memindex 1.4.5 → 1.4.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude-plugin/marketplace.json +1 -1
- package/.claude-plugin/plugin.json +1 -1
- package/.codex-plugin/plugin.json +1 -1
- package/.qoder-plugin/plugin.json +1 -1
- package/README.md +7 -1
- package/dist/bug-report/index.js +94 -83
- package/dist/npx-cli/index.js +3 -3
- package/dist/pi-extension/TYPEBOX-PROVENANCE.json +4 -4
- package/dist/pi-extension/index.js +9 -9
- package/integrations/dsh/package.json +1 -1
- package/integrations/openclaw/openclaw.plugin.json +1 -5
- package/integrations/pi/TYPEBOX-PROVENANCE.json +4 -4
- package/package.json +5 -5
- package/plugin/.claude-plugin/plugin.json +1 -1
- package/plugin/.codex-plugin/plugin.json +1 -1
- package/plugin/.qoder-plugin/plugin.json +1 -1
- package/plugin/package.json +1 -1
- package/plugin/scripts/context-generator.cjs +62 -61
- package/plugin/scripts/context-generator.cjs.map +3 -3
- package/plugin/scripts/mcp-server.cjs +2 -2
- package/plugin/scripts/transcript-watcher.cjs +2 -2
- package/plugin/scripts/transcript-watcher.cjs.map +1 -1
- package/plugin/scripts/worker-service.cjs +677 -669
- package/plugin/scripts/worker-service.cjs.map +3 -3
- package/plugin/skills/distill/SKILL.md +22 -0
- package/plugin/skills/how-it-works/SKILL.md +1 -1
- package/plugin/skills/how-it-works/onboarding-explainer.md +1 -1
- package/plugin/skills/import-chats/SKILL.md +22 -0
- package/plugin/skills/import-jira/SKILL.md +22 -0
- package/plugin/skills/import-reviews/SKILL.md +22 -0
- package/plugin/skills/pin/SKILL.md +26 -0
- package/plugin/skills/savings/SKILL.md +22 -0
- package/integrations/openclaw/skills/agent-cost-report/CHECKSUMS.txt +0 -50
- package/integrations/openclaw/skills/agent-cost-report/SKILL.md +0 -241
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/__init__.py +0 -1
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/behavior.py +0 -303
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/classify.py +0 -83
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/costs.py +0 -119
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/devices.py +0 -88
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/evidence.py +0 -160
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/labels.py +0 -115
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/measure.py +0 -89
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/mistakes.py +0 -310
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/patterns.py +0 -391
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/period.py +0 -98
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/prices.py +0 -95
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/render.py +0 -586
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/rollup.py +0 -355
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/rules.json +0 -4
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/rules.py +0 -76
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/sample.py +0 -45
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/snapshot.py +0 -53
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/sync.py +0 -79
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/transcripts.py +0 -126
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr/wins.py +0 -341
- package/integrations/openclaw/skills/agent-cost-report/scripts/acr.py +0 -219
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/_db.py +0 -49
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/_paths.py +0 -9
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/codex/sessions/2026/09/18/rollout-codex-1.jsonl +0 -4
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/make_report_fixtures.py +0 -46
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/prices_sample.json +0 -6
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/projects/-workspace-proj-a/sess-aaaa.jsonl +0 -8
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/projects/-workspace-proj-b/sess-bbbb.jsonl +0 -1
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/report-1day.json +0 -823
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/report-30day.json +0 -2213
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/report-7day.json +0 -1252
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/report-empty.json +0 -536
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/fixtures/report-session.json +0 -662
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_behavior.py +0 -299
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_devices.py +0 -99
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_labels.py +0 -96
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_measure.py +0 -120
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_pdf_windows.py +0 -54
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_period.py +0 -137
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_prices.py +0 -126
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_render.py +0 -238
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_render_encoding.py +0 -70
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_rollup.py +0 -263
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_snapshot.py +0 -63
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_sync.py +0 -52
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_transcript_encoding.py +0 -109
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_transcripts.py +0 -151
- package/integrations/openclaw/skills/agent-cost-report/scripts/tests/test_wins.py +0 -222
- package/plugin/skills/agent-cost-report/CHECKSUMS.txt +0 -50
- package/plugin/skills/agent-cost-report/SKILL.md +0 -241
- package/plugin/skills/agent-cost-report/scripts/acr/__init__.py +0 -1
- package/plugin/skills/agent-cost-report/scripts/acr/behavior.py +0 -303
- package/plugin/skills/agent-cost-report/scripts/acr/classify.py +0 -83
- package/plugin/skills/agent-cost-report/scripts/acr/costs.py +0 -119
- package/plugin/skills/agent-cost-report/scripts/acr/devices.py +0 -88
- package/plugin/skills/agent-cost-report/scripts/acr/evidence.py +0 -160
- package/plugin/skills/agent-cost-report/scripts/acr/labels.py +0 -115
- package/plugin/skills/agent-cost-report/scripts/acr/measure.py +0 -89
- package/plugin/skills/agent-cost-report/scripts/acr/mistakes.py +0 -310
- package/plugin/skills/agent-cost-report/scripts/acr/patterns.py +0 -391
- package/plugin/skills/agent-cost-report/scripts/acr/period.py +0 -98
- package/plugin/skills/agent-cost-report/scripts/acr/prices.py +0 -95
- package/plugin/skills/agent-cost-report/scripts/acr/render.py +0 -586
- package/plugin/skills/agent-cost-report/scripts/acr/rollup.py +0 -355
- package/plugin/skills/agent-cost-report/scripts/acr/rules.json +0 -4
- package/plugin/skills/agent-cost-report/scripts/acr/rules.py +0 -76
- package/plugin/skills/agent-cost-report/scripts/acr/sample.py +0 -45
- package/plugin/skills/agent-cost-report/scripts/acr/snapshot.py +0 -53
- package/plugin/skills/agent-cost-report/scripts/acr/sync.py +0 -79
- package/plugin/skills/agent-cost-report/scripts/acr/transcripts.py +0 -126
- package/plugin/skills/agent-cost-report/scripts/acr/wins.py +0 -341
- package/plugin/skills/agent-cost-report/scripts/acr.py +0 -219
- package/plugin/skills/agent-cost-report/scripts/tests/_db.py +0 -49
- package/plugin/skills/agent-cost-report/scripts/tests/_paths.py +0 -9
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/codex/sessions/2026/09/18/rollout-codex-1.jsonl +0 -4
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/make_report_fixtures.py +0 -46
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/prices_sample.json +0 -6
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/projects/-workspace-proj-a/sess-aaaa.jsonl +0 -8
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/projects/-workspace-proj-b/sess-bbbb.jsonl +0 -1
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/report-1day.json +0 -823
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/report-30day.json +0 -2213
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/report-7day.json +0 -1252
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/report-empty.json +0 -536
- package/plugin/skills/agent-cost-report/scripts/tests/fixtures/report-session.json +0 -662
- package/plugin/skills/agent-cost-report/scripts/tests/test_behavior.py +0 -299
- package/plugin/skills/agent-cost-report/scripts/tests/test_devices.py +0 -99
- package/plugin/skills/agent-cost-report/scripts/tests/test_labels.py +0 -96
- package/plugin/skills/agent-cost-report/scripts/tests/test_measure.py +0 -120
- package/plugin/skills/agent-cost-report/scripts/tests/test_pdf_windows.py +0 -54
- package/plugin/skills/agent-cost-report/scripts/tests/test_period.py +0 -137
- package/plugin/skills/agent-cost-report/scripts/tests/test_prices.py +0 -126
- package/plugin/skills/agent-cost-report/scripts/tests/test_render.py +0 -238
- package/plugin/skills/agent-cost-report/scripts/tests/test_render_encoding.py +0 -70
- package/plugin/skills/agent-cost-report/scripts/tests/test_rollup.py +0 -263
- package/plugin/skills/agent-cost-report/scripts/tests/test_snapshot.py +0 -63
- package/plugin/skills/agent-cost-report/scripts/tests/test_sync.py +0 -52
- package/plugin/skills/agent-cost-report/scripts/tests/test_transcript_encoding.py +0 -109
- package/plugin/skills/agent-cost-report/scripts/tests/test_transcripts.py +0 -151
- package/plugin/skills/agent-cost-report/scripts/tests/test_wins.py +0 -222
- package/plugin/skills/babysit/SKILL.md +0 -87
- package/plugin/skills/design-is/SKILL.md +0 -312
- package/plugin/skills/do/SKILL.md +0 -45
- package/plugin/skills/learn-codebase/SKILL.md +0 -21
- package/plugin/skills/make-plan/SKILL.md +0 -67
- package/plugin/skills/oh-my-issues/SKILL.md +0 -226
- package/plugin/skills/pathfinder/SKILL.md +0 -111
- package/plugin/skills/screenpipe/SKILL.md +0 -78
- package/plugin/skills/screenpipe/screenpipe.mjs +0 -196
- package/plugin/skills/smart-explore/SKILL.md +0 -193
- package/plugin/skills/standup/SKILL.md +0 -142
- package/plugin/skills/standup/agent-brief.md +0 -47
- package/plugin/skills/standup/standup.mjs +0 -586
- package/plugin/skills/what-the/SKILL.md +0 -6
- package/plugin/skills/wowerpoint/SKILL.md +0 -205
|
@@ -1,222 +0,0 @@
|
|
|
1
|
-
import datetime as dt
|
|
2
|
-
import json
|
|
3
|
-
import os
|
|
4
|
-
import shutil
|
|
5
|
-
import tempfile
|
|
6
|
-
import unittest
|
|
7
|
-
|
|
8
|
-
import _paths
|
|
9
|
-
from acr import wins
|
|
10
|
-
from acr.period import PT
|
|
11
|
-
|
|
12
|
-
S = int(dt.datetime(2026, 9, 18, tzinfo=PT).timestamp() * 1000)
|
|
13
|
-
E = int(dt.datetime(2026, 9, 21, tzinfo=PT).timestamp() * 1000)
|
|
14
|
-
H = 3600 * 1000
|
|
15
|
-
DAYS = ["2026-09-18", "2026-09-19", "2026-09-20"]
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
def iso(ms):
|
|
19
|
-
return dt.datetime.fromtimestamp(ms / 1000, dt.timezone.utc).isoformat().replace("+00:00", "Z")
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
def usage_row(cs, ms, x1e6, out=100):
|
|
23
|
-
return dict(session=cs, ts_ms=ms, x1e6=x1e6, input=1000, output=out, cache_write_5m=0, cache_write_1h=0, cache_read=0)
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
def tool_lines(session, ms, cmd, result="", is_error=False, tid="toolu_1"):
|
|
27
|
-
a = dict(type="assistant", timestamp=iso(ms), sessionId=session, cwd="/w/p", message=dict(role="assistant", content=[dict(type="tool_use", id=tid, name="Bash", input=dict(command=cmd))]))
|
|
28
|
-
u = dict(type="user", timestamp=iso(ms + 1000), sessionId=session, message=dict(role="user", content=[dict(type="tool_result", tool_use_id=tid, content=result, is_error=is_error)]),
|
|
29
|
-
toolUseResult=dict(stdout=result, stderr=""))
|
|
30
|
-
return [json.dumps(a), json.dumps(u)]
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
def fake_gh(table):
|
|
34
|
-
calls = []
|
|
35
|
-
def gh(number, repo):
|
|
36
|
-
calls.append((number, repo)); return table.get(f"{repo}#{number}")
|
|
37
|
-
gh.calls = calls; return gh
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
class Commands(unittest.TestCase):
|
|
41
|
-
def test_classify(self):
|
|
42
|
-
c = wins.classify_command("gh pr merge 12 --repo o/r --squash")
|
|
43
|
-
self.assertEqual((c["kind"], c["number"], c["repo"]), ("pr", 12, "o/r"))
|
|
44
|
-
c = wins.classify_command("cd /x && gh pr merge https://github.com/o/r/pull/7 --merge")
|
|
45
|
-
self.assertEqual((c["number"], c["repo"]), (7, "o/r"))
|
|
46
|
-
self.assertEqual(wins.classify_command("gh pr merge --auto")["number"], None)
|
|
47
|
-
self.assertEqual(wins.classify_command("npm publish --access public")["how"], "npm publish")
|
|
48
|
-
self.assertEqual(wins.classify_command("gh release create v1.2.3 --notes x")["tag"], "v1.2.3")
|
|
49
|
-
self.assertEqual(wins.classify_command("git push origin v1.2.3")["tag"], "v1.2.3")
|
|
50
|
-
self.assertEqual(wins.classify_command("git push --tags")["tag"], None)
|
|
51
|
-
self.assertIsNone(wins.classify_command("git push origin main"))
|
|
52
|
-
self.assertIsNone(wins.classify_command("grep -l 'gh pr merge' *.jsonl | head"))
|
|
53
|
-
self.assertIsNone(wins.classify_command("cat > f.py <<'PY'\nnpm publish\ngh pr merge 1\nPY"))
|
|
54
|
-
self.assertIsNone(wins.classify_command("echo done"))
|
|
55
|
-
|
|
56
|
-
def test_trailers(self):
|
|
57
|
-
msg = "feat: x\n\nbody\n\nClaude-Session: 0e3f6cc9-3feb-4e4a-b20e-cc3704ef868d\nCo-Authored-By: x"
|
|
58
|
-
self.assertEqual(wins.session_trailers(msg), ["0e3f6cc9-3feb-4e4a-b20e-cc3704ef868d"])
|
|
59
|
-
self.assertEqual(wins.session_trailers("Session: 0e3f6cc9-3feb-4e4a-b20e-cc3704ef868d"), [])
|
|
60
|
-
self.assertEqual(wins.session_trailers("Claude-Session: https://x/0e3f6cc9"), [])
|
|
61
|
-
self.assertEqual(wins.session_trailers("Claude-Session: abc def"), []); self.assertEqual(wins.session_trailers("Claude-Session: (abc)"), [])
|
|
62
|
-
self.assertEqual(wins.session_trailers("Claude-Session: a\nClaude-Session: a\nClaude-Session: b"), ["a", "b"])
|
|
63
|
-
|
|
64
|
-
def test_no_fallback_estimate_anywhere(self):
|
|
65
|
-
for f in ("wins.py", "rollup.py", "costs.py"):
|
|
66
|
-
with open(os.path.join(_paths.SCRIPTS, "acr", f)) as fh: src = fh.read()
|
|
67
|
-
self.assertNotIn("fallback_estimate", src, f); self.assertNotIn("lineage", src.lower().replace("no lineage fallback", ""), f)
|
|
68
|
-
|
|
69
|
-
def test_praise_stub(self):
|
|
70
|
-
self.assertEqual(wins.detect_praise(), []); self.assertEqual(wins.detect_praise(prompts=["boom love it"]), [])
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
class Transcripts(unittest.TestCase):
|
|
74
|
-
def setUp(self):
|
|
75
|
-
self.tmp = tempfile.mkdtemp(prefix="acr-wins-")
|
|
76
|
-
|
|
77
|
-
def tearDown(self):
|
|
78
|
-
shutil.rmtree(self.tmp, ignore_errors=True)
|
|
79
|
-
|
|
80
|
-
def write(self, name, lines):
|
|
81
|
-
p = os.path.join(self.tmp, name)
|
|
82
|
-
with open(p, "w") as fh: fh.write("\n".join(lines) + "\n")
|
|
83
|
-
return p
|
|
84
|
-
|
|
85
|
-
def test_scan_success_error_and_window(self):
|
|
86
|
-
f = self.write("a.jsonl", tool_lines("cs1", S + 2 * H, "gh pr merge 5 --repo o/r", "", tid="t1")
|
|
87
|
-
+ tool_lines("cs1", S + 3 * H, "gh pr merge 6 --repo o/r", "GraphQL: Pull request not mergeable", is_error=False, tid="t2")
|
|
88
|
-
+ tool_lines("cs1", S + 4 * H, "gh pr merge 7 --repo o/r", "", is_error=True, tid="t3")
|
|
89
|
-
+ tool_lines("cs1", E + H, "gh pr merge 8 --repo o/r", "", tid="t4")
|
|
90
|
-
+ tool_lines("cs1", S + 5 * H, "npm publish", "npm notice Publishing to https://registry.npmjs.org/\n+ memindex@13.25.3", tid="t5")
|
|
91
|
-
+ ['{"type":"assistant","message":{"usage":{}}}', "not json"])
|
|
92
|
-
got = list(wins.scan_transcripts([f], S, E))
|
|
93
|
-
self.assertEqual([(c["kind"], c.get("number")) for c in got], [("pr", 5), ("publish", None)])
|
|
94
|
-
self.assertIn("memindex@13.25.3", got[1]["result"])
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
class Build(Transcripts):
|
|
98
|
-
def sessions(self):
|
|
99
|
-
return {"cs1": dict(project="memindex", memory_session_id="m1", has_transcript=True, observer_tokens=100, cwd="/w/p"),
|
|
100
|
-
"cs2": dict(project="memindex", memory_session_id="m2", has_transcript=False, observer_tokens=1000, cwd=None)}
|
|
101
|
-
|
|
102
|
-
def ship(self, n=4125, ms=S + 6 * H):
|
|
103
|
-
return [dict(id=9, title=f"PR #{n} merged to main: x", ts_ms=ms, project="memindex", memory_session_id="m1", content_session_id="cs1")]
|
|
104
|
-
|
|
105
|
-
def build(self, files=(), ship_obs=(), rows=None, gh=None, ratio=None, remote=lambda cwd: "o/r", gh_list=None, repos=("o/r",)):
|
|
106
|
-
return wins.build(list(files), S, E, list(ship_obs), self.sessions(), rows or {}, ratio, DAYS, gh=gh or fake_gh({}), remote=remote,
|
|
107
|
-
use_gh=(gh is not None or gh_list is not None), repos=repos, gh_list=gh_list)
|
|
108
|
-
|
|
109
|
-
def test_no_link_means_every_usd_null(self):
|
|
110
|
-
f = self.write("a.jsonl", tool_lines("cs1", S + 2 * H, "gh pr merge 5 --repo o/r"))
|
|
111
|
-
block, by_day = self.build([f], self.ship(), rows={"cs1": [usage_row("cs1", S + H, 1_000_000)]})
|
|
112
|
-
self.assertEqual(len(block["items"]), 2); self.assertEqual(block["cost_status"], "unmeasured")
|
|
113
|
-
self.assertTrue(all(i["usd"] is None and i["cost_status"] == "unmeasured" for i in block["items"]))
|
|
114
|
-
self.assertIsNone(block["attribution_method"]); self.assertIsNone(block["total_attributed_usd"]); self.assertIsNone(block["unattributed_usd"])
|
|
115
|
-
self.assertEqual(block["praise_n"], 0); self.assertEqual([d["day_pt"] for d in by_day], DAYS)
|
|
116
|
-
self.assertEqual([d["count"] for d in by_day], [2, 0, 0]); self.assertTrue(all(d["usd"] is None for d in by_day))
|
|
117
|
-
by_key = {i["key"]: i for i in block["items"]}
|
|
118
|
-
self.assertEqual(by_key["o/r#4125"]["title"], "PR #4125 merged to main: x")
|
|
119
|
-
self.assertEqual(by_key["o/r#5"]["title"], "o/r#5")
|
|
120
|
-
|
|
121
|
-
def test_pr_in_transcript_and_observation_counts_once(self):
|
|
122
|
-
f = self.write("a.jsonl", tool_lines("cs1", S + 2 * H, "gh pr merge 4125 --repo o/r"))
|
|
123
|
-
block, _ = self.build([f], self.ship(4125))
|
|
124
|
-
self.assertEqual(len(block["items"]), 1); self.assertEqual(block["items"][0]["key"], "o/r#4125")
|
|
125
|
-
self.assertEqual(set(block["items"][0]["sources"]), {"transcript", "ship_observation"}); self.assertEqual(block["items"][0]["evidence_ids"], [9])
|
|
126
|
-
|
|
127
|
-
def test_observation_repo_from_remote_of_project_cwd(self):
|
|
128
|
-
block, _ = self.build([], self.ship(1), remote=lambda cwd: "imogenlabs/memindex" if cwd == "/w/p" else None)
|
|
129
|
-
self.assertEqual(block["items"][0]["key"], "imogenlabs/memindex#1")
|
|
130
|
-
block, _ = self.build([], self.ship(1), remote=lambda cwd: None)
|
|
131
|
-
self.assertEqual(block["items"][0]["key"], "memindex#1")
|
|
132
|
-
|
|
133
|
-
def test_gh_confirmation_window_and_not_merged(self):
|
|
134
|
-
merged_in = iso(S + 7 * H); merged_out = iso(E + 5 * H)
|
|
135
|
-
gh = fake_gh({"o/r#1": dict(mergedAt=merged_in, url="u1", title="T1", commits=[]), "o/r#2": dict(mergedAt=merged_out, url="u2", title="T2", commits=[]),
|
|
136
|
-
"o/r#3": dict(mergedAt=None, url="u3", title="T3", commits=[])})
|
|
137
|
-
obs = self.ship(1) + self.ship(2) + self.ship(3)
|
|
138
|
-
block, _ = self.build([], obs, gh=gh)
|
|
139
|
-
self.assertEqual([i["key"] for i in block["items"]], ["o/r#1"]); self.assertTrue(block["items"][0]["confirmed"])
|
|
140
|
-
self.assertEqual((block["items"][0]["title"], block["items"][0]["url"]), ("T1", "u1"))
|
|
141
|
-
self.assertEqual(sorted(d["reason"] for d in block["dropped"]), ["gh pr view: not merged", "outside window"])
|
|
142
|
-
self.assertEqual(len(gh.calls), 3)
|
|
143
|
-
|
|
144
|
-
def test_trailer_links_and_shared_turn_assigned_once(self):
|
|
145
|
-
c = lambda sid: [dict(messageHeadline="x", messageBody=f"body\n\nClaude-Session: {sid}")]
|
|
146
|
-
gh = fake_gh({"o/r#1": dict(mergedAt=iso(S + 5 * H), url="u", title="first", commits=c("cs1")),
|
|
147
|
-
"o/r#2": dict(mergedAt=iso(S + 9 * H), url="u", title="second", commits=c("cs1") + [dict(messageHeadline="y", messageBody="Session: cs2")])})
|
|
148
|
-
rows = {"cs1": [usage_row("cs1", S + H, 1_000_000), usage_row("cs1", S + 2 * H, 2_000_000), usage_row("cs1", S + 7 * H, 4_000_000), usage_row("cs1", S + 20 * H, 8_000_000)]}
|
|
149
|
-
block, by_day = self.build([], self.ship(1) + self.ship(2), rows=rows, gh=gh)
|
|
150
|
-
w1, w2 = block["items"]
|
|
151
|
-
self.assertEqual((w1["usd"], w2["usd"]), (3.0, 4.0))
|
|
152
|
-
self.assertEqual((w1["turns_n"], w2["turns_n"], w1["sessions_n"]), (2, 1, 1))
|
|
153
|
-
self.assertEqual(block["cost_status"], "session_linked"); self.assertEqual(block["total_attributed_usd"], 7.0)
|
|
154
|
-
self.assertEqual(w1["cost_label"], "ESTIMATED · session-linked"); self.assertEqual(w1["cost_basis"], "estimated_usage")
|
|
155
|
-
self.assertEqual(w2["linked_sessions"], ["cs1"])
|
|
156
|
-
self.assertEqual(w1["tokens"]["input"], 2000); self.assertEqual(by_day[0]["usd"], 7.0)
|
|
157
|
-
|
|
158
|
-
def test_replica_session_win_is_extrapolated_only_with_link(self):
|
|
159
|
-
gh = fake_gh({"o/r#1": dict(mergedAt=iso(S + 5 * H), url="u", title="t", commits=[dict(messageHeadline="x", messageBody="Claude-Session: cs2")])})
|
|
160
|
-
block, _ = self.build([], self.ship(1), gh=gh, ratio=2.0)
|
|
161
|
-
w = block["items"][0]
|
|
162
|
-
self.assertEqual((w["cost_status"], w["cost_basis"], w["usd"]), ("session_linked", "extrapolated", 0.0))
|
|
163
|
-
self.assertIn("EXTRAPOLATED (low confidence)", w["cost_label"])
|
|
164
|
-
block, _ = self.build([], self.ship(1), gh=gh, ratio=2.0)
|
|
165
|
-
self.assertEqual(block["items"][0]["usd"], round(1000 * 2.0 / 1e6, 2))
|
|
166
|
-
gh2 = fake_gh({"o/r#1": dict(mergedAt=iso(S + 5 * H), url="u", title="t", commits=[])})
|
|
167
|
-
block, _ = self.build([], self.ship(1), gh=gh2, ratio=2.0)
|
|
168
|
-
self.assertEqual((block["items"][0]["cost_status"], block["items"][0]["usd"], block["items"][0]["cost_basis"]), ("unmeasured", None, None))
|
|
169
|
-
|
|
170
|
-
def test_finished_work_is_not_a_win(self):
|
|
171
|
-
block, by_day = self.build([], [])
|
|
172
|
-
self.assertEqual(block["items"], []); self.assertEqual([d["count"] for d in by_day], [0, 0, 0])
|
|
173
|
-
|
|
174
|
-
def test_gh_list_adds_wins_dedupes_and_labels_source(self):
|
|
175
|
-
f = self.write("a.jsonl", tool_lines("cs1", S + 2 * H, "gh pr merge 4125 --repo o/r"))
|
|
176
|
-
lister = lambda repo, s, e: ([dict(number=4125, title="from github", url="https://github.com/o/r/pull/4125", ts_ms=S + 3 * H),
|
|
177
|
-
dict(number=4200, title="only on github", url="https://github.com/o/r/pull/4200", ts_ms=S + 30 * H)], "ok")
|
|
178
|
-
block, by_day = self.build([f], self.ship(4125), gh_list=lister)
|
|
179
|
-
keys = {i["key"]: i for i in block["items"]}
|
|
180
|
-
self.assertEqual(sorted(keys), ["o/r#4125", "o/r#4200"])
|
|
181
|
-
self.assertEqual(set(keys["o/r#4125"]["sources"]), {"transcript", "ship_observation", wins.GH_SOURCE}); self.assertTrue(keys["o/r#4125"]["confirmed"])
|
|
182
|
-
self.assertEqual((keys["o/r#4200"]["sources"], keys["o/r#4200"]["title"], keys["o/r#4200"]["usd"], keys["o/r#4200"]["cost_status"]), ([wins.GH_SOURCE], "only on github", None, "unmeasured"))
|
|
183
|
-
src = block["sources"][wins.GH_SOURCE]; self.assertEqual((src["status"], src["found"], src["repos"]["o/r"]["status"]), ("ok", 2, "ok"))
|
|
184
|
-
self.assertEqual([d["count"] for d in by_day], [1, 1, 0])
|
|
185
|
-
|
|
186
|
-
def test_gh_list_unavailable_never_zero_and_report_completes(self):
|
|
187
|
-
f = self.write("a.jsonl", tool_lines("cs1", S + 2 * H, "gh pr merge 4125 --repo o/r"))
|
|
188
|
-
for reason in ("gh missing", "unauthenticated", "rate-limited"):
|
|
189
|
-
block, _ = self.build([f], gh_list=lambda repo, s, e, r=reason: ([], r))
|
|
190
|
-
src = block["sources"][wins.GH_SOURCE]
|
|
191
|
-
self.assertEqual((src["status"], src["repos"]["o/r"]["status"], src["repos"]["o/r"]["found"]), ("unavailable", reason, None), reason)
|
|
192
|
-
self.assertEqual(len(block["items"]), 1)
|
|
193
|
-
block, _ = self.build([f], gh=fake_gh({}), gh_list=None)
|
|
194
|
-
self.assertEqual(block["sources"][wins.GH_SOURCE]["repos"]["o/r"]["status"], "unavailable (gh disabled for this run)")
|
|
195
|
-
|
|
196
|
-
def test_gh_merged_prs_window_edges_and_failures(self):
|
|
197
|
-
rows = [dict(number=1, title="a", url="u1", mergedAt=iso(S)), dict(number=2, title="b", url="u2", mergedAt=iso(E)), dict(number=3, title="c", url="u3", mergedAt=iso(E - 1000)), dict(number=4, title="no merge", url="u4", mergedAt=None)]
|
|
198
|
-
ok = lambda cmd, **k: type("R", (), dict(returncode=0, stdout=json.dumps(rows), stderr=""))()
|
|
199
|
-
out, st = wins.gh_merged_prs("o/r", S, E, run=ok)
|
|
200
|
-
self.assertEqual((st, [x["number"] for x in out]), ("ok", [1, 3]))
|
|
201
|
-
def missing(cmd, **k): raise FileNotFoundError("gh")
|
|
202
|
-
self.assertEqual(wins.gh_merged_prs("o/r", S, E, run=missing), ([], "gh missing"))
|
|
203
|
-
auth = lambda cmd, **k: type("R", (), dict(returncode=4, stdout="", stderr="To get started with GitHub CLI, please run: gh auth login"))()
|
|
204
|
-
self.assertEqual(wins.gh_merged_prs("o/r", S, E, run=auth)[1], "unauthenticated")
|
|
205
|
-
rl = lambda cmd, **k: type("R", (), dict(returncode=1, stdout="", stderr="GraphQL: API rate limit exceeded for user"))()
|
|
206
|
-
self.assertEqual(wins.gh_merged_prs("o/r", S, E, run=rl)[1], "rate-limited")
|
|
207
|
-
junk = lambda cmd, **k: type("R", (), dict(returncode=0, stdout="not json", stderr=""))()
|
|
208
|
-
self.assertEqual(wins.gh_merged_prs("o/r", S, E, run=junk)[1], "gh error (unparseable output)")
|
|
209
|
-
seen = []
|
|
210
|
-
def spy(cmd, **k): seen.append(cmd); return ok(cmd)
|
|
211
|
-
wins.gh_merged_prs("o/r", S, E, run=spy)
|
|
212
|
-
self.assertEqual(seen[0][:6], ["gh", "pr", "list", "--repo", "o/r", "--state"]); self.assertEqual(seen[0][1:3], ["pr", "list"]); self.assertNotIn("merge", seen[0][2])
|
|
213
|
-
|
|
214
|
-
def test_publish_from_transcript_and_observation_dedup(self):
|
|
215
|
-
f = self.write("a.jsonl", tool_lines("cs1", S + 2 * H, "npm publish", "npm notice Publishing to https://registry.npmjs.org/\n+ memindex@13.25.3"))
|
|
216
|
-
obs = [dict(id=3, title="memindex 13.25.3 published to npm", ts_ms=S + 3 * H, project="memindex/wt", memory_session_id="m1", content_session_id="cs1")]
|
|
217
|
-
block, _ = self.build([f], obs)
|
|
218
|
-
self.assertEqual([i["key"] for i in block["items"]], ["memindex@13.25.3"]); self.assertEqual(block["items"][0]["kind"], "publish")
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
if __name__ == "__main__":
|
|
222
|
-
unittest.main()
|
|
@@ -1,50 +0,0 @@
|
|
|
1
|
-
ff7b223f562519ade842573f431917d221b3e7d719f5d4281200949572b9de44 SKILL.md
|
|
2
|
-
3aee0f7d817e53646d551f144b3741b03099966a047cdfd278b8cc9f6dd2d534 scripts/acr.py
|
|
3
|
-
09a83d4838add1313209d767513dfc84eb316bf137658483ef164a5465fdbfdd scripts/acr/__init__.py
|
|
4
|
-
c9e34e6c9e4f3e50d530adf27b52e030f8ede445a526b1501677e56db46aa8f5 scripts/acr/behavior.py
|
|
5
|
-
dc80ec35fc10d182c32a9bf45acc13b87ef4469f7fa7fdef525c2ed3cc0e673f scripts/acr/classify.py
|
|
6
|
-
5c42fa1677f07bd467d8ff94110c8b8d0bd47d570930fa73b137b276b2fa6e07 scripts/acr/costs.py
|
|
7
|
-
c296851652e13a2f2db53e87e59c6f15ef01a4780024f4b9d3827c5a6f25a0f0 scripts/acr/devices.py
|
|
8
|
-
46c26b86028f5fff1aa0021a9aa71f3b4ec2b9107f5321e72722fecce62291e5 scripts/acr/evidence.py
|
|
9
|
-
f851d9cbd30c56e4401b758f6b2f60aa88b3080e2759d9fd13518c4dc017bb6e scripts/acr/labels.py
|
|
10
|
-
8084ff9e633fd3a1b2bc66d4744653560fab464df1d33054f51b94d7b0c57b89 scripts/acr/measure.py
|
|
11
|
-
1a8a903467c1cd358d0d7f2717568c02d61cc389cd321f1ab9b64d708ed60314 scripts/acr/mistakes.py
|
|
12
|
-
662f6b4ce10833b5a7a1a99b4a98fc45c3311bdc62c749913717cd519ffade65 scripts/acr/patterns.py
|
|
13
|
-
94c062d58c2a9d7f3ac25357e69c0cac3c7762ff7698f91e32b4b98d897525d3 scripts/acr/period.py
|
|
14
|
-
84f9f6fdfd0017080261de42e3b89211e9da1c4c69958febccadcb045e186b82 scripts/acr/prices.py
|
|
15
|
-
10f8d20e22d2b6ed61431a58d74cc49ef44e3ec2b7c36fbb6b4c1f06b00c2062 scripts/acr/render.py
|
|
16
|
-
dfaa9485f96ff0bbb60761e3adcde37967cbfdc6cc7e632d165685feb60b31d9 scripts/acr/rollup.py
|
|
17
|
-
91a2e6ef09fe5f2a2aeb086519253c0d1ed42c24b2c40db4561dbc658430486d scripts/acr/rules.json
|
|
18
|
-
bff0e48603cbd66c6951e084e519c91a760d496b9d023ee64f1c5ab0014d1231 scripts/acr/rules.py
|
|
19
|
-
11d8da12ef882827fa47c7b37f37d84053790f25c8ead2a756db9672a8883a5e scripts/acr/sample.py
|
|
20
|
-
32b2b5d60879c722e7ad5a136d818e1d9659f052db6118fdcfcb3f7a06e01010 scripts/acr/snapshot.py
|
|
21
|
-
efd8e7c6f079b9c3adee2625d31ca583a1463d78af10540429cd6da6d4c2c25e scripts/acr/sync.py
|
|
22
|
-
2d6b0721cc6b092932a7c3b240815bd48747d593fff96b59860f04b7e6f082f7 scripts/acr/transcripts.py
|
|
23
|
-
23260ecf473d012364a631b2cfc050d7282f84f38aefafd8773210a7f0b0b839 scripts/acr/wins.py
|
|
24
|
-
f5a5a85d015614c206150a7c90f34bd321aca5b0b5c81917a8dbf1bd31b01b46 scripts/tests/_db.py
|
|
25
|
-
1baa143bea04326b7096a0654f91c22255d1f131ed23af454395b9d3c959f78d scripts/tests/_paths.py
|
|
26
|
-
1b635a61006492d13e8981e5f997d6594d6b96cfe4d6c0ec331448e13c6b2f49 scripts/tests/test_behavior.py
|
|
27
|
-
1cbdff1dd09c178b5e1bfd445d1cd63c0953b796daf4b220a9d7128c181373e6 scripts/tests/test_devices.py
|
|
28
|
-
208896741e6f6b85c8ad8d968080c1a15adbeb9cc1f13d9dd1d5690f0a8ec8ea scripts/tests/test_labels.py
|
|
29
|
-
2ac92bb696c5ed13bcae9e88e1d3b8598af38f46a764fc17d146357db3820bda scripts/tests/test_measure.py
|
|
30
|
-
fb00297709c4ccb0c29d5728508cb060007e1cefb7752f056f84ef2f35bcd439 scripts/tests/test_pdf_windows.py
|
|
31
|
-
c4b9b88868a3513ae4884ee58e3532695ed0c7a57f7f71e9d19ceb71ac7e9b30 scripts/tests/test_period.py
|
|
32
|
-
77a16deda2df742acb5c5405fb014584d270b686fc3d2e38e5e1d7e25eb3f3a7 scripts/tests/test_prices.py
|
|
33
|
-
0fda33db7e4fea2b8ffdd9448e3d5335012873ae465dc1dc3293714511faeb24 scripts/tests/test_render.py
|
|
34
|
-
0143a3d34b86dd67c0c0af444258625798ebfe71d83494d12b0837245ef350a4 scripts/tests/test_render_encoding.py
|
|
35
|
-
3663890648b1e04f1a8c0f9fe8f8dd03aca76e1914fb4b02b84d51c12da63ce2 scripts/tests/test_rollup.py
|
|
36
|
-
00444fa59eaeb0df491696771a8dd6ee4d8b32b7c88208c164512e7fa18e436b scripts/tests/test_snapshot.py
|
|
37
|
-
21bcf77ba286e4b2d49903e5ca7c83a2369df89739e27bdad32ae2319550a5a7 scripts/tests/test_sync.py
|
|
38
|
-
e59880d991a3ce9ec2a2f40ed8836c2a2d77fa85093650b9952e830e83dbcd8b scripts/tests/test_transcript_encoding.py
|
|
39
|
-
2c39ce509e7293e95ffa81f766f46792c76a0497f491806d797e9ec76a822d52 scripts/tests/test_transcripts.py
|
|
40
|
-
a74d987bfd568209a8a4105f63052f4af0b0fcd09dbf966bde9921b893f1f91d scripts/tests/test_wins.py
|
|
41
|
-
e18dd87adc7439dbc3485328eb39bdf8c72231713412a7d34f095007e939576f scripts/tests/fixtures/make_report_fixtures.py
|
|
42
|
-
f89b3a779a195670f381c40ec2fcda56b049e14c3cefa0483d583fc11209ab06 scripts/tests/fixtures/prices_sample.json
|
|
43
|
-
204c3b7375c73d6f3a14a381d94f1430494cecb4bf4d55d8809a78aacfb8c794 scripts/tests/fixtures/report-1day.json
|
|
44
|
-
bcc9bd05b1107b46d996441b256676634417cadcfdacb7e54a46b703120225c0 scripts/tests/fixtures/report-30day.json
|
|
45
|
-
dacd1b90b94e5b4a3901e6c4040a386ba5b05ca5bbd5dcb08bb3342af0685425 scripts/tests/fixtures/report-7day.json
|
|
46
|
-
6b9bc94d5483a10da12826a8ccd910268888b4e5ae19cabb8affa145b08d309b scripts/tests/fixtures/report-empty.json
|
|
47
|
-
9257b83ff2103a02257ce11a05c5ce790cd3cdc9cab1821a34827c7775433501 scripts/tests/fixtures/report-session.json
|
|
48
|
-
9ed8ae28c7bb8dd86be98617819abfc8b4557af7c8efa4ee36a339efd0cb62b5 scripts/tests/fixtures/codex/sessions/2026/09/18/rollout-codex-1.jsonl
|
|
49
|
-
d3285c6a012ff622f104fd0ab8452e0d586233aec1f4562a0f0655cbadf004b9 scripts/tests/fixtures/projects/-workspace-proj-a/sess-aaaa.jsonl
|
|
50
|
-
cd0219f1b8e202d04c7c07ce7c2dd52fc5ec78f30ae8254defe4cae2880bb000 scripts/tests/fixtures/projects/-workspace-proj-b/sess-bbbb.jsonl
|
|
@@ -1,241 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: Agent Cost Report
|
|
3
|
-
description: >-
|
|
4
|
-
Believable agent cost report for any period, default the last 7 full days PT, not counting today.
|
|
5
|
-
Measured tokens from Claude Code transcripts priced at OpenRouter list prices (ESTIMATED), measured
|
|
6
|
-
provider spend when a sanctioned source exists, note-taker cost separate, Timing-style HTML/PDF plus
|
|
7
|
-
report.json, line-items.csv, evidence.json.
|
|
8
|
-
allowed-tools:
|
|
9
|
-
- Bash
|
|
10
|
-
- Read
|
|
11
|
-
- Write
|
|
12
|
-
- AskUserQuestion
|
|
13
|
-
- mcp__plugin_memindex_mcp-search__search
|
|
14
|
-
- mcp__plugin_memindex_mcp-search__timeline
|
|
15
|
-
- mcp__plugin_memindex_mcp-search__get_observations
|
|
16
|
-
---
|
|
17
|
-
|
|
18
|
-
# Agent Cost Report
|
|
19
|
-
|
|
20
|
-
**MemIndex / Claude Code skill.** Runtime is the `scripts/` pipeline (transcripts → tokens → dollars → Timing-style report) plus a progressive Mem Search review pass that confirms the drafted labels.
|
|
21
|
-
|
|
22
|
-
Resolve the absolute directory containing this `SKILL.md`; all helper paths are relative to that directory. `${CLAUDE_SKILL_DIR}` is the shortcut: `python3 "${CLAUDE_SKILL_DIR}/scripts/acr.py" …`. Python 3.9+ standard library only, with IANA timezone data for `America/Los_Angeles`; no pip installs. `tzdata` is the only exception to the no-pip-installs rule, and only when `acr.py` exits saying that timezone data is unavailable (stock Windows Python ships none): show the user the lines it printed and ask (AskUserQuestion) before installing it. On a yes, run the line for your shell (the plain quoted line in Bash or cmd, the `PowerShell:` line in PowerShell), then rerun the failed command. Setting `PYTHONTZPATH` instead needs no install. The look lives in `scripts/acr/render.py`, never here.
|
|
23
|
-
|
|
24
|
-
## Purpose
|
|
25
|
-
|
|
26
|
-
Turn MemIndex activity into a **manager-readable cost and failure report**.
|
|
27
|
-
|
|
28
|
-
Product idea: a reusable skill that searches MemIndex via Mem Search, reconstructs real units of work, assigns cost and failure categories, and renders a printable report.
|
|
29
|
-
|
|
30
|
-
## Insight north star
|
|
31
|
-
|
|
32
|
-
The headline is dollars, to two decimals, labeled. The dollars come from Claude Code transcripts (exact per-reply token usage) priced at OpenRouter public list prices, so they are **ESTIMATED**. Measured provider spend appears only when a sanctioned source gives it. Directly under the dollars: what the mistakes cost, what shipped, and both on one time axis.
|
|
33
|
-
|
|
34
|
-
## Questions the report must answer
|
|
35
|
-
|
|
36
|
-
1. What work was completed?
|
|
37
|
-
2. What did each outcome cost?
|
|
38
|
-
3. What was wasted through looping, hedging, wrong turns, rework, or poor routing?
|
|
39
|
-
4. Were any unauthorized actions attempted?
|
|
40
|
-
5. What should the manager change next?
|
|
41
|
-
|
|
42
|
-
**Primary unit = cost per completed outcome** (not cost per observation).
|
|
43
|
-
|
|
44
|
-
## When to use
|
|
45
|
-
|
|
46
|
-
- "Agent cost report" / "cost per outcome" / "failure economics" / "was this session worth it" / "what did the agents cost this week"
|
|
47
|
-
- After a real Mem session dig when leadership needs outcome economics
|
|
48
|
-
- Sample / ship packs that need self-contained HTML + JSON + CSV + evidence
|
|
49
|
-
|
|
50
|
-
Memory dig mechanics: the memindex `mem-search` skill (progressive recall).
|
|
51
|
-
|
|
52
|
-
## Default scope (ALWAYS)
|
|
53
|
-
|
|
54
|
-
- Unless the user names a **specific session / range / project**, the window is the **last 7 full days in PT, not counting today**: `end` = the PT midnight that started today (exclusive), `start` = `end − 7 days`. The default window never contains a partial day.
|
|
55
|
-
- Explicit windows: `--start YYYY-MM-DD --end YYYY-MM-DD` (PT calendar days, end exclusive). An explicit `--end` later than today marks the last day "partial, generated HH:MM PT".
|
|
56
|
-
- One session: `--session <content_session_id>`. One project plus a period: `--project <name> --start … --end …` (worktrees of the project are included).
|
|
57
|
-
- The report **Scope** strip shows the PT range, and Details list every session id in scope.
|
|
58
|
-
|
|
59
|
-
## Progressive Mem Search (ALWAYS)
|
|
60
|
-
|
|
61
|
-
Follow the memindex `mem-search` three layers. Keep spend light.
|
|
62
|
-
|
|
63
|
-
1. **Search** — get an index of IDs (titles, types, token hints).
|
|
64
|
-
2. **Timeline** — only around anchors you care about.
|
|
65
|
-
3. **Observations** — `get_observations` for the filtered IDs you will cite as evidence.
|
|
66
|
-
|
|
67
|
-
Recipe:
|
|
68
|
-
|
|
69
|
-
1. Resolve scope (default: the last 7 full PT days).
|
|
70
|
-
2. Search → collect IDs.
|
|
71
|
-
3. Timeline for thin context only.
|
|
72
|
-
4. Observations for intended / actual / outcome / waste / rework / blocked / unauthorized / status.
|
|
73
|
-
5. Group into named work items + failure events.
|
|
74
|
-
6. Calculate line-item costs.
|
|
75
|
-
7. Render HTML + optional PDF (+ json/csv/evidence).
|
|
76
|
-
8. Keep evidence IDs in the appendix — do not dump entire timelines into the main report.
|
|
77
|
-
|
|
78
|
-
Process bullets:
|
|
79
|
-
|
|
80
|
-
- Search first → pick IDs → timeline only if context is thin → fetch only needed obs.
|
|
81
|
-
- Work-item titles are invented for managers ("Restore search after Chroma crash-loop"); observation titles stay evidence-only.
|
|
82
|
-
- Evidence appendix lists obs IDs + short titles; main sections stay outcome-first.
|
|
83
|
-
|
|
84
|
-
In this skill the search pass is the **review step** (see Recipe step 4): the pipeline drafts categories and failure types from keywords (`label_source: keyword`); the orchestrator confirms or changes each line item's `category` and `failure_type` from its cited evidence IDs and applies the result with `review --apply`. Items left unreviewed keep the "draft label" mark and the footer counts them.
|
|
85
|
-
|
|
86
|
-
## Work categories (ALWAYS)
|
|
87
|
-
|
|
88
|
-
`Feature` · `Bug fix` · `Incident` · `Maintenance` · `Investigation` · `Experiment`
|
|
89
|
-
|
|
90
|
-
## Failure / waste types (ALWAYS)
|
|
91
|
-
|
|
92
|
-
`Looping` · `Hedging` · `Wrong turn` · `Rework` · `Regression` · `Premature completion` · `Unauthorized action` · `Suboptimal path` · `Duplicate work` · `Blocked work` · `Missed requirement` · `Unnecessary escalation` · `Context re-read` · `Model thrash` · `Fan-out waste` · `Recovery after miss`
|
|
93
|
-
|
|
94
|
-
**Rework lock (ALWAYS):** Rework lives **only** under `failure_type` — never as a work category. Keep category as the intended job type; set `failure_type: Rework` when rework occurred.
|
|
95
|
-
|
|
96
|
-
A line item can have a work category **and** a failure_type (e.g. Maintenance + Looping).
|
|
97
|
-
|
|
98
|
-
## Cost model
|
|
99
|
-
|
|
100
|
-
Measured tokens come from Claude Code transcripts (`~/.claude/projects/**/*.jsonl`, assistant replies deduped on `(message.id, requestId)`); Codex transcripts are read the same way. Prices are OpenRouter public list prices per million tokens, fetched at run time and saved with the report.
|
|
101
|
-
|
|
102
|
-
```
|
|
103
|
-
agent_cost_i (per reply, micro-dollars) =
|
|
104
|
-
input × price.input + output × price.output
|
|
105
|
-
+ cache_write_5m × price.cache_write + cache_write_1h × price.cache_write_1h
|
|
106
|
-
+ cache_read × price.cache_read # cache_write_1h = listed rate, else 2 × input
|
|
107
|
-
|
|
108
|
-
agent_estimated_usd = Σ agent_cost_i over every reply in the window (matched or not) # ESTIMATED, the headline
|
|
109
|
-
extrapolated_unmeasured = observer tokens of sessions with no transcript here × (measured $ per observer token)
|
|
110
|
-
# EXTRAPOLATED (low confidence); "all sessions measured" when nothing remains
|
|
111
|
-
observer_note_taker_est = note-taker (observer) tokens, deduped per reply, × its input list price
|
|
112
|
-
# priced separately, never agent cost, never in the headline
|
|
113
|
-
mistakes_estimated_usd = Σ agent_cost_i over the same-session union of wasted turns, each turn once # low figure
|
|
114
|
-
cost_per_completed_outcome = Σ attributed $ for status ∈ {shipped, completed} / count(those work items)
|
|
115
|
-
waste_rate = Σ wasted_cost / Σ attributed $ recovery_share = Σ recovery_cost / Σ attributed $
|
|
116
|
-
```
|
|
117
|
-
|
|
118
|
-
Line items are sessions: `attributed_usd` = estimated (transcript on this box) or extrapolated (no transcript). `wasted_cost` and `recovery_cost` come from the behavior pass (below), one union set for the ribbon, the line items and the mistakes line. The upper bound (redo windows plus project-wide fallback) stays in Details.
|
|
119
|
-
|
|
120
|
-
**Unauthorized blocked:** `direct_cost` $0, `risk_exposure` high, `action_status` blocked. Always keep risk_exposure non-dollar unless real cash/remediation is at stake — never invent risk dollars.
|
|
121
|
-
|
|
122
|
-
`confidence` — `high` when tokens + model + outcome are clear; `medium` when allocation across obs is judgmental; `low` when evidence is thin.
|
|
123
|
-
|
|
124
|
-
Unpriced models (not in the price list, or a negative "variable" price) are listed by name with their tokens and add nothing; they are never priced at zero silently.
|
|
125
|
-
|
|
126
|
-
## Behavior metrics (heuristic until reviewed)
|
|
127
|
-
|
|
128
|
-
A second pass over the same transcripts tags every user turn human / bot / unknown (relayed agent prompts are never the user's words), finds frustration episodes, and runs the behavior pattern detectors: invented human gates, broke working things, wrong or expensive model, over-engineering, did something not asked, fake output, false "done", wrong tool or contact, bad outbound (incidents × recipients, never dollars), memory or rule loss, jargon, unclear cause, plus tool errors and hedging (a caveat given when the answer was already available). Four summary tiles; everything else in Details. Every count is labeled `heuristic` until reviewed or classified. The optional classifier (`--classify`) is off by default, capped at $2.00 per run, uses only a regular inference `OPENROUTER_API_KEY`, and its spend is shown separately.
|
|
129
|
-
|
|
130
|
-
Three optional environment settings tune the detectors to the user; all are off by default:
|
|
131
|
-
|
|
132
|
-
- `ACR_OWNER_EMAIL` — the user's own address; an outbound send from it is flagged as "sent as the user".
|
|
133
|
-
- `ACR_DENY_INPUT` — a regex of accounts, contacts or sites the agent must never use; matching tool inputs are flagged.
|
|
134
|
-
- `ACR_ALLOWED_MODELS` — a regex of model ids that never count as "wrong model" (default: Claude models).
|
|
135
|
-
|
|
136
|
-
## Money labeling (ALWAYS)
|
|
137
|
-
|
|
138
|
-
| Label | Meaning |
|
|
139
|
-
|-------|---------|
|
|
140
|
-
| **Measured** | Provider-reported spend from a sanctioned source (today: the OpenRouter per-key snapshot, shown with its UTC bucket label) |
|
|
141
|
-
| **Estimated** | Measured transcript tokens × OpenRouter list price per MTok (the formula above) |
|
|
142
|
-
| **Extrapolated** | Sessions with no transcript on this machine, from the observer-token ratio; always "low confidence" |
|
|
143
|
-
| **Unavailable** | No measured value — write `measured spend unavailable`, never `$0 spent` for unknown |
|
|
144
|
-
| **risk_exposure** | Severity / qualitative unless real cash is at stake (seats, refunds, SLA) |
|
|
145
|
-
|
|
146
|
-
`cost_basis` on each line item: `estimated_usage`, `measured_provider`, or `extrapolated`. Dollars are shown to two decimals everywhere (`$109.25`); every figure carries its label and basis.
|
|
147
|
-
|
|
148
|
-
## Line-item schema
|
|
149
|
-
|
|
150
|
-
Each row in `line-items.csv` / `report.json.line_items`:
|
|
151
|
-
|
|
152
|
-
| Field | Notes |
|
|
153
|
-
|-------|-------|
|
|
154
|
-
| `work_item_id` | Stable id (`WI-1` …) |
|
|
155
|
-
| `title` | Human work-item title (ALWAYS invent at review; drafts use the session's first prompt) |
|
|
156
|
-
| `scope` / `project` / `worktree` | Period, session or project; primary project; worktree when relevant |
|
|
157
|
-
| `session_ids` / `content_session_id` | Mem session id(s) and the transcript join key |
|
|
158
|
-
| `status` | `shipped` / `completed` / `in_progress` / `abandoned` / `blocked` |
|
|
159
|
-
| `category` / `work_category` | One of the six work categories |
|
|
160
|
-
| `failure_type` / `failure_signals` | Empty or failure types (Rework lives here) |
|
|
161
|
-
| `cost_measured` | Number or `unavailable` |
|
|
162
|
-
| `cost_estimated` / `cost_extrapolated` / `attributed_usd` | List-price estimate; extrapolation for sessions with no transcript; the one used |
|
|
163
|
-
| `wasted_cost` / `recovery_cost` / `productive_cost` | USD from the behavior union (0 if none) |
|
|
164
|
-
| `risk_exposure` | `none` / `low` / `medium` / `high` (non-dollar unless real cash) |
|
|
165
|
-
| `evidence_ids` / `summary_ids` | Observation and summary IDs |
|
|
166
|
-
| `agent_tokens` | `{input, output, cache_write_5m, cache_write_1h, cache_read}` |
|
|
167
|
-
| `observer_tokens` | Note-taker tokens, Details only |
|
|
168
|
-
| `model` / `model_prices_usd_per_mtok` | Dominant model and the prices used |
|
|
169
|
-
| `cost_basis` | `estimated_usage` / `measured_provider` / `extrapolated` |
|
|
170
|
-
| `label_source` | `keyword` (draft) / `llm` / `human`, with `reviewed_by` and `reviewed_at_pt` |
|
|
171
|
-
| `device` | `local`, `remote-N`, or an export label such as `mac` |
|
|
172
|
-
| `behavior_counts` | Pattern hits in this session |
|
|
173
|
-
| `recommended_action` / `confidence` / `date_pt` / `notes` | One lever; high/medium/low; PT day; one plain sentence |
|
|
174
|
-
|
|
175
|
-
## Deliverables (ALWAYS)
|
|
176
|
-
|
|
177
|
-
`acr.py` writes one directory containing:
|
|
178
|
-
|
|
179
|
-
1. **`report.html`** — self-contained (inline CSS, no script, no external resources), Timing-style
|
|
180
|
-
2. **`report.pdf`** — from `report.print.html` with headless Chrome (`google-chrome` on PATH, else Chrome's standard install location on macOS and Windows); when Chrome is missing the run says "PDF skipped, HTML is canonical"
|
|
181
|
-
3. **`report.json`** — window, scope, spend, totals, by_day, by_model, by_device, line_items, wins, behavior, timeline
|
|
182
|
-
4. **`line-items.csv`** — one row per work item
|
|
183
|
-
5. **`evidence.json`** — observation IDs cited, short titles, observer tokens, model, session ids
|
|
184
|
-
6. **`labels.review.json`** — the drafted labels for the review pass
|
|
185
|
-
|
|
186
|
-
### Manager-facing HTML sections (required order)
|
|
187
|
-
|
|
188
|
-
1. **Hero** — dollars with its tag (ESTIMATE / MEASURED), the basis sentence, "N things finished, about $X each", the extrapolated line, "Measured provider spend: unavailable" when it is, and the story sentence (computed, never hand-written)
|
|
189
|
-
2. **Wins vs mistakes** — the cost-of-mistakes line (low figure), wins shipped (merged PRs, published versions, praise; cost "unmeasured" until sessions are linked to PRs), and two timelines on one PT-day axis
|
|
190
|
-
3. **Cost ribbon** — every piece of work as a block sized by cost, waste hatched, in-progress striped
|
|
191
|
-
4. **Where the money went** (donut by kind of work) · **Day by day** (stacked bars, empty days say "no agent work") · **How much was useful** (the ring, the behavior strip with at most four tiles)
|
|
192
|
-
5. **What got done** — one folded row per work item, most expensive first, draft labels marked
|
|
193
|
-
6. **Worth your attention** — at most three, computed
|
|
194
|
-
7. **Details & evidence** (folded; open in the PDF) — wins, mistakes by day, behavior pattern table with the upper bound, rule effectiveness, spend and pricing, models, failure accounting, line items, devices, unmatched transcripts, unpriced models, label counts, the honesty rules, links to the CSV and JSON
|
|
195
|
-
|
|
196
|
-
## Truthfulness ALWAYS rules
|
|
197
|
-
|
|
198
|
-
Positive truth rules only:
|
|
199
|
-
|
|
200
|
-
1. Always label estimate vs measured (`cost_basis` = `estimated_usage` or `measured_provider`).
|
|
201
|
-
2. Always say **measured spend unavailable** when unmatched — never "$0 spent" for unknown.
|
|
202
|
-
3. Always use **completed outcomes** as the unit (cost per completed outcome).
|
|
203
|
-
4. Always keep `risk_exposure` non-dollar unless real cash/remediation is at stake.
|
|
204
|
-
5. Always invent human work-item titles; observation titles are evidence-only.
|
|
205
|
-
6. Always progressive Mem Search; evidence IDs live in the appendix.
|
|
206
|
-
7. Always put Rework under `failure_type` only.
|
|
207
|
-
8. Always default scope to most recent sessions unless the user names another.
|
|
208
|
-
|
|
209
|
-
Added by the rebuild, same spirit: the note-taker's cost is separate and never in the headline; keyword labels are drafts until reviewed; Grok Bot and other-machine figures are "unavailable" or "extrapolated (low confidence)", never $0 and never guessed; the live database is only ever opened read-only (every run works on a snapshot).
|
|
210
|
-
|
|
211
|
-
## Recipe (orchestrator)
|
|
212
|
-
|
|
213
|
-
All commands from the skill directory; `OUT` is one run directory (for example `/tmp/acr-weekly/run`). Every step is idempotent.
|
|
214
|
-
|
|
215
|
-
1. **Prices** — `python3 scripts/acr.py prices --out $OUT` (OpenRouter list prices; `--prices <saved prices.json>` to run offline; no key needed).
|
|
216
|
-
2. **Collect** — `python3 scripts/acr.py collect --out $OUT` (default window) or `… --start YYYY-MM-DD --end YYYY-MM-DD`, `… --session <id>`, `… --project <name> --start … --end …`.
|
|
217
|
-
3. **Rollup** — `python3 scripts/acr.py rollup --out $OUT` with the same period flags. Writes report.json, line-items.csv, evidence.json, labels.review.json. Options: `--no-gh` (skip the read-only `gh pr view` confirmation of merged PRs), `--classify` (see above), `--rules-dir <your rule files>` (rule effectiveness: dated `HARD` headers such as `# Ask before sending email (HARD 2026-09-18)`; `scripts/acr/rules.json` ships empty).
|
|
218
|
-
4. **Review** — for each line item in `labels.review.json`, `get_observations` on its `evidence_ids` (not whole timelines), confirm or change `category`, `failure_type`, `status`, and the title; write the reviewed file with `reviewed_by` (model id or `user`); apply with `python3 scripts/acr.py review --apply <reviewed.json> --out $OUT`. Items left unreviewed stay marked "draft label".
|
|
219
|
-
5. **Render** — `python3 scripts/acr.py render --in $OUT/report.json --out $OUT --print`.
|
|
220
|
-
6. **PDF** — `python3 scripts/acr.py pdf --out $OUT` (skipped with a message when Chrome is missing).
|
|
221
|
-
7. **Optional, needs the user's go — measured OpenRouter spend**: with a regular inference key in the environment, `OPENROUTER_API_KEY=… python3 scripts/acr.py measure-openrouter --out $OUT` before step 3. Without the key the file says `unavailable` and nothing else changes.
|
|
222
|
-
8. **Optional, needs the user's go — another machine's export**: `python3 scripts/acr.py rollup … --device-usage <device-usage-<label>.json>` (see Gaps and gates).
|
|
223
|
-
9. Return absolute paths, the skill slug `agent-cost-report`, the headline dollars with their label, the mistakes line, the wins, PDF yes/no, and how the report answers the five product questions.
|
|
224
|
-
|
|
225
|
-
## Gaps and gates
|
|
226
|
-
|
|
227
|
-
Each of these needs the user's explicit go; the report shows the honest fallback until then.
|
|
228
|
-
|
|
229
|
-
- **OpenRouter key (needs the user)** — a regular inference `OPENROUTER_API_KEY` supplied as an environment variable; never a management or provisioning key, never read from a settings file, never printed or written to any output. The per-key endpoint gives UTC day/week/month snapshots, so the figure is shown with its bucket label and switches the hero to MEASURED only when the bucket fully covers the report window. A range-by-day source (`/api/v1/activity`) exists but needs a management key, so it is not implemented.
|
|
230
|
-
- **Other machines (needs the user)** — sessions that ran on another machine (for example a laptop) have no transcript on this one, so their cost is EXTRAPOLATED (low confidence) until that machine's export is merged. The user runs, on the other machine, from a plain checkout of this skill (system `python3` 3.9+): `python3 scripts/acr.py collect --export-device laptop --start YYYY-MM-DD --end YYYY-MM-DD --out ~/acr-export` and shares `~/acr-export/device-usage-laptop.json` (ids, timestamps, token counts, model names; no prompt or observation text, no settings, no paths). Never run it on another machine without the user's go for that specific run. Merge with `--device-usage`; joined sessions flip to `estimated_usage` with `device: laptop`.
|
|
231
|
-
- **Grok Bot (ALWAYS unavailable, no seat count)** — Grok Bot is Cursor's cloud agent. Cursor shows its weekly usage only on the plan screen; there is no API or export for it. The report says exactly "Grok Bot usage: unavailable": no dollars, no guessed per-seat price, no seat count. A figure the user supplies by hand would be entered as `measured_manual` with `entered_by: the user` and the date, never inferred; that would be a new decision.
|
|
232
|
-
- **Wins from GitHub (read-only)** — besides box transcripts and ship observations, `rollup` lists merged PRs with `gh pr list --state merged` for the repos in `--wins-repo` (default `imogenlabs/memindex`), filtered by merge time inside the PT window and deduplicated by repo and PR number. When `gh` is missing, unauthenticated or rate-limited, that source reads "unavailable" in Details and the report still completes; it is never a zero. `--no-gh` turns every gh call off.
|
|
233
|
-
- **Win cost** — sessions are not linked to PRs today, so every win reads "unmeasured (session not linked to PR)". Once commits carry a `Claude-Session: <content_session_id>` trailer (trailer only, bare id), PR and publish wins get `ESTIMATED · session-linked` costs. There is no fallback estimate.
|
|
234
|
-
|
|
235
|
-
## Related
|
|
236
|
-
|
|
237
|
-
- memindex `mem-search` skill — progressive recall
|
|
238
|
-
|
|
239
|
-
## Creed
|
|
240
|
-
|
|
241
|
-
**MemIndex skill · outcomes first · estimates labeled · most recent by default · rework under failure_type · progressive recall · HTML is canonical · humans see what the agent did and why it cost money.**
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
"""agent-cost-report helpers (stdlib only)."""
|