agent-coach-cli 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 agent-coach contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
@@ -0,0 +1,242 @@
1
+ Metadata-Version: 2.4
2
+ Name: agent-coach-cli
3
+ Version: 0.1.0
4
+ Summary: Grade and coach your Claude Code and Codex prompts from your own conversation history, against Anthropic's and OpenAI's official guides.
5
+ License-Expression: MIT
6
+ Project-URL: Homepage, https://github.com/1017toa/agent-coach
7
+ Project-URL: Issues, https://github.com/1017toa/agent-coach/issues
8
+ Keywords: claude-code,codex,openai-codex,anthropic,claude,prompt-engineering,prompting,coaching,ai-agents,coding-agents,agentic-coding,developer-tools,cli,claude-md,agents-md
9
+ Classifier: Programming Language :: Python :: 3
10
+ Classifier: Programming Language :: Python :: 3 :: Only
11
+ Classifier: Operating System :: OS Independent
12
+ Classifier: Topic :: Software Development
13
+ Classifier: Environment :: Console
14
+ Classifier: Intended Audience :: Developers
15
+ Classifier: Natural Language :: English
16
+ Classifier: Natural Language :: Korean
17
+ Requires-Python: >=3.10
18
+ Description-Content-Type: text/markdown
19
+ License-File: LICENSE
20
+ Dynamic: license-file
21
+
22
+ <div align="center">
23
+
24
+ <h1>agent-coach</h1>
25
+
26
+ <p><b>Grade and coach your Claude Code and Codex prompts from your own conversation history, against Anthropic's and OpenAI's official guides.</b></p>
27
+
28
+ <p>
29
+ <a href="https://github.com/1017toa/agent-coach/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT license"></a>
30
+ <a href="https://pypi.org/project/agent-coach-cli/"><img src="https://img.shields.io/pypi/v/agent-coach-cli.svg?label=PyPI" alt="PyPI"></a>
31
+ <img src="https://img.shields.io/badge/python-3.10%2B-3776AB.svg?logo=python&logoColor=white" alt="Python 3.10+">
32
+ <img src="https://img.shields.io/badge/dependencies-0-brightgreen.svg" alt="Zero dependencies">
33
+ <img src="https://img.shields.io/badge/works%20with-Claude%20Code%20%7C%20Codex-8A63D2.svg" alt="Works with Claude Code and Codex">
34
+ <img src="https://img.shields.io/badge/runs-100%25%20local-555.svg" alt="Runs locally">
35
+ </p>
36
+
37
+ <p>
38
+ <a href="#quick-start">Quick start</a> ·
39
+ <a href="#what-it-grades">What it grades</a> ·
40
+ <a href="#missed-responses">Missed responses</a> ·
41
+ <a href="#faq">FAQ</a> ·
42
+ <a href="https://github.com/1017toa/agent-coach/blob/main/README.ko.md">한국어</a>
43
+ </p>
44
+
45
+ <picture>
46
+ <source media="(prefers-color-scheme: dark)" srcset="https://github.com/1017toa/agent-coach/raw/main/docs/images/overview-dark.png">
47
+ <img alt="agent-coach report: overall score, the three habits to work on, a radar chart of 11 prompting habits and a weakest-first list" src="https://github.com/1017toa/agent-coach/raw/main/docs/images/overview-light.png" width="900">
48
+ </picture>
49
+
50
+ <sub>Demo report built from made-up conversations (<code>python scripts/demo_report.py</code>).</sub>
51
+
52
+ </div>
53
+
54
+ **agent-coach** is a command-line tool that reads the transcripts Claude Code and OpenAI Codex already save on your machine (`~/.claude/projects`, `~/.codex/sessions`) and scores how you direct the agent on 11 habits taken from the official Anthropic and OpenAI guides and academies. It rewrites prompts you actually typed into better versions, finds the turns where you didn't get what you wanted, and tracks your scores run after run so you can see whether you're improving.
55
+
56
+ ## Quick start
57
+
58
+ ```bash
59
+ pipx install agent-coach-cli # or: uv tool install agent-coach-cli · pip install agent-coach-cli
60
+ agent-coach run
61
+ ```
62
+
63
+ The package on PyPI is `agent-coach-cli`; the command it installs is `agent-coach`. `run` grades your recent conversations through the `claude` or `codex` CLI you're already signed in to, saves the result and opens an HTML report. No API key, no server, no dependencies. Want to look before grading anything? `agent-coach scan` lists what would be graded, without calling a model.
64
+
65
+ ### Or use it inside Claude Code or Codex
66
+
67
+ Install the plugin and let the agent run it and coach you in the chat, with nothing else to install.
68
+
69
+ ```text
70
+ # Claude Code
71
+ /plugin install agent-coach --marketplace 1017toa/agent-coach
72
+ /agent-coach:coach
73
+ ```
74
+
75
+ ```bash
76
+ # Codex
77
+ codex plugin marketplace add 1017toa/agent-coach
78
+ codex plugin add agent-coach@agent-coach
79
+ ```
80
+
81
+ Then ask Codex to "grade my prompting with agent-coach". The skill previews what will be graded, runs the grading, and walks you through your weakest habits, rewrites and rules worth adding to `CLAUDE.md` / `AGENTS.md`.
82
+
83
+ ## Highlights
84
+
85
+ - **Grounded in the official guides.** Every habit cites the guide section it comes from: Claude Code best practices, Claude Academy, Codex best practices, OpenAI Academy ([rubric](https://github.com/1017toa/agent-coach/blob/main/agentcoach/rubric/default.json)).
86
+ - **Rewrites of your real prompts.** "You wrote this; this would have worked better, because…", not generic tips.
87
+ - **Short prompts aren't penalized.** In a long conversation, "ok do it" is often all you need. Coaching targets turns that actually missed, and only when the cause was a gap in your prompt.
88
+ - **Finds rules worth writing down.** Gaps that recur across conversations become one-line rules, with where to put them: `CLAUDE.md` / `AGENTS.md`, memory, a skill, or a hook.
89
+ - **Tracks progress.** Each run is saved under `~/.agent-coach/history/`, so your trend survives after the tools delete old transcripts.
90
+ - **Claude Code and Codex on one scale**, in one report.
91
+ - **Private by default.** Secrets and contact details are masked before anything reaches the judge, and results stay on your machine.
92
+ - **English and Korean.** Report, CLI and coaching in either language (`--lang en|ko`).
93
+
94
+ ## Supported tools
95
+
96
+ | Tool | Transcripts read from | Notes |
97
+ |---|---|---|
98
+ | Claude Code (CLI, desktop, IDE) | `~/.claude/projects/**/*.jsonl` | Scripted `claude -p` runs and app-generated turns are skipped |
99
+ | OpenAI Codex (CLI, IDE) | `~/.codex/sessions/**/rollout-*.jsonl` | `codex exec` and agent-spawned sessions are skipped |
100
+
101
+ Judge: `claude` CLI, `codex` CLI, or offline keyword scoring (`--judge heuristic`).
102
+
103
+ ## How it works
104
+
105
+ ```
106
+ ~/.claude, ~/.codex transcripts
107
+ │
108
+ ● collect your own typed turns only (automation, slash commands, app messages filtered out)
109
+ │
110
+ ● redact keys, emails, phone / ID / card numbers, addresses, names you list
111
+ │
112
+ ● judge one call per conversation via your claude / codex CLI, rubric v2 (11 habits)
113
+ │
114
+ ● save ~/.agent-coach/history/run-*.json
115
+ │
116
+ ✓ report one self-contained HTML file: overview · coaching · missed responses · conversations · progress
117
+ ```
118
+
119
+ ## What it grades
120
+
121
+ Eleven habits, each scored 0 / 1 / 2 only in the conversations where it mattered, averaged to 0–100.
122
+
123
+ | Habit | What the guides recommend |
124
+ |---|---|
125
+ | Specific context | State the goal and point at the file, screen and symptom from the start |
126
+ | Done criteria and verification | Give a pass/fail check (tests, build, screenshot) and let the agent run it |
127
+ | Constraints and preferences | Say what must not change and which approach you prefer |
128
+ | Hand over the real material | Attach the @file, screenshot or full error instead of describing it |
129
+ | Scope and output shape | Minimal or full version; format and length of the result |
130
+ | Plan or interview first | For big or fuzzy work, plan or get interviewed before building |
131
+ | Stop points and working style | How far to go alone, where to stop and ask, how to report |
132
+ | Ask for evidence, review the result | Commands, output, diff; a fresh session to review |
133
+ | Course correction | After two failed corrections, start fresh with a better prompt |
134
+ | Move repeated rules into files | Put recurring rules in CLAUDE.md, AGENTS.md or a skill |
135
+ | One outcome per conversation | New work in a new conversation; tidy long ones |
136
+
137
+ <img alt="Coaching tab: a habit's score, rewrites of the user's own prompts side by side with better versions, and the official guidance behind it" src="https://github.com/1017toa/agent-coach/raw/main/docs/images/coaching.png" width="900">
138
+
139
+ ## Missed responses
140
+
141
+ Context builds up over a conversation, so a short instruction can be exactly right. agent-coach never marks a prompt down for being short. It looks for **turns that didn't get what you wanted**, judged by what happened next:
142
+
143
+ - **Signals**: you corrected the result, repeated an earlier instruction, asked to redo or undo, reported damage, showed frustration, or the agent had to ask back.
144
+ - **Causes**: *gap in your prompt* (something only you knew was missing and context didn't cover it) / *agent error* (context was enough) / *changed your mind* / *external limit*.
145
+ - **Coaching**: only prompt gaps get a rewrite, phrased as "the one missing piece". Gaps that recur become rule candidates.
146
+
147
+ <img alt="Missed responses tab: turns split by cause, the prompt, what happened, the one missing piece and a recurring-gap rule" src="https://github.com/1017toa/agent-coach/raw/main/docs/images/missed-responses.png" width="900">
148
+
149
+ Causes are classified only with an LLM judge; keyword scoring finds the signals and leaves the cause unclassified.
150
+
151
+ ## Usage
152
+
153
+ ```bash
154
+ agent-coach scan # preview which conversations would be graded (no model calls)
155
+ agent-coach run # grade → save the run → open the report
156
+ agent-coach summary # latest run as text: weakest habits, rewrites, rules to add
157
+ agent-coach history # score table per run
158
+ agent-coach report # rebuild the report from saved runs
159
+ ```
160
+
161
+ | Option | Meaning |
162
+ |---|---|
163
+ | `--days 30` / `--all` | Time window (default: since the previous run; 14 days on the first run) |
164
+ | `--judge claude\|codex\|heuristic` | Which judge (default: auto) |
165
+ | `--model sonnet` | Judge model |
166
+ | `--max-sessions 20` | Max conversations to grade; each one is one model call |
167
+ | `--source claude` / `--source codex` | Only one tool |
168
+ | `--rubric my-rubric.json` | Your own rubric, for a course or a team |
169
+ | `--lang en\|ko` | Language of the report and coaching (default: OS locale, or `AGENT_COACH_LANG`) |
170
+
171
+ From a checkout without installing: `python -m agentcoach run`. Latest development version: `pip install git+https://github.com/1017toa/agent-coach.git`.
172
+
173
+ **Grade on a schedule.** Run `agent-coach run` every week or two; each run grades only what's new and adds to the trend. Claude Code deletes transcripts older than 30 days by default, so keep the interval shorter than that.
174
+
175
+ ## How it compares
176
+
177
+ | | agent-coach | Token and cost trackers (e.g. ccusage) | Prompt eval frameworks (e.g. promptfoo) |
178
+ |---|---|---|---|
179
+ | Looks at | How **you** direct the agent | Tokens and spend | Your app's prompts and model outputs |
180
+ | Input | Your Claude Code / Codex history | Your Claude Code / Codex history | Test cases you write |
181
+ | Output | Scores per habit, rewrites, missed turns, rules to add | Usage tables | Pass/fail matrices |
182
+ | For | Developers, teams and courses learning to work with coding agents | Budgeting | Building LLM apps |
183
+
184
+ They answer different questions and work fine side by side.
185
+
186
+ ## FAQ
187
+
188
+ ### How do I improve my Claude Code or Codex prompts?
189
+ Run `agent-coach run` and open the **Coaching** tab. It starts with your three weakest habits and shows prompts you actually wrote next to rewritten versions, with the official guide section behind each one. Re-run in a week or two and check the **Progress** tab.
190
+
191
+ ### Does it send my conversations anywhere?
192
+ Raw transcripts never leave your machine. With an LLM judge, the masked prompts of each graded conversation and one-line summaries of what the agent did are sent to the Claude or Codex account you're signed in to, the same place your conversations already went. `--judge heuristic` grades fully offline.
193
+
194
+ ### Do I need an API key?
195
+ No. Grading runs through the `claude` or `codex` CLI you're already logged in to, using your existing plan. Without either, it falls back to offline keyword scoring.
196
+
197
+ ### How much does a run cost?
198
+ One judge call per conversation, on your existing plan. Use `--max-sessions` to cap it and `agent-coach scan` to preview the count first.
199
+
200
+ ### Why wasn't my short prompt marked down?
201
+ Because short prompts are often right. "Yes, do it" after a long discussion has all the context it needs. agent-coach only coaches a turn when what happened next shows it missed, and only rewrites it when the missing piece was something only you knew.
202
+
203
+ ### Can I use my own rubric for a course or a team?
204
+ Yes. Copy [`agentcoach/rubric/default.json`](https://github.com/1017toa/agent-coach/blob/main/agentcoach/rubric/default.json), edit the habits and examples, and pass `--rubric your-rubric.json`. A rubric without English fields is shown as written.
205
+
206
+ ### Where are results stored?
207
+ `~/.agent-coach/history/run-*.json` (one file per run) and `~/.agent-coach/report.html`. Set `AGENT_COACH_HOME` to move them.
208
+
209
+ ### How do I hide my name or company?
210
+ Names can't be detected reliably, so list yours (name, romanized name, company, ...) one per line in `~/.agent-coach/redact.txt`. Keys, tokens, emails, phone, national ID and card numbers, and Korean road addresses are masked automatically.
211
+
212
+ ### Does it support Cursor, Gemini CLI or Aider?
213
+ Not yet; Claude Code and Codex only. Support for more tools is on the roadmap, and parsers are small (see [`agentcoach/collect.py`](https://github.com/1017toa/agent-coach/blob/main/agentcoach/collect.py)).
214
+
215
+ ## Sources
216
+
217
+ The default rubric (v2) is built from:
218
+
219
+ - **Anthropic**: [Best practices for Claude Code](https://code.claude.com/docs/en/best-practices), [Getting the most out of Opus 5.5](https://claude.dev/blog/getting-the-most-out-of-opus-5-5/)
220
+ - **Claude Academy**: [AI Fluency: Framework & Foundations](https://academy.claude.com/courses/ai-fluency-framework-foundations), [Claude Code 101](https://academy.claude.com/courses/claude-code-101), [Claude Code in Action](https://academy.claude.com/courses/claude-code-in-action), [Claude 101](https://academy.claude.com/courses/claude-101)
221
+ - **OpenAI**: [Codex Best practices](https://learn.chatgpt.com/guides/best-practices), [Codex Prompting](https://learn.chatgpt.com/docs/prompting)
222
+ - **OpenAI Academy**: [Get Started with Codex](https://academy.openai.com/public/courses/get-started-with-codex-jkhsq), [Agents and Workflows](https://academy.openai.com/public/courses/agents-and-workflows-y0qoc), [Extend Codex Workflows](https://academy.openai.com/public/courses/extend-codex-workflows-4e61p) (course bodies need sign-in; only the public overviews are used)
223
+
224
+ When the rubric changes, earlier runs have no score for new habits and the report says the versions differ.
225
+
226
+ ## Roadmap
227
+
228
+ - Anonymized aggregate export for courses and teams
229
+ - Example custom rubrics
230
+ - Cursor and Gemini CLI transcripts
231
+
232
+ ## Contributing
233
+
234
+ Issues and pull requests are welcome; see [CONTRIBUTING.md](https://github.com/1017toa/agent-coach/blob/main/CONTRIBUTING.md). Run the tests with:
235
+
236
+ ```bash
237
+ python -m unittest discover -s tests
238
+ ```
239
+
240
+ ## License
241
+
242
+ [MIT](https://github.com/1017toa/agent-coach/blob/main/LICENSE)
@@ -0,0 +1,221 @@
1
+ <div align="center">
2
+
3
+ <h1>agent-coach</h1>
4
+
5
+ <p><b>Grade and coach your Claude Code and Codex prompts from your own conversation history, against Anthropic's and OpenAI's official guides.</b></p>
6
+
7
+ <p>
8
+ <a href="https://github.com/1017toa/agent-coach/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="MIT license"></a>
9
+ <a href="https://pypi.org/project/agent-coach-cli/"><img src="https://img.shields.io/pypi/v/agent-coach-cli.svg?label=PyPI" alt="PyPI"></a>
10
+ <img src="https://img.shields.io/badge/python-3.10%2B-3776AB.svg?logo=python&logoColor=white" alt="Python 3.10+">
11
+ <img src="https://img.shields.io/badge/dependencies-0-brightgreen.svg" alt="Zero dependencies">
12
+ <img src="https://img.shields.io/badge/works%20with-Claude%20Code%20%7C%20Codex-8A63D2.svg" alt="Works with Claude Code and Codex">
13
+ <img src="https://img.shields.io/badge/runs-100%25%20local-555.svg" alt="Runs locally">
14
+ </p>
15
+
16
+ <p>
17
+ <a href="#quick-start">Quick start</a> ·
18
+ <a href="#what-it-grades">What it grades</a> ·
19
+ <a href="#missed-responses">Missed responses</a> ·
20
+ <a href="#faq">FAQ</a> ·
21
+ <a href="https://github.com/1017toa/agent-coach/blob/main/README.ko.md">한국어</a>
22
+ </p>
23
+
24
+ <picture>
25
+ <source media="(prefers-color-scheme: dark)" srcset="https://github.com/1017toa/agent-coach/raw/main/docs/images/overview-dark.png">
26
+ <img alt="agent-coach report: overall score, the three habits to work on, a radar chart of 11 prompting habits and a weakest-first list" src="https://github.com/1017toa/agent-coach/raw/main/docs/images/overview-light.png" width="900">
27
+ </picture>
28
+
29
+ <sub>Demo report built from made-up conversations (<code>python scripts/demo_report.py</code>).</sub>
30
+
31
+ </div>
32
+
33
+ **agent-coach** is a command-line tool that reads the transcripts Claude Code and OpenAI Codex already save on your machine (`~/.claude/projects`, `~/.codex/sessions`) and scores how you direct the agent on 11 habits taken from the official Anthropic and OpenAI guides and academies. It rewrites prompts you actually typed into better versions, finds the turns where you didn't get what you wanted, and tracks your scores run after run so you can see whether you're improving.
34
+
35
+ ## Quick start
36
+
37
+ ```bash
38
+ pipx install agent-coach-cli # or: uv tool install agent-coach-cli · pip install agent-coach-cli
39
+ agent-coach run
40
+ ```
41
+
42
+ The package on PyPI is `agent-coach-cli`; the command it installs is `agent-coach`. `run` grades your recent conversations through the `claude` or `codex` CLI you're already signed in to, saves the result and opens an HTML report. No API key, no server, no dependencies. Want to look before grading anything? `agent-coach scan` lists what would be graded, without calling a model.
43
+
44
+ ### Or use it inside Claude Code or Codex
45
+
46
+ Install the plugin and let the agent run it and coach you in the chat, with nothing else to install.
47
+
48
+ ```text
49
+ # Claude Code
50
+ /plugin install agent-coach --marketplace 1017toa/agent-coach
51
+ /agent-coach:coach
52
+ ```
53
+
54
+ ```bash
55
+ # Codex
56
+ codex plugin marketplace add 1017toa/agent-coach
57
+ codex plugin add agent-coach@agent-coach
58
+ ```
59
+
60
+ Then ask Codex to "grade my prompting with agent-coach". The skill previews what will be graded, runs the grading, and walks you through your weakest habits, rewrites and rules worth adding to `CLAUDE.md` / `AGENTS.md`.
61
+
62
+ ## Highlights
63
+
64
+ - **Grounded in the official guides.** Every habit cites the guide section it comes from: Claude Code best practices, Claude Academy, Codex best practices, OpenAI Academy ([rubric](https://github.com/1017toa/agent-coach/blob/main/agentcoach/rubric/default.json)).
65
+ - **Rewrites of your real prompts.** "You wrote this; this would have worked better, because…", not generic tips.
66
+ - **Short prompts aren't penalized.** In a long conversation, "ok do it" is often all you need. Coaching targets turns that actually missed, and only when the cause was a gap in your prompt.
67
+ - **Finds rules worth writing down.** Gaps that recur across conversations become one-line rules, with where to put them: `CLAUDE.md` / `AGENTS.md`, memory, a skill, or a hook.
68
+ - **Tracks progress.** Each run is saved under `~/.agent-coach/history/`, so your trend survives after the tools delete old transcripts.
69
+ - **Claude Code and Codex on one scale**, in one report.
70
+ - **Private by default.** Secrets and contact details are masked before anything reaches the judge, and results stay on your machine.
71
+ - **English and Korean.** Report, CLI and coaching in either language (`--lang en|ko`).
72
+
73
+ ## Supported tools
74
+
75
+ | Tool | Transcripts read from | Notes |
76
+ |---|---|---|
77
+ | Claude Code (CLI, desktop, IDE) | `~/.claude/projects/**/*.jsonl` | Scripted `claude -p` runs and app-generated turns are skipped |
78
+ | OpenAI Codex (CLI, IDE) | `~/.codex/sessions/**/rollout-*.jsonl` | `codex exec` and agent-spawned sessions are skipped |
79
+
80
+ Judge: `claude` CLI, `codex` CLI, or offline keyword scoring (`--judge heuristic`).
81
+
82
+ ## How it works
83
+
84
+ ```
85
+ ~/.claude, ~/.codex transcripts
86
+ │
87
+ ● collect your own typed turns only (automation, slash commands, app messages filtered out)
88
+ │
89
+ ● redact keys, emails, phone / ID / card numbers, addresses, names you list
90
+ │
91
+ ● judge one call per conversation via your claude / codex CLI, rubric v2 (11 habits)
92
+ │
93
+ ● save ~/.agent-coach/history/run-*.json
94
+ │
95
+ ✓ report one self-contained HTML file: overview · coaching · missed responses · conversations · progress
96
+ ```
97
+
98
+ ## What it grades
99
+
100
+ Eleven habits, each scored 0 / 1 / 2 only in the conversations where it mattered, averaged to 0–100.
101
+
102
+ | Habit | What the guides recommend |
103
+ |---|---|
104
+ | Specific context | State the goal and point at the file, screen and symptom from the start |
105
+ | Done criteria and verification | Give a pass/fail check (tests, build, screenshot) and let the agent run it |
106
+ | Constraints and preferences | Say what must not change and which approach you prefer |
107
+ | Hand over the real material | Attach the @file, screenshot or full error instead of describing it |
108
+ | Scope and output shape | Minimal or full version; format and length of the result |
109
+ | Plan or interview first | For big or fuzzy work, plan or get interviewed before building |
110
+ | Stop points and working style | How far to go alone, where to stop and ask, how to report |
111
+ | Ask for evidence, review the result | Commands, output, diff; a fresh session to review |
112
+ | Course correction | After two failed corrections, start fresh with a better prompt |
113
+ | Move repeated rules into files | Put recurring rules in CLAUDE.md, AGENTS.md or a skill |
114
+ | One outcome per conversation | New work in a new conversation; tidy long ones |
115
+
116
+ <img alt="Coaching tab: a habit's score, rewrites of the user's own prompts side by side with better versions, and the official guidance behind it" src="https://github.com/1017toa/agent-coach/raw/main/docs/images/coaching.png" width="900">
117
+
118
+ ## Missed responses
119
+
120
+ Context builds up over a conversation, so a short instruction can be exactly right. agent-coach never marks a prompt down for being short. It looks for **turns that didn't get what you wanted**, judged by what happened next:
121
+
122
+ - **Signals**: you corrected the result, repeated an earlier instruction, asked to redo or undo, reported damage, showed frustration, or the agent had to ask back.
123
+ - **Causes**: *gap in your prompt* (something only you knew was missing and context didn't cover it) / *agent error* (context was enough) / *changed your mind* / *external limit*.
124
+ - **Coaching**: only prompt gaps get a rewrite, phrased as "the one missing piece". Gaps that recur become rule candidates.
125
+
126
+ <img alt="Missed responses tab: turns split by cause, the prompt, what happened, the one missing piece and a recurring-gap rule" src="https://github.com/1017toa/agent-coach/raw/main/docs/images/missed-responses.png" width="900">
127
+
128
+ Causes are classified only with an LLM judge; keyword scoring finds the signals and leaves the cause unclassified.
129
+
130
+ ## Usage
131
+
132
+ ```bash
133
+ agent-coach scan # preview which conversations would be graded (no model calls)
134
+ agent-coach run # grade → save the run → open the report
135
+ agent-coach summary # latest run as text: weakest habits, rewrites, rules to add
136
+ agent-coach history # score table per run
137
+ agent-coach report # rebuild the report from saved runs
138
+ ```
139
+
140
+ | Option | Meaning |
141
+ |---|---|
142
+ | `--days 30` / `--all` | Time window (default: since the previous run; 14 days on the first run) |
143
+ | `--judge claude\|codex\|heuristic` | Which judge (default: auto) |
144
+ | `--model sonnet` | Judge model |
145
+ | `--max-sessions 20` | Max conversations to grade; each one is one model call |
146
+ | `--source claude` / `--source codex` | Only one tool |
147
+ | `--rubric my-rubric.json` | Your own rubric, for a course or a team |
148
+ | `--lang en\|ko` | Language of the report and coaching (default: OS locale, or `AGENT_COACH_LANG`) |
149
+
150
+ From a checkout without installing: `python -m agentcoach run`. Latest development version: `pip install git+https://github.com/1017toa/agent-coach.git`.
151
+
152
+ **Grade on a schedule.** Run `agent-coach run` every week or two; each run grades only what's new and adds to the trend. Claude Code deletes transcripts older than 30 days by default, so keep the interval shorter than that.
153
+
154
+ ## How it compares
155
+
156
+ | | agent-coach | Token and cost trackers (e.g. ccusage) | Prompt eval frameworks (e.g. promptfoo) |
157
+ |---|---|---|---|
158
+ | Looks at | How **you** direct the agent | Tokens and spend | Your app's prompts and model outputs |
159
+ | Input | Your Claude Code / Codex history | Your Claude Code / Codex history | Test cases you write |
160
+ | Output | Scores per habit, rewrites, missed turns, rules to add | Usage tables | Pass/fail matrices |
161
+ | For | Developers, teams and courses learning to work with coding agents | Budgeting | Building LLM apps |
162
+
163
+ They answer different questions and work fine side by side.
164
+
165
+ ## FAQ
166
+
167
+ ### How do I improve my Claude Code or Codex prompts?
168
+ Run `agent-coach run` and open the **Coaching** tab. It starts with your three weakest habits and shows prompts you actually wrote next to rewritten versions, with the official guide section behind each one. Re-run in a week or two and check the **Progress** tab.
169
+
170
+ ### Does it send my conversations anywhere?
171
+ Raw transcripts never leave your machine. With an LLM judge, the masked prompts of each graded conversation and one-line summaries of what the agent did are sent to the Claude or Codex account you're signed in to, the same place your conversations already went. `--judge heuristic` grades fully offline.
172
+
173
+ ### Do I need an API key?
174
+ No. Grading runs through the `claude` or `codex` CLI you're already logged in to, using your existing plan. Without either, it falls back to offline keyword scoring.
175
+
176
+ ### How much does a run cost?
177
+ One judge call per conversation, on your existing plan. Use `--max-sessions` to cap it and `agent-coach scan` to preview the count first.
178
+
179
+ ### Why wasn't my short prompt marked down?
180
+ Because short prompts are often right. "Yes, do it" after a long discussion has all the context it needs. agent-coach only coaches a turn when what happened next shows it missed, and only rewrites it when the missing piece was something only you knew.
181
+
182
+ ### Can I use my own rubric for a course or a team?
183
+ Yes. Copy [`agentcoach/rubric/default.json`](https://github.com/1017toa/agent-coach/blob/main/agentcoach/rubric/default.json), edit the habits and examples, and pass `--rubric your-rubric.json`. A rubric without English fields is shown as written.
184
+
185
+ ### Where are results stored?
186
+ `~/.agent-coach/history/run-*.json` (one file per run) and `~/.agent-coach/report.html`. Set `AGENT_COACH_HOME` to move them.
187
+
188
+ ### How do I hide my name or company?
189
+ Names can't be detected reliably, so list yours (name, romanized name, company, ...) one per line in `~/.agent-coach/redact.txt`. Keys, tokens, emails, phone, national ID and card numbers, and Korean road addresses are masked automatically.
190
+
191
+ ### Does it support Cursor, Gemini CLI or Aider?
192
+ Not yet; Claude Code and Codex only. Support for more tools is on the roadmap, and parsers are small (see [`agentcoach/collect.py`](https://github.com/1017toa/agent-coach/blob/main/agentcoach/collect.py)).
193
+
194
+ ## Sources
195
+
196
+ The default rubric (v2) is built from:
197
+
198
+ - **Anthropic**: [Best practices for Claude Code](https://code.claude.com/docs/en/best-practices), [Getting the most out of Opus 5.5](https://claude.dev/blog/getting-the-most-out-of-opus-5-5/)
199
+ - **Claude Academy**: [AI Fluency: Framework & Foundations](https://academy.claude.com/courses/ai-fluency-framework-foundations), [Claude Code 101](https://academy.claude.com/courses/claude-code-101), [Claude Code in Action](https://academy.claude.com/courses/claude-code-in-action), [Claude 101](https://academy.claude.com/courses/claude-101)
200
+ - **OpenAI**: [Codex Best practices](https://learn.chatgpt.com/guides/best-practices), [Codex Prompting](https://learn.chatgpt.com/docs/prompting)
201
+ - **OpenAI Academy**: [Get Started with Codex](https://academy.openai.com/public/courses/get-started-with-codex-jkhsq), [Agents and Workflows](https://academy.openai.com/public/courses/agents-and-workflows-y0qoc), [Extend Codex Workflows](https://academy.openai.com/public/courses/extend-codex-workflows-4e61p) (course bodies need sign-in; only the public overviews are used)
202
+
203
+ When the rubric changes, earlier runs have no score for new habits and the report says the versions differ.
204
+
205
+ ## Roadmap
206
+
207
+ - Anonymized aggregate export for courses and teams
208
+ - Example custom rubrics
209
+ - Cursor and Gemini CLI transcripts
210
+
211
+ ## Contributing
212
+
213
+ Issues and pull requests are welcome; see [CONTRIBUTING.md](https://github.com/1017toa/agent-coach/blob/main/CONTRIBUTING.md). Run the tests with:
214
+
215
+ ```bash
216
+ python -m unittest discover -s tests
217
+ ```
218
+
219
+ ## License
220
+
221
+ [MIT](https://github.com/1017toa/agent-coach/blob/main/LICENSE)