@olegkoval/agent-skills 1.41.4 → 1.42.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/copilot-instructions.md +1 -0
- package/.github/prompts/agent-ops-retro.prompt.md +178 -0
- package/.kiro/steering/agent-ops-retro.md +177 -0
- package/.windsurf/rules/agent-ops-retro.md +176 -0
- package/README.md +3 -2
- package/adapters/claude/olko-reflection/skills/agent-ops-retro/SKILL.md +199 -0
- package/adapters/claude/olko-reflection/skills/agent-ops-retro/scripts/mine-transcripts.mjs +245 -0
- package/adapters/codex/olko-reflection/README.md +1 -0
- package/adapters/cursor/olko-reflection/skills/agent-ops-retro/SKILL.md +199 -0
- package/adapters/cursor/olko-reflection/skills/agent-ops-retro/scripts/mine-transcripts.mjs +245 -0
- package/adapters/grok/olko-reflection/skills/agent-ops-retro/SKILL.md +199 -0
- package/adapters/grok/olko-reflection/skills/agent-ops-retro/scripts/mine-transcripts.mjs +245 -0
- package/catalog/skills.json +24 -0
- package/package.json +1 -1
- package/plugins/olko-apple-kit/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-creative/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-garmin-kit/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-git-tools/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-github-pr/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-obsidian/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-product/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-reflection/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-reflection/skills/agent-ops-retro/SKILL.md +198 -0
- package/plugins/olko-reflection/skills/agent-ops-retro/scripts/mine-transcripts.mjs +245 -0
- package/plugins/olko-release/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-skill-meta/.claude-plugin/plugin.json +1 -1
- package/plugins/olko-web-ops/.claude-plugin/plugin.json +1 -1
- package/scripts/lib/catalog.mjs +1 -1
|
@@ -41,6 +41,7 @@ Available reusable prompt files:
|
|
|
41
41
|
- self-critique: .github/prompts/self-critique.prompt.md
|
|
42
42
|
- review-past-performance: .github/prompts/review-past-performance.prompt.md
|
|
43
43
|
- retro-analysis: .github/prompts/retro-analysis.prompt.md
|
|
44
|
+
- agent-ops-retro: .github/prompts/agent-ops-retro.prompt.md
|
|
44
45
|
- crash-course: .github/prompts/crash-course.prompt.md
|
|
45
46
|
- wrap-up: .github/prompts/wrap-up.prompt.md
|
|
46
47
|
- obsidian-pr-sync: .github/prompts/obsidian-pr-sync.prompt.md
|
|
@@ -0,0 +1,178 @@
|
|
|
1
|
+
<!-- Generated by scripts/build-adapters.sh. Do not edit directly. -->
|
|
2
|
+
|
|
3
|
+
---
|
|
4
|
+
description: "Retrospective on how the agents themselves are being operated, from local transcripts plus the reports other jobs already produce: what the human keeps repeating, what the guardrails caught, and where delivery leaked, with unactioned findings carried forward."
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
Use the olko:agent-ops-retro skill.
|
|
8
|
+
|
|
9
|
+
# Agent Ops Retro
|
|
10
|
+
|
|
11
|
+
A retrospective on the operating layer, not the code. It asks whether the agents are being run
|
|
12
|
+
well: what the human had to repeat, what the guardrails caught, what they cost, and which findings
|
|
13
|
+
have been reported before and never actioned.
|
|
14
|
+
|
|
15
|
+
## Scope boundary
|
|
16
|
+
|
|
17
|
+
Three retro skills exist and they do not overlap.
|
|
18
|
+
|
|
19
|
+
| Skill | Looks at | Answers |
|
|
20
|
+
|---|---|---|
|
|
21
|
+
| `retro-analysis` | Repository history, delivery evidence, code quality | What shipped and how the code is trending |
|
|
22
|
+
| `wrap-up` | One finished change | Did this specific piece of work meet its goal |
|
|
23
|
+
| `agent-ops-retro` | Transcripts, hooks, skills, ledger, delegation logs | How the agents are being operated |
|
|
24
|
+
|
|
25
|
+
If the user asks about shipped work, hand off to `retro-analysis` and say so.
|
|
26
|
+
|
|
27
|
+
## Invocation
|
|
28
|
+
|
|
29
|
+
Accept a window: `14d` (default), `7d`, `30d`, or an explicit ISO date range. State the absolute
|
|
30
|
+
range at the top of the output. Convert relative dates to absolute.
|
|
31
|
+
|
|
32
|
+
## Step 0. Read, do not recompute
|
|
33
|
+
|
|
34
|
+
Several jobs already report on this state. Adding a fifth independent opinion is the failure mode
|
|
35
|
+
this skill exists to fix, so read their outputs first and only compute what none of them cover.
|
|
36
|
+
|
|
37
|
+
| Source | Path | Gives you |
|
|
38
|
+
|---|---|---|
|
|
39
|
+
| Skill audit | `~/.claude/skill-audit/latest.json` and the dated `.txt` beside it | Installed vs never-invoked skills |
|
|
40
|
+
| Delegation log | `~/.claude/delegation-metrics/delegations.jsonl` | Model, tier, depth, outcome, rework per spawn |
|
|
41
|
+
| Tier check | `~/.claude/delegation-metrics/tier-check-*.md` | Opus share and spend against a baseline |
|
|
42
|
+
| Usage miner | `~/obsidian/cloud-opus/Lead/Automation/Weekly Automation Report *.md` | Repeated commands, proposal fatigue, approval queue |
|
|
43
|
+
| Miner state | `~/.claude/usage-miner/state/last-run.json` | What was staged, queued, blocked |
|
|
44
|
+
| Knowledge ledger | `~/.local/share/agent-context/repo/ledger.json` | Traps already recorded, so you do not re-derive them |
|
|
45
|
+
| Prior run | the previous output of this skill (see Step 6) | Findings to carry forward |
|
|
46
|
+
|
|
47
|
+
Two traps when reading these:
|
|
48
|
+
|
|
49
|
+
- A skill-usage figure counted from attributed messages is not an invocation count. The audit's own
|
|
50
|
+
numbers run three orders of magnitude above the Skill-tool invocation count. Say which unit you
|
|
51
|
+
are quoting, every time.
|
|
52
|
+
- When two sources disagree on the same number, report both with their sources and say the figure
|
|
53
|
+
is unreconciled. Never silently pick one.
|
|
54
|
+
|
|
55
|
+
## Step 1. Mine the transcripts
|
|
56
|
+
|
|
57
|
+
This is the axis nothing else covers. Run `scripts/mine-transcripts.mjs` from this skill directory,
|
|
58
|
+
or reimplement it, over `~/.claude/projects/**/*.jsonl`.
|
|
59
|
+
|
|
60
|
+
Non-negotiable filters, each of which has burned a previous run:
|
|
61
|
+
|
|
62
|
+
- Filter by the record's own `.timestamp`, never by file mtime. A resumed session rewrites old
|
|
63
|
+
files and pulls pre-window turns into the window.
|
|
64
|
+
- A `user` record is not a human turn. Exclude `tool_result` blocks, `<system-reminder>` wrappers,
|
|
65
|
+
`<command-name>` and hook preambles, task notifications and bash echoes. In one measured corpus
|
|
66
|
+
45 percent of `user` text records were tool echoes, so the raw count nearly doubles the real one.
|
|
67
|
+
- Exclude `isSidechain` records and anything under a `subagents/` directory from session and
|
|
68
|
+
human-turn counts. Count them separately for the delegation view.
|
|
69
|
+
- Empty output is a state, not a success. If a scan returns nothing, prove the scan works before
|
|
70
|
+
concluding the thing is absent.
|
|
71
|
+
|
|
72
|
+
Extract: organic human turns per day and per session, turn-length distribution, tool call counts,
|
|
73
|
+
`tool_result` errors with `is_error`, Agent spawns with `subagent_type` and `model`, Skill
|
|
74
|
+
invocations, interruptions, compaction events, per-model token usage including cache reads, and
|
|
75
|
+
synthetic records naming a rate or spend limit.
|
|
76
|
+
|
|
77
|
+
## Step 2. Classify what the human repeated
|
|
78
|
+
|
|
79
|
+
Group the organic turns. The strongest automation signal is not raw frequency, it is the number of
|
|
80
|
+
distinct sessions a correction appears in: a thing said twenty times in one session is one
|
|
81
|
+
argument, a thing said once in twenty sessions is a missing default.
|
|
82
|
+
|
|
83
|
+
Report each recurring pattern as: name, count, distinct sessions, two or three verbatim dated
|
|
84
|
+
quotes, and the one concrete change that removes it. Name the mechanism, not the mood. Write
|
|
85
|
+
"stops for a one-word ack on reversible steps", not "autonomy could be better".
|
|
86
|
+
|
|
87
|
+
## Step 3. Split what holds from what is broken
|
|
88
|
+
|
|
89
|
+
Produce two explicit lists, never a narrative.
|
|
90
|
+
|
|
91
|
+
**Verified as working:** the claim and what proved it. Guardrails that fired correctly count here.
|
|
92
|
+
|
|
93
|
+
**Defects:** the mechanism and its consequence. Include the governance frictions, which usually
|
|
94
|
+
outnumber the external failures: permission denials, classifier blocks, path-gate refusals, agent
|
|
95
|
+
output failing its own schema. Give each a count and a share of total errors.
|
|
96
|
+
|
|
97
|
+
## Step 4. Delivery leakage
|
|
98
|
+
|
|
99
|
+
Delivery here means whether agent output landed, not whether it was produced.
|
|
100
|
+
|
|
101
|
+
- Open PRs authored by the user, bucketed by age, with fan-out batches identified as batches.
|
|
102
|
+
- Tickets whose completion timestamps cluster in minutes, which is a status backfill rather than
|
|
103
|
+
shipping. State both the raw completion count and the count after removing the cluster.
|
|
104
|
+
- Any fan-out with no named landing mechanism. That is backlog, not delivery.
|
|
105
|
+
- Scheduled jobs: check `launchctl list` exit codes and each job's own log. A wrapper exit code is
|
|
106
|
+
not completion.
|
|
107
|
+
|
|
108
|
+
## Step 5. Close the knowledge circle
|
|
109
|
+
|
|
110
|
+
This is the half that makes the retro compound instead of repeating.
|
|
111
|
+
|
|
112
|
+
1. Before writing a finding, check whether the ledger already holds it:
|
|
113
|
+
`~/.local/share/agent-context/repo/recipes/tools/ledger-index --kind trap --scope <scope>`.
|
|
114
|
+
A finding the ledger already records is a compliance gap, not a discovery. Say which it is.
|
|
115
|
+
2. For each genuinely new, durable, cross-cutting trap, draft a note: `kind`, `scope`, `title`,
|
|
116
|
+
`body`, and a `why` naming the concrete failure. A finding that is only true this week, or only
|
|
117
|
+
true in one repository, does not go in the ledger. It goes in the report.
|
|
118
|
+
3. Appending is a shared write. Take the lease first
|
|
119
|
+
(`recipes/tools/task-claim acquire ledger-append <agent> 20`), append without touching any
|
|
120
|
+
existing note, run `node recipes/tools/validate-ledger.js`, assert the note count rose by
|
|
121
|
+
exactly the number added and that no prior id vanished, commit, push, then verify
|
|
122
|
+
`git rev-parse HEAD` against `git ls-remote origin main` and quote both. Release the lease even
|
|
123
|
+
on failure.
|
|
124
|
+
4. Never append a credential, a personal path that is not the user's own machine, or a fact that is
|
|
125
|
+
only true this week.
|
|
126
|
+
|
|
127
|
+
Present the drafted notes to the user and get an explicit go before pushing. The ledger is shared
|
|
128
|
+
with another agent, so an unrequested push is an external side effect.
|
|
129
|
+
|
|
130
|
+
## Step 6. Carry findings forward
|
|
131
|
+
|
|
132
|
+
The reason four correct reports changed nothing is that each one started clean. This skill does
|
|
133
|
+
not.
|
|
134
|
+
|
|
135
|
+
Write the output to a dated file and, on every run, read the previous one. Each finding carries:
|
|
136
|
+
|
|
137
|
+
- `first_seen`: the date it was first reported
|
|
138
|
+
- `runs_seen`: how many consecutive runs have reported it
|
|
139
|
+
- `status`: `new`, `carried`, `actioned`, or `closed-not-possible`
|
|
140
|
+
|
|
141
|
+
A finding at `runs_seen: 3` or more gets its own section at the top of the report titled with its
|
|
142
|
+
age, for example "Open for 3 runs". Reporting the same defect a fourth time without escalating it
|
|
143
|
+
is the failure this section prevents. `closed-not-possible` is a real terminal state and must
|
|
144
|
+
record why, so it is not rediscovered.
|
|
145
|
+
|
|
146
|
+
Use `context-repo` to resolve a durable private store for the snapshots if one is not already
|
|
147
|
+
configured. Do not write snapshots to a repo-local scratch directory.
|
|
148
|
+
|
|
149
|
+
## Step 7. Output
|
|
150
|
+
|
|
151
|
+
Lead with a verdict in one line naming what is wrong, not what was analysed. Then, in this order:
|
|
152
|
+
|
|
153
|
+
1. Open for N runs (carried findings, oldest first) - omit the section on a first run
|
|
154
|
+
2. Verified as working
|
|
155
|
+
3. Defects
|
|
156
|
+
4. What repeated, ranked by distinct sessions
|
|
157
|
+
5. Delivery leakage
|
|
158
|
+
6. Skills and knowledge, including the ledger notes drafted this run
|
|
159
|
+
7. The ranked change list, config edits before projects
|
|
160
|
+
8. Receipts
|
|
161
|
+
|
|
162
|
+
Hard rules for the output:
|
|
163
|
+
|
|
164
|
+
- Numbers over adjectives. "1120 organic turns, 46 percent under 40 characters" beats "a lot of
|
|
165
|
+
short prompts".
|
|
166
|
+
- No em dashes anywhere. Hyphens or restructure.
|
|
167
|
+
- Every PR, issue or ticket line carries a clickable URL.
|
|
168
|
+
- Anything not run is named as not run. A step that was blocked is stated as blocked, with the
|
|
169
|
+
reason and what remains untrue because of it.
|
|
170
|
+
- Receipts are facts with values: paths, counts, SHAs, exit codes, what was deliberately not run.
|
|
171
|
+
|
|
172
|
+
## Cost
|
|
173
|
+
|
|
174
|
+
Mining is grunt work. Run the extraction and the per-axis analysis as parallel leaf agents on
|
|
175
|
+
Sonnet or Haiku, and keep only the synthesis and the ledger decision on the session model. Pass the
|
|
176
|
+
agents the mined JSON paths rather than the transcripts, and require each to return its full report
|
|
177
|
+
in its final message: a subagent's transcript is not visible to the caller, and a report that says
|
|
178
|
+
"see above" arrives empty.
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
<!-- Generated by scripts/build-adapters.sh. Do not edit directly. -->
|
|
2
|
+
|
|
3
|
+
---
|
|
4
|
+
inclusion: manual
|
|
5
|
+
description: "Retrospective on how the agents themselves are being operated, from local transcripts plus the reports other jobs already produce: what the human keeps repeating, what the guardrails caught, and where delivery leaked, with unactioned findings carried forward."
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
# Agent Ops Retro
|
|
9
|
+
|
|
10
|
+
A retrospective on the operating layer, not the code. It asks whether the agents are being run
|
|
11
|
+
well: what the human had to repeat, what the guardrails caught, what they cost, and which findings
|
|
12
|
+
have been reported before and never actioned.
|
|
13
|
+
|
|
14
|
+
## Scope boundary
|
|
15
|
+
|
|
16
|
+
Three retro skills exist and they do not overlap.
|
|
17
|
+
|
|
18
|
+
| Skill | Looks at | Answers |
|
|
19
|
+
|---|---|---|
|
|
20
|
+
| `retro-analysis` | Repository history, delivery evidence, code quality | What shipped and how the code is trending |
|
|
21
|
+
| `wrap-up` | One finished change | Did this specific piece of work meet its goal |
|
|
22
|
+
| `agent-ops-retro` | Transcripts, hooks, skills, ledger, delegation logs | How the agents are being operated |
|
|
23
|
+
|
|
24
|
+
If the user asks about shipped work, hand off to `retro-analysis` and say so.
|
|
25
|
+
|
|
26
|
+
## Invocation
|
|
27
|
+
|
|
28
|
+
Accept a window: `14d` (default), `7d`, `30d`, or an explicit ISO date range. State the absolute
|
|
29
|
+
range at the top of the output. Convert relative dates to absolute.
|
|
30
|
+
|
|
31
|
+
## Step 0. Read, do not recompute
|
|
32
|
+
|
|
33
|
+
Several jobs already report on this state. Adding a fifth independent opinion is the failure mode
|
|
34
|
+
this skill exists to fix, so read their outputs first and only compute what none of them cover.
|
|
35
|
+
|
|
36
|
+
| Source | Path | Gives you |
|
|
37
|
+
|---|---|---|
|
|
38
|
+
| Skill audit | `~/.claude/skill-audit/latest.json` and the dated `.txt` beside it | Installed vs never-invoked skills |
|
|
39
|
+
| Delegation log | `~/.claude/delegation-metrics/delegations.jsonl` | Model, tier, depth, outcome, rework per spawn |
|
|
40
|
+
| Tier check | `~/.claude/delegation-metrics/tier-check-*.md` | Opus share and spend against a baseline |
|
|
41
|
+
| Usage miner | `~/obsidian/cloud-opus/Lead/Automation/Weekly Automation Report *.md` | Repeated commands, proposal fatigue, approval queue |
|
|
42
|
+
| Miner state | `~/.claude/usage-miner/state/last-run.json` | What was staged, queued, blocked |
|
|
43
|
+
| Knowledge ledger | `~/.local/share/agent-context/repo/ledger.json` | Traps already recorded, so you do not re-derive them |
|
|
44
|
+
| Prior run | the previous output of this skill (see Step 6) | Findings to carry forward |
|
|
45
|
+
|
|
46
|
+
Two traps when reading these:
|
|
47
|
+
|
|
48
|
+
- A skill-usage figure counted from attributed messages is not an invocation count. The audit's own
|
|
49
|
+
numbers run three orders of magnitude above the Skill-tool invocation count. Say which unit you
|
|
50
|
+
are quoting, every time.
|
|
51
|
+
- When two sources disagree on the same number, report both with their sources and say the figure
|
|
52
|
+
is unreconciled. Never silently pick one.
|
|
53
|
+
|
|
54
|
+
## Step 1. Mine the transcripts
|
|
55
|
+
|
|
56
|
+
This is the axis nothing else covers. Run `scripts/mine-transcripts.mjs` from this skill directory,
|
|
57
|
+
or reimplement it, over `~/.claude/projects/**/*.jsonl`.
|
|
58
|
+
|
|
59
|
+
Non-negotiable filters, each of which has burned a previous run:
|
|
60
|
+
|
|
61
|
+
- Filter by the record's own `.timestamp`, never by file mtime. A resumed session rewrites old
|
|
62
|
+
files and pulls pre-window turns into the window.
|
|
63
|
+
- A `user` record is not a human turn. Exclude `tool_result` blocks, `<system-reminder>` wrappers,
|
|
64
|
+
`<command-name>` and hook preambles, task notifications and bash echoes. In one measured corpus
|
|
65
|
+
45 percent of `user` text records were tool echoes, so the raw count nearly doubles the real one.
|
|
66
|
+
- Exclude `isSidechain` records and anything under a `subagents/` directory from session and
|
|
67
|
+
human-turn counts. Count them separately for the delegation view.
|
|
68
|
+
- Empty output is a state, not a success. If a scan returns nothing, prove the scan works before
|
|
69
|
+
concluding the thing is absent.
|
|
70
|
+
|
|
71
|
+
Extract: organic human turns per day and per session, turn-length distribution, tool call counts,
|
|
72
|
+
`tool_result` errors with `is_error`, Agent spawns with `subagent_type` and `model`, Skill
|
|
73
|
+
invocations, interruptions, compaction events, per-model token usage including cache reads, and
|
|
74
|
+
synthetic records naming a rate or spend limit.
|
|
75
|
+
|
|
76
|
+
## Step 2. Classify what the human repeated
|
|
77
|
+
|
|
78
|
+
Group the organic turns. The strongest automation signal is not raw frequency, it is the number of
|
|
79
|
+
distinct sessions a correction appears in: a thing said twenty times in one session is one
|
|
80
|
+
argument, a thing said once in twenty sessions is a missing default.
|
|
81
|
+
|
|
82
|
+
Report each recurring pattern as: name, count, distinct sessions, two or three verbatim dated
|
|
83
|
+
quotes, and the one concrete change that removes it. Name the mechanism, not the mood. Write
|
|
84
|
+
"stops for a one-word ack on reversible steps", not "autonomy could be better".
|
|
85
|
+
|
|
86
|
+
## Step 3. Split what holds from what is broken
|
|
87
|
+
|
|
88
|
+
Produce two explicit lists, never a narrative.
|
|
89
|
+
|
|
90
|
+
**Verified as working:** the claim and what proved it. Guardrails that fired correctly count here.
|
|
91
|
+
|
|
92
|
+
**Defects:** the mechanism and its consequence. Include the governance frictions, which usually
|
|
93
|
+
outnumber the external failures: permission denials, classifier blocks, path-gate refusals, agent
|
|
94
|
+
output failing its own schema. Give each a count and a share of total errors.
|
|
95
|
+
|
|
96
|
+
## Step 4. Delivery leakage
|
|
97
|
+
|
|
98
|
+
Delivery here means whether agent output landed, not whether it was produced.
|
|
99
|
+
|
|
100
|
+
- Open PRs authored by the user, bucketed by age, with fan-out batches identified as batches.
|
|
101
|
+
- Tickets whose completion timestamps cluster in minutes, which is a status backfill rather than
|
|
102
|
+
shipping. State both the raw completion count and the count after removing the cluster.
|
|
103
|
+
- Any fan-out with no named landing mechanism. That is backlog, not delivery.
|
|
104
|
+
- Scheduled jobs: check `launchctl list` exit codes and each job's own log. A wrapper exit code is
|
|
105
|
+
not completion.
|
|
106
|
+
|
|
107
|
+
## Step 5. Close the knowledge circle
|
|
108
|
+
|
|
109
|
+
This is the half that makes the retro compound instead of repeating.
|
|
110
|
+
|
|
111
|
+
1. Before writing a finding, check whether the ledger already holds it:
|
|
112
|
+
`~/.local/share/agent-context/repo/recipes/tools/ledger-index --kind trap --scope <scope>`.
|
|
113
|
+
A finding the ledger already records is a compliance gap, not a discovery. Say which it is.
|
|
114
|
+
2. For each genuinely new, durable, cross-cutting trap, draft a note: `kind`, `scope`, `title`,
|
|
115
|
+
`body`, and a `why` naming the concrete failure. A finding that is only true this week, or only
|
|
116
|
+
true in one repository, does not go in the ledger. It goes in the report.
|
|
117
|
+
3. Appending is a shared write. Take the lease first
|
|
118
|
+
(`recipes/tools/task-claim acquire ledger-append <agent> 20`), append without touching any
|
|
119
|
+
existing note, run `node recipes/tools/validate-ledger.js`, assert the note count rose by
|
|
120
|
+
exactly the number added and that no prior id vanished, commit, push, then verify
|
|
121
|
+
`git rev-parse HEAD` against `git ls-remote origin main` and quote both. Release the lease even
|
|
122
|
+
on failure.
|
|
123
|
+
4. Never append a credential, a personal path that is not the user's own machine, or a fact that is
|
|
124
|
+
only true this week.
|
|
125
|
+
|
|
126
|
+
Present the drafted notes to the user and get an explicit go before pushing. The ledger is shared
|
|
127
|
+
with another agent, so an unrequested push is an external side effect.
|
|
128
|
+
|
|
129
|
+
## Step 6. Carry findings forward
|
|
130
|
+
|
|
131
|
+
The reason four correct reports changed nothing is that each one started clean. This skill does
|
|
132
|
+
not.
|
|
133
|
+
|
|
134
|
+
Write the output to a dated file and, on every run, read the previous one. Each finding carries:
|
|
135
|
+
|
|
136
|
+
- `first_seen`: the date it was first reported
|
|
137
|
+
- `runs_seen`: how many consecutive runs have reported it
|
|
138
|
+
- `status`: `new`, `carried`, `actioned`, or `closed-not-possible`
|
|
139
|
+
|
|
140
|
+
A finding at `runs_seen: 3` or more gets its own section at the top of the report titled with its
|
|
141
|
+
age, for example "Open for 3 runs". Reporting the same defect a fourth time without escalating it
|
|
142
|
+
is the failure this section prevents. `closed-not-possible` is a real terminal state and must
|
|
143
|
+
record why, so it is not rediscovered.
|
|
144
|
+
|
|
145
|
+
Use `context-repo` to resolve a durable private store for the snapshots if one is not already
|
|
146
|
+
configured. Do not write snapshots to a repo-local scratch directory.
|
|
147
|
+
|
|
148
|
+
## Step 7. Output
|
|
149
|
+
|
|
150
|
+
Lead with a verdict in one line naming what is wrong, not what was analysed. Then, in this order:
|
|
151
|
+
|
|
152
|
+
1. Open for N runs (carried findings, oldest first) - omit the section on a first run
|
|
153
|
+
2. Verified as working
|
|
154
|
+
3. Defects
|
|
155
|
+
4. What repeated, ranked by distinct sessions
|
|
156
|
+
5. Delivery leakage
|
|
157
|
+
6. Skills and knowledge, including the ledger notes drafted this run
|
|
158
|
+
7. The ranked change list, config edits before projects
|
|
159
|
+
8. Receipts
|
|
160
|
+
|
|
161
|
+
Hard rules for the output:
|
|
162
|
+
|
|
163
|
+
- Numbers over adjectives. "1120 organic turns, 46 percent under 40 characters" beats "a lot of
|
|
164
|
+
short prompts".
|
|
165
|
+
- No em dashes anywhere. Hyphens or restructure.
|
|
166
|
+
- Every PR, issue or ticket line carries a clickable URL.
|
|
167
|
+
- Anything not run is named as not run. A step that was blocked is stated as blocked, with the
|
|
168
|
+
reason and what remains untrue because of it.
|
|
169
|
+
- Receipts are facts with values: paths, counts, SHAs, exit codes, what was deliberately not run.
|
|
170
|
+
|
|
171
|
+
## Cost
|
|
172
|
+
|
|
173
|
+
Mining is grunt work. Run the extraction and the per-axis analysis as parallel leaf agents on
|
|
174
|
+
Sonnet or Haiku, and keep only the synthesis and the ledger decision on the session model. Pass the
|
|
175
|
+
agents the mined JSON paths rather than the transcripts, and require each to return its full report
|
|
176
|
+
in its final message: a subagent's transcript is not visible to the caller, and a report that says
|
|
177
|
+
"see above" arrives empty.
|
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
<!-- Generated by scripts/build-adapters.sh. Do not edit directly. -->
|
|
2
|
+
|
|
3
|
+
---
|
|
4
|
+
description: "Retrospective on how the agents themselves are being operated, from local transcripts plus the reports other jobs already produce: what the human keeps repeating, what the guardrails caught, and where delivery leaked, with unactioned findings carried forward."
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
# Agent Ops Retro
|
|
8
|
+
|
|
9
|
+
A retrospective on the operating layer, not the code. It asks whether the agents are being run
|
|
10
|
+
well: what the human had to repeat, what the guardrails caught, what they cost, and which findings
|
|
11
|
+
have been reported before and never actioned.
|
|
12
|
+
|
|
13
|
+
## Scope boundary
|
|
14
|
+
|
|
15
|
+
Three retro skills exist and they do not overlap.
|
|
16
|
+
|
|
17
|
+
| Skill | Looks at | Answers |
|
|
18
|
+
|---|---|---|
|
|
19
|
+
| `retro-analysis` | Repository history, delivery evidence, code quality | What shipped and how the code is trending |
|
|
20
|
+
| `wrap-up` | One finished change | Did this specific piece of work meet its goal |
|
|
21
|
+
| `agent-ops-retro` | Transcripts, hooks, skills, ledger, delegation logs | How the agents are being operated |
|
|
22
|
+
|
|
23
|
+
If the user asks about shipped work, hand off to `retro-analysis` and say so.
|
|
24
|
+
|
|
25
|
+
## Invocation
|
|
26
|
+
|
|
27
|
+
Accept a window: `14d` (default), `7d`, `30d`, or an explicit ISO date range. State the absolute
|
|
28
|
+
range at the top of the output. Convert relative dates to absolute.
|
|
29
|
+
|
|
30
|
+
## Step 0. Read, do not recompute
|
|
31
|
+
|
|
32
|
+
Several jobs already report on this state. Adding a fifth independent opinion is the failure mode
|
|
33
|
+
this skill exists to fix, so read their outputs first and only compute what none of them cover.
|
|
34
|
+
|
|
35
|
+
| Source | Path | Gives you |
|
|
36
|
+
|---|---|---|
|
|
37
|
+
| Skill audit | `~/.claude/skill-audit/latest.json` and the dated `.txt` beside it | Installed vs never-invoked skills |
|
|
38
|
+
| Delegation log | `~/.claude/delegation-metrics/delegations.jsonl` | Model, tier, depth, outcome, rework per spawn |
|
|
39
|
+
| Tier check | `~/.claude/delegation-metrics/tier-check-*.md` | Opus share and spend against a baseline |
|
|
40
|
+
| Usage miner | `~/obsidian/cloud-opus/Lead/Automation/Weekly Automation Report *.md` | Repeated commands, proposal fatigue, approval queue |
|
|
41
|
+
| Miner state | `~/.claude/usage-miner/state/last-run.json` | What was staged, queued, blocked |
|
|
42
|
+
| Knowledge ledger | `~/.local/share/agent-context/repo/ledger.json` | Traps already recorded, so you do not re-derive them |
|
|
43
|
+
| Prior run | the previous output of this skill (see Step 6) | Findings to carry forward |
|
|
44
|
+
|
|
45
|
+
Two traps when reading these:
|
|
46
|
+
|
|
47
|
+
- A skill-usage figure counted from attributed messages is not an invocation count. The audit's own
|
|
48
|
+
numbers run three orders of magnitude above the Skill-tool invocation count. Say which unit you
|
|
49
|
+
are quoting, every time.
|
|
50
|
+
- When two sources disagree on the same number, report both with their sources and say the figure
|
|
51
|
+
is unreconciled. Never silently pick one.
|
|
52
|
+
|
|
53
|
+
## Step 1. Mine the transcripts
|
|
54
|
+
|
|
55
|
+
This is the axis nothing else covers. Run `scripts/mine-transcripts.mjs` from this skill directory,
|
|
56
|
+
or reimplement it, over `~/.claude/projects/**/*.jsonl`.
|
|
57
|
+
|
|
58
|
+
Non-negotiable filters, each of which has burned a previous run:
|
|
59
|
+
|
|
60
|
+
- Filter by the record's own `.timestamp`, never by file mtime. A resumed session rewrites old
|
|
61
|
+
files and pulls pre-window turns into the window.
|
|
62
|
+
- A `user` record is not a human turn. Exclude `tool_result` blocks, `<system-reminder>` wrappers,
|
|
63
|
+
`<command-name>` and hook preambles, task notifications and bash echoes. In one measured corpus
|
|
64
|
+
45 percent of `user` text records were tool echoes, so the raw count nearly doubles the real one.
|
|
65
|
+
- Exclude `isSidechain` records and anything under a `subagents/` directory from session and
|
|
66
|
+
human-turn counts. Count them separately for the delegation view.
|
|
67
|
+
- Empty output is a state, not a success. If a scan returns nothing, prove the scan works before
|
|
68
|
+
concluding the thing is absent.
|
|
69
|
+
|
|
70
|
+
Extract: organic human turns per day and per session, turn-length distribution, tool call counts,
|
|
71
|
+
`tool_result` errors with `is_error`, Agent spawns with `subagent_type` and `model`, Skill
|
|
72
|
+
invocations, interruptions, compaction events, per-model token usage including cache reads, and
|
|
73
|
+
synthetic records naming a rate or spend limit.
|
|
74
|
+
|
|
75
|
+
## Step 2. Classify what the human repeated
|
|
76
|
+
|
|
77
|
+
Group the organic turns. The strongest automation signal is not raw frequency, it is the number of
|
|
78
|
+
distinct sessions a correction appears in: a thing said twenty times in one session is one
|
|
79
|
+
argument, a thing said once in twenty sessions is a missing default.
|
|
80
|
+
|
|
81
|
+
Report each recurring pattern as: name, count, distinct sessions, two or three verbatim dated
|
|
82
|
+
quotes, and the one concrete change that removes it. Name the mechanism, not the mood. Write
|
|
83
|
+
"stops for a one-word ack on reversible steps", not "autonomy could be better".
|
|
84
|
+
|
|
85
|
+
## Step 3. Split what holds from what is broken
|
|
86
|
+
|
|
87
|
+
Produce two explicit lists, never a narrative.
|
|
88
|
+
|
|
89
|
+
**Verified as working:** the claim and what proved it. Guardrails that fired correctly count here.
|
|
90
|
+
|
|
91
|
+
**Defects:** the mechanism and its consequence. Include the governance frictions, which usually
|
|
92
|
+
outnumber the external failures: permission denials, classifier blocks, path-gate refusals, agent
|
|
93
|
+
output failing its own schema. Give each a count and a share of total errors.
|
|
94
|
+
|
|
95
|
+
## Step 4. Delivery leakage
|
|
96
|
+
|
|
97
|
+
Delivery here means whether agent output landed, not whether it was produced.
|
|
98
|
+
|
|
99
|
+
- Open PRs authored by the user, bucketed by age, with fan-out batches identified as batches.
|
|
100
|
+
- Tickets whose completion timestamps cluster in minutes, which is a status backfill rather than
|
|
101
|
+
shipping. State both the raw completion count and the count after removing the cluster.
|
|
102
|
+
- Any fan-out with no named landing mechanism. That is backlog, not delivery.
|
|
103
|
+
- Scheduled jobs: check `launchctl list` exit codes and each job's own log. A wrapper exit code is
|
|
104
|
+
not completion.
|
|
105
|
+
|
|
106
|
+
## Step 5. Close the knowledge circle
|
|
107
|
+
|
|
108
|
+
This is the half that makes the retro compound instead of repeating.
|
|
109
|
+
|
|
110
|
+
1. Before writing a finding, check whether the ledger already holds it:
|
|
111
|
+
`~/.local/share/agent-context/repo/recipes/tools/ledger-index --kind trap --scope <scope>`.
|
|
112
|
+
A finding the ledger already records is a compliance gap, not a discovery. Say which it is.
|
|
113
|
+
2. For each genuinely new, durable, cross-cutting trap, draft a note: `kind`, `scope`, `title`,
|
|
114
|
+
`body`, and a `why` naming the concrete failure. A finding that is only true this week, or only
|
|
115
|
+
true in one repository, does not go in the ledger. It goes in the report.
|
|
116
|
+
3. Appending is a shared write. Take the lease first
|
|
117
|
+
(`recipes/tools/task-claim acquire ledger-append <agent> 20`), append without touching any
|
|
118
|
+
existing note, run `node recipes/tools/validate-ledger.js`, assert the note count rose by
|
|
119
|
+
exactly the number added and that no prior id vanished, commit, push, then verify
|
|
120
|
+
`git rev-parse HEAD` against `git ls-remote origin main` and quote both. Release the lease even
|
|
121
|
+
on failure.
|
|
122
|
+
4. Never append a credential, a personal path that is not the user's own machine, or a fact that is
|
|
123
|
+
only true this week.
|
|
124
|
+
|
|
125
|
+
Present the drafted notes to the user and get an explicit go before pushing. The ledger is shared
|
|
126
|
+
with another agent, so an unrequested push is an external side effect.
|
|
127
|
+
|
|
128
|
+
## Step 6. Carry findings forward
|
|
129
|
+
|
|
130
|
+
The reason four correct reports changed nothing is that each one started clean. This skill does
|
|
131
|
+
not.
|
|
132
|
+
|
|
133
|
+
Write the output to a dated file and, on every run, read the previous one. Each finding carries:
|
|
134
|
+
|
|
135
|
+
- `first_seen`: the date it was first reported
|
|
136
|
+
- `runs_seen`: how many consecutive runs have reported it
|
|
137
|
+
- `status`: `new`, `carried`, `actioned`, or `closed-not-possible`
|
|
138
|
+
|
|
139
|
+
A finding at `runs_seen: 3` or more gets its own section at the top of the report titled with its
|
|
140
|
+
age, for example "Open for 3 runs". Reporting the same defect a fourth time without escalating it
|
|
141
|
+
is the failure this section prevents. `closed-not-possible` is a real terminal state and must
|
|
142
|
+
record why, so it is not rediscovered.
|
|
143
|
+
|
|
144
|
+
Use `context-repo` to resolve a durable private store for the snapshots if one is not already
|
|
145
|
+
configured. Do not write snapshots to a repo-local scratch directory.
|
|
146
|
+
|
|
147
|
+
## Step 7. Output
|
|
148
|
+
|
|
149
|
+
Lead with a verdict in one line naming what is wrong, not what was analysed. Then, in this order:
|
|
150
|
+
|
|
151
|
+
1. Open for N runs (carried findings, oldest first) - omit the section on a first run
|
|
152
|
+
2. Verified as working
|
|
153
|
+
3. Defects
|
|
154
|
+
4. What repeated, ranked by distinct sessions
|
|
155
|
+
5. Delivery leakage
|
|
156
|
+
6. Skills and knowledge, including the ledger notes drafted this run
|
|
157
|
+
7. The ranked change list, config edits before projects
|
|
158
|
+
8. Receipts
|
|
159
|
+
|
|
160
|
+
Hard rules for the output:
|
|
161
|
+
|
|
162
|
+
- Numbers over adjectives. "1120 organic turns, 46 percent under 40 characters" beats "a lot of
|
|
163
|
+
short prompts".
|
|
164
|
+
- No em dashes anywhere. Hyphens or restructure.
|
|
165
|
+
- Every PR, issue or ticket line carries a clickable URL.
|
|
166
|
+
- Anything not run is named as not run. A step that was blocked is stated as blocked, with the
|
|
167
|
+
reason and what remains untrue because of it.
|
|
168
|
+
- Receipts are facts with values: paths, counts, SHAs, exit codes, what was deliberately not run.
|
|
169
|
+
|
|
170
|
+
## Cost
|
|
171
|
+
|
|
172
|
+
Mining is grunt work. Run the extraction and the per-axis analysis as parallel leaf agents on
|
|
173
|
+
Sonnet or Haiku, and keep only the synthesis and the ledger decision on the session model. Pass the
|
|
174
|
+
agents the mined JSON paths rather than the transcripts, and require each to return its full report
|
|
175
|
+
in its final message: a subagent's transcript is not visible to the caller, and a report that says
|
|
176
|
+
"see above" arrives empty.
|
package/README.md
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
<p><strong>Agent-agnostic skill catalog for Codex, Claude, Cursor, Grok, Copilot, Windsurf, Kiro, and other skill-aware tools.</strong></p>
|
|
4
4
|
<p>
|
|
5
5
|
<img src="https://img.shields.io/badge/license-MIT-16a34a" alt="MIT license">
|
|
6
|
-
<img src="https://img.shields.io/badge/skills-
|
|
6
|
+
<img src="https://img.shields.io/badge/skills-52-2563eb" alt="52 skills">
|
|
7
7
|
<img src="https://img.shields.io/badge/platforms-Codex%20%7C%20Claude%20%7C%20Cursor%20%7C%20Grok%20%7C%20Copilot%20%7C%20Windsurf%20%7C%20Kiro-111827" alt="Codex Claude Cursor Grok Copilot Windsurf Kiro">
|
|
8
8
|
<img src="https://img.shields.io/badge/status-public%20catalog-16a34a" alt="Public catalog">
|
|
9
9
|
</p>
|
|
@@ -244,7 +244,7 @@ Author and maintain agent skills and the AI toolchain itself.
|
|
|
244
244
|
| [relay](plugins/olko-skill-meta/skills/relay/SKILL.md) | Uses `claude-relay` to run long or rate-limit-prone tasks autonomously across subscription accounts | A task will outlive one session or hit rate limits partway through |
|
|
245
245
|
| [shared-knowledge-artifact](plugins/olko-skill-meta/skills/shared-knowledge-artifact/SKILL.md) | Builds a shared, self-persisting knowledge ledger as a Claude Artifact, a private page that stores its own data, renders itself from it, and publishes new versions of itself so several agents read the same lessons and append to them | Giving multiple agents one place to learn from each other instead of repeating the same mistakes |
|
|
246
246
|
|
|
247
|
-
### olko-reflection (
|
|
247
|
+
### olko-reflection (6)
|
|
248
248
|
|
|
249
249
|
Look back and improve: self-critique, retrospectives, performance review, rapid learning.
|
|
250
250
|
|
|
@@ -253,6 +253,7 @@ Look back and improve: self-critique, retrospectives, performance review, rapid
|
|
|
253
253
|
| [self-critique](plugins/olko-reflection/skills/self-critique/SKILL.md) | Adversarially critiques your own last answer: spawns a critic agent that verifies claims against live sources, then loops until satisfied and reports where you were wrong | Checking a substantial answer before the user has to |
|
|
254
254
|
| [review-past-performance](plugins/olko-reflection/skills/review-past-performance/SKILL.md) | Pulls 24h of ICM memories, git history, and skill analytics; detects repeated mistakes and slow workflows; proposes 1-3 concrete fixes | Daily self-improvement loop or codifying a repeated workflow |
|
|
255
255
|
| [retro-analysis](plugins/olko-reflection/skills/retro-analysis/SKILL.md) | Produces repository, comparison, and cross-project retrospectives from delivery, code-quality, work-pattern, and trend evidence | Recurring engineering retrospective or “what did we ship?” analysis |
|
|
256
|
+
| [agent-ops-retro](plugins/olko-reflection/skills/agent-ops-retro/SKILL.md) | Retrospective on how the agents themselves are being operated: mines local transcripts and reads the reports other jobs already produce to surface what the human keeps repeating, what the guardrails caught, and where delivery leaked, carrying unactioned findings forward | Checking agent usage, delegation cost, or repeating corrections across sessions rather than shipped code |
|
|
256
257
|
| [crash-course](plugins/olko-reflection/skills/crash-course/SKILL.md) | Expert tutor for rapid, source-grounded learning of any topic: a timed 4-hour sprint plus cheat-sheet, learning-ladder, quiz-me, Feynman, and resource-curation modes | Ramping up on an unfamiliar codebase, project, or concept under time pressure |
|
|
257
258
|
| [wrap-up](plugins/olko-reflection/skills/wrap-up/SKILL.md) | Verifies a completed task against its original objective, confirms applicable checks, and safely tidies task-owned artifacts, worktrees, and local branches | Periodic or end-of-task delivery and cleanup review |
|
|
258
259
|
|