trace-to-skill 0.1.26 → 0.1.35
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +52 -28
- package/dist/src/agentsLint.js +697 -2
- package/dist/src/agentsLint.js.map +1 -1
- package/dist/src/cli.js +18 -0
- package/dist/src/cli.js.map +1 -1
- package/dist/src/doctor.js +2 -1
- package/dist/src/doctor.js.map +1 -1
- package/dist/src/index.d.ts +1 -0
- package/dist/src/index.js +1 -0
- package/dist/src/index.js.map +1 -1
- package/dist/src/init.js +3 -3
- package/dist/src/init.js.map +1 -1
- package/dist/src/redact.d.ts +20 -0
- package/dist/src/redact.js +166 -0
- package/dist/src/redact.js.map +1 -0
- package/docs/ADOPTION_GUIDE.md +5 -3
- package/docs/DISCOVERY.md +76 -0
- package/docs/FAILURE_TAXONOMY.md +4 -1
- package/docs/USE_CASES.md +114 -0
- package/llms.txt +88 -0
- package/package.json +33 -2
- package/schemas/redact-result.schema.json +65 -0
package/README.md
CHANGED
|
@@ -3,24 +3,26 @@
|
|
|
3
3
|
[](https://github.com/grnbtqdbyx-create/trace-to-skill/actions/workflows/ci.yml)
|
|
4
4
|
[](https://github.com/grnbtqdbyx-create/trace-to-skill/actions/workflows/codex-readiness.yml)
|
|
5
5
|
[](https://github.com/grnbtqdbyx-create/trace-to-skill/releases)
|
|
6
|
+
[](https://www.npmjs.com/package/trace-to-skill)
|
|
7
|
+
[](https://www.npmjs.com/package/trace-to-skill)
|
|
6
8
|
[](LICENSE)
|
|
7
9
|
[](package.json)
|
|
8
10
|
|
|
9
11
|
Turn failed AI coding-agent runs into reusable `AGENTS.md` rules, `SKILL.md` files, and eval evidence.
|
|
10
12
|
|
|
11
13
|
```bash
|
|
12
|
-
npx
|
|
13
|
-
npx
|
|
14
|
-
npx
|
|
15
|
-
npx
|
|
16
|
-
npx
|
|
17
|
-
npx
|
|
18
|
-
npx
|
|
19
|
-
npx
|
|
20
|
-
npx
|
|
21
|
-
npx
|
|
22
|
-
npx
|
|
23
|
-
npx
|
|
14
|
+
npx trace-to-skill doctor .
|
|
15
|
+
npx trace-to-skill lint-agents .
|
|
16
|
+
npx trace-to-skill analyze ./runs
|
|
17
|
+
npx trace-to-skill init --comment --sarif
|
|
18
|
+
npx trace-to-skill suggest ./runs --target agents-md
|
|
19
|
+
npx trace-to-skill eval ./runs --threshold 80
|
|
20
|
+
npx trace-to-skill benchmark
|
|
21
|
+
npx trace-to-skill scorecard .
|
|
22
|
+
npx trace-to-skill scorecard-comment . --dry-run
|
|
23
|
+
npx trace-to-skill guard-github-event "$GITHUB_EVENT_PATH"
|
|
24
|
+
npx trace-to-skill comment ./runs --dry-run
|
|
25
|
+
npx trace-to-skill compare --before ./runs/before --after ./runs/after
|
|
24
26
|
```
|
|
25
27
|
|
|
26
28
|
AI coding agents are getting good enough to change real repositories, but they still repeat the same workflow mistakes: claiming success without tests, ignoring repo instructions, over-editing, inventing files, leaking secrets into traces, or enabling risky MCP tools.
|
|
@@ -33,6 +35,17 @@ failed agent run -> failure class -> reusable rule/skill -> eval gate -> keep or
|
|
|
33
35
|
|
|
34
36
|
It is built for maintainers using Codex, Claude Code, Cursor, Copilot coding agent, Gemini CLI, OpenCode, or MCP-enabled workflows.
|
|
35
37
|
|
|
38
|
+
## Fast Use Cases
|
|
39
|
+
|
|
40
|
+
Use it when you need to:
|
|
41
|
+
|
|
42
|
+
- **Gate Codex-ready PRs:** run `trace-to-skill scorecard .` in CI and post a reviewer-friendly readiness comment.
|
|
43
|
+
- **Harden agent instructions:** run `trace-to-skill lint-agents .` to catch missing `AGENTS.md`, conflicting tool instructions, missing includes, nested instruction drift, encoding issues, and risky MCP config.
|
|
44
|
+
- **Protect agent context:** run `trace-to-skill guard-github-event "$GITHUB_EVENT_PATH"` before feeding issue, PR, comment, discussion, check-run, or commit text into an agent.
|
|
45
|
+
- **Share failed traces safely:** run `trace-to-skill redact ./runs --output redacted-runs` before publishing anonymized failure fixtures.
|
|
46
|
+
|
|
47
|
+
For copy-paste workflows, see [docs/USE_CASES.md](https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/USE_CASES.md). For crawler-friendly metadata, see [docs/DISCOVERY.md](https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/DISCOVERY.md) and [llms.txt](https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/llms.txt).
|
|
48
|
+
|
|
36
49
|
## Why This Exists
|
|
37
50
|
|
|
38
51
|
Open-source maintainers do not need more AI-generated noise. They need agents that learn from concrete failures and produce reviewable evidence.
|
|
@@ -112,22 +125,22 @@ Trace analysis detects run-level failures:
|
|
|
112
125
|
|
|
113
126
|
## Installation
|
|
114
127
|
|
|
115
|
-
|
|
128
|
+
Run from npm:
|
|
116
129
|
|
|
117
130
|
```bash
|
|
118
|
-
npx
|
|
131
|
+
npx trace-to-skill analyze ./runs
|
|
119
132
|
```
|
|
120
133
|
|
|
121
|
-
|
|
134
|
+
Or install in a repository:
|
|
122
135
|
|
|
123
136
|
```bash
|
|
124
137
|
npm install -D trace-to-skill
|
|
125
138
|
```
|
|
126
139
|
|
|
127
|
-
|
|
140
|
+
GitHub source installs also work:
|
|
128
141
|
|
|
129
142
|
```bash
|
|
130
|
-
npx trace-to-skill analyze ./runs
|
|
143
|
+
npx github:grnbtqdbyx-create/trace-to-skill analyze ./runs
|
|
131
144
|
```
|
|
132
145
|
|
|
133
146
|
Requires Node.js 20+.
|
|
@@ -150,7 +163,17 @@ trace-to-skill lint-agents .
|
|
|
150
163
|
trace-to-skill lint-agents . --format json
|
|
151
164
|
```
|
|
152
165
|
|
|
153
|
-
This focused linter checks whether `AGENTS.md` exists as the canonical instruction source, whether validation commands are discoverable, whether `AGENTS.md` / `CLAUDE.md` / Cursor / Copilot guidance conflicts, and whether MCP configs expose risky capabilities or
|
|
166
|
+
This focused linter checks whether `AGENTS.md` exists as the canonical instruction source, whether validation commands are discoverable, whether `AGENTS.md` / `CLAUDE.md` / Cursor / Copilot guidance conflicts, whether instruction files reference missing paths, missing `@file.md` includes, nested `AGENTS.md` files that the root instructions do not mention, invalid UTF-8, or grow large enough to risk ignored guidance, and whether JSON or `.codex/config.toml` MCP/Codex configs expose risky capabilities, secrets, unresolved commands, missing `cwd` values, placeholder env vars, wrong `mcpServers` casing, unresolved plugin placeholders, deprecated `codex_hooks`, missing `default_permissions` profiles, or synced `projects.* trusted_level` state.
|
|
167
|
+
|
|
168
|
+
Redact traces before sharing them:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
trace-to-skill redact ./runs --output redacted-runs
|
|
172
|
+
trace-to-skill redact ./runs/failed-run.md > failed-run.redacted.md
|
|
173
|
+
trace-to-skill redact ./runs --output redacted-runs --format json
|
|
174
|
+
```
|
|
175
|
+
|
|
176
|
+
This removes common API keys, GitHub/npm/Slack tokens, bearer tokens, email addresses, local home paths, and hidden Unicode controls while preserving enough context for maintainer review.
|
|
154
177
|
|
|
155
178
|
Scaffold a repo:
|
|
156
179
|
|
|
@@ -246,9 +269,9 @@ trace-to-skill compare --before ./runs/before --after ./runs/after
|
|
|
246
269
|
|
|
247
270
|
JSONL traces are normalized by extracting common fields such as `message`, `content`, `text`, `output`, and `error`. Codex-style JSONL traces with `response_item`, `function_call`, `function_call_output`, and `event_msg` payloads are normalized into readable evidence lines.
|
|
248
271
|
|
|
249
|
-
MCP configs with `mcpServers` are parsed for capability hints such as filesystem, shell, browser, network, database, container, and secret-bearing environment variables.
|
|
272
|
+
MCP configs with `mcpServers`, `.mcp.json`, or project-local `.codex/config.toml` are parsed for capability hints such as filesystem, shell, browser, network, database, container, and secret-bearing environment variables. `lint-agents` also checks static startup inputs such as `command`, `cwd`, env placeholders, unresolved `$VARS`, `${CLAUDE_PLUGIN_ROOT}`-style plugin placeholders, local stdio commands without explicit `cwd`, and the common JSON `mcp_servers` / `mcpServers` casing mismatch. Codex config hygiene checks catch deprecated `[features].codex_hooks`, missing `default_permissions` profile definitions, and machine-local `projects.* trusted_level` metadata in synced config files.
|
|
250
273
|
|
|
251
|
-
Instruction files such as `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `.cursor/rules`, and `.github/copilot-instructions.md` are checked for obvious contradictions in validation commands, test requirements,
|
|
274
|
+
Instruction files such as `AGENTS.md`, `CLAUDE.md`, `GEMINI.md`, `.cursor/rules`, and `.github/copilot-instructions.md` are checked for obvious contradictions in validation commands, test requirements, destructive-command approval rules, invalid UTF-8, missing include targets, and nested `AGENTS.md` files that may not be loaded automatically.
|
|
252
275
|
|
|
253
276
|
## JSON Schemas
|
|
254
277
|
|
|
@@ -257,6 +280,7 @@ Stable machine-readable contracts are published with the npm package and release
|
|
|
257
280
|
- [`schemas/analysis-result.schema.json`](schemas/analysis-result.schema.json) describes `trace-to-skill analyze --format json`.
|
|
258
281
|
- [`schemas/agents-lint-result.schema.json`](schemas/agents-lint-result.schema.json) describes `trace-to-skill lint-agents --format json`.
|
|
259
282
|
- [`schemas/doctor-result.schema.json`](schemas/doctor-result.schema.json) describes `trace-to-skill doctor --format json`.
|
|
283
|
+
- [`schemas/redact-result.schema.json`](schemas/redact-result.schema.json) describes `trace-to-skill redact --format json`.
|
|
260
284
|
- [`schemas/scorecard-result.schema.json`](schemas/scorecard-result.schema.json) describes `trace-to-skill scorecard --format json`.
|
|
261
285
|
|
|
262
286
|
These schemas let downstream Codex workflows, dashboards, and CI bots consume reports without scraping Markdown.
|
|
@@ -285,7 +309,7 @@ jobs:
|
|
|
285
309
|
issues: write
|
|
286
310
|
steps:
|
|
287
311
|
- uses: actions/checkout@v5
|
|
288
|
-
- uses: grnbtqdbyx-create/trace-to-skill@v0.1.
|
|
312
|
+
- uses: grnbtqdbyx-create/trace-to-skill@v0.1.35
|
|
289
313
|
with:
|
|
290
314
|
mode: all
|
|
291
315
|
doctor-threshold: "85"
|
|
@@ -316,15 +340,15 @@ jobs:
|
|
|
316
340
|
- uses: actions/setup-node@v5
|
|
317
341
|
with:
|
|
318
342
|
node-version: 20
|
|
319
|
-
- run: npx
|
|
320
|
-
- run: npx
|
|
321
|
-
- run: npx
|
|
343
|
+
- run: npx trace-to-skill analyze ./runs --output agent-learning-report.md
|
|
344
|
+
- run: npx trace-to-skill comment ./runs --token "${{ github.token }}"
|
|
345
|
+
- run: npx trace-to-skill eval ./runs --threshold 80
|
|
322
346
|
```
|
|
323
347
|
|
|
324
348
|
Code scanning / SARIF upload:
|
|
325
349
|
|
|
326
350
|
```yaml
|
|
327
|
-
- run: npx
|
|
351
|
+
- run: npx trace-to-skill analyze ./runs --format sarif --output trace-to-skill.sarif
|
|
328
352
|
- uses: github/codeql-action/upload-sarif@v4
|
|
329
353
|
with:
|
|
330
354
|
sarif_file: trace-to-skill.sarif
|
|
@@ -334,7 +358,7 @@ Composite action usage:
|
|
|
334
358
|
|
|
335
359
|
```yaml
|
|
336
360
|
- id: trace-to-skill
|
|
337
|
-
uses: grnbtqdbyx-create/trace-to-skill@v0.1.
|
|
361
|
+
uses: grnbtqdbyx-create/trace-to-skill@v0.1.35
|
|
338
362
|
with:
|
|
339
363
|
mode: all
|
|
340
364
|
doctor-threshold: "85"
|
|
@@ -376,7 +400,7 @@ Action outputs:
|
|
|
376
400
|
|
|
377
401
|
By default, generated reports are also appended to the GitHub Actions Job Summary. Set `job-summary: "false"` to disable that UI output.
|
|
378
402
|
|
|
379
|
-
Tagged Action releases build and run the CLI from `$GITHUB_ACTION_PATH`, so a workflow pinned to `@v0.1.
|
|
403
|
+
Tagged Action releases build and run the CLI from `$GITHUB_ACTION_PATH`, so a workflow pinned to a release tag such as `@v0.1.35` executes that release's checked-out source instead of pulling the default branch at runtime.
|
|
380
404
|
|
|
381
405
|
## Codex Skill
|
|
382
406
|
|
|
@@ -409,7 +433,7 @@ The goal is not to let agents autonomously rewrite project policy. The goal is t
|
|
|
409
433
|
- Codex session JSONL adapters
|
|
410
434
|
- Claude Code transcript adapters
|
|
411
435
|
- `AGENTS.md` contradiction detector
|
|
412
|
-
- MCP config parser with explicit capability scoring
|
|
436
|
+
- MCP/Codex config parser with explicit capability scoring, JSON/TOML startup diagnostics, and config drift checks
|
|
413
437
|
- GitHub PR comment mode
|
|
414
438
|
- before/after eval runner
|
|
415
439
|
- SARIF output for GitHub code scanning
|