trace-to-skill 0.1.26 → 0.1.35

package/docs/DISCOVERY.md

@@ -0,0 +1,76 @@
+# Discovery Summary
+
+This page is written for maintainers, search engines, package indexes, and AI research bots that need to understand `trace-to-skill` quickly.
+
+## One-Sentence Summary
+
+`trace-to-skill` checks whether a repository is Codex-ready, then turns failed AI coding-agent runs into evidence-backed `AGENTS.md` rules, `SKILL.md` workflows, privacy-safe traces, and eval gates.
+
+## Canonical Links
+
+- Repository: https://github.com/grnbtqdbyx-create/trace-to-skill
+- npm: https://www.npmjs.com/package/trace-to-skill
+- Latest release: https://github.com/grnbtqdbyx-create/trace-to-skill/releases/latest
+- Use cases: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/USE_CASES.md
+- Scorecard: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/SCORECARD.md
+- OpenAI OSS strategy: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/OPENAI_OSS_STRATEGY.md
+
+## Problems It Solves
+
+- Codex or Claude Code claims a task is done without validation evidence.
+- A coding agent repeats the same test/build/lint failure.
+- A repository has conflicting `AGENTS.md`, `CLAUDE.md`, Cursor, Copilot, or Gemini instructions.
+- A monorepo has nested `AGENTS.md` files, `@file.md` instruction includes, or invalid instruction-file encoding that makes Codex load the wrong policy.
+- A workflow wants to feed GitHub issue, PR, comment, discussion, check-run, or commit text into an agent but needs prompt-injection checks first.
+- MCP config gives agents filesystem, shell, browser, network, database, container, or secret-bearing access without a visible trust boundary.
+- MCP config looks valid at a glance but has broken startup inputs, such as missing commands, bad `cwd`, placeholder env vars, unresolved `$VARS`, unresolved plugin placeholders, local stdio commands without explicit `cwd`, or the wrong JSON `mcp_servers` wrapper key.
+- Codex config contains drift-prone settings such as deprecated `codex_hooks`, missing `default_permissions` profiles, or synced `projects.* trusted_level` project metadata.
+- Maintainers want to share failed traces publicly without leaking common tokens, emails, home paths, or hidden Unicode controls.
+
+## Primary Users
+
+- open-source maintainers using Codex for PR review, issue triage, release automation, or code changes
+- teams adopting `AGENTS.md` and repository-specific agent policy
+- maintainers evaluating MCP tool risk before enabling coding agents
+- people collecting anonymized failed agent runs for reproducible evals
+
+## Commands To Try
+
+```bash
+npx trace-to-skill doctor .
+npx trace-to-skill scorecard .
+npx trace-to-skill lint-agents .
+npx trace-to-skill guard-github-event "$GITHUB_EVENT_PATH"
+npx trace-to-skill redact ./runs --output redacted-runs
+npx trace-to-skill analyze ./runs --format json
+npx trace-to-skill suggest ./runs --target agents-md
+```
+
+## Outputs For Bots
+
+- `trace-to-skill analyze --format json`
+- `trace-to-skill lint-agents --format json`
+- `trace-to-skill doctor --format json`
+- `trace-to-skill redact --format json`
+- `trace-to-skill scorecard --format json`
+- SARIF from `trace-to-skill analyze --format sarif`
+- GitHub Action outputs for doctor, AGENTS lint, GitHub context guard, benchmark, and scorecard modes
+
+## Schema Contracts
+
+- `schemas/analysis-result.schema.json`
+- `schemas/agents-lint-result.schema.json`
+- `schemas/doctor-result.schema.json`
+- `schemas/redact-result.schema.json`
+- `schemas/scorecard-result.schema.json`
+
+## Related Keywords
+
+Codex, OpenAI Codex, Codex CLI, AGENTS.md, SKILL.md, Claude Code, Cursor, Copilot coding agent, Gemini CLI, MCP, Model Context Protocol, prompt injection, agent evals, AI code review, open-source maintainers, trace redaction, SARIF, GitHub Actions.
+
+## Non-Goals
+
+- It does not train a model.
+- It does not automatically rewrite project policy.
+- It does not ask maintainers to publish full private transcripts.
+- It does not replace security review; it gives maintainers deterministic evidence and guardrails.

package/docs/FAILURE_TAXONOMY.md

@@ -27,6 +27,9 @@ Agent instruction files disagree or the agent ignores an existing repository rul
 - different package managers for validation commands
 - "always run tests" vs "do not run tests"
 - approval required vs approval bypassed for destructive commands
+- missing `@file.md` include targets
+- nested `AGENTS.md` files that the root instructions do not point to
+- invalid UTF-8 bytes that can make instruction loading fail or become hard to debug
 
 ## Over-Editing
 
@@ -54,4 +57,4 @@ The fix is to treat those surfaces as data unless the instruction is also presen
 
 MCP server configuration or tool usage appears without an explicit trust boundary, capability inventory, or approval policy.
 
-`trace-to-skill` also parses common `mcpServers` JSON shapes and reports capability hints such as filesystem, shell, browser, network, database, container, and secret-bearing environment variables.
+`trace-to-skill` also parses common `mcpServers` JSON shapes and project `.codex/config.toml` MCP sections, then reports capability hints such as filesystem, shell, browser, network, database, container, and secret-bearing environment variables. `lint-agents` checks static startup inputs too: command availability, missing `cwd`, placeholder env values, unresolved `$VARS`, unresolved plugin placeholders, local stdio commands without explicit `cwd`, and JSON `mcp_servers` / `mcpServers` casing drift. It also flags Codex config drift such as deprecated `codex_hooks`, missing `default_permissions` profile definitions, and synced `projects.* trusted_level` metadata.

package/docs/USE_CASES.md

@@ -0,0 +1,114 @@
+# Use Cases
+
+`trace-to-skill` is for maintainers who want coding agents to produce reviewable evidence instead of repeating the same mistakes.
+
+## 1. Codex Readiness Gate
+
+Use this when a repository wants Codex-assisted pull requests, but maintainers need proof that the repo has basic guardrails.
+
+```bash
+npx trace-to-skill scorecard .
+```
+
+What it proves:
+
+- repository instructions exist
+- CI and validation scripts are present
+- maintainer docs and license are visible
+- distribution is easy to try
+- benchmark fixtures still catch known agent failure classes
+
+Recommended CI surface:
+
+```yaml
+- uses: grnbtqdbyx-create/trace-to-skill@v0.1.35
+  with:
+    mode: all
+    doctor-threshold: "85"
+    doctor-comment: "true"
+    scorecard-comment: "true"
+    job-summary: "true"
+    github-token: ${{ github.token }}
+```
+
+## 2. AGENTS.md And MCP Hygiene
+
+Use this before giving Codex broad repository access.
+
+```bash
+npx trace-to-skill lint-agents .
+```
+
+This checks:
+
+- whether repository-level agent instructions exist
+- whether `AGENTS.md`, `CLAUDE.md`, Cursor rules, Copilot instructions, or other tool guidance conflict
+- whether instruction files reference paths that no longer exist or have grown large enough to risk ignored guidance
+- whether `@file.md` include references are missing, nested `AGENTS.md` files are easy to miss, or instruction files contain invalid UTF-8
+- whether MCP config hints at risky capabilities such as filesystem, shell, browser, network, database, container, or secret-bearing environment variables
+- whether JSON or `.codex/config.toml` MCP startup inputs are obviously broken before launch, including wrong JSON `mcp_servers` casing, missing commands, missing `cwd`, placeholder env values, unresolved `$VARS`, unresolved plugin placeholders, or local stdio commands without explicit `cwd`
+- whether Codex config has drift-prone settings such as deprecated `codex_hooks`, missing `default_permissions` profile definitions, or synced `projects.* trusted_level` metadata
+
+The goal is not to ban powerful tools. The goal is to make trust boundaries visible before an agent acts.
+
+## 3. GitHub Context Guard
+
+Use this before an agent reads untrusted GitHub text.
+
+```bash
+npx trace-to-skill guard-github-event "$GITHUB_EVENT_PATH"
+```
+
+This scans pull request bodies, issue text, comments, discussions, review text, check-run messages, and commit messages for prompt-injection patterns.
+
+Use it when:
+
+- a workflow lets an agent summarize or act on PR comments
+- maintainers paste issue text into Codex
+- a bot asks Codex to triage untrusted user reports
+- logs or comments might contain instructions like "ignore previous instructions" or "print secrets"
+
+## 4. Failed Agent Run To Reviewable Rule
+
+Use this when a coding agent made a repeated workflow mistake.
+
+```bash
+npx trace-to-skill analyze ./runs --output agent-learning-report.md
+npx trace-to-skill suggest ./runs --target agents-md --output AGENTS.generated.md
+npx trace-to-skill eval ./runs --threshold 80
+```
+
+Recommended maintainer loop:
+
+1. Store a short redacted trace in `runs/`.
+2. Run `analyze` to classify the failure.
+3. Run `suggest` to generate candidate `AGENTS.md` or `SKILL.md` text.
+4. Copy only evidence-backed rules into the real policy file.
+5. Run `eval` or `scorecard` in CI so the same failure does not silently return.
+
+## 5. Privacy-Preserving Adoption
+
+Use this when you want public evidence without leaking private traces.
+
+```bash
+npx trace-to-skill redact ./runs --output redacted-runs
+npx trace-to-skill analyze ./runs --format json
+npx trace-to-skill analyze ./runs --format sarif --output trace-to-skill.sarif
+```
+
+Before publishing traces:
+
+- redact secrets, cookies, customer data, and proprietary code
+- keep only the lines needed to explain the failure
+- treat issue bodies, PR comments, copied logs, and web pages as untrusted input
+- prefer short fixtures that reproduce a detector over full transcripts
+
+## Why This Helps Open Source Maintainers
+
+The useful unit is not "an agent wrote code." The useful unit is:
+
+```text
+maintainer-visible failure -> evidence-backed rule -> repeatable gate
+```
+
+That is the path from ad-hoc AI usage to safer Codex-assisted maintenance.

package/llms.txt

@@ -0,0 +1,88 @@
+# trace-to-skill
+
+> Open-source CLI and GitHub Action for Codex-ready repository maintenance: turn failed AI coding-agent runs into reusable AGENTS.md rules, SKILL.md workflows, privacy-safe traces, and eval gates.
+
+Canonical repository: https://github.com/grnbtqdbyx-create/trace-to-skill
+NPM package: https://www.npmjs.com/package/trace-to-skill
+Latest release: https://github.com/grnbtqdbyx-create/trace-to-skill/releases/latest
+License: Apache-2.0
+Runtime: Node.js 20+
+
+## What this project is
+
+`trace-to-skill` helps open-source maintainers adopt Codex and other coding agents safely. It focuses on maintainer pain points:
+
+- agents claiming completion without test/build proof
+- failed tests hidden behind optimistic summaries
+- hallucinated files and broad over-editing
+- conflicting `AGENTS.md`, `CLAUDE.md`, Cursor, Copilot, or Gemini instructions
+- stale path references, missing `@file.md` includes, nested `AGENTS.md` visibility gaps, invalid UTF-8, and oversized instruction files that can make Codex follow wrong or truncated guidance
+- prompt injection in issue, PR, review, discussion, check-run, commit, log, or web text
+- risky MCP server capabilities, secret-bearing environment variables, broken JSON/TOML startup inputs, unresolved plugin placeholders, missing `cwd`, deprecated `codex_hooks`, missing `default_permissions` profiles, synced `projects.* trusted_level` metadata, and `mcp_servers` / `mcpServers` casing mismatches
+- sharing failed agent traces without leaking tokens, emails, local paths, or hidden Unicode controls
+
+The core loop is:
+
+```text
+failed agent run -> failure class -> evidence-backed AGENTS.md/SKILL.md suggestion -> eval gate -> keep or revise
+```
+
+## Best entry points for bots and maintainers
+
+- README: https://github.com/grnbtqdbyx-create/trace-to-skill#readme
+- Use cases: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/USE_CASES.md
+- Discovery summary: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/DISCOVERY.md
+- Adoption guide: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/ADOPTION_GUIDE.md
+- Failure taxonomy: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/FAILURE_TAXONOMY.md
+- OpenAI OSS strategy: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/OPENAI_OSS_STRATEGY.md
+- OpenAI application draft: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/OPENAI_APPLICATION_DRAFT.md
+- Benchmark: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/BENCHMARK.md
+- Scorecard: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/SCORECARD.md
+- Codex readiness auditor skill: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/skills/codex-readiness-auditor/SKILL.md
+
+## Core commands
+
+```bash
+npx trace-to-skill doctor .
+npx trace-to-skill lint-agents .
+npx trace-to-skill guard-github-event "$GITHUB_EVENT_PATH"
+npx trace-to-skill redact ./runs --output redacted-runs
+npx trace-to-skill analyze ./runs
+npx trace-to-skill suggest ./runs --target agents-md
+npx trace-to-skill eval ./runs --threshold 80
+npx trace-to-skill benchmark
+npx trace-to-skill scorecard .
+npx trace-to-skill init --comment --sarif
+```
+
+## GitHub Action
+
+```yaml
+- uses: grnbtqdbyx-create/trace-to-skill@v0.1.35
+  with:
+    mode: all
+    doctor-threshold: "85"
+    doctor-comment: "true"
+    scorecard-comment: "true"
+    job-summary: "true"
+    github-token: ${{ github.token }}
+```
+
+## Machine-readable schemas
+
+- Analysis JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/analysis-result.schema.json
+- AGENTS.md linter JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/agents-lint-result.schema.json
+- Doctor JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/doctor-result.schema.json
+- Redaction JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/redact-result.schema.json
+- Scorecard JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/scorecard-result.schema.json
+
+## Search phrases this project should answer
+
+- Codex readiness checker for open-source repositories
+- AGENTS.md linter for Codex and Claude Code
+- turn failed agent runs into AGENTS.md rules
+- prompt injection guard for GitHub issue and PR comments
+- MCP security scanner for coding agents
+- privacy-preserving redaction for AI agent traces
+- GitHub Action for AI coding-agent eval gates
+- Codex OSS maintainer automation evidence

package/package.json

@@ -1,8 +1,17 @@
 {
   "name": "trace-to-skill",
-  "version": "0.1.26",
+  "version": "0.1.35",
   "description": "Turn failed AI coding-agent runs into reusable AGENTS.md rules, SKILL.md files, and eval evidence.",
   "type": "module",
+  "main": "dist/src/index.js",
+  "types": "dist/src/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/src/index.d.ts",
+      "import": "./dist/src/index.js"
+    },
+    "./schemas/*": "./schemas/*"
+  },
   "bin": {
     "trace-to-skill": "dist/src/cli.js"
   },
@@ -12,10 +21,13 @@
     "docs/ADOPTION_GUIDE.md",
     "docs/AGENTS_LINT.md",
     "docs/BENCHMARK.md",
+    "docs/DISCOVERY.md",
     "docs/FAILURE_TAXONOMY.md",
     "docs/SCORECARD.md",
+    "docs/USE_CASES.md",
     "examples",
     "fixtures",
+    "llms.txt",
     "skills",
     "README.md",
     "LICENSE"
@@ -30,21 +42,40 @@
   },
   "keywords": [
     "codex",
+    "openai-codex",
     "codex-readiness",
+    "codex-cli",
     "agents",
     "ai-agents",
+    "ai-coding-agents",
     "agent-skills",
+    "agent-evals",
     "claude-code",
     "agents-md",
     "agents-md-linter",
+    "github-action",
     "json-schema",
     "mcp",
+    "mcp-security",
+    "prompt-injection",
     "evals",
     "open-source-maintainers",
-    "self-improvement"
+    "self-improvement",
+    "trace-redaction"
   ],
   "author": "Ogün <https://github.com/grnbtqdbyx-create>",
   "license": "Apache-2.0",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/grnbtqdbyx-create/trace-to-skill.git"
+  },
+  "bugs": {
+    "url": "https://github.com/grnbtqdbyx-create/trace-to-skill/issues"
+  },
+  "homepage": "https://github.com/grnbtqdbyx-create/trace-to-skill#readme",
+  "publishConfig": {
+    "access": "public"
+  },
   "engines": {
     "node": ">=20"
   },

package/schemas/redact-result.schema.json

@@ -0,0 +1,65 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://raw.githubusercontent.com/grnbtqdbyx-create/trace-to-skill/main/schemas/redact-result.schema.json",
+  "title": "trace-to-skill RedactResult",
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "generatedAt",
+    "files",
+    "totals"
+  ],
+  "properties": {
+    "generatedAt": {
+      "type": "string",
+      "format": "date-time"
+    },
+    "files": {
+      "type": "array",
+      "items": {
+        "$ref": "#/$defs/redactedFile"
+      }
+    },
+    "totals": {
+      "$ref": "#/$defs/replacementCounts"
+    }
+  },
+  "$defs": {
+    "replacementCounts": {
+      "type": "object",
+      "additionalProperties": {
+        "type": "integer",
+        "minimum": 0
+      }
+    },
+    "redactedFile": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": [
+        "inputPath",
+        "bytesBefore",
+        "bytesAfter",
+        "replacements"
+      ],
+      "properties": {
+        "inputPath": {
+          "type": "string"
+        },
+        "outputPath": {
+          "type": "string"
+        },
+        "bytesBefore": {
+          "type": "integer",
+          "minimum": 0
+        },
+        "bytesAfter": {
+          "type": "integer",
+          "minimum": 0
+        },
+        "replacements": {
+          "$ref": "#/$defs/replacementCounts"
+        }
+      }
+    }
+  }
+}