trace-to-skill 0.1.26 → 0.1.34

package/docs/USE_CASES.md

@@ -0,0 +1,113 @@
+# Use Cases
+
+`trace-to-skill` is for maintainers who want coding agents to produce reviewable evidence instead of repeating the same mistakes.
+
+## 1. Codex Readiness Gate
+
+Use this when a repository wants Codex-assisted pull requests, but maintainers need proof that the repo has basic guardrails.
+
+```bash
+npx trace-to-skill scorecard .
+```
+
+What it proves:
+
+- repository instructions exist
+- CI and validation scripts are present
+- maintainer docs and license are visible
+- distribution is easy to try
+- benchmark fixtures still catch known agent failure classes
+
+Recommended CI surface:
+
+```yaml
+- uses: grnbtqdbyx-create/trace-to-skill@v0.1.34
+  with:
+    mode: all
+    doctor-threshold: "85"
+    doctor-comment: "true"
+    scorecard-comment: "true"
+    job-summary: "true"
+    github-token: ${{ github.token }}
+```
+
+## 2. AGENTS.md And MCP Hygiene
+
+Use this before giving Codex broad repository access.
+
+```bash
+npx trace-to-skill lint-agents .
+```
+
+This checks:
+
+- whether repository-level agent instructions exist
+- whether `AGENTS.md`, `CLAUDE.md`, Cursor rules, Copilot instructions, or other tool guidance conflict
+- whether instruction files reference paths that no longer exist or have grown large enough to risk ignored guidance
+- whether MCP config hints at risky capabilities such as filesystem, shell, browser, network, database, container, or secret-bearing environment variables
+- whether JSON or `.codex/config.toml` MCP startup inputs are obviously broken before launch, including wrong JSON `mcp_servers` casing, missing commands, missing `cwd`, placeholder env values, unresolved `$VARS`, unresolved plugin placeholders, or local stdio commands without explicit `cwd`
+- whether Codex config has drift-prone settings such as deprecated `codex_hooks`, missing `default_permissions` profile definitions, or synced `projects.* trusted_level` metadata
+
+The goal is not to ban powerful tools. The goal is to make trust boundaries visible before an agent acts.
+
+## 3. GitHub Context Guard
+
+Use this before an agent reads untrusted GitHub text.
+
+```bash
+npx trace-to-skill guard-github-event "$GITHUB_EVENT_PATH"
+```
+
+This scans pull request bodies, issue text, comments, discussions, review text, check-run messages, and commit messages for prompt-injection patterns.
+
+Use it when:
+
+- a workflow lets an agent summarize or act on PR comments
+- maintainers paste issue text into Codex
+- a bot asks Codex to triage untrusted user reports
+- logs or comments might contain instructions like "ignore previous instructions" or "print secrets"
+
+## 4. Failed Agent Run To Reviewable Rule
+
+Use this when a coding agent made a repeated workflow mistake.
+
+```bash
+npx trace-to-skill analyze ./runs --output agent-learning-report.md
+npx trace-to-skill suggest ./runs --target agents-md --output AGENTS.generated.md
+npx trace-to-skill eval ./runs --threshold 80
+```
+
+Recommended maintainer loop:
+
+1. Store a short redacted trace in `runs/`.
+2. Run `analyze` to classify the failure.
+3. Run `suggest` to generate candidate `AGENTS.md` or `SKILL.md` text.
+4. Copy only evidence-backed rules into the real policy file.
+5. Run `eval` or `scorecard` in CI so the same failure does not silently return.
+
+## 5. Privacy-Preserving Adoption
+
+Use this when you want public evidence without leaking private traces.
+
+```bash
+npx trace-to-skill redact ./runs --output redacted-runs
+npx trace-to-skill analyze ./runs --format json
+npx trace-to-skill analyze ./runs --format sarif --output trace-to-skill.sarif
+```
+
+Before publishing traces:
+
+- redact secrets, cookies, customer data, and proprietary code
+- keep only the lines needed to explain the failure
+- treat issue bodies, PR comments, copied logs, and web pages as untrusted input
+- prefer short fixtures that reproduce a detector over full transcripts
+
+## Why This Helps Open Source Maintainers
+
+The useful unit is not "an agent wrote code." The useful unit is:
+
+```text
+maintainer-visible failure -> evidence-backed rule -> repeatable gate
+```
+
+That is the path from ad-hoc AI usage to safer Codex-assisted maintenance.

package/llms.txt

@@ -0,0 +1,88 @@
+# trace-to-skill
+
+> Open-source CLI and GitHub Action for Codex-ready repository maintenance: turn failed AI coding-agent runs into reusable AGENTS.md rules, SKILL.md workflows, privacy-safe traces, and eval gates.
+
+Canonical repository: https://github.com/grnbtqdbyx-create/trace-to-skill
+NPM package: https://www.npmjs.com/package/trace-to-skill
+Latest release: https://github.com/grnbtqdbyx-create/trace-to-skill/releases/latest
+License: Apache-2.0
+Runtime: Node.js 20+
+
+## What this project is
+
+`trace-to-skill` helps open-source maintainers adopt Codex and other coding agents safely. It focuses on maintainer pain points:
+
+- agents claiming completion without test/build proof
+- failed tests hidden behind optimistic summaries
+- hallucinated files and broad over-editing
+- conflicting `AGENTS.md`, `CLAUDE.md`, Cursor, Copilot, or Gemini instructions
+- stale path references and oversized instruction files that can make Codex follow wrong or truncated guidance
+- prompt injection in issue, PR, review, discussion, check-run, commit, log, or web text
+- risky MCP server capabilities, secret-bearing environment variables, broken JSON/TOML startup inputs, unresolved plugin placeholders, missing `cwd`, deprecated `codex_hooks`, missing `default_permissions` profiles, synced `projects.* trusted_level` metadata, and `mcp_servers` / `mcpServers` casing mismatches
+- sharing failed agent traces without leaking tokens, emails, local paths, or hidden Unicode controls
+
+The core loop is:
+
+```text
+failed agent run -> failure class -> evidence-backed AGENTS.md/SKILL.md suggestion -> eval gate -> keep or revise
+```
+
+## Best entry points for bots and maintainers
+
+- README: https://github.com/grnbtqdbyx-create/trace-to-skill#readme
+- Use cases: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/USE_CASES.md
+- Discovery summary: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/DISCOVERY.md
+- Adoption guide: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/ADOPTION_GUIDE.md
+- Failure taxonomy: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/FAILURE_TAXONOMY.md
+- OpenAI OSS strategy: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/OPENAI_OSS_STRATEGY.md
+- OpenAI application draft: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/OPENAI_APPLICATION_DRAFT.md
+- Benchmark: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/BENCHMARK.md
+- Scorecard: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/docs/SCORECARD.md
+- Codex readiness auditor skill: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/skills/codex-readiness-auditor/SKILL.md
+
+## Core commands
+
+```bash
+npx trace-to-skill doctor .
+npx trace-to-skill lint-agents .
+npx trace-to-skill guard-github-event "$GITHUB_EVENT_PATH"
+npx trace-to-skill redact ./runs --output redacted-runs
+npx trace-to-skill analyze ./runs
+npx trace-to-skill suggest ./runs --target agents-md
+npx trace-to-skill eval ./runs --threshold 80
+npx trace-to-skill benchmark
+npx trace-to-skill scorecard .
+npx trace-to-skill init --comment --sarif
+```
+
+## GitHub Action
+
+```yaml
+- uses: grnbtqdbyx-create/trace-to-skill@v0.1.34
+  with:
+    mode: all
+    doctor-threshold: "85"
+    doctor-comment: "true"
+    scorecard-comment: "true"
+    job-summary: "true"
+    github-token: ${{ github.token }}
+```
+
+## Machine-readable schemas
+
+- Analysis JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/analysis-result.schema.json
+- AGENTS.md linter JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/agents-lint-result.schema.json
+- Doctor JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/doctor-result.schema.json
+- Redaction JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/redact-result.schema.json
+- Scorecard JSON schema: https://github.com/grnbtqdbyx-create/trace-to-skill/blob/main/schemas/scorecard-result.schema.json
+
+## Search phrases this project should answer
+
+- Codex readiness checker for open-source repositories
+- AGENTS.md linter for Codex and Claude Code
+- turn failed agent runs into AGENTS.md rules
+- prompt injection guard for GitHub issue and PR comments
+- MCP security scanner for coding agents
+- privacy-preserving redaction for AI agent traces
+- GitHub Action for AI coding-agent eval gates
+- Codex OSS maintainer automation evidence

package/package.json

@@ -1,8 +1,17 @@
 {
   "name": "trace-to-skill",
-  "version": "0.1.26",
+  "version": "0.1.34",
   "description": "Turn failed AI coding-agent runs into reusable AGENTS.md rules, SKILL.md files, and eval evidence.",
   "type": "module",
+  "main": "dist/src/index.js",
+  "types": "dist/src/index.d.ts",
+  "exports": {
+    ".": {
+      "types": "./dist/src/index.d.ts",
+      "import": "./dist/src/index.js"
+    },
+    "./schemas/*": "./schemas/*"
+  },
   "bin": {
     "trace-to-skill": "dist/src/cli.js"
   },
@@ -12,10 +21,13 @@
     "docs/ADOPTION_GUIDE.md",
     "docs/AGENTS_LINT.md",
     "docs/BENCHMARK.md",
+    "docs/DISCOVERY.md",
     "docs/FAILURE_TAXONOMY.md",
     "docs/SCORECARD.md",
+    "docs/USE_CASES.md",
     "examples",
     "fixtures",
+    "llms.txt",
     "skills",
     "README.md",
     "LICENSE"
@@ -30,21 +42,40 @@
   },
   "keywords": [
     "codex",
+    "openai-codex",
     "codex-readiness",
+    "codex-cli",
     "agents",
     "ai-agents",
+    "ai-coding-agents",
     "agent-skills",
+    "agent-evals",
     "claude-code",
     "agents-md",
     "agents-md-linter",
+    "github-action",
     "json-schema",
     "mcp",
+    "mcp-security",
+    "prompt-injection",
     "evals",
     "open-source-maintainers",
-    "self-improvement"
+    "self-improvement",
+    "trace-redaction"
   ],
   "author": "Ogün <https://github.com/grnbtqdbyx-create>",
   "license": "Apache-2.0",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/grnbtqdbyx-create/trace-to-skill.git"
+  },
+  "bugs": {
+    "url": "https://github.com/grnbtqdbyx-create/trace-to-skill/issues"
+  },
+  "homepage": "https://github.com/grnbtqdbyx-create/trace-to-skill#readme",
+  "publishConfig": {
+    "access": "public"
+  },
   "engines": {
     "node": ">=20"
   },

package/schemas/redact-result.schema.json

@@ -0,0 +1,65 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "$id": "https://raw.githubusercontent.com/grnbtqdbyx-create/trace-to-skill/main/schemas/redact-result.schema.json",
+  "title": "trace-to-skill RedactResult",
+  "type": "object",
+  "additionalProperties": false,
+  "required": [
+    "generatedAt",
+    "files",
+    "totals"
+  ],
+  "properties": {
+    "generatedAt": {
+      "type": "string",
+      "format": "date-time"
+    },
+    "files": {
+      "type": "array",
+      "items": {
+        "$ref": "#/$defs/redactedFile"
+      }
+    },
+    "totals": {
+      "$ref": "#/$defs/replacementCounts"
+    }
+  },
+  "$defs": {
+    "replacementCounts": {
+      "type": "object",
+      "additionalProperties": {
+        "type": "integer",
+        "minimum": 0
+      }
+    },
+    "redactedFile": {
+      "type": "object",
+      "additionalProperties": false,
+      "required": [
+        "inputPath",
+        "bytesBefore",
+        "bytesAfter",
+        "replacements"
+      ],
+      "properties": {
+        "inputPath": {
+          "type": "string"
+        },
+        "outputPath": {
+          "type": "string"
+        },
+        "bytesBefore": {
+          "type": "integer",
+          "minimum": 0
+        },
+        "bytesAfter": {
+          "type": "integer",
+          "minimum": 0
+        },
+        "replacements": {
+          "$ref": "#/$defs/replacementCounts"
+        }
+      }
+    }
+  }
+}