agent-config-score 0.18.0__tar.gz

agent_config_score-0.18.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Zhibo Lin
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agent_config_score-0.18.0/PKG-INFO

@@ -0,0 +1,532 @@
+Metadata-Version: 2.4
+Name: agent-config-score
+Version: 0.18.0
+Summary: Regression gate for AI coding-agent configs: catch PRs that make AGENTS.md, CLAUDE.md, Cursor or Copilot instructions worse.
+Author: Zhibo Lin
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/LE0-Lin/AgentConfigScore
+Project-URL: Repository, https://github.com/LE0-Lin/AgentConfigScore
+Project-URL: Issues, https://github.com/LE0-Lin/AgentConfigScore/issues
+Project-URL: Changelog, https://github.com/LE0-Lin/AgentConfigScore/blob/main/CHANGELOG.md
+Keywords: agents-md,claude-code,cursor,copilot,ai-agents,github-actions,ci,developer-tools
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Operating System :: OS Independent
+Classifier: Topic :: Software Development :: Quality Assurance
+Classifier: Topic :: Software Development :: Testing
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# AgentConfigScore
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/LE0-Lin/AgentConfigScore/main/assets/agent-config-score.svg" alt="AgentConfigScore A 100" />
+</p>
+
+<p align="center"><strong>Codecov for AI coding-agent instructions.</strong></p>
+
+<p align="center">
+  Stop pull requests from quietly making <code>AGENTS.md</code>, <code>CLAUDE.md</code>, Cursor, Copilot and Gemini instructions worse.
+</p>
+
+<p align="center">
+  <a href="https://github.com/LE0-Lin/AgentConfigScore/actions/workflows/ci.yml"><img src="https://github.com/LE0-Lin/AgentConfigScore/actions/workflows/ci.yml/badge.svg" alt="CI" /></a>
+  <a href="https://pypi.org/project/agent-config-score/"><img src="https://img.shields.io/pypi/v/agent-config-score" alt="PyPI" /></a>
+  <img src="https://img.shields.io/badge/python-3.10%2B-blue" alt="Python 3.10+" />
+  <img src="https://img.shields.io/badge/platforms-Linux%20%7C%20Windows%20%7C%20macOS-informational" alt="Linux, Windows, macOS" />
+  <img src="https://img.shields.io/badge/runtime_dependencies-0-brightgreen" alt="0 runtime dependencies" />
+  <img src="https://img.shields.io/badge/license-MIT-green" alt="MIT" />
+</p>
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/LE0-Lin/AgentConfigScore/main/assets/agent-config-score-demo.gif" alt="AgentConfigScore detects an instruction regression and blocks the pull request" width="100%" />
+</p>
+
+AgentConfigScore is a deterministic regression gate for coding-agent configuration. It compares a change with its baseline and answers one CI-friendly question:
+
+> **Did this change make our agent instructions worse?**
+
+It is regression-first rather than perfection-first: an existing repository can start at 72/100 and adopt the gate immediately. A pull request that stays at 72 can pass; one that drops to 65 can fail.
+
+| What usually goes wrong | What the gate does |
+|---|---|
+| A pull request weakens its own quality threshold | Uses the baseline branch's policy to judge the change |
+| A new instruction contradicts or duplicates existing guidance | Reports deterministic, line-addressable findings |
+| An exception becomes a permanent silent ignore | Requires a reason and expiry, then preserves an audit trail |
+| A scanner update becomes noisy on real projects | Replays a pinned, manually reviewed [real-repository benchmark](https://github.com/LE0-Lin/AgentConfigScore/blob/v0/benchmarks/README.md) |
+
+## Get running
+
+```bash
+python -m pip install agent-config-score
+agent-config-score init
+```
+
+To test the latest stable pre-1.0 source directly from GitHub instead:
+
+```bash
+python -m pip install "git+https://github.com/LE0-Lin/AgentConfigScore.git@v0"
+```
+
+`init` safely creates:
+
+- `.agentconfigscore.json` — version-controlled policy, suppressions, and editor schema annotation
+- `.github/workflows/agent-config-score.yml` — pull-request regression gate
+
+Review and commit those files. Pull requests are then checked automatically.
+
+Validate the integration and run the same regression check locally before pushing:
+
+```bash
+agent-config-score doctor
+agent-config-score diff
+```
+
+`diff` auto-detects a safe local default-branch baseline when possible. You can still pass an explicit ref such as `origin/main` whenever you want full control.
+
+Initialization is conservative: every target is preflighted before anything is written, conflicting files are never overwritten by default, and rerunning against matching generated files is idempotent.
+
+```bash
+agent-config-score init --dry-run      # preview without writing
+agent-config-score init --no-workflow  # config only
+agent-config-score init --force        # intentionally replace conflicting generated files
+```
+
+`v0` is the rolling stable ref for the current pre-1.0 series.
+
+## Repository policy + editor validation
+
+A generated config looks like this:
+
+```json
+{
+  "$schema": "https://raw.githubusercontent.com/LE0-Lin/AgentConfigScore/v0/schema/agentconfigscore.schema.json",
+  "version": 1,
+  "policy": {
+    "max_drop": 0,
+    "fail_on_new_errors": true
+  }
+}
+```
+
+The `$schema` annotation enables live validation and completion in editors that support JSON Schema. It can catch misspelled policy keys, wrong value types, unknown suppression rule IDs, missing suppression fields, and malformed path scopes before CI runs.
+
+The schema is Draft 2020-12 and lives at `schema/agentconfigscore.schema.json`. AgentConfigScore's tests verify that its suppression rule enum stays exactly aligned with the stable Rule Catalog, so editor hints cannot silently drift away from scanner behavior.
+
+Policy fields:
+
+| Key | Default | Meaning |
+|---|---:|---|
+| `max_drop` | `0` | Maximum score decrease allowed by `diff` / `compare` |
+| `fail_on_new_errors` | `false` in the CLI; `true` in the Action when no policy file exists | Fail on newly introduced active error findings |
+| `fail_under` | unset | Optional absolute score floor for a normal scan |
+
+An absolute floor is optional:
+
+```json
+{
+  "$schema": "https://raw.githubusercontent.com/LE0-Lin/AgentConfigScore/v0/schema/agentconfigscore.schema.json",
+  "version": 1,
+  "policy": {
+    "max_drop": 0,
+    "fail_on_new_errors": true,
+    "fail_under": 90
+  }
+}
+```
+
+### Why the baseline policy governs a PR
+
+A pull request must not be able to weaken the gate that reviews it.
+
+For `diff`, `compare`, and the reusable GitHub Action, AgentConfigScore uses the **baseline** policy to decide pass/fail. The candidate config is still parsed and validated, but new thresholds only become active after that config is reviewed, merged, and becomes a later baseline.
+
+A candidate changing `max_drop` to `100` therefore cannot use that weaker value to pass itself. Explicit CLI or Action inputs remain available as trusted invocation-time overrides.
+
+## Auditable exceptions
+
+Sometimes a finding is understood and intentionally accepted. AgentConfigScore supports suppressions, but does not treat them as a permanent ignore list.
+
+Every suppression must name a stable rule, explain the exception, and expire:
+
+```json
+{
+  "$schema": "https://raw.githubusercontent.com/LE0-Lin/AgentConfigScore/v0/schema/agentconfigscore.schema.json",
+  "version": 1,
+  "policy": {
+    "max_drop": 0,
+    "fail_on_new_errors": true
+  },
+  "suppressions": [
+    {
+      "rule": "dead-path",
+      "reason": "Generated docs reference paths that only exist after deployment.",
+      "expires": "2026-12-31",
+      "paths": ["docs/**"]
+    }
+  ]
+}
+```
+
+Rules for suppressions:
+
+- `rule` must be a known stable AgentConfigScore rule ID
+- `reason` must be non-empty and is limited to 500 characters
+- `expires` must be `YYYY-MM-DD`
+- an expired suppression is a configuration error; remove it or explicitly renew it
+- `paths` is optional and accepts unique instruction-file glob patterns
+- path-scoped suppressions never hide repository-level findings such as cross-file contradiction
+- duplicate rule/path scopes are rejected
+
+Suppressions are **not silent**. A matching finding stops contributing to score and regression decisions, but remains visible in the `suppressed_findings` audit trail in JSON and is shown with its reason and expiry in terminal, HTML, and Markdown output.
+
+Suppressions are baseline-governed too: a PR cannot add a suppression and use it to excuse a finding introduced by that same PR. The baseline suppression set is applied to both sides of a regression comparison.
+
+## Check locally before you push
+
+The common path is now zero-config:
+
+```bash
+agent-config-score doctor
+agent-config-score diff
+```
+
+`doctor` validates the repository integration. `diff` compares the current working tree—including uncommitted changes—with a safe local default-branch baseline.
+
+When no `BASE_REF` is provided, AgentConfigScore detects a baseline conservatively in this order:
+
+1. locally configured `origin/HEAD`
+2. local `main`
+3. local `master`
+4. local `trunk`
+5. an upstream whose branch name itself is `main`, `master`, or `trunk`
+
+It deliberately does **not** use a feature-branch upstream such as `origin/my-feature` as the baseline, because doing so could hide committed changes relative to the real default branch.
+
+AgentConfigScore creates an isolated detached baseline worktree, compares it with your current repository, then removes the temporary worktree automatically. Automatic detection is fully offline and never runs `git fetch`.
+
+If no safe local baseline can be identified, pass one explicitly:
+
+```bash
+agent-config-score diff origin/main
+```
+
+If that ref is not available locally, fetch it yourself and retry. AgentConfigScore never fetches a missing ref behind your back.
+
+Trusted threshold overrides are explicit:
+
+```bash
+agent-config-score diff \
+  --max-drop 3 \
+  --no-fail-on-new-errors
+```
+
+You can run `diff` from any subdirectory inside the repository. Use `--path DIR` to point at another local repository.
+
+## What a failing PR looks like
+
+```text
+AgentConfigScore regression  A 96 → B 84 (-12)
+New findings: 2   Resolved: 0   Suppressed: 0
+
++ ERROR   curl-pipe-shell      Remote script piped directly to a shell
+          .github/copilot-instructions.md:12
++ WARNING dead-path            Referenced path does not exist: src/legacy_auth.py
+          AGENTS.md:31
+```
+
+**Live proof:** [PR #6](https://github.com/LE0-Lin/AgentConfigScore/pull/6) deliberately added an unsafe `curl | bash` instruction. AgentConfigScore changed the score from **A 100 → B 82 (-18)**, reported a new `curl-pipe-shell` error, failed the GitHub Actions job, and the PR was closed without merging.
+
+## Real-repository benchmark
+
+The v0.17.0 scanner was replayed against pinned commits from three public AI coding projects; source code was scanned but never executed.
+
+| Repository | Commit | Files found | Result | Manually reviewed signal |
+|---|---|---:|---:|---|
+| [`openai/codex`](https://github.com/openai/codex) | `d58d0e5` | 2 | B 82 | 3 references to absent `.rs` files; 1 context-size warning |
+| [`anomalyco/opencode`](https://github.com/anomalyco/opencode) | `9f69463` | 18 | A 94 | 1 context-size warning |
+| [`browser-use/browser-use`](https://github.com/browser-use/browser-use) | `d379a32` | 2 | B 88 | 1 context-size warning |
+
+All six findings matched their rule definitions in manual review. This small corpus is a reproducible smoke benchmark, not a quality leaderboard or a claim of broad statistical accuracy. The pinned inputs, reviewed expectations, limitations, and one-command runner live in [`benchmarks/`](https://github.com/LE0-Lin/AgentConfigScore/blob/v0/benchmarks/README.md).
+
+## Manual GitHub Actions setup
+
+If you do not want to use `init`, create `.github/workflows/agent-config-score.yml` yourself:
+
+```yaml
+name: agent-config-regression
+
+on:
+  pull_request:
+
+permissions:
+  contents: read
+
+jobs:
+  regression:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+        with:
+          fetch-depth: 0
+      - uses: LE0-Lin/AgentConfigScore@v0
+```
+
+The Action installs AgentConfigScore, resolves the PR base commit, applies baseline policy and suppressions, writes a Markdown report to the GitHub Actions Step Summary, exposes structured outputs, and returns the final regression status.
+
+Without a policy file, the Action preserves its conservative compatibility defaults: `max_drop = 0` and `fail_on_new_errors = true`.
+
+## Keep score history
+
+Optionally record a read-only score snapshot whenever your default branch changes:
+
+```yaml
+name: agent-config-score-history
+
+on:
+  push:
+    branches: [main]
+
+permissions:
+  contents: read
+
+jobs:
+  snapshot:
+    uses: LE0-Lin/AgentConfigScore/.github/workflows/score-history.yml@v0
+```
+
+Each successful run uploads an immutable artifact containing `snapshot.json`, `report.json`, `report.html`, and `badge.svg`. Artifact names include the commit SHA and run attempt, and retention defaults to 30 days. The history workflow does not write generated files back to the repository and does not fail merely because the current score is low.
+
+See [`docs/score-history.md`](https://github.com/LE0-Lin/AgentConfigScore/blob/v0/docs/score-history.md) for retention controls, monorepo paths, workflow outputs, and the history/security model. A copy-paste workflow is also available at [`examples/score-history.yml`](https://github.com/LE0-Lin/AgentConfigScore/blob/v0/examples/score-history.yml).
+
+## Nested AGENTS.md scopes and overrides
+
+AgentConfigScore recursively discovers both `AGENTS.md` and Codex `AGENTS.override.md` files and understands their directory scoping instead of treating every instruction file as one global policy.
+
+For a nested AGENTS-family file, file-like path references may be repository-root-relative, package-root-relative, or relative to the directory containing that file. The conservative `dead-path` analysis ignores ambiguous extensionless tokens, code fences, URLs, package imports, API symbols, absolute paths, Windows drive-qualified paths, and relative paths that escape the repository.
+
+Codex gives `AGENTS.override.md` priority over `AGENTS.md` in the same directory. Across directories, sibling scopes do not overlap and deeper instructions are more specific inside their subtree. AgentConfigScore therefore does not report an exact positive/negative pair across two different AGENTS-family files as an ambiguous contradiction. A contradiction inside one file still fails, and cross-system contradictions such as AGENTS-family instructions versus `CLAUDE.md` remain detectable.
+
+Codex can also be configured with custom `project_doc_fallback_filenames`. AgentConfigScore does not guess those user-specific filenames automatically; only the standard built-in AGENTS filenames are discovered by default.
+
+## Stable rule catalog
+
+Scoring logic is not hidden inside prose. Every rule has one canonical definition containing:
+
+- rule ID
+- default severity
+- scoring category
+- penalty before category caps
+- short summary
+- longer explanation
+
+Inspect all rules or one rule:
+
+```bash
+agent-config-score rules
+agent-config-score rules curl-pipe-shell
+agent-config-score rules --json
+agent-config-score rules dead-path --json
+```
+
+The scanner, scoring categories, CLI rule inspection, SARIF metadata, suppression validation, and config JSON Schema all derive from or are tested against the same stable catalog.
+
+Current rule families include context size, cross-file duplication, contradictions, dead paths, dangerous shell commands, common credential patterns, and missing canonical `AGENTS.md` coordination.
+
+## CLI reference
+
+Score the current repository:
+
+```bash
+agent-config-score .
+```
+
+Validate the repository integration:
+
+```bash
+agent-config-score doctor
+agent-config-score doctor --json
+```
+
+Run a local regression check with automatic baseline detection:
+
+```bash
+agent-config-score diff
+```
+
+Save a regression Markdown report:
+
+```bash
+agent-config-score diff --markdown regression.md
+```
+
+Use an explicit baseline when needed:
+
+```bash
+agent-config-score diff origin/main
+```
+
+Advanced: compare two already checked-out trees directly:
+
+```bash
+agent-config-score compare ../repo-base . --markdown regression.md
+```
+
+Generate local reports:
+
+```bash
+agent-config-score . \
+  --html .agent-config-score/report.html \
+  --badge .agent-config-score/badge.svg \
+  --sarif .agent-config-score/results.sarif
+```
+
+Override an absolute floor:
+
+```bash
+agent-config-score . --fail-under 90
+```
+
+Machine-readable output is available with `--json` for scans, `doctor`, `diff`, `compare`, and the rule catalog.
+
+```bash
+agent-config-score --version
+agent-config-score --help
+```
+
+## GitHub code scanning with SARIF
+
+`--sarif` writes SARIF 2.1.0 with rule IDs, severity, source location, descriptions, categories, and scoring penalties. Active findings can therefore be uploaded into GitHub code scanning. Suppressed findings remain in AgentConfigScore's explicit audit output rather than being emitted as active SARIF results.
+
+```yaml
+name: agent-config-code-scanning
+
+on:
+  push:
+
+permissions:
+  contents: read
+  security-events: write
+
+jobs:
+  scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v7
+        with:
+          python-version: "3.12"
+      - run: python -m pip install "git+https://github.com/LE0-Lin/AgentConfigScore.git@v0"
+      - run: agent-config-score . --sarif agent-config-score.sarif
+      - uses: github/codeql-action/upload-sarif@v4
+        with:
+          sarif_file: agent-config-score.sarif
+          category: agent-config-score
+```
+
+Repository-level findings remain repository-level SARIF results instead of receiving invented file locations.
+
+## What it scans
+
+Supported discovery includes:
+
+- `**/AGENTS.md` — root and nested directory scopes
+- `**/AGENTS.override.md` — Codex local overrides
+- `CLAUDE.md`
+- `GEMINI.md`
+- `.cursorrules`
+- `.cursor/rules/*.md` / `*.mdc`
+- `.github/copilot-instructions.md`
+- `.github/instructions/*.md`
+- `.claude/**/*.md`
+- `.clinerules`
+- `.windsurfrules`
+
+`.agentconfigscoreignore` can exclude discovery paths when needed.
+
+## Action inputs
+
+| Input | Default | Meaning |
+|---|---:|---|
+| `base-sha` | PR base SHA | Explicit baseline commit SHA |
+| `max-drop` | empty | Optional trusted override for baseline `policy.max_drop` |
+| `fail-on-new-errors` | empty | Optional trusted override for baseline `policy.fail_on_new_errors` |
+| `python-version` | `3.12` | Python used by the composite action |
+
+## Action outputs
+
+| Output | Meaning |
+|---|---|
+| `base-score` | Baseline AgentConfigScore |
+| `head-score` | Candidate AgentConfigScore |
+| `delta` | Candidate minus baseline score |
+| `new-findings` | Number of newly introduced active findings |
+| `new-errors` | Number of newly introduced active error findings |
+| `resolved-findings` | Number of active findings resolved by the candidate |
+
+```yaml
+- name: Check agent config
+  id: acs
+  uses: LE0-Lin/AgentConfigScore@v0
+
+- name: Print regression metrics
+  if: always()
+  run: |
+    echo "score: ${{ steps.acs.outputs.base-score }} -> ${{ steps.acs.outputs.head-score }}"
+    echo "delta: ${{ steps.acs.outputs.delta }}"
+    echo "new errors: ${{ steps.acs.outputs.new-errors }}"
+```
+
+The Action emits outputs before returning its final regression status. Use `if: always()` when a downstream step must consume metrics even after the gate intentionally fails.
+
+## Design principles
+
+1. **Regression-first.** Existing repositories do not have to become perfect before CI becomes useful.
+2. **Baseline-governed policy.** A candidate cannot weaken the gate evaluating itself.
+3. **Auditable exceptions.** Suppressions need a reason and expiry, remain visible, and cannot self-authorize.
+4. **Local-first.** Repository content is not uploaded to a hosted analysis service, and automatic Git baseline detection does not fetch.
+5. **Explainable.** Scores map to stable rule IDs and visible findings.
+6. **Conservative.** Prefer a missed warning over noisy fake certainty.
+7. **Zero runtime dependencies.** Python 3.10+ standard library only.
+
+## Roadmap
+
+- [x] deterministic 0–100 scoring
+- [x] baseline → candidate regression comparison
+- [x] Markdown PR / Step Summary report
+- [x] first-class reusable GitHub Action
+- [x] structured GitHub Action outputs
+- [x] SARIF output for GitHub code scanning
+- [x] Git-aware local diff without manual worktrees
+- [x] automatic local default-branch baseline detection
+- [x] baseline-governed repository policy
+- [x] safe one-command repository initialization
+- [x] repository integration doctor
+- [x] stable Rule Catalog and rule inspection
+- [x] reasoned / expiring suppressions
+- [x] JSON Schema + editor validation
+- [x] end-to-end behavior contract fixtures
+- [x] score-history artifact / badge workflow
+- [x] scope-aware nested `AGENTS.md` analysis
+- [x] Codex `AGENTS.override.md` discovery / precedence
+- [ ] optional semantic contradiction plugin
+
+## Development
+
+```bash
+python -m pip install -e .
+python -m unittest discover -s tests -v
+agent-config-score . --fail-under 90
+```
+
+MIT licensed. Contributions and real-world agent-config failure examples are welcome.
+
+If AgentConfigScore catches a regression in your repository, a ⭐ helps other maintainers discover it.