llmlint-cli 0.3.4__py3-none-win_amd64.whl

llmlint_cli-0.3.4.dist-info/METADATA

@@ -0,0 +1,789 @@
+Metadata-Version: 2.4
+Name: llmlint-cli
+Version: 0.3.4
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Rust
+Classifier: Topic :: Software Development :: Quality Assurance
+License-File: LICENSE
+Summary: LLM-as-judge linter: enforce code-quality checks deterministic linters can't express, by driving real coding harnesses through oneharness.
+Keywords: cli,linter,llm,code-review,oneharness
+Home-Page: https://github.com/nickderobertis/llmlint
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
+Project-URL: Changelog, https://github.com/nickderobertis/llmlint/blob/main/CHANGELOG.md
+Project-URL: Repository, https://github.com/nickderobertis/llmlint
+
+# llmlint
+
+![llmlint's live progress view: rules resolving one by one as their judges return, then clearing to reveal the report](docs/screenshots/demo.gif)
+
+**The next generation of linting: an LLM as a judge.** `llmlint` enforces the
+code-quality checks a human reviewer normally makes — adherence to architectural
+patterns, coding-style intent, alignment to organization objectives — that
+deterministic linters can't express. It is **additive** to your existing linters,
+not a replacement: keep using deterministic tools for everything they can already
+check, and reach for llmlint only for the judgment calls.
+
+Each check is a **rule**: a statement about your code that is judged `true`
+(holds) or `false` (a violation). A single fast Rust binary, llmlint **batches**
+your rules into as few harness calls as it can, then drives a real coding harness
+(Claude Code, Codex, Cursor, …) through
+[`oneharness`](https://github.com/nickderobertis/oneharness) to read the relevant
+files and decide, and reports the violations — with file and line numbers where
+they can be pinned down. Because the gate is "just a config file," llmlint drops
+into CI next to your other linters.
+
+By default llmlint reports the failing rules (with the locations it could pin
+down) and a one-line summary — passing, skipped, and not-relevant rules are just
+counted:
+
+![llmlint's default report: a red FAIL with its pinned violation and a colorized summary line counting passed, failed, skipped, and not-relevant rules](docs/screenshots/lint-default.svg)
+
+Add `-v` to itemize *every* rule (passed, skipped, and not relevant too) and to
+print the oneharness debug view — the exact `oneharness run …` command and the
+raw result for each judge — to **stderr**, so the report on stdout stays clean:
+
+![llmlint's verbose report: green PASS, red FAIL, yellow SKIP, and dim N/A not-relevant lines above the summary](docs/screenshots/lint-verbose.svg)
+
+<details>
+<summary>The <code>-v</code> debug view (oneharness command + raw result per judge, on stderr)</summary>
+
+![llmlint's -v oneharness debug view: the exact oneharness run command and the raw JSON result for each judge](docs/screenshots/lint-debug.svg)
+
+</details>
+
+> These are real captures of the CLI, rendered from the actual colorized output
+> by [`just screenshots`](screenshots/AGENTS.md) and gated by
+> [screencomp](https://github.com/nickderobertis/screencomp).
+
+The exit code is unaffected by verbosity (`0` all-pass, `1` a violation, `2`
+the run couldn't complete); operational errors are always shown. Use
+`--format json` for the full machine-readable report.
+
+The human report is **colorized** — green `PASS`, red `FAIL`/`ERROR` — when
+stdout is a terminal. Coloring follows the [`NO_COLOR`](https://no-color.org)
+convention and a `--color <auto|always|never>` flag: `auto` (the default) colors
+only an interactive terminal, `always` forces it (e.g. through a pager or to
+capture a screenshot), `never` disables it. `--format json` is never colorized.
+
+While the judges run, llmlint draws a **live progress view** on stderr — rules
+resolving one by one as their judges return (the GIF above) — then clears it and
+prints the report. Like the color, it is audience-aware: `--progress
+<auto|always|never>` (default `auto`) shows it only for an interactive human — a
+terminal, not CI, and **not an AI coding agent** (Claude Code, Codex, Cursor,
+detected via their environment variables). Piped, redirected, in CI, or captured
+by an agent, the view is fully suppressed so it never spams a log or an agent's
+context — the report on stdout, and `--format json`, are byte-for-byte the same
+either way.
+
+## How it works
+
+1. You declare **rules** (and optionally **agents** that group them) in a YAML
+   config — like any other linter.
+2. For each agent, llmlint renders a system prompt from a template (the rules +
+   the target file paths) and calls `oneharness run` with a generated **JSON
+   Schema** for structured output. oneharness constrains and validates the
+   harness's answer, so llmlint gets a checked verdict per rule, not prose.
+3. The harness reads the target files on demand with its own tools to gather
+   evidence, then returns `{ "rule_name": { "holds": bool, "violations": [...] } }`.
+4. llmlint aggregates (majority vote across judges when configured), reports, and
+   exits non-zero if any rule was violated.
+
+llmlint **shells out to oneharness** — it is a runtime prerequisite (see Install).
+
+## Install
+
+`llmlint` needs the `oneharness` binary on your `PATH`.
+
+```console
+# 1) oneharness (the harness driver)
+curl -fsSL https://raw.githubusercontent.com/nickderobertis/oneharness/main/scripts/install.sh | sh
+#    (or: cargo install --git https://github.com/nickderobertis/oneharness --locked)
+
+# 2) llmlint
+curl -fsSL https://raw.githubusercontent.com/nickderobertis/llmlint/main/scripts/install.sh | sh
+#    (or: pip install llmlint-cli — prebuilt binary wheel, no Rust toolchain needed)
+#    (or: cargo install llmlint --locked)
+#    (or, without a crates.io release: cargo install --git https://github.com/nickderobertis/llmlint --locked)
+
+llmlint doctor      # confirms oneharness is reachable
+```
+
+The installer honors `LLMLINT_VERSION` / `LLMLINT_INSTALL_DIR` (or the `--version`
+/ `--to` flags), works on Linux, macOS, and Windows under a POSIX shell
+(Git Bash / MSYS / WSL), and refuses a binary it cannot verify. Each tagged
+release publishes prebuilt binaries for those platforms, each with a SHA-256
+checksum and a keyless [Sigstore](https://www.sigstore.dev/) build-provenance
+attestation bundle (`.sigstore.json`); on native Windows PowerShell, use
+`cargo install llmlint --locked`.
+
+**Via pip.** Each release also ships per-platform wheels wrapping the same
+prebuilt binary, so anywhere Python is present, `pip install llmlint-cli` (or
+`uv tool install llmlint-cli` / `pipx install llmlint-cli`) is a seconds-fast
+binary install with no Rust toolchain — handy in restricted-egress environments
+where package registries are reachable but `github.com` is not. (The PyPI
+package is `llmlint-cli` — PyPI reserves names too similar to existing projects
+— but the installed binary is `llmlint`.) Wheels are published
+with PyPI [Trusted Publishing](https://docs.pypi.org/trusted-publishers/) and
+carry [PEP 740](https://peps.python.org/pep-0740/) attestations — the same
+Sigstore build provenance as the GitHub release assets.
+
+**Behind a mirror.** In a network that can reach a release-proxy mirror but not
+`github.com`, point the archive download at it:
+
+```console
+LLMLINT_RELEASE_BASE_URL=https://mirror.example/llmlint \
+  curl -fsSL https://raw.githubusercontent.com/nickderobertis/llmlint/main/scripts/install.sh | sh
+```
+
+The archive comes from the mirror, but its integrity is checked against a trust
+root the mirror does not control. If a Sigstore verifier is installed —
+[`cosign`](https://github.com/sigstore/cosign), the official
+[`sigstore`](https://pypi.org/project/sigstore/) Python client, or
+[`gh`](https://cli.github.com/) — the installer downloads the `.sigstore.json`
+bundle from the mirror and verifies it **offline** — no GitHub API — against the
+keyless signature bound to this repo's release workflow; the trusted digest comes
+from the *signed* attestation, so a mirror cannot forge it. With no verifier
+installed it falls back to a `.sha256` fetched from canonical GitHub
+(`LLMLINT_CHECKSUM_BASE_URL` overrides that root) — but it **refuses** a checksum
+that shares the mirror's origin, since a tampered mirror would just serve a
+matching tampered checksum. If nothing independent of the mirror can vouch for
+the archive, the install aborts.
+
+A verifier is one install away even where `github.com` itself is unreachable,
+because all three ship through package registries:
+
+```console
+pip install sigstore                                        # PyPI
+npm install -g @sigstore/cli                                # npm
+go install github.com/sigstore/cosign/v2/cmd/cosign@latest  # Go module proxy
+```
+
+(`go install` fetches through `proxy.golang.org` and integrity-checks against the
+`sum.golang.org` transparency log — it never contacts github.com.) So the pattern
+for a restricted-egress host or sandbox image is: provision one verifier from
+whichever registry is reachable, then run the installer with
+`LLMLINT_RELEASE_BASE_URL` pointing at your mirror — every byte comes from the
+mirror or a registry, and the signature still chains to the Sigstore root.
+Elsewhere, `brew install cosign`, a distro sigstore package, or the
+[`sigstore/cosign-installer`](https://github.com/sigstore/cosign-installer)
+GitHub Action in CI all work too.
+
+You also need a coding harness installed and authenticated (e.g. Claude Code).
+See `oneharness list` / `oneharness detect --all`.
+
+## Quick start
+
+```console
+llmlint init                 # write a starter llmlint.yml (config-lint plugin on)
+llmlint init --with-template # ...and embed the prompt template to customize
+$EDITOR llmlint.yml          # write your rules
+llmlint                      # lint the configured files
+llmlint src/api/**/*.rs      # ...or lint specific files
+llmlint --format json        # machine-readable output
+```
+
+## Configuration
+
+`llmlint.yml` (discovered by walking up from the working directory; override with
+`-c/--config`, repeatable). Discovery is **nested** in both directions. Walking
+**up**, *every* config found (one per directory) is merged, nearest first, so a
+config beside the files being linted, a project config above it, and a user-level
+config higher still layer together — the most-local config wins each top-level
+scalar, every config contributes its rules, and a more distant config fills only
+the gaps (the same nearest-wins precedence as [plugins](#plugins-shared-rule-sets)).
+Walking **down**, a config in a subdirectory governs *its own* part of the project:
+its `files` globs are rooted at that directory (a `frontend/llmlint.yml` with
+`*.txt` matches `frontend/`'s files, never a same-named file elsewhere), so you can
+keep per-area rules next to the code they check. A subtree config scopes *rules*,
+not session-wide settings (model, timeout, template, rationales come from the
+working directory and up); `--config` replaces the whole walk with no cascade.
+`llmlint init` writes it with a leading
+`# yaml-language-server: $schema=…` modeline pointing at llmlint's
+[published JSON Schema](assets/llmlint.schema.json), so editors with the YAML
+language server (e.g. VS Code's [YAML extension](https://marketplace.visualstudio.com/items?itemName=redhat.vscode-yaml))
+give completion and validation as you write. Add the same line to a hand-written
+config to opt in.
+
+```yaml
+version: 1                     # this config's published version (used when it is consumed as a plugin)
+
+# Files linted when none are passed on the CLI. Omit the whole block (or leave
+# `include` empty) to lint every file in the tree from the current directory;
+# `exclude` and `.gitignore` still apply.
+files:
+  include: ["src/**/*.rs"]
+  exclude: ["**/generated/**"]
+
+# Require a short `rationale` for every verdict (default true). See Rationales below.
+rationales: true
+
+# Default base for `--diff` (when `--diff-base` isn't passed). Any git revision —
+# a branch, tag, commit, or `A..B`/`A...B` range. Set it to your default branch so
+# a CI quality gate reviews whatever the current branch changed; unset = `HEAD`.
+diff_base: main
+
+# Pull in shared rule sets / plugins with one line each. An entry is a local
+# path or a URL (`http(s)://`, `file://`); pin a URL to a version with `@`.
+plugins:
+  - "https://raw.githubusercontent.com/nickderobertis/llmlint/main/assets/config_lint.yml@1"  # bundled: lints this config's own rules
+  - "https://example.com/org-rules.yml@1.2.3"   # pinned; fetched + cached once
+  - "./team-rules.yml"
+
+# Agents group rules and add reviewer context + harness/model/batch config.
+# YAML anchors let you share prompt text with zero framework support.
+agents:
+  architecture:
+    harness: claude-code       # any id from `oneharness list`; omit to use oneharness's own default
+    model: opus
+    batch_size: 15             # rules per judge run (default 20)
+    prompt_template: |         # appended to the master template before render
+      You are a senior software architect reviewing service boundaries.
+
+rules:
+  - name: handlers_delegate_to_services   # unique, terse, descriptive
+    description: |
+      true when every HTTP handler delegates business logic to a service layer.
+      false when a handler performs business logic (DB queries, domain rules)
+      inline.
+    agent: architecture        # optional; omit to use the default agent
+    # override: true           # optional; extend a same-named plugin rule, inheriting unset fields
+    judges: 3                  # optional; independent judges, majority wins (default 1)
+    rationale: true            # optional; override the session-wide `rationales` for this rule
+    relevance: true            # optional; when to evaluate — see Relevance below (default true)
+    files:                     # optional; override the target files for this rule
+      include: ["src/api/**"]
+```
+
+### Nested & per-directory configs
+
+Configs **nest** — discovery walks both up from the working directory and down
+into its subtree, merging every `llmlint.yml` it finds. This lets you layer a
+user-level config, a project config, and per-area configs that live next to the
+code they govern, with no extra wiring.
+
+```
+~/.llmlint.yml                 # user-level defaults (model, rationales…)
+my-project/
+├── llmlint.yml                # project rules + settings  (run from here)
+├── backend/
+│   └── llmlint.yml            # rules for backend/**, globs rooted at backend/
+│       #   files: { include: ["**/*.py"] }
+│       #   rules: [{ name: no_print_debugging, … }]
+└── frontend/
+    └── llmlint.yml            # rules for frontend/**, globs rooted at frontend/
+        #   files: { include: ["**/*.ts"] }
+        #   rules: [{ name: no_inline_styles, … }]
+```
+
+Running `llmlint` from `my-project/` evaluates **all** of these together:
+
+- `no_print_debugging` runs only on `backend/**/*.py`, and `no_inline_styles`
+  only on `frontend/**/*.ts` — each subtree config's `files` globs are **rooted
+  at its own directory**, so `**/*.py` under `backend/llmlint.yml` means
+  `backend/**/*.py`, never a stray `.py` elsewhere.
+- The project's own rules and settings apply across the whole run; a more-local
+  config **wins** each setting and can `override` a rule from a config above it.
+- **Session settings** (model, timeout, prompt template, rationales) come from
+  the working directory and up — a subtree config scopes *rules*, it doesn't
+  retune the whole run. Run from `my-project/backend/` instead and that config
+  becomes the most-local one, layering under the project and user configs.
+
+Passing **explicit files** (`llmlint backend/svc.py`) narrows this to just the
+configs that govern those files: a subtree config is consulted only when a passed
+file lives under it, and each rule judges only the passed files inside its own
+directory. So `llmlint backend/svc.py` never loads `frontend/llmlint.yml` (nor
+fetches its plugins, nor collides with a rule it happens to share a name with) —
+you get exactly the rules that apply to what you asked to lint.
+
+Use `llmlint config` to see the merged result and `llmlint config --sources`
+(or `llmlint where rules.<name>`) to trace any rule, agent, or setting back to
+the exact file it came from. To bypass discovery entirely, pass explicit
+configs with `-c/--config` (repeatable) — that roots every glob at the working
+directory with no cascade.
+
+### Writing good rules
+
+- **Phrase each rule as a positive invariant.** `holds = true` means the code
+  complies; `holds = false` is a violation that llmlint reports and fails on.
+- **Make the verdict unambiguous.** Often a plain statement of the property is
+  enough — `every public item has a doc comment` already says what passes and,
+  by negation, what fails; there's no need to bolt on a "false when…" clause that
+  only restates the inverse. Spell out the violating case *only* when it carries
+  meaningful detail — concrete examples, easily-confused edge cases — that the
+  positive statement leaves unclear. When you do state both, keep them mutually
+  exclusive. The bundled config-lint plugin (the `config_lint.yml` URL above)
+  lints your config for exactly this, plus descriptive (non-placeholder) names
+  that match what each rule checks.
+- **Names** are unique, terse, and descriptive (`^[A-Za-z][A-Za-z0-9_]*$`); they
+  become the JSON keys of the structured output.
+- **Scope a rule to the changes it applies to with `relevance`** (see below)
+  instead of bolting "…or not applicable" onto the description — that keeps the
+  true/false outcome clean and lets llmlint tell "didn't apply" apart from "true".
+- **Keep each `description` and `relevance` concise.** A judge call batches an
+  agent's rules into one prompt, so tokens one bloated rule spends dilute every
+  other rule in that batch. State the invariant in the fewest words that keep it
+  unambiguous; add length only when it buys clarity (concrete examples, tricky
+  edge cases). config-lint checks this too.
+
+### The prompt template
+
+llmlint renders the judge's system prompt from a
+[minijinja](https://docs.rs/minijinja) (Jinja2-style) template. The bundled
+default lives in [`assets/default_template.md`](assets/default_template.md); embed
+a copy to customize with `llmlint init --with-template`, or set `prompt_template`
+yourself. The top-level `prompt_template` *replaces* the master template; an
+agent's `prompt_template` is **appended** to it before rendering, so reviewer
+context you add per-agent sees the same variables.
+
+These variables are in scope when a template renders:
+
+| Variable | Type | Description |
+| --- | --- | --- |
+| `files` | list of strings | The target file paths for this run — relative to the working directory, always forward-slashed (so a Windows run reads the same as Linux/macOS). |
+| `rules` | list of objects | The rules in this batch. Each has `.name` (the identifier, also the JSON key in the structured output), `.description` (the invariant to judge), `.rationale` (whether this rule wants a justification), `.relevance` (the relevance condition string, or unset for an always-evaluated rule), `.require_line_attribution` (whether every violation must cite a `file` + `line`), and `.files` (the subset of `files` this rule applies to). |
+| `file_rules` | list of objects | Per-file applicability — one entry per target file, in the same order as `files`. Each has `.file` (the path), `.mode` (`"include"` or `"exclude"`), `.rules` (the rule names to apply or, when `mode == "exclude"`, to skip — whichever list is shorter), and `.diff` (that file's unified diff, present only under `--diff` when the file changed). The default template's "Target files" section is built from this. |
+| `diffs` | list of objects | Per-file changed-line diffs — one entry per *changed* file, present only under `--diff` (empty otherwise). Each has `.file` (matching its entry in `files`) and `.diff` (the unified diff text). The default template inlines these per file via `file_rules`; kept separately for custom templates. |
+| `rationales` | bool | True when any rule in this batch wants a rationale — gate the rationale guidance on it. |
+| `relevance` | bool | True when any rule in this batch carries a relevance condition — gate the relevance guidance on it. |
+| `line_attribution` | bool | True when any rule in this batch requires line attribution — gate the line-attribution guidance on it. |
+
+```jinja
+## Target files
+{% for f in files %}- {{ f }}
+{% endfor %}
+## Rules to evaluate
+{% for r in rules %}### {{ r.name }}
+
+{{ r.description }}
+{% endfor %}
+```
+
+A run is one `(agent, file set, judge)` batch, so `rules` is that batch's slice
+(see `batch_size`), not necessarily every rule in the config.
+
+### Rationales
+
+By default each judge must justify every verdict with a short **rationale**. The
+structured output for each rule is ordered deliberately — the judge echoes the
+rule **name**, writes the **rationale**, then commits to the **result**
+(`holds` + any `violations`):
+
+```jsonc
+{
+  "no_inline_sql": {
+    "name": "no_inline_sql",                       // 1. anchor on the rule
+    "rationale": "raw SQL built inline in db.rs:42, not via the query layer",  // 2. reason
+    "holds": false,                                // 3. conclude
+    "violations": [{ "file": "src/db.rs", "line": 42, "message": "inline SQL" }]
+  }
+}
+```
+
+Reasoning *before* concluding (and naming the rule first) keeps each verdict
+consistent and targeted — it leans on the model's next-token prediction so the
+`holds` follows from the evidence just written, not the other way round. Beyond
+that, rationales buy you:
+
+- **Auditability** — a durable record of *why* each verdict landed, carried in
+  `--format json` for every rule (pass or fail).
+- **Debugging** — when a verdict looks wrong, you see the judge's reasoning, not
+  just a bare pass/fail.
+- **Reliability** — verdicts are measurably steadier when the judge must commit
+  to evidence first.
+
+The cost is **extra output tokens on every request**. Turn rationales off to
+save tokens:
+
+```yaml
+rationales: false            # session-wide default (CLI --no-rationales overrides it)
+
+rules:
+  - name: handlers_delegate_to_services
+    description: ...
+    rationale: true           # …but keep them for this high-stakes rule
+```
+
+Precedence, lowest to highest: the session default `rationales` (default `true`)
+→ a per-rule `rationale` → the `--rationales` / `--no-rationales` CLI flags
+(which set the session default for the run; a per-rule `rationale` still wins).
+In the human report, a rule's rationale is shown for every **failure** by
+default, and for **every evaluated rule** at `-v`. The default prompt template
+asks for rationales that are terse and pithy — the fewest tokens that still cite
+the evidence — so the token cost stays small. See
+[Cost vs performance](#cost-vs-performance-token-usage) to trade it for a cheaper run.
+
+For a **multi-judge** rule (`judges: N`), the report and `--format json` show
+**each judge's** result and rationale, not just one representative — so you can
+see exactly where the judges agreed or split:
+
+![llmlint's multi-judge report: a FAIL headed "1/3 judges held" with each judge's held/violated line and rationale, then the pinned violation](docs/screenshots/multi-judge.svg)
+
+### Relevance
+
+Not every rule applies to every change. Rather than make each `description`
+carry its own "…or not applicable" escape hatch — which muddies the true/false
+outcome and hides *why* a rule passed — declare when a rule should be evaluated
+with **`relevance`**:
+
+```yaml
+rules:
+  # Always evaluated (the default). The judge may not opt out.
+  - name: public_items_are_documented
+    description: ...
+    # relevance: true            # implicit
+
+  # Never evaluated — disabled deterministically, with no judge call.
+  - name: legacy_only_check
+    description: ...
+    relevance: false
+
+  # Conditionally evaluated. The judge decides whether the condition holds for
+  # the change *before* the verdict; if it doesn't, the rule is "not relevant".
+  - name: errors_are_contextualized
+    description: |
+      TRUE when every returned error adds context about the operation that
+      failed. FALSE when an error is propagated with no added context.
+    relevance: the change adds or modifies error handling
+```
+
+For a conditional rule the structured output gains a `relevant` boolean, decided
+before the verdict — so a not-applicable rule is distinguishable from a true one:
+
+```jsonc
+// Not relevant: the object ends after `relevant`; the rationale explains why.
+{ "errors_are_contextualized": {
+    "name": "errors_are_contextualized",
+    "rationale": "the change only renames a struct field; no error handling touched",
+    "relevant": false } }
+
+// Relevant: proceed to the verdict as usual.
+{ "errors_are_contextualized": {
+    "name": "errors_are_contextualized",
+    "rationale": "every `?` propagation wraps with `.context(...)`",
+    "relevant": true,
+    "holds": true } }
+```
+
+A **not-relevant** rule is neither a pass nor a violation — it never fails the
+build. The human report counts it in a `… not relevant` summary segment and, at
+`-v`, itemizes it as a dim `N/A <rule> (not relevant)` line with the reason;
+`--format json` carries `"outcome": "not_relevant"` and a `not_relevant` summary
+count. For a multi-judge rule, relevance is decided by majority first, then the
+verdict is tallied over the judges that found it relevant.
+
+### Ignore directives
+
+Suppress a rule at a specific place with an inline comment in the target file —
+the same idea as `# noqa` / `// eslint-disable`, but **strict**: a directive must
+name the specific rule(s) and give a reason.
+
+```rust
+let q = format!("SELECT * FROM users WHERE id = {id}"); // llmlint: ignore[no_inline_sql] one-off migration, not user-facing
+```
+
+```python
+# llmlint: ignore-file[public_items_are_documented] generated stubs, documented upstream
+```
+
+```rust
+// llmlint: ignore-block[no_inline_sql] legacy query layer, migration tracked in JIRA-42
+fn legacy_queries() { /* … */ }
+// llmlint: ignore-end[no_inline_sql]
+```
+
+- `llmlint: ignore[rule, ...] <reason>` is **line-scoped** — it covers the line it
+  sits on (a trailing comment) or the line right below it (a comment on its own line).
+- `llmlint: ignore-file[rule, ...] <reason>` is **file-scoped** — it covers the
+  whole file.
+- `llmlint: ignore-block[rule, ...] <reason>` … `llmlint: ignore-end[rule, ...]` is
+  **block-scoped** — it covers every line between the open and its matching close.
+  The closing `ignore-end` names the same rule(s) and needs no reason. Blocks track
+  each rule independently, so rules opened together may be closed at different points
+  and blocks for different rules may overlap.
+
+Use whatever comment syntax the file's language uses (`//`, `#`, `/* … */`, `<!-- … -->`);
+llmlint keys off the reserved `llmlint: ignore` / `llmlint: ignore-file` /
+`llmlint: ignore-block` / `llmlint: ignore-end` prefix.
+
+**Two layers, by design.** llmlint deterministically validates each directive's
+*structure* before any judge runs — it must name **specific, configured** rule(s)
+and carry a **reason** (except `ignore-end`, which only closes a block). A directive
+with no brackets, an empty list, an unknown or misspelled rule, or no reason is a
+hard `file:line:` error (exit 2), so a typo fails loudly instead of silently
+suppressing nothing. Block pairing is checked too: an unclosed `ignore-block`, an
+`ignore-end` with no open block, or re-opening a rule already open is a hard error. Actually *honoring* a
+well-formed directive is the judge's job: the default prompt tells it to skip a
+named rule's violation at the directive's location. (A custom `prompt_template`
+should carry the same guidance if you want directives honored.) Because the
+prefix is reserved, a *linted* file that merely documents the feature must use
+real rule names or avoid the literal `llmlint: ignore[…]` form.
+
+This structural check is **deterministic and free** — no model call — so it is
+also exposed as its own command, [`llmlint check-ignores`](#commands--exit-codes).
+Run it in your tight, fast linter loop (next to `cargo fmt` / `clippy`, in a
+pre-commit hook, or as a quick CI step), where it catches a typo'd or
+reason-less directive in milliseconds. The full `llmlint` run performs the same
+check as a pre-flight, so the two never disagree — `check-ignores` just gives you
+the fast feedback without waiting on (or paying for) a judge.
+
+### Batching
+
+Model calls are the slow, paid part, so llmlint packs rules into as few as it can.
+Rules group **by agent**, then split into batches of at most `batch_size`
+(default 20) — one `oneharness run` per batch, over the union of its files, each
+rule scoped to its own files in the prompt. Multi-judge rules fan out per judge
+(judge `j` runs the rules with `judges >= j`). Fewer, fuller batches, fewer
+round-trips.
+
+### Judges and voting
+
+`judges: N` runs a rule through `N` independent judges and takes the **majority**
+verdict. `N` must be **odd** (1, 3, 5, …) so the vote can't tie — an even count is
+a config error. Only rules that opt in pay the extra cost: judge 1 runs all rules,
+judge 2 only the rules with `judges >= 2`, and so on.
+
+### Cost vs performance (token usage)
+
+Defaults favor **judgment quality** over cost: rationales on, a thorough prompt,
+every file read in full. Trade some back for fewer tokens, roughly by impact:
+
+- **`judges`** — each extra judge is a full extra pass ([Judges](#judges-and-voting)).
+  Keep it at 1 except for high-stakes rules.
+- **`rationales: false`** — drops the per-verdict justification, output tokens on
+  *every* rule ([Rationales](#rationales)). Re-enable `rationale: true` per rule;
+  `--no-rationales` for one run.
+- **Fewer agents, bigger `batch_size`** — every batch re-sends the prompt and
+  re-reads its files ([Batching](#batching)). Merge rules onto one agent; split
+  only for a different harness, model, or reviewer context.
+- **Read less** — narrow `files.include`/`exclude`; `--diff` reviews only changed
+  lines; `FILES`/`--rule`/`--agent` lint a subset.
+- **`require_line_attribution`** off unless you need pinned locations — on can
+  trigger localize re-prompts.
+- **`oneharness.schema_max_retries`** — caps re-asks on a schema-invalid answer.
+- **`model`** — dollars, not tokens: a cheaper model per agent or run.
+
+`llmlint check-ignores` spends no tokens at all.
+
+### oneharness passthrough
+
+llmlint lets oneharness discover its own `oneharness.toml` by default. To force a
+specific oneharness config, use `--oneharness-config <path>` (or `oneharness.config`
+in the llmlint config); it is forwarded via oneharness's `--config`. Override the
+binary with `--oneharness-bin` or `$LLMLINT_ONEHARNESS_BIN`.
+
+### Plugins (shared rule sets)
+
+`plugins` pulls other llmlint configs into this one — their rules and agents are
+merged in. For the **top-level settings** (template, files, oneharness,
+rationales), **the nearer config to the root wins**: your config's settings take
+precedence over a plugin's, a plugin's over its own plugins', and an
+earlier-listed plugin over a later sibling. A plugin only *fills in* a setting
+the including config left unset, so a shared plugin can ship sensible defaults
+without overriding what you set locally. The CLI overrides all of them (see
+Commands). Each entry is a config file:
+
+- a **local path** (`./team-rules.yml`), resolved relative to the including file;
+- a **URL** — `http(s)://` (fetched over HTTPS) or `file://` (read directly).
+
+Resolution is **transitive**: a pulled-in config's own `plugins` are pulled in
+turn, and so on. Diamonds and cycles are de-duplicated (each config loads once),
+and the chain is bounded at a depth of 100 to fail fast on a pathological graph.
+
+By default a rule name is **unique** across the whole merged config — declaring
+the same name twice is an error. To **adjust** a rule a plugin gave you without
+restating it, re-declare it with `override: true` and set only the fields you
+want to change; every other field (including the `description`) is inherited from
+the plugin's rule:
+
+```yaml
+plugins:
+  - "https://example.com/org-rules.yml@1"   # ships `no_inline_sql`, 1 judge
+
+rules:
+  # Keep the org rule's text, but vote it across 3 judges and scope it tighter.
+  - name: no_inline_sql
+    override: true
+    judges: 3
+    files:
+      include: ["src/db/**"]
+```
+
+The override must be set on the **nearer-root** config, and there must be exactly
+one base rule (the same name declared *without* `override`) for it to extend — an
+`override` with nothing to override is an error, so a typo'd name can't silently
+do nothing. When several configs override the same base, the nearest-root
+override wins each field.
+
+URL fetching is built in (a pure-Rust HTTPS client — no `curl` or other external
+tools, no system OpenSSL) and honors the standard `HTTP(S)_PROXY` / `NO_PROXY`
+env vars. The bundled config-lint plugin ships inside the binary and resolves
+**offline**.
+
+A URL may be **pinned to a version** with an `@` suffix matching the plugin
+config's own top-level `version`: `@1` accepts any `1.x`, `@1.2` any `1.2.x`,
+`@1.2.3` exactly that. The pin is both an assertion (a mismatch is a hard error)
+and the **cache key**: a pinned URL is fetched once into the cache and reused on
+later runs without refetching — bump the pin to pull a new version. An *unpinned*
+URL is fetched every run.
+
+The cache lives under `$XDG_CACHE_HOME/llmlint/plugins` (override with
+`LLMLINT_CACHE_DIR`). Set `LLMLINT_PLUGIN_REFRESH=1` to force a refetch.
+
+### Linting your llmlint configs
+
+llmlint ships with a **config-lint** rule set that lints llmlint config files
+themselves — that every rule's `description` yields a clear, unambiguous verdict,
+its `name` is descriptive (non-placeholder) and matches what the description
+checks, and a conditional rule uses `relevance` instead of bolting "…or not
+applicable" onto the description. It's the [Writing good rules](#writing-good-rules)
+guidance, enforced (and each rule is phrased to pass its own checks). Each finding
+cites the config file + the offending rule's line (`require_line_attribution`).
+There are two ways to use it:
+
+- **As a plugin** — add the bundled URL to your `plugins` (it's on by default in
+  `llmlint init`), and its rules run against your config files on every normal
+  `llmlint` run, alongside your own rules:
+
+  ```yaml
+  plugins:
+    - "https://raw.githubusercontent.com/nickderobertis/llmlint/main/assets/config_lint.yml@1"
+  ```
+
+  The URL ships **inside the binary** and resolves offline (no network, no
+  cache), so it works disconnected and needs no pin bump to stay current.
+
+- **As a subcommand** — `llmlint lint-config` is the `lint` command with that
+  plugin included by default, so you don't have to add it to your config. It
+  first runs the deterministic ignore-directive (comment) check over the config
+  files, then judges each config's rules. Point it at specific files
+  (`llmlint lint-config path/to/llmlint.yml`) or let it discover every llmlint
+  config in the tree. Handy in CI as a standalone "is my config well-authored?"
+  gate.
+
+### Finding where something is defined
+
+Once configs merge across files and plugins, a rule, agent, or setting in the
+effective config can come from any of them. Two commands trace an item back to
+the file (or plugin URL) you'd edit to change it.
+
+`llmlint where <path>` answers one lookup and prints **just the source**, so it
+composes in scripts. The path mirrors the config structure:
+
+```console
+$ llmlint where oneharness.model            # a top-level setting
+./shared/team.yml
+$ llmlint where agents.security             # an agent
+./shared/team.yml
+$ llmlint where rules.no_inline_sql         # where a rule is defined
+https://example.com/org-rules.yml@1
+$ llmlint where rules.no_inline_sql.judges  # the file an override set a field in
+./llmlint.yml
+$ editor "$(llmlint where rules.no_inline_sql.judges)"
+```
+
+Because an `override` resolves **field by field**, a single rule can draw its
+`description` from the plugin that defined it and its `judges` from your config —
+`where rules.<name>.<field>` points at the file that actually set that field (or
+the definition site when no override did). An unknown name lists what's available,
+and a setting left at its built-in default says so, both exiting non-zero.
+
+For the whole picture at once, `llmlint config --sources` adds a `sources` block:
+
+```jsonc
+{
+  "config_files": ["./llmlint.yml", "./shared/team.yml", "https://example.com/org-rules.yml@1"],
+  "sources": {
+    "settings": { "version": "./llmlint.yml", "oneharness.model": "./shared/team.yml" },
+    "agents":   { "security": "./shared/team.yml" },
+    "rules": {
+      "no_inline_sql": {
+        "source": "https://example.com/org-rules.yml@1",  // where the rule is defined
+        "fields": { "judges": "./llmlint.yml" }            // a field an override moved
+      }
+    }
+  },
+  "config": { /* … the merged config … */ }
+}
+```
+
+A rule with no cross-file override has no `fields` entry; settings and agents are
+each kept whole from the nearest-root config that set them, so they have a single
+source.
+
+## Commands & exit codes
+
+- `llmlint [FILES...]` — lint (the default). `--format human|json`, `--agent`,
+  `--rule`, `--max-parallel`, `--timeout`, `--cwd`. Target individual rules with
+  `--rule NAME` (repeatable) or a whole group with `--agent NAME`; an unknown
+  rule/agent name is an exit-2 error that lists the available names. Every
+  top-level setting also has a flag that wins over the config:
+  `--rationales`/`--no-rationales`, `--model NAME`, `--schema-max-retries N`,
+  `--prompt-template PATH`, plus `--oneharness-bin`/`--oneharness-config`. Pass
+  `--diff [<backend>]` to add each changed file's diff to the judge prompt so it
+  reviews only the changed lines; bare `--diff` uses the `git` backend (compared
+  against `HEAD`). Add `--diff-base <REF>` to compare against a different git
+  revision instead of `HEAD` — a branch, tag, commit, or `A..B`/`A...B` range —
+  so `--diff --diff-base main` reviews exactly what the current branch changed
+  versus `main` (the PR-review case). The base can also be set once in config as
+  `diff_base:` (the flag overrides it).
+- `llmlint check-ignores [FILES...]` — validate the *structure* of inline
+  `llmlint: ignore` directives in the target files, **deterministically and with
+  no model call** (`-c/--config`, `--cwd`; pass `FILES` to scope it, e.g. the
+  changed files in a pre-commit hook). This is the same pre-flight `lint` runs,
+  split out for the fast static-check loop: exit `0` when every directive is
+  well-formed, exit `2` (located `file:line:`) on a typo'd / reason-less /
+  unbalanced one.
+- `llmlint lint-config [FILES...]` — lint llmlint config files with the bundled
+  [config-lint](#linting-your-llmlint-configs) rules, without adding the plugin to
+  your own config. It's the `lint` engine with that plugin forced on: it first
+  runs the deterministic comment (ignore-directive) check, then judges each
+  config's rules. Shares `lint`'s flags (`--format`, `--model`, `--timeout`,
+  `--cwd`, `--diff`, …); the config source is fixed, so `--config`/`--agent`/
+  `--rule` aren't taken. Same exit codes as `lint`.
+- `llmlint init` — write a starter config (`--with-template`, `--global`, `--force`).
+
+  ![llmlint init writing a starter llmlint.yml](docs/screenshots/init.svg)
+- `llmlint config` — print the merged config and the ordered list of sources that
+  contributed, as JSON. Add `--sources` to also trace every rule, agent, and
+  setting back to the file (or plugin URL) it came from — see
+  [Finding where something is defined](#finding-where-something-is-defined).
+
+  ![llmlint config printing the merged config and its sources as JSON](docs/screenshots/config.svg)
+- `llmlint where <path>` — print the single source of one config item: a setting
+  (`oneharness.model`, `version`), `agents.<name>`, `rules.<name>`, or a rule
+  field `rules.<name>.<field>`. See
+  [Finding where something is defined](#finding-where-something-is-defined).
+- `llmlint doctor` — check that oneharness is installed and reachable.
+
+  ![llmlint doctor reporting the resolved oneharness version](docs/screenshots/doctor.svg)
+
+Exit codes: `0` all rules hold · `1` at least one violation · `2` usage,
+configuration, or harness error (could not complete the lint).
+
+## Development
+
+```console
+just bootstrap   # toolchain components + fetch (from a clean clone)
+just check       # full gate: fmt, clippy -D warnings, tests + 95% coverage, docs
+just test-e2e    # the e2e binary journeys in isolation
+just deps-check  # cargo deny + cargo machete
+just lint-live   # opt-in: ad-hoc lint against the REAL oneharness + a real harness
+just live-claude # opt-in: live e2e — built llmlint → real oneharness → real harness
+```
+
+Tests drive the real `llmlint` binary against a hermetic mock-oneharness fixture.
+The live tier (`just live-claude`, and the ad-hoc `just lint-live`) drives the
+whole stack end to end against a real, authenticated harness — the only thing that
+makes real model calls, and out of the `check` gate. It runs on PRs in its own
+workflow across Linux/macOS/Windows, so a missing CLI, auth, or oneharness is a
+hard failure, not a skip. See `AGENTS.md` and `tests/AGENTS.md`.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+