ai-rulez 5.2.1 → 5.4.0

package/README.md

@@ -43,14 +43,48 @@ repository can loosen.
 - **Supply-chain-grade trust.** Sigstore signing, in-toto/DSSE attestations, CycloneDX/SPDX SBOMs and reviewer approvals, so "what prompt is actually running in production" has a verifiable answer.
 - **Org-wide policy, not just repo config.** A central [policy](docs/policy.md) sets tighten-only floors; `verify` and CI gates enforce it everywhere.
 
-| Stage | What happens | Commands | Docs |
-| ----- | ------------ | -------- | ---- |
-| **Author** | Rules, context, skills, agents, commands and checks as markdown in `.ai-rulez/`, with domains, profiles, roles and includes. `.ai-rulez/` is an [OKF](docs/okf.md) bundle: OKF is the internal format, and `init`, `add` and the MCP tools write it. A pre-OKF tree still loads, with a deprecation notice; `migrate okf` converts it. | `init`, `add`, `convert`, `import`, `migrate okf` | [Author](docs/configuration.md) |
-| **Generate** | Native files for 52 harnesses (Claude Code, Cursor, Codex, Copilot, Gemini CLI, OpenCode, Devin, Kilo and more), per project or per user. | `generate`, `doctor`, `clean` | [Harnesses](docs/harnesses.md) |
-| **Bundle** | Distributable plugin bundles, an Agent Plugins package, an OKF bundle, an ARD manifest, `AGENTS.md` and `llms.txt`. | `generate --plugin`, `export okf` | [Plugins](docs/plugins.md) |
-| **Lint and validate** | Content and security checks with stable `AR` codes, plus each standard's own schema or rules. | `validate`, `scan`, `okf validate`, `verify` | [Validate](docs/strict-validation.md) |
-| **Govern** | A content-pinning lock, reviewer approvals, Sigstore signing, an organization policy and an SBOM. | `lock`, `approve`, `sign`, `verify --attestation`, `sbom` | [Trust model](docs/trust-model.md) |
-| **Publish** | Reproducible, checksummed release artifacts to GitHub releases, npm, OCI and marketplaces. | `publish` | [Publish](docs/publish.md) |
+## Features
+
+The table covers the implemented capabilities. The [CLI reference](docs/cli-reference.md) lists every command and flag;
+[standards](#standards) below distinguishes supported formats from partial integrations.
+
+| Feature | What it does | Commands / configuration | Docs |
+| ------- | ------------ | ------------------------ | ---- |
+| Content authoring | Manage rules, context, skills with scripts and references, agents, commands and code-review checks. | `init`, `add`, `edit`, `show`, `list`, `remove` | [Configuration](docs/configuration.md), [Skills](docs/skills.md), [Checks](docs/checks.md) |
+| Open-standard sources | Use `.ai-rulez/` as an OKF bundle, or author in `AGENTS.md` and `.agents/` without generation overwriting that source. Convert a pre-OKF content tree in place. | `source = "agents"`, `migrate okf`, `export okf`, `import okf`, `okf validate` | [OKF](docs/okf.md), [Agents source](docs/agents-source.md) |
+| Domains, profiles and roles | Organize content by domain, select domains by profile, and compose roles that filter items and control skill modes. | `domain`, `profile`, `roles`, `--profile`, `--role` | [Domains and profiles](docs/domains.md), [Roles](docs/roles.md) |
+| Configuration layers and local overrides | Merge named TOML layers, a developer's local overlay and user-level settings; inspect the effective local changes. | `config layers`, `local`, `config.<name>.toml`, `--no-local` | [Layers](docs/configuration.md#layered-configuration), [Local configuration](docs/local-overrides.md) |
+| Shared includes and skill installation | Reuse local or Git content, choose merge precedence, install individual skills and load skill sources with pinned revisions or version ranges. | `include`, `skill install/list/remove/update`, `includes`, `installed_skills`, `skill_sources` | [Includes](docs/includes.md), [Installed skills](docs/installed-skills.md) |
+| Builtin content | Select bundled domains and individual guidance items, or exclude items from a selected pack. | `builtins list/show`, `builtins` | [Builtin domains](docs/domains.md) |
+| Import and migration | Convert existing harness files and supported third-party layouts with a lossiness report; migrate a 4.x configuration to v5. | `convert`, `migrate v5` | [Conversion](docs/cli.md#convert-command), [Migration](docs/migration-v5.md) |
+| Native generation | Render the file layouts, frontmatter and settings of 52 built-in harnesses, including Claude Code, Cursor, Codex, Copilot and Gemini CLI. | `generate`, `presets` | [Harness matrix](docs/harnesses.md) |
+| Custom presets | Define templates or provider-backed presets for an additional harness and its supported content and settings. | `presets` inline tables, provider specs | [Custom presets](docs/profiles.md), [Provider-backed presets](docs/configuration.md#provider-backed-presets-full-parity) |
+| Hooks, permissions and MCP settings | Translate shared hooks, allow/deny rules and MCP server definitions to each harness's supported native settings; report unsupported translations. | `hooks`, `permissions`, `mcp_servers` | [Settings](docs/settings.md), [Permissions](docs/permissions.md) |
+| Generated-file guard | Install a supported harness's PreToolUse hook to block agent edits to generated files. | `[guard] generated = true`, `guard` | [Guard](docs/cli.md#guard-command) |
+| User scope and monorepos | Generate user-level assistant configuration, watch source changes, or process nested project roots with ordered reports. | `generate --user`, `generate --watch`, `generate/validate --recursive` | [User scope](docs/user-scope.md), [Monorepos](docs/monorepo.md) |
+| Drift, cleanup and diagnostics | Preview a generation plan, detect output drift, remove generated files and diagnose configuration, locks, tools and MCP placeholders. | `generate --dry-run/--emit-plan/--check`, `clean`, `doctor` | [CLI reference](docs/cli.md), [Architecture](docs/architecture.md) |
+| Plugin and marketplace authoring | Bundle runtime-specific plugins and marketplace indexes, including multi-plugin definitions, Agent Plugins and opt-in Pi packages. | `generate --plugin`, `plugin`, `plugins_authoring`, `marketplaces_authoring` | [Plugins](docs/plugins.md), [Agent Plugins](docs/agent-plugins.md) |
+| Discovery and documentation formats | Generate `AGENTS.md` and `llms.txt`; emit ARD manifests and experimental MCP Server Cards. | `agents_md`, `llms-txt` preset, `publish --emit`, `publish emit` | [AGENTS.md](docs/agents-md.md), [llms.txt](docs/llms-txt.md), [ARD](docs/ard.md), [MCP Server Cards](docs/mcp-server-card.md) |
+| Release artifacts and publishing | Build reproducible archives, manifests and checksums, optionally sign them, verify a dist directory and explicitly upload to GitHub releases, npm or OCI. | `publish`, `publish verify`, `--to`, `--execute --yes` | [Publishing](docs/publish.md) |
+| Static validation and security scanning | Check config, content, scripts and conventional harness files with stable `AR` codes, without requiring an ai-rulez config. Scope by paths, globs or Git changes; use fixes, suppressions, baselines and JSON/SARIF reports. | `validate`, `scan`, `--strict`, `--path`, `--glob`, `--outputs`, `--changed` | [Validation and security rules](docs/strict-validation.md), [Harness traps](docs/harness-traps.md) |
+| External scanners | Run configured scanner programs with explicit execution/egress controls and merge their findings into validation reports. | `lint.external`, `scanners list/doctor`, `validate/scan --external` | [Scanner integrations](docs/strict-validation.md) |
+| Repository verifiers | Run declared deterministic checks, opt into command or LLM predicates, link checks to rules, test fixtures and calibrate model-backed checks. | `verifiers run/list/explain/test/calibrate/suggest`, `validate --verifiers` | [Verifiers](docs/verifiers.md) |
+| Model-based judge | Review agent configuration for security, quality and governance with quoted evidence, an adjudicator, rubric skills, change-impact selection and explicit incomplete coverage. Estimate before sending; cap spend and emit JSON or SARIF. | `judge`, `--estimate`, `--allow-llm`, `--changed`, `--skill`, `--fail-on` | [Judge](docs/judge.md) |
+| Rubric review and patch proposals | Score configured items offline or with a semantic model, calibrate a rubric's gate and propose verified patches for findings. | `review`, `review calibrate/explain/fix`, `rubric` | [Review](docs/review.md) |
+| Skill evaluation | Run harness-neutral eval cases through adapters; measure pass rate, trigger precision/recall, ablation and cost; import scenarios and calibrate estimates. | `eval run/import/calibrate-estimate` | [Evals](docs/evals.md) |
+| Skill improvement (experimental) | Run an external optimizer on a copy, gate candidates against held-out evals, inspect results, apply an accepted change or open an isolated PR. | `improve run/show/apply/pr/clean/adapters` | [Improve](docs/improve.md) |
+| Locks and source updates | Pin remote sources, installed skills, authored content and outputs; check drift, inspect outdated sources and update version-range pins. | `lock`, `lock --check/--diff/--outdated`, `update`, `skill update` | [Lock file](docs/lockfile.md) |
+| Organization policy | Apply central floors for sources, approvals, signatures, execution and model use that repository settings cannot loosen. | `--policy`, `AI_RULEZ_POLICY`, `validate --show-policy` | [Policy](docs/policy.md), [Trust model](docs/trust-model.md) |
+| Reviewer approvals | Bind approvals to content digests, review diffs, revoke or deny content, enforce authorized reviewers and verify signed approvals. | `approve`, `verify --approvals` | [Approvals](docs/approvals.md) |
+| Signatures and attestations | Sign locks, bundles, skills, SBOMs and policies; verify Sigstore bundles, in-toto/DSSE attestations and publisher provenance. | `sign`, `verify --attestation/--self/--plugin`, `trust update` | [Signing](docs/signing.md) |
+| Software bills of materials | Inventory AI content, dependencies and MCP packages in CycloneDX 1.6 or SPDX 2.3; detect SBOM drift and enforce lock and pin gates. | `sbom`, `sbom --check -o <file>`, `--require-lock`, `--strict-pins`, `--type spdx-json` | [SBOM](docs/sbom.md) |
+| MCP management and skill serving | Expose configuration tools and reports to assistants; serve skills and supporting resources on demand with role filters and session budgets. Select a read-only server or reuse a local daemon. | `mcp`, `--serve-skills`, `--read-only`, `--daemon` | [MCP server](docs/mcp-server.md) |
+| Skill search | Rank skills lexically, by vectors or with hybrid embeddings, maintain the index, evaluate ranking and mine queries from usage. | `search`, `search index/status/mine`, `search --eval` | [Search](docs/search.md) |
+| Catalog and inventory | Inspect item metadata, owners, roles, dependencies and lock status; join eval/usage results, compare catalogs and build a static catalog website. | `catalog`, `catalog diff`, `catalog --html` | [Catalog](docs/catalog.md) |
+| Token and context budgets | Report generated prompt tokens, compare profiles and roles, identify costly content and gate always-loaded or on-demand budgets. | `tokens`, `cost`, `--budget`, `--on-demand-budget` | [Token reports](docs/cli.md#tokens-command), [Cost reports](docs/cli.md#cost-command) |
+| Usage and item-load telemetry | Opt into identifier-only skill invocation, feedback and item-load records; report usage and optionally export item-load telemetry through OTLP. | `telemetry record/report/feedback/enable/export` | [Usage telemetry](docs/usage-telemetry.md), [Item loads and OTLP](docs/telemetry.md) |
+| Shared model access | Route model calls through liter-llm, including AWS Bedrock credential refresh and OpenAI-compatible gateways, with network consent, budgets, authenticated caching and redacted diagnostics. | `[llm]`, `llm doctor`, `llm estimate` | [LLM access](docs/llm.md) |
+| Automation and embedding | Use the composite GitHub Action, pre-commit/prek, Poly or Lefthook; call the experimental Go API to load, validate, plan/apply generation and manage locks. Shell completion and JSON schemas support other integrations. | [Action](action.yml), `init --setup-hooks`, `completion`, `pkg/airulez` | [Git hooks](#git-hooks), [GitHub Actions](#github-actions), [Go API](docs/embedding.md), [Schemas](docs/schema.md) |
 
 <p align="center">
   <img src="https://raw.githubusercontent.com/Goldziher/ai-rulez/main/docs/assets/quickstart.gif" alt="ai-rulez init and generate: one .ai-rulez source becomes the native files of four AI tools" width="820" />
@@ -126,8 +160,7 @@ full parity through a [custom provider-backed preset](docs/configuration.md#prov
 
 ## Standards
 
-Generation isn't limited to harness files — ai-rulez aims to be compliant with the major emerging standards in the agentic ecosystem. A standard is marked **supported** when the code generates it and
-validates it today, and **planned** when it does not yet.
+Generation isn't limited to harness files — ai-rulez aims to be compliant with the major emerging standards in the agentic ecosystem. A standard is listed when the code generates and validates it today.
 [docs/standards.md](docs/standards.md) carries the pinned spec version, the conformance test suite and the known gaps for
 each row, and is the source of truth for test status.
 
@@ -144,7 +177,6 @@ each row, and is the source of truth for test status.
 | [CycloneDX / SPDX SBOM](docs/sbom.md) | supported | `sbom` | `sbom --check` |
 | [in-toto / DSSE / Sigstore](docs/signing.md) | supported | `sign` | `verify --attestation` |
 | [OpenTelemetry (OTLP)](docs/telemetry.md) | supported | `telemetry export --to otlp` | not applicable |
-| "Agent bundle" | planned, no spec yet | none | none |
 
 ## Usage
 
@@ -202,6 +234,8 @@ ai-rulez validate --strict
 ```
 
 The module path needs `/v5`; without it `@latest` resolves to an old build.
+This local build needs cgo, a C toolchain and the Go module cache at runtime.
+For a statically linked build, use `task build` from a clone; see [source installation](docs/installation.md).
 
 </details>
 
@@ -302,6 +336,9 @@ configured scope and plugin name for the released package.
 ai-rulez validate                        # config plus content checks, exit 2 on findings
 ai-rulez validate --explain AR001        # what a rule checks and how to suppress it
 ai-rulez validate --format sarif --output ai-rulez.sarif
+ai-rulez validate --path catalog --glob "plugins/**/*.json"   # also lint files outside .ai-rulez/
+ai-rulez validate --outputs              # also lint the generated files; each problem is reported once
+ai-rulez validate --strict               # in any repo: with no config it lints AGENTS.md, SKILL.md, .claude/, .agents/
 ai-rulez okf validate docs/okf           # lint any OKF bundle, ai-rulez's or a third party's
 ai-rulez generate --check                # generated files still match their sources
 ```
@@ -344,13 +381,20 @@ An organization [policy](docs/policy.md) sets tighten-only floors a repository c
 - **Telemetry.** Item-load records with an optional OTLP export. See [telemetry](docs/telemetry.md).
 - **Evals and improve.** `ai-rulez eval` scores skills; `improve` (experimental) optimizes them behind a held-out eval
   gate. See [evals](docs/evals.md) and [improve](docs/improve.md).
+- **Judge.** `ai-rulez judge` reviews skills, agents, commands, rules, `CLAUDE.md`, `AGENTS.md`, MCP and settings files
+  with models, from the command line alone: security, quality and governance reviewers, a verifier that checks quoted
+  evidence, an independent adjudicator, and scores computed in code. Failed rubric criteria without a verifiable quote
+  are reported separately as unit-level findings. Governance reads the rules and
+  `CLAUDE.md` files that apply to each unit, nested `.claude/` directories at any depth. `ai-rulez judge --estimate`, then
+  `--allow-llm` to send; `--changed=origin/main` for a pull request, `--format sarif` for code scanning. See [judge](docs/judge.md).
 - **Cost and inventory.** `tokens`, `cost`, `catalog` and `search` report what the configuration contains and costs.
 
 Existing project on 4.x? Run `ai-rulez migrate v5 --dry-run`, then see [Migrating to v5](docs/migration-v5.md).
 
 ## Documentation
 
-Full documentation at [goldziher.github.io/ai-rulez](https://goldziher.github.io/ai-rulez/).
+Full documentation at [goldziher.github.io/ai-rulez](https://goldziher.github.io/ai-rulez/). For how a source becomes
+generated files, and the package map, see [Architecture](docs/architecture.md).
 
 ## Git Hooks
 
@@ -364,7 +408,7 @@ Add to `.pre-commit-config.yaml`:
 ```yaml
 repos:
   - repo: https://github.com/Goldziher/ai-rulez
-    rev: v5.2.1
+    rev: v5.4.0
     hooks:
       - id: ai-rulez-recursive   # generate: write outputs for every nested .ai-rulez/ root
       - id: ai-rulez-validate    # validate --recursive: content, security and link checks
@@ -395,7 +439,7 @@ repository needs:
 [[hooks.sources]]
 id = "ai-rulez"
 git = "https://github.com/Goldziher/ai-rulez.git"
-revision = "v5.2.1"
+revision = "v5.4.0"
 hooks = ["ai-rulez-recursive", "ai-rulez-plugin-verify"]
 ```
 
@@ -467,10 +511,10 @@ jobs:
   check:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
-      - uses: Goldziher/ai-rulez@main   # pin a release tag once one exists past v5.1.0 (the tag predates action.yml)
+      - uses: actions/checkout@v7
+      - uses: Goldziher/ai-rulez@v5.4.0   # pin a release you reviewed
         # with:
-        #   version: 5.1.0   # optional; omit (or "latest") to always use the newest release
+        #   version: 5.4.0   # optional; omit (or "latest") to always use the newest release
       - run: ai-rulez generate --check   # the committed output matches the sources
       - run: ai-rulez validate --strict  # content and security checks, warnings fail
       - run: ai-rulez lock --check        # pins were updated when the sources were

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "ai-rulez",
-  "version": "5.2.1",
-  "description": "Standards-compliant lifecycle tool for agent knowledge and capabilities. Author rules, skills, agents and MCP servers once; generate for 52 harnesses; bundle as OKF, Agent Plugins and ARD; validate against each standard; govern with signing, approvals and policy; publish.",
+  "version": "5.4.0",
+  "description": "Author, generate, validate and govern AI agent rules, skills, plugins and MCP configuration across 52 coding assistants. Review with verified model evidence, evaluate skills, lock and sign dependencies, and publish bundles. CLI, MCP server and Go API.",
   "keywords": [
     "agent-plugins",
     "agent-skills",
@@ -10,6 +10,7 @@
     "ai",
     "ai-assistant",
     "ai-development",
+    "ai-governance",
     "ai-rules",
     "ard",
     "assistant-configuration",
@@ -39,6 +40,7 @@
     "hooks",
     "in-toto",
     "lifecycle",
+    "llm-evaluation",
     "llms-txt",
     "mcp",
     "model-context-protocol",
@@ -92,7 +94,8 @@
     "ai-rulez.js",
     "install.js",
     "README.md",
-    "bin/ai-rulez-*"
+    "bin/ai-rulez-*",
+    "bin/liter_llm_ffi.dll"
   ],
   "os": [
     "darwin",
@@ -101,8 +104,7 @@
   ],
   "cpu": [
     "x64",
-    "arm64",
-    "ia32"
+    "arm64"
   ],
   "engines": {
     "node": ">=22.0.0"