rubrkit 0.1.0 → 0.3.0

package/README.md

@@ -1,126 +1,417 @@
-# Rubrkit CLI
-
-Local v1 package for pulling Rubrkit artifact-bundle files into agent projects and testing artifacts from local projects or CI.
-
-The package folder is `tools/rubrkit-cli`, but the intended published package name and executable are `rubrkit` so users can run:
-
-```bash
-npx rubrkit pull [all|<artifact-bundle-or-artifact-selector>]
-npx rubrkit validate <path|glob>
-npx rubrkit test <path|glob|artifact-bundle-or-artifact-selector>
-npx rubrkit audit <artifact-bundle-or-selector>
-npx rubrkit eval <artifact-bundle-or-selector>
-npx rubrkit report <job-or-run-id>
-```
-
-## Auth
-
-Use a Rubrkit API key for pull commands and remote checks. Prefer a pull key scoped to `artifacts:pull`. For `rubrkit test <path> --remote`, use a testing key scoped to the existing composed API flow: `artifact_bundles:write`, `files:write`, `audits:run`, `jobs:read`, and `credits:read`.
-
-```bash
-RUBRKIT_API_KEY=... rubrkit pull all --yes
-```
-
-`--api-key` is also supported for secret-manager wrappers. API keys are never written to `.rubrkit/manifest.json`, local reports, fixtures, or snapshots.
-
-## Pull Examples
-
-```bash
-rubrkit pull
-rubrkit pull all --yes
-rubrkit pull team-agent-kit --destination .
-rubrkit pull --artifact-bundle team-agent-kit --artifact AGENTS.md --yes
-rubrkit pull all --yes --dry-run
-```
-
-Useful flags:
-
-```text
---destination <path>
---agent <auto|codex|claude|generic>
---artifact-bundle <id-or-name>
---artifact <id-or-path>
---rubric <rubric-id-or-path>
---format <text|json|junit>
---output <path>
---fail-under <score>
---fail-on <critical|high|medium|low>
---ci
---local
---remote
---no-ai
---watch
---changed
---all
---yes
---dry-run
---force
---prune
---update-only
---config <path>
---api-url <url>
---api-key <key>
-```
-
-## Testing Examples
-
-Local validation runs without network access or credits where possible:
-
-```bash
-rubrkit validate docs/example.rubr_flow
-rubrkit test "prompts/**/*.md" --local --format json
-```
-
-Remote checks use the public `/api/v1` API, create async jobs, and poll `/jobs/{jobId}`. Local-file remote tests create or select an artifact bundle, upload the local text files, then start an audit job:
-
-```bash
-RUBRKIT_API_KEY=... rubrkit test team-agent-kit --remote --format json
-RUBRKIT_API_KEY=... rubrkit audit team-agent-kit --fail-under 85 --ci
-RUBRKIT_API_KEY=... rubrkit eval team-agent-kit --format junit --output rubrkit-junit.xml
-RUBRKIT_API_KEY=... rubrkit report <job-id>
-```
-
-Use `--dry-run --remote` to inspect the remote request shape without calling the API.
-
-Exit codes:
-
-- `0`: checks passed.
-- `1`: checks completed and failed validation or quality gates.
-- `2`: invalid CLI usage or invalid config.
-- `3`: authentication or authorization failure.
-- `4`: credit, usage-limit, or circuit-breaker block.
-- `5`: network, provider, or server failure.
-
-## SDK
-
-The current local package exports the SDK from `rubrkit` while final package naming awaits owner approval.
-
-```js
-import { Rubrkit } from 'rubrkit';
-
-const client = new Rubrkit({ apiKey: process.env.RUBRKIT_API_KEY });
-const started = await client.artifacts.test({ artifactBundleId: 'team-agent-kit' });
-const job = await client.jobs.wait(started.jobId);
-```
-
-## Placement
-
-- Codex primary artifacts are placed as `AGENTS.md`; supporting files go under `.rubrkit/artifacts/<bundle>/`.
-- Claude primary artifacts are placed as `CLAUDE.md`, or `.claude/CLAUDE.md` when that file already exists; supporting files go under `.rubrkit/artifacts/<bundle>/`.
-- Generic placement preserves artifact paths under `.rubrkit/artifacts/<bundle>/`.
-
-The CLI protects existing local files unless they were previously managed by Rubrkit and match the local manifest. Use `--force` to overwrite after reviewing the plan.
-
-## Postinstall
-
-Rubrkit does not run network sync from its own package lifecycle. Consuming projects can opt in:
-
-```json
-{
-  "scripts": {
-    "postinstall": "rubrkit pull all --yes"
-  }
-}
-```
-
-Postinstall commands should use `RUBRKIT_API_KEY` and an unambiguous selector.
+# Rubrkit CLI
+
+Pull Rubrkit artifact bundles into agent projects and validate/test artifacts locally or remotely.
+
+```bash
+npx rubrkit pull [all|<artifact-bundle-or-artifact-selector>] [options]
+npx rubrkit validate <path|glob> [options]
+npx rubrkit test <path|glob|artifact-bundle-or-artifact-selector> [options]
+npx rubrkit audit <artifact-bundle-or-selector> [options]
+npx rubrkit eval <artifact-bundle-or-selector> [options]
+npx rubrkit report <job-or-run-id> [options]
+```
+
+## Commands
+
+### pull
+
+Downloads artifact bundles from Rubrkit and writes them to your local project.
+
+**Syntax:**
+```bash
+rubrkit pull [all|<bundle>] [options]
+```
+
+**Examples:**
+```bash
+rubrkit pull                                              # Interactive mode
+rubrkit pull all --yes                                    # Pull all bundles without confirmation
+rubrkit pull team-agent-kit                               # Pull one bundle
+rubrkit pull --artifact-bundle team-agent-kit --artifact AGENTS.md  # Pull specific artifact
+rubrkit pull --label prod --yes                           # Pull every bundle carrying a label
+rubrkit pull all --yes --dry-run                          # Preview what would be written
+```
+
+**Requires:** `RUBRKIT_API_KEY` environment variable or `--api-key` flag. Use a key scoped to `artifacts:pull`.
+
+### validate
+
+Checks artifact syntax and structure locally (no API calls, no credits).
+
+**Syntax:**
+```bash
+rubrkit validate <path|glob> [options]
+```
+
+**Examples:**
+```bash
+rubrkit validate docs/example.rubr_flow
+rubrkit validate "prompts/**/*.md" --format json
+```
+
+### test
+
+Validates artifacts locally or submits them to Rubrkit for remote testing and quality scoring.
+
+**Syntax:**
+```bash
+rubrkit test <path|glob|artifact-bundle-or-artifact-selector> [options]
+```
+
+**Examples:**
+```bash
+rubrkit test "prompts/**/*.md" --local --format json         # Local only, no API
+RUBRKIT_API_KEY=... rubrkit test team-agent-kit --remote    # Remote test with scoring
+```
+
+### audit
+
+Submits artifacts to Rubrkit and runs audit checks with quality gates.
+
+**Syntax:**
+```bash
+rubrkit audit <artifact-bundle-or-selector> [options]
+```
+
+**Examples:**
+```bash
+RUBRKIT_API_KEY=... rubrkit audit team-agent-kit --fail-under 85 --ci
+RUBRKIT_API_KEY=... rubrkit audit team-agent-kit --no-cache   # Force a fresh run
+```
+
+**Deterministic results (caching):** Audit results are cached per account by a content hash of the inputs (artifact contents, rubric, artifact type, and the scoring engine/model version). Re-running an audit on unchanged inputs returns the identical stored result instantly, charges no credits, and creates no job — so repeated CI runs are reproducible. The result is reported as cached (the text output prints `Cached: yes`; JSON/JUnit payloads carry a `cached` field). Changing any input — or a server-side engine/model version bump — produces a fresh result automatically. Use `--no-cache` to bypass the cache and force a fresh run.
+
+**Requires:** `RUBRKIT_API_KEY` environment variable or `--api-key` flag. Use a key scoped to: `artifact_bundles:write`, `files:write`, `audits:run`, `jobs:read`, `credits:read`.
+
+### eval
+
+Evaluates artifacts against rubrics with detailed scoring and reporting.
+
+**Syntax:**
+```bash
+rubrkit eval <artifact-bundle-or-selector> [options]
+```
+
+**Examples:**
+```bash
+RUBRKIT_API_KEY=... rubrkit eval team-agent-kit --format junit --output rubrkit-junit.xml
+```
+
+**Requires:** API key with testing scope (same as `audit`).
+
+### report
+
+Fetches results from a completed async job.
+
+**Syntax:**
+```bash
+rubrkit report <job-id> [options]
+```
+
+**Examples:**
+```bash
+RUBRKIT_API_KEY=... rubrkit report job-abc-123 --format json
+```
+
+**Requires:** API key with `jobs:read` scope.
+
+---
+
+## Flags
+
+### Output & Formatting
+
+**`--format <text|json|junit>`**
+Output format for test/audit/eval results. Defaults to `text`.
+- `text`: Human-readable summary
+- `json`: Structured JSON for parsing
+- `junit`: JUnit XML for CI systems
+
+**`--output <path>`**
+Write results to a file instead of stdout.
+
+### Artifact Selection
+
+**`--artifact-bundle <id-or-name>`**
+Select a specific artifact bundle to pull or test.
+
+**`--artifact <id-or-path>`**
+Select a specific artifact file within a bundle.
+
+**`--label <name>`**
+Pull every artifact in all bundles carrying this label. Repeatable and comma-separated, with **OR** semantics — a bundle matches if it has ANY of the given labels (`--label prod --label api` or `--label prod,api`). Cannot be combined with a positional selector, `--artifact-bundle`, or `--artifact`. Create and attach labels in the Rubrkit dashboard.
+
+**`--rubric <rubric-id-or-path>`**
+Select a specific evaluation rubric for audits.
+
+**`--all`**
+Pull all available bundles (requires `--yes` for safety). Equivalent to `rubrkit pull all`.
+
+### Quality Gates
+
+**`--fail-under <score>`**
+Exit with code 1 if the quality score falls below this threshold (0-100). Only applies to test/audit/eval.
+
+**`--fail-on <critical|high|medium|low>`**
+Exit with code 1 if any findings at this severity level or higher are detected.
+
+### Execution Mode
+
+**`--local`**
+Run checks locally without making API calls or consuming credits. Cannot be combined with `--remote`.
+
+**`--remote`**
+Submit artifacts to Rubrkit for remote testing. Requires API key. Cannot be combined with `--local`. Default for `test`/`audit`/`eval` without `--local`.
+
+**`--ci`**
+Run in CI mode: non-interactive, structured output, appropriate exit codes.
+
+**`--no-ai`**
+Skip AI-powered analysis (reduces cost, available for some commands).
+
+**`--no-cache`**
+Bypass the per-account audit result cache and force a fresh run, even when an identical prior result exists. Applies to `audit` (and `test`, which composes an audit). The fresh result becomes the new cached result for those inputs. Without this flag, identical inputs return the stored result instantly at no credit cost.
+
+### Pull-Specific Flags
+
+**`--agent <auto|codex|claude|generic>`**
+Target agent type for placement decisions. `auto` detects from project structure.
+
+**`--destination <path>`**
+Target directory for pulling artifacts. Defaults to current directory.
+
+**`--force`**
+Overwrite protected local changes. By default, `pull` stops if existing files differ from tracked versions.
+
+**`--prune`**
+Remove manifest-tracked files that are no longer selected.
+
+**`--update-only`**
+Only update files already tracked in `.rubrkit/manifest.json`. Skip new files.
+
+**`--yes` or `-y`**
+Skip interactive confirmation for unambiguous pulls.
+
+### Testing & Debugging
+
+**`--dry-run`**
+Preview planned actions without changing files or making API calls. Useful with `--remote` to inspect request payloads.
+
+**`--watch`**
+Re-run checks when local files change (development mode).
+
+**`--changed`**
+Only test files changed since last run.
+
+### Configuration
+
+**`--config <path>`**
+Load options from a JSON config file. API keys are not allowed in config files; use environment variables instead.
+
+**`--api-url <url>`**
+Override the Rubrkit API endpoint. Defaults to `RUBRKIT_API_URL` env var or `https://rubrkit.com/api/v1`.
+
+**`--api-key <key>`**
+Provide the API key explicitly (use `RUBRKIT_API_KEY` env var for security).
+
+### Help
+
+**`--help` or `-h`**
+Show command help.
+
+**`--version` or `-v`**
+Show CLI version.
+
+---
+
+## Authentication
+
+Rubrkit requires API keys for `pull`, `test`, `audit`, `eval`, and `report` commands.
+
+**Set the key via environment variable (recommended):**
+```bash
+export RUBRKIT_API_KEY=your_api_key_here
+rubrkit pull all --yes
+```
+
+**Or pass it directly (avoid in scripts/CI):**
+```bash
+rubrkit pull all --yes --api-key your_api_key_here
+```
+
+**API key scopes:**
+- **Pull:** `artifacts:pull`
+- **Test/Audit/Eval/Report:** `artifact_bundles:write`, `files:write`, `audits:run`, `jobs:read`, `credits:read`
+
+Keys are never saved to `.rubrkit/manifest.json`, logs, or client bundles.
+
+---
+
+## Artifact Placement
+
+Artifacts are placed based on your agent type (`--agent auto|codex|claude|generic`):
+
+| Agent | Primary Artifact | Supporting Files | Auto-Detect |
+|-------|------------------|------------------|------------|
+| **codex** | `AGENTS.md` | `.rubrkit/artifacts/<bundle>/` | Looks for `AGENTS.md` |
+| **claude** | `CLAUDE.md` (or `.claude/CLAUDE.md` if exists) | `.rubrkit/artifacts/<bundle>/` | Looks for `CLAUDE.md` or `.claude/` |
+| **generic** | Preserves original paths | `.rubrkit/artifacts/<bundle>/` | Default fallback |
+| **auto** | Detects from project structure | `.rubrkit/artifacts/<bundle>/` | Recommended for new projects |
+
+Pull creates a `.rubrkit/manifest.json` to track managed files. Existing files are protected unless they match the manifest or you use `--force`.
+
+---
+
+## Exit Codes
+
+- **0:** Success. Checks passed or command completed without errors.
+- **1:** Validation/quality checks failed (threshold not met, findings detected, etc.).
+- **2:** Invalid CLI usage or configuration error.
+- **3:** Authentication or authorization failure (missing or invalid API key).
+- **4:** Rate limited, usage blocked, or circuit breaker active.
+- **5:** Network, API, or server failure.
+
+---
+
+## Examples
+
+### Pull artifacts
+```bash
+# Interactive: prompts for bundle and artifact selection
+rubrkit pull
+
+# Pull all bundles non-interactively
+RUBRKIT_API_KEY=my_key rubrkit pull all --yes
+
+# Pull one bundle to a custom location
+RUBRKIT_API_KEY=my_key rubrkit pull team-agent-kit --destination ./my-agents
+
+# Preview without writing
+RUBRKIT_API_KEY=my_key rubrkit pull all --yes --dry-run
+
+# Update only already-tracked files, skip new ones
+RUBRKIT_API_KEY=my_key rubrkit pull --update-only --yes
+
+# Pull every bundle carrying a label (OR across multiple labels)
+RUBRKIT_API_KEY=my_key rubrkit pull --label prod --label api --yes
+```
+
+### Validate artifacts locally
+```bash
+# Check syntax
+rubrkit validate docs/example.rubr_flow
+
+# Batch validate with glob
+rubrkit validate "prompts/**/*.md" --format json
+```
+
+### Test artifacts
+```bash
+# Local validation (no API, no cost)
+rubrkit test "prompts/**/*.md" --local --format json
+
+# Remote test with scoring (requires API key)
+RUBRKIT_API_KEY=my_key rubrkit test team-agent-kit --remote
+
+# Remote with quality gates
+RUBRKIT_API_KEY=my_key rubrkit test team-agent-kit --remote --fail-under 80
+```
+
+### Audit with CI output
+```bash
+# Generate JUnit XML for GitHub Actions / GitLab CI
+RUBRKIT_API_KEY=my_key rubrkit audit team-agent-kit \
+  --format junit \
+  --output test-results.xml \
+  --ci \
+  --fail-under 85
+```
+
+### Postinstall hook
+```json
+{
+  "scripts": {
+    "postinstall": "rubrkit pull all --yes"
+  }
+}
+```
+
+---
+
+## SDK
+
+Import the Rubrkit SDK for programmatic use:
+
+```js
+import { Rubrkit } from 'rubrkit';
+
+const client = new Rubrkit({ apiKey: process.env.RUBRKIT_API_KEY });
+
+// Test an artifact bundle
+const started = await client.artifacts.test({ artifactBundleId: 'team-agent-kit' });
+const job = await client.jobs.wait(started.jobId);
+
+// Get job results
+const results = await client.jobs.get(job.jobId);
+console.log(results.score, results.findings);
+```
+
+---
+
+## Configuration File
+
+Create `.rubrkit/config.json` (or pass `--config path/to/config.json`):
+
+```json
+{
+  "destination": ".",
+  "agent": "claude",
+  "format": "json",
+  "failUnder": 85,
+  "failOn": "high"
+}
+```
+
+**Note:** API keys are not allowed in config files. Always use `RUBRKIT_API_KEY` environment variable for security.
+
+---
+
+## Common Patterns
+
+### CI/CD Integration
+```bash
+# GitHub Actions
+- name: Pull Rubrkit artifacts
+  env:
+    RUBRKIT_API_KEY: ${{ secrets.RUBRKIT_API_KEY }}
+  run: npx rubrkit pull all --yes
+
+- name: Audit artifacts
+  env:
+    RUBRKIT_API_KEY: ${{ secrets.RUBRKIT_API_KEY }}
+  run: npx rubrkit audit team-agent-kit --ci --format junit --output results.xml --fail-under 80
+```
+
+### Development Workflow
+```bash
+# Watch for changes and re-validate
+rubrkit test "prompts/**/*.md" --local --watch
+
+# Or in package.json
+"scripts": {
+  "validate": "rubrkit validate docs/CLAUDE.md",
+  "test": "rubrkit test docs/CLAUDE.md --local"
+}
+```
+
+---
+
+## Troubleshooting
+
+**Missing API key:** Set `RUBRKIT_API_KEY` or pass `--api-key` for commands requiring authentication.
+
+**Local file conflicts:** Use `--dry-run` to preview, then `--force` to overwrite, or manually resolve and retry.
+
+**Slow remote tests:** Use `--local` for quick validation, `--remote --no-ai` to skip expensive analysis.
+
+**Rate limited:** Wait and retry. Consider using `--ci` to reduce request overhead in high-volume scenarios.

package/package.json

@@ -1,28 +1,28 @@
-{
-  "name": "rubrkit",
-  "version": "0.1.0",
-  "type": "module",
-  "description": "Rubrkit CLI for pulling artifact bundles into local agent projects.",
-  "bin": {
-    "rubrkit": "./bin/rubrkit.js"
-  },
-  "exports": {
-    ".": {
-      "types": "./src/index.d.ts",
-      "default": "./src/sdk.js"
-    }
-  },
-  "types": "./src/index.d.ts",
-  "files": [
-    "bin",
-    "src",
-    "README.md",
-    "package.json"
-  ],
-  "engines": {
-    "node": ">=22"
-  },
-  "scripts": {
-    "test": "node --test \"test/**/*.test.js\""
-  }
-}
+{
+  "name": "rubrkit",
+  "version": "0.3.0",
+  "type": "module",
+  "description": "Rubrkit CLI for pulling artifact bundles into local agent projects.",
+  "bin": {
+    "rubrkit": "./bin/rubrkit.js"
+  },
+  "exports": {
+    ".": {
+      "types": "./src/index.d.ts",
+      "default": "./src/sdk.js"
+    }
+  },
+  "types": "./src/index.d.ts",
+  "files": [
+    "bin",
+    "src",
+    "README.md",
+    "package.json"
+  ],
+  "engines": {
+    "node": ">=22"
+  },
+  "scripts": {
+    "test": "node --test \"test/**/*.test.js\""
+  }
+}