rlenv-audit 0.3.0__tar.gz

rlenv_audit-0.3.0/.claude-plugin/marketplace.json

@@ -0,0 +1,11 @@
+{
+  "name": "rlenv-audit",
+  "owner": { "name": "Vivek", "email": "vivekvkashyap10@gmail.com" },
+  "plugins": [
+    {
+      "name": "env-audit",
+      "source": "./",
+      "description": "Audit a verifiers RL environment before training on it: six agent-run checks, one scorecard. Ask: \"Audit primeintellect/gsm8k\"."
+    }
+  ]
+}

rlenv_audit-0.3.0/.claude-plugin/plugin.json

@@ -0,0 +1,10 @@
+{
+  "name": "env-audit",
+  "description": "Audit a Prime Intellect / verifiers RL environment before training on it. Six judgment-based checks (integrity, problem-statement alignment, reward design, latency, rollout quality, contamination) run by the agent, producing a scorecard with scores and written justifications.",
+  "version": "0.3.0",
+  "author": { "name": "Vivek", "email": "vivekvkashyap10@gmail.com" },
+  "homepage": "https://github.com/vivekvkashyap/RLEnv_audit",
+  "repository": "https://github.com/vivekvkashyap/RLEnv_audit",
+  "license": "MIT",
+  "keywords": ["rl", "reinforcement-learning", "verifiers", "prime-intellect", "audit"]
+}

rlenv_audit-0.3.0/.gitignore

@@ -0,0 +1,32 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+
+# Virtual envs
+.venv/
+venv/
+env/
+
+# Tooling / caches
+.pytest_cache/
+.ruff_cache/
+.mypy_cache/
+
+# RLEnv_audit local cache (downloaded eval sets for the contamination check)
+.rlenv_audit_cache/
+
+# Reports written during runs
+report.json
+*.report.json
+
+# OS
+.DS_Store
+
+# Survey run artifacts
+survey_reports/
+survey.json
+.venv311/

rlenv_audit-0.3.0/COMMITS.md

@@ -0,0 +1,56 @@
+# COMMITS — build plan for RLEnv_audit
+
+Built commit-by-commit; each commit builds and is self-contained. Tick a box when
+the commit lands. Milestone is commit 4: the moment `rlenv-audit run gsm8k --only
+determinism` prints `determinism PASS` on a real Hub env, the architecture is
+proven.
+
+- [x] **1. chore: scaffold** — `pyproject.toml` (verifiers==0.1.14, click, rich,
+  docker), `.gitignore`, README stub, empty package dirs, `DESIGN.md`,
+  `COMMITS.md`. Dedicated venv (`uv`, py3.10); `verifiers==0.1.14` + editable
+  install; `vf-install gsm8k -r` to get the reference env.
+- [x] **2. feat(base+report): data model** — `checks/base.py` (`CheckStatus`,
+  `CheckResult`), `report.py` (`Scorecard.to_terminal` / `to_json`).
+- [x] **3. feat(adapter): verifiers EnvHandle** — load env; normalize
+  RubricGroup-aware rubric / parser / dataset; synchronous `score()` over the
+  async path; `teardown()`. Graceful load failures.
+- [x] **4. feat(determinism) + CLI** — determinism check + `audit()` orchestrator
+  + `cli.py` (`run`, `list-checks`). **Milestone proven on gsm8k.**
+- [x] **5. feat(sandbox+exploits)** — `sandbox.py` Docker isolation + exploits
+  check running cheat patterns inside it; SKIP cleanly if Docker is down.
+- [x] **6. feat(parser)** — parser-robustness check.
+- [x] **7. feat(contamination)** — n-gram overlap vs cached eval sets.
+- [x] **8. feat(latency)** — timing check.
+- [x] **9. feat(distribution)** — GPU/vLLM check; SKIP-degrading when absent.
+- [x] **10. docs+tests** — README, sample scorecard, what each check means; real
+  tests in `tests/`.
+- [x] **11. push** — `git push -u origin main` after confirming the remote +
+  stored credentials work.
+
+## v0.2 — the skill-based rewrite
+
+Everything above built the v0.1 script battery. v0.2 scrapped the deterministic
+checks: the audits are judgment-heavy, so each check became a **skill file**
+(`skills/<check>/SKILL.md`) executed by an agent, leaning on a thin deterministic
+tool layer (`rlenv-audit inspect / score / rollouts / scorecard`). See
+`DESIGN.md` for the current architecture.
+
+- [x] **12. feat!: skill-based audit** — delete `checks/`, `core.py`,
+  `report.py`, `skills.py`; add `tools.py` (inspect / score / rollouts /
+  scorecard), rewrite `cli.py` around the four tools, add the six check skills +
+  the `env-audit` orchestrator, rewrite README/DESIGN, port `scripts/survey.py`
+  to the inspect tool, new `tests/test_tools.py`.
+- [x] **13. feat(distribution): one-command install** — `.claude-plugin/`
+  manifests (repo doubles as a Claude Code plugin marketplace), skills bundled
+  into the wheel, `rlenv-audit install-skills`, self-bootstrapping setup step in
+  the orchestrator skill (pip-installs the tools + `vf-install`s the env).
+
+## Guardrails (apply throughout)
+
+- Library-first: every check callable programmatically; CLI is a thin shell.
+- Keep `verifiers` pinned to `==0.1.14`; never pull `verifiers-rl`/torch/vLLM as
+  hard deps.
+- Fail gracefully: env won't load / no vLLM / Docker down → clear SKIP or error,
+  never a traceback dump.
+- No plugin system, no config framework, no multi-format support. Six checks,
+  one format, that's v0.

rlenv_audit-0.3.0/DESIGN.md

@@ -0,0 +1,140 @@
+# DESIGN — env_audit
+
+> A skill-based auditing system for RL environments. An agent (Claude Code /
+> Codex) runs six judgment-based checks over a `verifiers` environment and emits
+> a scorecard with scores + written justifications.
+
+## 1. Why this exists
+
+RL post-training environments are treated like training data, but nobody tests
+them before burning GPU hours. A broken reward doesn't crash — it silently
+teaches the policy garbage: non-deterministic rewards (noisy gradient),
+exploitable rewards (the policy cheats), rewards that don't discriminate (no
+signal), brittle parsers (correct answers scored wrong), contaminated datasets
+(memorized eval). env_audit catches these first.
+
+## 2. The core decision: skills, not scripts
+
+The six checks are **judgment-heavy and non-deterministic** — "does this reward
+agree with a competent grader?", "is the system prompt missing something?", "does
+this dataset overlap a benchmark?". A deterministic script can only approximate
+these with brittle heuristics. An agent does them well.
+
+So each check is a **skill file** (`skills/<check>/SKILL.md`, SKILL.md style with
+`name` + `description` frontmatter) that an agent reads and executes with its own
+reasoning. The agent leans on a thin **tool layer** (`rlenv-audit ...`) only for
+the parts that must be exact and reproducible:
+
+- **load + introspect** the environment,
+- **score** agent-written completions through the real reward function,
+- run + cache a **shared set of rollouts**,
+- **render** the scorecard.
+
+Tools are pure JSON-in / JSON-out so a skill can shell out and read the result.
+This split keeps judgment in the agent and determinism in the code.
+
+## 3. Target format: `verifiers==0.1.14`
+
+We build against the `verifiers` library (the Hub standard), **pinned to 0.1.14**
+(newer versions drag in torch/vLLM; the target box has old CUDA). The adapter was
+written against the actually-installed source. Facts that shaped it:
+
+| Concern | Reality in 0.1.14 |
+| --- | --- |
+| Loading | `verifiers.load_environment(env_id)` is **synchronous**; it imports a module named `env_id.replace("-","_").split("/")[-1]` and calls its `load_environment()`. The env must be pip-installed (`vf-install`). |
+| Rubric | Often a **`RubricGroup`** whose own `.funcs` is empty — real reward funcs surface via `rubric._get_reward_func_names()`. |
+| Scoring | **Async**, mutates a `state` dict in place. Branch on `rubric.has_group_rewards` (`score_group` vs `score_rollout`). |
+| Reward funcs | May own a `ProcessPoolExecutor` (e.g. `MathRubric`) → must `teardown()`. |
+| Parser | `parser.parse_answer(messages) -> str | None`. |
+| Dataset | HF `Dataset`; rows carry `prompt` (chat messages), `answer`, plus env columns. Many Hub envs are eval-only. |
+
+## 4. The adapter — `EnvHandle` (`adapters/verifiers.py`)
+
+The only code that touches `verifiers`. It normalizes a loaded env into a stable,
+synchronous handle the tools use:
+
+```python
+EnvHandle:
+    load_handle(env_id) -> EnvHandle          # clean EnvLoadError on failure
+    reward_func_names() / reward_sources()    # names + getsource of reward fns (RubricGroup-aware)
+    system_prompt() / module_file()           # env framing + source file
+    dataset(n) / dataset_size()               # normalized rows, train↔eval fallback
+    score(text, prompt, answer, columns) -> (reward, metrics)   # sync over async scoring
+    canonical_answer(answer) / teardown()
+```
+
+`score()` builds a `state` dict, runs the rubric (group-aware), and returns the
+reward — uniform across `Rubric`, `MathRubric`, and `RubricGroup`. All dataset
+columns are threaded through so reward funcs that read custom fields work.
+
+## 5. The tool layer (`tools.py`, `cli.py`)
+
+Four commands, each JSON-in/JSON-out:
+
+- `rlenv-audit inspect <env> -n 20` → `{loaded, env_type, parser_type, module_file,
+  dataset_size, system_prompt, reward_funcs:[{name,weight,source}], sample:[...]}`.
+  Load failures are captured as `{loaded: false, error}` so the integrity check
+  sees them as data. Used by checks 1, 2, 3, 6.
+- `rlenv-audit score <env> completions.json` → scores agent-written
+  `[{prompt_index, label, text}]` through the reward function. Used by check 3.
+- `rlenv-audit rollouts <env> --endpoint --model -n 20 -k 8` (or `--dummy`) →
+  generates 8 rollouts over ~20 tasks **once**, scores + times them, caches to
+  JSON. Checks 4 and 5 share this single cache.
+- `rlenv-audit scorecard results.json` → computes the overall grade + rating
+  (average of the checks that ran; N/A excluded) and renders the table.
+
+## 6. The six checks (`skills/`)
+
+1. **integrity** — does it run and is it shaped right (dataset, reward, conventions,
+   imports). No endpoint.
+2. **problem-alignment** (conditional) — given the user's problem statement, does
+   the env actually test it. **N/A** without a problem statement. No endpoint.
+3. **reward-design** — agent writes ~20 synthetic completions (correct / wrong /
+   edge / format perturbations), scores them, and checks (a) variance &
+   discrimination and (b) agreement between the reward and the agent's own quality
+   judgment. No endpoint.
+4. **latency** — end-to-end rollout timing from the shared cache. Needs an endpoint.
+5. **rollout-quality** — reads actual rollouts and judges the env setup (system
+   prompt, output sensibility, env-caused failure modes). Needs an endpoint.
+6. **contamination** — infer domain → pick benchmarks → check dataset overlap. No
+   endpoint.
+
+The **env-audit** orchestrator skill gathers inputs (env id, optional problem
+statement, optional endpoint), runs the no-endpoint checks, generates the shared
+rollouts once if an endpoint is given, runs the endpoint checks from that cache,
+and assembles the scorecard.
+
+## 7. Scoring model
+
+Each check returns `{name, status, score (0–100|null), justification}`.
+
+- **status**: PASS (~75–100) / WARN (~40–74) / FAIL (~0–39) / **N/A** (documented
+  skip: no problem statement, no endpoint).
+- **rating**: the mean of the numeric scores over checks that actually ran (N/A
+  excluded), mapped to an A–F letter.
+- **grade**: the worst meaningful status (any FAIL → FAIL).
+
+Every score must be grounded in observed evidence — tool output, completions the
+agent wrote, rollouts it read — never a vibe. `REWARD_DESIGN.md` is the rubric the
+reward-design and rollout-quality checks judge against.
+
+## 8. Distribution
+
+One repo, published two ways, so a user never clones it:
+
+- **Claude Code plugin** — `.claude-plugin/marketplace.json` makes the repo a
+  one-plugin marketplace; `/plugin install env-audit@rlenv-audit` ships the
+  skill files straight from GitHub.
+- **PyPI wheel** — the wheel force-includes `skills/` as package data;
+  `rlenv-audit install-skills` (or `uvx rlenv-audit install-skills`) copies them
+  into `~/.claude/skills/`.
+
+The skills carry no other setup burden: the orchestrator's bootstrap step has
+the agent pip-install the tools and `vf-install` the target environment itself
+on first run. The user types one install command once, then just "audit <env>".
+
+## 9. Honest scope
+
+Six checks, one format (`verifiers`), agent-driven. Determinism lives in the
+tools; judgment lives in the skills. No plugin system, no config framework — the
+`adapters/` + `skills/` seams make extension possible without building it now.

rlenv_audit-0.3.0/PKG-INFO

@@ -0,0 +1,150 @@
+Metadata-Version: 2.4
+Name: rlenv-audit
+Version: 0.3.0
+Summary: env_audit — a skill-based auditing system for Prime Intellect `verifiers` RL environments
+Project-URL: Repository, https://github.com/vivekvkashyap/RLEnv_audit
+Author-email: Vivek <vivekvkashyap10@gmail.com>
+License: MIT
+Keywords: audit,prime-intellect,reinforcement-learning,reward-hacking,rl,skills,verifiers
+Requires-Python: >=3.10
+Requires-Dist: click>=8.1
+Requires-Dist: docker>=7.0
+Requires-Dist: openai>=1.0
+Requires-Dist: rich>=13.0
+Requires-Dist: verifiers==0.1.14
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Provides-Extra: gpu
+Requires-Dist: vllm; extra == 'gpu'
+Description-Content-Type: text/markdown
+
+# env_audit
+
+**A skill-based auditing system for RL environments.** Point an agent (Claude
+Code / Codex) at a [`verifiers`](https://github.com/PrimeIntellect-ai/verifiers)
+environment from the Prime Intellect Hub and it runs **six checks** and produces
+a scorecard — *before* you spend GPU hours training on a broken reward.
+
+RL environments are treated like training data, but nobody tests them first. A
+broken reward function doesn't crash — it silently teaches the policy garbage.
+env_audit catches that.
+
+## Why skills, not scripts
+
+The six checks are **judgment-heavy, non-deterministic evaluations** — "does this
+reward agree with a competent grader?", "is the system prompt missing something?",
+"does this dataset overlap a benchmark?". Those are done well by an *agent*, not a
+hard-coded script. So each check is a **skill file** (`skills/<check>/SKILL.md`)
+that the agent reads and executes with its own reasoning, leaning on a small layer
+of deterministic **tools** (`rlenv-audit ...`) for the exact parts: loading the
+env, calling the reward function, running rollouts, rendering the scorecard.
+
+Each check returns a **score (0–100), a status, and a written justification**.
+
+## The six checks
+
+| # | Check | Needs | What it does |
+|---|-------|-------|--------------|
+| 1 | **integrity** | — | Does it even run and is it shaped right: dataset loads & is well-formed, reward present & callable, follows verifiers conventions, no missing fields / broken imports. |
+| 2 | **problem-statement alignment** | *(a problem statement)* | Given what the user says the env is *for*, judge whether the dataset + reward + prompt actually test that. **N/A** if no problem statement is provided. |
+| 3 | **reward design** | — | Stress-tests the reward without the policy: the agent writes ~20 synthetic completions (correct / wrong / edge / format perturbations), scores them through the real reward, and checks (a) the reward varies & discriminates sensibly and (b) each reward matches the agent's own judgment of quality. |
+| 4 | **latency** | model endpoint | How long rollouts take end to end. Reads the shared cached rollouts. |
+| 5 | **rollout quality** | model endpoint | Reads actual rollouts and judges whether the env is set up well in practice — system prompt right, outputs sensible, obvious env-caused failure modes. |
+| 6 | **contamination** | — | Infers the domain, picks the public benchmarks for it, and checks whether dataset instances match/near-match benchmark instances. |
+
+**Shared rollouts (checks 4 & 5).** Both need a model, so env_audit asks once
+which endpoint/model to use (or "dummy"), runs rollouts **once** (8 rollouts over
+~20 samples, scored + timed, cached), and both checks read that single cache.
+Checks 1, 2, 3, 6 need no endpoint. No endpoint → 4 & 5 are **N/A**.
+
+## Quickstart
+
+```bash
+# Install the skills (pick one)
+uvx --from git+https://github.com/vivekvkashyap/RLEnv_audit.git rlenv-audit install-skills
+pip install git+https://github.com/vivekvkashyap/RLEnv_audit.git && rlenv-audit install-skills
+```
+
+Or as a Claude Code plugin, no terminal needed:
+
+```
+/plugin marketplace add vivekvkashyap/RLEnv_audit
+/plugin install env-audit@rlenv-audit
+```
+
+Then point your agent (Claude Code / Codex) at an environment:
+
+> "Audit the `gsm8k` environment." &nbsp; / &nbsp; "Audit `primeintellect/aime2024`
+> — I'm trying to train a competition-math solver — using my vLLM at
+> `http://localhost:8000/v1`."
+
+That's it — everything else is self-bootstrapping: on the first audit the skill
+installs the `rlenv-audit` tools (if missing) and `vf-install`s the environment
+itself. The agent runs the six checks and prints the scorecard:
+
+```
+                               env_audit · gsm8k
+┃ check             ┃ status ┃ score ┃ justification                           ┃
+│ integrity         │ PASS   │    95 │ loads, reward callable, well-formed     │
+│ problem_alignment │ N/A    │     — │ no problem statement provided           │
+│ reward_design     │ PASS   │    88 │ discriminates; matches judgment 18/20   │
+│ latency           │ N/A    │     — │ no endpoint                             │
+│ rollout_quality   │ N/A    │     — │ no endpoint                             │
+│ contamination     │ WARN   │    60 │ 3 near-matches with GSM8K test          │
+overall: WARN   rating: B (81/100)
+```
+
+### From a checkout (development)
+
+```bash
+pip install -e .                    # the rlenv-audit / env-audit tools
+rlenv-audit install-skills          # copy skills/ into ~/.claude/skills
+vf-install primeintellect/gsm8k     # install an environment to audit by hand
+```
+
+> Most Hub envs require **Python 3.11+**; `verifiers==0.1.14` (pinned) also runs
+> on 3.10 for old-CUDA boxes, where you can install the older example envs. The
+> env must be installed into the **same Python environment** as `rlenv-audit` —
+> verifiers loads environments by importing them.
+
+### The tools (what the skills call)
+
+```bash
+rlenv-audit inspect <env> -n 20            # load + introspect -> JSON (reward source, samples, prompt)
+rlenv-audit score <env> completions.json   # score agent-written completions through the reward fn
+rlenv-audit rollouts <env> --endpoint <url> --model <m> -n 20 -k 8   # run+cache shared rollouts
+rlenv-audit rollouts <env> --dummy         # fake rollouts, no endpoint (dry run)
+rlenv-audit scorecard results.json         # render the final scorecard
+```
+
+These are deterministic and JSON-in/JSON-out — usable directly, but normally
+driven by the skills.
+
+## What good looks like
+
+[`REWARD_DESIGN.md`](REWARD_DESIGN.md) is the reference the reward-design and
+rollout-quality checks judge against — determinism, discrimination, baseline
+floor, partial credit, bounds, anti-hacking, parser contract, contamination.
+
+## Layout
+
+```
+skills/                 the six checks + the env-audit orchestrator (SKILL.md each)
+.claude-plugin/         plugin + marketplace manifests (repo doubles as a Claude Code plugin)
+rlenv_audit/
+  adapters/verifiers.py EnvHandle — the only code that touches verifiers
+  tools.py              inspect / score / rollouts / scorecard
+  sandbox.py            Docker isolation (for executing risky completions)
+  cli.py                the rlenv-audit / env-audit CLI (+ install-skills)
+REWARD_DESIGN.md        the design guide the judgment checks cite
+```
+
+## Development
+
+```bash
+pip install -e ".[dev]" && pytest tests/
+```
+
+## License
+
+MIT

rlenv_audit-0.3.0/README.md

@@ -0,0 +1,130 @@
+# env_audit
+
+**A skill-based auditing system for RL environments.** Point an agent (Claude
+Code / Codex) at a [`verifiers`](https://github.com/PrimeIntellect-ai/verifiers)
+environment from the Prime Intellect Hub and it runs **six checks** and produces
+a scorecard — *before* you spend GPU hours training on a broken reward.
+
+RL environments are treated like training data, but nobody tests them first. A
+broken reward function doesn't crash — it silently teaches the policy garbage.
+env_audit catches that.
+
+## Why skills, not scripts
+
+The six checks are **judgment-heavy, non-deterministic evaluations** — "does this
+reward agree with a competent grader?", "is the system prompt missing something?",
+"does this dataset overlap a benchmark?". Those are done well by an *agent*, not a
+hard-coded script. So each check is a **skill file** (`skills/<check>/SKILL.md`)
+that the agent reads and executes with its own reasoning, leaning on a small layer
+of deterministic **tools** (`rlenv-audit ...`) for the exact parts: loading the
+env, calling the reward function, running rollouts, rendering the scorecard.
+
+Each check returns a **score (0–100), a status, and a written justification**.
+
+## The six checks
+
+| # | Check | Needs | What it does |
+|---|-------|-------|--------------|
+| 1 | **integrity** | — | Does it even run and is it shaped right: dataset loads & is well-formed, reward present & callable, follows verifiers conventions, no missing fields / broken imports. |
+| 2 | **problem-statement alignment** | *(a problem statement)* | Given what the user says the env is *for*, judge whether the dataset + reward + prompt actually test that. **N/A** if no problem statement is provided. |
+| 3 | **reward design** | — | Stress-tests the reward without the policy: the agent writes ~20 synthetic completions (correct / wrong / edge / format perturbations), scores them through the real reward, and checks (a) the reward varies & discriminates sensibly and (b) each reward matches the agent's own judgment of quality. |
+| 4 | **latency** | model endpoint | How long rollouts take end to end. Reads the shared cached rollouts. |
+| 5 | **rollout quality** | model endpoint | Reads actual rollouts and judges whether the env is set up well in practice — system prompt right, outputs sensible, obvious env-caused failure modes. |
+| 6 | **contamination** | — | Infers the domain, picks the public benchmarks for it, and checks whether dataset instances match/near-match benchmark instances. |
+
+**Shared rollouts (checks 4 & 5).** Both need a model, so env_audit asks once
+which endpoint/model to use (or "dummy"), runs rollouts **once** (8 rollouts over
+~20 samples, scored + timed, cached), and both checks read that single cache.
+Checks 1, 2, 3, 6 need no endpoint. No endpoint → 4 & 5 are **N/A**.
+
+## Quickstart
+
+```bash
+# Install the skills (pick one)
+uvx --from git+https://github.com/vivekvkashyap/RLEnv_audit.git rlenv-audit install-skills
+pip install git+https://github.com/vivekvkashyap/RLEnv_audit.git && rlenv-audit install-skills
+```
+
+Or as a Claude Code plugin, no terminal needed:
+
+```
+/plugin marketplace add vivekvkashyap/RLEnv_audit
+/plugin install env-audit@rlenv-audit
+```
+
+Then point your agent (Claude Code / Codex) at an environment:
+
+> "Audit the `gsm8k` environment." &nbsp; / &nbsp; "Audit `primeintellect/aime2024`
+> — I'm trying to train a competition-math solver — using my vLLM at
+> `http://localhost:8000/v1`."
+
+That's it — everything else is self-bootstrapping: on the first audit the skill
+installs the `rlenv-audit` tools (if missing) and `vf-install`s the environment
+itself. The agent runs the six checks and prints the scorecard:
+
+```
+                               env_audit · gsm8k
+┃ check             ┃ status ┃ score ┃ justification                           ┃
+│ integrity         │ PASS   │    95 │ loads, reward callable, well-formed     │
+│ problem_alignment │ N/A    │     — │ no problem statement provided           │
+│ reward_design     │ PASS   │    88 │ discriminates; matches judgment 18/20   │
+│ latency           │ N/A    │     — │ no endpoint                             │
+│ rollout_quality   │ N/A    │     — │ no endpoint                             │
+│ contamination     │ WARN   │    60 │ 3 near-matches with GSM8K test          │
+overall: WARN   rating: B (81/100)
+```
+
+### From a checkout (development)
+
+```bash
+pip install -e .                    # the rlenv-audit / env-audit tools
+rlenv-audit install-skills          # copy skills/ into ~/.claude/skills
+vf-install primeintellect/gsm8k     # install an environment to audit by hand
+```
+
+> Most Hub envs require **Python 3.11+**; `verifiers==0.1.14` (pinned) also runs
+> on 3.10 for old-CUDA boxes, where you can install the older example envs. The
+> env must be installed into the **same Python environment** as `rlenv-audit` —
+> verifiers loads environments by importing them.
+
+### The tools (what the skills call)
+
+```bash
+rlenv-audit inspect <env> -n 20            # load + introspect -> JSON (reward source, samples, prompt)
+rlenv-audit score <env> completions.json   # score agent-written completions through the reward fn
+rlenv-audit rollouts <env> --endpoint <url> --model <m> -n 20 -k 8   # run+cache shared rollouts
+rlenv-audit rollouts <env> --dummy         # fake rollouts, no endpoint (dry run)
+rlenv-audit scorecard results.json         # render the final scorecard
+```
+
+These are deterministic and JSON-in/JSON-out — usable directly, but normally
+driven by the skills.
+
+## What good looks like
+
+[`REWARD_DESIGN.md`](REWARD_DESIGN.md) is the reference the reward-design and
+rollout-quality checks judge against — determinism, discrimination, baseline
+floor, partial credit, bounds, anti-hacking, parser contract, contamination.
+
+## Layout
+
+```
+skills/                 the six checks + the env-audit orchestrator (SKILL.md each)
+.claude-plugin/         plugin + marketplace manifests (repo doubles as a Claude Code plugin)
+rlenv_audit/
+  adapters/verifiers.py EnvHandle — the only code that touches verifiers
+  tools.py              inspect / score / rollouts / scorecard
+  sandbox.py            Docker isolation (for executing risky completions)
+  cli.py                the rlenv-audit / env-audit CLI (+ install-skills)
+REWARD_DESIGN.md        the design guide the judgment checks cite
+```
+
+## Development
+
+```bash
+pip install -e ".[dev]" && pytest tests/
+```
+
+## License
+
+MIT

rlenv_audit-0.3.0/REWARD_DESIGN.md

@@ -0,0 +1,115 @@
+# REWARD_DESIGN.md — how to design a good `verifiers` RL environment
+
+This is the reference the `rlenv-audit` checks point at. Every recommendation in
+a scorecard cites a section here. It's a checklist for building an environment
+whose reward is actually a good training signal — written so you can fix a flagged
+issue without guessing what "good" means.
+
+The one-line principle: **the reward must go up if and only if the policy gets
+better at the task.** Everything below is a way that principle breaks.
+
+---
+
+## §determinism — the same completion must always score the same
+
+Score a fixed completion repeatedly; the reward must not move. Sources of
+non-determinism to remove:
+
+- unseeded RNG anywhere in scoring (`random`, `numpy`, sampling a judge);
+- wall-clock or timeout-dependent scoring (a slow machine scores differently);
+- network/API calls whose result varies (rate limits, model drift);
+- LLM judges with `temperature > 0` — pin to `0` and ideally cache.
+
+A reward that varies across identical re-scores injects pure noise into the
+gradient: the policy chases randomness, not skill.
+
+## §discrimination — correct must out-score garbage
+
+A correct answer must score strictly higher than an empty or nonsense
+completion. If it doesn't, the gradient points nowhere. The usual culprit is the
+**parser/matcher rejecting the dataset's own gold answers** — if the env can't
+reward its own answer key, no policy can learn from it. Test: run each row's gold
+`answer` through the verifier and confirm it earns the max reward.
+
+## §baseline-floor — don't pay everyone
+
+If every response (even garbage) earns a constant positive reward — a
+participation, length, or format bonus applied unconditionally — the *relative*
+advantage of solving the task shrinks. Prefer a zero floor. If you keep a format
+reward, weight it near zero and gate it on the answer being present, not on mere
+formatting.
+
+## §partial-credit — graded beats binary on hard tasks
+
+Strict 0-or-1 reward is correct but sparse: on tasks the policy almost never
+solves, the gradient is almost always zero. Where the task allows, give graded
+partial credit (fraction of unit tests passing, edit-distance/LCS similarity,
+sub-goal completion) so there's signal at the frontier.
+
+## §bounds — keep the aggregate in [0, 1]
+
+Keep the summed reward in a known, bounded range (normally `[0, 1]`). Unbounded or
+wildly-scaled rewards complicate advantage normalization and make environments
+incomparable. If you must use another range, document it.
+
+## §weights — sanity-check the weighted sum
+
+- At least one reward function must have a non-zero weight (all-zero weights →
+  reward is 0 by construction → no learning).
+- Negative-weight penalties are easy to get backwards; make sure a penalty can't
+  dominate the positive signal and reward the policy for doing nothing.
+
+## §anti-hacking — assume the policy will cheat
+
+RL policies find every shortcut. A robust verifier:
+
+- never trusts an exit code alone (a `sys.exit(0)` before tests must not pass);
+- keeps expected outputs / test files **out of** the execution working directory
+  (so the solution can't read the answer off disk);
+- re-asserts results *after* the submission runs (so monkeypatching `assert`,
+  overriding builtins, or printing the answer without computing it fails);
+- rejects empty / no-op submissions.
+
+If a no-solution completion earns reward, the policy will learn that shortcut
+instead of the task.
+
+## §parser-contract — parse what models actually emit
+
+The parser must extract the answer from *real* model output, not just the
+canonical format. Be liberal in what you accept: strip surrounding whitespace and
+trailing punctuation, match case-insensitively, take the last occurrence when the
+answer is restated, tolerate reasoning before the answer. And state the required
+format explicitly in the system prompt — if the parser extracts nothing from real
+rollouts, the format contract isn't being communicated.
+
+## §difficulty-curriculum — avoid all-pass / all-fail batches
+
+Generate rollouts with a reference policy and histogram the rewards. An all-zero
+batch (too hard / broken) or all-one batch (trivial) gives zero gradient. Aim for
+a spread of outcomes at the model's current ability — mix difficulties, or filter
+the dataset to the band where the policy sometimes-but-not-always succeeds.
+
+## §contamination — don't train on the eval set
+
+N-gram/embedding-check your training tasks against the benchmarks you'll report
+on (AIME, MATH, GSM8K, HumanEval, LiveCodeBench, …). Overlap means measured
+"improvement" is partly memorization. For an explicitly *eval-only* environment,
+overlap with its own benchmark is expected — just never use it for training.
+
+---
+
+### How the audit checks map to these sections
+
+| Check | Sections it judges against |
+| --- | --- |
+| integrity | §parser-contract, §weights |
+| reward_design | §determinism, §discrimination, §baseline-floor, §partial-credit, §bounds, §weights, §anti-hacking, §parser-contract |
+| rollout_quality | §parser-contract, §difficulty-curriculum |
+| contamination | §contamination |
+
+(problem_alignment and latency judge things outside this guide: the user's stated
+goal and throughput.)
+
+A clean scorecard means none of these failure modes were detected on the slice we
+could measure — not a proof of correctness, but the cheap faults are ruled out
+before you spend GPU hours.