rlm-harness 1.0.0__tar.gz

rlm_harness-1.0.0/.claude/rules/handoff.md

@@ -0,0 +1,55 @@
+# Context preservation (read before auto-compacting)
+
+`rlm-harness` already routes durable knowledge into its tracked docs — keep using them, and
+when the conversation is about to compact, preserve only what they do NOT already hold:
+
+- **Stable invariants** → the **Invariants** section of `CLAUDE.md`.
+- **Resolved decisions / shipped changes** → `CHANGELOG.md` (under the current version).
+- **Open / proposed work** → the issue tracker, or the CHANGELOG's unreleased section.
+
+So a handoff summary should carry the *in-flight session state* those files miss. Prioritize,
+in order:
+
+1. **Decisions we agreed on this session** that are not yet in CHANGELOG/CLAUDE — design choices
+   ("depth stays 1 — depth>1 recursion is out of scope", "skills are knowledge-only, no script
+   exec", "dataset split by tool name, not `kind=='tool'`"), API-shape calls, and the *reason*.
+   Promote durable ones into CLAUDE.md (invariant) or CHANGELOG.md (change) before they fade.
+2. **Files / symbols changed**, as `path:symbol` one-liners on the *final* shape — e.g.
+   `sub_lm.py:_InterceptedSubLM.__call__ — records the escalation input on the sub_call event`,
+   `dataset.py:export_actions — per-action (planner/tool/sub) records with run reward`. Drop
+   diffs and intermediate revisions.
+3. **Current status.** What passes `uv run pytest` (and the count), what is broken, last command
+   run + result. One paragraph.
+4. **Open suggestions / TODOs** not yet tracked — mark each `proposed`,
+   `accepted-not-done`, or `rejected`, then move the durable ones into the issue tracker.
+5. **In-flight consumer signal.** What the downstream consumer surfaced in the current
+   work and what it needs — the dogfooding driver.
+6. **In-flight user intent + acceptance criteria** for this session. Without it a resumed
+   session drifts.
+
+**Do NOT preserve** (reconstructable / already durable):
+
+- Anything already in `CLAUDE.md`, `CHANGELOG.md`, `README.md`, or `pyproject.toml`.
+- Tool-call transcripts, `grep` output, file listings, full file contents readable from disk.
+- Step-by-step exploration narration; speculative reasoning that led to no decision.
+
+**Format for a handoff summary** (use when compaction is imminent or the user asks for a recap):
+
+```
+## Session state
+- Goal: <one sentence>
+- Consumer driving it: <the downstream consumer | none>
+- Status: <what passes pytest, what doesn't, last command + result>
+
+## Decisions
+- <decision> — <why>   (→ promote to CLAUDE.md invariant / CHANGELOG.md)
+
+## Changed
+- <path:symbol> — <what & why>
+
+## Open
+- [proposed|accepted-not-done|rejected] <item>   (→ issue tracker if durable)
+```
+
+Keep it under ~40 lines. If something fits one of the three tracked docs, put it THERE instead
+of in the summary.

rlm_harness-1.0.0/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,27 @@
+---
+name: Bug report
+about: Report a defect in rlm-harness
+labels: bug
+---
+
+**What happened**
+
+A clear description of the bug.
+
+**Reproduction**
+
+Minimal steps or a code snippet (include the `RLMTask` declaration if relevant).
+
+**Expected**
+
+What you expected instead.
+
+**Environment**
+
+- rlm-harness version / commit:
+- Python version:
+- Interpreter (sandbox `pyodide`/`deno`, or `local`):
+
+**Notes**
+
+Trace excerpts, stack traces, or logs — redact anything sensitive.

rlm_harness-1.0.0/.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,5 @@
+blank_issues_enabled: false
+contact_links:
+  - name: Security vulnerability
+    url: https://github.com/qazbnm456/rlm-harness/blob/main/SECURITY.md
+    about: Report security issues privately per SECURITY.md — do not open a public issue.

rlm_harness-1.0.0/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,22 @@
+---
+name: Feature request / reusable gap
+about: Propose an improvement, or a generic gap the kit should absorb
+labels: enhancement
+---
+
+**The gap or need**
+
+What's missing or awkward. If a downstream consumer surfaced it, describe the friction
+generically — no consumer-specific names or values, per the vendor-neutral rule.
+
+**Is it generic?**
+
+Could this be a reusable mechanic promoted into the kit via the base/wrap split (a generic
+base + guard + factory here; the provider + tracing in the consumer)? Or is it
+consumer-specific (and therefore belongs in the consumer)?
+
+**Proposed shape**
+
+An API sketch, or which module it would live in.
+
+**Alternatives considered**

rlm_harness-1.0.0/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,12 @@
+## What & why
+
+<!-- What does this change, and why? Reference any issue it closes, e.g. "Closes #12". -->
+
+## Checklist
+
+- [ ] `uv run --group dev python -m pytest` is green
+- [ ] `uvx ruff check .` is clean
+- [ ] New behavior has a test
+- [ ] No new top-level `dspy` import in a dspy-free module (`config` / `_retry` / `sandbox` / `tools` / `trace` / `skills` / `replay` / `dataset`)
+- [ ] The `rlm-harness/trace/v1` trace is unchanged, or the change is additive (a new optional field) — `tests/test_contract.py` still green
+- [ ] The public surface stays vendor-neutral (no specific downstream project names or values)

rlm_harness-1.0.0/.github/workflows/ci.yml

@@ -0,0 +1,54 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+# Least privilege: this workflow only checks out code and runs tests/lint — it never writes. Specifying
+# `permissions:` also drops every unlisted scope to `none`, so a compromised dep/action on an untrusted
+# fork PR can't use the token to push, tag, or open issues. The third-party setup-uv action is SHA-pinned
+# (a repointed tag could inject code that runs with the token); GitHub-owned checkout stays on a major tag.
+# CI needs no live model, no vLLM, no network, no Deno: the dspy-bearing tests use a DummyLM or skip when
+# dspy is absent.
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    name: test (py${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11", "3.12", "3.13"]
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
+        with:
+          python-version: ${{ matrix.python-version }}
+          enable-cache: true
+      - name: Run tests
+        # --extra mcp so the MCP-client tests run (they spawn a stdio MCP subprocess) rather than skip
+        run: uv run --group dev --extra mcp python -m pytest -q
+
+  lint:
+    name: ruff
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
+        with:
+          enable-cache: true
+      - name: Lint
+        # PINNED on purpose. `uvx ruff` resolves the latest release at run time, and ruff's DEFAULT
+        # rule set is not a stable contract — 0.16 expanded it far past the pycodestyle/pyflakes
+        # baseline and turned this job red overnight with no code change. Bump it deliberately, with
+        # the resulting fixes in the same commit, rather than letting a release do it for you.
+        run: uvx ruff@0.16.0 check .

rlm_harness-1.0.0/.github/workflows/release.yml

@@ -0,0 +1,52 @@
+name: Release
+
+# Publish to PyPI via Trusted Publishing (OIDC) — NO API token. This is INERT until a
+# GitHub Release is published, and requires one-time setup before the first release:
+#   1. On PyPI, add a Trusted Publisher for project `rlm-harness`:
+#        owner = qazbnm456, repo = rlm-harness, workflow = release.yml, environment = pypi
+#   2. In this repo's settings, create an environment named `pypi`.
+# Then publishing a GitHub Release (from a `vX.Y.Z` tag) builds + uploads automatically.
+# Supply-chain: the two third-party actions are SHA-pinned — setup-uv, and (highest blast radius, it
+# uploads to PyPI) pypi-publish. Bump a SHA and its trailing version comment together. GitHub-owned
+# checkout/*-artifact stay on major tags.
+
+on:
+  release:
+    types: [published]
+
+permissions:
+  contents: read
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install uv
+        uses: astral-sh/setup-uv@d4b2f3b6ecc6e67c4457f6d3e41ec42d3d0fcb86 # v5.4.2
+      - name: Build sdist + wheel
+        run: uv build
+      - uses: actions/upload-artifact@v4
+        with:
+          name: dist
+          path: dist/
+
+  publish:
+    needs: build
+    runs-on: ubuntu-latest
+    # Fork guard. `release: published` is inherited by every fork, so a fork cutting its own release
+    # would fire this job and ask PyPI to publish `rlm-harness` over OIDC. The Trusted Publisher binding
+    # (owner+repo+workflow+environment) would reject it, but that is a rejected publish attempt against
+    # OUR project, plus a probe surface and a confusing red run in someone else's repo. `build` above is
+    # deliberately NOT guarded — it only builds an artifact, and a fork verifying its own build is fine.
+    if: github.repository == 'qazbnm456/rlm-harness'
+    environment: pypi
+    permissions:
+      id-token: write          # OIDC token for PyPI Trusted Publishing
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: dist
+          path: dist/
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # v1.14.0

rlm_harness-1.0.0/.gitignore

@@ -0,0 +1,9 @@
+__pycache__/
+*.py[cod]
+.venv/
+.pytest_cache/
+*.egg-info/
+.env
+dist/
+build/
+traces/