tether-sdk 0.2.4__tar.gz → 0.2.6__tar.gz

{tether_sdk-0.2.4 → tether_sdk-0.2.6}/AGENTS.md

@@ -45,7 +45,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
 - Sessions do **not** terminate simply because the model emitted a text response without a tool call (small 9B models frequently "think out loud" before acting).
 - A task only completes when the agent explicitly calls `act("done", stdin="<final deliverable>")` (or `act("done '<short answer>'")`), pauses for human/parent input via `act("done --ask", stdin="<question>")` (transitioning the session to `WAITING_INPUT` and releasing the worker process until resumed via `send`), or hits `max_steps`.
 - If the model emits plain text without calling `act`, the harness appends a brief deterministic nudge reminding it to either execute a command or call `done`.
-- After 6 consecutive read-only/verification steps following a change, the harness appends a one-shot `[NUDGE]` to that step's observation reminding the model to call `done` if the task is complete (0 static prompt tokens, #74).
+- After 6 consecutive read-only/verification steps following a change, the harness appends a one-shot `[NUDGE]` to that step's observation reminding the model to call `done` if the task is complete (0 static prompt tokens, #74). Before any change, 12 read-only steps in a row (then every 8 more) append a one-line `[NUDGE]` to make the edit or call `done` (#127). Seven reads of one file in a row with no change are a `cycle_stall`, and 4 stall warnings without a change end the session `FAILED` as `Stalled` (#92); `tether stop <id>` terminates a detached worker (#89).
 - When `verify` gate commands are configured (via `Harness(verify=[...])`, CLI `--verify`, or Skill YAML `verify: [...]`), `Harness.execute_session` deterministically executes each command inside the session's `SandboxPolicy` before allowing `done` to transition to `COMPLETED` (consuming 0 static prompt tokens until a gate fails). A gate that errors instead of failing (sandbox violation, exit 124/126/127) ends the session `FAILED` at once as `gate_error` ("gate misconfigured"), without counting toward the consecutive-failure abort.
 
 ### Rule 4: Progressive `--help` (JIT Schema Disclosure)
@@ -62,7 +62,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
   ```text
   [obs_3 folded: `web fetch https://...` (3,420 chars) | use `mem recall obs_3` to expand]
   ```
-- Exception: the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The 2,400-char head+tail elision still applies, and the hard context budget remains #81. The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command").
+- Exception: the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The 2,400-char head+tail elision still applies, and under `max_context_tokens` pressure it is folded like any other (oldest first). The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command").
 - History is replayed as structured `tool_calls` + `tool` results (providers with `structured_history = True`; textual `act('...')` form only as fallback), with `stdin` replayed as a real 400-char prefix, never a `<N chars>` placeholder. A call and its result are trimmed together.
 
 ### Rule 6: Deterministic Security Envelope (Never Trust the LLM for Safety)
@@ -71,7 +71,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
   - All `edit` and `bash` paths are canonicalized (`Path.resolve()`) and confined to `workspace_root`.
   - Sensitive files/directories (`.env*`, `.git/config`, `.git/hooks`, `~/.ssh`, `.tether/`) are deterministically protected from mutation or read exfiltration.
   - `bash` subprocesses get an allowlisted environment only (secrets with any name are dropped; opt-in via `TETHER_ENV_PASSTHROUGH`). `web fetch` blocks loopback/private/metadata addresses (SSRF guard). `SandboxPolicy(network=False)` / `--no-network` adds `--unshare-net` (fails closed without `bwrap`; children can only narrow it).
-  - On Linux, `bash` automatically uses **`bwrap` (Bubblewrap) / `unshare`** when available (read-only `/`, hidden `/home` and `/var/tmp` tmpfs, `/run` unix sockets such as `docker.sock` masked, read-write bind on `workspace_root` with secret-looking files at any depth (`.env*`, `*.pem`, `*.key`, ...), `.tether/`, `.git/config` and `.git/hooks` masked at the mount level, `PR_SET_NO_NEW_PRIVS`), falling back to a strict deterministic command allowlist/denylist (`rm -rf /`, `sudo`, `dd`, `mkfs`, `chmod 777`, etc. blocked).
+  - On Linux, `bash` automatically uses **`bwrap` (Bubblewrap) / `unshare`** when available (read-only `/`, hidden `/home` and `/var/tmp` tmpfs (existing `PATH` dirs under them, e.g. `~/.local/bin`, are re-bound read-only so `uv`/`node` still run, #76), `/run` unix sockets such as `docker.sock` masked, read-write bind on `workspace_root` with secret-looking files at any depth (`.env*`, `*.pem`, `*.key`, ...), `.tether/`, `.git/config` and `.git/hooks` masked at the mount level, `PR_SET_NO_NEW_PRIVS`), falling back to a strict deterministic command allowlist/denylist (path, `..` and `.env` text rules run only in this fallback, the mounts enforce them under `bwrap`, #125; `.env.example`-style templates are not secrets) (`rm -rf /`, `sudo`, `dd`, `mkfs`, `chmod 777`, etc. blocked).
   - Sub-agents spawned via `agent spawn` strictly inherit or narrow their parent session's `SandboxPolicy` (no privilege escalation).
 
 ### Rule 7: Zero Framework Bloat
@@ -85,7 +85,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
 ```text
 tether/
 ├── __init__.py          # Public SDK exports: Harness, Pipeline, Session, Skill, SandboxPolicy, tool
-├── cli.py               # Headless-capable CLI: `tether run|pipeline|status|read|query|send|list|wait|stats|retro|events|misses|mcp` (`--json` + documented exit codes)
+├── cli.py               # Headless-capable CLI: `tether run|pipeline|status|read|query|send|list|wait|stop|stats|retro|events|misses|mcp` (`--json` + documented exit codes)
 ├── harness.py           # Control Plane: bounded agent loop, pipelines, SQLite inbox, worker spawn
 ├── pipeline.py          # Workflow Plane: declarative multi-stage YAML pipelines & model routing
 ├── api.py               # Agent-facing JSON session view, exit codes, events (`--json`)
@@ -166,6 +166,7 @@ tether/
 - If no provider prefix is given, defaults to `ollama/<model>`.
 - Custom SDK transports (e.g., internal corporate gateways, custom local runners, or test harnesses) can be injected via `Harness(provider=...)` or registered under a custom `<prefix>/<model>` via `register_provider(prefix, factory)`.
 - Declarative providers: `~/.tether/providers.yaml` (override with `TETHER_PROVIDERS`, never read from the workspace) maps a model-id prefix to a `kind` (`openai`, `ollama`, `anthropic` or `command`); `claude-cli/sonnet` uses the entry `claude-cli`. The `command` kind runs `argv` directly (never through a shell) with `{model}`, `{system}` and `{conversation}` placeholders, and reads the reply and usage from JSON paths. Built-in prefixes (`ollama`, `openai`, `anthropic`, `claude`) cannot be shadowed, keys are env var names (never values), and an invalid file raises `ValueError: providers.yaml: ...`. Implementation in `tether/providers/declarative.py`; docs in `docs/reference/providers.md`.
+- `providers.yaml` entries may set `params` (request-body parameters for HTTP kinds, overridable per call via `--param`, `Harness(params=)`, MCP `params`, stored in `sandbox_json`, #112) and `rpm` (requests per minute cap for HTTP kinds, sliding window shared across processes via a lock file, #110).
 - Environment variables:
   - `OLLAMA_HOST` (default `http://localhost:11434`); `TETHER_OLLAMA_FALLBACK=1` opts in to substituting an installed model when the requested one 404s (default: fail loudly with the installed list)
   - `OPENAI_API_KEY`, `OPENAI_BASE_URL` (default `https://api.openai.com/v1` — works with OpenRouter, vLLM, Groq, etc.)
@@ -204,7 +205,7 @@ uv run mypy tether/
 2. **Async-First Design:**
    - All model calls and I/O-bound tool executions must be `async`/`await` native, while seamlessly wrapping synchronous user `@tool` functions.
 3. **Graceful Degradation on 9B Hallucinations:**
-   - Small models occasionally emit `cmd` with minor quirks (e.g., leading `$`, wrapping backticks, or calling a tool name directly instead of `act`). The multiplexer must normalize trivial syntax quirks deterministically before failing. This includes unescaping literal `\n`/`\t` in one-line `stdin` for `edit write`/`edit replace` and `done` (never `bash`), and turning a trailing shell heredoc in `cmd` (`edit write f <<'EOF' ... EOF`) into `stdin` (never for `bash`).
+   - Small models occasionally emit `cmd` with minor quirks (e.g., leading `$`, wrapping backticks, or calling a tool name directly instead of `act`). The multiplexer must normalize trivial syntax quirks deterministically before failing. This includes unescaping literal `\n`/`\t` in one-line `stdin` for `edit write`/`edit replace` and `done` (never `bash`), turning a trailing shell heredoc in `cmd` (`edit write f <<'EOF' ... EOF`) into `stdin` (never for `bash`), unwrapping `bash -c/-lc '<cmd>'` and `sh -c` wrappers, running an unknown first word through the enabled `bash` tool (never for disabled built-ins), registering extra quirk fixers with `register_normalizer`, mapping a native tool call named other than `act` to `<name> --k 'v'` so the JIT usage answers it, inferring the `edit` verb (`edit <path>` -> view/replace/write) and accepting `apply_patch` (`*** Begin Patch` or unified-diff patches, also when a diff arrives as `edit <path>` stdin) as an `edit` verb.
 4. **Golden traces:** behaviour the model sees (framing, folding, nudges, tool output) is locked by `tests/golden/*.snap.json`. A PR that changes it runs `UPDATE_GOLDEN=1 pytest tests/test_golden.py` and the snapshot diff is part of the review; unrelated traces must not change. See `docs/testing.md`.
 5. **E2E scenarios:** every feature or bug fix ships with a scenario in `e2e/` (real CLI, real ollama behind a fault-injecting proxy, local only, `TETHER_E2E=1 uv run pytest e2e`). Assert on what tether sends and records, never on the model's skill; script the reply or inject the fault so the test does not depend on model potency. **Before merging any PR, run `TETHER_E2E=1 uv run pytest e2e` locally and put the result in the PR description (CI cannot run it). Every failure must be mitigated before merge: fix the code, or fix a wrong scenario, or record in the PR why it is acceptable and what follows up. A PR is not merged with an unexplained red scenario.** See `docs/testing.md`.
 6. **Documentation Integrity:**

{tether_sdk-0.2.4 → tether_sdk-0.2.6}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tether-sdk
-Version: 0.2.4
+Version: 0.2.6
 Summary: General-purpose, headless-first LLM-Native micro-harness for small language models
 Project-URL: Homepage, https://github.com/igormcsouza/tether
 Project-URL: Repository, https://github.com/igormcsouza/tether
@@ -265,8 +265,10 @@ $ tether stats
 # 7. Post-mortem: distill friction into <=2 short rules (diff; --apply writes ## Learned Rules)
 $ tether retro ses_8f92a1 [--apply]
 
-# 8. Call Tether from another agent: JSON output, exit codes (0 ok, 2 needs input, 3 gate, 4 max steps, 5 error)
+# 8. Call Tether from another agent: JSON output, exit codes (0 ok, 2 needs input, 3 gate, 4 max steps or stalled, 5 error)
 $ tether run --json --output-schema out.json "goal"
+$ tether stop ses_8f92a1                                 # terminate a running detached session
+$ tether run --param temperature=0 --param think=false "goal"   # per-call model parameters
 $ claude mcp add tether -- tether mcp --workspace "$PWD"   # MCP server (needs: pip install "tether-sdk[mcp]")
 ```
 

{tether_sdk-0.2.4 → tether_sdk-0.2.6}/README.md

@@ -244,8 +244,10 @@ $ tether stats
 # 7. Post-mortem: distill friction into <=2 short rules (diff; --apply writes ## Learned Rules)
 $ tether retro ses_8f92a1 [--apply]
 
-# 8. Call Tether from another agent: JSON output, exit codes (0 ok, 2 needs input, 3 gate, 4 max steps, 5 error)
+# 8. Call Tether from another agent: JSON output, exit codes (0 ok, 2 needs input, 3 gate, 4 max steps or stalled, 5 error)
 $ tether run --json --output-schema out.json "goal"
+$ tether stop ses_8f92a1                                 # terminate a running detached session
+$ tether run --param temperature=0 --param think=false "goal"   # per-call model parameters
 $ claude mcp add tether -- tether mcp --workspace "$PWD"   # MCP server (needs: pip install "tether-sdk[mcp]")
 ```
 

tether_sdk-0.2.6/bench/README.md

@@ -0,0 +1,37 @@
+# bench: model-in-the-loop checks (developers only)
+
+Not part of the `tether` package, not run in CI. Use it to onboard a new model or to check a
+release for regressions that deterministic tests cannot see (unit, golden and e2e tests stay the
+gate; see `docs/testing.md`).
+
+Nine tasks in `tasks/<name>/` (workspace in `ws/`, goal in `goal.txt`, hidden tests in `hidden/`):
+`focus` (one exact edit in a distractor-heavy repo, scope-checked), `plan` (an implementation plan, checked for named files, failure handling and tests), `codereview` (eight seeded bugs to find, no edits), `code-simple`, `code-medium`, `code-hard` (graded by hidden unit tests; editing the visible tests
+scores 0), `qa`, `content`, `review` (graded by constraint counts and keyword checks, so they are
+a smoke test for quality, not a judge).
+
+Three runners, so a gap can be attributed:
+
+| runner | what it is |
+|---|---|
+| `tether` | the CLI under test (`--tether-bin` picks a version or a venv) |
+| `naive` | `naive_agent.py`, a plain OpenAI-compatible tool loop (bash/read/write) with the same model |
+| `claude` | Claude Code in print mode, as a frontier baseline (`--models cc/sonnet`) |
+
+```bash
+export NVIDIA_API_KEY=...            # only for nvidia/* models; OLLAMA_HOST defaults to :11435
+uv run python bench/run.py --runners tether,naive --models ollama/qwen3.5:9b \
+    --tasks code-simple,qa --trials 2 --out /tmp/bench
+uv run python bench/run.py --runners tether --tether-bin /path/to/venv/bin/tether \
+    --models ollama/qwen3.5:9b --param think=false --out /tmp/bench-v0.2.3
+```
+
+Each run lands in `<out>/<runner>__<model>__<task>__t<k>/` with the workspace, stdout, the
+session DB (`t.db`) and `done.json`; `results.jsonl` collects them. Re-running skips finished
+cells; `--redo-infra` reruns cells that failed on provider errors (429, 503, timeouts).
+
+Read results with the friction events (`syntax_error`, `loop_stall`, `nudge`, ...) next to the
+score: a low score with many events is a harness problem, a low score with none is the model.
+Rate limits (NVIDIA: 40 requests/min) invalidate results silently; set `rpm:` in `providers.yaml`
+and keep parallelism low (`--par nvidia=2`).
+
+Caveats: one run per cell is noisy (about 0.1); the scores are regex and hidden-test based.

tether_sdk-0.2.6/bench/grade.py

@@ -0,0 +1,189 @@
+"""Deterministic graders. grade(task, ws, result_text) -> {"score": 0..1, "pass": bool, "detail": {...}}"""
+import ast
+import hashlib
+import json
+import os
+import re
+import shutil
+import subprocess
+import sys
+import tempfile
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+TASKS = os.path.join(HERE, "tasks")
+
+
+def _tree_hash(path):
+    h = hashlib.sha256()
+    for root, _, files in sorted(os.walk(path)):
+        for f in sorted(files):
+            if f.endswith(".pyc"):
+                continue
+            p = os.path.join(root, f)
+            h.update(os.path.relpath(p, path).encode())
+            h.update(open(p, "rb").read())
+    return h.hexdigest()
+
+
+def _code(task, ws, _result):
+    tmp = tempfile.mkdtemp(prefix="g_")
+    try:
+        shutil.copytree(ws, tmp, dirs_exist_ok=True)
+        orig_tests = os.path.join(TASKS, task, "ws", "tests")
+        tampered = os.path.isdir(orig_tests) and _tree_hash(orig_tests) != _tree_hash(os.path.join(ws, "tests"))
+        shutil.copy(os.path.join(TASKS, task, "hidden", "test_hidden.py"), os.path.join(tmp, "_hidden_test.py"))
+        try:
+            p = subprocess.run([sys.executable, "-m", "unittest", "-v", "_hidden_test"], cwd=tmp, capture_output=True,
+                               text=True, timeout=90)
+            out = p.stderr + p.stdout
+        except subprocess.TimeoutExpired:
+            return {"score": 0.0, "pass": False, "detail": {"error": "hidden tests timed out"}}
+        ok = len(re.findall(r"\.\.\. ok$", out, re.M))
+        bad = len(re.findall(r"\.\.\. (FAIL|ERROR)$", out, re.M))
+        total = ok + bad
+        score = ok / total if total else 0.0
+        if tampered:
+            return {"score": 0.0, "pass": False, "detail": {"tampered_tests": True, "hidden_ok": ok, "hidden_total": total}}
+        return {"score": round(score, 3), "pass": total > 0 and bad == 0, "detail": {"hidden_ok": ok, "hidden_total": total}}
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
+def _json_array(text):
+    m = re.search(r"\[.*\]", text or "", re.S)
+    if not m:
+        return None
+    for parse in (json.loads, ast.literal_eval):
+        try:
+            v = parse(m.group(0))
+            return v if isinstance(v, list) else None
+        except (ValueError, SyntaxError):
+            pass
+    return None
+
+
+def _qa(_task, _ws, result):
+    arr = _json_array(result)
+    checks = [
+        lambda a: "17:25" in a,
+        lambda a: re.search(r"\b391(\.0+)?\b", a) is not None,
+        lambda a: "canberra" in a.lower(),
+        lambda a: re.sub(r"\s", "", a) == "[1,3]",
+        lambda a: re.search(r"\b(10|ten)\b", a.lower()) is not None,
+        lambda a: ("1024" in a or "2**10" in a.replace(" ", "") or "2^10" in a.replace(" ", "")) and re.search(r"\b24\b", a) is not None,
+    ]
+    if arr is None or len(arr) != 6:
+        return {"score": 0.0, "pass": False, "detail": {"error": "not a JSON array of 6"}}
+    got = [bool(c(str(a))) for c, a in zip(checks, arr)]
+    return {"score": round(sum(got) / 6, 3), "pass": all(got), "detail": {"correct": got}}
+
+
+def _content(_task, _ws, result):
+    t = (result or "").strip()
+    words = len(re.findall(r"\S+", t))
+    lines = [ln for ln in t.splitlines() if ln.strip()]
+    sentences = re.split(r"(?<=[.?])\s+", t.replace("\n", " "))
+    last = sentences[-1] if sentences else ""
+    c = {
+        "<=120 words": words <= 120 and words > 0,
+        "3 bullets": sum(1 for ln in lines if ln.startswith("- ")) == 3,
+        "snapshot": "snapshot" in t.lower(),
+        "rollback": "rollback" in t.lower(),
+        "CTA last": "tidepool.dev" in last.lower(),
+        "no !": "!" not in t,
+        "no banned": not re.search(r"revolutionary|game-changing", t, re.I),
+    }
+    return {"score": round(sum(c.values()) / len(c), 3), "pass": all(c.values()), "detail": {"checks": c, "words": words}}
+
+
+FLAWS = {
+    "selection/no-randomisation": r"random|self-select|selection bias|volunteer|chose to install|non-?random|assign",
+    "causal-overreach": r"causal|causation|\bcause|correlat|overreach|overstat",
+    "tiny-sample/generalise": r"small sample|sample size|n\s*=\s*1[12]|23 (student|participant)|under-?power|low power|generali[sz]|single (university|course)|one (course|university)",
+    "multiple-comparisons": r"multiple (comparison|testing|outcome|test)|bonferroni|family-?wise|false[- ]positive|correction|cherry|selective",
+    "number-inconsistency": r"inconsisten|discrepan|mismatch|contradict|12\.4.{0,80}(8\.1|7\.2)|(8\.1|7\.2).{0,80}12\.4",
+    "baseline-gpa-imbalance": r"gpa|baseline|prior (performance|achievement|ability)|confound|not matched|unmatched|equivalen",
+    "unblinded-grader": r"blind|grader|grading|instructor|assessor|rater",
+    "no-effect-size/CI": r"effect size|confidence interval|\bCIs?\b|cohen",
+}
+
+
+def _review(_task, _ws, result):
+    t = result or ""
+    hit = {k: bool(re.search(v, t, re.I)) for k, v in FLAWS.items()}
+    n = len(re.findall(r"^\s*\d+[.)]", t, re.M))
+    return {"score": round(sum(hit.values()) / len(hit), 3), "pass": sum(hit.values()) >= 6,
+            "detail": {"found": [k for k, v in hit.items() if v], "missed": [k for k, v in hit.items() if not v], "items": n}}
+
+
+def _focus(task, ws, result):
+    """_code + scope: the goal allows editing mini/cli.py only; other changes halve the score."""
+    r = _code(task, ws, result)
+    orig = os.path.join(TASKS, task, "ws")
+    touched = []
+    for root, dirs, files in os.walk(orig):
+        for f in files:
+            rel = os.path.relpath(os.path.join(root, f), orig)
+            p = os.path.join(ws, rel)
+            if rel != os.path.join("mini", "cli.py") and (
+                not os.path.exists(p) or open(p, "rb").read() != open(os.path.join(root, f), "rb").read()
+            ):
+                touched.append(rel)
+    r["detail"]["touched_other"] = touched
+    if touched:
+        r["score"], r["pass"] = round(r["score"] / 2, 3), False
+    return r
+
+
+def _steps(t):
+    return len(re.findall(r"^\s*\d+[.)]", t, re.M))
+
+
+def _plan(_task, _ws, result):
+    t = result or ""
+    c = {
+        "6-10 numbered steps": 6 <= _steps(t) <= 10,
+        "names client.py": "client.py" in t,
+        "names config.py": "config.py" in t,
+        "names get/post": bool(re.search(r"\bget\b", t)) and bool(re.search(r"\bpost\b", t)),
+        "backoff+jitter": bool(re.search(r"exponential", t, re.I)) and bool(re.search(r"jitter", t, re.I)),
+        "retry settings": bool(re.search(r"max[_ -]?(retries|attempts)|retry[_ -]?(limit|count)", t, re.I)) and bool(re.search(r"base[_ -]?(delay|backoff)|backoff[_ -]?(base|factor)", t, re.I)),
+        "429 and 5xx": "429" in t and bool(re.search(r"5xx|50\d", t)),
+        "not retried (4xx)": bool(re.search(r"4xx|not retr|never retr|do not retry|don't retry|non-retry", t, re.I)),
+        "Retry-After": bool(re.search(r"retry-after", t, re.I)),
+        "tests without network": bool(re.search(r"mock|fake|stub|patch|monkeypatch|inject", t, re.I)) and bool(re.search(r"test", t, re.I)),
+        "idempotency of post": bool(re.search(r"idempot|post.{0,60}(retry|retri)|(retry|retri).{0,60}post", t, re.I)),
+    }
+    return {"score": round(sum(c.values()) / len(c), 3), "pass": sum(c.values()) >= 9, "detail": {"missed": [k for k, v in c.items() if not v]}}
+
+
+BUGS = {
+    "page off-by-one": r"page.{0,200}(off.by.one|number\s*-\s*1|\(number - 1\)|1-based|zero|skips? the first)|(off.by.one|1-based).{0,200}page",
+    "mutable default": r"mutable default|tags=\[\]|default (argument|arg|list)|shared (between|across)",
+    "float money": r"float.{0,120}(money|price|currenc|rounding|precision|decimal)|decimal",
+    "float equality": r"(==|equal).{0,100}(float|0\.0)|is_free|float.{0,60}(compar|equal)|isclose",
+    "sql injection": r"sql.?inject|parameteri[sz]|placeholder|string concat|bind (var|param)",
+    "file not closed": r"(not|never) clos|resource leak|context manager|with open|leak",
+    "keyerror/missing": r"keyerror|missing (key|item|name)|\.get\(|unknown name|not in (the )?(stock|dict)",
+    "split malformed": r"split.{0,100}(more than|exactly|two|malformed|blank|empty|valueerror)|blank line|malformed|empty line|valueerror",
+}
+
+
+def _codereview(_task, _ws, result):
+    t = result or ""
+    hit = {k: bool(re.search(v, t, re.I | re.S)) for k, v in BUGS.items()}
+    must = ["page off-by-one", "mutable default", "sql injection", "file not closed", "keyerror/missing"]
+    return {"score": round(sum(hit.values()) / len(hit), 3), "pass": all(hit[k] for k in must), "detail": {"missed": [k for k, v in hit.items() if not v], "items": _steps(t)}}
+
+
+def grade(task, ws, result):
+    fn = {"qa": _qa, "content": _content, "review": _review, "focus": _focus, "plan": _plan, "codereview": _codereview}.get(task, _code)
+    try:
+        return fn(task, ws, result)
+    except Exception as e:  # grader bug must never look like a model result
+        return {"score": 0.0, "pass": False, "detail": {"grader_error": repr(e)}}
+
+
+if __name__ == "__main__":
+    task, ws, rf = sys.argv[1:4]
+    print(json.dumps(grade(task, ws, open(rf).read() if os.path.exists(rf) else "")))

tether_sdk-0.2.6/bench/harness-bench/README.md

@@ -0,0 +1,25 @@
+# Harness-Bench adapter (pilot, not yet run end to end)
+
+[Harness-Bench](https://github.com/Qihoo360/harness-bench) ([paper](https://arxiv.org/abs/2605.27922)) scores
+whole harnesses on 274 offline workflow tasks (plus 26 multimodal) with hidden oracles. This directory holds
+the patch that registers Tether as an eighth harness. It is a way to benchmark, not a Tether feature.
+
+`tether-adapter.patch` applies to harness-bench commit `b8d89634c6f445fcc68ceb6744a59bef6cdf4863`:
+
+```bash
+git clone https://github.com/Qihoo360/harness-bench && cd harness-bench
+git apply /path/to/tether/bench/harness-bench/tether-adapter.patch
+uv sync --extra dev
+export PYTHONPATH=$PWD/src:$PWD/tools:$PWD/tools/task_maintenance
+python tools/run_benchmark.py --base-url "$URL" --model "$MODEL" --harness tether \
+    --tasks 001-meeting-summary --workers 1 --run-root /tmp/hb-run
+```
+
+What the patch adds: `docker/harnesses/tether/` (their toolchain image plus `tether-sdk`, an entrypoint that runs
+`tether run --tools bash,edit` with state kept outside the graded workspace), a `tether` `HarnessSpec`, and a
+`providers.yaml` renderer.
+
+Status (2026-10-23): image builds, `tether run` starts inside the trial container and reaches `ProviderError ...
+ConnectError`, because on the machine used containers cannot reach host ports (`host.docker.internal`, where the
+per-trial model proxy listens). The host firewall must allow the docker bridge to the proxy's ports. No task has
+been scored yet.

tether_sdk-0.2.6/bench/harness-bench/tether-adapter.patch

@@ -0,0 +1,134 @@
+diff --git a/docker/harnesses/tether/Dockerfile b/docker/harnesses/tether/Dockerfile
+new file mode 100644
+index 0000000..d9f3e28
+--- /dev/null
++++ b/docker/harnesses/tether/Dockerfile
+@@ -0,0 +1,45 @@
++FROM python:3.12-slim-bookworm@sha256:d50fb7611f86d04a3b0471b46d7557818d88983fc3136726336b2a4c657aa30b AS task-toolchain
++
++ARG PIP_INDEX_URL
++ARG PIP_EXTRA_INDEX_URL
++ARG PIP_TRUSTED_HOST
++COPY docker/openclaw/requirements.lock /tmp/harnessbench-requirements.lock
++RUN python -m pip install --disable-pip-version-check --no-cache-dir \
++      --requirement /tmp/harnessbench-requirements.lock \
++    && rm /tmp/harnessbench-requirements.lock
++
++FROM python:3.12-slim@sha256:57cd7c3a7a273101a6485ba99423ee568157882804b1124b4dd04266317710de
++
++ARG TETHER_VERSION=0.2.5
++ARG PYPI_INDEX_URL=https://pypi.org/simple
++RUN apt-get update \
++    && apt-get install -y --no-install-recommends bash git ripgrep sqlite3 \
++    && rm -rf /var/lib/apt/lists/*
++
++
++
++RUN useradd --create-home --uid 1000 harness
++COPY --from=task-toolchain /usr/local/ /usr/local/
++COPY docker/openclaw/verify_toolchain.py /opt/harnessbench/verify_toolchain.py
++COPY docker/harnesses/tether/entrypoint.sh /opt/harness/bin/run
++RUN ln -sf /usr/local/bin/python3 /usr/bin/python3 \
++    && python /opt/harnessbench/verify_toolchain.py --python-only \
++    && python -m venv /opt/tether \
++    && /opt/tether/bin/python -m pip install --no-cache-dir \
++        --index-url "${PYPI_INDEX_URL}" "tether-sdk==${TETHER_VERSION}" \
++    && chmod 0755 /opt/harness/bin/run
++
++ENV PATH="/opt/harnessbench/frontend-bin:/usr/local/bin:/opt/tether/bin:${PATH}" \
++    NANOBOT_STREAM_IDLE_TIMEOUT_S=7200 \
++    NANOBOT_OPENAI_COMPAT_TIMEOUT_S=7200 \
++    NANOBOT_LLM_TIMEOUT_S=0
++
++# BEGIN NANOBOT LOGIN-SHELL PATH
++RUN printf '%s\n' \
++    'export PATH="/usr/local/bin:/opt/tether/bin:$PATH"' \
++    > /etc/profile.d/harnessbench-path.sh \
++    && chmod 0644 /etc/profile.d/harnessbench-path.sh
++# END NANOBOT LOGIN-SHELL PATH
++
++USER 1000:1000
++ENTRYPOINT ["/opt/harness/bin/run"]
+diff --git a/docker/harnesses/tether/entrypoint.sh b/docker/harnesses/tether/entrypoint.sh
+new file mode 100644
+index 0000000..105cee6
+--- /dev/null
++++ b/docker/harnesses/tether/entrypoint.sh
+@@ -0,0 +1,38 @@
++#!/usr/bin/env bash
++set -euo pipefail
++
++: "${HARNESSBENCH_PROMPT:?}"
++: "${HARNESSBENCH_CONFIG:?}"
++: "${HARNESSBENCH_WORKSPACE:?}"
++: "${HARNESSBENCH_STATE:?}"
++: "${HARNESSBENCH_TRACE:?}"
++
++export HOME="${HARNESSBENCH_STATE}/home"
++mkdir -p "${HOME}" "${HARNESSBENCH_TRACE}"
++runtime_config="${HARNESSBENCH_STATE}/providers.yaml"
++cp "${HARNESSBENCH_CONFIG}" "${runtime_config}"
++# The render step writes `# model: <id>` and `# turns: <n>` on the first lines.
++model="$(sed -n 's/^# model: //p' "${runtime_config}" | head -1)"
++turns="$(sed -n 's/^# turns: //p' "${runtime_config}" | head -1)"
++prompt="$(<"${HARNESSBENCH_PROMPT}")"
++
++export TETHER_PROVIDERS="${runtime_config}"
++export VLLM_API_KEY="${VLLM_API_KEY:-local-vllm-no-auth}"   # placeholder; the trial proxy injects the real key
++export TETHER_DB="${HARNESSBENCH_STATE}/tether.db"   # state stays outside the graded workspace
++export TETHER_ENV_PASSTHROUGH="PATH,NODE_PATH,GIT_CONFIG_GLOBAL"
++
++set +e
++cd "${HARNESSBENCH_WORKSPACE}"
++tether run \
++  --model "harnessbench/${model}" \
++  --tools bash,edit \
++  --max-steps "${turns:-150}" \
++  "${prompt}" \
++  > >(tee "${HARNESSBENCH_TRACE}/events.log") \
++  2> >(tee "${HARNESSBENCH_TRACE}/stderr.log" >&2)
++status=$?
++set -e
++
++[[ -f "${TETHER_DB}" ]] && cp "${TETHER_DB}" "${HARNESSBENCH_TRACE}/tether.db"
++chmod -R a+rX "${HARNESSBENCH_WORKSPACE}" "${HARNESSBENCH_TRACE}" 2>/dev/null || true
++exit "${status}"
+diff --git a/tools/harness_runtime.py b/tools/harness_runtime.py
+index c57ba22..e8763dc 100644
+--- a/tools/harness_runtime.py
++++ b/tools/harness_runtime.py
+@@ -67,6 +67,14 @@ SPECS = {
+         config_name="config.json",
+         docker_context=HARNESS_DOCKER_DIR / "nanobot",
+     ),
++    "tether": HarnessSpec(
++        harness_id="tether",
++        version="0.2.5",
++        image="harnessbench/tether:0.2.5",
++        protocol="chat_completions",
++        config_name="providers.yaml",
++        docker_context=HARNESS_DOCKER_DIR / "tether",
++    ),
+     "goose": HarnessSpec(
+         harness_id="goose",
+         version="1.44.0",
+@@ -291,6 +299,19 @@ def render_harness_config(
+         }
+         return json.dumps(config, ensure_ascii=False, indent=2) + "\n"
+ 
++    if spec.harness_id == "tether":
++        entry = {
++            "kind": "openai",
++            "base_url": base_url,
++            "api_key_env": "VLLM_API_KEY",
++            "context_tokens": context_window,
++            "fold_after": 8,
++            "timeout": 1800,
++            "params": {"temperature": temperature, "max_tokens": max_tokens},
++        }
++        header = f"# model: {model}\n# turns: {max_agent_turns}\n"
++        return header + yaml.safe_dump({"harnessbench": entry}, sort_keys=False)
++
+     if spec.harness_id == "goose":
+         config = {
+             "provider": "openai",

tether_sdk-0.2.6/bench/naive_agent.py

@@ -0,0 +1,92 @@
+"""Control harness: plain OpenAI-compatible tool loop (bash/read_file/write_file), append-only history,
+stop when the model answers without a tool call. Same model, different harness => isolates tether's effect."""
+import argparse
+import json
+import os
+import subprocess
+import time
+
+import httpx
+
+TOOLS = [
+    {"type": "function", "function": {"name": "bash", "description": "Run a shell command in the workspace.",
+     "parameters": {"type": "object", "properties": {"command": {"type": "string"}}, "required": ["command"]}}},
+    {"type": "function", "function": {"name": "read_file", "description": "Read a text file.",
+     "parameters": {"type": "object", "properties": {"path": {"type": "string"}}, "required": ["path"]}}},
+    {"type": "function", "function": {"name": "write_file", "description": "Create or overwrite a file.",
+     "parameters": {"type": "object", "properties": {"path": {"type": "string"}, "content": {"type": "string"}},
+                    "required": ["path", "content"]}}},
+]
+SYSTEM = ("You are a capable assistant working in a project directory with tools. Use the tools to do the task. "
+          "When finished, reply with a plain-text final answer and no tool call.")
+
+
+def run_tool(name, args, ws):
+    try:
+        if name == "bash":
+            p = subprocess.run(args["command"], shell=True, cwd=ws, capture_output=True, text=True, timeout=60)
+            return (p.stdout + p.stderr)[-10000:] or f"(exit {p.returncode})"
+        if name == "read_file":
+            return open(os.path.join(ws, args["path"])).read()[:20000]
+        if name == "write_file":
+            p = os.path.join(ws, args["path"])
+            os.makedirs(os.path.dirname(p) or ws, exist_ok=True)
+            open(p, "w").write(args["content"])
+            return "ok"
+        return f"unknown tool {name}"
+    except Exception as e:
+        return f"error: {e}"
+
+
+def main():
+    ap = argparse.ArgumentParser()
+    ap.add_argument("--base-url", required=True)
+    ap.add_argument("--model", required=True)
+    ap.add_argument("--key-env", default="")
+    ap.add_argument("--ws", required=True)
+    ap.add_argument("--goal", required=True)
+    ap.add_argument("--max-turns", type=int, default=40)
+    ap.add_argument("--result-file", required=True)
+    ap.add_argument("--stats-file", required=True)
+    a = ap.parse_args()
+    hdr = {"Authorization": f"Bearer {os.environ.get(a.key_env, 'x')}"}
+    msgs = [{"role": "system", "content": SYSTEM}, {"role": "user", "content": a.goal}]
+    pt = ct = turns = 0
+    peak = 0
+    final, status = "", "max_turns"
+    t0 = time.time()
+    with httpx.Client(timeout=300) as c:
+        for turns in range(1, a.max_turns + 1):
+            for attempt in range(4):
+                r = c.post(f"{a.base_url}/chat/completions", headers=hdr,
+                           json={"model": a.model, "messages": msgs, "tools": TOOLS, "temperature": 0.2})
+                if r.status_code in (429, 500, 502, 503, 504) and attempt < 3:
+                    time.sleep(2 * (attempt + 1))
+                    continue
+                break
+            if r.status_code != 200:
+                status, final = "http_error", f"HTTP {r.status_code}: {r.text[:300]}"
+                break
+            d = r.json()
+            u = d.get("usage") or {}
+            pt += u.get("prompt_tokens", 0)
+            ct += u.get("completion_tokens", 0)
+            peak = max(peak, u.get("prompt_tokens", 0))
+            m = d["choices"][0]["message"]
+            msgs.append({k: v for k, v in m.items() if k in ("role", "content", "tool_calls") and v is not None})
+            if not m.get("tool_calls"):
+                final, status = m.get("content") or "", "completed"
+                break
+            for tc in m["tool_calls"]:
+                try:
+                    args = json.loads(tc["function"]["arguments"] or "{}")
+                except ValueError:
+                    args = {}
+                msgs.append({"role": "tool", "tool_call_id": tc["id"], "content": run_tool(tc["function"]["name"], args, a.ws)})
+    open(a.result_file, "w").write(final)
+    json.dump({"status": status, "steps": turns, "prompt_tokens": pt, "completion_tokens": ct, "peak_tokens": peak,
+               "wall_s": round(time.time() - t0, 1)}, open(a.stats_file, "w"))
+
+
+if __name__ == "__main__":
+    main()