tether-sdk 0.2.5__tar.gz → 0.2.7__tar.gz

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/AGENTS.md

@@ -45,7 +45,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
 - Sessions do **not** terminate simply because the model emitted a text response without a tool call (small 9B models frequently "think out loud" before acting).
 - A task only completes when the agent explicitly calls `act("done", stdin="<final deliverable>")` (or `act("done '<short answer>'")`), pauses for human/parent input via `act("done --ask", stdin="<question>")` (transitioning the session to `WAITING_INPUT` and releasing the worker process until resumed via `send`), or hits `max_steps`.
 - If the model emits plain text without calling `act`, the harness appends a brief deterministic nudge reminding it to either execute a command or call `done`.
-- After 6 consecutive read-only/verification steps following a change, the harness appends a one-shot `[NUDGE]` to that step's observation reminding the model to call `done` if the task is complete (0 static prompt tokens, #74). Seven reads of one file in a row with no change are a `cycle_stall`, and 4 stall warnings without a change end the session `FAILED` as `Stalled` (#92); `tether stop <id>` terminates a detached worker (#89).
+- After 6 consecutive read-only/verification steps following a change, the harness appends a one-shot `[NUDGE]` to that step's observation reminding the model to call `done` if the task is complete (0 static prompt tokens, #74). Before any change, 12 read-only steps in a row (then every 8 more) append a one-line `[NUDGE]` to make the edit or call `done` (#127). Seven reads of one file in a row with no change are a `cycle_stall`, and 4 stall warnings without a change end the session `FAILED` as `Stalled` (#92); `tether stop <id>` terminates a detached worker (#89).
 - When `verify` gate commands are configured (via `Harness(verify=[...])`, CLI `--verify`, or Skill YAML `verify: [...]`), `Harness.execute_session` deterministically executes each command inside the session's `SandboxPolicy` before allowing `done` to transition to `COMPLETED` (consuming 0 static prompt tokens until a gate fails). A gate that errors instead of failing (sandbox violation, exit 124/126/127) ends the session `FAILED` at once as `gate_error` ("gate misconfigured"), without counting toward the consecutive-failure abort.
 
 ### Rule 4: Progressive `--help` (JIT Schema Disclosure)
@@ -62,7 +62,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
   ```text
   [obs_3 folded: `web fetch https://...` (3,420 chars) | use `mem recall obs_3` to expand]
   ```
-- Exception: the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The 2,400-char head+tail elision still applies, and under `max_context_tokens` pressure it is folded like any other (oldest first). The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command").
+- Exception: the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The head+tail elision still applies (an observation may use up to a third of `max_context_tokens`, at least 2,400 chars), and under `max_context_tokens` pressure it is folded like any other (oldest first). The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command").
 - History is replayed as structured `tool_calls` + `tool` results (providers with `structured_history = True`; textual `act('...')` form only as fallback), with `stdin` replayed as a real 400-char prefix, never a `<N chars>` placeholder. A call and its result are trimmed together.
 
 ### Rule 6: Deterministic Security Envelope (Never Trust the LLM for Safety)
@@ -71,7 +71,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
   - All `edit` and `bash` paths are canonicalized (`Path.resolve()`) and confined to `workspace_root`.
   - Sensitive files/directories (`.env*`, `.git/config`, `.git/hooks`, `~/.ssh`, `.tether/`) are deterministically protected from mutation or read exfiltration.
   - `bash` subprocesses get an allowlisted environment only (secrets with any name are dropped; opt-in via `TETHER_ENV_PASSTHROUGH`). `web fetch` blocks loopback/private/metadata addresses (SSRF guard). `SandboxPolicy(network=False)` / `--no-network` adds `--unshare-net` (fails closed without `bwrap`; children can only narrow it).
-  - On Linux, `bash` automatically uses **`bwrap` (Bubblewrap) / `unshare`** when available (read-only `/`, hidden `/home` and `/var/tmp` tmpfs (existing `PATH` dirs under them, e.g. `~/.local/bin`, are re-bound read-only so `uv`/`node` still run, #76), `/run` unix sockets such as `docker.sock` masked, read-write bind on `workspace_root` with secret-looking files at any depth (`.env*`, `*.pem`, `*.key`, ...), `.tether/`, `.git/config` and `.git/hooks` masked at the mount level, `PR_SET_NO_NEW_PRIVS`), falling back to a strict deterministic command allowlist/denylist (`rm -rf /`, `sudo`, `dd`, `mkfs`, `chmod 777`, etc. blocked).
+  - On Linux, `bash` automatically uses **`bwrap` (Bubblewrap) / `unshare`** when available (read-only `/`, hidden `/home` and `/var/tmp` tmpfs (existing `PATH` dirs under them, e.g. `~/.local/bin`, are re-bound read-only so `uv`/`node` still run, #76), `/run` unix sockets such as `docker.sock` masked, read-write bind on `workspace_root` with secret-looking files at any depth (`.env*`, `*.pem`, `*.key`, ...), `.tether/`, `.git/config` and `.git/hooks` masked at the mount level, `PR_SET_NO_NEW_PRIVS`), falling back to a strict deterministic command allowlist/denylist (path, `..` and `.env` text rules run only in this fallback, the mounts enforce them under `bwrap`, #125; `.env.example`-style templates are not secrets) (`rm -rf /`, `sudo`, `dd`, `mkfs`, `chmod 777`, etc. blocked).
   - Sub-agents spawned via `agent spawn` strictly inherit or narrow their parent session's `SandboxPolicy` (no privilege escalation).
 
 ### Rule 7: Zero Framework Bloat

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tether-sdk
-Version: 0.2.5
+Version: 0.2.7
 Summary: General-purpose, headless-first LLM-Native micro-harness for small language models
 Project-URL: Homepage, https://github.com/igormcsouza/tether
 Project-URL: Repository, https://github.com/igormcsouza/tether

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/bench/README.md

@@ -4,11 +4,19 @@ Not part of the `tether` package, not run in CI. Use it to onboard a new model o
 release for regressions that deterministic tests cannot see (unit, golden and e2e tests stay the
 gate; see `docs/testing.md`).
 
-Seven tasks in `tasks/<name>/` (workspace in `ws/`, goal in `goal.txt`, hidden tests in `hidden/`):
-`focus` (one exact edit in a distractor-heavy repo, scope-checked), `code-simple`, `code-medium`, `code-hard` (graded by hidden unit tests; editing the visible tests
+Nine tasks in `tasks/<name>/` (workspace in `ws/`, goal in `goal.txt`, hidden tests in `hidden/`):
+`focus` (one exact edit in a distractor-heavy repo, scope-checked), `plan` (an implementation plan, checked for named files, failure handling and tests), `codereview` (eight seeded bugs to find, no edits), `code-simple`, `code-medium`, `code-hard` (graded by hidden unit tests; editing the visible tests
 scores 0), `qa`, `content`, `review` (graded by constraint counts and keyword checks, so they are
 a smoke test for quality, not a judge).
 
+**Polyglot-Python** is the score we answer to. `--tasks polyglot` runs the 34 Python exercises of Aider's
+[polyglot-benchmark](https://github.com/Aider-AI/polyglot-benchmark) (cloned once into
+`~/.cache/tether-bench/polyglot-benchmark`, or set `POLYGLOT_DIR`); `--tasks polyglot-fast` runs a 12-exercise
+subset that mixes passes, partials and failures. Each task is graded by the exercise's own tests (the test file is
+restored before grading, so editing it cannot pass), `--poly-verify` adds a `--verify` gate on those tests. Then
+`python3 bench/report.py <out>` prints pass rate and mean score with a failure breakdown that keeps provider outages
+(`outage`, `timeout`) apart from harness outcomes (`max_steps`, `stalled`, `wrong_answer`, `worker_error`).
+
 Three runners, so a gap can be attributed:
 
 | runner | what it is |

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/bench/grade.py

@@ -135,10 +135,96 @@ def _focus(task, ws, result):
     return r
 
 
+def _steps(t):
+    return len(re.findall(r"^\s*\d+[.)]", t, re.M))
+
+
+def _plan(_task, _ws, result):
+    t = result or ""
+    c = {
+        "6-10 numbered steps": 6 <= _steps(t) <= 10,
+        "names client.py": "client.py" in t,
+        "names config.py": "config.py" in t,
+        "names get/post": bool(re.search(r"\bget\b", t)) and bool(re.search(r"\bpost\b", t)),
+        "backoff+jitter": bool(re.search(r"exponential", t, re.I)) and bool(re.search(r"jitter", t, re.I)),
+        "retry settings": bool(re.search(r"max[_ -]?(retries|attempts)|retry[_ -]?(limit|count)", t, re.I)) and bool(re.search(r"base[_ -]?(delay|backoff)|backoff[_ -]?(base|factor)", t, re.I)),
+        "429 and 5xx": "429" in t and bool(re.search(r"5xx|50\d", t)),
+        "not retried (4xx)": bool(re.search(r"4xx|not retr|never retr|do not retry|don't retry|non-retry", t, re.I)),
+        "Retry-After": bool(re.search(r"retry-after", t, re.I)),
+        "tests without network": bool(re.search(r"mock|fake|stub|patch|monkeypatch|inject", t, re.I)) and bool(re.search(r"test", t, re.I)),
+        "idempotency of post": bool(re.search(r"idempot|post.{0,60}(retry|retri)|(retry|retri).{0,60}post", t, re.I)),
+    }
+    return {"score": round(sum(c.values()) / len(c), 3), "pass": sum(c.values()) >= 9, "detail": {"missed": [k for k, v in c.items() if not v]}}
+
+
+BUGS = {
+    "page off-by-one": r"page.{0,200}(off.by.one|number\s*-\s*1|\(number - 1\)|1-based|zero|skips? the first)|(off.by.one|1-based).{0,200}page",
+    "mutable default": r"mutable default|tags=\[\]|default (argument|arg|list)|shared (between|across)",
+    "float money": r"float.{0,120}(money|price|currenc|rounding|precision|decimal)|decimal",
+    "float equality": r"(==|equal).{0,100}(float|0\.0)|is_free|float.{0,60}(compar|equal)|isclose",
+    "sql injection": r"sql.?inject|parameteri[sz]|placeholder|string concat|bind (var|param)",
+    "file not closed": r"(not|never) clos|resource leak|context manager|with open|leak",
+    "keyerror/missing": r"keyerror|missing (key|item|name)|\.get\(|unknown name|not in (the )?(stock|dict)",
+    "split malformed": r"split.{0,100}(more than|exactly|two|malformed|blank|empty|valueerror)|blank line|malformed|empty line|valueerror",
+}
+
+
+def _codereview(_task, _ws, result):
+    t = result or ""
+    hit = {k: bool(re.search(v, t, re.I | re.S)) for k, v in BUGS.items()}
+    must = ["page off-by-one", "mutable default", "sql injection", "file not closed", "keyerror/missing"]
+    return {"score": round(sum(hit.values()) / len(hit), 3), "pass": all(hit[k] for k in must), "detail": {"missed": [k for k, v in hit.items() if not v], "items": _steps(t)}}
+
+
+# Fast set (12): easy passes, partials and hard failures from the 0.2.6 baseline; tether's weak spots.
+POLY_FAST = ("affine-cipher bottle-song list-ops paasio dot-dsl food-chain forth go-counting poker pov react "
+             "rest-api").split()
+
+
+def poly_dir(ex=""):
+    """Aider polyglot-benchmark Python exercises (cloned once, never vendored)."""
+    root = os.environ.get("POLYGLOT_DIR") or os.path.expanduser("~/.cache/tether-bench/polyglot-benchmark")
+    if not os.path.isdir(root):
+        subprocess.run(["git", "clone", "--depth", "1", "https://github.com/Aider-AI/polyglot-benchmark", root], check=True)
+    return os.path.join(root, "python", "exercises", "practice", ex)
+
+
+def poly_goal(ex):
+    d = poly_dir(ex)
+    docs = "\n\n".join(open(os.path.join(d, ".docs", f)).read() for f in ("instructions.md", "instructions.append.md")
+                        if os.path.exists(os.path.join(d, ".docs", f)))
+    sol = ex.replace("-", "_") + ".py"
+    return (f"Implement `{sol}` in the current directory so that `python3 -m unittest discover -p '*_test.py'` passes. "
+            f"Do not edit the test file.\n\n{docs}")
+
+
+def _poly(task, ws, _result):
+    ex = task.removeprefix("poly-")
+    tmp = tempfile.mkdtemp(prefix="g_")
+    try:
+        shutil.copytree(ws, tmp, dirs_exist_ok=True)
+        test = ex.replace("-", "_") + "_test.py"
+        shutil.copy(os.path.join(poly_dir(ex), test), os.path.join(tmp, test))  # restore: editing tests cannot pass
+        try:
+            p = subprocess.run([sys.executable, "-m", "unittest", "-v", test[:-3]], cwd=tmp, capture_output=True, text=True, timeout=90)
+        except subprocess.TimeoutExpired:
+            return {"score": 0.0, "pass": False, "detail": {"error": "tests timed out"}}
+        out = p.stderr + p.stdout
+        ok = len(re.findall(r"\.\.\. ok$", out, re.M))
+        bad = len(re.findall(r"\.\.\. (FAIL|ERROR)$", out, re.M))
+        total = ok + bad
+        return {"score": round(ok / total, 3) if total else 0.0, "pass": p.returncode == 0 and total > 0, "detail": {"ok": ok, "total": total}}
+    finally:
+        shutil.rmtree(tmp, ignore_errors=True)
+
+
 def grade(task, ws, result):
-    fn = {"qa": _qa, "content": _content, "review": _review, "focus": _focus}.get(task, _code)
+    orig = task
+    if task.startswith("poly-"):
+        task = "poly"
+    fn = {"poly": _poly, "qa": _qa, "content": _content, "review": _review, "focus": _focus, "plan": _plan, "codereview": _codereview}.get(task, _code)
     try:
-        return fn(task, ws, result)
+        return fn(orig, ws, result)
     except Exception as e:  # grader bug must never look like a model result
         return {"score": 0.0, "pass": False, "detail": {"grader_error": repr(e)}}
 

tether_sdk-0.2.7/bench/harness-bench/README.md

@@ -0,0 +1,25 @@
+# Harness-Bench adapter (pilot, not yet run end to end)
+
+[Harness-Bench](https://github.com/Qihoo360/harness-bench) ([paper](https://arxiv.org/abs/2605.27922)) scores
+whole harnesses on 274 offline workflow tasks (plus 26 multimodal) with hidden oracles. This directory holds
+the patch that registers Tether as an eighth harness. It is a way to benchmark, not a Tether feature.
+
+`tether-adapter.patch` applies to harness-bench commit `b8d89634c6f445fcc68ceb6744a59bef6cdf4863`:
+
+```bash
+git clone https://github.com/Qihoo360/harness-bench && cd harness-bench
+git apply /path/to/tether/bench/harness-bench/tether-adapter.patch
+uv sync --extra dev
+export PYTHONPATH=$PWD/src:$PWD/tools:$PWD/tools/task_maintenance
+python tools/run_benchmark.py --base-url "$URL" --model "$MODEL" --harness tether \
+    --tasks 001-meeting-summary --workers 1 --run-root /tmp/hb-run
+```
+
+What the patch adds: `docker/harnesses/tether/` (their toolchain image plus `tether-sdk`, an entrypoint that runs
+`tether run --tools bash,edit` with state kept outside the graded workspace), a `tether` `HarnessSpec`, and a
+`providers.yaml` renderer.
+
+Status (2026-10-23): image builds, `tether run` starts inside the trial container and reaches `ProviderError ...
+ConnectError`, because on the machine used containers cannot reach host ports (`host.docker.internal`, where the
+per-trial model proxy listens). The host firewall must allow the docker bridge to the proxy's ports. No task has
+been scored yet.

tether_sdk-0.2.7/bench/harness-bench/tether-adapter.patch

@@ -0,0 +1,134 @@
+diff --git a/docker/harnesses/tether/Dockerfile b/docker/harnesses/tether/Dockerfile
+new file mode 100644
+index 0000000..d9f3e28
+--- /dev/null
++++ b/docker/harnesses/tether/Dockerfile
+@@ -0,0 +1,45 @@
++FROM python:3.12-slim-bookworm@sha256:d50fb7611f86d04a3b0471b46d7557818d88983fc3136726336b2a4c657aa30b AS task-toolchain
++
++ARG PIP_INDEX_URL
++ARG PIP_EXTRA_INDEX_URL
++ARG PIP_TRUSTED_HOST
++COPY docker/openclaw/requirements.lock /tmp/harnessbench-requirements.lock
++RUN python -m pip install --disable-pip-version-check --no-cache-dir \
++      --requirement /tmp/harnessbench-requirements.lock \
++    && rm /tmp/harnessbench-requirements.lock
++
++FROM python:3.12-slim@sha256:57cd7c3a7a273101a6485ba99423ee568157882804b1124b4dd04266317710de
++
++ARG TETHER_VERSION=0.2.5
++ARG PYPI_INDEX_URL=https://pypi.org/simple
++RUN apt-get update \
++    && apt-get install -y --no-install-recommends bash git ripgrep sqlite3 \
++    && rm -rf /var/lib/apt/lists/*
++
++
++
++RUN useradd --create-home --uid 1000 harness
++COPY --from=task-toolchain /usr/local/ /usr/local/
++COPY docker/openclaw/verify_toolchain.py /opt/harnessbench/verify_toolchain.py
++COPY docker/harnesses/tether/entrypoint.sh /opt/harness/bin/run
++RUN ln -sf /usr/local/bin/python3 /usr/bin/python3 \
++    && python /opt/harnessbench/verify_toolchain.py --python-only \
++    && python -m venv /opt/tether \
++    && /opt/tether/bin/python -m pip install --no-cache-dir \
++        --index-url "${PYPI_INDEX_URL}" "tether-sdk==${TETHER_VERSION}" \
++    && chmod 0755 /opt/harness/bin/run
++
++ENV PATH="/opt/harnessbench/frontend-bin:/usr/local/bin:/opt/tether/bin:${PATH}" \
++    NANOBOT_STREAM_IDLE_TIMEOUT_S=7200 \
++    NANOBOT_OPENAI_COMPAT_TIMEOUT_S=7200 \
++    NANOBOT_LLM_TIMEOUT_S=0
++
++# BEGIN NANOBOT LOGIN-SHELL PATH
++RUN printf '%s\n' \
++    'export PATH="/usr/local/bin:/opt/tether/bin:$PATH"' \
++    > /etc/profile.d/harnessbench-path.sh \
++    && chmod 0644 /etc/profile.d/harnessbench-path.sh
++# END NANOBOT LOGIN-SHELL PATH
++
++USER 1000:1000
++ENTRYPOINT ["/opt/harness/bin/run"]
+diff --git a/docker/harnesses/tether/entrypoint.sh b/docker/harnesses/tether/entrypoint.sh
+new file mode 100644
+index 0000000..105cee6
+--- /dev/null
++++ b/docker/harnesses/tether/entrypoint.sh
+@@ -0,0 +1,38 @@
++#!/usr/bin/env bash
++set -euo pipefail
++
++: "${HARNESSBENCH_PROMPT:?}"
++: "${HARNESSBENCH_CONFIG:?}"
++: "${HARNESSBENCH_WORKSPACE:?}"
++: "${HARNESSBENCH_STATE:?}"
++: "${HARNESSBENCH_TRACE:?}"
++
++export HOME="${HARNESSBENCH_STATE}/home"
++mkdir -p "${HOME}" "${HARNESSBENCH_TRACE}"
++runtime_config="${HARNESSBENCH_STATE}/providers.yaml"
++cp "${HARNESSBENCH_CONFIG}" "${runtime_config}"
++# The render step writes `# model: <id>` and `# turns: <n>` on the first lines.
++model="$(sed -n 's/^# model: //p' "${runtime_config}" | head -1)"
++turns="$(sed -n 's/^# turns: //p' "${runtime_config}" | head -1)"
++prompt="$(<"${HARNESSBENCH_PROMPT}")"
++
++export TETHER_PROVIDERS="${runtime_config}"
++export VLLM_API_KEY="${VLLM_API_KEY:-local-vllm-no-auth}"   # placeholder; the trial proxy injects the real key
++export TETHER_DB="${HARNESSBENCH_STATE}/tether.db"   # state stays outside the graded workspace
++export TETHER_ENV_PASSTHROUGH="PATH,NODE_PATH,GIT_CONFIG_GLOBAL"
++
++set +e
++cd "${HARNESSBENCH_WORKSPACE}"
++tether run \
++  --model "harnessbench/${model}" \
++  --tools bash,edit \
++  --max-steps "${turns:-150}" \
++  "${prompt}" \
++  > >(tee "${HARNESSBENCH_TRACE}/events.log") \
++  2> >(tee "${HARNESSBENCH_TRACE}/stderr.log" >&2)
++status=$?
++set -e
++
++[[ -f "${TETHER_DB}" ]] && cp "${TETHER_DB}" "${HARNESSBENCH_TRACE}/tether.db"
++chmod -R a+rX "${HARNESSBENCH_WORKSPACE}" "${HARNESSBENCH_TRACE}" 2>/dev/null || true
++exit "${status}"
+diff --git a/tools/harness_runtime.py b/tools/harness_runtime.py
+index c57ba22..e8763dc 100644
+--- a/tools/harness_runtime.py
++++ b/tools/harness_runtime.py
+@@ -67,6 +67,14 @@ SPECS = {
+         config_name="config.json",
+         docker_context=HARNESS_DOCKER_DIR / "nanobot",
+     ),
++    "tether": HarnessSpec(
++        harness_id="tether",
++        version="0.2.5",
++        image="harnessbench/tether:0.2.5",
++        protocol="chat_completions",
++        config_name="providers.yaml",
++        docker_context=HARNESS_DOCKER_DIR / "tether",
++    ),
+     "goose": HarnessSpec(
+         harness_id="goose",
+         version="1.44.0",
+@@ -291,6 +299,19 @@ def render_harness_config(
+         }
+         return json.dumps(config, ensure_ascii=False, indent=2) + "\n"
+ 
++    if spec.harness_id == "tether":
++        entry = {
++            "kind": "openai",
++            "base_url": base_url,
++            "api_key_env": "VLLM_API_KEY",
++            "context_tokens": context_window,
++            "fold_after": 8,
++            "timeout": 1800,
++            "params": {"temperature": temperature, "max_tokens": max_tokens},
++        }
++        header = f"# model: {model}\n# turns: {max_agent_turns}\n"
++        return header + yaml.safe_dump({"harnessbench": entry}, sort_keys=False)
++
+     if spec.harness_id == "goose":
+         config = {
+             "provider": "openai",

tether_sdk-0.2.7/bench/report.py

@@ -0,0 +1,48 @@
+"""Failure breakdown for a results dir: `python3 bench/report.py <out> [<out> ...]`.
+
+Splits every run into pass, model-side outcomes (wrong answer, ran out of steps, stalled) and
+failures that say nothing about the harness's logic (provider outage, killed by timeout), so a
+score never hides infrastructure noise and infrastructure never hides a harness bug.
+"""
+import collections
+import json
+import os
+import re
+import sys
+
+OUTAGE = re.compile(r"unavailable after|429|503|Not Found|ConnectError|TimeoutError")
+
+
+def outcome(r):
+    if r["pass"]:
+        return "pass"
+    try:
+        j = json.loads(open(r["dir"] + "/stdout.txt").read())
+    except (OSError, ValueError):
+        return "timeout" if r.get("run_status") == "timeout" else "no_output"
+    f = j.get("failure") or {}
+    kind, msg = f.get("kind"), f.get("message", "")
+    if kind == "worker_error":
+        return "outage" if OUTAGE.search(msg) else "worker_error"
+    return kind or ("wrong_answer" if j.get("status") == "COMPLETED" else "timeout")
+
+
+def main(dirs):
+    rows = []
+    for d in dirs:
+        rows += [json.loads(line) for line in open(os.path.join(d, "results.jsonl"))]
+    groups = collections.defaultdict(list)
+    for r in rows:
+        groups[(r["runner"], r["model"])].append(r)
+    for (runner, model), rs in sorted(groups.items()):
+        c = collections.Counter(outcome(r) for r in rs)
+        valid = [r for r in rs if outcome(r) not in ("outage", "timeout", "no_output")]
+        mean = lambda l: round(sum(x["score"] for x in l) / len(l), 3) if l else 0.0  # noqa: E731
+        wall = round(sum(r["wall_s"] for r in rs) / len(rs))
+        print(f"{runner} {model}: {len(rs)} runs, pass {c['pass']} ({c['pass'] / len(rs):.0%}), mean {mean(rs)}, "
+              f"mean over valid runs {mean(valid)}, {wall}s/run")
+        print("   " + ", ".join(f"{k} {v}" for k, v in c.most_common()))
+
+
+if __name__ == "__main__":
+    main(sys.argv[1:])

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/bench/run.py

@@ -20,10 +20,22 @@ HERE = os.path.dirname(os.path.abspath(__file__))
 TASK_CFG = {  # tether tools / step cap / wall timeout
     "code-simple": ("edit,bash", 30, 600), "code-medium": ("edit,bash", 40, 900), "code-hard": ("edit,bash", 50, 1200),
     "focus": ("edit,bash,mem", 30, 600),
+    "plan": ("edit,bash", 20, 420), "codereview": ("edit,bash", 20, 420),
     "qa": ("", 10, 300), "content": ("", 10, 300), "review": ("edit", 15, 420),
 }
 
 
+POLY_VERIFY = False
+
+
+def cfg(task):
+    return ("edit,bash", 30, 600) if task.startswith("poly-") else TASK_CFG[task]
+
+
+def poly_tasks():
+    return ",".join("poly-" + e for e in sorted(os.listdir(grade.poly_dir())))
+
+
 def group(model):
     return "nvidia" if model.startswith("openai/") else "ollama" if model.startswith("ollama/") else "claude" if model.startswith(("claude", "cc/")) else "nvidia"
 
@@ -39,7 +51,7 @@ def flags(tether_bin):
 
 
 def build_cmd(runner, model, task, goal, ws, rd, tether_bin, params=()):
-    tools, steps, _ = TASK_CFG[task]
+    tools, steps, _ = cfg(task)
     if runner == "tether":
         h = flags(tether_bin)
         cmd = [tether_bin, "run", "--model", model, "--max-steps", str(steps), "--db", f"{rd}/t.db"]
@@ -49,6 +61,8 @@ def build_cmd(runner, model, task, goal, ws, rd, tether_bin, params=()):
             cmd += ["--tools", tools]
         for item in params:
             cmd += ["--param", item]
+        if task.startswith("poly-") and POLY_VERIFY:
+            cmd += ["--verify", "python3 -m unittest discover -p '*_test.py'"]
         return cmd + [goal]
     if runner == "naive":
         base, key = ("http://localhost:11435/v1", "") if model.startswith("ollama/") else ("https://integrate.api.nvidia.com/v1", "NVIDIA_API_KEY")
@@ -98,8 +112,13 @@ async def one(sem, rec, args, results):
     async with sem[group(model)]:
         shutil.rmtree(rd, ignore_errors=True)
         os.makedirs(rd)
-        shutil.copytree(f"{HERE}/tasks/{task}/ws", ws)
-        goal = open(f"{HERE}/tasks/{task}/goal.txt").read().strip()
+        if task.startswith("poly-"):
+            ex = task[5:]
+            shutil.copytree(grade.poly_dir(ex), ws, ignore=shutil.ignore_patterns(".meta", ".docs"))
+            goal = grade.poly_goal(ex)
+        else:
+            shutil.copytree(f"{HERE}/tasks/{task}/ws", ws)
+            goal = open(f"{HERE}/tasks/{task}/goal.txt").read().strip()
         cmd = build_cmd(runner, model, task, goal, ws, rd, args.tether_bin, args.param)
         env = dict(os.environ, OLLAMA_HOST="http://localhost:11435", PYTHONDONTWRITEBYTECODE="1")
         if model.startswith("openai/"):  # old tether has no providers.yaml: route openai/ to NVIDIA's OpenAI-compatible API
@@ -108,7 +127,7 @@ async def one(sem, rec, args, results):
         p = await asyncio.create_subprocess_exec(*cmd, cwd=ws, env=env, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, start_new_session=True)
         status = "ok"
         try:
-            out, err = await asyncio.wait_for(p.communicate(), TASK_CFG[task][2])
+            out, err = await asyncio.wait_for(p.communicate(), cfg(task)[2])
         except asyncio.TimeoutError:
             os.killpg(p.pid, signal.SIGKILL)
             out, err = await p.communicate()
@@ -160,7 +179,14 @@ async def main():
     ap.add_argument("--par", default="nvidia=4,ollama=1,claude=3")
     ap.add_argument("--force", action="store_true")
     ap.add_argument("--redo-infra", action="store_true", help="rerun cells that failed on provider errors")
+    ap.add_argument("--poly-verify", action="store_true", help="verify gate on the exercise tests: gate runs at `done`")
     args = ap.parse_args()
+    global POLY_VERIFY
+    POLY_VERIFY = args.poly_verify
+    if args.tasks == "polyglot":
+        args.tasks = poly_tasks()
+    elif args.tasks == "polyglot-fast":
+        args.tasks = ",".join("poly-" + e for e in grade.POLY_FAST)
     os.makedirs(args.out, exist_ok=True)
     sem = {k: asyncio.Semaphore(int(v)) for k, v in (x.split("=") for x in args.par.split(","))}
     recs = [(r, m, t, k) for r in args.runners.split(",") for m in args.models.split(",") for t in args.tasks.split(",") for k in range(args.trials)

tether_sdk-0.2.7/bench/tasks/codereview/goal.txt

@@ -0,0 +1 @@
+Do not edit any file. Review inventory.py. Call done with a numbered list of the real defects you find: for each, name the function, say what is wrong and give the one-line fix. Report only real bugs, no style remarks.

tether_sdk-0.2.7/bench/tasks/codereview/ws/inventory.py

@@ -0,0 +1,40 @@
+"""Inventory helpers for a small shop."""
+import sqlite3
+
+
+def page(items, number, size=10):
+    """Return page `number` (1-based) of `items`."""
+    start = number * size
+    return items[start:start + size]
+
+
+def add_tag(item, tag, tags=[]):
+    tags.append(tag)
+    item["tags"] = tags
+    return item
+
+
+def total_price(lines):
+    total = 0.0
+    for line in lines:
+        total += line["price"] * line["qty"]
+    return total
+
+
+def is_free(price):
+    return price == 0.0
+
+
+def find_by_name(conn, name):
+    cur = conn.execute("SELECT id, name FROM items WHERE name = '" + name + "'")
+    return cur.fetchall()
+
+
+def load_stock(path):
+    f = open(path)
+    data = f.read()
+    return dict(line.split(",") for line in data.splitlines())
+
+
+def stock_of(stock, name):
+    return int(stock[name])

tether_sdk-0.2.7/bench/tasks/plan/goal.txt

@@ -0,0 +1 @@
+Do not edit any file. Read svc/client.py and svc/config.py, then call done with an implementation plan (a numbered list of 6 to 10 steps) for adding client-side retries with exponential backoff and jitter to the HTTP client. The plan must name the exact files and functions to change, say how the retry limit and base delay become settings, say which failures are retried (for example HTTP 429 and 5xx, timeouts) and which are not, say how Retry-After is honoured, and say how it will be tested without a real network. Plan only, no code.

tether_sdk-0.2.7/bench/tasks/plan/ws/README.md

@@ -0,0 +1,3 @@
+# svc
+
+Tiny service layer. All network calls go through `svc/client.py` (`get`, `post`). Settings live in `svc/config.py`. Tests live in `tests/` (none for the client yet).

tether_sdk-0.2.7/bench/tasks/plan/ws/svc/client.py

@@ -0,0 +1,18 @@
+"""HTTP client used by every service call."""
+import json
+import urllib.request
+
+from svc.config import BASE_URL, TIMEOUT
+
+
+def get(path):
+    with urllib.request.urlopen(BASE_URL + path, timeout=TIMEOUT) as resp:
+        return json.loads(resp.read())
+
+
+def post(path, payload):
+    req = urllib.request.Request(
+        BASE_URL + path, data=json.dumps(payload).encode(), headers={"Content-Type": "application/json"}
+    )
+    with urllib.request.urlopen(req, timeout=TIMEOUT) as resp:
+        return json.loads(resp.read())

tether_sdk-0.2.7/bench/tasks/plan/ws/svc/config.py

@@ -0,0 +1,3 @@
+"""Settings."""
+BASE_URL = "https://api.example.internal"
+TIMEOUT = 5

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/docs/architecture.md

@@ -53,12 +53,12 @@ In *"MemGPT: Towards LLMs as Operating Systems"*, Packer et al. (2023) introduce
 
 Tether implements a deterministic three-tier memory hierarchy backed by `.tether/tether.db` (SQLite WAL + FTS5):
 
-1. **Head + Tail L1 Active Elision**: Even within the active $K=2$ turn window, if a command emits an oversized output (`> 2,400 chars`), `ContextAssembler` keeps the first 900 chars (headers/command summary) + last 900 chars (stack trace/exit status) and elides the middle with a pointer to `mem recall <obs_id>`.
+1. **Head + Tail L1 Active Elision**: Even within the active $K=2$ turn window, if a command emits an oversized output (more than a third of `max_context_tokens` in chars, at least 2,400), `ContextAssembler` keeps the first 60% of the cap (headers/command summary) + the last 40% (stack trace/exit status) and elides the middle with a pointer to `mem recall <obs_id>`.
 2. **Ephemeral $K$-Turn Folding**: Once an observation ages past $K=2$ steps, it is folded into a single-line tombstone:
    ```text
    [obs_1 folded: `web fetch https://...` (3,420 chars) | use `mem recall obs_1` or `mem search <query>`]
    ```
-   Exception (#69): the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The 2,400-char head+tail elision still applies, and the hard context budget remains #81. The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command").
+   Exception (#69): the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The head+tail elision still applies (an observation may use up to a third of `max_context_tokens`, at least 2,400 chars), and the hard context budget remains #81. The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command").
 3. **Sub-Millisecond SQLite FTS5 BM25 RAG**: Every raw observation and final deliverable is automatically chunked (~150 tokens per paragraph/section) and indexed into an FTS5 virtual table (`chunks_fts`) so the agent (or a parent agent) can retrieve exact passages via `mem search` or `agent query`.
 
 ---

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/docs/reference/context-and-store.md

@@ -22,7 +22,7 @@ When the provider sets `structured_history = True` (Ollama, OpenAI-compatible, A
 
 Raw tool outputs stay in the prompt for $K=2$ turns, then fold into a one-line tombstone (see [Built-in Tools](../tutorial/built-in-tools.md#mem-pinned-scratchpad-bm25-observation-recall)). For small models this has consequences ([#27](https://github.com/igormcsouza/tether/issues/27)), seen in the [course-builder example](https://github.com/igormcsouza/tether/tree/main/examples/course):
 
-- **Fix (#69).** Folding exception: the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The 2,400-char head+tail elision still applies, and the hard context budget remains #81. The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command"). (`ContextAssembler(mem_enabled=..., workspace_root=...)`, `MAX_LIVE_READS`).
+- **Fix (#69).** Folding exception: the latest read of each of the last 8 files (`edit view <p>`, `bash cat/head/tail/sed -n <p>`; paths normalized against the workspace) is not folded until `<p>` is edited (`edit write|replace`, or any other bash command naming it: `sed -i`, `tee`, redirects, `python -c`) or re-read. The head+tail elision still applies (an observation may use up to a third of `max_context_tokens`, at least 2,400 chars), and the hard context budget remains #81. The `mem recall` hint is shown only when `mem` is enabled (else "re-run the command"). (`ContextAssembler(mem_enabled=..., workspace_root=...)`, `MAX_LIVE_READS`).
 - **Re-viewing files loops, then invents data.** The model runs `edit view`, the output folds two turns later, it views again, and eventually writes from memory.
 - **What worked: one compact joined view, used right before the write, that ends with the exact next call.** The `course-source <module>` tool returns a module's plan and chosen videos in one output and finishes with `NOW WRITE THIS MODULE: act("edit write part-N.md", ...)`, so the data is still in the prompt when the write happens.
 - **Write long outputs in small pieces.** One big write dropped a whole section. The writer skill writes one `part-N.md` per module with `edit write`, then a `course-build` tool concatenates them into `course.md`.

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/docs/reference/providers.md

@@ -48,7 +48,7 @@ Every entry, of any kind, may set optional positive integers: `context_tokens` (
 
 ### Timeout and retry
 
-Every entry may set `timeout` (positive seconds). It applies to the HTTP kinds (`openai`, `ollama`, `anthropic`; defaults 120 / 120 / 180 s) and to `command` (default 300 s). The HTTP providers retry automatically, at most 2 times with 1 s then 2 s backoff, on timeouts, network errors and HTTP 429/500/502/503/504. A 401/403 is never retried: it raises `ProviderError` naming the provider and the credential variable (`openai: HTTP 401: `NVIDIA_API_KEY` is missing or rejected; set it to a valid key`; the value is never printed), and a declared `api_key_env` that is unset or empty fails the session before any request with the same message. Other 4xx responses are never retried. When retries run out the call raises `ProviderError`, so the session fails with a readable `Worker error: ProviderError: openai model `gpt-4o` unavailable after 3 attempts (timeout 120s, last error: ReadTimeout)`. The retry is invisible to the prompt (0 tokens, no prefix change).
+Every entry may set `timeout` (positive seconds). It applies to the HTTP kinds (`openai`, `ollama`, `anthropic`; defaults 120 / 120 / 180 s) and to `command` (default 300 s). The HTTP providers retry automatically, at most 2 times with 1 s then 2 s backoff, on timeouts, network errors and HTTP 429/500/502/503/504. A 401/403 is never retried: it raises `ProviderError` naming the provider and the credential variable (`openai: HTTP 401: `NVIDIA_API_KEY` is missing or rejected; set it to a valid key`; the value is never printed), and a declared `api_key_env` that is unset or empty fails the session before any request with the same message. Other 4xx responses are never retried. An HTTP 500 whose body names a rejected tool call (Ollama: `invalid tool call arguments`, a cut-off tool-call JSON) is not retried either, since the same request fails the same way; `ollama`, `openai` and `anthropic` turn it into a one-line hint to the model (a no-action reply), so the session continues instead of failing. The match is the text `tool call` in a 500 body, for any server. When retries run out the call raises `ProviderError`, so the session fails with a readable `Worker error: ProviderError: openai model `gpt-4o` unavailable after 3 attempts (timeout 120s, last error: ReadTimeout)`. The retry is invisible to the prompt (0 tokens, no prefix change).
 
 ### Kinds
 

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/docs/release-notes.md

@@ -1,5 +1,29 @@
 # Release Notes
 
+## v0.2.7 (2026-10-24)
+
+Found by running Tether on the Polyglot-Python benchmark (34 Aider polyglot Python exercises) and reading why runs failed instead of guessing: sessions that ended mid-task on a server error, and models that re-read one file until the step cap. Closes #138, #141.
+
+### Benchmark
+
+- `bench/run.py --tasks polyglot|polyglot-fast` runs the Polyglot-Python exercises, graded by each exercise's own tests; `bench/report.py` prints pass rate, mean score and a failure breakdown that keeps provider outages apart from harness outcomes. On the 12-exercise fast set (2 trials), the observation-cap fix below moved `qwen3.5:9b` from mean 0.64 to 0.80 and cut re-reads of one file from 32% to 19% of steps; `nemotron-3.5-lightning` went from 10 to 12 passes of 24.
+
+### Fixes
+
+- Context: one active observation may now use up to a third of the context budget (`max_context_tokens * 4 / 3` chars, at least 2,400) before head+tail elision, instead of a fixed 2,400 chars. Before, a 6-12 KB test file or a 100-line slice of one was always cut to head+tail, so the model asked for another slice, got that cut too, and re-read the same file 10+ times: on the Polyglot-Python fast set 24 of 48 runs ended at the step cap, 45% of their steps were reads and only 19% were edits. The hard budget still folds older observations first, so the prompt cannot overflow; `ContextAssembler(active_char_cap=...)` still sets a fixed cap.
+- Providers: when a server (Ollama, vLLM, llama.cpp behind `openai`, `anthropic`) answers HTTP 500 because it rejected the model's own tool call (`invalid tool call arguments for "act": unexpected end of JSON input`), the call is no longer retried and no longer ends the session as `worker_error` (#138). The model gets one line (`Your last call was rejected: the tool-call JSON was invalid or cut off. Send a shorter call, e.g. write long files in smaller pieces.`) through the existing no-action reminder and carries on, bounded by `max_steps`. Before, 13 of 34 Polyglot-Python runs on `qwen3.5:9b` died this way, mid-task and often with most tests passing. `ModelResponse` gains an optional `hint`. 0 static prompt tokens.
+
+## v0.2.6 (2026-10-23)
+
+Found by using tether as its own SWE agent and by the v0.2.5 post-release check: a sandbox that refused harmless commands, sessions that only read, and a `query` that could not say why it found nothing. Closes #77, #93, #125, #127.
+
+### Fixes
+
+- CLI: `tether query` now says why nothing matched (#93): `Error: unknown session` with exit code 1 for a wrong id, `No chunks are indexed for <id> yet` for an empty session, and the number of indexed chunks otherwise. `search_chunks` also tries a prefix match (`loop*`) after the AND and OR matches, so `loop` finds `looping`.
+- Store: `SQLiteStore(":memory:")` no longer creates a file literally named `:memory:` in the working directory (#77). The path now maps to a temporary file outside the workspace, so every store method still shares one database.
+- Sandbox: with `bwrap` active, `bash` no longer refuses commands by text for `..`, absolute paths outside the workspace, or `.env`/`.tether`/`.git/config` mentions (#125). The kernel mounts already enforce them (read-only root, hidden `$HOME`, masked secrets), so the text rules only produced false refusals such as `ls ~/..` or `cat /etc/hosts`. They remain the fallback when `bwrap` is missing, and `sudo`, `su`, disk tools, `rm -rf /`, `chmod 777`, fork bombs and `~/.ssh` stay refused in both modes.
+- Sandbox: `.env.example`, `.env.sample`, `.env.template` and `.env.dist` are readable and writable; real `.env`, `.env.local`, `.env.production` stay masked and protected. Templates hold no secrets.
+
 ## v0.2.5 (2026-10-23)
 
 Found by benchmarking the same models inside Tether and inside a plain tool loop (7 tasks, 10+ models): model idioms made the single-`act` grammar fail invisibly, and loops or oversized prompts burned steps and tokens. Closes #76, #81, #86, #89, #92 and the shell-idiom bugs #109, #110.
@@ -46,6 +70,8 @@ Dogfooding fixes: the four milestone bugs (#74 #75 #85 #88) plus the friction fo
 
 ### Fixes
 
+- Harness: a session that only reads gets a deterministic hint (#127). After 12 consecutive read-only steps with no change at all, then every 8 more, the observation ends with `[NUDGE] No file changes in N steps. Make your edit now, or call `done` with what you found.` (0 static prompt tokens). Before, the quiet nudge started only after a first change, so a model that never edited (nemotron-3-ultra and glm-5.3 on scoped briefs: 40 steps, 150k to 215k prompt tokens, no change) was never told.
+- Harness: `awk`/`gawk`/`mawk` count as read-only (unless they print to a file, pipe to a shell, call `system()` or edit in place) and as range-shift reads of one file, so the #92 stall guard sees `awk 'NR>=a && NR<=b' f` loops. `NR>=5` used to look like a redirect, also inside compound commands (`grep ...; awk 'NR>=5' f`), which silently turned the nudges off.
 - Store: a `RUNNING` session whose worker pid no longer exists is flipped to `FAILED` (`Worker died (pid N) before finishing`) when read via `get_session`/`list_sessions`, so `status`, `list`, `wait` and `read` no longer report a killed worker as running forever (#85). A pid owned by another user counts as alive.
 - Providers: `429` retries wait `Retry-After` (else 10s doubling, capped 30s) and get up to 5 attempts instead of 3 quick ones, so rate-limited hosted models (NVIDIA free tier) no longer fail the session after 3 quick attempts.
 - Providers: each `post_with_retry` attempt is now bounded by a total `asyncio.wait_for(timeout)` deadline, so a trickling/stalled response cannot hang past the configured timeout; the deadline is retried like other transient errors and shows as `TimeoutError` (vs `ReadTimeout`) in the final `ProviderError` (#88).

{tether_sdk-0.2.5 → tether_sdk-0.2.7}/docs/testing.md

@@ -85,7 +85,7 @@ trigger it, script the reply; if infrastructure must fail, add a fault to the pr
 
 ## Model conformance bench (developers only)
 
-`bench/` holds seven tasks (the `focus` task checks that an exact single-file goal is not lost in a distractor-heavy repo) with hidden-test graders and three runners (Tether, a plain tool loop,
+`bench/` holds nine tasks (the `focus` task checks that an exact single-file goal is not lost in a distractor-heavy repo) with hidden-test graders and three runners (Tether, a plain tool loop,
 Claude Code) so the same model can be compared across harnesses and releases. It needs real
 models, so it is not in CI and not in the package: use it when onboarding a model or before a
 release to see friction that scripted tests cannot (the shell-wrapper, bare-command and