tether-sdk 0.2.9__tar.gz → 0.2.10__tar.gz

tether_sdk-0.2.10/.claude/gru/active/north-star.md

@@ -0,0 +1,40 @@
+# North Star: Polyglot-Python 80% on qwen3.5:9b
+- Slug: north-star
+- Status: active
+- Started: 2026-10-27   Updated: 2026-10-27   Closed: —
+- Branches / PRs: branch fix/north-star-cheap-wins (worktree .claude/worktrees/north-star-c852b8), tracker #173
+- Budget: warn 300k / stop 400k paid tokens, max escalations 3, Tier 3 never without ask   Used: ~small (gru only)
+
+## Goal
+Polyglot-Python (34 ex, 2 trials, `--poly-verify`, default 8192 ctx) pass rate on ollama/qwen3.5:9b: 51.5% -> 80% by v0.3.0. Lanes: default (official) and `swe` role (#172, must not score below default).
+
+## Requests
+- 2026-10-27 PO: "Get to the North Star"; dev too slow, benches unreported. Asked to review the last cross-session and close it (done: v0-2-bugs archived).
+
+## Decisions
+- 2026-10-27 PO answers: lane 1 = deterministic fixes first (no GPU); merge proof = tests + offline replay, one batched full 34x2 bench after 2-3 PRs, reported immediately; budget default 300k/400k.
+- Ranked levers (Opus assessment): (1) runaway thinking hits num_ctx, +8-12pp, two attempts inconclusive (think=false 20 rows +3; length-retry rejected); (2) compact test output (relativize 195-char workspace path = 14% of obs chars, tracebacks = 51%), +3-6pp; (3) 4/68 runs died on 120s ProviderError timeout ~6pp (floor #171); (4) test-file access #170.
+- STOP list: nudge variants, whole-set 24-run A/Bs, wall cap as part of the north star.
+
+## Problems
+- 14 exercises never pass: book-store bowling connect forth hangman paasio poker pov react rest-api scale-generator sgf-parsing transpose zipper. Bench GPU: 8GB RTX 5050, one Ollama.
+
+## Plan (task cards)
+- A: ollama default timeout 120 -> 300 (tests/providers/test_ollama.py test written; docs/reference/providers.md line 95 says 120). Gru patches (1 line).
+- B: bash output: strip workspace prefix + ==== / ---- rule lines in sandbox.execute_bash_with_code (tests/test_sandbox.py test written). Gru patches (small).
+- C (next): collapse unittest tracebacks; build bench/replay.py offline measure first.
+- Then: output headroom / bounded thinking (#135/#147), #170, #171.
+
+## Ledger
+| Task | Tier | Model | Harness | Session | Attempt | Tokens | Time | Result |
+|---|---|---|---|---|---|---|---|---|
+| A,B | plan | gru | - | - | 1 | small | - | tests written failing, patch pending |
+
+## Progress log
+- 2026-10-27 branch fix/north-star-cheap-wins; failing tests added (sandbox prefix, ollama timeout). Closed v0-2-bugs log.
+
+## Next steps
+- [ ] Patch A and B, update docs (providers.md, release notes), unit+e2e, PR with `north star:` line
+- [ ] bench/replay.py offline proof for B
+- [ ] One batched full 34x2 after A+B+C, report at once
+- 2026-10-27 merged #180 (timeout 300s, compact bash, canary). PR #181 traceback collapse (canary +0.12, offline -12.6% obs chars, e2e 69). PR #182 outage retry (#171, e2e new scenario). PO: 80% is the impossible focus goal; wants v0.2.10 plan ranked by north star. Canary tool: bench/canary.py (12 min, 7 rows). Note: bench `pass` is graded by exercise tests, so #177 (done at max steps) does NOT move the bench number, only the user-facing floor.

{tether_sdk-0.2.9/.claude/gru/active → tether_sdk-0.2.10/.claude/gru/archive}/v0-2-bugs.md

@@ -1,7 +1,7 @@
 # v0.2 bugs
 - Slug: v0-2-bugs
-- Status: active
-- Started: 2026-10-20   Updated: 2026-10-24   Closed: —
+- Status: done
+- Started: 2026-10-20   Updated: 2026-10-24   Closed: 2026-10-27
 - Branches / PRs: branch `claude/keep-feature-context-resume-0ec918` (worktree); base `main` at 125b386 (release v0.2.2)
 
 ## Goal
@@ -108,3 +108,29 @@ Track and fix bugs found in Tether v0.2.x. Exact bug list not yet given by PO.
 - 2026-10-25 PO: milestones reshuffled: v0.2.8 = release now (#144 #145 + sandbox venv + CLI/providers docs), v0.2.9 = docs overhaul (#150-#154, #63-#66), v0.2.10 = #27 #79 #132-#137. Merged: #146 #148 #149 #155 #156 #157 #158. Release PR v0.2.8 (version bump, notes) prepared; GitHub Release only on PO yes.
 - 2026-10-25 RELEASED v0.2.8 'Room to Answer' (release PR #159 -> f4370b1, GitHub Release, delivery.yml success, PyPI tether-sdk 0.2.8 live, milestone closed). Docs v0.2.9 started: #151 site structure PR (nav = Start here/Guides/Concepts/Reference/Help, 'For:' line per page, CONTRIBUTING.md replaces docs/testing.md + releasing.md, tests/test_docs_structure.py). Lesson: ruff format --check also formats Python blocks in docs; renamed anchors break the strict mkdocs build via old release-notes links -> run both before pushing docs.
 - 2026-10-25 PO: 'merge all docs PRs and cut the next version'. Docs plan (#150) done: #155 #156 #157 #160 #161 #162 #165 #166 #167 merged, epic closed. Found #163 while writing guide 1 (qwen3.5:9b fixes the test, passes, never calls done -> FAILED Stalled); #164 merged on PO yes (Rule 3 reading: passing verify gates at stall time -> COMPLETED). Release PR v0.2.9 prepared. Next: v0.2.10 = #27 #79 #132-#137 (8 issues; pick by evidence, e.g. bench failure breakdown after v0.2.9). Tether friction this stretch: docs rewrites by Nemotron invent API facts and end without done; harness.py cards need 40+ steps.
+- 2026-10-26 v0.2.9 released (#168). Polyglot fast set + poly-verify on v0.2.9: 12/24, mean 0.765; 8192 vs 16384 context on 12 exercises: 16384 not a regression (easy 9/12 both, mean 0.872->0.937; hard 3->5/12) but ~+50% s/step (decode-bound), more 600s bench-cap timeouts; default stays 8192 (token saver), --context-tokens bench option on branch bench/137-context-tokens (no PR). Root-cause read of the breakdown found #169: bare `done` / typed `act("done", ...)` rejected, 8 of 36 runs wasted 1-13 steps, 2 false FAILED at score 1.0.
+- 2026-10-26 #169 fix (PR on branch fix/169-bare-done): tests first (e4f0feb), fix 143d79e. Ledger: #169 card A (bare done) | Tier 0 | z-ai/glm-5.3 | tether | ses_152e5df2 | attempt 2 | 7 steps, 6.6k tokens | ok (3 lines). Card B (act() text unwrap) | nemotron-3-ultra ses_d9caa19b (combined card A+B): 25 steps, 230k prompt tokens, zero edits; glm-5.3 ses (800s timeout, rate-limited, zero edits) -> escalated to plan level, 5-line patch written by gru.
+- Tether friction (#169): (1) nemotron-3-ultra on a 2-file card read 3 large files and ran pytest 12 times in 25 steps without ever calling `edit write` (read-only loop, no pre-change nudge fired before step 12, prompt 230k tokens); (2) nvidia free tier + glm-5.3 ~70s per step (rate limit) so an 800s timeout ended with no edit. Cards under ~15 lines and one file each worked (glm-5.3 7 steps).
+- 2026-10-26 #169 bench (done-fix vs v0.2.9, qwen3.5:9b, poly-verify, 2 trials, 8192): done-rejected steps 18 (4 runs) -> 0; pass 12/24 -> 10/24, mean 0.765 -> 0.694 (one infra outage cell rerun). Within noise: every failing run in the new set made 0 done calls (30 steps, never called done), so none touched the changed code. Affine-cipher t0 28 -> 7 steps, bottle-song t0 21 -> 14. Remaining dominant failure: max_steps without calling done (rewrite loops, e.g. food-chain t1 scores 1.0 yet FAILED). e2e 69 passed, unit 391 passed, 100% cov.
+- 2026-10-27 North star set (#173, AGENTS.md "North Star"): Polyglot-Python on qwen3.5:9b default 8192, full 34: 80% by v0.3.0. Baseline pinned (#178, bench/history.jsonl): main@b25cbbe 35/68 = 51.5%, mean 0.718; 17 fails at 30-step cap, 11 at the 600s bench wall cap, 14 exercises never pass (book-store bowling connect forth hangman paasio poker pov react rest-api scale-generator sgf-parsing transpose zipper). Naive no-harness loop on fast-12: 3/24 (12.5%) vs tether 12/24.
+- 2026-10-27 Experiments, all unmerged (data on issues): ctx 16384 fast-12 11/24 vs 12/24 (mean 0.807 vs 0.711, 9/13 fails hit 600s cap) -> rejected (#170); paging nudge flat; bounded cmd replay worse (10->8/24); think=false on 20 targeted rows 9/20 vs 6/20 (net +3, pre-reg +4) ; adaptive length-retry (branch fix/135-length-retry) 1/12 on failing rows -> rejected (#135); blanket think=false on fast-12 stopped at 11/24 runs (7 pass), inconclusive.
+- 2026-10-27 OPUS ASSESSMENT (architect, read-only, 13 tool calls over base34 t.db): 
+  * First-order failure nobody had measured = runaway thinking hitting num_ctx: 53 assistant turns with no action, 45 have prompt+completion in [8192,9216) (re-verified by gru: 45/53, 15 runs, 3672 s). Model thinks until the window is full, Ollama stops, tether records "(no output)" and nudges; ~85-110 s per turn. In the 11 wall-cap runs 41% of model time was empty turns: wall-cap failures are this, not slow progress. context.py:333 budgets prompt to 3/4 of ctx and leaves ~2048 for output; thinking does not fit.
+  * 4 of 68 runs died of ProviderError timeout 120s x3 (book-store t1, transpose t0, hangman t1, pig-latin t1): unretried infra loss ~6pp, breaks the floor (#171).
+  * Second mechanism: read-loop to the step cap (paasio poker react zipper sgf pov-t1 grep-t0 phone-t0 bowling-t1; fast 6-11 s/step). Test-run output (unittest tracebacks) = 51% of all observation chars; the 195-char bench workspace path = 14% of observation chars (~60 tokens per traceback line); react prints 6.5KB per test run, paasio 12KB.
+  * Why no progress: picked by proxy (re-read %) not mechanism; thinking not recorded (#32); 24-run bench sd ~2.4 passes = a +10pp change is ~1 sigma; whole-set A/B dilutes effects; the 600s wall cap measures GPU speed and thinking length (confound; ctx16 raised mean score but lost on wall); infra deaths baked into baseline.
+  * Fast assessment tiers: T0 offline replay `bench/replay.py` (~150 lines; rebuild each step's prompt from a recorded t.db through the candidate assembler; prompt tokens, output headroom, turns exceeding num_ctx, test-output chars; seconds, no GPU; proves prompt shape only). T1 determinism: same exercise twice with --param seed=1 (temp already 0.2), compare command sequences; first divergence = did the change fire; identical = zero effect (10 min). T2 counterfactual fork: copy a failing t.db, cut where the mechanism first fires, resume with the change (5-15 min/run). T3 targeted subset: only rows where the mechanism fired + 4 always-pass heavy-thinking canaries (zebra t1, vlq t0, two-bucket t0, go-counting t1), paired seeds, sequential stop (ship +4 net flips, kill at -2), <=1h. T4 full 34x2 overnight. Rule: a change gets GPU time only if T0/T1 shows its target metric moves.
+  * Parallelism: GPU is an 8GB RTX 5050; two Ollama instances will not fit; try OLLAMA_NUM_PARALLEL=2 (~1.5x?) for 10 min, only after dropping the wall cap (contention distorts wall).
+  * Ranked levers: (1) output headroom + bounded thinking, expected +8-12pp (15 affected runs, partial 0.6-0.75): test think=false T3, real fix = on done_reason=="length" with no tool call retry once with think=false, reserve output headroom in context.py, scale provider timeout to the generation budget, overlaps #135/#147; (2) compact test-run output: relativize workspace_root in bash observations, collapse unittest tracebacks to `test_x:105 AssertionError: 2 != 4`, drop ====/---- lines, +3-6pp, verify via T0 then T3; (3) test-file access #170, +2-4pp low confidence (def test_ index when file exceeds cap).
+  * 2-week workflow: D1 build replay.py + seed determinism check + queue think=false T3; D2 headroom/think-retry PR; D3-4 output-compaction PR proven with T0 then T3; bench-rules PR (official cap = 30 steps only with 1800s safety wall, p50/p90 wall as indicator, short workspace path /tmp/tb/<ex>, provider timeout no longer kills a run silently, re-pin baseline overnight once); end wk1 full T4; wk2 #170 then #171. Opus does diagnosis/tooling/acceptance tests/verdicts, minions mechanical PRs only; one GPU queue. STOP: nudge variants, whole-set 24-run A/Bs, wall cap as part of the north star.
+- 2026-10-27 Result of following it (gru): lever 1 tested twice, neither clean: think=false 20 rows +3 net, adaptive length-retry rejected (empty turns 3672s -> 332s but 20 turns still >2500 completion tokens: thinking is slow even when it fits; blanket think=false ran 405 turns in the time retry ran 243). T0/T1/T2 never built.
+- 2026-10-27 PO feedback (verbatim gist): dev is far too slow ("1 line of code and an entire day benching it"), benches finish unreported, results get thrown away because of the impossible goal; gru is losing itself on the work; wants the Opus findings logged. LESSON: stop bench-per-line-of-code loops; decide with a fast tier first, ship deterministic/obviously-correct improvements with unit+e2e evidence, report every bench end immediately (watcher fires on the results file, not on the launcher shell).
+## Next steps (as of 2026-10-27, awaiting PO decision)
+- [ ] PO to choose: (a) finish blanket think=false fast-12 (~13 runs, 45 min), (b) offline output compaction (no GPU), (c) plan only. Nothing is running; no open PRs.
+
+
+## Wrap-up (2026-10-27, closed by PO request)
+- Outcome: v0.2.3 -> v0.2.9 released (PyPI tether-sdk 0.2.9). Bugs and docs overhaul done; goal shifted to the North Star (#173): Polyglot-Python 80% on qwen3.5:9b by v0.3.0. Baseline 35/68 = 51.5% (main@b25cbbe).
+- Durable lessons: (1) never bench per line of code: decide with a fast tier first (offline replay of recorded t.db, determinism check), GPU only for targeted rows; (2) report every bench end immediately; (3) cards under ~15 lines, one file, worked on tether minions (glm-5.3 7 steps); 2-file cards and docs rewrites on Nemotron Ultra loop or invent API facts; (4) run `ruff format --check` and strict mkdocs before docs pushes; (5) nudge/wording variants and whole-set 24-run A/Bs do not move the number (sd ~2.4 passes).
+- Open debt: #27 #79 #132-#137 #170 #171; branches unmerged: fix/170-bounded-cmd-replay (rejected), fix/135-length-retry (rejected), bench/137-context-tokens; Harness-Bench #123 blocked (docker firewall + key).
+- Tier cost: Tier 0 (Nemotron Ultra, GLM-5.3 on tether) solved only tiny single-file cards; everything else was finished at plan level by gru.

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/AGENTS.md

@@ -28,6 +28,15 @@ Tether stays **lightweight, reliable, extensible, model-agnostic, easy to use, r
 
 Reviewers and the Team Leader cite the failing line when they reject a change.
 
+### North Star (the one number; see tracker #173)
+**Polyglot-Python pass rate on `ollama/qwen3.5:9b` (our weakest model), default config, full 34 exercises: 80% by v0.3.0.** Nothing important is set aside for it, but everything is ranked by it:
+1. Every issue, PR and benchmark must say how it moves this number (or tokens per solved exercise at the same pass rate). A benchmark that cannot show a change in it is not run; an issue that cannot name the link waits behind one that can.
+1b. Two lanes, both tracked: the **general-purpose default** (no role) is the official number and must reach 80%; the shipped **`swe` role** (#172) is reported next to it, must never score below the default, and gets its own target once its baseline is pinned.
+2. Weakest-model rule: every stronger model must score at least what `qwen3.5:9b` scores (and at least 80% once reached). If we plateau, stronger models may not fall below the plateau.
+3. Other indicators (tokens per pass, steps, wall-clock, prompt size, coverage) may slow or pause but never degrade between releases.
+4. Floor, a promise rather than a goal: every task delivers a real, honest, usable result in a fair time; no unretried outage, no empty result, no unrelated hallucinated deliverable (#171).
+5. Every PR body and issue-close comment ends with `north star: <before> -> <after> (runs, noise, run dir)`; the tracker's History table and `bench/history.jsonl` are updated at every release.
+
 ---
 
 ## 2. The Golden Rules (Non-Negotiable Invariants)
@@ -43,7 +52,7 @@ Reviewers and the Team Leader cite the failing line when they reject a change.
 
 ### Rule 3: Explicit Termination (`act("done", stdin="...")`), Interactive Inquiry (`done --ask`), & Pre-`done` Verification Gates
 - Sessions do **not** terminate simply because the model emitted a text response without a tool call (small 9B models frequently "think out loud" before acting).
-- A task only completes when the agent explicitly calls `act("done", stdin="<final deliverable>")` (or `act("done '<short answer>'")`), pauses for human/parent input via `act("done --ask", stdin="<question>")` (transitioning the session to `WAITING_INPUT` and releasing the worker process until resumed via `send`), or hits `max_steps`.
+- A task only completes when the agent explicitly calls `act("done", stdin="<final deliverable>")` (or `act("done '<short answer>'")`), pauses for human/parent input via `act("done --ask", stdin="<question>")` (transitioning the session to `WAITING_INPUT` and releasing the worker process until resumed via `send`), or hits `max_steps`. A bare `done` (no answer, no stdin) completes with the result `Done.` and still runs the `verify` gates; `done --ask` with no question is an error (#169).
 - If the model emits plain text without calling `act`, the harness appends a brief deterministic nudge reminding it to either execute a command or call `done`.
 - After 6 consecutive read-only/verification steps following a change, the harness appends a one-shot `[NUDGE]` to that step's observation reminding the model to call `done` if the task is complete (0 static prompt tokens, #74). Before any change, 12 read-only steps in a row (then every 8 more) append a one-line `[NUDGE]` to make the edit or call `done` (#127). Seven reads of one file in a row with no change are a `cycle_stall`, and 4 stall warnings without a change end the session `FAILED` as `Stalled` (#92); `tether stop <id>` terminates a detached worker (#89).
 - When `verify` gate commands are configured (via `Harness(verify=[...])`, CLI `--verify`, or Skill YAML `verify: [...]`), `Harness.execute_session` deterministically executes each command inside the session's `SandboxPolicy` before allowing `done` to transition to `COMPLETED` (consuming 0 static prompt tokens until a gate fails). A gate that errors instead of failing (sandbox violation, exit 124/126/127) ends the session `FAILED` at once as `gate_error` ("gate misconfigured"), without counting toward the consecutive-failure abort. When the stall guard is about to end a session as `Stalled` and gates are configured, it runs them first: a change was made and every gate passes, so the session ends `COMPLETED` ("Finished by the harness"), since the deterministic gate is the completion criterion the caller asked for (#163).
@@ -205,7 +214,7 @@ uv run mypy tether/
 2. **Async-First Design:**
    - All model calls and I/O-bound tool executions must be `async`/`await` native, while seamlessly wrapping synchronous user `@tool` functions.
 3. **Graceful Degradation on 9B Hallucinations:**
-   - Small models occasionally emit `cmd` with minor quirks (e.g., leading `$`, wrapping backticks, or calling a tool name directly instead of `act`). The multiplexer must normalize trivial syntax quirks deterministically before failing. This includes unescaping literal `\n`/`\t` in one-line `stdin` for `edit write`/`edit replace` and `done` (never `bash`), turning a trailing shell heredoc in `cmd` (`edit write f <<'EOF' ... EOF`) into `stdin` (never for `bash`), unwrapping `bash -c/-lc '<cmd>'` and `sh -c` wrappers, running an unknown first word through the enabled `bash` tool (never for disabled built-ins), registering extra quirk fixers with `register_normalizer`, mapping a native tool call named other than `act` to `<name> --k 'v'` so the JIT usage answers it, inferring the `edit` verb (`edit <path>` -> view/replace/write) and accepting `apply_patch` (`*** Begin Patch` or unified-diff patches, also when a diff arrives as `edit <path>` stdin) as an `edit` verb.
+   - Small models occasionally emit `cmd` with minor quirks (e.g., leading `$`, wrapping backticks, or calling a tool name directly instead of `act`). The multiplexer must normalize trivial syntax quirks deterministically before failing. This includes unescaping literal `\n`/`\t` in one-line `stdin` for `edit write`/`edit replace` and `done` (never `bash`), turning a trailing shell heredoc in `cmd` (`edit write f <<'EOF' ... EOF`) into `stdin` (never for `bash`), unwrapping `bash -c/-lc '<cmd>'` and `sh -c` wrappers, unwrapping an `act("<cmd>", stdin="...")` call typed as text in `cmd` (also after `bash`) into the command and its `stdin` (#169), running an unknown first word through the enabled `bash` tool (never for disabled built-ins), registering extra quirk fixers with `register_normalizer`, mapping a native tool call named other than `act` to `<name> --k 'v'` so the JIT usage answers it, inferring the `edit` verb (`edit <path>` -> view/replace/write) and accepting `apply_patch` (`*** Begin Patch` or unified-diff patches, also when a diff arrives as `edit <path>` stdin) as an `edit` verb.
 4. **Golden traces:** behaviour the model sees (framing, folding, nudges, tool output) is locked by `tests/golden/*.snap.json`. A PR that changes it runs `UPDATE_GOLDEN=1 pytest tests/test_golden.py` and the snapshot diff is part of the review; unrelated traces must not change. See `CONTRIBUTING.md`.
 5. **E2E scenarios:** every feature or bug fix ships with a scenario in `e2e/` (real CLI, real ollama behind a fault-injecting proxy, local only, `TETHER_E2E=1 uv run pytest e2e`). Assert on what tether sends and records, never on the model's skill; script the reply or inject the fault so the test does not depend on model potency. **Before merging any PR, run `TETHER_E2E=1 uv run pytest e2e` locally and put the result in the PR description (CI cannot run it). Every failure must be mitigated before merge: fix the code, or fix a wrong scenario, or record in the PR why it is acceptable and what follows up. A PR is not merged with an unexplained red scenario.** See `CONTRIBUTING.md`.
 6. **Documentation Integrity:**
@@ -225,5 +234,5 @@ uv run mypy tether/
 - Hard stops (ask first): force push / history rewrite; merging a PR and cutting a release need the PO's explicit yes each time
 - Design docs: `AGENTS.md` sections 1-3, `docs/architecture.md`, `docs/reference/*`
 - Orchestration: **`/gru` runs this repo** (Team Leader + router). Strong model plans, writes the failing acceptance tests first, reviews and decides; tether does the typing as the minion at Tier 0 on the strongest free model (`nvidia/nvidia/nemotron-3-ultra-550b-a55b`, fallback `nvidia/z-ai/glm-5.3`), escalating one tier only on evidence. Tiers, context sizes and sensitive paths: `.claude/gru.json`. tether's own failures while doing a card are logged as friction and filed as issues (with the PO's OK).
-- Improvement loop: **every PR leaves tether a little better**, measured, not asserted: a behaviour change ships with a bench or e2e delta (Polyglot-Python fast set on `ollama/qwen3.5:9b`, `bench/`) in the PR body, reported honestly even when it is flat. After a merge, gru re-reads the next planned milestone and the other open bug issues and picks the one that clearly improves tether most (evidence from bench failure breakdowns and `tether stats` friction beats guesses); features and docs wait behind bugs that cost steps or pass rate.
+- Improvement loop: **every PR leaves tether a little better**, measured, not asserted: a behaviour change ships with a bench or e2e delta (Polyglot-Python on `ollama/qwen3.5:9b`, `bench/`) in the PR body as a `north star:` line (see North Star), reported honestly even when it is flat. After a merge, gru re-reads the next planned milestone and the other open bug issues and picks the one that clearly improves tether most (evidence from bench failure breakdowns and `tether stats` friction beats guesses); features and docs wait behind bugs that cost steps or pass rate.
 - gru store: .claude/gru/   (committed; legacy `feature-context` logs were moved here)

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tether-sdk
-Version: 0.2.9
+Version: 0.2.10
 Summary: General-purpose, headless-first LLM-Native micro-harness for small language models
 Project-URL: Homepage, https://github.com/igormcsouza/tether
 Project-URL: Repository, https://github.com/igormcsouza/tether

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/bench/README.md

@@ -17,6 +17,10 @@ restored before grading, so editing it cannot pass), `--poly-verify` adds a `--v
 `python3 bench/report.py <out>` prints pass rate and mean score with a failure breakdown that keeps provider outages
 (`outage`, `timeout`) apart from harness outcomes (`max_steps`, `stalled`, `wrong_answer`, `worker_error`).
 
+**Did my change help? (15 min)** `python3 bench/canary.py /tmp/canary-<change> [--tether-bin <venv>/bin/tether]` runs 7 exercises
+once (4 flaky ones that can flip, 3 fast guards) and prints each score next to the pinned baseline with a verdict
+(`REJECT` / `PROMISING` / `FLAT`). Only `PROMISING` earns the full 34x2 run. `--tasks canary` is the same set for `run.py`.
+
 Three runners, so a gap can be attributed:
 
 | runner | what it is |
@@ -43,3 +47,11 @@ Rate limits (NVIDIA: 40 requests/min) invalidate results silently; set `rpm:` in
 and keep parallelism low (`--par nvidia=2`).
 
 Caveats: one run per cell is noisy (about 0.1); the scores are regex and hidden-test based.
+
+## History (the north-star ledger)
+
+`bench/history.jsonl` is the single source for the tracker's History table (#173). One JSON object per line: `date`, `version`, `model`, `runner`, `tasks`, `trials`, `context`, `runs`, `passed`, `pass_rate`, `mean_score`, failure counts, mean steps/prompt tokens/wall, `note`.
+
+A PR that changes behaviour the model sees appends one line for its own run (same model, default 8192 context, `--poly-verify`) and ends its body with `north star: <before> -> <after> (runs, noise, run dir)`. Noise is about 0.1 per run, so a claim needs enough runs to beat it; a flat result is appended and reported as flat.
+
+The official number is the full 34 exercises × 2 trials (68 runs) with the 600 s per-run wall cap. Use `--tasks polyglot-fast` for quick checks and label it `polyglot-fast-12`.

tether_sdk-0.2.10/bench/canary.py

@@ -0,0 +1,39 @@
+"""15-minute check: did this change help? `python3 bench/canary.py <out-dir> [--tether-bin ...]`.
+
+Runs 7 Polyglot exercises once (4 flaky ones that can flip, 3 fast guards), then compares each
+score with the pinned baseline (`canary_baseline.json`, mean of 2 runs on main@b25cbbe).
+Verdict: REJECT if a guard drops >0.2, PROMISING if the flaky mean gains >=0.1, else FLAT.
+Only PROMISING changes earn a full 34x2 run. One run per exercise is noisy: a FLAT verdict
+means "no evidence", not "proven useless".
+"""
+
+import json
+import os
+import subprocess
+import sys
+
+HERE = os.path.dirname(os.path.abspath(__file__))
+sys.path.insert(0, HERE)
+import grade  # noqa: E402
+
+
+def main(out, extra):
+    cmd = [sys.executable, f"{HERE}/run.py", "--models", "ollama/qwen3.5:9b", "--tasks", "canary",
+           "--poly-verify", "--par", "ollama=1", "--out", out, *extra]
+    subprocess.run(cmd, check=True)
+    base = json.load(open(f"{HERE}/canary_baseline.json"))
+    now = {}
+    for line in open(f"{out}/results.jsonl"):
+        r = json.loads(line)
+        now[r["task"][5:]] = r["score"]
+    flaky = [e for e in base if e not in grade.GUARDS]
+    for e in base:
+        print(f"{e:20} {base[e]:.2f} -> {now.get(e, float('nan')):.2f}{'  (guard)' if e in grade.GUARDS else ''}")
+    gain = sum(now.get(e, 0) - base[e] for e in flaky) / len(flaky)
+    drops = [e for e in grade.GUARDS if now.get(e, 0) < base[e] - 0.2]
+    verdict = "REJECT" if drops else "PROMISING" if gain >= 0.1 else "FLAT"
+    print(f"flaky mean delta {gain:+.2f}; guard drops {drops or 'none'} -> {verdict}")
+
+
+if __name__ == "__main__":
+    main(sys.argv[1], sys.argv[2:])

tether_sdk-0.2.10/bench/canary_baseline.json

@@ -0,0 +1 @@
+{"phone-number": 0.69, "tree-building": 0.92, "grep": 0.66, "pig-latin": 0.91, "proverb": 1.0, "robot-name": 1.0, "list-ops": 1.0}

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/bench/grade.py

@@ -180,6 +180,10 @@ def _codereview(_task, _ws, result):
 POLY_FAST = ("affine-cipher bottle-song list-ops paasio dot-dsl food-chain forth go-counting poker pov react "
              "rest-api").split()
 
+# ~15 min: 4 flaky exercises that can flip + 3 fast guards that must not regress (baseline in canary_baseline.json)
+POLY_CANARY = "phone-number tree-building grep pig-latin proverb robot-name list-ops".split()
+GUARDS = ("proverb", "robot-name", "list-ops")
+
 
 def poly_dir(ex=""):
     """Aider polyglot-benchmark Python exercises (cloned once, never vendored)."""

tether_sdk-0.2.10/bench/history.jsonl

@@ -0,0 +1,3 @@
+{"date": "2026-10-27", "version": "main@b25cbbe (post v0.2.9, #176)", "model": "ollama/qwen3.5:9b", "runner": "tether", "tasks": "polyglot-34", "trials": 2, "context": 8192, "note": "baseline; wall cap 600s; 14 exercises never pass", "runs": 68, "passed": 35, "pass_rate": 0.515, "mean_score": 0.718, "fail_max_steps": 17, "fail_wall_cap": 11, "mean_steps": 18.2, "mean_prompt_tokens": 82120, "mean_wall_s": 274}
+{"date": "2026-10-27", "version": "main@b25cbbe", "model": "ollama/qwen3.5:9b", "runner": "tether", "tasks": "polyglot-fast-12", "trials": 2, "context": 8192, "note": "same runs as baseline, fast subset", "runs": 24, "passed": 12, "pass_rate": 0.5, "mean_score": 0.711, "fail_max_steps": 8, "fail_wall_cap": 3, "mean_steps": 20.7, "mean_prompt_tokens": 94828, "mean_wall_s": 265}
+{"date": "2026-10-27", "version": "naive_agent", "model": "ollama/qwen3.5:9b", "runner": "naive", "tasks": "polyglot-fast-12", "trials": 2, "context": 8192, "note": "no-harness ceiling reference", "runs": 24, "passed": 3, "pass_rate": 0.125, "mean_score": 0.243, "fail_max_steps": 0, "fail_wall_cap": 0, "mean_steps": 4.7, "mean_prompt_tokens": 8054, "mean_wall_s": 55}

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/bench/run.py

@@ -50,7 +50,7 @@ def flags(tether_bin):
     return _HELP[tether_bin]
 
 
-def build_cmd(runner, model, task, goal, ws, rd, tether_bin, params=()):
+def build_cmd(runner, model, task, goal, ws, rd, tether_bin, params=(), context_tokens=None):
     tools, steps, _ = cfg(task)
     if runner == "tether":
         h = flags(tether_bin)
@@ -61,6 +61,8 @@ def build_cmd(runner, model, task, goal, ws, rd, tether_bin, params=()):
             cmd += ["--tools", tools]
         for item in params:
             cmd += ["--param", item]
+        if context_tokens:
+            cmd += ["--context-tokens", str(context_tokens)]
         if task.startswith("poly-") and POLY_VERIFY:
             cmd += ["--verify", "python3 -m unittest discover -p '*_test.py'"]
         return cmd + [goal]
@@ -119,7 +121,9 @@ async def one(sem, rec, args, results):
         else:
             shutil.copytree(f"{HERE}/tasks/{task}/ws", ws)
             goal = open(f"{HERE}/tasks/{task}/goal.txt").read().strip()
-        cmd = build_cmd(runner, model, task, goal, ws, rd, args.tether_bin, args.param)
+        cmd = build_cmd(
+            runner, model, task, goal, ws, rd, args.tether_bin, args.param, args.context_tokens
+        )
         env = dict(os.environ, OLLAMA_HOST="http://localhost:11435", PYTHONDONTWRITEBYTECODE="1")
         if model.startswith("openai/"):  # old tether has no providers.yaml: route openai/ to NVIDIA's OpenAI-compatible API
             env.update(OPENAI_BASE_URL="https://integrate.api.nvidia.com/v1", OPENAI_API_KEY=os.environ.get("NVIDIA_API_KEY", ""))
@@ -177,6 +181,7 @@ async def main():
     ap.add_argument("--tether-bin", default="tether")
     ap.add_argument("--param", action="append", default=[], help="forwarded to `tether run --param`")
     ap.add_argument("--par", default="nvidia=4,ollama=1,claude=3")
+    ap.add_argument("--context-tokens", type=int, default=None, help="forwarded to `tether run`")
     ap.add_argument("--force", action="store_true")
     ap.add_argument("--redo-infra", action="store_true", help="rerun cells that failed on provider errors")
     ap.add_argument("--poly-verify", action="store_true", help="verify gate on the exercise tests: gate runs at `done`")
@@ -185,6 +190,8 @@ async def main():
     POLY_VERIFY = args.poly_verify
     if args.tasks == "polyglot":
         args.tasks = poly_tasks()
+    elif args.tasks == "canary":
+        args.tasks = ",".join("poly-" + e for e in grade.POLY_CANARY)
     elif args.tasks == "polyglot-fast":
         args.tasks = ",".join("poly-" + e for e in grade.POLY_FAST)
     os.makedirs(args.out, exist_ok=True)

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/docs/reference/providers.md

@@ -92,12 +92,12 @@ Every entry, of any kind, may set optional positive integers: `context_tokens` (
 
 ### Timeout and retry
 
-Every entry may set `timeout` (positive seconds). It applies to the HTTP kinds (`openai`, `ollama`, `anthropic`; defaults 120 / 120 / 180 s) and to `command` (default 300 s). The HTTP providers retry automatically, at most 2 times with 1 s then 2 s backoff, on timeouts, network errors and HTTP 429/500/502/503/504. A 401/403 is never retried: it raises `ProviderError` naming the provider and the credential variable (`openai: HTTP 401: `MY_GATEWAY_KEY` is missing or rejected; set it to a valid key`; the value is never printed), and a declared `api_key_env` that is unset or empty fails the session before any request with the same message. Other 4xx responses are never retried. An HTTP 500 whose body names a rejected tool call (Ollama: `invalid tool call arguments`, a cut-off tool-call JSON) is not retried either, since the same request fails the same way; `ollama`, `openai` and `anthropic` turn it into a one-line hint to the model (a no-action reply), so the session continues instead of failing. The match is the text `tool call` in a 500 body, for any server. When retries run out the call raises `ProviderError`, so the session fails with a readable `Worker error: ProviderError: openai model `gpt-4o` unavailable after 3 attempts (timeout 120s, last error: ReadTimeout)`. The retry is invisible to the prompt (0 tokens, no prefix change).
+Every entry may set `timeout` (positive seconds). It applies to the HTTP kinds (`openai`, `ollama`, `anthropic`; defaults 120 / 300 / 180 s) and to `command` (default 300 s). The HTTP providers retry automatically, at most 2 times with 1 s then 2 s backoff, on timeouts, network errors and HTTP 429/500/502/503/504. A 401/403 is never retried: it raises `ProviderError` naming the provider and the credential variable (`openai: HTTP 401: `MY_GATEWAY_KEY` is missing or rejected; set it to a valid key`; the value is never printed), and a declared `api_key_env` that is unset or empty fails the session before any request with the same message. Other 4xx responses are never retried. An HTTP 500 whose body names a rejected tool call (Ollama: `invalid tool call arguments`, a cut-off tool-call JSON) is not retried either, since the same request fails the same way; `ollama`, `openai` and `anthropic` turn it into a one-line hint to the model (a no-action reply), so the session continues instead of failing. The match is the text `tool call` in a 500 body, for any server. When retries run out the call raises `ProviderError`, so the session fails with a readable `Worker error: ProviderError: openai model `gpt-4o` unavailable after 3 attempts (timeout 120s, last error: ReadTimeout)`. When they run out the harness waits 10 s (set `TETHER_OUTAGE_BACKOFF`) and tries the whole call again, up to 3 more times with doubling waits, before the session fails. The retry is invisible to the prompt (0 tokens, no prefix change).
 
 
 ### Request parameters (`params`)
 
-HTTP kinds (`openai`, `ollama`, `anthropic`) accept `params`, merged into the request body over the adapter's defaults (temperature 0.2). For `ollama`, `think`, `keep_alive`, `format` and `stream` are request-level keys and everything else (`temperature`, `top_p`, ...) goes into `options`.
+HTTP kinds (`openai`, `ollama`, `anthropic`) accept `params`, merged into the request body over the adapter's defaults (temperature 0.2). `ollama` also sends `think=false` on a turn whose prompt leaves under 3072 tokens of `num_ctx` free, unless you set `think` yourself. For `ollama`, `think`, `keep_alive`, `format` and `stream` are request-level keys and everything else (`temperature`, `top_p`, ...) goes into `options`.
 
 ```yaml
 my-gateway:

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/docs/release-notes.md

@@ -1,5 +1,22 @@
 # Release Notes
 
+## Unreleased
+
+## v0.2.10 (2026-10-28)
+
+The run-killers found in the v0.2.9 Polyglot breakdown on `qwen3.5:9b`: provider outages, a window filled by thinking, noisy test output and big test files re-read slice by slice. The full 34x2 benchmark for this release is published separately once it finishes.
+
+### Fixes
+
+- When a long observation is cut (head + tail), the tests defined in the cut middle are listed as `name:line` (up to 30): a test file too big for the window is no longer re-read slice after slice, which cost 54% of the steps in max-step runs (#170).
+- A session that ends without `done` (max steps or stall) now says what it did: the last change commands and, with `--verify`, whether the gates pass or which one fails. Status and exit code are unchanged (#171).
+- Ollama: when the prompt leaves less than 3072 tokens of `num_ctx` free, the request sends `think=false`. A thinking reply no longer fills the window and comes back empty (45 of 53 empty baseline turns sat at the window limit). An explicit `--param think=...` still wins (#183).
+- A provider outage that outlasts the provider's own retries no longer ends the session: the harness tries the call again up to 3 more times (waits of 10, 20, 40 s; `TETHER_OUTAGE_BACKOFF` changes the first wait) and only then fails with the same `ProviderError`. 4 of 68 baseline Polyglot runs died of one timeout (#171).
+- `bash` observations keep only the last two frames of a Python traceback (the call site and the cause) with `... N frames ...` for the rest, and drop the `^^^^` marker rows: test failures cost fewer tokens.
+- Ollama default timeout is now 300 s (was 120 s): a model that thinks until the context window is full needs 85-110 s per turn, and 4 of 68 baseline Polyglot runs died on the old limit.
+- `bash` observations no longer carry the absolute workspace path (`<root>/`) or `====`/`----` rule lines: less noise in test output, fewer tokens per step.
+- `done`: a bare `done` with no answer now completes the session with the result `Done.` (the `--verify` gates still run first) instead of returning an error the model repeated until `max_steps`. `done --ask` with no question is still an error. A model that types the call as text, `act("done", stdin="...")` or `bash act("done", ...)`, in `cmd` is understood the same way as a real call (#169). Found in the v0.2.9 Polyglot breakdown: 8 of 36 `qwen3.5:9b` runs wasted 1-13 steps on a rejected `done`, and 2 runs that passed every test ended `FAILED` at max steps.
+
 ## v0.2.9 (2026-10-25)
 
 The docs are rebuilt around the people who use tether, and writing the first guide with a real run found a real bug: a small model that fixed a failing test, saw it pass, then repeated the check until the stall guard ended the session as a failure. Closes #163, #61, #60, #62, #63, #64, #151, #152, #153, #154.

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/docs/troubleshooting.md

@@ -18,7 +18,9 @@ With `--verify` gates configured, a stall after a change whose gates all pass en
 
 **Cause.** The task needs more steps than allowed, or the model never found the way to finish.
 
-**Fix.** Check `tether read <id> --steps`: if the work is done and the model keeps verifying, tell it to call `done` in the goal. If it needs more room, raise `--max-steps` (for a pipeline, per stage). `tether send <id> "keep going"` resumes a finished session with a fresh budget.
+The result is never bare: it lists the last change commands and, with `--verify`, whether the gates pass now (`Stalled` ends the same way). A run that stopped with passing gates already holds a usable result in the workspace.
+
+**Fix.** Check `tether read <id> --steps`: if the work is done and the model keeps verifying, tell it to call `done` in the goal. A bare `done` is enough: it ends the session with the result `Done.` once the gates pass. If it needs more room, raise `--max-steps` (for a pipeline, per stage). `tether send <id> "keep going"` resumes a finished session with a fresh budget.
 
 ### `FAILED` with `Verification gate ... misconfigured`
 

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/e2e/conftest.py

@@ -108,6 +108,7 @@ class Tether:
             "HOME": str(self.home),
             "TETHER_PROVIDERS": str(providers),
             "E2E_KEY": "sekret-value-123",
+            "TETHER_OUTAGE_BACKOFF": "0",
         }
 
     def cli(

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/e2e/test_faults.py

@@ -20,12 +20,20 @@ def test_429_is_retried_honoring_retry_after(tmp_path: Path, chaos, model: str)
 
 
 def test_persistent_5xx_fails_loudly_with_exit_5(tether: Tether, chaos) -> None:
-    chaos.fault("status", times=10, code=503)
+    chaos.fault("status", times=20, code=503)
     r = tether.run("anything")
     assert (r["rc"], r["status"]) == (5, "FAILED")
     assert "unavailable after 3 attempts" in r["failure"]["message"]
 
 
+def test_outage_that_outlasts_provider_retries_is_survived(tether: Tether, chaos) -> None:
+    """#171: 3 failed attempts end one provider call, not the session; the next try succeeds."""
+    chaos.fault("status", times=3, code=503)
+    chaos.script(("done", "ok"))
+    r = tether.run("go", "--max-steps", "6")
+    assert (r["rc"], r["status"]) == (0, "COMPLETED"), r
+
+
 def test_rejected_tool_call_reaches_the_model_instead_of_ending_the_session(
     tether: Tether, chaos
 ) -> None:
@@ -40,13 +48,13 @@ def test_rejected_tool_call_reaches_the_model_instead_of_ending_the_session(
 
 def test_trickling_response_hits_the_per_attempt_deadline(tmp_path: Path, chaos) -> None:
     """#88: bytes keep arriving so httpx's read timeout never fires; the deadline must."""
-    chaos.fault("trickle", times=3, seconds=60)
+    chaos.fault("trickle", times=12, seconds=60)  # 3 attempts x 4 turn-level tries
     t = Tether(tmp_path, chaos, timeout=3)
     start = time.time()
     r = t.run("anything")
     assert (r["rc"], r["status"]) == (5, "FAILED")
     assert "TimeoutError" in r["failure"]["message"], r["failure"]
-    assert time.time() - start < 40, "must not hang past attempts x timeout"
+    assert time.time() - start < 80, "must not hang past attempts x timeout"
 
 
 def test_auth_error_names_the_variable_never_the_value(tether: Tether, chaos) -> None:

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/e2e/test_quirks.py

@@ -86,6 +86,13 @@ def test_stall_with_a_failing_gate_stays_failed(tether: Tether, chaos) -> None:
     assert r["status"] == "FAILED" and r["failure"]["message"].startswith("Stalled")
 
 
+def test_unfinished_session_reports_changes_and_gate(tether: Tether, chaos) -> None:
+    chaos.script(*(["bash touch fixed.txt"] * 6), ("done", "ok"))  # #171: never a bare error
+    r = tether.run("loop", "--tools", "bash", "--verify", "test -f missing.txt")
+    msg = r["failure"]["message"]
+    assert "Changed: `bash touch fixed.txt`" in msg and "Gate `test -f missing.txt` fails" in msg
+
+
 def test_two_command_cycle_is_a_cycle_stall(tether: Tether, chaos) -> None:
     chaos.script(*(["bash echo a", "bash echo b"] * 3), ("done", "ok"))
     r = tether.run("cycle", "--tools", "bash")
@@ -162,3 +169,16 @@ def test_skill_enables_its_tools_without_touching_the_schema(tether: Tether, cha
     sysmsg = req["messages"][0]["content"]
     assert [t["function"]["name"] for t in req["tools"]] == ["act"]
     assert "- bash:" in sysmsg and "[SKILL]" in sysmsg and "Always run `bash true` first." in sysmsg
+
+
+def test_bare_done_completes_instead_of_looping(tether: Tether, chaos) -> None:
+    chaos.script("done")  # #169: was "Error: provide final deliverable", repeated to max_steps
+    r = tether.run("finish", "--tools", "bash")
+    assert r["status"] == "COMPLETED"
+    assert tether.events(r["id"]) == ["completed"]
+
+
+def test_act_call_typed_as_text_is_unwrapped(tether: Tether, chaos) -> None:
+    chaos.script('act("done", stdin="all green")')
+    r = tether.run("finish", "--tools", "bash")
+    assert r["status"] == "COMPLETED" and r["result"] == "all green"

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/tests/providers/test_ollama.py

@@ -109,3 +109,36 @@ async def test_ollama_unknown_model_fails_loudly_by_default() -> None:
     op = OllamaProvider("doesnotexist", client=client, fallback=False)
     with pytest.raises(RuntimeError, match="doesnotexist.*installed: qwen3.5:9b"):
         await op.generate([{"role": "user", "content": "hi"}], [])
+
+
+def test_default_timeout_fits_a_full_window_of_thinking() -> None:
+    """North star: 4/68 baseline runs died on a 120s ReadTimeout while the model was thinking."""
+    from tether.providers.ollama import OllamaProvider
+
+    assert OllamaProvider().timeout >= 300
+
+
+async def _sent_payload(provider_kwargs: dict, text: str) -> dict:
+    seen: dict = {}
+
+    def handler(request: httpx.Request) -> httpx.Response:
+        seen.update(__import__("json").loads(request.content))
+        return httpx.Response(200, json={"message": {"content": "hi"}})
+
+    client = httpx.AsyncClient(transport=httpx.MockTransport(handler))
+    provider = OllamaProvider("m", client=client, **provider_kwargs)
+    await provider.generate([{"role": "user", "content": text}], [])
+    return seen
+
+
+async def test_thinking_is_off_when_little_window_is_left_for_the_reply() -> None:
+    """#183: a prompt that leaves < 3072 tokens of num_ctx cannot hold a thinking reply."""
+    small = await _sent_payload({"num_ctx": 8192}, "x" * 400)
+    assert "think" not in small
+    full = await _sent_payload({"num_ctx": 8192}, "x" * 4 * 5200)
+    assert full["think"] is False
+    # an explicit `think` param always wins
+    forced = await _sent_payload({"num_ctx": 8192, "params": {"think": True}}, "x" * 4 * 5200)
+    assert forced["think"] is True
+    # headroom scales with the window: the same prompt is fine in a 32k window
+    assert "think" not in await _sent_payload({"num_ctx": 32768}, "x" * 4 * 5200)

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/tests/test_context.py

@@ -349,3 +349,19 @@ def test_budget_reserves_a_quarter_of_the_window_for_the_reply() -> None:
     # #144: a prompt that fills the window cuts off the model's file writes
     ca, msgs = _budget_run(8, 2400, 2000, fold_after_turns=8)
     assert estimate_messages_tokens(msgs, [ACT_TOOL_SCHEMA]) <= 2000 * 3 // 4
+
+
+def test_elided_test_file_keeps_an_index_of_its_tests() -> None:
+    """#170: the model stops re-reading a test file when the cut part still names its tests."""
+    body = "".join(
+        f"    def test_case_{i}(self):\n        assert f({i}) == {i}\n" for i in range(60)
+    )
+    out = elide_active_observation("obs_1", body, max_chars=600)
+    assert "elided in L1" in out
+    assert "tests in the elided part: " in out and "test_case_30:61" in out
+    index = out.split("tests in the elided part: ")[1].splitlines()[0]
+    assert len(index) < 700  # bounded
+    # nothing to index in an ordinary elided output
+    assert "tests in the elided" not in elide_active_observation(
+        "obs_2", "x\n" * 900, max_chars=600
+    )

{tether_sdk-0.2.9 → tether_sdk-0.2.10}/tests/test_harness.py

@@ -12,6 +12,7 @@ from tests.conftest import FakeScriptedProvider
 from tether import ActionCall, Harness, ModelResponse, register_provider, tool
 from tether.decorator import ToolSpec
 from tether.harness import detect_cycle, is_quiet, read_target, spawn_detached_worker
+from tether.providers.base import ProviderError
 from tether.store import SQLiteStore
 
 
@@ -237,6 +238,37 @@ async def test_pre_done_verification_gates_and_self_healing(tmp_path: Path) -> N
     assert "Fix the reported issues before calling `done` again." in step2_prompt
 
 
+async def test_bare_done_still_runs_verification_gates(tmp_path: Path) -> None:
+    """A bare `done` completes the session only once the gates pass (#169)."""
+    from tether.sandbox import SandboxPolicy
+
+    ws = tmp_path / "ws_bare"
+    ws.mkdir()
+    (ws / "module.py").write_text("VALUE = 1\n", encoding="utf-8")
+    provider = FakeScriptedProvider(
+        [
+            ModelResponse(content="", action=ActionCall(cmd="done")),
+            ModelResponse(
+                content="", action=ActionCall(cmd="edit write module.py", stdin="VALUE = 42\n")
+            ),
+            ModelResponse(content="", action=ActionCall(cmd="done")),
+        ]
+    )
+    harness = Harness(
+        db_path=tmp_path / "tether.db",
+        tools=["edit"],
+        verify=["python3 -c 'import module, sys; sys.exit(0 if module.VALUE == 42 else 1)'"],
+        sandbox=SandboxPolicy(workspace_root=ws, use_kernel_sandbox=False),
+        provider=provider,
+    )
+    session = await harness.run("Set VALUE = 42 in module.py")
+    assert session.status == "COMPLETED" and session.step_count == 3
+    assert session.result == "Done."
+    assert "[VERIFICATION GATE FAILED:" in "\n".join(
+        m["content"] for m in provider.recorded_calls[1]
+    )
+
+
 async def test_verification_gate_circuit_breaker_and_skill_verify(tmp_path: Path) -> None:
     """Verify circuit breaker trips on consecutive gate failures and skills declare verify gates."""
     from tether.sandbox import SandboxPolicy
@@ -757,3 +789,67 @@ async def test_stall_after_a_change_with_passing_gate_is_completed(tmp_path: Pat
 async def test_stall_with_failing_gate_stays_failed(tmp_path: Path) -> None:
     session = await _stalled_after_a_change(tmp_path, "false")
     assert session.status == "FAILED" and (session.result or "").startswith("Stalled")
+
+
+class _FlakyProvider:
+    """Raises ProviderError `fails` times, then calls done."""
+
+    model_name = "qwen3.5:9b"
+
+    def __init__(self, fails: int) -> None:
+        self.fails = fails
+        self.calls = 0
+
+    async def generate(self, messages: Any, tools: Any) -> ModelResponse:
+        self.calls += 1
+        if self.calls <= self.fails:
+            raise ProviderError("ollama model `qwen3.5:9b` unavailable after 3 attempts")
+        return ModelResponse(content="", action=ActionCall(cmd="done ok"))
+
+
+async def test_outage_is_retried_at_turn_level_then_fails_loudly(
+    tmp_path: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """North star floor (#171): 4/68 baseline runs died on one provider outage; retry 3 times."""
+    sleeps: list[float] = []
+
+    async def fake_sleep(delay: float) -> None:
+        sleeps.append(delay)
+
+    monkeypatch.setattr("tether.harness.asyncio.sleep", fake_sleep)
+
+    ok = _FlakyProvider(fails=2)
+    done = await Harness(db_path=tmp_path / "a.db", provider=ok, max_steps=3).run("go")
+    assert done.status == "COMPLETED" and ok.calls == 3 and len(sleeps) == 2
+
+    dead = _FlakyProvider(fails=99)
+    failed = await Harness(db_path=tmp_path / "b.db", provider=dead, max_steps=3).run("go")
+    assert failed.status == "FAILED" and "unavailable after 3 attempts" in (failed.result or "")
+    assert dead.calls == 4  # first try + 3 turn-level retries
+
+
+async def _unfinished(tmp_path: Path, cmds: list[str], verify: list[str]) -> str:
+    from tether.sandbox import SandboxPolicy
+
+    script = [ModelResponse(content="", action=ActionCall(cmd=c)) for c in cmds]
+    h = Harness(
+        db_path=tmp_path / "t.db",
+        tools=["edit"],
+        provider=FakeScriptedProvider(script),
+        max_steps=len(cmds),
+        verify=verify,
+        sandbox=SandboxPolicy(workspace_root=tmp_path, use_kernel_sandbox=False),
+    )
+    return (await h.run("goal")).result or ""
+
+
+async def test_max_steps_result_reports_changes_and_gate(tmp_path: Path) -> None:
+    """#171 floor: a session that never calls done still ends with something usable."""
+    write = "edit write a.py\nprint(1)"
+    res = await _unfinished(tmp_path, [write], ["false"])
+    assert res.startswith("Max steps (1) reached without calling `done`.")
+    assert "Changed: `edit write a.py" in res and "Gate `false` fails (exit 1)" in res
+    (tmp_path / "ok").mkdir()
+    assert "Gate passes" in await _unfinished(tmp_path / "ok", [write], ["true"])
+    res = await _unfinished(tmp_path, ["ls"], [])
+    assert res.endswith("No change was made.")