teamrunner 0.0.2__tar.gz → 0.0.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {teamrunner-0.0.2 → teamrunner-0.0.4}/.github/ISSUE_TEMPLATE/bug_report.yml +1 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/.github/ISSUE_TEMPLATE/question.yml +1 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/.github/workflows/tests.yml +9 -0
- teamrunner-0.0.4/CHANGELOG.md +103 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/NOTICE +11 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/PKG-INFO +64 -129
- teamrunner-0.0.4/PREVIEW.md +128 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/README.md +63 -128
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/fact_checked/against.md +0 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/fact_checked/chair.md +3 -3
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/fact_checked/fact_checker.md +4 -4
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/fact_checked/for.md +0 -1
- teamrunner-0.0.4/examples/fact_checked/judge.md +20 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/fact_checked/sandbox.md +1 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/fact_checked/team.yaml +2 -0
- teamrunner-0.0.4/examples/fact_checked/web.md +19 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/hiring_committee/models/member.yaml +3 -3
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/hiring_committee/team.yaml +1 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/multi_agent_debate/against.md +0 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/multi_agent_debate/for.md +0 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/multi_agent_debate/judge.md +0 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/multi_agent_debate/team.yaml +2 -0
- teamrunner-0.0.4/examples/orchestrators_lite/models/engineer.yaml +7 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/team.yaml +3 -3
- teamrunner-0.0.4/examples/reference_check/catalogue.md +19 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/context_checker.md +3 -2
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/editor.md +4 -2
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/intake.md +1 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/metadata_checker.md +11 -5
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/reader.md +7 -2
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/team.yaml +2 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/round_robin/01_for.md +0 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/round_robin/02_against.md +0 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/round_robin/03_judge.md +0 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/round_robin/team.yaml +2 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/pyproject.toml +1 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/__init__.py +1 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/authoring/AGENTS.md +8 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/build.py +20 -12
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/builtin_tools.py +103 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/checks.py +122 -43
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/cli.py +11 -6
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/console.py +23 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/engine.py +12 -2
- teamrunner-0.0.4/src/teamrunner/flowgraph.py +136 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/ir.py +18 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/litellm_model.py +14 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/new.py +2 -2
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/run.py +83 -19
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/runfolder.py +9 -6
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/runs.py +85 -14
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/schemas/member.schema.json +2 -2
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/schemas/model_profile.schema.json +2 -2
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/schemas/team.schema.json +2 -2
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/show.py +4 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/tool_registry.py +2 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/turns.py +1 -0
- teamrunner-0.0.4/src/teamrunner/ui/__init__.py +1 -0
- teamrunner-0.0.4/src/teamrunner/ui/api.py +297 -0
- teamrunner-0.0.4/src/teamrunner/ui/diagrams.py +131 -0
- teamrunner-0.0.4/src/teamrunner/ui/jobs.py +272 -0
- teamrunner-0.0.4/src/teamrunner/ui/runviews.py +231 -0
- teamrunner-0.0.4/src/teamrunner/ui/server.py +265 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/app.css +89 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/app.js +346 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/diagrams.js +53 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/index.html +134 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/run.js +144 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/runs.js +249 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/LICENSE +21 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/SOURCE.txt +5 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/dompurify-LICENSE.txt +202 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/dompurify-SOURCE.txt +5 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/dompurify.js +7 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/editor.css +344 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/editor.js +11383 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/marked-LICENSE.txt +44 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/marked-SOURCE.txt +5 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/marked.js +86 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/mermaid-LICENSE.txt +21 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/mermaid-SOURCE.txt +5 -0
- teamrunner-0.0.4/src/teamrunner/ui/static/vendor/mermaid.js +4329 -0
- teamrunner-0.0.4/src/teamrunner/ui/stream.py +78 -0
- teamrunner-0.0.4/src/teamrunner/ui/trash.py +79 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/validate.py +14 -2
- teamrunner-0.0.4/tests/fixtures/ui_history/README.txt +1 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-05_090000_debate/events.jsonl +9 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-05_090000_debate/output.md +14 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-05_090000_debate/run.json +26 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-06_100000_debate/events.jsonl +9 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-06_100000_debate/inputs/brief.txt +1 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-06_100000_debate/output.md +14 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-06_100000_debate/run.json +26 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-06_100000_debate/work/scores.csv +2 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-07_101500_review/events.jsonl +9 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-07_101500_review/output.md +14 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/runs/2026-11-07_101500_review/run.json +26 -0
- teamrunner-0.0.4/tests/fixtures/ui_history/team.yaml +2 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_new.py +2 -1
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_run.py +122 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_show.py +11 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_tools.py +51 -0
- teamrunner-0.0.4/tests/test_ui.py +793 -0
- teamrunner-0.0.4/tests/test_ui_diagrams.py +128 -0
- teamrunner-0.0.4/tests/test_ui_review.py +145 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_validate.py +29 -0
- teamrunner-0.0.4/tests/ui_transcript_check.js +44 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/uv.lock +1 -1
- teamrunner-0.0.2/CHANGELOG.md +0 -39
- teamrunner-0.0.2/examples/fact_checked/judge.md +0 -19
- teamrunner-0.0.2/examples/fact_checked/web.md +0 -13
- teamrunner-0.0.2/examples/orchestrators_lite/models/engineer.yaml +0 -7
- teamrunner-0.0.2/examples/reference_check/catalogue.md +0 -22
- {teamrunner-0.0.2 → teamrunner-0.0.4}/.github/ISSUE_TEMPLATE/config.yml +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/.github/workflows/publish.yml +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/.gitignore +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/CLA.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/CONTRIBUTING.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/LICENSE +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/SECURITY.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/LICENSE +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/fact_checked/.gitignore +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/hiring_committee/chair.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/hiring_committee/chen.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/hiring_committee/clerk.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/hiring_committee/diaz.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/hiring_committee/okafor.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/multi_agent_debate/.gitignore +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/multi_agent_debate/chair.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/.gitignore +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/chair.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/derek.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/emma.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/lily.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/noah.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/orchestrators_lite/zoe.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/chair.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/reference_check/library.md +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/examples/round_robin/.gitignore +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/__main__.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/authoring/LICENSE +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/authoring/__init__.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/describe.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/desk.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/environment.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/events.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/explain.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/flow.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/gather.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/ids.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/inputs.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/keys.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/loader.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/minutes.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/model.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/prices.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/problems.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/rehearsal.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/sandbox/__init__.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/sandbox/run.js +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/schema_check.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/src/teamrunner/vocabulary.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_authoring.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_engine.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_explain.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_flow.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_minutes.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_prices.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_runs.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_sandbox.py +0 -0
- {teamrunner-0.0.2 → teamrunner-0.0.4}/tests/test_turns.py +0 -0
|
@@ -68,3 +68,12 @@ jobs:
|
|
|
68
68
|
teamrunner explain debate
|
|
69
69
|
teamrunner run debate --rehearse -m "Should cities ban cars from their centres?"
|
|
70
70
|
teamrunner runs debate
|
|
71
|
+
|
|
72
|
+
# The website's tutorial (teamrunner.ai/tutorial.html), replayed with this build:
|
|
73
|
+
# a change that would break the tutorial fails here, before it is released
|
|
74
|
+
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
75
|
+
with:
|
|
76
|
+
repository: teamrunner/teamrunner.ai
|
|
77
|
+
path: website
|
|
78
|
+
- name: The website's tutorial
|
|
79
|
+
run: python website/tests/test_tutorial.py website/tutorial.html
|
|
@@ -0,0 +1,103 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
TeamRunner is in preview: 0.0.x versions are a soft launch, and until 0.1.0
|
|
4
|
+
is announced a version may change in ways that break teams and run folders
|
|
5
|
+
made with an earlier one. Each such change is listed here.
|
|
6
|
+
|
|
7
|
+
## 0.0.4
|
|
8
|
+
|
|
9
|
+
- **Local browser interface:** `teamrunner ui [folder]` (`gui` is an alias)
|
|
10
|
+
opens the team's files in a browser. Edit with offline syntax highlighting,
|
|
11
|
+
save, validate with clickable problems, inspect resolved instructions and
|
|
12
|
+
flow diagrams, create teams from examples, manage members and set up keys.
|
|
13
|
+
- Start one run at a time with defaults, a message, a file or a task file;
|
|
14
|
+
rehearse for free, follow events live, stop cleanly and re-run saved inputs
|
|
15
|
+
with the current team. Browse and search run history, read conversations,
|
|
16
|
+
outputs and records, inspect what members could read, and download raw files.
|
|
17
|
+
- Compare two runs' numbered member versions, outputs, cost and paths. New
|
|
18
|
+
event logs keep exact member source text and fingerprints, the declared
|
|
19
|
+
flow graph and the recorded engine path for source diffs and arrow counts.
|
|
20
|
+
Older logs remain readable and label missing historical data explicitly.
|
|
21
|
+
A saved `resolved/team.yaml`, when present, can be copied to a new team for
|
|
22
|
+
editing; ordinary runs do not yet save a complete resolved team.
|
|
23
|
+
- The interface uses the CLI's Python functions, adds no Python runtime
|
|
24
|
+
dependencies or database, and bundles its editor, Markdown renderer and
|
|
25
|
+
Mermaid diagrams for offline use. It binds only to `127.0.0.1`, requires a
|
|
26
|
+
random access token, confines editing to the opened folder, keeps run
|
|
27
|
+
records read-only and never displays stored key values.
|
|
28
|
+
- The README's tutorial gives a browser alternative beside each CLI step.
|
|
29
|
+
- UI review fixes: teams under an unrelated folder named `runs` remain editable;
|
|
30
|
+
actual run records stay read-only. Case-only renames work on macOS, rename
|
|
31
|
+
selects the old filename, and new-member names are checked before creation.
|
|
32
|
+
Non-UTF-8 files and validation in an empty workspace show friendly messages.
|
|
33
|
+
- Files changed on disk reload automatically when there are no unsaved edits.
|
|
34
|
+
The file tree refreshes automatically and after runs finish. macOS uses its
|
|
35
|
+
native Trash operation, preserving Finder's Put Back support. The runs list
|
|
36
|
+
keeps comparison controls aligned and offers column navigation on narrow screens.
|
|
37
|
+
- Live transcripts append new cards and populate read context on demand;
|
|
38
|
+
member-version history is cached in memory and invalidated when run files change.
|
|
39
|
+
The historical diagram helpers have been reformatted for readability.
|
|
40
|
+
- **Two email tools**: `read_email` hands a member one email from a folder of
|
|
41
|
+
`.eml` files in the package (`mailbox`, `max_chars`); `save_draft` saves a
|
|
42
|
+
reply in the run's `work/outbox/` and never sends it. By default a draft can
|
|
43
|
+
answer only the email's sender (`reply_only: false` lifts that).
|
|
44
|
+
- Runs started in the same second get their own folders instead of failing
|
|
45
|
+
with `FileExistsError`.
|
|
46
|
+
|
|
47
|
+
## 0.0.3
|
|
48
|
+
|
|
49
|
+
- **Backup models**: `model:` and `default_model:` may name several models,
|
|
50
|
+
separated by commas, in order of preference
|
|
51
|
+
(`default_model: openai/gpt-6-luna, anthropic/claude-sonnet-4-5`). Before the
|
|
52
|
+
first turn each member gets the first one available, for the whole run; the
|
|
53
|
+
console says so whenever a member is not on its first choice, and `run.json`,
|
|
54
|
+
`events.jsonl` and `teamrunner runs` record it. A profile's `name:` still
|
|
55
|
+
names one model.
|
|
56
|
+
- The examples name their model once, as `default_model:` in `team.yaml`.
|
|
57
|
+
- All six examples use Gemma 4 31B through OpenRouter with one API key.
|
|
58
|
+
The fact checker reads public sources instead of requiring Brave Search;
|
|
59
|
+
reference lookups use Crossref without a catalogue key. Reference lists
|
|
60
|
+
and findings are shared in the meeting messages so the next checker and
|
|
61
|
+
the editor can read them.
|
|
62
|
+
- Gemma 4 tool calls on OpenRouter no longer compete with strict JSON
|
|
63
|
+
formatting: during tool use the final format is requested in the prompt,
|
|
64
|
+
and the runner still validates the declared outputs.
|
|
65
|
+
- The README's tutorial and its five ideas hold only what works today; what is
|
|
66
|
+
designed but not built yet (experiments, the whiteboard) is in `PREVIEW.md`.
|
|
67
|
+
The README installs with uv or pip and needs one key.
|
|
68
|
+
- Comments in the examples, and the `.gitignore` `teamrunner new` writes, no
|
|
69
|
+
longer cite internal design notes.
|
|
70
|
+
|
|
71
|
+
## 0.0.2 — the first preview
|
|
72
|
+
|
|
73
|
+
What the roadmap calls 0.1, the org package: a folder of text files that gets
|
|
74
|
+
a job done.
|
|
75
|
+
|
|
76
|
+
- **Teams from files**: a folder with `team.yaml` is a team; each member is a
|
|
77
|
+
Markdown file (settings on top, the prompt as prose); `chair.md` is the
|
|
78
|
+
script of who acts when, with rounds and decisions; with no chair, members
|
|
79
|
+
take turns in name order. Model profiles in `models/` share a model and its
|
|
80
|
+
settings across a team.
|
|
81
|
+
- **Commands**: `new` (empty, `--from` an example or another team, or
|
|
82
|
+
`--describe` for a team written by a model), `validate` (with `--online`
|
|
83
|
+
to check the models), `explain`, `run` (with `--message`, `--file`,
|
|
84
|
+
`--task-file`; `--rehearse` runs the whole flow with no model and no cost),
|
|
85
|
+
`runs`, `show`, `gather`, `skill`.
|
|
86
|
+
- **Toolkeepers** hold the tools, with an access list and guardrails, and four
|
|
87
|
+
built-in tools: `web_search_brave`, `read_page`, `read_file`, and `run_code`
|
|
88
|
+
(Python in a WebAssembly sandbox).
|
|
89
|
+
- **Every run is logged** in the team's `runs/` folder: `run.json`,
|
|
90
|
+
`events.jsonl` and `output.md`, with costs, and the Python, LiteLLM and Deno
|
|
91
|
+
versions it ran on. Search results are read by the team but not kept,
|
|
92
|
+
unless `keep_results: true`.
|
|
93
|
+
- **Six examples ship**: `round_robin`, `multi_agent_debate`,
|
|
94
|
+
`orchestrators_lite`, `fact_checked`, `reference_check`, and
|
|
95
|
+
`hiring_committee` (a hidden-profile task, after Stasser and Titus 1985).
|
|
96
|
+
- Models from OpenAI, Anthropic and Google, and local models through Ollama.
|
|
97
|
+
macOS, Linux and Windows; Python 3.10 to 3.13.
|
|
98
|
+
- Licensed GPL-3.0-or-later; the examples, the files `new` writes and the
|
|
99
|
+
authoring guide are MIT-0.
|
|
100
|
+
|
|
101
|
+
## 0.0.1
|
|
102
|
+
|
|
103
|
+
Reserved the name on PyPI. No working code.
|
|
@@ -5,7 +5,17 @@ TeamRunner is licensed under GPL-3.0-or-later (see LICENSE). The shipped
|
|
|
5
5
|
examples, the files `teamrunner new` writes and the authoring guide are
|
|
6
6
|
MIT-0 (see examples/LICENSE).
|
|
7
7
|
|
|
8
|
-
TeamRunner bundles
|
|
8
|
+
TeamRunner bundles CodeMirror 5.65.21 (MIT), by Marijn Haverbeke and others,
|
|
9
|
+
for offline browser editing. Its licence and pinned source information are
|
|
10
|
+
in src/teamrunner/ui/static/vendor/LICENSE and SOURCE.txt.
|
|
11
|
+
The offline run viewer also bundles Marked 18.1.3 (MIT) and DOMPurify
|
|
12
|
+
3.4.17 (Apache-2.0 OR MPL-2.0). Their licences and pinned source information
|
|
13
|
+
are in the same vendor folder, under marked-* and dompurify-*.
|
|
14
|
+
|
|
15
|
+
Offline flow diagrams bundle Mermaid 12.3.0 (MIT); its licence, source
|
|
16
|
+
information and bundled dependency notices are in vendor/mermaid-*.
|
|
17
|
+
|
|
18
|
+
It depends on these packages, which
|
|
9
19
|
pip installs separately, each under its own licence:
|
|
10
20
|
|
|
11
21
|
LiteLLM MIT
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: teamrunner
|
|
3
|
-
Version: 0.0.
|
|
3
|
+
Version: 0.0.4
|
|
4
4
|
Summary: The experimentation platform for agent organizations — run multi-agent orchestrations and factorial experiments entirely from text files.
|
|
5
5
|
Project-URL: Homepage, https://github.com/teamrunner/teamrunner
|
|
6
6
|
Project-URL: Repository, https://github.com/teamrunner/teamrunner
|
|
@@ -53,7 +53,7 @@ experiments/advocate_model/ [v0.2]
|
|
|
53
53
|
└── experiment.yaml ← base: ../../debate — GPT vs. Claude as advocate, 20 replications
|
|
54
54
|
```
|
|
55
55
|
|
|
56
|
-
> Everything unmarked
|
|
56
|
+
> Everything unmarked works today, in the preview. Marked parts are designed but not built yet: **[v0.2]** arrives with experiments; **[v0.3]** with richer organizations (managers, private information, subteams, parallel work).
|
|
57
57
|
|
|
58
58
|
```bash
|
|
59
59
|
teamrunner run debate --message "Remote work makes teams more productive"
|
|
@@ -68,13 +68,14 @@ No Python. No graph-building API. No framework vocabulary. If you can edit a tex
|
|
|
68
68
|
- [Why we built it](#why-we-built-it)
|
|
69
69
|
- [The five ideas](#the-five-ideas)
|
|
70
70
|
- [Tutorial: your first debate in 10 minutes](#tutorial-your-first-debate-in-10-minutes)
|
|
71
|
-
- [
|
|
71
|
+
- [Use TeamRunner in your browser](#use-teamrunner-in-your-browser)
|
|
72
72
|
- [What lands in a run folder](#what-lands-in-a-run-folder)
|
|
73
73
|
- [What your runs keep, and where your words go](#what-your-runs-keep-and-where-your-words-go)
|
|
74
74
|
- [Going further: the worked examples](#going-further-the-worked-examples)
|
|
75
75
|
- [What TeamRunner is not](#what-teamrunner-is-not)
|
|
76
76
|
- [Status and roadmap](#status-and-roadmap)
|
|
77
77
|
- [Questions, bugs and ideas](#questions-bugs-and-ideas)
|
|
78
|
+
- [What's coming](PREVIEW.md): designed, not built yet
|
|
78
79
|
|
|
79
80
|
---
|
|
80
81
|
|
|
@@ -142,7 +143,7 @@ not your own opinion of the motion.
|
|
|
142
143
|
|
|
143
144
|
**2. A folder is a team if it contains `team.yaml`.** Everything inside is a member, and a member is one of three things: a **worker** (an agent with a goal), a **team** (**[v0.3]** a subfolder with its own `team.yaml` — just another node in its parent's flow: *its turn means running its own internal flow to completion*, like a subcommittee: it deliberates on its own whiteboard, and the parent hears only its report — `report_from:` in its `team.yaml` says whose words those are), or a **toolkeeper** (the member who holds the keys to a tool). Complexity is contained by hierarchy, not by syntax. There is no registry file listing your agents; the file browser is the org chart.
|
|
144
145
|
|
|
145
|
-
|
|
146
|
+
The whole folder — team and toolkeepers — is a **package**: copy it anywhere and it gets the same job done. The work itself comes from outside: you hand the team a topic when you run it, or it uses the default written in `team.yaml`.
|
|
146
147
|
|
|
147
148
|
**3. Every team has a chair, and there are no hidden if-statements.** Someone always decides who acts next, and TeamRunner makes you say who:
|
|
148
149
|
|
|
@@ -152,45 +153,40 @@ Tools are not members. An organization doesn't hand every worker every tool; a f
|
|
|
152
153
|
|
|
153
154
|
Every routing decision is logged as an event attributed to whoever made it. Which means *protocol vs. discretion* — a script vs. a manager exercising judgment — is itself something you can A/B test by swapping one file for the other (**[v0.3]**).
|
|
154
155
|
|
|
155
|
-
**4.
|
|
156
|
+
**4. Tools sit with a toolkeeper.** Tools are not members. An organization doesn't hand every worker every tool; a factory has a tool crib, with someone at the window who checks who may take what, and everyone else asks there. So the only members who touch tools are toolkeepers — a toolkeeper holds the guardrails (timeouts, no network, which folders), decides by a plain access list rather than a prompt, and its text body is simply what the team is told about it. Who may use a toolkeeper, and on what terms, is up to the toolkeeper's own access list — one toolkeeper can give different members different terms, the way a library gives professors and students different cards.
|
|
156
157
|
|
|
157
|
-
|
|
158
|
-
|---|---|---|---|
|
|
159
|
-
| **The whiteboard** (`whiteboard.yaml`) | the meeting, and the official record | everyone, unless a `read_by` line says otherwise | per item `written_by`; whatever anyone says is posted automatically; always logged |
|
|
160
|
-
| **Memory** | what you were in the room for | you | nobody — history is append-only |
|
|
158
|
+
**5. Every run is kept, and no agent can touch the record.** Every run gets its own folder in `runs/` with every prompt, everything said, every decision about who acts next, every tool call, the model that answered, tokens, time and cost (see [What lands in a run folder](#what-lands-in-a-run-folder)). The run log is the researcher's camera, outside the simulation: **no agent can ever read or write it** — a hard guarantee, without which every study of who knows what leaks.
|
|
161
159
|
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
Information asymmetry is not a feature we had to invent; it is a narrow `read_by`. Incentives are numeric items on the whiteboard. Private briefs are items only one party can read.
|
|
165
|
-
|
|
166
|
-
And one thing that is emphatically *not* a layer: the run log. It is the researcher's camera, outside the simulation. **No agent can ever read or write it** — a hard guarantee, without which every asymmetry design leaks.
|
|
167
|
-
|
|
168
|
-
**5. Experiments are overlays.** **[v0.2]** Your package stays clean and runnable. An `experiment.yaml` lives *outside* it, names it with `base:`, and declares factors as *patches* against it, plus replications and a seed. Single runs and 200-run factorials go through the same engine. When a variant wins, `teamrunner promote` copies the exact version that ran back into the package — same agent, new version — and `export` hands you a standalone copy for production.
|
|
160
|
+
Two more ideas are designed but not built yet — a whiteboard for who sees what, and experiments as overlays — and are described in [PREVIEW.md](PREVIEW.md).
|
|
169
161
|
|
|
170
162
|
---
|
|
171
163
|
|
|
172
164
|
## Tutorial: your first debate in 10 minutes
|
|
173
165
|
|
|
174
|
-
We'll build a three-agent debate, run it, give it rounds, a fact checker and the web
|
|
166
|
+
We'll build a three-agent debate, run it, give it rounds, a fact checker and the web. Zero code throughout. Turning it into a 40-run experiment comes with 0.2: see [PREVIEW.md](PREVIEW.md#experiments-v02).
|
|
175
167
|
|
|
176
|
-
>
|
|
168
|
+
> Every step works in a terminal, and from 0.0.4 also in your browser: see [Use TeamRunner in your browser](#use-teamrunner-in-your-browser).
|
|
177
169
|
|
|
178
170
|
### Step 0 — install
|
|
179
171
|
|
|
180
172
|
```bash
|
|
181
|
-
|
|
173
|
+
uv tool install teamrunner # uv fetches the right Python for you: https://docs.astral.sh/uv/
|
|
174
|
+
pip install teamrunner # or this, if you already have Python 3.10 or later
|
|
182
175
|
```
|
|
183
176
|
|
|
184
|
-
Put your
|
|
177
|
+
Put your key in TeamRunner's key file, `~/.teamrunner/.env` (in your home folder, outside every team). One key is enough: the one for the provider your team's model uses. This tutorial's team runs on OpenAI:
|
|
185
178
|
|
|
186
179
|
```
|
|
187
180
|
OPENAI_API_KEY=sk-...
|
|
188
|
-
ANTHROPIC_API_KEY=sk-ant-...
|
|
189
181
|
```
|
|
190
182
|
|
|
183
|
+
A team on another provider takes that provider's key instead (`ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, …). The shipped examples run on OpenRouter, so they need only `OPENROUTER_API_KEY`. If a key is missing, the run stops before spending anything and says which one. You can try every step without a key: `teamrunner run --rehearse` runs the whole team with no model, for free.
|
|
184
|
+
|
|
191
185
|
Keys stay with you, never with your teams: team folders get copied, zipped and sent to people, and a key inside one goes with it. If you already have a key set in your shell, TeamRunner offers to save it to the key file the first time it needs it (or to use it just this once). If it finds a `.env` inside a team folder, it offers to move the key out and delete it there. And it will **refuse to run** if it finds something that looks like an API key inside a spec file.
|
|
192
186
|
|
|
193
|
-
Models on your own machine work too, free: install [Ollama](https://ollama.com), pull a model (`ollama pull
|
|
187
|
+
Models on your own machine work too, free: install [Ollama](https://ollama.com), pull a model (`ollama pull gpt-oss:20b`), and write `default_model: ollama/gpt-oss:20b` in `team.yaml`. If Ollama runs somewhere other than this computer, put its address in the key file as `OLLAMA_API_BASE`. Choose one of about 20 billion parameters or more: the smallest models often fail at exactly what teams need most — answering in a fixed form and asking toolkeepers properly — so they test the wiring better than they stand in for a real team. Before the first turn of every run, TeamRunner checks that each model your team names is available and lists every problem it finds in one table, so you never pay for half a run. `teamrunner validate --online` runs the same check without a run.
|
|
188
|
+
|
|
189
|
+
Or open `teamrunner ui` and choose **Set up keys** after selecting a team. It saves keys to the same key file and shows whether each key is present, never its saved value. **Check keys and models online** is a separate check.
|
|
194
190
|
|
|
195
191
|
### Step 1 — make a team
|
|
196
192
|
|
|
@@ -206,14 +202,18 @@ Two other ways to start, when you'd rather not begin from nothing:
|
|
|
206
202
|
- `teamrunner new my_debate --from fact_checked` copies one of the examples that ship with TeamRunner (or any team folder you name), with fresh ids; each file remembers what it was copied from.
|
|
207
203
|
- `teamrunner new my_debate --describe "two debaters argue a topic, a fact checker checks each speech, and a judge decides"` asks a model to write the team for you. It checks its own work with `validate`, never overwrites files, tells you what it cost, and ends by printing `teamrunner explain`, so you can read what it made before you run it. (If you work with an AI coding assistant, `teamrunner skill` gives it the same instructions: it installs a skill for Claude Code and Codex, and `teamrunner skill --agents-md <folder>` writes an `AGENTS.md` for other assistants that read one.)
|
|
208
204
|
|
|
205
|
+
Or run `teamrunner ui` in the folder where you want to keep your teams. Choose **New team**, enter `debate`, and select an empty team or a shipped example. Select `team.yaml` on the left to edit it.
|
|
206
|
+
|
|
209
207
|
This tutorial starts from nothing, so you see every file.
|
|
210
208
|
|
|
211
|
-
`team.yaml` is the only required file. It marks the folder as a team
|
|
209
|
+
`team.yaml` is the only required file. It marks the folder as a team, says what the team does, and names the model its agents run on:
|
|
212
210
|
|
|
213
211
|
```yaml
|
|
214
212
|
# team.yaml
|
|
215
213
|
name: debate
|
|
216
214
|
description: Two debaters argue a topic; a judge decides
|
|
215
|
+
|
|
216
|
+
default_model: openai/gpt-6-luna
|
|
217
217
|
```
|
|
218
218
|
|
|
219
219
|
Two things worth knowing now:
|
|
@@ -225,11 +225,12 @@ Two things worth knowing now:
|
|
|
225
225
|
|
|
226
226
|
An agent file is Markdown. Frontmatter configures, body prompts.
|
|
227
227
|
|
|
228
|
+
In the browser, select the team, choose **New member** for each Markdown file below, and paste its contents into the editor. **Save** (Ctrl/Cmd+S), then use **Show as it will run** to read the instructions with inputs filled in.
|
|
229
|
+
|
|
228
230
|
```markdown
|
|
229
231
|
<!-- advocate.md -->
|
|
230
232
|
---
|
|
231
233
|
description: Argues in favor of the topic
|
|
232
|
-
model: openai/gpt-6-luna
|
|
233
234
|
---
|
|
234
235
|
You are debating in favor of this topic:
|
|
235
236
|
|
|
@@ -244,7 +245,6 @@ recent point before adding your own. Be persuasive but honest.
|
|
|
244
245
|
<!-- critic.md -->
|
|
245
246
|
---
|
|
246
247
|
description: Argues against the topic
|
|
247
|
-
model: openai/gpt-6-luna
|
|
248
248
|
---
|
|
249
249
|
You are debating against this topic:
|
|
250
250
|
|
|
@@ -259,7 +259,6 @@ Be persuasive but honest.
|
|
|
259
259
|
<!-- judge.md -->
|
|
260
260
|
---
|
|
261
261
|
description: Decides who argued better
|
|
262
|
-
model: openai/gpt-6-luna
|
|
263
262
|
output:
|
|
264
263
|
winner: [advocate, critic]
|
|
265
264
|
reasoning: ""
|
|
@@ -275,7 +274,7 @@ explain your decision in one paragraph.
|
|
|
275
274
|
|
|
276
275
|
Four things just happened:
|
|
277
276
|
|
|
278
|
-
**
|
|
277
|
+
**The team names the model once.** No agent above names a model, so all three run on `default_model:` from `team.yaml`. An agent that should run on a different one says so itself (`model: anthropic/…`), and its own line wins. A model with its settings can live in a profile, `models/careful.yaml`, named as `default_model: careful.yaml` or `model: careful.yaml`. `teamrunner show judge` prints exactly which model and settings the judge will run with, and where each came from. `examples/orchestrators_lite` does both: its five engineers share one profile, `models/engineer.yaml`, named once as the team's `default_model:`. Models churn, so either line can name backups in order of preference, separated by commas: `default_model: openai/gpt-6-luna, anthropic/claude-sonnet-4-5`. Before the first turn TeamRunner picks the first one available, keeps it for the whole run, and says so whenever it isn't the first (`↪ judge runs on anthropic/claude-sonnet-4-5, its 2nd choice — …`); the run's record says so too.
|
|
279
278
|
|
|
280
279
|
**Alphabetical order is the flow.** With no `chair.md`, members act in alphabetical filename order, one turn each — here `advocate → critic → judge`. That is also why the files above have no `name:` line: an agent's name is its filename. The folder listing *is* the running order, visible in any file browser. Rename to reorder.
|
|
281
280
|
|
|
@@ -322,6 +321,8 @@ teamrunner run . --message "Four-day weeks make teams more productive"
|
|
|
322
321
|
|
|
323
322
|
The first runs the default topic; the second replaces it. Either way, the run log records the topic and whether it was given or the default.
|
|
324
323
|
|
|
324
|
+
Or do the editing, validation and running in `teamrunner ui .`: **Validate** checks the same files, and clicking a problem opens its file at the line to fix. Choose **Run…**, keep **Rehearse (free, no model)** checked for a free test, and start. To use a model, clear that checkbox; choose team defaults, a message, a file or a task file. The page shows the conversation as it happens and a **Stop run** button.
|
|
325
|
+
|
|
325
326
|
What you see while it runs (the words are invented; the layout is the real one):
|
|
326
327
|
|
|
327
328
|
```
|
|
@@ -349,6 +350,8 @@ Each turn shows who spoke, what they decided, how long it took, what it cost and
|
|
|
349
350
|
|
|
350
351
|
Later, `teamrunner runs` lists every run of this package — when, which team, the input, the result (`winner = advocate`), status, time and cost — and `teamrunner runs latest` (or `teamrunner runs 14:32`) shows one again exactly as it looked while it ran.
|
|
351
352
|
|
|
353
|
+
In the browser, choose **Runs** to reopen the conversation, output, path or record. Select two runs with the checkboxes and choose **Compare selected runs** to see member versions, outputs, cost and paths side by side.
|
|
354
|
+
|
|
352
355
|
That's a working multi-agent system: four text files, one command.
|
|
353
356
|
|
|
354
357
|
### Step 5 — take control of the flow
|
|
@@ -372,7 +375,7 @@ The flow grammar is deliberately tiny — it is a whiteboard sketch that happens
|
|
|
372
375
|
- **[v0.3]** `[a, b]` — **at the same time**: `editor -> [reviewer_1, reviewer_2] -> editor` sends the paper to both reviewers at once, and the editor waits for both reports. Neither sees the other's work in progress.
|
|
373
376
|
- `done` ends the flow — or, inside a named block, just that block; `stop` ends the whole run immediately.
|
|
374
377
|
|
|
375
|
-
Now the debate runs `advocate → critic → advocate → critic → advocate → critic → judge`. To see it as a picture, `teamrunner explain --diagram` opens the flow as a flowchart in your browser.
|
|
378
|
+
Now the debate runs `advocate → critic → advocate → critic → advocate → critic → judge`. To see it as a picture, `teamrunner explain --diagram` opens the flow as a flowchart in your browser. Or, in the local interface, refresh the file tree after adding `chair.md`, select it, and choose **Flow diagram**. Save an edit to redraw the diagram and plain-words summary.
|
|
376
379
|
|
|
377
380
|
### Step 6 — add a fact checker and a decision
|
|
378
381
|
|
|
@@ -382,7 +385,6 @@ Suppose we want each speech fact-checked before it stands. Write a fact checker
|
|
|
382
385
|
<!-- fact_checker.md -->
|
|
383
386
|
---
|
|
384
387
|
description: Checks the factual claims in the latest speech
|
|
385
|
-
model: openai/gpt-6-luna
|
|
386
388
|
output:
|
|
387
389
|
verdict: [pass, fail]
|
|
388
390
|
reasons: ""
|
|
@@ -468,114 +470,36 @@ A toolkeeper step means the member just before it must end its turn with a reque
|
|
|
468
470
|
|
|
469
471
|
`teamrunner explain .` prints, in plain English, who may use which toolkeeper and on what terms.
|
|
470
472
|
|
|
471
|
-
TeamRunner 0.1 has
|
|
472
|
-
|
|
473
|
-
### Step 8 — private information and shared records **[v0.3]**
|
|
474
|
-
|
|
475
|
-
The debate doesn't need this, but the moment you study organizations you will. `whiteboard.yaml` is the team's whiteboard: declared, typed items, each with who may read and write it. Everything anyone says is posted there too, open to all unless a `read_by` line says otherwise.
|
|
476
|
-
|
|
477
|
-
```yaml
|
|
478
|
-
# whiteboard.yaml
|
|
479
|
-
|
|
480
|
-
# Bare list = a choice, starting at the first option.
|
|
481
|
-
# Read by everyone, written by the chair — those are the defaults.
|
|
482
|
-
stage: [opening, rebuttal, counter, closing, done]
|
|
483
|
-
|
|
484
|
-
# A bare value = a starting value, read and written by the defaults.
|
|
485
|
-
rounds_left: 3
|
|
486
|
-
|
|
487
|
-
# Spell it out only when you change who reads or writes it.
|
|
488
|
-
score:
|
|
489
|
-
starts_as: 0
|
|
490
|
-
written_by: [judge]
|
|
491
|
-
|
|
492
|
-
# Information asymmetry IS a read permission.
|
|
493
|
-
buyer_ceiling:
|
|
494
|
-
starts_as: 850
|
|
495
|
-
read_by: [buyer]
|
|
496
|
-
```
|
|
497
|
-
|
|
498
|
-
Every field is declared up front — no ad-hoc keys appearing mid-run. Every write is harness-mediated and logged as a `state_write` event: an agent listed in `written_by` returns the new value as a declared output field; nothing pokes state directly. Agents reference readable fields in their prompts as `{{whiteboard.buyer_ceiling}}`, and the validator checks that permission before you spend a cent.
|
|
499
|
-
|
|
500
|
-
This one file is doing a surprising amount of work. Private valuations, confidential negotiating briefs, payoffs and incentives, task inputs, an artifact handed from one department to the next — all of them are just whiteboard items with the right permissions.
|
|
473
|
+
TeamRunner 0.1 has six built-in tools: `web_search_brave` (Brave Search), `read_page` (a link in, the page's text out), `read_file` (a file the run was given with `--file`, its text out), `run_code` (Python in a sandbox: no internet unless the toolkeeper allows it, and files written only in the run's own work folder), `read_email` (one email from a folder of `.eml` files in the package) and `save_draft` (a reply saved in the run's `work/outbox/`, never sent, addressed only to the email's sender unless `reply_only: false`). The sandbox is Python running in WebAssembly, and it installs with TeamRunner — nothing else to set up; it can use packages built for it (numpy, pandas, scipy, sympy and many more), but not every package. `examples/fact_checked` gives the fact checker a second toolkeeper, `sandbox.md`, so it can check the sums in a claim by running them. A toolkeeper holds one tool in 0.1, so its limits always mean one thing; a member who needs two tools asks two toolkeepers, as the fact checker does here. `examples/reference_check` uses `read_page` and `read_file`: it checks every reference in a paper.
|
|
501
474
|
|
|
502
475
|
---
|
|
503
476
|
|
|
504
|
-
##
|
|
477
|
+
## Use TeamRunner in your browser
|
|
505
478
|
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
Don't touch the debate. Make a folder next to it with one file:
|
|
509
|
-
|
|
510
|
-
> **[v0.2]** Experiments are built after the package is right. The file below is the agreed shape: it lives *outside* the package (`experiments/advocate_model/experiment.yaml`) and names it with `base:`.
|
|
511
|
-
|
|
512
|
-
```yaml
|
|
513
|
-
# experiments/advocate_model/experiment.yaml
|
|
514
|
-
name: advocate-model-ab
|
|
515
|
-
base: ../../debate
|
|
516
|
-
task: tasks/remote_work
|
|
517
|
-
|
|
518
|
-
factors:
|
|
519
|
-
advocate_model:
|
|
520
|
-
- label: gpt
|
|
521
|
-
patch: {advocate.model: "openai/gpt-6-luna"}
|
|
522
|
-
- label: claude
|
|
523
|
-
patch: {advocate.model: "anthropic/claude-sonnet-4-5"}
|
|
524
|
-
|
|
525
|
-
replications: 20 # 2 conditions × 20 = 40 runs
|
|
526
|
-
seed: 42
|
|
527
|
-
|
|
528
|
-
limits:
|
|
529
|
-
max_concurrent: 4
|
|
530
|
-
max_cost_usd: 40.00
|
|
531
|
-
```
|
|
479
|
+
From 0.0.4, open a team folder with:
|
|
532
480
|
|
|
533
481
|
```bash
|
|
534
|
-
teamrunner
|
|
482
|
+
teamrunner ui debate
|
|
535
483
|
```
|
|
536
484
|
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
**Factors are patches, so the base spec stays clean.** `advocate.model` means "the `model` field of the component `advocate`". Patch paths address any field of any component, including the prompt body (the field is called `prompt`), and any value can be `file:<path>` to swap in a whole file. `patch: {}` is the *unmodified base spec* — the control condition, written honestly as a level rather than smuggled in as "the normal run."
|
|
540
|
-
|
|
541
|
-
**Anything in a file is a factor.** That's not a slogan, it's the mechanism. A few real examples from our worked recasts:
|
|
485
|
+
`teamrunner ui` opens the current folder when you leave out `debate`; `teamrunner gui` is an alias. To create teams, open their parent folder instead. The command opens a browser tab and prints its local address. Keep that terminal running while you work; press Ctrl+C there to stop the server. If the tab does not open, paste the full printed address, including its token, into your browser.
|
|
542
486
|
|
|
543
|
-
|
|
544
|
-
# A prompt/strategy A/B — same agent, two personalities
|
|
545
|
-
- {label: tough, patch: {buyer.prompt: file:variants/buyer_tough.md}}
|
|
546
|
-
|
|
547
|
-
# A treatment: one member is secretly self-interested
|
|
548
|
-
- {label: saboteur, patch: {council.prompt: file:variants/saboteur.md}}
|
|
487
|
+
The page works on the same files as the CLI. Choose a team or file on the left. You can create an empty team or copy a shipped example, add members, edit and save, rename members, or move files to the system trash. The editor warns about unsaved edits and files changed by another editor. Validation points to the file, line and suggested fix. **Show as it will run** fills in a member's instructions; **Flow diagram** shows the team's flow and summary. For a ready-made flow to edit, start from the `fact_checked` example.
|
|
549
488
|
|
|
550
|
-
|
|
551
|
-
# same reviewers, same paper — only who can read whose review changes
|
|
552
|
-
- label: public
|
|
553
|
-
patch:
|
|
554
|
-
whiteboard.said.reviewer_1.read_by: everyone
|
|
555
|
-
whiteboard.said.reviewer_2.read_by: everyone
|
|
556
|
-
whiteboard.said.reviewer_3.read_by: everyone
|
|
557
|
-
```
|
|
489
|
+
**Run…** accepts team defaults, a message, a file or a YAML task file. Rehearsal is selected by default for a team with no prior run; it uses placeholder words and random answers, needs no key and costs nothing. A model run uses your configured provider and its normal charges. The interface validates before starting, allows one active run at a time, streams the recorded events, and can stop the run cleanly. Key setup uses `~/.teamrunner/.env`; stored key values are never displayed.
|
|
558
490
|
|
|
559
|
-
|
|
491
|
+
**Runs** is the archive. Filter by team or status, search the message and output, then open a run:
|
|
560
492
|
|
|
561
|
-
|
|
493
|
+
- **Conversation:** member messages, decisions, tool calls, retries, tokens, models and cost, with a toggle for what each member could read.
|
|
494
|
+
- **Path:** the saved flow with taken arrows highlighted and transition counts. Older logs without a saved engine path show only observed completed-turn transitions, labelled accordingly.
|
|
495
|
+
- **Output:** the rendered `output.md`.
|
|
496
|
+
- **Record:** inputs, numbered member versions, recorded settings and files. Version numbers come from the available run history.
|
|
562
497
|
|
|
563
|
-
|
|
498
|
+
Select two runs to compare their outputs, cost, time, paths and member versions. Exact member source diffs are available for new logs that record the source and its fingerprint; older logs show their recorded prompts, which may include substituted inputs.
|
|
564
499
|
|
|
565
|
-
|
|
500
|
+
**Re-run with these inputs…** uses the current team with the old run's input values and copied input files. It does not restore old member versions. **Edit a copy of the saved team…** appears only when a run contains `resolved/team.yaml`; it creates an independent team with fresh IDs and lineage. Open the teams' parent folder to give that copy a place outside the original team. Ordinary runs do not yet save a complete resolved team, so their original files cannot be reconstructed from prompts alone.
|
|
566
501
|
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
```bash
|
|
570
|
-
teamrunner runs # browse what you've run
|
|
571
|
-
teamrunner export --experiment advocate-model-ab --format csv
|
|
572
|
-
```
|
|
573
|
-
|
|
574
|
-
Long format, one row per observation, straight into R, Stata, or pandas. Your analysis never lives inside our dashboard — that's a promise, not an oversight.
|
|
575
|
-
|
|
576
|
-
The declarative metrics layer (ground truth, LLM-as-judge with a separately pinned judge model, cost and time) is specified but its file format is still open. Today's `metrics:` references in the examples are placeholders.
|
|
577
|
-
|
|
578
|
-
---
|
|
502
|
+
Every run has **Open folder** and raw-file download links. The editor stays inside the folder you opened, and run records remain read-only. The server listens only on this computer at `127.0.0.1`, with a random access token. Editor, Markdown and diagram assets are bundled for offline use; model calls and online checks still need their usual services. The page adds no database or separate project format, and shows equivalent CLI commands alongside actions.
|
|
579
503
|
|
|
580
504
|
## What lands in a run folder
|
|
581
505
|
|
|
@@ -597,12 +521,12 @@ runs/01JZXG7T3M9QKWP5RVB2C8HD6N/
|
|
|
597
521
|
|
|
598
522
|
| Event | What it records |
|
|
599
523
|
|---|---|
|
|
600
|
-
| `run_started` | the inputs, and every member exactly as it will run — model, settings, prompt |
|
|
524
|
+
| `run_started` | the inputs, and every member exactly as it will run — model, settings, prompt; from 0.0.4, also each member's source text, content fingerprints and the declared flow graph |
|
|
601
525
|
| `agent_message` | from, **to** (the *realized* topology), stage, attempt, **what it could read**, everything it said, typed outputs, the model actually used, tokens, cost, time |
|
|
602
526
|
| `route_decision` | who decided, which option, which arrow — the audit trail for "no hidden if-statements" |
|
|
603
527
|
| `tool_call` | who asked, which toolkeeper, the toolkeeper's yes or no, tool, arguments, the full result (for a search, how many results came back, unless `keep_results: true`) |
|
|
604
528
|
| `state_write` **[v0.3]** | item, old → new, writer |
|
|
605
|
-
| `check`, `retry`, `warning`, `error`, `run_finished` | the pre-run check, retries, warnings, failures, and the totals |
|
|
529
|
+
| `check`, `retry`, `warning`, `error`, `run_finished` | the pre-run check, retries, warnings, failures, and the totals; from 0.0.4, `run_finished` also saves the path taken, for arrow counts |
|
|
606
530
|
|
|
607
531
|
A field never changes its meaning between versions — later versions only add — so an analysis script you write today keeps working.
|
|
608
532
|
|
|
@@ -634,7 +558,17 @@ replays that exact spec. Every run folder is a self-contained replication packag
|
|
|
634
558
|
|
|
635
559
|
## Going further: the worked examples
|
|
636
560
|
|
|
637
|
-
Six examples ship with TeamRunner, in `examples/`, simplest first — each adds something the one before it doesn't have. Copy one by name to start from it: `teamrunner new my_team --from orchestrators_lite`. Most are published multi-agent papers or projects, recast as TeamRunner files.
|
|
561
|
+
Six examples ship with TeamRunner, in `examples/`, simplest first — each adds something the one before it doesn't have. Copy one by name to start from it: `teamrunner new my_team --from orchestrators_lite`. Most are published multi-agent papers or projects, recast as TeamRunner files. All six use `openrouter/google/gemma-4-31b-it` and need only an OpenRouter API key, with credits for model usage. They do not require OpenAI, Anthropic, Brave or catalogue keys.
|
|
562
|
+
|
|
563
|
+
Set the key outside the team folder (never commit it):
|
|
564
|
+
|
|
565
|
+
```bash
|
|
566
|
+
export OPENROUTER_API_KEY="your-openrouter-key"
|
|
567
|
+
teamrunner new my_debate --from round_robin
|
|
568
|
+
teamrunner run my_debate
|
|
569
|
+
```
|
|
570
|
+
|
|
571
|
+
On PowerShell, set `$env:OPENROUTER_API_KEY="your-openrouter-key"` instead. You can also keep `OPENROUTER_API_KEY` in `~/.teamrunner/.env`. Add `--rehearse` to check a flow without calling a model; a rehearsal does not verify live model or tool behavior. The two source-checking examples read public pages and Crossref without extra keys; access restrictions, rate limits and missing abstracts or PDFs can still limit their evidence. The fact checker's Python sandbox downloads its runtime on first use. The code-review example uses demo model settings rather than the source paper's model.
|
|
638
572
|
|
|
639
573
|
| Example | Recast of | What it adds |
|
|
640
574
|
|---|---|---|
|
|
@@ -642,7 +576,7 @@ Six examples ship with TeamRunner, in `examples/`, simplest first — each adds
|
|
|
642
576
|
| **`multi_agent_debate/`** | muthuspark/multi-agent-debate | A chair: the debate as a script, with a repeated block of rounds. |
|
|
643
577
|
| **`hiring_committee/`** | Stasser & Titus 1985 (hidden profile) | A committee picks a candidate when the key facts are spread across its members: facts only one member knows, kept in that member's own file; a stage in the chair; a clerk who records the decision; and both meeting formats in one `chair.md`, one commented out. |
|
|
644
578
|
| **`orchestrators_lite/`** | Fukui 2026 (lite) | Five engineers review code against planted bugs: a model profile the whole team shares, and two inputs. |
|
|
645
|
-
| **`fact_checked/`** | — | The debate with a fact checker: decisions that send a speech back, counted tries, and toolkeepers — a
|
|
579
|
+
| **`fact_checked/`** | — | The debate with a fact checker: decisions that send a speech back, counted tries, and toolkeepers — a public-source reader with different terms for different members, a required source-reading step, and a Python sandbox. |
|
|
646
580
|
| **`reference_check/`** | sciwrite-lint (Samsonau 2026) | Checks every reference in a paper: that it exists, that its details are right, and that the cited paper says what the citing sentence claims. A paper given as a link or a file (`--file`), toolkeepers holding `read_page` and `read_file`, the same tool under two toolkeepers with different `domains`, a decision that can stop the run, and batches of ten as a loop with a counted exit. |
|
|
647
581
|
|
|
648
582
|
**Coming with 0.3**, recast on paper and waiting for the features they need: **Deb8flow** (subteams — each side a debater plus its own fact checker), **NegotiationGym** (private valuations, a loop either side can end), **LLM-Deliberation** (four parties, confidential briefs, one secretly given a hidden agenda), **AgentReview** (three reviewers at once, sealed reviews — do reviewers herd when reviews are public?), **ChatDev** (a three-level company, each department on its own whiteboard), and the full **Invisible Orchestrators** study (a hidden orchestrator whose words reach the team only as unsigned "policy updates").
|
|
@@ -665,15 +599,16 @@ The recasting campaign is the point, not decoration: each one is run *before* th
|
|
|
665
599
|
|
|
666
600
|
## Status and roadmap
|
|
667
601
|
|
|
668
|
-
**Preview.** What the roadmap calls 0.1 is out as a soft launch, numbered 0.0.x: every command in the tutorial not
|
|
602
|
+
**Preview.** What the roadmap calls 0.1 is out as a soft launch, numbered 0.0.x: every command in the tutorial works; what is designed but not built yet is in [PREVIEW.md](PREVIEW.md). It may still change in ways that break earlier teams and runs until 0.1.0 is announced. What changed in each version is in the [changelog](https://github.com/teamrunner/teamrunner/blob/main/CHANGELOG.md).
|
|
669
603
|
|
|
670
604
|
| Version | |
|
|
671
605
|
|---|---|
|
|
672
606
|
| **Design** | Spec drafted; paper recasts written as pure text ✅ |
|
|
673
607
|
| **0.1 · The org package** ← now: in preview as 0.0.x | A folder that gets a job done: validation with friendly errors, its own runner, the event log, toolkeepers and four built-in tools, rehearsals, `run`, `runs`, `explain`, `new`, the authoring skill ✅ |
|
|
608
|
+
| **Before 0.2 · Local browser interface** | In 0.0.4 ✅ `teamrunner ui` / `gui`: edit, validate, rehearse or run, browse recorded conversations and paths, compare two runs |
|
|
674
609
|
| **0.2 · Experiments** | Experiments outside the package (`base:`), factorial designs, seeded replications, manifests, response cache, the answer-key metric, `teamrunner experiment`, export; `promote` / `export`. First real result: leader vs. flat |
|
|
675
610
|
| **0.3 · Richer organizations** | Managers and rules-table chairs, `whiteboard.yaml` (private information, sealed and conditional visibility, addressed messages), subteams, parallel work, announcers; the recasts run for real, and the Invisible Orchestrators replication |
|
|
676
|
-
| **0.4+ · Later** | LLM-as-judge metrics,
|
|
611
|
+
| **0.4+ · Later** | LLM-as-judge metrics, communication-network statistics and visualization, template library of canonical org structures, human-in-the-loop, automated improvement, running packages in production |
|
|
677
612
|
|
|
678
613
|
Human-in-the-loop nodes (`kind: human` — the runner pauses and asks a person) are for mixed human–agent studies.
|
|
679
614
|
|
|
@@ -701,7 +636,7 @@ Inside a package, these names are reserved. **Every one of them is optional exce
|
|
|
701
636
|
| `chair.md` | How a protocol team is run: the script, as the file's body, or a `kind: rules` table **[v0.3]** | Everyone may talk to everyone; members act in alphabetical filename order, one turn each |
|
|
702
637
|
| `manager.md` **[v0.3]** | How a managed team is run: an LLM that decides who acts next; only it may use input placeholders. A team has at most one of `chair.md` and `manager.md` | — |
|
|
703
638
|
| `whiteboard.yaml` **[v0.3]** | Who sees what: declared items with `read_by` / `written_by`, and who can read what each member says (`said`) | Everyone reads everything said; built-in items only (e.g. `stage`) |
|
|
704
|
-
| `tools/` | Reserved for a package's own tools, which come after 0.1. In 0.1 toolkeepers use the built-ins: `web_search_brave`, `read_page`, `read_file`, `run_code` | Built-ins only |
|
|
639
|
+
| `tools/` | Reserved for a package's own tools, which come after 0.1. In 0.1 toolkeepers use the built-ins: `web_search_brave`, `read_page`, `read_file`, `run_code`, `read_email`, `save_draft` | Built-ins only |
|
|
705
640
|
| `models/` | Model profiles: a model and its settings in one `.yaml`, named by agents as `model: careful.yaml` or by `team.yaml` as `default_model:` | Agents name provider models directly |
|
|
706
641
|
| `runs/` | Where every run of this package is logged, whichever team you started; git-ignored | Created on the first run |
|
|
707
642
|
| *subfolder with `team.yaml`* **[v0.3]** | A subteam — a node in the parent flow, with its own whiteboard; the parent hears only its report (`report_from:`) | — |
|