tare-cli 0.2.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,505 @@
1
+ Metadata-Version: 2.5
2
+ Name: tare-cli
3
+ Version: 0.2.0
4
+ Summary: Prove your coding agent starts without your personal setup before you measure it.
5
+ Project-URL: Homepage, https://github.com/a-ratz/tare
6
+ Project-URL: Repository, https://github.com/a-ratz/tare
7
+ Project-URL: Issues, https://github.com/a-ratz/tare/issues
8
+ Project-URL: Concept, https://github.com/a-ratz/tare/blob/main/CONCEPT.md
9
+ Author: Andre Ratzenberger
10
+ License-Expression: MIT
11
+ License-File: LICENSE
12
+ Keywords: agent-evaluation,ai-agents,claude-code,codex,coding-agents,evals,llm-evaluation,sandbox
13
+ Classifier: Development Status :: 3 - Alpha
14
+ Classifier: Environment :: Console
15
+ Classifier: Intended Audience :: Developers
16
+ Classifier: Operating System :: POSIX :: Linux
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3.11
19
+ Classifier: Programming Language :: Python :: 3.12
20
+ Classifier: Programming Language :: Python :: 3.13
21
+ Classifier: Topic :: Software Development :: Testing
22
+ Requires-Python: >=3.11
23
+ Description-Content-Type: text/markdown
24
+
25
+ <div align="center">
26
+ <picture>
27
+ <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/a-ratz/tare/main/.github/logo-dark.svg">
28
+ <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/a-ratz/tare/main/.github/logo-light.svg">
29
+ <img alt="tare" src="https://raw.githubusercontent.com/a-ratz/tare/main/.github/logo-light.svg" width="260">
30
+ </picture>
31
+
32
+ <p>Prove your coding agent starts without your personal setup before you measure it.</p>
33
+ </div>
34
+
35
+ <div align="center">
36
+
37
+ [![License: MIT][license-shield]][license-url]
38
+ [![PyPI][version-shield]][version-url]
39
+ [![Python 3.11+][python-shield]][python-url]
40
+ [![Linux | WSL][platform-shield]][platform-url]
41
+ [![Claude Code | Codex | Pi | Antigravity][agent-shield]][agent-url]
42
+
43
+ </div>
44
+
45
+ <div align="center">
46
+ <a href="https://github.com/a-ratz/tare#install">Install</a> &middot;
47
+ <a href="#quick-start">Quick start</a> &middot;
48
+ <a href="#features">Features</a> &middot;
49
+ <a href="#how-it-works">How it works</a> &middot;
50
+ <a href="https://github.com/a-ratz/tare/blob/main/CONCEPT.md">Concept</a> &middot;
51
+ <a href="https://github.com/a-ratz/tare/issues/new?template=bug_report.md">Report Bug</a>
52
+ </div>
53
+
54
+ <br>
55
+
56
+ ---
57
+
58
+ ## Why tare?
59
+
60
+ You test a skill by comparing agent runs with the skill against runs without it. On your own
61
+ machine, both runs also carry your personal setup: your global instructions, memories, installed
62
+ skills, MCP servers and connected accounts. The comparison then measures your machine, not the
63
+ skill.
64
+
65
+ A separate config directory (`CLAUDE_CONFIG_DIR` for Claude Code, `CODEX_HOME` for Codex) does not
66
+ fix this. With Claude Code, the login alone brings your connected accounts (mail, calendar,
67
+ documents), your account's skills and your email address into the session. Evaluation frameworks
68
+ that drive agent CLIs, such as UiPath's coder_eval, leave it to you to provide a clean
69
+ environment, and none of them checks it.
70
+
71
+ tare starts the agent in a clean environment and proves that it is clean before every run. It
72
+ works with four coding agents on Linux and WSL: Claude Code, Codex, Pi
73
+ ([`@earendil-works/pi-coding-agent`](https://www.npmjs.com/package/@earendil-works/pi-coding-agent))
74
+ and Google's Antigravity CLI (`agy`).
75
+
76
+ ## The words tare uses
77
+
78
+ The name comes from weighing. You tare a scale, which sets it to zero, before you weigh
79
+ something. tare does the same for an agent before you measure it.
80
+
81
+ | Word | Meaning |
82
+ |---|---|
83
+ | room | An isolated environment for one agent run. It has an empty home directory, a fresh copy of your login (the file in which the agent keeps your sign-in) and only four basic environment variables from your shell (an allowlist), plus its own `HOME` and `PATH`. Your project appears at `/work`. tare builds the room with [bubblewrap](https://github.com/containers/bubblewrap). |
84
+ | probe | The test that tare runs before an agent starts. It makes no paid model calls. The agent talks to a fake model server on your machine, which records what the agent sends. |
85
+ | reading | The result of a probe, one of three. A clean room reads `tare: 0.00`, the zeroed scale. A room with leaks reads, for example, `tare: 4 leaks`. A probe that could not see your setup reads `tare: not proven (blind)`. There are no values in between. |
86
+ | leak | Something of your personal setup that reaches the room, for example your global instructions. The reading names each leak and the file it came from. |
87
+ | declared | Something of yours that the room holds on purpose, for example the copy of your login. The reading lists it, but it is not a leak and does not stop the run. |
88
+ | dirty twin | The same agent run in your real setup, outside the room. It is part of the probe and proves that the probe can see your setup at all. |
89
+
90
+ ## Features
91
+
92
+ - **Checks the room before every run.** The agent starts only when the probe reads
93
+ `tare: 0.00`. Otherwise you see every leak with the file it came from.
94
+ - **Measures what the model receives.** tare does not ask the agent about its context. It records
95
+ the request that the agent sends to its model. An agent asked about its own context can refuse
96
+ or be wrong.
97
+ - **Tells you when it cannot see.** tare looks at your files to learn what your setup holds, for
98
+ example a global instructions file or installed skills. If the dirty twin's request lacks one of
99
+ these, the probe could not see it. A clean room would then prove nothing. tare runs the dirty
100
+ twin once more, and if that kind of context is still missing, the reading is
101
+ `tare: not proven (blind)`.
102
+ - **Keeps your files out of reach.** Your home directory, the credentials of your other tools
103
+ and the secret environment variables of your shell do not exist inside the room.
104
+ - **Keeps your own login working.** Each room gets a fresh copy of your login, which tare deletes
105
+ after the run. Claude Code may replace its login token when it renews it, which could log you
106
+ out outside tare. That has not been tested, so tare refuses to start when the copy would need
107
+ renewal within the next hour. Start the agent once outside tare, which renews
108
+ your login, and try again.
109
+ - **Passes all your arguments to the agent.** Interactive or one-shot, the agent runs as it does
110
+ outside tare.
111
+ - **Compares skills, plugins and models.** `tare calibrate` starts each side (an agent with its
112
+ arguments) several times from scratch, each in its own room. It reports how often each side
113
+ passes a check, a shell command such as your test suite.
114
+ - **Finds out whether the workspace (the run's copy of your project) or the agent caused a
115
+ failure.** `tare swap` lets two agents
116
+ continue each other's unfinished work and compares the results.
117
+ - **Looks for the step where a failed run went wrong** (experimental). `tare cliff` restarts a
118
+ failed run from its saved steps (one per tool call) and names the step after which it no
119
+ longer succeeds.
120
+
121
+ ## When to use
122
+
123
+ | Use tare when | Look elsewhere when |
124
+ |---|---|
125
+ | you compare agent runs with and without a skill, plugin or prompt | you need protection from a hostile agent, because the room has network access and a working login |
126
+ | you benchmark agents on your own machine | you are on macOS (a room is in progress, not released yet) |
127
+ | you want a fresh-machine run without a fresh machine | you log in with an API key only (untested) |
128
+ | an agent failed a task and you want to know which step lost it | you want a single run explained without running the agent again, because Cliff runs it many times |
129
+
130
+ ## Install
131
+
132
+ | Requirement | Notes |
133
+ |---|---|
134
+ | Linux or WSL2 | tare builds the room with user namespaces |
135
+ | [bubblewrap](https://github.com/containers/bubblewrap) | `sudo apt install bubblewrap` |
136
+ | at least one of the four agents | on `PATH` and logged in. Codex must resolve to a path under `/usr` for now, because tare mounts Codex's npm package and Node.js from there. If `which codex` points elsewhere, for example into nvm, install it with the system's npm (`sudo npm install -g @openai/codex`) and put `/usr/bin` first on `PATH`. The other agents can live anywhere on `PATH`. |
137
+ | Google Chrome, only for `tare judge` | `google-chrome` on `PATH`, installed under `/usr` or `/opt`. `tare judge` scores web pages and takes their screenshots with it. |
138
+ | [uv](https://docs.astral.sh/uv/) or [pipx](https://pipx.pypa.io/) | Python 3.11 or newer |
139
+
140
+ Install tare from PyPI. The package is `tare-cli`, the command is `tare`:
141
+
142
+ ```bash
143
+ uv tool install tare-cli # or: pipx install tare-cli
144
+ ```
145
+
146
+ To work on tare itself, clone the repository and run it from the checkout with `uv run tare ...`:
147
+
148
+ ```bash
149
+ git clone https://github.com/a-ratz/tare.git && cd tare
150
+ uv run tare --version
151
+ ```
152
+
153
+ ## Quick start
154
+
155
+ With tare installed:
156
+
157
+ ```bash
158
+ tare probe claude
159
+ ```
160
+
161
+ ```text
162
+ tare probe · claude 2.1.289 · /home/me/tare
163
+ control dirty twin shows: instructions, home path, mcp, skills, plugins, email, reach
164
+ leaks none
165
+ declared
166
+ email account email subscription login
167
+ credentials /home/tare/.claude-config/.credentials.json copy of the login, needed by the CLI
168
+ tare: 0.00
169
+ ```
170
+
171
+ This is the output of a real run, with only the home path shortened. Line by line:
172
+
173
+ - `control` lists the kinds of context that the dirty twin found in your real setup. The probe
174
+ could therefore have seen each of them in the room. Connected accounts count under `mcp`,
175
+ because they reach the agent as MCP tools. `home path` means that your real home path appears
176
+ in the agent's request, for example in the path of a memory file. `reach` comes from a plain
177
+ script that looks for your files. It finds them outside the room, which shows that the script
178
+ works. So when it finds nothing inside the room, that result can be trusted.
179
+ - `leaks none` says that none of these reached the room. Inside the room, tare starts Claude Code
180
+ with flags that keep connected accounts and account skills out.
181
+ - `declared` lists the two things that the room holds on purpose. With a subscription login, your
182
+ account email arrives with the login, and tare cannot remove it without removing the login. It
183
+ is one line of text that names you, not instructions, memories or tools. So tare reports it
184
+ and lets the run start. The room also holds a copy of your login, because the agent needs one
185
+ to run.
186
+
187
+ For the dirty twin, the probe starts your real agent with your real setup, so your hooks and MCP
188
+ servers start too. The Antigravity CLI writes into its config directory on every run, so tare
189
+ lets its dirty twin write into a temporary overlay that is discarded after the run. Your
190
+ `~/.gemini` stays unchanged.
191
+
192
+ `tare claude` then starts Claude Code in that room. `tare codex`, `tare pi` and `tare agy` do
193
+ the same for the other agents.
194
+
195
+ ## Usage
196
+
197
+ ```bash
198
+ tare probe claude # probe the room: tare: 0.00, the leaks and their sources, or blind
199
+ tare claude # probe, then start Claude Code in the room
200
+ tare claude -- -p "fix the tests" # arguments after -- go to claude
201
+ tare claude --yolo # skip the agent's permission prompts
202
+ tare claude --allow-dirty # start even though the reading is not tare: 0.00
203
+ tare claude --project ../other # another project directory (default: the current one)
204
+
205
+ tare probe codex # the same for Codex, Pi and the Antigravity CLI
206
+ tare codex -- exec -m MODEL "..." # one-shot Codex run: arguments after -- go to codex
207
+ tare pi # Pi in a room
208
+ tare agy # the Antigravity CLI in a room
209
+ tare prices update # fetch the price table for costs an agent does not report
210
+ ```
211
+
212
+ `tare claude`, `tare codex`, `tare pi` and `tare agy` mount your project at `/work` directly. The
213
+ agent changes your real files, as it would outside the room. `--yolo` adds the agent's own flag
214
+ for skipping permission prompts. Pi has no such prompts. The commands under
215
+ [Measuring agents in rooms](#measuring-agents-in-rooms) work on copies instead.
216
+
217
+ `tare probe` exits with 0 on `tare: 0.00` and with 1 otherwise. `tare claude` and the other agent
218
+ commands refuse to start with exit code 1 unless the reading is `tare: 0.00` or you pass
219
+ `--allow-dirty`. When they start, they return the agent's own exit code.
220
+
221
+ The room starts with the agent's defaults, not with your settings. Codex, for example, runs its
222
+ default model unless you pass `-m`. Set the model explicitly when you compare runs.
223
+
224
+ When a room is not clean, the reading names each leak and where it came from. The excerpt below
225
+ shows a room that was given copies of real settings on purpose, with the home path shortened.
226
+
227
+ ```text
228
+ leaks
229
+ instructions global instructions /home/me/.claude/CLAUDE.md
230
+ home path /home/me/ your files
231
+ skills 1 skill /home/me/.claude/skills
232
+ env SPIKE_API_KEY inherited environment
233
+ tare: 4 leaks
234
+ ```
235
+
236
+ ## Measuring agents in rooms
237
+
238
+ The commands in this section start many agent runs. Each run works on its own copy of your
239
+ project in its own room. tare scores each run with a **check**: any shell command that you pass
240
+ with `--check`. tare runs it in the finished workspace, and exit code 0 means that the run
241
+ passed. A **step** is one tool call of the agent, such as running a command or writing a file.
242
+
243
+ Each command saves everything in a **run directory** under `~/.local/state/tare/`. It also starts
244
+ a live dashboard on your machine and prints its address. Open the address in a browser to follow
245
+ the runs.
246
+
247
+ A pass rate comes with a **95% interval**: the range that holds the true pass rate with 95%
248
+ confidence. Few runs give a wide interval. Three passes out of three, for example, still allow a
249
+ true rate as low as 0.44.
250
+
251
+ ### Calibrate: how often does each side pass?
252
+
253
+ ```bash
254
+ tare calibrate "fix the failing test" --check "uv run pytest -q" \
255
+ --side "claude --model haiku" --side "codex -m gpt-6.1-sol" --runs 10
256
+ ```
257
+
258
+ Each `--side` is an agent with its arguments. tare starts every side `--runs` times from a fresh
259
+ copy of the project and reports each side's pass rate with its interval. Use it to compare
260
+ skills, plugins or models. It also tells you whether a task suits Swap or Cliff. Swap needs a
261
+ task where one side mostly fails and the other mostly passes. Cliff needs a task where a fresh
262
+ start passes at least sometimes.
263
+
264
+ Some skills start with their own command, for example a slash command, so the sides need
265
+ different prompts. `--side-prompt` gives a side its own prompt. The sides are named a, b and c in
266
+ the order of `--side`:
267
+
268
+ ```bash
269
+ tare calibrate --side "claude --plugin-dir plugins/a" --side "claude --plugin-dir plugins/b" \
270
+ --side-prompt "a=/a:ideate a todo app" --side-prompt "b=/b:ideate a todo app" --runs 5
271
+ ```
272
+
273
+ Without `--check`, a run counts as finished when its agent exits with 0. tare keeps each finished
274
+ workspace, and the report shows the finished runs and their times, so that you can score the
275
+ results elsewhere. `--keep` keeps the workspaces when there is a check too.
276
+
277
+ ### Judge: scoring results that no test can decide
278
+
279
+ Some results have no test, for example how good a web page is. A **judge** is a second agent that
280
+ scores the result from 0 to 100 by your **rubric**, a Markdown file that says what earns points.
281
+ The judge is Claude Code with Sonnet unless you choose another agent with `--judge`.
282
+
283
+ `tare judge DIR` opens the page `DIR/index.html` (another file with `--page`) in headless Google
284
+ Chrome inside a room of its own and takes a screenshot. Without `DIR`, it uses the current
285
+ directory. The judge then gets the screenshot, the page's source and the rubric in a fresh room,
286
+ without any agent or model names, so it cannot favour an agent or a model. `tare judge` exits
287
+ with 0 when the score reaches the `--threshold` value, so it works as a check:
288
+
289
+ ```bash
290
+ tare calibrate "build the page" --check "tare judge --rubric $PWD/rubric.md --threshold 60" \
291
+ --side "claude --model haiku" --runs 5
292
+ ```
293
+
294
+ A check runs in the finished workspace, so `tare judge` without `DIR` judges the page that the
295
+ agent built. The check does not run where you typed the command. So the rubric needs an absolute
296
+ path, which your shell makes from `$PWD/rubric.md`. And `tare` must be on `PATH`, or written as
297
+ `uv run --project <checkout> tare`.
298
+
299
+ A judge does not give the same page the same score every time. One test page scored 55, 60, 58,
300
+ 58, 66 and 62 in six judgements. Before you set a threshold, let `tare judge-noise` score a few
301
+ finished pages several times each, ideally a good one and a weak one:
302
+
303
+ ```bash
304
+ tare judge-noise good-page/ weak-page/ --rubric rubric.md --times 10 --threshold 60
305
+ ```
306
+
307
+ Each directory holds one finished page. `judge-noise` lists every page's scores and refuses a
308
+ threshold that lies inside the range of a page's scores, because there the judge's noise would
309
+ decide between pass and fail.
310
+
311
+ ### Swap: was it the workspace or the agent?
312
+
313
+ ```bash
314
+ tare swap "fix the failing test" --check "uv run pytest -q" --a claude --a-args "--model sonnet" --b codex
315
+ ```
316
+
317
+ Two agents, a and b, each attempt the same task once. When one of them fails, two causes are
318
+ possible. The agent may be the weaker one. Or its workspace may have reached a state that no
319
+ agent can finish. Swap separates the two causes.
320
+
321
+ Swap cuts both runs at several points. `--cuts` sets them as fractions of each run's steps,
322
+ where 0 is the start and 1 is the end. The default is `0,0.5,1`. At each cut, each agent continues
323
+ each workspace, three times by default (`--tails`). Each such continuation is a **tail**. A tail
324
+ gets the workspace and a **handoff**: a written summary of the steps so far, in the same form for
325
+ every agent. Swap then reports two numbers per cut, each with an interval:
326
+
327
+ - The **state effect** says how much better the tails do in a's workspace than in b's, which is
328
+ how much the state of the workspace matters.
329
+ - The **model effect** says how much better agent a does than agent b, in the same workspaces.
330
+ The model effect covers the agent CLI and its model together.
331
+
332
+ At cut 0 both workspaces are your untouched project, so the state effect there must be zero. Swap
333
+ checks this and calls it the **null check**. Where an agent can continue its own saved session
334
+ (Claude Code, Codex and Pi), Swap also lets each agent continue its own session from each cut.
335
+ That measures what the handoff itself costs. The report shows it per agent as a **handoff cost**
336
+ line: the agent's pass rate when it continues its own session, minus its pass rate from the
337
+ handoff, in its own workspace. A value near zero means that the handoff loses nothing.
338
+
339
+ The report below comes from tare's own test. It drives the real Claude Code CLI against two
340
+ scripted models: fake models that follow a fixed script, so the right verdict is known. The test
341
+ runs four tails per workspace and agent instead of three. Agent a is competent but trusts a note in the
342
+ workspace. Agent b is weak. Run b wrote a wrong note at step 2.
343
+
344
+ Each cell shows passes out of tails and their interval. Positive effects favour a's workspace or
345
+ agent a. The report continues with cut 1.00, left out here.
346
+
347
+ ```text
348
+ cut 0.00: workspace a at step 0, workspace b at step 0
349
+ agent a agent b
350
+ workspace a 4/4 0.51-1.00 1/4 0.05-0.70
351
+ workspace b 4/4 0.51-1.00 1/4 0.05-0.70
352
+ state effect +0.00 [-0.35, +0.35] (positive: workspace a better)
353
+ model effect +0.75 [+0.28, +0.89] (positive: agent a better)
354
+ handoff cost a +0.00 [-0.49, +0.49] (own session 4/4 against handoff 4/4, both in its own workspace)
355
+ handoff cost b +0.50 [-0.14, +0.79] (own session 3/4 against handoff 1/4, both in its own workspace)
356
+
357
+ cut 0.50: workspace a at step 2, workspace b at step 2
358
+ agent a agent b
359
+ workspace a 4/4 0.51-1.00 1/4 0.05-0.70
360
+ workspace b 0/4 0.00-0.49 2/4 0.15-0.85
361
+ state effect +0.38 [-0.03, +0.66] (positive: workspace a better)
362
+ model effect +0.12 [-0.25, +0.44] (positive: agent a better)
363
+ handoff cost a +0.00 [-0.49, +0.49] (own session 4/4 against handoff 4/4, both in its own workspace)
364
+ handoff cost b -0.25 [-0.66, +0.32] (own session 1/4 against handoff 2/4, both in its own workspace)
365
+
366
+ null check passed: no state effect at cut 0, where both workspaces are the untouched project
367
+ verdict Blame passes from the model to the workspace between cut 0.00 and cut 0.50. The intervals behind this verdict include zero. More tails would settle it.
368
+ ```
369
+
370
+ At the start, the model effect explains the difference. Agent a passes, agent b mostly fails.
371
+ After the wrong note, a's workspace still works and b's does not, even for the competent agent
372
+ a. The verdict says so.
373
+
374
+ ### Cliff: where did a failed run go wrong?
375
+
376
+ ```bash
377
+ tare cliff claude "fix the failing test" --check "uv run pytest -q" -- --model sonnet
378
+ ```
379
+
380
+ A failed run is a chain of steps. Somewhere in that chain, the run may have taken a turn that it
381
+ could not recover from. Cliff looks for that step, the **cliff**. Before it the run could still
382
+ succeed, after it the run no longer does.
383
+
384
+ > **Experimental.** Cliff finds the known step in tare's scripted test. On real runs it has
385
+ > never blamed a step that was not the cause. But it has not yet found a real cliff either,
386
+ > because neither of two real test rounds had a failure that sat in one step. The rounds were a
387
+ > [data migration](https://github.com/a-ratz/tare/blob/main/experiments/migration/RESULTS.md) and a [web page](https://github.com/a-ratz/tare/blob/main/experiments/html/RESULTS.md).
388
+
389
+ tare runs the task once in a room and saves the workspace and the conversation after every step.
390
+ If the check fails, Cliff starts tails from saved steps in fresh rooms. Tails from the beginning
391
+ are the **baseline**. They show how often the task succeeds at all.
392
+
393
+ Cliff first runs three baseline tails and three tails from the last step. A step counts as good
394
+ while its tails pass at least half as often as the baseline. Cliff then halves the range between
395
+ the last good step and the first bad one, three tails at a time. When the two are neighbours, it
396
+ adds tails to both until their intervals no longer overlap, or until the budget (`--budget`,
397
+ default 30 tails) is spent. Everything after `--` goes to every agent run, so pin the model there.
398
+
399
+ Cliff can also end without naming a step. When even the baseline almost never passes, the task is
400
+ too hard for the model, and Cliff reports a model gap. It does so only when the upper end of the
401
+ baseline's interval lies below `--gap-below` (default 0.2). When tails from the last step pass at
402
+ least half as often as the baseline, the run was unlucky, not lost, and Cliff says that.
403
+
404
+ The report below comes from tare's scripted test, with a scripted model that writes the wrong
405
+ answer into a note at step 4. Steps that Cliff did not need are missing from the table, and `<`
406
+ marks the cliff. `monotone` says whether a step after the cliff passes again.
407
+
408
+ ```text
409
+ step what the step did tails pass rate 95% interval
410
+ 0 start 3 3 1.00 0.44-1.00
411
+ 3 Bash echo hi > scratch.txt 6 5 0.83 0.44-0.97
412
+ 4 Bash echo answer=41 > notes.txt 6 0 0.00 0.00-0.39 <
413
+ 6 Bash echo 41 > answer.txt 3 0 0.00 0.00-0.56
414
+
415
+ verdict The run became lost at step 4.
416
+ at step 4: Bash echo answer=41 > notes.txt
417
+ changed notes.txt
418
+ monotone yes: no step after the cliff passes again
419
+ tails 18 of a budget of 30
420
+ ```
421
+
422
+ Cliff works with all four agents (`tare cliff pi ...`). Claude Code, Codex and Pi continue their
423
+ own saved session. The Antigravity CLI stores its sessions in a format that tare cannot cut, so
424
+ Cliff continues it with a handoff, as Swap does. Each tail is a real agent run, so a search costs
425
+ as much as its tails.
426
+
427
+ ### Tokens and cost
428
+
429
+ Every Calibrate, Swap and Cliff report ends with the tokens and the cost of each side, of the
430
+ check (a judge agent costs tokens too) and of the whole run:
431
+
432
+ ```text
433
+ usage a claude --model haiku: 1 run, 42.3k in (34.8k cached), 352 out, $0.020. Subscription login, so the cost is notional
434
+ b codex -m gpt-6.1-sol: 1 run, 42.1k in (38.5k cached), 129 out, about $0.012 (estimated from LiteLLM prices fetched 2026-10-05). Subscription login, so the cost is notional
435
+ check (the judge): 2 runs, 102.0k in (92.7k cached), 471 out, $0.061
436
+ total: 4 runs, 186.4k in (166.0k cached), 952 out, about $0.093 (estimated from LiteLLM prices fetched 2026-10-05)
437
+ ```
438
+
439
+ Claude Code and Pi report their cost themselves. Codex and the Antigravity CLI report only
440
+ tokens, so tare estimates their cost from a price table, and only when it knows the model: pass
441
+ it with `-m` or `--model`. `tare prices update` fetches LiteLLM's public price table, and
442
+ `~/.config/tare/prices.json` overrides single models with your own prices (in dollars per million
443
+ tokens, with a source and a date). With a subscription login the cost is what the same tokens
444
+ would cost through the API, so the report calls it notional. The dashboard shows the same
445
+ totals while the run goes on, and the run directory keeps them in `usage.json`.
446
+
447
+ ### Watch it live, repeat it
448
+
449
+ The live dashboard of `tare calibrate`, `tare swap` and `tare cliff` runs at
450
+ `http://localhost:8777/` unless that port is taken. It shows the setup of the run, the probe
451
+ readings, each run step by step, every tail with what its agent is doing, the results as they
452
+ come in, and the final report.
453
+
454
+ ```bash
455
+ tare watch ~/.local/state/tare/cliff/myproject-20261005-142000 # any run, live or finished
456
+ tare watch ~/.local/state/tare/calibrate/* # several runs: one row each
457
+ tare rerun ~/.local/state/tare/cliff/myproject-20261005-142000 # the same run again
458
+ ```
459
+
460
+ Every run directory holds a `recipe.json` with the command, the task and the check, a fingerprint
461
+ of the project, the agents with their versions and arguments, and the settings. `tare rerun`
462
+ repeats the run in a new directory with the same settings and says what has changed since and
463
+ what the repeated run used.
464
+ Agent runs are not deterministic, so a repeated run can still come out differently.
465
+
466
+ ## How it works
467
+
468
+ A probe has three parts, and none of them asks the agent anything. First, the real agent runs
469
+ inside the room against the fake model server, which keeps the request that it receives. That
470
+ request is the context the agent was given. Second, the dirty twin runs the same agent in your
471
+ real setup, so tare knows that it can see your setup at all. Third, a plain script inside the
472
+ room looks for your files and for secret environment variables.
473
+
474
+ → [Concept: the design and the measurements behind it](https://github.com/a-ratz/tare/blob/main/CONCEPT.md)
475
+ → [The dirty-twin experiments that decided the probe design: Claude Code](https://github.com/a-ratz/tare/blob/main/experiments/dirty-twin/RESULTS.md), [Codex](https://github.com/a-ratz/tare/blob/main/experiments/dirty-twin-codex/RESULTS.md), [Pi](https://github.com/a-ratz/tare/blob/main/experiments/dirty-twin-pi/RESULTS.md), [Antigravity CLI](https://github.com/a-ratz/tare/blob/main/experiments/dirty-twin-agy/RESULTS.md)
476
+
477
+ ## Roadmap
478
+
479
+ A room for macOS, built on Seatbelt, is in progress ([epic #85](https://github.com/a-ratz/tare/issues/85)).
480
+ Until it is released, tare runs on Linux and WSL.
481
+
482
+ ## Contributing
483
+
484
+ Every change starts as an issue. Larger work is an epic, split into features, and one epic
485
+ becomes one branch and one pull request. [AGENTS.md](https://github.com/a-ratz/tare/blob/main/AGENTS.md) describes the process, the checks
486
+ and the conventions. Run the tests with `uv run pytest`.
487
+
488
+ ## License
489
+
490
+ MIT. See [LICENSE](https://github.com/a-ratz/tare/blob/main/LICENSE).
491
+
492
+ ---
493
+
494
+ Crafted with [Readme Craft](https://github.com/motiful/readme-craft)
495
+
496
+ [license-shield]: https://img.shields.io/badge/license-MIT-blue.svg
497
+ [license-url]: https://github.com/a-ratz/tare/blob/main/LICENSE
498
+ [version-shield]: https://img.shields.io/pypi/v/tare-cli.svg
499
+ [version-url]: https://github.com/a-ratz/tare/blob/main/pyproject.toml
500
+ [python-shield]: https://img.shields.io/badge/python-3.11%2B-3776AB.svg
501
+ [python-url]: https://www.python.org/
502
+ [platform-shield]: https://img.shields.io/badge/platform-Linux%20%7C%20WSL-555555.svg
503
+ [platform-url]: #install
504
+ [agent-shield]: https://img.shields.io/badge/agents-Claude%20Code%20%7C%20Codex%20%7C%20Pi%20%7C%20Antigravity-D97757.svg
505
+ [agent-url]: #install
@@ -0,0 +1,23 @@
1
+ tare/__init__.py,sha256=rZuSHkVpIuRmCEmJ6b-GmiJOd9CIpJpmq_hAW5Jxvjg,67
2
+ tare/agents.py,sha256=Rd2YmHpnTZnNqgsnb_ERlc1yxFcULkV1M0_yRxu6C3k,31963
3
+ tare/calibrate.py,sha256=LnYDgZLVipqdACyDVZSNyd2RpFJrBpVn-7m0HbEg73A,4673
4
+ tare/capsule.py,sha256=ZSUmtYBQGi3Buvy4KmgjxELy3UyoaBU1uD7hnmeI2WE,11340
5
+ tare/cli.py,sha256=KkSUhdQ2zKPO-_ea5KpltRdhfyIC7hQGLjAwMIRW6KM,18484
6
+ tare/cliff.py,sha256=cxN6xx4t75Kqif9TYKG6qGT3V45HzR0Myg38KP7n0UQ,8633
7
+ tare/dashboard.html,sha256=CtX3WB1gXvvxoXocZ_WZLUmzWyjau4kJQwvht2NEQYE,19129
8
+ tare/dashboard.py,sha256=I-rCL30aj3FkU1zlmMI9FMiMFMfsKRXj0JuoQZf4b2s,9878
9
+ tare/fake.py,sha256=3U3qBd3OKcTB5f4VtAbbWdmU_8aMEmjBhA7hrXgJmWY,9690
10
+ tare/journal.py,sha256=XRpgGjzn5YrRRa8v49t-kKQ4vtoK1wJsy0yodfj68Cc,1091
11
+ tare/judge.py,sha256=o1mM5lkG9s9vi4bb8hi1CatV3t6n0sGThgYN81Cl23U,6129
12
+ tare/overview.html,sha256=6KrIreXi5pYMGY84VXHn_xCMWA9O2QG7jHtmUO5kNEw,3917
13
+ tare/probe.py,sha256=lJS4CWhncp9v_KXdLavhi_hllOOIP40PFvrtUSWnfKg,12952
14
+ tare/recipe.py,sha256=-LFnjTrcpfzmGjhilGWworlAxOK8c3psp_mGiLADLYM,3903
15
+ tare/room.py,sha256=Ls3DirJcCkGLMz6n5XFI7_-zBlYeng-OUAtoHJ3yhgU,3019
16
+ tare/swap.py,sha256=4HEAWY7AIHdRKTC3NqQFLIMr-J9Frh8rRnpr8mVKTqE,11397
17
+ tare/trail.py,sha256=vAUOPz9m1CwESTMjs8sJ5YpdwdAdtjbd-sZBC33iGsQ,9952
18
+ tare/usage.py,sha256=O4Od_v-ulJ-mylxY55djFLQIufn4dFVd1Zqv_m739tU,10812
19
+ tare_cli-0.2.0.dist-info/METADATA,sha256=4pWSDzqd7FA8dZtG6qAL3W9C1wGcWFb746sV_vA0Xw4,28628
20
+ tare_cli-0.2.0.dist-info/WHEEL,sha256=W3fkpkm7-wf9vBI5Z-7s0eWkeM-spu78I8Neb98DeEg,87
21
+ tare_cli-0.2.0.dist-info/entry_points.txt,sha256=BhoQrZYP3BRJuI7i3u3G5m50IrH6MDFcla6OL51d31U,39
22
+ tare_cli-0.2.0.dist-info/licenses/LICENSE,sha256=ZN4Y2sSRK1FL3-LSXp_sEfXthN-X-zLQeO7LzHeD4fY,1075
23
+ tare_cli-0.2.0.dist-info/RECORD,,
@@ -0,0 +1,4 @@
1
+ Wheel-Version: 1.0
2
+ Generator: hatchling 1.32.4
3
+ Root-Is-Purelib: true
4
+ Tag: py3-none-any
@@ -0,0 +1,2 @@
1
+ [console_scripts]
2
+ tare = tare.cli:main
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Andre Ratzenberger
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.