callva-harness-runner 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,12 @@
1
+ __pycache__/
2
+ *.py[cod]
3
+ *.egg-info/
4
+ build/
5
+ dist/
6
+ .venv/
7
+ venv/
8
+ .pytest_cache/
9
+ .mypy_cache/
10
+ .ruff_cache/
11
+ .env
12
+ .DS_Store
@@ -0,0 +1,107 @@
1
+ # Design
2
+
3
+ `callva-harness-runner` runs one turn of an agent harness from a profile and hands back one result. It exists because the same fifty lines of launch-and-parse code had been written nine times across one owner's tools, each with a different bug, and a tenth was about to be written.
4
+
5
+ ## 1. Scope
6
+
7
+ One turn: a prompt goes in, a harness runs to its end or its deadline, one `Result` comes out. The library runs the harness through its vendor's SDK against the CLI installed on the machine, configures it only from the profile's explicit knobs, keeps every process it starts under the deadline, reads the harness's report, and classifies a failure by what the harness said. That is the whole job.
8
+
9
+ ### Non-goals
10
+
11
+ It does not schedule, queue, retry, or loop. It holds no ledger, no session store, no conversation memory. It renders no progress; it hands events to a callback and the caller renders. It does not know what a task, a message or a job is. It ships no presets and no fences: it never decides what a turn may do. Presets belong to consumers.
12
+
13
+ ## 2. Names
14
+
15
+ The vocabulary: a harness is the vendor's agent runtime (Claude Code, the Codex CLI); a profile is one harness configuration; a run is one turn. The library holds no queue and no long-lived consumer of work; those belong to its callers.
16
+
17
+ The distribution is `callva-harness-runner` and the import path is `callva.harness_runner`: the distribution name is the import path with dashes, the same rule `callva-livekit` follows, and `callva` is a namespace package with no `__init__.py` so the two coexist in one environment. It was `callva-agentworker` (`callva.agentworker`) up to 0.2.0; the README maps each old name to its new one, and every old name is refused with a message naming the new one. The three nouns a caller uses are `Profile` (how to run), `Session` (which conversation), and `Result` (what came back). `run()` is the one verb. `probe()` answers whether a harness is present.
18
+
19
+ ## 3. Harnesses
20
+
21
+ `claude` is Claude Code through `claude-agent-sdk`; `codex` is Codex through `openai-codex`, which drives `codex app-server`. Both are dependencies pinned exactly, because an SDK release changes the protocol it speaks and the flags it passes.
22
+
23
+ Both SDKs bundle a CLI, and the library never runs it: it always passes the installed CLI (`cli_path` on claude, the `app-server` launch on codex). The CLI is the profile's `cli_path`, or else the first `claude` or `codex` on the caller's `PATH`, then in the places a login shell would have added and a launchd job does not: `~/.local/bin`, fnm and nvm node directories, Homebrew. Codex is started at the file a symlinked install points to, because it finds its companion executables beside the path it was started from.
24
+
25
+ Claude always runs on the `claude_code` system-prompt preset. The SDK's own default is a minimal prompt that is not Claude Code's; the preset keeps the prompt a `claude -p` turn has. Neither harness's system prompt is ever replaced; `append_system_prompt` adds to claude's and is codex's developer instructions.
26
+
27
+ Codex's turn is read from the turn's notification stream, not from `thread.run()`, which raises on a failed turn and drops `codexErrorInfo`.
28
+
29
+ ## 4. Profile
30
+
31
+ A `Profile` is a frozen, flat record of explicit knobs. There is no inheritance and no named fence. A knob at its default leaves the harness's own default in place. A knob one harness cannot honour is refused for that harness when the profile is built, with a `ProfileError` naming the knob; it is never ignored while the turn runs. Each knob's docstring on `Profile` is the home of what it does on each harness, and the README carries the same table for readers of the package page.
32
+
33
+ The knobs: `harness`, `cli_path`, `model`, `effort`, `timeout_seconds`, `budget_usd`; `tools`, `allowed_tools`, `disallowed_tools`, `permission_mode`, `setting_sources`; `settings`, `mcp_config`, `strict_mcp`; `add_dirs`, `append_system_prompt`, `output_schema`, `name`; `env_set`, `env_remove`, `strip_session_markers`; `codex_config`, `approval_policy`, `service_tier`, `bypass_hook_trust`; `claude_extra_args`, `codex_extra_args`.
34
+
35
+ `Profile.from_dict()` validates against `profile.schema.json`, which ships in the package so that every consumer's configuration file is checked by the same rule. Unknown keys are refused, and the 0.1.x keys (`fence`, `allow_tools`, `env`, `extra_args`) are refused with a message naming the knobs that replace them.
36
+
37
+ Hook trust on codex is a knob that is refused on both harnesses. Over `app-server` there is no way to run a hook the machine has not trusted: `bypass_hook_trust` is not a config key (`--strict-config` rejects it) and the CLI's `--dangerously-bypass-hook-trust`, accepted before `app-server`, does not make an untrusted hook run, although the same hook runs under `codex exec` with it. Hooks the machine trusts run.
38
+
39
+ ### Environment
40
+
41
+ Both SDKs merge the profile's variables over the environment the Python process has, and neither can remove one. The library therefore puts a launcher of its own where the SDK expects the CLI: a two-line shell script that runs `_launcher.py` with `python -I -S`, which removes the variables the turn removes and `exec`s the real CLI. What reaches the launcher is a spec of names and paths; a variable's value only ever travels through the SDK's own env option.
42
+
43
+ The harness's environment is the caller's `environ` (this process's by default), with the profile's `env_remove` globs and, unless `strip_session_markers` is false, `SESSION_MARKERS` removed, and with `env_set` and the call's `extra_env` set on top. A variable in `env_set` is never removed, and a profile that both sets and removes one name is refused. On claude the variables the SDK itself sets for the CLI (its entrypoint, its version, its session-state flag, `PWD`) are the SDK's values and are not removed.
44
+
45
+ ## 5. Session
46
+
47
+ `Session.fresh()` starts a new conversation. On claude the library generates the id before launch and pins it, so a turn that is killed is still addressable. On codex the id is the thread id the app-server gives, reported even when the turn is later killed. `Session.pinned(id)` lets the caller choose the id; it is claude-only and refused on codex. `Session.resume(id)` continues an existing conversation on either harness; on codex the thread is resumed with the profile's model, config, developer instructions, service tier and approval policy.
48
+
49
+ ## 6. Harness guard
50
+
51
+ `TESTED_VERSIONS` declares, per harness, the inclusive range of CLI versions the release was tested with: the low end is the CLI installed where the release was tested, the high end is the CLI the pinned SDK bundles. Before every turn the installed CLI's `--version` is read (cached per binary file):
52
+
53
+ - below the range, or unreadable: the turn is refused with `incompatible_harness`, naming both versions;
54
+ - above the range: the turn runs and `Result.warnings` names both versions;
55
+ - no CLI: `binary_missing`, as it was.
56
+
57
+ ## 7. Running
58
+
59
+ The turn runs on a thread of its own; the calling thread holds the deadline and the cancel. The launcher records the harness's pid and command line and makes the harness a session leader. When the deadline passes or the caller's `threading.Event` is set, the library first writes a stop file that the launcher checks before any later start, then finds the harness's tree three ways: by parentage from the harness, by the process groups of what it found, and by the sessions of what it found, the harness's own included. The harnesses give each tool command a process group of its own, and a command that backgrounds a child and exits leaves that child with its group and session but a new parent, so parentage alone would miss it. The tree gets SIGTERM, three seconds, then SIGKILL; the caller's own group and session are never touched. Then the SDK is closed.
60
+
61
+ A caller killed outright cannot do any of this, and the harness's tree outlives it.
62
+
63
+ ## 8. Reading
64
+
65
+ Claude: the SDK's `ResultMessage` gives `result` as the answer (its `errors` when there is none), `structured_output`, `session_id`, `total_cost_usd`, `duration_ms`, `num_turns`, the four token counts from `usage`, the model that ran from `modelUsage`, `permission_denials`, and `subtype` with `is_error` as the verdict. The SDK raises after it hands over an error result; the result is the report and the raise is not. The last `api_retry` event is kept, because a turn that dies by the deadline while the harness retries a 401 or a 429 has already said why.
66
+
67
+ Codex: the turn's `item/completed` notifications give the answer (the last agent message), `thread/tokenUsage/updated` gives this turn's tokens (the thread's running total less what it held before the turn), and `turn/completed` gives the status, the duration and the error with its `codexErrorInfo`. Cost is `None`: codex reports none under ChatGPT authentication.
68
+
69
+ When `output_schema` is given and the harness returned no structured field, the answer text is parsed as JSON; if that fails, the turn is an `invalid_output` failure with the text kept.
70
+
71
+ ## 9. Result
72
+
73
+ | Field | Meaning |
74
+ |---|---|
75
+ | `ok` | the harness confirmed a completed turn: claude `subtype == "success"` and `is_error` false; codex status `completed`, no error, and a non-empty answer. |
76
+ | `harness` | which harness ran. |
77
+ | `answer` | the final text, possibly empty on failure. |
78
+ | `structured` | the parsed structured answer, or `None`. |
79
+ | `session_id` | claude session id or codex thread id; set whenever it is known, including after a kill. |
80
+ | `model` | the model that actually ran when claude reports it, else the requested one, else `None`. |
81
+ | `cost_usd`, `duration_ms`, `num_turns`, `tokens` | as reported; `None` where the harness does not say. |
82
+ | `permission_denials` | claude's list, empty on codex. |
83
+ | `failure` | `None` when `ok`; otherwise a kind and the harness's own sentence. |
84
+ | `exit_code` | the exit code the claude SDK reported for a failed process, else `None`. |
85
+ | `stderr_tail` | the last lines of the harness's stderr, for a person. |
86
+ | `command` | the CLI command line the launcher started. |
87
+ | `raw` | claude's result message as a dict, or codex's `{"turn", "items", "errors"}`, for anything this table left out. |
88
+ | `harness_version` | the CLI version the guard read. |
89
+ | `warnings` | the guard's warning when the CLI is newer than tested. |
90
+
91
+ ## 10. Failure kinds
92
+
93
+ `quota`: the subscription or rate limit is spent; the turn can be retried later without changing anything. `model_refused`: the harness will not run this model; retrying changes nothing. `not_authenticated`: no login or an invalid key. `timeout`: the deadline passed and the tree was killed. `cancelled`: the caller's event was set. `binary_missing`: no harness found. `max_turns`, `budget`: the harness ended the turn on its own limit. `invalid_output`: a schema was required and nothing parsed. `error`: the harness reported a failure this table does not name; the sentence is in `message`. `crash`: the harness ended without a report. `incompatible_harness`: the installed CLI is older than the tested range, or its version cannot be read.
94
+
95
+ Classification reads the harness's own verdict first (claude's `error_max_turns` and budget subtypes; codex's `codexErrorInfo`: `usageLimitExceeded` and `rateLimitExceeded` are `quota`, `unauthorized` is `not_authenticated`, `sessionBudgetExceeded` is `budget`), then an HTTP status the harness reports (claude's `api_error_status` or its last `api_retry` status, codex's `httpStatusCode`: 429 is `quota`, 401 is `not_authenticated`), then the sentence, in a fixed order: quota before model refusal, because a spent subscription on one model says both, and only one of them is a pause. Codex wraps a provider's HTTP error as JSON; the sentence inside it is what is read and kept.
96
+
97
+ ## 11. Constraints worth knowing
98
+
99
+ The cost claude reports is its own estimate and includes subagents; a turn that dies reports no cost at all, so an accumulated total is a floor. Codex under ChatGPT authentication reports tokens but no cost. A profile with no knobs runs the harness exactly as the machine configures it: user settings, hooks and MCP servers included. Hooks on codex run only when the machine trusts them.
100
+
101
+ ## 12. Versioning
102
+
103
+ SemVer with the 0.x rule: the minor is the breaking position. A changed field meaning, a removed knob or a changed default is a minor; an added result field, an added knob whose default leaves the harness alone, or a widened tested range is a patch. `callva/harness_runner/version.py` is the one home of the number.
104
+
105
+ ## 13. Later
106
+
107
+ Progress rendering helpers, if two consumers end up writing the same one. Cost lookup for codex by token price table, if anyone needs a number rather than `None`. A parent watch in the launcher, if a crashed caller's orphaned tree becomes a real problem.
@@ -0,0 +1,202 @@
1
+
2
+ Apache License
3
+ Version 2.0, January 2004
4
+ http://www.apache.org/licenses/
5
+
6
+ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
7
+
8
+ 1. Definitions.
9
+
10
+ "License" shall mean the terms and conditions for use, reproduction,
11
+ and distribution as defined by Sections 1 through 9 of this document.
12
+
13
+ "Licensor" shall mean the copyright owner or entity authorized by
14
+ the copyright owner that is granting the License.
15
+
16
+ "Legal Entity" shall mean the union of the acting entity and all
17
+ other entities that control, are controlled by, or are under common
18
+ control with that entity. For the purposes of this definition,
19
+ "control" means (i) the power, direct or indirect, to cause the
20
+ direction or management of such entity, whether by contract or
21
+ otherwise, or (ii) ownership of fifty percent (50%) or more of the
22
+ outstanding shares, or (iii) beneficial ownership of such entity.
23
+
24
+ "You" (or "Your") shall mean an individual or Legal Entity
25
+ exercising permissions granted by this License.
26
+
27
+ "Source" form shall mean the preferred form for making modifications,
28
+ including but not limited to software source code, documentation
29
+ source, and configuration files.
30
+
31
+ "Object" form shall mean any form resulting from mechanical
32
+ transformation or translation of a Source form, including but
33
+ not limited to compiled object code, generated documentation,
34
+ and conversions to other media types.
35
+
36
+ "Work" shall mean the work of authorship, whether in Source or
37
+ Object form, made available under the License, as indicated by a
38
+ copyright notice that is included in or attached to the work
39
+ (an example is provided in the Appendix below).
40
+
41
+ "Derivative Works" shall mean any work, whether in Source or Object
42
+ form, that is based on (or derived from) the Work and for which the
43
+ editorial revisions, annotations, elaborations, or other modifications
44
+ represent, as a whole, an original work of authorship. For the purposes
45
+ of this License, Derivative Works shall not include works that remain
46
+ separable from, or merely link (or bind by name) to the interfaces of,
47
+ the Work and Derivative Works thereof.
48
+
49
+ "Contribution" shall mean any work of authorship, including
50
+ the original version of the Work and any modifications or additions
51
+ to that Work or Derivative Works thereof, that is intentionally
52
+ submitted to Licensor for inclusion in the Work by the copyright owner
53
+ or by an individual or Legal Entity authorized to submit on behalf of
54
+ the copyright owner. For the purposes of this definition, "submitted"
55
+ means any form of electronic, verbal, or written communication sent
56
+ to the Licensor or its representatives, including but not limited to
57
+ communication on electronic mailing lists, source code control systems,
58
+ and issue tracking systems that are managed by, or on behalf of, the
59
+ Licensor for the purpose of discussing and improving the Work, but
60
+ excluding communication that is conspicuously marked or otherwise
61
+ designated in writing by the copyright owner as "Not a Contribution."
62
+
63
+ "Contributor" shall mean Licensor and any individual or Legal Entity
64
+ on behalf of whom a Contribution has been received by Licensor and
65
+ subsequently incorporated within the Work.
66
+
67
+ 2. Grant of Copyright License. Subject to the terms and conditions of
68
+ this License, each Contributor hereby grants to You a perpetual,
69
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
70
+ copyright license to reproduce, prepare Derivative Works of,
71
+ publicly display, publicly perform, sublicense, and distribute the
72
+ Work and such Derivative Works in Source or Object form.
73
+
74
+ 3. Grant of Patent License. Subject to the terms and conditions of
75
+ this License, each Contributor hereby grants to You a perpetual,
76
+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable
77
+ (except as stated in this section) patent license to make, have made,
78
+ use, offer to sell, sell, import, and otherwise transfer the Work,
79
+ where such license applies only to those patent claims licensable
80
+ by such Contributor that are necessarily infringed by their
81
+ Contribution(s) alone or by combination of their Contribution(s)
82
+ with the Work to which such Contribution(s) was submitted. If You
83
+ institute patent litigation against any entity (including a
84
+ cross-claim or counterclaim in a lawsuit) alleging that the Work
85
+ or a Contribution incorporated within the Work constitutes direct
86
+ or contributory patent infringement, then any patent licenses
87
+ granted to You under this License for that Work shall terminate
88
+ as of the date such litigation is filed.
89
+
90
+ 4. Redistribution. You may reproduce and distribute copies of the
91
+ Work or Derivative Works thereof in any medium, with or without
92
+ modifications, and in Source or Object form, provided that You
93
+ meet the following conditions:
94
+
95
+ (a) You must give any other recipients of the Work or
96
+ Derivative Works a copy of this License; and
97
+
98
+ (b) You must cause any modified files to carry prominent notices
99
+ stating that You changed the files; and
100
+
101
+ (c) You must retain, in the Source form of any Derivative Works
102
+ that You distribute, all copyright, patent, trademark, and
103
+ attribution notices from the Source form of the Work,
104
+ excluding those notices that do not pertain to any part of
105
+ the Derivative Works; and
106
+
107
+ (d) If the Work includes a "NOTICE" text file as part of its
108
+ distribution, then any Derivative Works that You distribute must
109
+ include a readable copy of the attribution notices contained
110
+ within such NOTICE file, excluding those notices that do not
111
+ pertain to any part of the Derivative Works, in at least one
112
+ of the following places: within a NOTICE text file distributed
113
+ as part of the Derivative Works; within the Source form or
114
+ documentation, if provided along with the Derivative Works; or,
115
+ within a display generated by the Derivative Works, if and
116
+ wherever such third-party notices normally appear. The contents
117
+ of the NOTICE file are for informational purposes only and
118
+ do not modify the License. You may add Your own attribution
119
+ notices within Derivative Works that You distribute, alongside
120
+ or as an addendum to the NOTICE text from the Work, provided
121
+ that such additional attribution notices cannot be construed
122
+ as modifying the License.
123
+
124
+ You may add Your own copyright statement to Your modifications and
125
+ may provide additional or different license terms and conditions
126
+ for use, reproduction, or distribution of Your modifications, or
127
+ for any such Derivative Works as a whole, provided Your use,
128
+ reproduction, and distribution of the Work otherwise complies with
129
+ the conditions stated in this License.
130
+
131
+ 5. Submission of Contributions. Unless You explicitly state otherwise,
132
+ any Contribution intentionally submitted for inclusion in the Work
133
+ by You to the Licensor shall be under the terms and conditions of
134
+ this License, without any additional terms or conditions.
135
+ Notwithstanding the above, nothing herein shall supersede or modify
136
+ the terms of any separate license agreement you may have executed
137
+ with Licensor regarding such Contributions.
138
+
139
+ 6. Trademarks. This License does not grant permission to use the trade
140
+ names, trademarks, service marks, or product names of the Licensor,
141
+ except as required for reasonable and customary use in describing the
142
+ origin of the Work and reproducing the content of the NOTICE file.
143
+
144
+ 7. Disclaimer of Warranty. Unless required by applicable law or
145
+ agreed to in writing, Licensor provides the Work (and each
146
+ Contributor provides its Contributions) on an "AS IS" BASIS,
147
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
148
+ implied, including, without limitation, any warranties or conditions
149
+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
150
+ PARTICULAR PURPOSE. You are solely responsible for determining the
151
+ appropriateness of using or redistributing the Work and assume any
152
+ risks associated with Your exercise of permissions under this License.
153
+
154
+ 8. Limitation of Liability. In no event and under no legal theory,
155
+ whether in tort (including negligence), contract, or otherwise,
156
+ unless required by applicable law (such as deliberate and grossly
157
+ negligent acts) or agreed to in writing, shall any Contributor be
158
+ liable to You for damages, including any direct, indirect, special,
159
+ incidental, or consequential damages of any character arising as a
160
+ result of this License or out of the use or inability to use the
161
+ Work (including but not limited to damages for loss of goodwill,
162
+ work stoppage, computer failure or malfunction, or any and all
163
+ other commercial damages or losses), even if such Contributor
164
+ has been advised of the possibility of such damages.
165
+
166
+ 9. Accepting Warranty or Additional Liability. While redistributing
167
+ the Work or Derivative Works thereof, You may choose to offer,
168
+ and charge a fee for, acceptance of support, warranty, indemnity,
169
+ or other liability obligations and/or rights consistent with this
170
+ License. However, in accepting such obligations, You may act only
171
+ on Your own behalf and on Your sole responsibility, not on behalf
172
+ of any other Contributor, and only if You agree to indemnify,
173
+ defend, and hold each Contributor harmless for any liability
174
+ incurred by, or claims asserted against, such Contributor by reason
175
+ of your accepting any such warranty or additional liability.
176
+
177
+ END OF TERMS AND CONDITIONS
178
+
179
+ APPENDIX: How to apply the Apache License to your work.
180
+
181
+ To apply the Apache License to your work, attach the following
182
+ boilerplate notice, with the fields enclosed by brackets "[]"
183
+ replaced with your own identifying information. (Don't include
184
+ the brackets!) The text should be enclosed in the appropriate
185
+ comment syntax for the file format. We also recommend that a
186
+ file or class name and description of purpose be included on the
187
+ same "printed page" as the copyright notice for easier
188
+ identification within third-party archives.
189
+
190
+ Copyright [yyyy] [name of copyright owner]
191
+
192
+ Licensed under the Apache License, Version 2.0 (the "License");
193
+ you may not use this file except in compliance with the License.
194
+ You may obtain a copy of the License at
195
+
196
+ http://www.apache.org/licenses/LICENSE-2.0
197
+
198
+ Unless required by applicable law or agreed to in writing, software
199
+ distributed under the License is distributed on an "AS IS" BASIS,
200
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
201
+ See the License for the specific language governing permissions and
202
+ limitations under the License.
@@ -0,0 +1,192 @@
1
+ Metadata-Version: 2.5
2
+ Name: callva-harness-runner
3
+ Version: 0.3.0
4
+ Summary: One headless turn of a coding agent (Claude Code or Codex) through the vendor SDKs: explicit knobs, a deadline that kills the tree, one result.
5
+ Project-URL: Homepage, https://github.com/callva-io/harness-runner
6
+ Project-URL: Source, https://github.com/callva-io/harness-runner
7
+ Author: CallVA
8
+ License-Expression: Apache-2.0
9
+ License-File: LICENSE
10
+ Keywords: agent,claude,claude-agent-sdk,codex,harness,headless,openai-codex
11
+ Classifier: Intended Audience :: Developers
12
+ Classifier: License :: OSI Approved :: Apache Software License
13
+ Classifier: Programming Language :: Python :: 3
14
+ Classifier: Programming Language :: Python :: 3 :: Only
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
17
+ Classifier: Topic :: Software Development :: Libraries
18
+ Requires-Python: >=3.11
19
+ Requires-Dist: claude-agent-sdk==0.2.161
20
+ Requires-Dist: openai-codex==0.159.0
21
+ Provides-Extra: dev
22
+ Requires-Dist: pytest>=8; extra == 'dev'
23
+ Requires-Dist: ruff>=0.6; extra == 'dev'
24
+ Description-Content-Type: text/markdown
25
+
26
+ # callva-harness-runner
27
+
28
+ One run of an agent harness, as a Python library. A harness is the vendor's agent runtime: Claude Code, driven through Anthropic's `claude-agent-sdk`, or the Codex CLI, driven through OpenAI's `openai-codex`, always the CLI installed on the machine. A profile is one harness configuration, a flat set of explicit knobs. A run is one turn: a prompt goes in, the harness works to its end or its deadline, and one `Result` comes out, with the failure named. The deadline kills everything the harness started. The library ships no presets and no fences: what a run may do is whatever its profile says, and nothing else.
29
+
30
+ `DESIGN.md` is the contract. The source is at https://github.com/callva-io/harness-runner.
31
+
32
+ Formerly `callva-agentworker`, whose last release is 0.2.0. The 0.3.0 renames, with no aliases (each old name is refused with a message naming the new one):
33
+
34
+ | 0.2.0 | 0.3.0 |
35
+ |---|---|
36
+ | distribution `callva-agentworker` | `callva-harness-runner` |
37
+ | import `callva.agentworker` | `callva.harness_runner` |
38
+ | profile knob `engine` | `harness` |
39
+ | `Result.engine` | `Result.harness` |
40
+ | `Result.engine_version` | `Result.harness_version` |
41
+ | failure kind `incompatible_engine` (`FailureKind.INCOMPATIBLE_ENGINE`) | `incompatible_harness` (`FailureKind.INCOMPATIBLE_HARNESS`) |
42
+ | `ENGINES` | `HARNESSES` |
43
+ | `Probe.engine` | `Probe.harness` |
44
+
45
+ ## Install
46
+
47
+ ```
48
+ pip install callva-harness-runner
49
+ ```
50
+
51
+ It depends on `claude-agent-sdk==0.2.161` and `openai-codex==0.159.0`, pinned exactly; together they add about 0.5 GB, most of it the CLIs both SDKs bundle. The bundled CLIs are never run: the library always runs the CLI installed on the machine.
52
+
53
+ In a PEP 723 script, pin the exact version so nothing outside the script's own release changes what it runs:
54
+
55
+ ```python
56
+ # /// script
57
+ # dependencies = ["callva-harness-runner==0.3.0"]
58
+ # ///
59
+ ```
60
+
61
+ ## Use
62
+
63
+ ```python
64
+ from callva.harness_runner import Profile, Session, run
65
+
66
+ profile = Profile.from_dict(config["profile"]) # your configuration, your knobs
67
+ result = run("Summarise README.md in one line.", profile, cwd="/path/to/project")
68
+ if result.ok:
69
+ print(result.answer, result.cost_usd, result.session_id)
70
+ else:
71
+ print(result.failure.kind, result.failure.message)
72
+ ```
73
+
74
+ `run()` never raises for anything the harness did. Every ending is a `Result`; `result.failure.kind` says what kind, and `result.failure.message` carries the harness's own sentence. It raises `ValueError` only for a request it cannot express, such as a pinned session on codex.
75
+
76
+ `run(prompt, profile, cwd, *, session=None, environ=None, extra_env=None, on_event=None, cancel=None, stderr_tail=40)`:
77
+
78
+ - `session`: `Session.fresh()` (the default), `Session.pinned(id)` or `Session.resume(id)`, below.
79
+ - `environ`: the environment the harness starts from; this process's by default. A variable absent from it is absent in the harness, even though both SDKs inherit this process's environment.
80
+ - `extra_env`: variables set on top of `environ` and the profile's `env_set`.
81
+ - `on_event`: called with one dict per harness event while the turn runs. On claude it is the SDK's message: a system message is the harness's own event (`subtype` `init` carries `apiKeySource`, `model` and `claude_code_version`), any other message is its fields plus `type`, the SDK class name. On codex it is `{"method", "params"}`, one app-server notification.
82
+ - `cancel`: a `threading.Event`; setting it ends the turn and kills its processes.
83
+
84
+ ## Profile
85
+
86
+ A profile is a flat set of explicit knobs. There is no inheritance, no preset and no fence. A knob left at its default leaves the harness's own default in place. A knob one harness cannot honour is refused for that harness with a `ProfileError` when the profile is built, never ignored while it runs. Each knob's docstring on `Profile` says what it does on each harness; `PROFILE_SCHEMA` (shipped as `profile.schema.json`) validates a profile kept in configuration, and `Profile.from_dict` refuses unknown keys.
87
+
88
+ | Knob | claude | codex |
89
+ |---|---|---|
90
+ | `harness` | `claude` | `codex` |
91
+ | `cli_path` | the CLI the SDK launches; `None` finds the installed `claude` | the binary whose `app-server` the SDK talks to; `None` finds the installed `codex` |
92
+ | `model` | `--model` | the thread's model |
93
+ | `effort` | `--effort` | the turn's reasoning effort |
94
+ | `timeout_seconds` | the deadline; default 600 | the same |
95
+ | `budget_usd` | `--max-budget-usd` | refused: codex reports no cost |
96
+ | `tools` | `--tools`, the built-in tools that exist; `[]` removes them all | refused |
97
+ | `allowed_tools` | `--allowedTools`, patterns that run without a prompt | refused |
98
+ | `disallowed_tools` | `--disallowedTools` | refused |
99
+ | `permission_mode` | `--permission-mode` | refused: use `approval_policy` and `codex_config` |
100
+ | `setting_sources` | `--setting-sources`; `["project", "local"]` loads the target's CLAUDE.md, rules and hooks without the user's | refused |
101
+ | `settings` | `--settings`, an object or path layered above the settings files: sandbox, permissions | refused: use `codex_config` |
102
+ | `mcp_config` | `--mcp-config`, `{"mcpServers": {...}}` or a path | refused: `mcp_servers` in `codex_config` |
103
+ | `strict_mcp` | `--strict-mcp-config` | refused |
104
+ | `add_dirs` | `--add-dir` per entry | refused: give the directory access in `codex_config` permissions |
105
+ | `append_system_prompt` | appended to the Claude Code system prompt | the thread's developer instructions |
106
+ | `output_schema` | `--json-schema` | the turn's output schema |
107
+ | `name` | `--name` | the thread's name |
108
+ | `env_set` | set in the harness's environment | the same |
109
+ | `env_remove` | names or globs removed from what the harness inherits | the same |
110
+ | `strip_session_markers` | also remove `SESSION_MARKERS`; default true | the same |
111
+ | `codex_config` | refused: use `settings` | the thread's config, keyed as in `config.toml`: permission profiles, `sandbox_mode`, network, `mcp_servers` |
112
+ | `approval_policy` | refused: use `permission_mode` | `never` (escalations refused) or `auto_review` (codex's reviewer decides); `None` is the SDK default, `auto_review` |
113
+ | `service_tier` | refused | the thread's service tier |
114
+ | `bypass_hook_trust` | refused: claude has no hook trust | refused when true: the app-server has no hook-trust bypass, so hooks run only when the machine trusts them |
115
+ | `claude_extra_args` | extra CLI flags, `{"flag": "value"}` or `{"flag": None}` | refused |
116
+ | `codex_extra_args` | refused | arguments before `app-server`, such as `["-c", "key=value"]` |
117
+
118
+ Claude always runs on the `claude_code` system-prompt preset, so a turn has the same prompt `claude -p` has; the library never replaces either harness's system prompt, and `append_system_prompt` only adds to it.
119
+
120
+ ### Environment
121
+
122
+ Both SDKs only add to the environment they inherit. To remove a variable for real, the library puts a small launcher of its own in front of the CLI: the SDK starts the launcher, the launcher removes what the profile removes and becomes the CLI. The launcher learns names, never values. `SESSION_MARKERS` are the variables a running Claude Code session leaves in its children (`CLAUDECODE`, `CLAUDE_CODE_SESSION_ID`, `CLAUDE_EFFORT`, `AI_AGENT` and the rest); a child harness that inherits them believes it runs inside that session, so they are removed unless `strip_session_markers` is false. On claude the values the SDK itself sets for the CLI, such as its entrypoint, stay.
123
+
124
+ ### Deadline and cancel
125
+
126
+ The launcher makes the harness a session leader. When the deadline passes, or `cancel` is set, no further harness start is allowed, and the harness, every process it started, and everything those left behind in their process group or session are sent SIGTERM and then SIGKILL, grandchildren included. A caller that is itself killed cannot do this, and the harness's tree then outlives it.
127
+
128
+ ## Harness guard
129
+
130
+ `TESTED_VERSIONS` holds, per harness, the range of CLI versions this release was tested with: claude 2.1.280 to 2.1.284, codex 0.159.0. Before a turn the library reads the installed CLI's `--version`:
131
+
132
+ - older than the range: the turn is refused with failure kind `incompatible_harness`, and the message names the installed version and the tested one;
133
+ - newer than the range: the turn runs, and `result.warnings` names both versions;
134
+ - no CLI: `binary_missing`, as before.
135
+
136
+ ## Session
137
+
138
+ `Session.fresh()` starts a conversation; on claude the id is chosen before launch so a killed turn is still addressable, on codex it is the thread id the app-server gives. `Session.resume(id)` continues one on either harness. `Session.pinned(id)` chooses the id and is claude-only.
139
+
140
+ ## Result
141
+
142
+ | Field | Meaning |
143
+ |---|---|
144
+ | `ok` | the harness confirmed a completed turn with an answer |
145
+ | `harness` | `claude` or `codex` |
146
+ | `answer` | the final text |
147
+ | `structured` | the parsed answer when `output_schema` was given |
148
+ | `session_id` | claude session id or codex thread id, set whenever known, including after a kill |
149
+ | `model` | the model that ran when claude reports it, else the requested one |
150
+ | `cost_usd` | claude's own estimate; `None` on codex |
151
+ | `duration_ms`, `num_turns` | as reported; `num_turns` is `None` on codex |
152
+ | `tokens` | `input`, `output`, `cache_read`, `cache_creation` for this turn |
153
+ | `permission_denials` | claude's list; empty on codex |
154
+ | `failure` | `None` when `ok`; else `kind` and the harness's own `message` |
155
+ | `exit_code` | the exit code the claude SDK reported for a failed process, else `None` |
156
+ | `stderr_tail` | the harness's last stderr lines, for a person |
157
+ | `command` | the CLI command line the launcher started |
158
+ | `raw` | claude's result message, or codex's `{"turn", "items", "errors"}` |
159
+ | `harness_version` | the installed CLI version the guard read |
160
+ | `warnings` | the guard's warning when the CLI is newer than tested |
161
+
162
+ ### Failure kinds
163
+
164
+ `quota`, `model_refused`, `not_authenticated`, `timeout`, `cancelled`, `binary_missing`, `max_turns`, `budget`, `invalid_output`, `error`, `crash`, `incompatible_harness`. The harness's own verdict is read first (claude's result subtype, codex's `codexErrorInfo`), then an HTTP status it reports, then its sentence: a spent quota before a refused model, because a subscription spent on one model says both and only one of them is a pause. A claude turn that times out while the harness is retrying a 401 or a 429 is `not_authenticated` or `quota`. `failure.retryable` is true for the kinds where running the same turn again later can succeed.
165
+
166
+ ## Changelog
167
+
168
+ ### 0.3.0
169
+
170
+ The library is renamed and speaks of harnesses, profiles and runs; its behaviour is exactly 0.2.0's. What a 0.2.0 consumer must change: depend on `callva-harness-runner` and import `callva.harness_runner`; write `harness` where a profile said `engine`; read `Result.harness` and `Result.harness_version`; branch on `incompatible_harness`; import `HARNESSES`. The table at the top maps every old name.
171
+
172
+ ### 0.2.0
173
+
174
+ The engines now run through the vendors' SDKs, and the profile is explicit knobs. What a 0.1.x consumer must change:
175
+
176
+ - `fence` is gone, and so is its default. A 0.1.x profile that named no fence ran fenced to `read`; the same profile now runs with the engine's own defaults. State what the turn may do with `tools`, `allowed_tools`, `disallowed_tools`, `permission_mode`, `setting_sources`, `settings`, `mcp_config` and `strict_mcp` on claude, and `codex_config` and `approval_policy` on codex. `Profile.from_dict` refuses a `fence` key with a message naming these knobs, and `Profile.fence` no longer exists.
177
+ - `allow_tools` becomes `tools` plus `allowed_tools`. `env` (`allow`, `deny`, `set`) becomes `env_set`, `env_remove` and `strip_session_markers`; there is no allow-list. `extra_args` becomes `claude_extra_args` (a mapping) or `codex_extra_args`. `EnvPolicy`, `DEFAULT_DENY` and `FENCES` are no longer exported.
178
+ - `run()` loses `schema` (now the profile's `output_schema`), `name` (now the profile's `name`) and `prompt_via` (the prompt always travels over the SDK's own channel).
179
+ - A knob the engine cannot honour is refused instead of ignored: `budget_usd` on codex, `service_tier` on claude.
180
+ - `on_event` receives SDK messages (claude) and app-server notifications (codex) instead of CLI JSON lines. `raw` changes shape the same way, `command` is the command line the launcher started, and `exit_code` is `None` unless the claude SDK reported one.
181
+ - A CLI older than the tested range is refused with the new kind `incompatible_engine`; `Result` gains `engine_version` and `warnings`.
182
+ - Two dependencies arrive, about 0.5 GB.
183
+
184
+ Unchanged: `Profile.from_dict`, `Session.fresh`, `Session.pinned`, `Session.resume`, `run(prompt, profile, cwd, session=, environ=, extra_env=, on_event=, cancel=)`, `FailureKind`, and every 0.1.x `Result` field.
185
+
186
+ ## Versioning
187
+
188
+ SemVer with the 0.x rule: the minor is the breaking position. A changed field meaning, a removed knob or a changed default is a minor; an added result field, an added knob whose default leaves the harness alone, or a widened tested range is a patch.
189
+
190
+ ## License
191
+
192
+ Apache-2.0.