b3-prosaic-runtime 0.8.1__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- b3_prosaic_runtime-0.8.1.dist-info/METADATA +650 -0
- b3_prosaic_runtime-0.8.1.dist-info/RECORD +46 -0
- b3_prosaic_runtime-0.8.1.dist-info/WHEEL +5 -0
- b3_prosaic_runtime-0.8.1.dist-info/entry_points.txt +2 -0
- b3_prosaic_runtime-0.8.1.dist-info/licenses/LICENSE +202 -0
- b3_prosaic_runtime-0.8.1.dist-info/licenses/LICENSE-MIT +21 -0
- b3_prosaic_runtime-0.8.1.dist-info/licenses/NOTICE.md +16 -0
- b3_prosaic_runtime-0.8.1.dist-info/top_level.txt +1 -0
- prosaic_runtime/__init__.py +27 -0
- prosaic_runtime/accounting.py +256 -0
- prosaic_runtime/accounting_capture.py +275 -0
- prosaic_runtime/admission.py +57 -0
- prosaic_runtime/anthropic.py +205 -0
- prosaic_runtime/anthropic_stream.py +179 -0
- prosaic_runtime/anthropic_usage.py +96 -0
- prosaic_runtime/artifacts.py +56 -0
- prosaic_runtime/budgets.py +67 -0
- prosaic_runtime/cli.py +153 -0
- prosaic_runtime/cli_tools.py +313 -0
- prosaic_runtime/config.py +127 -0
- prosaic_runtime/conformance.py +314 -0
- prosaic_runtime/console.py +37 -0
- prosaic_runtime/diagnostics.py +132 -0
- prosaic_runtime/events.py +56 -0
- prosaic_runtime/execution.py +442 -0
- prosaic_runtime/http_bounds.py +66 -0
- prosaic_runtime/openai_compatible.py +2373 -0
- prosaic_runtime/openai_compatible_compaction.py +219 -0
- prosaic_runtime/openai_compatible_filters.py +147 -0
- prosaic_runtime/openai_compatible_progress.py +405 -0
- prosaic_runtime/openai_compatible_transcript.py +79 -0
- prosaic_runtime/operation_context.py +39 -0
- prosaic_runtime/policy.py +50 -0
- prosaic_runtime/provider_types.py +19 -0
- prosaic_runtime/runtime.py +562 -0
- prosaic_runtime/sandbox.py +179 -0
- prosaic_runtime/smoke_prose/pilot.md +7 -0
- prosaic_runtime/smoke_prose/subagents/native-summarizer.md +10 -0
- prosaic_runtime/smoke_prose/subagents/reviewer.md +22 -0
- prosaic_runtime/smoke_prose/subagents/summarizer.md +11 -0
- prosaic_runtime/structured_tools.py +193 -0
- prosaic_runtime/telemetry.py +128 -0
- prosaic_runtime/tool_effects.py +112 -0
- prosaic_runtime/tool_registry.py +36 -0
- prosaic_runtime/tools.py +261 -0
- prosaic_runtime/types.py +23 -0
|
@@ -0,0 +1,650 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: b3-prosaic-runtime
|
|
3
|
+
Version: 0.8.1
|
|
4
|
+
Summary: Bounded execution of neutral Prosaic agents on OpenAI-compatible and native Anthropic endpoints
|
|
5
|
+
Author: B3 Cognition
|
|
6
|
+
License-Expression: Apache-2.0
|
|
7
|
+
Requires-Python: >=3.11
|
|
8
|
+
Description-Content-Type: text/markdown
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
License-File: LICENSE-MIT
|
|
11
|
+
License-File: NOTICE.md
|
|
12
|
+
Requires-Dist: b3-prosaic<0.5,>=0.4
|
|
13
|
+
Requires-Dist: PyYAML>=6.0
|
|
14
|
+
Requires-Dist: jsonschema>=4.23
|
|
15
|
+
Provides-Extra: test
|
|
16
|
+
Requires-Dist: pytest>=8; extra == "test"
|
|
17
|
+
Requires-Dist: build>=1.2; extra == "test"
|
|
18
|
+
Requires-Dist: setuptools>=77; extra == "test"
|
|
19
|
+
Requires-Dist: wheel; extra == "test"
|
|
20
|
+
Requires-Dist: packaging>=24; extra == "test"
|
|
21
|
+
Dynamic: license-file
|
|
22
|
+
|
|
23
|
+
# Prosaic Runtime
|
|
24
|
+
|
|
25
|
+
The **0.8.1** candidate is distributed as `b3-prosaic-runtime` and requires
|
|
26
|
+
`b3-prosaic>=0.4,<0.5` from the package index. Python imports (`prosaic_runtime`)
|
|
27
|
+
and the `prosaic-runtime` CLI remain unchanged. The separately installed recorder
|
|
28
|
+
is `b3-prosaic-runtime-postgres` **0.2.1**, selecting Runtime 0.8.x.
|
|
29
|
+
Candidates are not public index releases until the release qualification gate passes.
|
|
30
|
+
|
|
31
|
+
Runtime exports pure execution-artifact and native registry validation for host
|
|
32
|
+
workflow admission and reuses Core's canonical in-memory artifact validation.
|
|
33
|
+
|
|
34
|
+
Version **0.7.0** adds bounded native Anthropic text, streaming and tools, alongside
|
|
35
|
+
OpenAI-compatible execution. The separate `prosaic-runtime-postgres` **0.1.1**
|
|
36
|
+
recorder validates real schema and runtime grants without DDL. Release assets include both packages;
|
|
37
|
+
install the Runtime wheel before the adapter wheel. Accounting stays opt-in.
|
|
38
|
+
|
|
39
|
+
Optional customer usage metering is described in [Accounting](docs/accounting.md).
|
|
40
|
+
Existing callers need no IDs or database; durable tracking uses a separately installed
|
|
41
|
+
PostgreSQL recorder and does not enable customer charges.
|
|
42
|
+
|
|
43
|
+
Runtime also offers optional [bounded observation](docs/telemetry.md) through
|
|
44
|
+
`run(observer=..., operation_context=InvocationScope(...))` and the `observer_v1`
|
|
45
|
+
capability. Ordinary observer exceptions leave execution results unchanged.
|
|
46
|
+
The existing `on_event` callback remains a critical evidence hook. Operation
|
|
47
|
+
scope supplies opaque host correlation without enabling accounting.
|
|
48
|
+
|
|
49
|
+
Installation and CI do not require Node.js or npm.
|
|
50
|
+
|
|
51
|
+
Version 0.5.3 retains trusted macOS framework Python startup probes using their
|
|
52
|
+
declared sandbox working directory and pins the updated permission documentation.
|
|
53
|
+
|
|
54
|
+
See the [prompt-injection audit and remaining isolation gaps](docs/security.md)
|
|
55
|
+
before granting tools access to sensitive workspaces.
|
|
56
|
+
|
|
57
|
+
Version 0.5.0 adds [custom CLI tools, discovery and offline preflight](docs/cli-tools.md),
|
|
58
|
+
with a complete runnable CLI and an optional Understanding adapter.
|
|
59
|
+
|
|
60
|
+
Version 0.5.0 also adds [host-registered custom tools](examples/README.md#host-registered-custom-tools):
|
|
61
|
+
validated Python callbacks with native function calling. Both APIs are opt-in;
|
|
62
|
+
Python callbacks execute trusted host code and are not sandboxed. CLI subprocesses
|
|
63
|
+
are also unsandboxed by default; the opt-in below isolates them.
|
|
64
|
+
|
|
65
|
+
## Tool permissions and CLI isolation
|
|
66
|
+
|
|
67
|
+
Runtime **0.5.2** includes `cli_sandbox_v1` and opt-in CLI isolation.
|
|
68
|
+
Versions 0.5.1 and earlier do not include it; upgrading Prosaic alone does not
|
|
69
|
+
enable isolation. Start with the
|
|
70
|
+
[sandboxed CLI walkthrough](examples/README.md#sandboxed-cli-tools-runtime-052)
|
|
71
|
+
and its [complete configuration](examples/cli-tools-sandboxed.yml).
|
|
72
|
+
|
|
73
|
+
Markdown requests capabilities; it does not grant host authority:
|
|
74
|
+
|
|
75
|
+
| Layer | Example | Who owns it? |
|
|
76
|
+
| --- | --- | --- |
|
|
77
|
+
| Agent Markdown | `tools: [analyze_spec]` | Agent author declares what is needed |
|
|
78
|
+
| Runtime YAML | `allowed_tools: [analyze_spec]` | Operator allows the tool |
|
|
79
|
+
| Invocation | `--allow-tool analyze_spec --read-root evidence` | Host grants this call's tool and readable inputs |
|
|
80
|
+
| CLI OS boundary | `cli_sandbox: {mode: required}` in Runtime YAML | Operator requires isolation; prose cannot disable it |
|
|
81
|
+
|
|
82
|
+
All three tool declarations/grants must intersect. Merely installing a CLI or
|
|
83
|
+
adding it to frontmatter grants nothing. For custom CLIs, the operator also
|
|
84
|
+
reviews the executable and explicitly trusts its YAML manifest directory via
|
|
85
|
+
`tool_directories`; never automatically trust model-generated manifests.
|
|
86
|
+
|
|
87
|
+
Required mode confines CLI tools and their version probes to read-only evidence,
|
|
88
|
+
trusted dependencies and private writable HOME/scratch. Host IP network access
|
|
89
|
+
(including loopback) is blocked; model inference/auth stays in the parent.
|
|
90
|
+
Linux requires `/usr/bin/bwrap` >= 0.12.0 and working user namespaces; macOS
|
|
91
|
+
requires `/usr/bin/sandbox-exec`. Unavailable enforcement fails closed before
|
|
92
|
+
inference. There is no automatic unrestricted fallback.
|
|
93
|
+
|
|
94
|
+
This setting does **not** isolate builtin file tools, trusted Python callbacks,
|
|
95
|
+
validators, or native Codex/Claude providers. Builtin file tools still enforce
|
|
96
|
+
their own path grants. CLI workspace writes and external-service access are not
|
|
97
|
+
supported in required mode, even if builtin write paths are granted. Do not
|
|
98
|
+
grant HOME, `/`, or directories containing host Unix sockets as dependency/input
|
|
99
|
+
roots. See [the full setup and limitations](docs/cli-tools.md).
|
|
100
|
+
|
|
101
|
+
## Overview
|
|
102
|
+
|
|
103
|
+
Version 0.3.0 harness support: `tool_completed` events for successful `read_file`
|
|
104
|
+
now include `read_receipts` with path, SHA-256 of the bytes actually read, offset,
|
|
105
|
+
lines_read and line_count. Events contain no file text or raw tool arguments.
|
|
106
|
+
`RunPolicy(initial_tool="read_file", ...)` requests that granted function on the
|
|
107
|
+
first turn only; prose, runtime and host grants must all allow it. This cannot
|
|
108
|
+
elevate permissions or guarantee an endpoint honors tool choice. Consumers must
|
|
109
|
+
still validate the observed events. Defaults remain unchanged.
|
|
110
|
+
|
|
111
|
+
Execute neutral Prosaic commands and agents on OpenAI-compatible Chat Completions
|
|
112
|
+
or native Anthropic Messages endpoints. A Python library and CLI for small,
|
|
113
|
+
bounded tasks with streaming, tool calls, explicit filesystem permissions, and
|
|
114
|
+
structured results.
|
|
115
|
+
|
|
116
|
+
Version 0.4.0 hardening: explicit `initial_tool` selections now reject omitted,
|
|
117
|
+
substituted or additional first-turn tool calls before executing any of them.
|
|
118
|
+
Failure is `tool_choice_not_honored`, with reported token usage retained and no
|
|
119
|
+
automatic retry. This does not prove successful tool execution or correct tool
|
|
120
|
+
arguments; consumers must still validate receipts. Stream deltas are provisional
|
|
121
|
+
until the final result succeeds. Version 0.4.0 also adds opt-in acquisition prose
|
|
122
|
+
and the bounded execution fixes described below.
|
|
123
|
+
|
|
124
|
+
## First run: from installation to an answer
|
|
125
|
+
|
|
126
|
+
Follow these steps in order in one terminal. Commands below target macOS, Linux,
|
|
127
|
+
or Windows through WSL, using Bash or Zsh. You need an existing OpenAI-compatible
|
|
128
|
+
Chat Completions endpoint or an Anthropic API account: this runtime does **not**
|
|
129
|
+
install or host a model. The walkthrough below uses OpenAI-compatible endpoints;
|
|
130
|
+
see [native Anthropic setup](#native-anthropic-messages) for Claude.
|
|
131
|
+
Local and hosted endpoints both work; hosted calls may incur provider charges.
|
|
132
|
+
|
|
133
|
+
### 1. Check prerequisites
|
|
134
|
+
|
|
135
|
+
Install Git and Python 3.11 or newer (with pip and venv), then check:
|
|
136
|
+
|
|
137
|
+
```sh
|
|
138
|
+
git --version
|
|
139
|
+
python3 --version
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Use a Python executable that reports 3.11+ throughout. Prosaic 0.3.0 and the
|
|
143
|
+
runtime are Python packages; Node.js and npm are not required.
|
|
144
|
+
|
|
145
|
+
### 2. Install
|
|
146
|
+
|
|
147
|
+
Start in a directory where you keep projects. This creates a fresh checkout and
|
|
148
|
+
an isolated Python environment; no sudo or global Python installation is needed:
|
|
149
|
+
|
|
150
|
+
```sh
|
|
151
|
+
git clone https://github.com/B3Cognition/prosaic-runtime.git
|
|
152
|
+
cd prosaic-runtime
|
|
153
|
+
git checkout v0.8.1
|
|
154
|
+
python3 -m venv .venv
|
|
155
|
+
source .venv/bin/activate
|
|
156
|
+
python -m pip install .
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
The package installs compatible `b3-prosaic` 0.4.x into the same environment.
|
|
160
|
+
Use the release tag after publication; older release tags retain historical docs.
|
|
161
|
+
|
|
162
|
+
For index installation after publication, create a fresh virtual environment or
|
|
163
|
+
container image and run `python -m pip install 'b3-prosaic-runtime==0.8.1'`.
|
|
164
|
+
Install the optional recorder with
|
|
165
|
+
`python -m pip install 'b3-prosaic-runtime-postgres==0.2.1'`. Never add renamed
|
|
166
|
+
packages to an environment containing legacy `prosaic`, `prosaic-runtime` or
|
|
167
|
+
`prosaic-runtime-postgres`: they share import paths. Preserve old environments
|
|
168
|
+
for historical reconstruction. PyPI's unrelated `prosaic` is not a dependency.
|
|
169
|
+
Before publication, install only the explicit candidate wheels from an audited
|
|
170
|
+
wheelhouse with `--no-index --find-links /path/to/qualified-wheelhouse`.
|
|
171
|
+
|
|
172
|
+
Confirm both commands are available:
|
|
173
|
+
|
|
174
|
+
```sh
|
|
175
|
+
prosaic --help
|
|
176
|
+
prosaic-runtime --help
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
For future terminals, enter the checkout and run `source .venv/bin/activate`
|
|
180
|
+
again. The Python package installs Prosaic, PyYAML and jsonschema automatically.
|
|
181
|
+
|
|
182
|
+
### 3. Open the ready-made examples
|
|
183
|
+
|
|
184
|
+
```sh
|
|
185
|
+
cd examples
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
Stay in this directory for the remaining commands. The checkout already contains:
|
|
189
|
+
|
|
190
|
+
```text
|
|
191
|
+
examples/
|
|
192
|
+
.prosaic/subagents/summarizer.md # prose without tools
|
|
193
|
+
.prosaic/subagents/reviewer.md # prose requesting read tools
|
|
194
|
+
evidence/pilot.md # sample input for the reviewer
|
|
195
|
+
prosaic-runtime.yaml # no-tool endpoint configuration
|
|
196
|
+
with-tools.yml # read-tool endpoint configuration
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
There are no files to create before the first run. The neutral Markdown prose
|
|
200
|
+
uses YAML frontmatter; runtime configuration uses YAML too.
|
|
201
|
+
|
|
202
|
+
### 4. Configure your endpoint and credentials
|
|
203
|
+
|
|
204
|
+
Open `prosaic-runtime.yaml` and `with-tools.yml` in your editor. In **both** files,
|
|
205
|
+
replace `base_url` and `model` with values supplied by your endpoint operator.
|
|
206
|
+
Keep the different `allowed_tools` values: `[]` for the first example and
|
|
207
|
+
`[read_file]` for the second.
|
|
208
|
+
|
|
209
|
+
The no-tool file looks like this:
|
|
210
|
+
|
|
211
|
+
```yaml
|
|
212
|
+
default_profile: small
|
|
213
|
+
allowed_tools: []
|
|
214
|
+
limits:
|
|
215
|
+
timeout_s: 180
|
|
216
|
+
max_tool_rounds: 8
|
|
217
|
+
|
|
218
|
+
routes:
|
|
219
|
+
fast: small
|
|
220
|
+
|
|
221
|
+
profiles:
|
|
222
|
+
small:
|
|
223
|
+
base_url: http://127.0.0.1:8000/v1
|
|
224
|
+
model: your-model
|
|
225
|
+
api_key_env: LOCAL_LLM_API_KEY
|
|
226
|
+
max_tokens: 2048
|
|
227
|
+
features:
|
|
228
|
+
streaming: true
|
|
229
|
+
stream_options: true
|
|
230
|
+
```
|
|
231
|
+
|
|
232
|
+
The localhost URL and `your-model` are placeholders. Start your local model server
|
|
233
|
+
separately, or use your hosted endpoint's URL. Supply the API base (often ending
|
|
234
|
+
in `/v1`), **not** the full `/chat/completions` URL: the runtime appends that path.
|
|
235
|
+
Use the endpoint's exact model identifier. The tool example additionally needs
|
|
236
|
+
function-calling support.
|
|
237
|
+
|
|
238
|
+
If your endpoint requires a Bearer API key, enter it without putting it in shell
|
|
239
|
+
history or a configuration file:
|
|
240
|
+
|
|
241
|
+
```sh
|
|
242
|
+
export LOCAL_LLM_API_KEY="$(python -c 'import getpass; print(getpass.getpass("Endpoint API key: "))')"
|
|
243
|
+
```
|
|
244
|
+
|
|
245
|
+
For an unauthenticated local endpoint, leave that variable unset:
|
|
246
|
+
|
|
247
|
+
```sh
|
|
248
|
+
unset LOCAL_LLM_API_KEY
|
|
249
|
+
```
|
|
250
|
+
|
|
251
|
+
Run only the credential command appropriate for your endpoint. Do not commit
|
|
252
|
+
credentials. If the endpoint rejects usage streaming, set `stream_options: false`;
|
|
253
|
+
if it does not stream, set `streaming: false`. If it rejects `reasoning_effort`,
|
|
254
|
+
remove `effort: low` from `.prosaic/subagents/summarizer.md`.
|
|
255
|
+
|
|
256
|
+
Check setup before making a model request:
|
|
257
|
+
|
|
258
|
+
```sh
|
|
259
|
+
prosaic-runtime doctor --output text
|
|
260
|
+
```
|
|
261
|
+
|
|
262
|
+
This checks configuration, the Prosaic CLI, configured credentials and authenticated
|
|
263
|
+
model discovery. It does **not** run inference. A server without a `/models` route
|
|
264
|
+
produces a warning; authentication and connectivity failures produce a failed check.
|
|
265
|
+
Model discovery alone does not prove completion or tool support.
|
|
266
|
+
|
|
267
|
+
### 5. Run prose without tools
|
|
268
|
+
|
|
269
|
+
```sh
|
|
270
|
+
prosaic-runtime subagents/summarizer.md \
|
|
271
|
+
--arguments 'S1: The pilot processed 120 requests. S2: Three requests timed out. S3: The cause has not been established.' \
|
|
272
|
+
> summary.json
|
|
273
|
+
```
|
|
274
|
+
|
|
275
|
+
Check the result and print the answer:
|
|
276
|
+
|
|
277
|
+
```sh
|
|
278
|
+
python -c 'import json; r=json.load(open("summary.json")); print(r["stdout"]); print("exit_code:", r["exit_code"]); assert r["exit_code"] == 0, r["stderr"]'
|
|
279
|
+
```
|
|
280
|
+
|
|
281
|
+
An illustrative answer is:
|
|
282
|
+
|
|
283
|
+
```text
|
|
284
|
+
The pilot processed 120 requests (S1). Three requests timed out (S2).
|
|
285
|
+
The cause has not been established (S3).
|
|
286
|
+
exit_code: 0
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
Wording varies by model. `summary.json` contains a result object with
|
|
290
|
+
`event: "result"`, `exit_code`, `stdout`, `stderr`, usage, and metadata. No tool is offered
|
|
291
|
+
to this agent. Execution success does not establish factual correctness—compare
|
|
292
|
+
the answer against the supplied input.
|
|
293
|
+
|
|
294
|
+
### 6. Run prose with a read-only tool
|
|
295
|
+
|
|
296
|
+
```sh
|
|
297
|
+
prosaic-runtime subagents/reviewer.md \
|
|
298
|
+
--config with-tools.yml \
|
|
299
|
+
--arguments 'Review evidence/pilot.md. What do we know, and what remains unknown?' \
|
|
300
|
+
--allow-tool read_file \
|
|
301
|
+
--read-root ./evidence \
|
|
302
|
+
> review.json
|
|
303
|
+
python -c 'import json; r=json.load(open("review.json")); print(r["stdout"]); print("exit_code:", r["exit_code"]); assert r["exit_code"] == 0, r["stderr"]'
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
Expect a read of `evidence/pilot.md`, followed by **Observed**, **Unknown**, and
|
|
307
|
+
**Next check** sections. Findings should cite that path and source IDs: 120 requests,
|
|
308
|
+
three timeouts, correctness not evaluated, timeout cause unknown, and no comparison
|
|
309
|
+
against another endpoint. A suggested next check is collecting server logs, not
|
|
310
|
+
claiming a diagnosis.
|
|
311
|
+
|
|
312
|
+
The tool must be requested by prose, allowed in configuration, **and** granted by
|
|
313
|
+
the invocation. Reads are limited to `evidence/`; this example cannot write files,
|
|
314
|
+
execute shell commands, or spawn agents.
|
|
315
|
+
|
|
316
|
+
### 7. Watch streaming events
|
|
317
|
+
|
|
318
|
+
```sh
|
|
319
|
+
prosaic-runtime subagents/reviewer.md \
|
|
320
|
+
--config with-tools.yml \
|
|
321
|
+
--arguments 'Review evidence/pilot.md.' \
|
|
322
|
+
--allow-tool read_file \
|
|
323
|
+
--read-root ./evidence \
|
|
324
|
+
--events
|
|
325
|
+
```
|
|
326
|
+
|
|
327
|
+
This prints one JSON object per line: events followed by a final `result` object.
|
|
328
|
+
It is JSONL, not a single JSON document. Look for `text_delta` events when the
|
|
329
|
+
endpoint streams text, tool progress, and the final result's `exit_code: 0`.
|
|
330
|
+
Intermediate events are diagnostic; the final result is authoritative.
|
|
331
|
+
|
|
332
|
+
Without `--events`, status and a five-second elapsed-time heartbeat go to **stderr**,
|
|
333
|
+
including while the model is reasoning silently. They do not contaminate JSON or
|
|
334
|
+
text on stdout. Use `--quiet` to suppress progress; errors remain visible.
|
|
335
|
+
|
|
336
|
+
### 8. Run the repeatable live smoke test
|
|
337
|
+
|
|
338
|
+
```sh
|
|
339
|
+
prosaic-runtime smoke --live --config with-tools.yml
|
|
340
|
+
```
|
|
341
|
+
|
|
342
|
+
`--live` is required: this deliberately makes model requests and may incur charges.
|
|
343
|
+
The installed package includes the same neutral summarizer/reviewer prose and sample
|
|
344
|
+
evidence, so this works without the repository's examples directory when you supply
|
|
345
|
+
a configuration path. It uses `default_profile`, or the profile selected with
|
|
346
|
+
`--profile NAME`, for both probes.
|
|
347
|
+
|
|
348
|
+
The test creates a temporary directory with synthetic evidence and grants only
|
|
349
|
+
`read_file` within that directory for the reviewer. It never reads your project
|
|
350
|
+
files. This fixed test grant is independent of your configuration's `allowed_tools`;
|
|
351
|
+
`--live` opts into these two narrowly scoped probes, not general agent permissions.
|
|
352
|
+
Timeout and round limits still apply to each probe separately.
|
|
353
|
+
|
|
354
|
+
The JSON report separates `completion`, observed `streaming`, and `tool_execution`
|
|
355
|
+
for each example. Exit 0 requires both completions and an actual successful file read;
|
|
356
|
+
a model merely claiming it read the file is not enough. Streaming is reported separately
|
|
357
|
+
and is not required for success, because non-streaming endpoints are supported.
|
|
358
|
+
This is a capability smoke test, not a model-quality benchmark.
|
|
359
|
+
|
|
360
|
+
For a larger demonstration, see the [four-tier launch dossier example](examples/README.md#four-tier-launch-dossier-demo):
|
|
361
|
+
four neutral Markdown subagents, a synthetic 5,100-word dossier, YAML model-tier
|
|
362
|
+
routing, scoped read-only tools, and a runnable program with text or JSONL output.
|
|
363
|
+
Its documented live results distinguish successful execution from answer quality.
|
|
364
|
+
|
|
365
|
+
For a single no-tool inference check instead:
|
|
366
|
+
|
|
367
|
+
```sh
|
|
368
|
+
prosaic-runtime doctor --inference --timeout 180
|
|
369
|
+
```
|
|
370
|
+
|
|
371
|
+
Inference is never a fallback for a failed default doctor check: only the explicit
|
|
372
|
+
`--inference` flag enables that request.
|
|
373
|
+
|
|
374
|
+
For a canonical capability report, run
|
|
375
|
+
`prosaic-runtime conformance --config runtime.yml --profile small`. Its default
|
|
376
|
+
produces explicit unexecuted evidence with no Runtime construction or probes.
|
|
377
|
+
`--live` opts into a finite shared allowance for seven synthetic provider/tool
|
|
378
|
+
checks. Fixture results remain `fixture_passed` and cannot qualify a live model.
|
|
379
|
+
See [deterministic conformance](docs/conformance.md) for replay, profile/origin
|
|
380
|
+
binding, measured assertions, canonical serialization, and execution bounds.
|
|
381
|
+
|
|
382
|
+
### Troubleshooting
|
|
383
|
+
|
|
384
|
+
| Symptom | Check or fix |
|
|
385
|
+
| --- | --- |
|
|
386
|
+
| `python3` missing | Install the prerequisites from step 1; on some Linux systems, Python venv is a separate OS package. |
|
|
387
|
+
| `prosaic-runtime` not found | Activate `.venv` from the checkout root and rerun `python -m pip install .`. |
|
|
388
|
+
| `prosaic` not found | Activate `.venv` and repeat the pinned Python Prosaic installation in step 2. |
|
|
389
|
+
| No configuration or artifact found | Run from `examples/`; otherwise pass `--config` and `--source` explicitly. |
|
|
390
|
+
| YAML error | Use spaces for indentation and preserve the nested structure shown above. |
|
|
391
|
+
| Connection refused or timeout | Start the local endpoint or check its host, port, connectivity, and availability. |
|
|
392
|
+
| HTTP 401/403 | Check the API key, its permissions, and `api_key_env`; do not print the key in logs. |
|
|
393
|
+
| HTTP 404 or model error | Check the API base path and exact model ID; do not include `/chat/completions` in `base_url`. |
|
|
394
|
+
| Unsupported request field | Disable `stream_options` or `streaming`, or remove prose `effort`, as described in step 4. |
|
|
395
|
+
| Tool unavailable or read denied | Check all three grants and `--read-root`; use a function-calling model and a path inside `evidence/`. |
|
|
396
|
+
| Nonzero exit or incomplete output | Read the result's `stderr` and `metadata`; check token/round limits before retrying. Setup errors are printed to terminal stderr instead. |
|
|
397
|
+
|
|
398
|
+
The runtime exits nonzero on failure. Files such as `summary.json` can be empty
|
|
399
|
+
when configuration fails before execution; inspect the terminal error first.
|
|
400
|
+
There are no automatic retries or paid-model quality guarantees.
|
|
401
|
+
|
|
402
|
+
## Configuration and further examples
|
|
403
|
+
|
|
404
|
+
Configuration discovery checks `prosaic-runtime.yaml`, then `prosaic-runtime.yml`,
|
|
405
|
+
in the current directory. Use `--config path/to/config.yml` or pass a path to
|
|
406
|
+
`ProsaicRuntime.from_config()` to select another file. Only YAML/YML is supported.
|
|
407
|
+
|
|
408
|
+
See the [examples walkthrough](examples/README.md) for the neutral prose files,
|
|
409
|
+
permission details, and Python equivalents. To create your own agent, add a
|
|
410
|
+
Markdown file under `.prosaic/subagents/` with neutral YAML frontmatter and a body
|
|
411
|
+
using `{{args}}`, following the checked-in summarizer or reviewer.
|
|
412
|
+
|
|
413
|
+
## Plain-text output
|
|
414
|
+
|
|
415
|
+
For just the answer in the terminal, use `--output text`:
|
|
416
|
+
|
|
417
|
+
```sh
|
|
418
|
+
prosaic-runtime subagents/summarizer.md --arguments 'Text to summarize' --output text
|
|
419
|
+
```
|
|
420
|
+
|
|
421
|
+
JSON remains the default. `--events` emits JSONL and cannot be combined with
|
|
422
|
+
`--output`. On failures, text mode prints any partial answer to stdout, a diagnostic
|
|
423
|
+
to stderr, and exits nonzero. Do not treat a partial answer as a successful result.
|
|
424
|
+
|
|
425
|
+
## Python
|
|
426
|
+
|
|
427
|
+
The complete runnable program is [examples/run_examples.py](examples/run_examples.py).
|
|
428
|
+
From the checkout root, run `python examples/run_examples.py` after configuring the
|
|
429
|
+
example YAML files and setting your API-key environment variable. See its
|
|
430
|
+
[usage instructions](examples/README.md#runnable-python-sample) for endpoint/model
|
|
431
|
+
overrides and streaming events.
|
|
432
|
+
|
|
433
|
+
```python
|
|
434
|
+
from prosaic_runtime import ProsaicRuntime, RunPolicy
|
|
435
|
+
|
|
436
|
+
runtime = ProsaicRuntime.from_config("prosaic-runtime.yaml")
|
|
437
|
+
result = runtime.run(
|
|
438
|
+
"subagents/summarizer.md",
|
|
439
|
+
arguments="Text to summarize",
|
|
440
|
+
policy=RunPolicy(timeout_s=60),
|
|
441
|
+
)
|
|
442
|
+
if result.exit_code:
|
|
443
|
+
raise RuntimeError(result.stderr)
|
|
444
|
+
print(result.stdout)
|
|
445
|
+
```
|
|
446
|
+
|
|
447
|
+
An orchestrator can pass `ProsaicArtifact.from_inspection(inspect_json)` to avoid spawning Prosaic on every invocation. The snapshot includes artifact identity, neutral frontmatter, body, and bundled resources. Results identify the artifact by a SHA-256 digest. Bundle contents are appended as named context; external prose references are the caller's responsibility.
|
|
448
|
+
|
|
449
|
+
## Tools and authority
|
|
450
|
+
|
|
451
|
+
### Opt-in acquisition (v0.4.0+)
|
|
452
|
+
|
|
453
|
+
Pass `acquisition="subagents/acquire-pilot.md"` to `runtime.run()` to send a
|
|
454
|
+
short neutral Prosaic artifact first. The final artifact body, resources and
|
|
455
|
+
arguments are withheld until the explicitly selected native tool succeeds.
|
|
456
|
+
Both artifacts use one model, conversation, deadline and tool-round budget.
|
|
457
|
+
The acquisition declaration cannot broaden final prose/host/config grants;
|
|
458
|
+
it must declare `policy.initial_tool`. Its tier/effort must match the final
|
|
459
|
+
artifact or be omitted. Acquisition receives **no final arguments**; put the
|
|
460
|
+
needed path in its own prose. First-turn JSON response formatting is withheld
|
|
461
|
+
too, then restored for analysis. Subsequent tool selection is automatic.
|
|
462
|
+
|
|
463
|
+
A missing/substituted/extra first call blocks with `tool_choice_not_honored`;
|
|
464
|
+
a failed required tool blocks with `acquisition_failed`. Neither failure retries
|
|
465
|
+
nor switches to preloading. `acquisition_v1` advertises support. Existing calls
|
|
466
|
+
without this option retain their behavior. Acquisition requires Runtime v0.4.0+.
|
|
467
|
+
|
|
468
|
+
Run the complete staged and no-tool examples from this checkout:
|
|
469
|
+
|
|
470
|
+
```sh
|
|
471
|
+
python examples/run_acquisition.py --config examples/tokenproxy.yml --profile qwen
|
|
472
|
+
```
|
|
473
|
+
|
|
474
|
+
Set `TOKENPROXY_KEY` first. Use `--mode staged` or `--mode preloaded` to choose
|
|
475
|
+
explicitly, `--no-stream` for non-streaming, or another configured profile.
|
|
476
|
+
The preloaded mode reads a fixed example file on the host and supplies its text
|
|
477
|
+
to tool-free Markdown prose; it does not fabricate native tool receipts.
|
|
478
|
+
See [examples/README.md](examples/README.md#staged-acquisition-and-explicit-preloading).
|
|
479
|
+
This program demonstrates execution, not factual validation; the Harness
|
|
480
|
+
companion supplies schema and source/quote admission checks.
|
|
481
|
+
|
|
482
|
+
Tools are enabled only by the intersection of the artifact's declaration, configuration `allowed_tools`, and invocation `RunPolicy.allowed_tools`. Filesystem tools also require explicit roots or exact output paths. Missing and empty grants deny access. Prose and provider responses cannot expand the grant.
|
|
483
|
+
|
|
484
|
+
Supported declarations are omitted/empty/`none`, `read`, `write`, or a list of supported tool names in an inspection snapshot. `write` requests both read and write tools. `full` is rejected because this runtime does not implement arbitrary shell execution or delegation.
|
|
485
|
+
|
|
486
|
+
Read tools: `read_file`, `read_many_files`, `sha256_file`, `list_files`, `list_tree_with_sizes`, `grep_files`, `grep_context`. Write tools: `write_file`, `edit_file`.
|
|
487
|
+
|
|
488
|
+
For a read agent, set `tools: read` in its Prosaic frontmatter, `allowed_tools: [read_file]` in configuration, and invoke:
|
|
489
|
+
|
|
490
|
+
```sh
|
|
491
|
+
prosaic-runtime subagents/reviewer.md --allow-tool read_file --read-root ./evidence
|
|
492
|
+
```
|
|
493
|
+
|
|
494
|
+
Paths are relative to `--cwd` (the current directory by default) and must resolve inside it. `--write-path` grants one exact path; `--forbid-root` excludes a path and its descendants. An edit grant allows reading the target as part of replacing its contents. Resolved symlink escapes are rejected. These guards are for the provided file tools; they are not an OS sandbox against concurrent hostile filesystem changes.
|
|
495
|
+
|
|
496
|
+
The CLI provides no general-purpose shell, web browsing, recursive agents,
|
|
497
|
+
workflow scheduler, or application state writer. Explicitly trusted CLI manifests
|
|
498
|
+
can execute fixed application commands. They are unsandboxed in the default
|
|
499
|
+
`off` mode; `required` mode adds the CLI OS boundary described above.
|
|
500
|
+
Hosts retain their output validation, workflow decisions and publication rules.
|
|
501
|
+
|
|
502
|
+
## Limits and events
|
|
503
|
+
|
|
504
|
+
Default invocation limits: 120 seconds, 8 tool rounds, 256 KiB input. Endpoint defaults: 4,096 output tokens per turn and 8 MiB captured response per turn. Conversation requests are checked against the input limit before tool-loop turns; oversized responses fail. Old tool results can be compacted. Repeated identical tool rounds disable tools for a final answer.
|
|
505
|
+
|
|
506
|
+
YAML can override time and round defaults:
|
|
507
|
+
|
|
508
|
+
```yaml
|
|
509
|
+
limits:
|
|
510
|
+
timeout_s: 180
|
|
511
|
+
max_tool_rounds: 4
|
|
512
|
+
```
|
|
513
|
+
|
|
514
|
+
CLI `--timeout` and `--max-tool-rounds` override these values independently.
|
|
515
|
+
The Python API uses YAML limits when `policy` is omitted; an explicit `RunPolicy`
|
|
516
|
+
replaces them. The checked-in examples use 180 seconds to accommodate slower models.
|
|
517
|
+
|
|
518
|
+
`on_event` receives `started`, `text_delta`, `progress`, `text`, `tool_started`,
|
|
519
|
+
`tool_completed`, and `completed` dictionaries. Tool events contain `name`, `call_id`,
|
|
520
|
+
and `turn`; completion also contains `status` and `duration_ms`. They include denied
|
|
521
|
+
calls and omit arguments and file contents. The older diagnostic `progress` events
|
|
522
|
+
can contain paths or model text, so do not treat the entire event stream as redacted.
|
|
523
|
+
The final `Result` is authoritative. Event handlers execute synchronously; keep them
|
|
524
|
+
fast. The `cancelled` callback is checked between streaming reads, tool-loop operations,
|
|
525
|
+
and requests. In-flight blocking I/O is bounded by the HTTP timeout; cancellation is cooperative.
|
|
526
|
+
Staged invocations additionally emit `acquisition_completed` after the required
|
|
527
|
+
tool succeeds. If a later turn is cancelled, times out or exceeds an input/
|
|
528
|
+
response limit, already reported completed-turn usage is retained and marked
|
|
529
|
+
`usage_scope: reported_completed_turns`; it is not an estimate of an unfinished
|
|
530
|
+
request's usage.
|
|
531
|
+
|
|
532
|
+
Usage is reported when supplied by the endpoint. Cost estimation is unavailable in v0.2; `cost_usd = 0` is a legacy compatibility field, not a claim that inference was free. Check `metadata.cost_status`. No retries or model escalation occur automatically. An incomplete or truncated final response is unsuccessful.
|
|
533
|
+
|
|
534
|
+
## Native Anthropic Messages
|
|
535
|
+
|
|
536
|
+
Endpoint profiles accept `provider: anthropic` for native Messages requests.
|
|
537
|
+
Omitting `provider` retains `openai-compatible`; both can coexist in the same
|
|
538
|
+
configuration and use the existing `routes` by model tier. Configure an API base
|
|
539
|
+
such as `https://api.anthropic.com/v1`, without `/messages`, and a model available
|
|
540
|
+
to your account. Runtime appends `/messages` and sends `x-api-key` plus the
|
|
541
|
+
Anthropic API version header. Model discovery uses the same native credentials.
|
|
542
|
+
|
|
543
|
+
Start with [examples/anthropic.yml](examples/anthropic.yml). Replace
|
|
544
|
+
`your-anthropic-model`, set `ANTHROPIC_API_KEY` in your shell, and run from
|
|
545
|
+
`examples/`:
|
|
546
|
+
|
|
547
|
+
```sh
|
|
548
|
+
prosaic-runtime subagents/native-summarizer.md --config anthropic.yml \
|
|
549
|
+
--arguments "Summarize this text." --output text
|
|
550
|
+
prosaic-runtime subagents/reviewer.md --config anthropic.yml \
|
|
551
|
+
--arguments "Review evidence/pilot.md" --allow-tool read_file --read-root evidence
|
|
552
|
+
```
|
|
553
|
+
|
|
554
|
+
Use `temperature: null` to omit sampling parameters for models that do not
|
|
555
|
+
accept them. Native requests require a positive `max_tokens`. The historic
|
|
556
|
+
`summarizer.md` declares `effort: low`; use `native-summarizer.md` for a summary
|
|
557
|
+
without that control. This version explicitly rejects effort controls, JSON
|
|
558
|
+
mode, extended thinking, image inputs, and provider-hosted tools. It supports
|
|
559
|
+
text and Runtime-managed builtin, host-registered, and trusted CLI tools.
|
|
560
|
+
Runtime continues to enforce all existing grants, read receipts, first-tool
|
|
561
|
+
requirements, acquisition boundaries, cancellation, deadlines, and size limits.
|
|
562
|
+
Partial text events remain provisional, and incomplete streamed tool arguments
|
|
563
|
+
cannot execute. Provider failures are not automatically retried.
|
|
564
|
+
|
|
565
|
+
For SDK accounting, set the recorder's `provider_id='anthropic'`; a mismatch
|
|
566
|
+
with the routed endpoint fails before inference. For durable CLI accounting,
|
|
567
|
+
add `--accounting-provider anthropic` alongside the existing accounting options.
|
|
568
|
+
Each recorder serves its matching provider; use separate recorders for mixed
|
|
569
|
+
provider invocations. Ledger contracts remain unchanged.
|
|
570
|
+
|
|
571
|
+
Anthropic token totals include ordinary input, cache reads, cache creation, and
|
|
572
|
+
output. Cumulative stream usage snapshots are replaced rather than summed.
|
|
573
|
+
Missing details stay unknown; partial and conflicting evidence is retained.
|
|
574
|
+
The existing rate-card schema cannot represent cache-write pricing, nonstandard
|
|
575
|
+
service tiers, or regional pricing. Those categories and unfamiliar nonzero
|
|
576
|
+
usage categories remain unavailable for cost estimates. Raw native usage stays
|
|
577
|
+
in result metadata. See the official [streaming](https://platform.claude.com/docs/en/build-with-claude/streaming)
|
|
578
|
+
and [prompt caching](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)
|
|
579
|
+
documentation for native response semantics.
|
|
580
|
+
|
|
581
|
+
To explicitly request a live capability check, run
|
|
582
|
+
`prosaic-runtime smoke --live --config anthropic.yml` from `examples/`.
|
|
583
|
+
This makes paid API calls using bundled synthetic prose; automated tests use
|
|
584
|
+
local fixtures and require no Anthropic key.
|
|
585
|
+
|
|
586
|
+
## Echelon integration
|
|
587
|
+
|
|
588
|
+
The shared transport, stream parser, tool loop, compaction and filtering originated in Echelon. Echelon consumes the runtime through a compatibility adapter, retaining its own prompt policy, RE tools, `echelon_result` handling, journals, and delivery capabilities. The low-level `openai_compatible` module is an integration seam for that adapter, not the public agent-definition API. General consumers should use `ProsaicRuntime`.
|
|
589
|
+
|
|
590
|
+
## Development
|
|
591
|
+
|
|
592
|
+
Version 0.4.0 hardening: HTTP inference rejects redirects to protect endpoint
|
|
593
|
+
credentials. No-tool and tool-loop requests both obey the serialized byte bound.
|
|
594
|
+
Prosaic inspection timeouts return a failed `Result`. Public usage totals remain
|
|
595
|
+
unknown (`None`) when any completed turn lacks complete, nonnegative integer
|
|
596
|
+
usage. `token_usage_status` distinguishes unknown from reported totals; real
|
|
597
|
+
reported zero remains zero. Known partial sums are diagnostic
|
|
598
|
+
`reported_token_usage`, not a complete total for a consumer token budget.
|
|
599
|
+
|
|
600
|
+
For finite invocation allowances, set `RunPolicy(max_provider_requests=4,
|
|
601
|
+
max_tool_calls=2, max_reported_tokens=8192)`. Each field accepts a nonnegative
|
|
602
|
+
integer or `None` (legacy behavior); zero prevents dispatch in that unit.
|
|
603
|
+
Provider attempts are reserved before HTTP and accounting preparation. Tool
|
|
604
|
+
attempts include denied and malformed calls, with each call in a batch counted
|
|
605
|
+
once. Acquisition and follow-up turns share the same allowances.
|
|
606
|
+
|
|
607
|
+
With a reported-token cap, missing, invalid or partial totals and duplicate JSON
|
|
608
|
+
keys fail with `usage_unknown`; a terminal response above the cap fails with
|
|
609
|
+
`token_limit`.
|
|
610
|
+
A complete final response exactly at the cap succeeds, while further provider or
|
|
611
|
+
tool dispatch stops at equality. Returned text and already reported usage remain
|
|
612
|
+
available, including unknown totals and known partial sums. A failed provider
|
|
613
|
+
attempt keeps invocation usage unknown even after earlier turns reported complete
|
|
614
|
+
totals; its provider error remains available alongside the budget failure.
|
|
615
|
+
Accounting capture persists a completed response before budget admission. Token reports arrive
|
|
616
|
+
after a request, so one response can exceed the allowance; these controls do not
|
|
617
|
+
estimate an invoice ceiling or preempt a running callback. Endpoint `max_tokens`
|
|
618
|
+
keeps its provider meaning. Harness `max_calls` still counts workflow invocations.
|
|
619
|
+
|
|
620
|
+
The capability `invocation_budgets_v1` advertises support. When any allowance is
|
|
621
|
+
configured, `Result.metadata.invocation_budgets_v1` includes `provider_requests`,
|
|
622
|
+
`tool_calls`, `reported_tokens` and `usage_complete`. Exhausted request/tool
|
|
623
|
+
allowances return `provider_request_limit` or `tool_call_limit` in
|
|
624
|
+
`metadata.failure_reason`. Disabled allowances add no count metadata and retain
|
|
625
|
+
existing low-level consumer defaults.
|
|
626
|
+
|
|
627
|
+
```sh
|
|
628
|
+
python -m pip install -e '.[test]'
|
|
629
|
+
pytest
|
|
630
|
+
```
|
|
631
|
+
|
|
632
|
+
Tests cover extracted transport edge cases, tool denial, path containment, artifact loading, routing, cancellation, and local HTTP/SSE integration. The end-to-end Prosaic test requires the CLI on PATH; CI installs an explicit immutable Core candidate while index publication is pending. Tests do not require paid LLM access. Successful conformance tests do not establish model quality; evaluate each agent/model pair against its own acceptance criteria.
|
|
633
|
+
|
|
634
|
+
`tests/test_distribution_metadata.py` builds both distributions from `git archive`
|
|
635
|
+
(override the source with `PROSAIC_RUNTIME_RELEASE_REF`). The optional test extra
|
|
636
|
+
includes the backend required for no-isolation source-artifact verification.
|
|
637
|
+
Run `python -I scripts/wheel_smoke.py` with a fresh environment containing the
|
|
638
|
+
explicit Core, Runtime and recorder candidate wheels to check installed ownership,
|
|
639
|
+
CLI help, pure admission and documented recorder construction without model or DB calls.
|
|
640
|
+
|
|
641
|
+
Licensed under the [Apache License, Version 2.0](LICENSE). See [NOTICE.md](NOTICE.md)
|
|
642
|
+
for extraction provenance and [LICENSE-MIT](LICENSE-MIT) for the retained notice
|
|
643
|
+
covering historically MIT-licensed code. Third-party components retain their own licenses.
|
|
644
|
+
|
|
645
|
+
## Structured JSON tool transport
|
|
646
|
+
|
|
647
|
+
For endpoints where the host deliberately chooses JSON tool requests, use the
|
|
648
|
+
shared [`StructuredToolLoop`](docs/structured-tools.md). It reuses registered
|
|
649
|
+
`CustomTool` contracts and bounded execution, with application-owned final-result
|
|
650
|
+
admission. Harness can drive each turn without embedding product rules in Runtime.
|