cdx-manager 0.24.9 → 0.25.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -1
- package/VERSION +1 -1
- package/changelogs/CHANGELOGS_0_25_0.md +17 -0
- package/changelogs/CHANGELOGS_UNRELEASED.md +7 -0
- package/checksums/release-archives.json +11 -0
- package/docs/cli.md +13 -1
- package/package.json +1 -1
- package/pyproject.toml +1 -1
- package/src/cli_args.py +16 -1
- package/src/commands/runs.py +57 -9
- package/src/commands/status.py +1 -0
- package/src/provider_runtime.py +21 -2
- package/src/run_command.py +5 -1
package/README.md
CHANGED
|
@@ -4,7 +4,7 @@
|
|
|
4
4
|
|
|
5
5
|
<br clear="left"/>
|
|
6
6
|
|
|
7
|
-
[](LICENSE) ](LICENSE)  
|
|
8
8
|
|
|
9
9
|
**The right AI coding account, when you need it.** CDX Manager keeps your Codex and Claude accounts in separate profiles, shows which ones have quota, and helps you pick the next usable session from your terminal.
|
|
10
10
|
|
|
@@ -59,6 +59,16 @@ cdx run --continue-run RUN_ID --cwd /path/to/repo --prompt "Fix the remaining te
|
|
|
59
59
|
|
|
60
60
|
Conservative assumption: a continuation requires a finished Claude or Codex run, a locally available provider transcript, and recorded model and effort settings. Missing runs, profiles, expired sessions, and unavailable settings produce a clear error; the caller can retry without `--continue-run` for a cold run. `--failover` and `--isolate-workspace` cannot be combined with continuation. Capacity recovery and provider model fallback are disabled for continuations so they cannot silently switch model or account. Provider cache retention remains provider-controlled, so resuming does not guarantee a cache hit.
|
|
61
61
|
|
|
62
|
+
## Run costs and budgets
|
|
63
|
+
|
|
64
|
+
`cdx history --json` includes nullable `cost_usd` for each entry, estimated from its observed usage and serving model using the same table and `CDX_TOKEN_PRICES` overrides as `cdx stats`. Unknown prices or usage stay `null`.
|
|
65
|
+
|
|
66
|
+
```bash
|
|
67
|
+
cdx run work --cwd /path/to/repo --prompt "Fix the tests" --budget 2.50 --json
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
`--budget` stops a run once its measured estimated cost reaches the USD amount, including with `--detach` and `--continue-run`. A budget stop exits with code 125 and reports `error.code: "budget_exceeded"` and `error.spent_usd`. Detached callers read the final result through `cdx run-status` or `cdx run-report`. Usage arrives in provider updates, so the estimate can overshoot; unknown usage or prices cannot trigger a budget stop. See the [budget contract](docs/cli.md#run-budgets) for details.
|
|
71
|
+
|
|
62
72
|
## Codex model capacity recovery
|
|
63
73
|
|
|
64
74
|
`cdx run` (including `--detach`) automatically recovers from Codex's "Selected model is at capacity" failure. It retries the same prompt, profile, and model after 1 and 2 seconds, then tries the profile's `fallback_model` list in configured order, removing duplicates. Configure alternatives with `cdx set work --fallback-model MODEL_A,MODEL_B`.
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
0.
|
|
1
|
+
0.25.0
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# CDX Manager 0.25.0
|
|
2
|
+
|
|
3
|
+
Release date: 2026-10-31
|
|
4
|
+
|
|
5
|
+
This release adds per-run cost estimates and enforces estimated USD budgets. (#71)
|
|
6
|
+
|
|
7
|
+
## Run costs and budgets
|
|
8
|
+
|
|
9
|
+
- Include nullable `cost_usd` estimates in history and run results, using observed usage, the serving model, cache token rates, and the same pricing table and `CDX_TOKEN_PRICES` overrides as stats. Unknown usage or prices remain `null`. (#71)
|
|
10
|
+
- Add `cdx run --budget USD` to stop a run when its measured estimated spend reaches the limit. Retries, model recovery, and account failover share the run budget; a budget stop terminates the provider process tree and prevents further recovery or failover. (#71)
|
|
11
|
+
- Support budgets for detached and continued runs. A continuation starts a fresh budget that excludes earlier turns; detached results remain available through `cdx run-status` and `cdx run-report`. (#71)
|
|
12
|
+
- Report budget stops with exit code `125`, `error.code: "budget_exceeded"`, and `error.spent_usd`; include `cost_usd` and `budget_usd` in final results. Estimates depend on reported usage and known prices, so delayed updates can cause overshoot and unknown costs cannot trigger a stop. (#71)
|
|
13
|
+
|
|
14
|
+
## Maintenance
|
|
15
|
+
|
|
16
|
+
- Record tray companion checksums for v0.24.9. (`eb4de92877c91f339426b344f9f242bdede02eec`)
|
|
17
|
+
- Record release archive checksums for v0.24.9. (`afecbfdcf4b999663cd6937955349824728f621e`)
|
|
@@ -0,0 +1,7 @@
|
|
|
1
|
+
# Unreleased
|
|
2
|
+
|
|
3
|
+
## Run costs and budgets
|
|
4
|
+
|
|
5
|
+
- Add nullable `cost_usd` to JSON history entries using the same observed usage, model pricing, and price overrides as stats.
|
|
6
|
+
- Add `cdx run --budget USD` to stop on measured estimated spend with exit code 125, `budget_exceeded`, and `error.spent_usd` in the final JSON. Share the budget across recovery attempts and failover; support detached runs and fresh budgets for continued runs without charging earlier turns.
|
|
7
|
+
- Document unknown usage/prices and delayed provider reporting, including possible budget overshoot. Cover pricing, unknown costs, validation, live process termination, Claude transcript accumulation, continuation boundaries, and detached supervision in tests.
|
|
@@ -672,6 +672,17 @@
|
|
|
672
672
|
"x86_64-pc-windows-msvc": "6df3f25ab6a9aa02a45aa6f90f95c3484553b248175394af9ecfd12668479281",
|
|
673
673
|
"x86_64-unknown-linux-musl": "ea867d352a18db59e882d6abbdbafa61b54fbec7fafa66746852696cf18cd6c5"
|
|
674
674
|
}
|
|
675
|
+
},
|
|
676
|
+
"v0.24.9": {
|
|
677
|
+
"github_tarball_sha256": "1bf646b5f6dc5b3fa9ba99a7736d62fc0170469cbb6bf6617124dfb134cbbea7",
|
|
678
|
+
"github_zip_sha256": "b921117dfd70beb48e8688d94706f755714a4eb826a94dc4b73ef250e646324e",
|
|
679
|
+
"tray_assets": {
|
|
680
|
+
"aarch64-apple-darwin": "378b8b9d85159263f9ce4a130ad0c67d5f142b43609b3719be0664c9b3125fed",
|
|
681
|
+
"aarch64-unknown-linux-musl": "49f9f5e1d0a352935a9e7c738150d36f679b22c66b389c937124e01bf6d954ac",
|
|
682
|
+
"x86_64-apple-darwin": "40a12abb38b8b4ef61c110f6426847b190cb9dd7f146d8f33f3abf0e5b3d8488",
|
|
683
|
+
"x86_64-pc-windows-msvc": "21dfebd0265bd8705d381517357cd8166cb0e2da37b27ed3e3485c892d9289ee",
|
|
684
|
+
"x86_64-unknown-linux-musl": "8477262e55f31ee2217382048b967846478a34f222beeb4242fa9c4c2942f747"
|
|
685
|
+
}
|
|
675
686
|
}
|
|
676
687
|
}
|
|
677
688
|
}
|
package/docs/cli.md
CHANGED
|
@@ -56,7 +56,7 @@
|
|
|
56
56
|
| `cdx agents [--max-age-hours N] [--json]` | Each session's last known agent state (working, waiting, done, failed, limit, ended), waiting first; records older than N hours (default 24) are dropped. See [Agent state](sessions-and-notifications.md#agent-state) |
|
|
57
57
|
| `cdx next [--json] [--refresh]` | Select the best next assistant using the same priority logic as `cdx status` |
|
|
58
58
|
| `cdx select --provider PROVIDER [--min-reasoning-effort minimal\|low\|medium\|high\|xhigh] [--min-power minimal\|low\|medium\|high\|xhigh] [--require-ready] [--refresh] --json` | Select a suitable session for headless automation |
|
|
59
|
-
| `cdx run [session] --cwd PATH (--prompt-file PATH\|--prompt TEXT\|--prompt-file -) [--provider PROVIDER] [--model MODEL] [--kind assistant\|code-review\|retro] [--reasoning-effort minimal\|low\|medium\|high\|xhigh] [--power minimal\|low\|medium\|high\|xhigh] [--permission MODE] [--timeout-seconds N] [--label TEXT] [--max-parallel N] [--continue-run RUN_ID] [--detach] [--failover] [--refresh] --json` | Run one headless task and return a stable JSON result; `--detach` returns the `run_id` at launch without waiting, `--failover` continues the task on the next account when this one hits its rate limit (with `--detach` too: the detached cdx child applies it and the run's `occupancies` record each move), `--prompt-file -` reads the prompt from stdin, `--label` names the run in `cdx runs`, `--max-parallel N` (default `CDX_MAX_RUNS`; unset, invalid, 0 or negative means no limit) refuses to start with `too_many_runs` when N runs are already running, `--kind` (default `assistant`) labels the run record and `code-review` adds a review report to its final payload |
|
|
59
|
+
| `cdx run [session] --cwd PATH (--prompt-file PATH\|--prompt TEXT\|--prompt-file -) [--provider PROVIDER] [--model MODEL] [--kind assistant\|code-review\|retro] [--reasoning-effort minimal\|low\|medium\|high\|xhigh] [--power minimal\|low\|medium\|high\|xhigh] [--permission MODE] [--timeout-seconds N] [--budget USD] [--label TEXT] [--max-parallel N] [--continue-run RUN_ID] [--detach] [--failover] [--refresh] --json` | Run one headless task and return a stable JSON result; `--detach` returns the `run_id` at launch without waiting, `--failover` continues the task on the next account when this one hits its rate limit (with `--detach` too: the detached cdx child applies it and the run's `occupancies` record each move), `--prompt-file -` reads the prompt from stdin, `--label` names the run in `cdx runs`, `--max-parallel N` (default `CDX_MAX_RUNS`; unset, invalid, 0 or negative means no limit) refuses to start with `too_many_runs` when N runs are already running, `--kind` (default `assistant`) labels the run record and `code-review` adds a review report to its final payload |
|
|
60
60
|
| `cdx run-report <run_id> --json` | Full report, transcript metadata, and final payload for a run; `final_message` is the run's last assistant message, read from Claude's `result` envelope or Codex's last `agent_message` (null when neither exists) |
|
|
61
61
|
| `cdx run-status <run_id> --json` | Status of one run by id |
|
|
62
62
|
| `cdx run-tail <run_id> [--lines N] --json` | Last lines of a run's own output, while it is still running or after it finished |
|
|
@@ -245,6 +245,16 @@ Known headless and completed interactive token usage is persisted in launch hist
|
|
|
245
245
|
|
|
246
246
|
Continuation errors are `continue_run_not_found` (unknown parent), `continue_run_mismatch` (another or missing profile/provider), `continue_run_unavailable` (unfinished or unsupported run, expired transcript, or unavailable settings), and `continue_run_settings_mismatch` (changed model/effort or incompatible flags). Callers may start a normal run after a refusal by omitting `--continue-run`.
|
|
247
247
|
|
|
248
|
+
#### Run budgets
|
|
249
|
+
|
|
250
|
+
`cdx run --budget USD` accepts a finite positive amount in US dollars. It uses the same token pricing function, cache rates, and `CDX_TOKEN_PRICES` overrides as `cdx stats`. The supervisor checks reported stdout usage and the exact provider conversation transcript every 250 ms while waiting, and checks again at completion. Claude's transcript messages are summed; continuation measurements subtract the transcript snapshot taken before this run. Earlier turns do not consume the new budget. Measured costs from retries, model recovery, and account failover share the same run budget.
|
|
251
|
+
|
|
252
|
+
At or above the limit, cdx terminates the provider process tree, skips further recovery/failover, records the run as `failed`, and exits with **125**. The final JSON has `ok: false`, `exit_code: 125`, `error.source: "cdx"`, `error.code: "budget_exceeded"`, and `error.spent_usd` with the measured estimated spend across attempts. `cost_usd` repeats that amount and `budget_usd` records the requested limit. Normal results also include nullable `cost_usd` for their observed usage and `budget_usd` (`null` without the flag).
|
|
253
|
+
|
|
254
|
+
With `--detach`, the launch still exits successfully and includes `budget_usd`; cdx always uses its own child supervisor for budgeted runs, including when native background delegation is enabled. Inspect `cdx run-status RUN_ID --json` or `cdx run-report RUN_ID --json` for the child's final payload and budget error. `--continue-run` starts a fresh budget for the new run, including in detached mode.
|
|
255
|
+
|
|
256
|
+
Conservative assumption: this is a stop based on observed, priceable usage, not a prepaid billing limit. Provider updates can arrive late or only at completion, and termination can take time, so actual spend can exceed the amount. Missing usage, missing continuation boundaries, mixed/unobserved models, and unknown prices are not treated as free tokens and cannot trigger the budget; their estimated cost stays `null`. Supply a `CDX_TOKEN_PRICES` override for a model missing from the table. Existing profile-level `cdx set --budget` settings remain the provider-native Claude cap and apply independently.
|
|
257
|
+
|
|
248
258
|
Every usage record, whichever path produced it, uses one set of definitions:
|
|
249
259
|
|
|
250
260
|
| Field | Meaning |
|
|
@@ -267,6 +277,8 @@ History and stats text show `cache 72 %` beside tokens, or `cache n/d` for unkno
|
|
|
267
277
|
|
|
268
278
|
Ranking on the raw total would put a session that mostly replayed a cache above one that spent far more on generation, because cache reads are typically the overwhelming majority of tokens consumed while costing a fiftieth of an output token.
|
|
269
279
|
|
|
280
|
+
`cdx history --json` includes `cost_usd` on every entry, calculated when read using the current stats price table and overrides. Missing or untrusted usage and unknown serving models/prices yield `null`, never a fabricated zero. Cache reads, writes (including one-hour TTL), and output use the same rates as stats.
|
|
281
|
+
|
|
270
282
|
The **`USD~`** column turns that into money for runs whose serving model cdx recorded. It prices each token class directly using the built-in model table. To customize prices: set `CDX_TOKEN_PRICES` to a JSON object mapping model id to dollars per million tokens to override the built-in table, whose last review date is printed with the totals. A run whose model cdx never saw stays unpriced rather than being charged at a default tier, and the totals line says how many runs were actually priced.
|
|
271
283
|
|
|
272
284
|
The table requires `input` and `output` prices in USD per million tokens and accepts an optional `cache_read` price in the same units. Without it, cache reads cost 0.1x input; cache writes assume 1.25x input. GPT-6.1 Sol and Claude Opus 5.5 use 0.05x cache reads, while Claude Fable/Mythos 5.1 use 0.025x. Both `COST~` and `USD~` honor those rates. `CDX_TOKEN_PRICES` replaces the supplied model row: a two-field override keeps the legacy 0.1x fallback; include `cache_read` to override it (including zero).
|
package/package.json
CHANGED
package/pyproject.toml
CHANGED
package/src/cli_args.py
CHANGED
|
@@ -5,6 +5,7 @@ Split out of cli_commands.py: pure input-parsing layer (no command execution),
|
|
|
5
5
|
imported back there and used by the handlers.
|
|
6
6
|
"""
|
|
7
7
|
|
|
8
|
+
import math
|
|
8
9
|
from datetime import datetime, timedelta
|
|
9
10
|
|
|
10
11
|
from .config import (
|
|
@@ -47,7 +48,7 @@ CAN_RESUME_USAGE = "Usage: cdx can-resume <name> [--conversation <id>] [--json]"
|
|
|
47
48
|
SESSIONS_USAGE = "Usage: cdx sessions [--all] --json"
|
|
48
49
|
SELECT_USAGE = "Usage: cdx select --provider PROVIDER [--min-reasoning-effort minimal|low|medium|high|xhigh] [--min-power minimal|low|medium|high|xhigh] [--require-ready] [--refresh] --json"
|
|
49
50
|
NEXT_USAGE = "Usage: cdx next [--json] [--refresh]"
|
|
50
|
-
RUN_USAGE = "Usage: cdx run [session] (--cwd|--dir) PATH (--prompt-file PATH|--prompt TEXT|--prompt-file -) [--provider PROVIDER] [--model MODEL] [--kind assistant|code-review|retro] [--reasoning-effort minimal|low|medium|high|xhigh] [--power minimal|low|medium|high|xhigh] [--permission review|default|auto|full|workspace-write|read-only|danger-full-access] [--network-access] [--isolate-workspace] [--timeout-seconds N] [--label TEXT] [--max-parallel N] [--continue-run RUN_ID] [--detach] [--failover] [--refresh] --json"
|
|
51
|
+
RUN_USAGE = "Usage: cdx run [session] (--cwd|--dir) PATH (--prompt-file PATH|--prompt TEXT|--prompt-file -) [--provider PROVIDER] [--model MODEL] [--kind assistant|code-review|retro] [--reasoning-effort minimal|low|medium|high|xhigh] [--power minimal|low|medium|high|xhigh] [--permission review|default|auto|full|workspace-write|read-only|danger-full-access] [--network-access] [--isolate-workspace] [--timeout-seconds N] [--budget USD] [--label TEXT] [--max-parallel N] [--continue-run RUN_ID] [--detach] [--failover] [--refresh] --json"
|
|
51
52
|
RUN_JSON_REQUIRED = "cdx run: --json is required."
|
|
52
53
|
RUN_TARGET_REQUIRED = "cdx run: specify a session name or --provider PROVIDER."
|
|
53
54
|
RUN_SESSION_PROVIDER_CONFLICT = "cdx run: cannot specify both a session name and --provider."
|
|
@@ -425,6 +426,17 @@ def _parse_timeout_seconds(value, usage=RUN_USAGE):
|
|
|
425
426
|
return parsed
|
|
426
427
|
|
|
427
428
|
|
|
429
|
+
def _parse_run_budget(value):
|
|
430
|
+
try:
|
|
431
|
+
amount = float(value)
|
|
432
|
+
if not math.isfinite(amount) or amount <= 0:
|
|
433
|
+
raise ValueError
|
|
434
|
+
return amount
|
|
435
|
+
except (TypeError, ValueError) as error:
|
|
436
|
+
raise CdxArgumentError("cdx run: --budget must be a finite positive USD amount.",
|
|
437
|
+
code=ARG_CODE_OUT_OF_RANGE, arguments=["--budget"]) from error
|
|
438
|
+
|
|
439
|
+
|
|
428
440
|
def _parse_run_timeout_seconds(value):
|
|
429
441
|
try:
|
|
430
442
|
return _parse_timeout_seconds(value, RUN_USAGE)
|
|
@@ -539,6 +551,7 @@ def cdx_schema():
|
|
|
539
551
|
"provider_start_failed",
|
|
540
552
|
"provider_failed",
|
|
541
553
|
"provider_timeout",
|
|
554
|
+
"budget_exceeded",
|
|
542
555
|
"run_cancelled",
|
|
543
556
|
"too_many_runs",
|
|
544
557
|
"failover_exhausted",
|
|
@@ -671,6 +684,7 @@ def _parse_run_args(args):
|
|
|
671
684
|
"--power": {"key": "power", "type": "str", "default": None},
|
|
672
685
|
"--permission": {"key": "permission", "type": "str", "default": None, "transform": _normalize_run_permission},
|
|
673
686
|
"--timeout-seconds": {"key": "timeout_seconds", "type": "str", "default": None, "transform": _parse_run_timeout_seconds},
|
|
687
|
+
"--budget": {"key": "budget", "type": "str", "default": None, "transform": _parse_run_budget},
|
|
674
688
|
"--detach": {"key": "detach", "type": "bool", "default": False},
|
|
675
689
|
"--network-access": {"key": "network_access", "type": "bool", "default": False},
|
|
676
690
|
"--isolate-workspace": {"key": "isolate_workspace", "type": "bool", "default": False},
|
|
@@ -727,6 +741,7 @@ def _parse_run_args(args):
|
|
|
727
741
|
"kind": parsed["kind"],
|
|
728
742
|
"model": parsed["model"],
|
|
729
743
|
"permission": parsed["permission"],
|
|
744
|
+
"budget": parsed["budget"],
|
|
730
745
|
"timeout_seconds": parsed["timeout_seconds"],
|
|
731
746
|
"detach": parsed["detach"],
|
|
732
747
|
"network_access": parsed["network_access"],
|
package/src/commands/runs.py
CHANGED
|
@@ -25,7 +25,7 @@ from ..cli_args import (
|
|
|
25
25
|
from ..cli_helpers import API_SCHEMA_VERSION, _json_failure, _json_success, _write_json
|
|
26
26
|
from ..config import split_extra_args
|
|
27
27
|
from ..errors import CdxError
|
|
28
|
-
from ..interactive_usage import MEASURED_KEYS, extract_interactive_usage, usage_delta
|
|
28
|
+
from ..interactive_usage import MATCH_CONVERSATION_ID, MEASURED_KEYS, extract_interactive_usage, usage_delta
|
|
29
29
|
from ..provider_background import (
|
|
30
30
|
BACKGROUND_PATH_CDX,
|
|
31
31
|
BACKGROUND_PATH_PROVIDER,
|
|
@@ -58,7 +58,7 @@ from ..run_failover import (
|
|
|
58
58
|
looks_model_at_capacity,
|
|
59
59
|
)
|
|
60
60
|
from ..run_registry import RunRegistry, build_code_review_report, final_message
|
|
61
|
-
from ..run_usage import empty_usage, extract_run_conversation_id, extract_run_observation
|
|
61
|
+
from ..run_usage import empty_usage, estimate_cost, extract_run_conversation_id, extract_run_observation, token_prices
|
|
62
62
|
from ..session_ranking import FACTOR_DESCRIPTIONS, rank_sessions, selection_policy
|
|
63
63
|
from ..status_view import _now_timestamp, _priority_reset_timestamp
|
|
64
64
|
|
|
@@ -204,6 +204,7 @@ def _detached_child_argv(parsed, prompt_path, session_name, cwd):
|
|
|
204
204
|
# from the package root, so a relative "--cwd ." would resolve there.
|
|
205
205
|
argv = ["run", session_name, "--cwd", cwd, "--prompt-file", prompt_path]
|
|
206
206
|
for flag, key in (
|
|
207
|
+
("--budget", "budget"),
|
|
207
208
|
("--continue-run", "continue_run"),
|
|
208
209
|
("--model", "model"),
|
|
209
210
|
("--kind", "kind"),
|
|
@@ -563,6 +564,28 @@ def _observe_headless_usage(session, run_info, baseline=None):
|
|
|
563
564
|
return observation
|
|
564
565
|
|
|
565
566
|
|
|
567
|
+
def _budget_observation(session, paths, baseline):
|
|
568
|
+
observation = _observe_headless_usage(session, paths, baseline if session["provider"] == "codex" else None)
|
|
569
|
+
identifier = (extract_run_conversation_id(session["provider"], paths.get("stdout_path"))
|
|
570
|
+
or ((session.get("conversation") or {}).get("id")
|
|
571
|
+
if session["provider"] == "claude" or session.get("continue_run") else None))
|
|
572
|
+
if not identifier:
|
|
573
|
+
return observation
|
|
574
|
+
usage, path, match, model = extract_interactive_usage(
|
|
575
|
+
session["provider"], session.get("authHome"), conversation_id=identifier)
|
|
576
|
+
if usage is None or match != MATCH_CONVERSATION_ID:
|
|
577
|
+
return observation
|
|
578
|
+
if baseline is not None:
|
|
579
|
+
before, before_path, _match, _model = baseline
|
|
580
|
+
comparable = (before and path == before_path
|
|
581
|
+
and all((usage.get(key) is None) == (before.get(key) is None) for key in MEASURED_KEYS))
|
|
582
|
+
usage = usage_delta(usage, before, provider=session["provider"]) if comparable else None
|
|
583
|
+
if usage is not None:
|
|
584
|
+
return {**observation, "usage": usage, "usage_model": model, "provider_transcript_path": path,
|
|
585
|
+
"provider_transcript_match": match}
|
|
586
|
+
return observation
|
|
587
|
+
|
|
588
|
+
|
|
566
589
|
def _continue_run_session(parsed, ctx):
|
|
567
590
|
"""Resolve only the parent's profile and exact persisted conversation."""
|
|
568
591
|
previous = _run_registry(ctx).get(parsed["continue_run"])
|
|
@@ -624,6 +647,7 @@ def handle_run(rest, ctx):
|
|
|
624
647
|
detached_run_id = os.environ.pop(DETACHED_RUN_ID_ENV, None) or None
|
|
625
648
|
try:
|
|
626
649
|
parsed = _parse_run_args(rest)
|
|
650
|
+
budget_prices = token_prices()[0] if parsed.get("budget") is not None else None
|
|
627
651
|
session = _continue_run_session(parsed, ctx) if parsed.get("continue_run") else None
|
|
628
652
|
if session:
|
|
629
653
|
if session.get("enabled", True) is False:
|
|
@@ -738,7 +762,7 @@ def handle_run(rest, ctx):
|
|
|
738
762
|
launch = None
|
|
739
763
|
# A provider's own background agent has no cdx process watching it,
|
|
740
764
|
# so nothing could fail it over; --failover keeps the in-house child.
|
|
741
|
-
if not parsed.get("continue_run") and not parsed.get("isolate_workspace") and not parsed.get("network_access") and not parsed.get("failover") and supports_native_background(
|
|
765
|
+
if not parsed.get("budget") and not parsed.get("continue_run") and not parsed.get("isolate_workspace") and not parsed.get("network_access") and not parsed.get("failover") and supports_native_background(
|
|
742
766
|
run_session.get("provider"),
|
|
743
767
|
env=ctx.get("env"),
|
|
744
768
|
spawn_sync=ctx.get("spawn_text"),
|
|
@@ -773,6 +797,7 @@ def handle_run(rest, ctx):
|
|
|
773
797
|
failover_error_code = None
|
|
774
798
|
recovery_events = []
|
|
775
799
|
attempt = 0
|
|
800
|
+
spent_usd = 0.0
|
|
776
801
|
models = capacity_models(run_session)
|
|
777
802
|
model_index = 0
|
|
778
803
|
capacity_retries = 0
|
|
@@ -781,7 +806,20 @@ def handle_run(rest, ctx):
|
|
|
781
806
|
baseline = extract_interactive_usage(
|
|
782
807
|
run_session["provider"], run_session.get("authHome"),
|
|
783
808
|
conversation_id=(run_session.get("conversation") or {}).get("id"),
|
|
784
|
-
) if parsed.get("continue_run") and run_session["provider"] == "codex" else None
|
|
809
|
+
) if parsed.get("continue_run") and (run_session["provider"] == "codex" or parsed.get("budget")) else None
|
|
810
|
+
attempt_cost = None
|
|
811
|
+
budget_observation = None
|
|
812
|
+
|
|
813
|
+
def check_budget(paths, session=run_session, before=baseline, already_spent=spent_usd):
|
|
814
|
+
nonlocal attempt_cost, budget_observation
|
|
815
|
+
observation = _budget_observation(session, paths, before)
|
|
816
|
+
cost = estimate_cost(observation["usage"], observation["usage_model"], budget_prices)
|
|
817
|
+
if cost is not None:
|
|
818
|
+
if attempt_cost is None or cost >= attempt_cost:
|
|
819
|
+
attempt_cost = cost
|
|
820
|
+
budget_observation = observation
|
|
821
|
+
return attempt_cost is not None and already_spent + attempt_cost >= parsed["budget"]
|
|
822
|
+
|
|
785
823
|
run_info = _run_headless_provider_command(
|
|
786
824
|
run_session,
|
|
787
825
|
cwd=cwd,
|
|
@@ -790,8 +828,11 @@ def handle_run(rest, ctx):
|
|
|
790
828
|
timeout_seconds=parsed.get("timeout_seconds"),
|
|
791
829
|
spawn=ctx.get("spawn_headless") or ctx.get("spawn"),
|
|
792
830
|
run_id=run_id,
|
|
831
|
+
**({"budget_check": check_budget} if parsed.get("budget") is not None else {}),
|
|
793
832
|
)
|
|
794
|
-
observation = _observe_headless_usage(run_session, run_info, baseline)
|
|
833
|
+
observation = _observe_headless_usage(run_session, run_info, baseline if run_session["provider"] == "codex" else None)
|
|
834
|
+
if run_info.get("budget_exceeded") and budget_observation is not None:
|
|
835
|
+
observation = budget_observation
|
|
795
836
|
run_info = {**run_info, **observation,
|
|
796
837
|
"parent_run_id": parsed.get("continue_run"),
|
|
797
838
|
"conversation_id": extract_run_conversation_id(run_session.get("provider"), run_info.get("stdout_path"))
|
|
@@ -800,6 +841,10 @@ def handle_run(rest, ctx):
|
|
|
800
841
|
"reasoning_effort": _launch_power(run_session),
|
|
801
842
|
"effective_model": (run_session.get("launch") or {}).get("model"),
|
|
802
843
|
"recovery_events": recovery_events}
|
|
844
|
+
cost = estimate_cost(observation["usage"], observation["usage_model"], budget_prices)
|
|
845
|
+
spent_usd += max(attempt_cost or 0.0, cost or 0.0)
|
|
846
|
+
if parsed.get("budget") is not None and (run_info.get("budget_exceeded") or spent_usd >= parsed["budget"]):
|
|
847
|
+
run_info.update(budget_exceeded=True, returncode=125, cost_usd=spent_usd)
|
|
803
848
|
ctx["service"]["record_launch_history"](run_session["name"], {
|
|
804
849
|
"status": "success" if run_info.get("returncode") == 0 else "failed",
|
|
805
850
|
"action": "run",
|
|
@@ -809,7 +854,7 @@ def handle_run(rest, ctx):
|
|
|
809
854
|
"exit_code": run_info.get("returncode"),
|
|
810
855
|
**run_info,
|
|
811
856
|
})
|
|
812
|
-
if parsed.get("continue_run"):
|
|
857
|
+
if run_info.get("budget_exceeded") or parsed.get("continue_run"):
|
|
813
858
|
break
|
|
814
859
|
if looks_model_at_capacity(run_session.get("provider"), run_info):
|
|
815
860
|
event = {"reason": "model_at_capacity", "attempt": attempt,
|
|
@@ -934,7 +979,10 @@ def handle_run(rest, ctx):
|
|
|
934
979
|
)
|
|
935
980
|
_write_json(ctx, final_payload)
|
|
936
981
|
return 0
|
|
937
|
-
if
|
|
982
|
+
if run_info.get("budget_exceeded"):
|
|
983
|
+
failover_error_code = "budget_exceeded"
|
|
984
|
+
message = "Run exceeded its estimated USD budget."
|
|
985
|
+
elif failover_error_code == "capacity_recovery_exhausted":
|
|
938
986
|
message = "Codex model capacity recovery exhausted its retry, model, or profile budget."
|
|
939
987
|
elif failover_error_code == "failover_exhausted":
|
|
940
988
|
message = f"Every eligible session was rate limited: {', '.join(attempted)}."
|
|
@@ -952,7 +1000,7 @@ def handle_run(rest, ctx):
|
|
|
952
1000
|
run_session,
|
|
953
1001
|
run_info=run_info,
|
|
954
1002
|
error=error,
|
|
955
|
-
error_source="provider",
|
|
1003
|
+
error_source="cdx" if run_info.get("budget_exceeded") else "provider",
|
|
956
1004
|
# Running out of accounts is a different outcome from the task
|
|
957
1005
|
# failing, and a caller has to be able to branch on which happened.
|
|
958
1006
|
error_code=failover_error_code or (
|
|
@@ -962,7 +1010,7 @@ def handle_run(rest, ctx):
|
|
|
962
1010
|
)
|
|
963
1011
|
registry.finish(
|
|
964
1012
|
run_id,
|
|
965
|
-
status="timed_out" if run_info.get("timed_out") else "failed",
|
|
1013
|
+
status="timed_out" if run_info.get("timed_out") and not run_info.get("budget_exceeded") else "failed",
|
|
966
1014
|
final_payload=final_payload,
|
|
967
1015
|
run_info=run_info,
|
|
968
1016
|
error=final_payload.get("error"),
|
package/src/commands/status.py
CHANGED
|
@@ -672,6 +672,7 @@ def handle_history(rest, ctx):
|
|
|
672
672
|
entries = ctx["service"]["get_launch_history"](parsed["name"], limit=limit)
|
|
673
673
|
entries = _filter_history_period(entries, parsed["period"])
|
|
674
674
|
entries = [{**entry, "parent_run_id": entry.get("parent_run_id"),
|
|
675
|
+
"cost_usd": estimate_cost(entry.get("usage"), entry.get("usage_model")) if is_vouched(entry.get("usage")) else None,
|
|
675
676
|
"usage": {**entry["usage"], "cache_hit_ratio": cache_hit_ratio(entry["usage"], provider=entry.get("provider")) if is_vouched(entry["usage"]) else None}
|
|
676
677
|
if isinstance(entry.get("usage"), dict) else entry.get("usage")} for entry in entries]
|
|
677
678
|
if has_period and not parsed["summary"]:
|
package/src/provider_runtime.py
CHANGED
|
@@ -8,6 +8,7 @@ import signal
|
|
|
8
8
|
import subprocess
|
|
9
9
|
import sys
|
|
10
10
|
import threading
|
|
11
|
+
import time
|
|
11
12
|
import uuid
|
|
12
13
|
from datetime import datetime, timezone
|
|
13
14
|
|
|
@@ -1204,7 +1205,7 @@ def _read_permission_denials(stdout_path):
|
|
|
1204
1205
|
|
|
1205
1206
|
|
|
1206
1207
|
def _run_headless_provider_command(session, cwd=None, env_override=None, initial_prompt=None,
|
|
1207
|
-
timeout_seconds=None, spawn=None, run_id=None):
|
|
1208
|
+
timeout_seconds=None, spawn=None, run_id=None, budget_check=None):
|
|
1208
1209
|
spawn = spawn or subprocess.Popen
|
|
1209
1210
|
spec = _build_headless_launch_spec(
|
|
1210
1211
|
session,
|
|
@@ -1262,6 +1263,7 @@ def _run_headless_provider_command(session, cwd=None, env_override=None, initial
|
|
|
1262
1263
|
child = None
|
|
1263
1264
|
timed_out = False
|
|
1264
1265
|
cancelled = False
|
|
1266
|
+
budget_exceeded = False
|
|
1265
1267
|
with open(paths["stdout_path"], "w", encoding="utf-8", errors="replace") as stdout_file, \
|
|
1266
1268
|
open(paths["stderr_path"], "w", encoding="utf-8", errors="replace") as stderr_file:
|
|
1267
1269
|
options = {k: v for k, v in spec.get("options", {}).items() if k not in ("stdio", "stdout", "stderr")}
|
|
@@ -1290,7 +1292,23 @@ def _run_headless_provider_command(session, cwd=None, env_override=None, initial
|
|
|
1290
1292
|
# this supervisor and keep its run "running" forever.
|
|
1291
1293
|
previous_handlers = _route_termination_to_interrupt()
|
|
1292
1294
|
try:
|
|
1293
|
-
if
|
|
1295
|
+
if budget_check is not None:
|
|
1296
|
+
deadline = time.monotonic() + timeout_seconds if timeout_seconds is not None else None
|
|
1297
|
+
while True:
|
|
1298
|
+
if budget_check(paths):
|
|
1299
|
+
budget_exceeded = True
|
|
1300
|
+
_terminate_child_tree(child)
|
|
1301
|
+
break
|
|
1302
|
+
remaining = deadline - time.monotonic() if deadline is not None else None
|
|
1303
|
+
if remaining is not None and remaining <= 0:
|
|
1304
|
+
raise subprocess.TimeoutExpired(command, timeout_seconds)
|
|
1305
|
+
try:
|
|
1306
|
+
child.wait(timeout=min(0.25, remaining) if remaining is not None else 0.25)
|
|
1307
|
+
budget_exceeded = budget_check(paths)
|
|
1308
|
+
break
|
|
1309
|
+
except subprocess.TimeoutExpired:
|
|
1310
|
+
continue
|
|
1311
|
+
elif timeout_seconds is None:
|
|
1294
1312
|
child.wait()
|
|
1295
1313
|
else:
|
|
1296
1314
|
child.wait(timeout=timeout_seconds)
|
|
@@ -1337,6 +1355,7 @@ def _run_headless_provider_command(session, cwd=None, env_override=None, initial
|
|
|
1337
1355
|
"pid": getattr(child, "pid", None),
|
|
1338
1356
|
"returncode": returncode,
|
|
1339
1357
|
"timed_out": timed_out,
|
|
1358
|
+
"budget_exceeded": budget_exceeded,
|
|
1340
1359
|
"permission_denials": (
|
|
1341
1360
|
_read_permission_denials(paths["stdout_path"])
|
|
1342
1361
|
if session.get("provider") == PROVIDER_CLAUDE else []
|
package/src/run_command.py
CHANGED
|
@@ -4,7 +4,7 @@ import sys
|
|
|
4
4
|
from .cli_args import ARG_CODE_INVALID_VALUE
|
|
5
5
|
from .errors import CdxArgumentError, CdxError
|
|
6
6
|
from .provider_runtime import headless_permission_disables_network
|
|
7
|
-
from .run_usage import empty_usage, extract_run_observation
|
|
7
|
+
from .run_usage import empty_usage, estimate_cost, extract_run_observation
|
|
8
8
|
|
|
9
9
|
PROMPT_STDIN = "-"
|
|
10
10
|
|
|
@@ -172,6 +172,7 @@ def run_launch_payload(api_schema_version, parsed, session, artifacts, cwd=None,
|
|
|
172
172
|
"parent_run_id": parsed.get("continue_run"),
|
|
173
173
|
"conversation_id": conversation_id or run_conversation_id(session),
|
|
174
174
|
"detached": True,
|
|
175
|
+
"budget_usd": parsed.get("budget"),
|
|
175
176
|
"session": session.get("name") if session else None,
|
|
176
177
|
"provider": session.get("provider") if session else parsed.get("provider"),
|
|
177
178
|
"model": parsed.get("model") or ((session.get("launch") or {}).get("model") if session else None),
|
|
@@ -235,11 +236,14 @@ def run_result_payload(api_schema_version, ok, parsed, session, run_info=None,
|
|
|
235
236
|
"stderr_path": run_info.get("stderr_path"),
|
|
236
237
|
"usage": usage,
|
|
237
238
|
"usage_model": usage_model,
|
|
239
|
+
"cost_usd": run_info.get("cost_usd", estimate_cost(usage, usage_model)),
|
|
240
|
+
"budget_usd": parsed.get("budget"),
|
|
238
241
|
"warnings": warnings,
|
|
239
242
|
"error": None if ok else {
|
|
240
243
|
"source": error_source or "cdx",
|
|
241
244
|
"code": error_code or "cdx_error",
|
|
242
245
|
"message": message,
|
|
246
|
+
**({"spent_usd": run_info.get("cost_usd")} if error_code == "budget_exceeded" else {}),
|
|
243
247
|
"arguments": run_error_arguments(error),
|
|
244
248
|
"allowed_values": run_error_allowed_values(error),
|
|
245
249
|
"provider_code": run_info.get("returncode") if error_source == "provider" else None,
|