codecartographer-pi 0.21.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.codecarto/broadside/SKILL.md +8 -1
- package/.codecarto/broadside/config.yaml +9 -1
- package/.codecarto/workflow/scaffold-version.yaml +1 -1
- package/README.md +2 -2
- package/agent-skill/codecartographer/references/broadside.md +10 -0
- package/dist/core/broadside.d.ts +49 -0
- package/dist/core/broadside.js +199 -33
- package/dist/extensions/codecarto/broadside-flags.d.ts +5 -1
- package/dist/extensions/codecarto/broadside-flags.js +39 -0
- package/dist/extensions/codecarto/index.js +8 -4
- package/dist/mcp-server/server.d.ts +2 -0
- package/dist/mcp-server/server.js +41 -3
- package/package.json +1 -1
|
@@ -108,7 +108,14 @@ architecture map. Each override is priced, capability-checked, and clamped like
|
|
|
108
108
|
the default, the submit estimate breaks cost out per lens, and `run-meta.json`
|
|
109
109
|
records which lens ran on what. No stronger default is shipped: which model is
|
|
110
110
|
worth the money depends on the repository and the budget, so compare with the
|
|
111
|
-
`models` action and decide
|
|
111
|
+
`models` action and decide — for one run with the `model` and `lens_models`
|
|
112
|
+
parameters (Pi: `--model=ID`, `--lens-model=LENS:ID`), or here for the
|
|
113
|
+
repository. The `models` listing is advisory: OpenRouter's catalog returns a
|
|
114
|
+
`:batch` id for some models its Batch API refuses (`does not have a :batch
|
|
115
|
+
endpoint`), at no cost, and nothing in the catalog tells them apart. The
|
|
116
|
+
listing tags the ids this repository's own submits have seen accepted or
|
|
117
|
+
refused (`broadside/batch-endpoints.json`), and a refused lens says why in the
|
|
118
|
+
submit report.
|
|
112
119
|
|
|
113
120
|
Every run knob — `incremental`, `retry_truncated`, `include_synthesis`,
|
|
114
121
|
`include_triage`, `wait_seconds` — also has a repository default under the same
|
|
@@ -54,10 +54,18 @@
|
|
|
54
54
|
# Nothing in the catalog distinguishes them. Every Anthropic and OpenAI
|
|
55
55
|
# batch id tried so far is rejected this way; Google and DeepSeek work.
|
|
56
56
|
# A rejected batch costs nothing, so probe a candidate on one lens before
|
|
57
|
-
# relying on it
|
|
57
|
+
# relying on it: `codecarto_broadside {action: "submit", lenses:
|
|
58
|
+
# ["architecture"], model: "<id>"}` (Pi: `/codecarto-broadside architecture
|
|
59
|
+
# --model=<id>`). Submits remember the answer in batch-endpoints.json next
|
|
60
|
+
# to this file, and the `models` listing tags each id accordingly.
|
|
58
61
|
# 2. A reasoning-capable model spends its output budget thinking, and the
|
|
59
62
|
# thinking is billed at the full output rate. See `reasoning:` below.
|
|
60
63
|
#
|
|
64
|
+
# The same two keys are submit parameters for one run — `model` and
|
|
65
|
+
# `lens_models` on codecarto_broadside, `--model=ID` and `--lens-model=LENS:ID`
|
|
66
|
+
# on /codecarto-broadside — and a lens set both there and here takes the
|
|
67
|
+
# parameter's.
|
|
68
|
+
#
|
|
61
69
|
# lens_models:
|
|
62
70
|
# security: deepseek/deepseek-v4-pro-0813:batch
|
|
63
71
|
# defect: deepseek/deepseek-v4-pro-0813:batch
|
package/README.md
CHANGED
|
@@ -398,9 +398,9 @@ codecarto_broadside {cwd, action: "collect"} # poll, save, synt
|
|
|
398
398
|
|
|
399
399
|
Submit and collect are separate because batch jobs routinely take tens of minutes; collect is resumable and picks up whatever is still in flight (`wait_seconds: 0`, the default, polls once and returns). Submit prices the run from the collected file sizes against the model's live per-token pricing (cached 24h) and refuses when the estimate exceeds `max_cost` — $1.00 unless the config or the call sets another value, `0` for no limit — unless `force: true` is passed — a pre-flight estimate, not a runtime stop. Actual spend lands in each run's `run-meta.json`.
|
|
400
400
|
|
|
401
|
-
Repository defaults live in `.codecarto/broadside/config.yaml` (`model`, `api_key`, `default_lenses`, `max_cost`, `pricing` overrides, `lens_models`, `incremental`, `retry_truncated`, `include_synthesis`, `include_triage`, `wait_seconds`); an explicit tool parameter always wins. `lens_models` routes individual lenses to their own batch model — a stronger model changes security and defect findings far more than it changes an architecture map — and each override is priced, capability-checked, and clamped exactly like the default, with the estimate broken out per lens so a mixed-model run cannot be approved without seeing which lens costs what. CodeCartographer ships no stronger default: which model earns its price depends on your repository and budget, so compare candidates with the `models` action and choose. `codecarto_skill {cwd, name: "broadside"}` returns the reading guide for a completed run, and unlike post-pipeline skills it is not gated on a finished pipeline.
|
|
401
|
+
Repository defaults live in `.codecarto/broadside/config.yaml` (`model`, `api_key`, `default_lenses`, `max_cost`, `pricing` overrides, `lens_models`, `incremental`, `retry_truncated`, `include_synthesis`, `include_triage`, `wait_seconds`); an explicit tool parameter always wins. `lens_models` routes individual lenses to their own batch model — a stronger model changes security and defect findings far more than it changes an architecture map — and each override is priced, capability-checked, and clamped exactly like the default, with the estimate broken out per lens so a mixed-model run cannot be approved without seeing which lens costs what. CodeCartographer ships no stronger default: which model earns its price depends on your repository and budget, so compare candidates with the `models` action and choose — for one run with the `model` and `lens_models` parameters (Pi: `--model=ID`, `--lens-model=LENS:ID`), or for the repository in `config.yaml`. The `models` listing is advisory: OpenRouter's catalog returns a `:batch` id for some models its Batch API then refuses (`does not have a :batch endpoint`), at no cost, and nothing in the catalog tells them apart — so the listing tags the ids this repository's own submits have seen accepted or refused, and a refused lens says why in the submit report. `codecarto_skill {cwd, name: "broadside"}` returns the reading guide for a completed run, and unlike post-pipeline skills it is not gated on a finished pipeline.
|
|
402
402
|
|
|
403
|
-
On the Pi extension the same run is `/codecarto-broadside [submit|collect|status|models] [lenses…]`, with tab-completion for actions
|
|
403
|
+
On the Pi extension the same run is `/codecarto-broadside [submit|collect|status|models] [lenses…] [--model=ID] [--lens-model=LENS:ID]`, with tab-completion for actions, lens names, and flags and live per-lens progress while batches poll. The two surfaces differ in one deliberate place: MCP cannot ask a human, so it refuses a run over `max_cost` until you pass `force`; Pi shows the per-lens breakdown and asks, and your approval *is* the force flag. Neither surface takes an API key as a command argument — a key typed into a slash command lands in the session transcript.
|
|
404
404
|
|
|
405
405
|
Broad-Side needs runtime code, so firing a run is an executable-surface feature: Pi and MCP have it, the pure drop-in template does not (it carries only the reading guide).
|
|
406
406
|
|
|
@@ -82,6 +82,16 @@ Two more economies worth knowing:
|
|
|
82
82
|
conventions is usually a better trade than raising the model for everything.
|
|
83
83
|
Overrides are priced and capability-checked individually, and the estimate
|
|
84
84
|
breaks cost out per lens.
|
|
85
|
+
- `model` and `lens_models` are also submit parameters (Pi: `--model=ID`,
|
|
86
|
+
`--lens-model=LENS:ID`), for one run without editing the file. Choose from
|
|
87
|
+
`action: "models"`, and read that listing as advisory: OpenRouter's catalog
|
|
88
|
+
returns a `:batch` id for some models its Batch API then refuses (`does not
|
|
89
|
+
have a :batch endpoint`) — free, reported on the lens with the reason, and
|
|
90
|
+
remembered, so the listing tags ids this repository has seen accepted or
|
|
91
|
+
refused. Probe an untried model on one lens before a six-lens run. A
|
|
92
|
+
`job-submission-count` refusal is the account's concurrent-job quota (one
|
|
93
|
+
job per lens fills it fast across runs); collect or wait out what is in
|
|
94
|
+
flight, then re-submit.
|
|
85
95
|
|
|
86
96
|
## Reading a run
|
|
87
97
|
|
package/dist/core/broadside.d.ts
CHANGED
|
@@ -11,6 +11,13 @@ export declare const BROADSIDE_OUTPUT_PRICE_PER_M = 1.875;
|
|
|
11
11
|
export declare const BROADSIDE_MODELS_URL = "https://openrouter.ai/api/v1/models";
|
|
12
12
|
export declare const BROADSIDE_BENCHMARKS_URL = "https://openrouter.ai/api/v1/benchmarks";
|
|
13
13
|
export declare const BROADSIDE_CATALOG_CACHE_FILE = "model-catalog.json";
|
|
14
|
+
/**
|
|
15
|
+
* What this repository's own submits learned about batch endpoints: which
|
|
16
|
+
* `:batch` ids OpenRouter accepted a job for and which it refused with
|
|
17
|
+
* "does not have a :batch endpoint". The catalog cannot tell the two apart
|
|
18
|
+
* (#141), so the `models` action annotates its rows from this file.
|
|
19
|
+
*/
|
|
20
|
+
export declare const BROADSIDE_ENDPOINTS_FILE = "batch-endpoints.json";
|
|
14
21
|
export declare const BROADSIDE_CATALOG_CACHE_TTL_MS: number;
|
|
15
22
|
export declare const BROADSIDE_LENS_IDS: readonly ["architecture", "api", "security", "defect", "conventions", "porting"];
|
|
16
23
|
export type BroadsideLensId = (typeof BROADSIDE_LENS_IDS)[number];
|
|
@@ -539,6 +546,26 @@ export declare function loadBroadsideConfig(broadsideDir: string): Promise<Broad
|
|
|
539
546
|
export declare function defaultBroadsideConfig(): BroadsideConfig;
|
|
540
547
|
/** The catalog cache schema this build writes; a file from another is not read. */
|
|
541
548
|
export declare const BROADSIDE_CATALOG_CACHE_SCHEMA = 3;
|
|
549
|
+
/** One model's most recent submit outcome, as remembered in {@link BROADSIDE_ENDPOINTS_FILE}. */
|
|
550
|
+
export type BatchEndpointRecord = {
|
|
551
|
+
status: "accepted" | "rejected";
|
|
552
|
+
/** ISO timestamp of the submit that produced this record. */
|
|
553
|
+
at: string;
|
|
554
|
+
/** The provider's refusal, for a rejected endpoint. */
|
|
555
|
+
error?: string;
|
|
556
|
+
};
|
|
557
|
+
export declare function readBatchEndpoints(broadsideDir: string): Promise<Record<string, BatchEndpointRecord>>;
|
|
558
|
+
/**
|
|
559
|
+
* Remember what a submit learned about each model it posted to. An accepted
|
|
560
|
+
* job proves the endpoint exists; a "does not have a :batch endpoint"
|
|
561
|
+
* refusal proves it does not. Any other rejection (quota, malformed request,
|
|
562
|
+
* auth) says nothing about the endpoint and leaves the record alone.
|
|
563
|
+
*/
|
|
564
|
+
export declare function recordBatchEndpoints(broadsideDir: string, outcomes: Array<{
|
|
565
|
+
model: string;
|
|
566
|
+
batchId: string;
|
|
567
|
+
error?: unknown;
|
|
568
|
+
}>): Promise<void>;
|
|
542
569
|
export declare function builtInCatalogEntry(model: string): CatalogEntry | null;
|
|
543
570
|
export declare function builtInPricing(model: string): ModelPricing | null;
|
|
544
571
|
export declare function resolveCatalogEntry(broadsideDir: string, config: BroadsideConfig, model: string, apiKey: string, fetcher?: FetchLike): Promise<BroadsideCatalogResult>;
|
|
@@ -552,6 +579,8 @@ export declare function listBatchModels(broadsideDir: string, config: BroadsideC
|
|
|
552
579
|
source: string;
|
|
553
580
|
benchmarks: CodingBenchmarks | null;
|
|
554
581
|
defaultModel: string;
|
|
582
|
+
/** This repository's remembered submit outcomes per model, from {@link BROADSIDE_ENDPOINTS_FILE}. */
|
|
583
|
+
endpoints: Record<string, BatchEndpointRecord>;
|
|
555
584
|
}>;
|
|
556
585
|
export type FetchLike = (url: string, init: Record<string, unknown>) => Promise<Response>;
|
|
557
586
|
export declare function submitBatch(batchRequests: BatchRequest[], apiKey: string, fetcher?: FetchLike, model?: string): Promise<{
|
|
@@ -602,6 +631,12 @@ export declare function runBroadsideSubmit(cwd: string, apiKey: string, opts?: {
|
|
|
602
631
|
lenses?: BroadsideLensId[];
|
|
603
632
|
fetcher?: FetchLike;
|
|
604
633
|
model?: string;
|
|
634
|
+
/**
|
|
635
|
+
* Per-lens model overrides for this run, layered over config.yaml's
|
|
636
|
+
* `lens_models`: a lens named here runs on this model, a lens named only
|
|
637
|
+
* in the file runs on the file's, and the rest run on `model` (#141).
|
|
638
|
+
*/
|
|
639
|
+
lensModels?: Partial<Record<BroadsideLensId, string>>;
|
|
605
640
|
/** Approximate run expense limit in USD; 0 means no limit. */
|
|
606
641
|
maxCost?: number;
|
|
607
642
|
/** Submit even when the estimate exceeds maxCost. */
|
|
@@ -673,7 +708,21 @@ export declare function estimateSubmitText(result: BroadsideSubmitResult, lenses
|
|
|
673
708
|
export declare function modelsText(entries: CatalogEntry[], opts: {
|
|
674
709
|
benchmarks: CodingBenchmarks | null;
|
|
675
710
|
defaultModel: string;
|
|
711
|
+
endpoints?: Record<string, BatchEndpointRecord>;
|
|
676
712
|
}): string;
|
|
713
|
+
/**
|
|
714
|
+
* A provider refusal plus what to do about it, for the two refusals a batch
|
|
715
|
+
* run meets in practice and cannot fix by itself (#141):
|
|
716
|
+
*
|
|
717
|
+
* - `Model '<id>' does not have a :batch endpoint.` — the catalog advertises a
|
|
718
|
+
* `:batch` id that OpenRouter runs no batch endpoint for. Nothing in the
|
|
719
|
+
* catalog distinguishes these; the `models` action marks ids this
|
|
720
|
+
* repository has seen refused.
|
|
721
|
+
* - `job-submission-count … in use: 16, quota: 16` — the per-account limit
|
|
722
|
+
* on concurrent batch jobs. Broad-Side submits one job per lens, so a few
|
|
723
|
+
* runs in flight on the same key fill it; the refusal costs nothing.
|
|
724
|
+
*/
|
|
725
|
+
export declare function explainBatchError(error: unknown): string | null;
|
|
677
726
|
export declare function collectResultText(result: BroadsideCollectResult): string;
|
|
678
727
|
export declare function statusText(state: BroadsideStateFile): string;
|
|
679
728
|
export {};
|
package/dist/core/broadside.js
CHANGED
|
@@ -65,6 +65,13 @@ export const BROADSIDE_OUTPUT_PRICE_PER_M = 1.875;
|
|
|
65
65
|
export const BROADSIDE_MODELS_URL = "https://openrouter.ai/api/v1/models";
|
|
66
66
|
export const BROADSIDE_BENCHMARKS_URL = "https://openrouter.ai/api/v1/benchmarks";
|
|
67
67
|
export const BROADSIDE_CATALOG_CACHE_FILE = "model-catalog.json";
|
|
68
|
+
/**
|
|
69
|
+
* What this repository's own submits learned about batch endpoints: which
|
|
70
|
+
* `:batch` ids OpenRouter accepted a job for and which it refused with
|
|
71
|
+
* "does not have a :batch endpoint". The catalog cannot tell the two apart
|
|
72
|
+
* (#141), so the `models` action annotates its rows from this file.
|
|
73
|
+
*/
|
|
74
|
+
export const BROADSIDE_ENDPOINTS_FILE = "batch-endpoints.json";
|
|
68
75
|
export const BROADSIDE_CATALOG_CACHE_TTL_MS = 24 * 60 * 60 * 1000;
|
|
69
76
|
export const BROADSIDE_LENS_IDS = [
|
|
70
77
|
"architecture",
|
|
@@ -1621,6 +1628,67 @@ async function writeCatalogCache(broadsideDir, cache) {
|
|
|
1621
1628
|
await mkdir(broadsideDir, { recursive: true });
|
|
1622
1629
|
await writeFile(join(broadsideDir, BROADSIDE_CATALOG_CACHE_FILE), `${JSON.stringify(cache, null, "\t")}\n`, "utf8");
|
|
1623
1630
|
}
|
|
1631
|
+
const BROADSIDE_ENDPOINTS_SCHEMA = 1;
|
|
1632
|
+
export async function readBatchEndpoints(broadsideDir) {
|
|
1633
|
+
const path = join(broadsideDir, BROADSIDE_ENDPOINTS_FILE);
|
|
1634
|
+
if (!(await pathExists(path)))
|
|
1635
|
+
return {};
|
|
1636
|
+
try {
|
|
1637
|
+
const parsed = JSON.parse(await readFile(path, "utf8"));
|
|
1638
|
+
if (!parsed || typeof parsed !== "object" || parsed.schema_version !== BROADSIDE_ENDPOINTS_SCHEMA)
|
|
1639
|
+
return {};
|
|
1640
|
+
if (!parsed.models || typeof parsed.models !== "object")
|
|
1641
|
+
return {};
|
|
1642
|
+
const out = {};
|
|
1643
|
+
for (const [model, record] of Object.entries(parsed.models)) {
|
|
1644
|
+
if (!record || typeof record !== "object")
|
|
1645
|
+
continue;
|
|
1646
|
+
if (record.status !== "accepted" && record.status !== "rejected")
|
|
1647
|
+
continue;
|
|
1648
|
+
if (typeof record.at !== "string")
|
|
1649
|
+
continue;
|
|
1650
|
+
out[model] = { status: record.status, at: record.at, ...(typeof record.error === "string" && { error: record.error }) };
|
|
1651
|
+
}
|
|
1652
|
+
return out;
|
|
1653
|
+
}
|
|
1654
|
+
catch {
|
|
1655
|
+
// An unreadable memory is an empty one: it only annotates a listing.
|
|
1656
|
+
return {};
|
|
1657
|
+
}
|
|
1658
|
+
}
|
|
1659
|
+
/**
|
|
1660
|
+
* The refusal OpenRouter returns for a catalog id that has no batch endpoint
|
|
1661
|
+
* behind it. Matched loosely: the message is the only signal there is.
|
|
1662
|
+
*/
|
|
1663
|
+
const NO_BATCH_ENDPOINT_RE = /does not have a :batch endpoint/i;
|
|
1664
|
+
/** The refusal for a full per-account concurrent batch-job quota. */
|
|
1665
|
+
const BATCH_QUOTA_RE = /job-submission-count/i;
|
|
1666
|
+
/**
|
|
1667
|
+
* Remember what a submit learned about each model it posted to. An accepted
|
|
1668
|
+
* job proves the endpoint exists; a "does not have a :batch endpoint"
|
|
1669
|
+
* refusal proves it does not. Any other rejection (quota, malformed request,
|
|
1670
|
+
* auth) says nothing about the endpoint and leaves the record alone.
|
|
1671
|
+
*/
|
|
1672
|
+
export async function recordBatchEndpoints(broadsideDir, outcomes) {
|
|
1673
|
+
const at = new Date().toISOString();
|
|
1674
|
+
const updates = {};
|
|
1675
|
+
for (const { model, batchId, error } of outcomes) {
|
|
1676
|
+
if (batchId) {
|
|
1677
|
+
updates[model] = { status: "accepted", at };
|
|
1678
|
+
continue;
|
|
1679
|
+
}
|
|
1680
|
+
const message = describeBatchError(error);
|
|
1681
|
+
if (message && NO_BATCH_ENDPOINT_RE.test(message)) {
|
|
1682
|
+
updates[model] = { status: "rejected", at, error: message };
|
|
1683
|
+
}
|
|
1684
|
+
}
|
|
1685
|
+
if (Object.keys(updates).length === 0)
|
|
1686
|
+
return;
|
|
1687
|
+
const models = { ...(await readBatchEndpoints(broadsideDir)), ...updates };
|
|
1688
|
+
await mkdir(broadsideDir, { recursive: true });
|
|
1689
|
+
const file = { schema_version: BROADSIDE_ENDPOINTS_SCHEMA, models };
|
|
1690
|
+
await atomicWriteFile(join(broadsideDir, BROADSIDE_ENDPOINTS_FILE), `${JSON.stringify(file, null, "\t")}\n`);
|
|
1691
|
+
}
|
|
1624
1692
|
function parseCatalogEntry(raw) {
|
|
1625
1693
|
const id = String(raw.id ?? "");
|
|
1626
1694
|
if (!id)
|
|
@@ -1843,7 +1911,8 @@ export async function listBatchModels(broadsideDir, config, apiKey, opts = {}) {
|
|
|
1843
1911
|
cache.models[entry.id] = { ...entry, fetched_at: fetchedAt };
|
|
1844
1912
|
await writeCatalogCache(broadsideDir, cache);
|
|
1845
1913
|
const benchmarks = opts.includeBenchmarks ? await fetchCodingBenchmarks(apiKey, fetcher) : null;
|
|
1846
|
-
|
|
1914
|
+
const endpoints = await readBatchEndpoints(broadsideDir);
|
|
1915
|
+
return { entries, source: "live", benchmarks, defaultModel: config.model, endpoints };
|
|
1847
1916
|
}
|
|
1848
1917
|
export async function submitBatch(batchRequests, apiKey, fetcher = fetch, model = BROADSIDE_MODEL) {
|
|
1849
1918
|
// The OpenRouter batch endpoint stream-parses the body and requires
|
|
@@ -2001,7 +2070,8 @@ export async function runBroadsideSubmit(cwd, apiKey, opts = {}) {
|
|
|
2001
2070
|
throw new Error(`Broad-Side found no ${info.language} source files to scan (detected from ${info.manifest?.path ?? "the file counts"}; ` +
|
|
2002
2071
|
`the lenses look for ${info.sourceExts.join(", ")}). Nothing was submitted.`);
|
|
2003
2072
|
}
|
|
2004
|
-
const
|
|
2073
|
+
const lensModels = { ...config.lensModels, ...opts.lensModels };
|
|
2074
|
+
const modelForLens = (lensId) => lensModels[lensId] ?? model;
|
|
2005
2075
|
const resolved = new Map();
|
|
2006
2076
|
for (const candidate of new Set([model, ...lensIds.map(modelForLens)])) {
|
|
2007
2077
|
const catalog = await resolveCatalogEntry(broadsideDir, config, candidate, apiKey, opts.fetcher);
|
|
@@ -2215,6 +2285,12 @@ export async function runBroadsideSubmit(cwd, apiKey, opts = {}) {
|
|
|
2215
2285
|
}
|
|
2216
2286
|
await Promise.allSettled(submissions);
|
|
2217
2287
|
await persistBroadsideRun(broadsideDir, run);
|
|
2288
|
+
// What the provider just said about each model's batch endpoint outlives
|
|
2289
|
+
// the run: the `models` action reads it back (#141).
|
|
2290
|
+
await recordBatchEndpoints(broadsideDir, lensIds
|
|
2291
|
+
.map((lensId) => run.batches[lensId])
|
|
2292
|
+
.filter((entry) => Boolean(entry) && entry.status !== "skipped")
|
|
2293
|
+
.map((entry) => ({ model: entry.model ?? model, batchId: entry.batchId, error: entry.error })));
|
|
2218
2294
|
// Persist the exact request bodies so collect can re-submit a truncated
|
|
2219
2295
|
// slice (bumped output cap) without re-walking the repo (#133). The run
|
|
2220
2296
|
// dir is created here rather than waiting for collect so a crash between
|
|
@@ -2520,7 +2596,19 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
|
|
|
2520
2596
|
truncatedCount += truncated;
|
|
2521
2597
|
totalCost += cost ?? 0;
|
|
2522
2598
|
await writeFile(join(runDir, `raw-${lensId}.json`), `${JSON.stringify(batch, null, "\t")}\n`, "utf8");
|
|
2523
|
-
|
|
2599
|
+
// A batch can complete with every request failed — the account's
|
|
2600
|
+
// concurrent-job quota filling after acceptance does exactly this.
|
|
2601
|
+
// The per-request errors are on disk as `<id>.error.json`, but a
|
|
2602
|
+
// lens reporting "completed, 0 result(s)" with the reason buried
|
|
2603
|
+
// there read as an empty repository rather than a refused run.
|
|
2604
|
+
const results = Array.isArray(batch.results) ? batch.results : [];
|
|
2605
|
+
const failed = results.filter((r) => r.error && extractContent(r) === null);
|
|
2606
|
+
const allFailed = stored.length === 0 && failed.length > 0
|
|
2607
|
+
? `all ${failed.length} request(s) failed: ${explainBatchError(failed[0].error)}`
|
|
2608
|
+
: null;
|
|
2609
|
+
if (allFailed)
|
|
2610
|
+
entry.error = allFailed;
|
|
2611
|
+
lensOutcomes[lensId] = { status, cost: entry.cost, resultCount: entry.resultCount, truncated, ...(allFailed && { error: allFailed }) };
|
|
2524
2612
|
}
|
|
2525
2613
|
else {
|
|
2526
2614
|
// Every non-completed outcome still has to reach the report.
|
|
@@ -2532,7 +2620,7 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
|
|
|
2532
2620
|
// indistinguishable in the output from one that was never requested.
|
|
2533
2621
|
if (batch.error)
|
|
2534
2622
|
entry.error = batch.error;
|
|
2535
|
-
const error =
|
|
2623
|
+
const error = explainBatchError(batch.error);
|
|
2536
2624
|
lensOutcomes[lensId] = { status, cost: entry.cost, resultCount: entry.resultCount, ...(error && { error }) };
|
|
2537
2625
|
}
|
|
2538
2626
|
await persistBroadsideRun(broadsideDir, run);
|
|
@@ -2540,9 +2628,19 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
|
|
|
2540
2628
|
// #133: re-submit truncated slices once with a bumped output cap. Batch
|
|
2541
2629
|
// requests are pure, so re-running is always safe; the aim is to recover
|
|
2542
2630
|
// coverage the first pass lost to a max_tokens cutoff, not to loop forever.
|
|
2631
|
+
//
|
|
2632
|
+
// All bumped requests for one model go out as ONE batch, and the batches
|
|
2633
|
+
// (one per model, since a batch carries a single model) are polled
|
|
2634
|
+
// together against the shared deadline. Each truncated slice used to be
|
|
2635
|
+
// submitted and polled to terminal before the next was submitted, so a
|
|
2636
|
+
// model that truncated 11 of 13 slices turned a five-minute collect into
|
|
2637
|
+
// eleven sequential round trips — the serialization #136 removed from the
|
|
2638
|
+
// lens pass, still present here (#206). Grouping also keeps the retry to
|
|
2639
|
+
// one job per model against OpenRouter's 16-concurrent-job quota.
|
|
2543
2640
|
let retriedCount = 0;
|
|
2544
2641
|
if (opts.retryTruncated !== false && truncatedCount > 0) {
|
|
2545
2642
|
const requestsByCustomId = await loadStoredRequests(runDir);
|
|
2643
|
+
const byModel = new Map();
|
|
2546
2644
|
for (const stored of allLensResults) {
|
|
2547
2645
|
if (!stored.truncated)
|
|
2548
2646
|
continue;
|
|
@@ -2559,42 +2657,57 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
|
|
|
2559
2657
|
const bumpedMax = lensCap ? Math.min(previousMax * 2, lensCap) : previousMax * 2;
|
|
2560
2658
|
if (bumpedMax <= previousMax)
|
|
2561
2659
|
continue; // already at the ceiling
|
|
2562
|
-
const
|
|
2563
|
-
|
|
2564
|
-
|
|
2565
|
-
|
|
2660
|
+
const group = byModel.get(lensModel) ?? { requests: [], slices: new Map() };
|
|
2661
|
+
group.requests.push({ ...original, body: { ...original.body, max_tokens: bumpedMax } });
|
|
2662
|
+
group.slices.set(stored.customId, stored);
|
|
2663
|
+
byModel.set(lensModel, group);
|
|
2664
|
+
}
|
|
2665
|
+
// Submit every group, then poll whatever was accepted, together.
|
|
2666
|
+
const submitted = [];
|
|
2667
|
+
for (const [model, group] of byModel) {
|
|
2566
2668
|
try {
|
|
2567
|
-
const { batchId, error } = await submitBatch(
|
|
2568
|
-
if (error)
|
|
2669
|
+
const { batchId, error } = await submitBatch(group.requests, apiKey, opts.fetcher, model);
|
|
2670
|
+
if (!error && batchId)
|
|
2671
|
+
submitted.push({ model, batchId });
|
|
2672
|
+
}
|
|
2673
|
+
catch {
|
|
2674
|
+
// A retry batch that fails to submit leaves its slices' original
|
|
2675
|
+
// truncated results in place — nothing is lost.
|
|
2676
|
+
}
|
|
2677
|
+
}
|
|
2678
|
+
const polled = await pollBatchesConcurrently(submitted.map(({ model, batchId }) => ({ lensId: `retry:${model}`, batchId })), apiKey, {
|
|
2679
|
+
// Share the caller's deadline. Each of these polls used to start a
|
|
2680
|
+
// fresh 25-minute budget, so `wait_seconds` bounded only the lens
|
|
2681
|
+
// poll and a collect could run for the caller's budget plus fifty
|
|
2682
|
+
// minutes.
|
|
2683
|
+
deadlineMs: Math.max(0, deadline - Date.now()),
|
|
2684
|
+
fetcher: opts.fetcher,
|
|
2685
|
+
onStatus: opts.onStatus,
|
|
2686
|
+
});
|
|
2687
|
+
for (const { model, batchId } of submitted) {
|
|
2688
|
+
const batch = polled.get(batchId);
|
|
2689
|
+
if (!batch || batch.status !== "completed")
|
|
2690
|
+
continue;
|
|
2691
|
+
const group = byModel.get(model);
|
|
2692
|
+
const usage = (batch.usage ?? {});
|
|
2693
|
+
totalCost += typeof usage.cost === "number" ? usage.cost : 0;
|
|
2694
|
+
const results = Array.isArray(batch.results) ? batch.results : [];
|
|
2695
|
+
for (const result of results) {
|
|
2696
|
+
const stored = group.slices.get(String(result.custom_id ?? ""));
|
|
2697
|
+
if (!stored)
|
|
2569
2698
|
continue;
|
|
2570
|
-
const
|
|
2571
|
-
|
|
2572
|
-
// start a fresh 25-minute budget, so `wait_seconds` bounded
|
|
2573
|
-
// only the lens poll and a collect could run for the caller's
|
|
2574
|
-
// budget plus fifty minutes.
|
|
2575
|
-
deadlineMs: Math.max(0, deadline - Date.now()),
|
|
2576
|
-
onStatus: (status, counts) => opts.onStatus?.(`${stored.lensId}:retry`, status, counts),
|
|
2577
|
-
fetcher: opts.fetcher,
|
|
2578
|
-
});
|
|
2579
|
-
if (batch.status !== "completed")
|
|
2699
|
+
const content = extractContent(result);
|
|
2700
|
+
if (content === null)
|
|
2580
2701
|
continue;
|
|
2581
|
-
const results = Array.isArray(batch.results) ? batch.results : [];
|
|
2582
|
-
const content = results.length > 0 ? extractContent(results[0]) : null;
|
|
2583
|
-
if (content === null || parseLensJson(content) === null)
|
|
2584
|
-
continue; // still no good
|
|
2585
|
-
const usage = (batch.usage ?? {});
|
|
2586
|
-
totalCost += typeof usage.cost === "number" ? usage.cost : 0;
|
|
2587
2702
|
const parsed = parseLensJson(content);
|
|
2703
|
+
if (parsed === null)
|
|
2704
|
+
continue; // still no good
|
|
2588
2705
|
await writeFile(join(runDir, `${sanitizeId(stored.customId)}.json`), `${JSON.stringify(parsed, null, "\t")}\n`, "utf8");
|
|
2589
2706
|
await writeFile(join(runDir, `${sanitizeId(stored.customId)}.md`), renderFindingsMarkdown(content), "utf8");
|
|
2590
2707
|
stored.content = content;
|
|
2591
2708
|
stored.truncated = false;
|
|
2592
2709
|
retriedCount += 1;
|
|
2593
2710
|
}
|
|
2594
|
-
catch {
|
|
2595
|
-
// A retry that fails to submit/poll leaves the original
|
|
2596
|
-
// truncated result in place — nothing is lost.
|
|
2597
|
-
}
|
|
2598
2711
|
}
|
|
2599
2712
|
truncatedCount = allLensResults.filter((s) => s.truncated).length;
|
|
2600
2713
|
for (const [lensId, outcome] of Object.entries(lensOutcomes)) {
|
|
@@ -2878,7 +2991,11 @@ export function estimateSubmitText(result, lenses) {
|
|
|
2878
2991
|
continue;
|
|
2879
2992
|
const status = entry.batchId ? `batch ${entry.batchId}` : entry.status;
|
|
2880
2993
|
const override = entry.model ? ` on ${entry.model}` : "";
|
|
2881
|
-
|
|
2994
|
+
// A rejected lens says why: the message is the only way to tell a
|
|
2995
|
+
// catalog id with no batch endpoint from a full job quota, and both
|
|
2996
|
+
// used to read as a bare "rejected".
|
|
2997
|
+
const reason = !entry.batchId && entry.error ? ` — ${explainBatchError(entry.error)}` : "";
|
|
2998
|
+
lines.push(` ${lens.name}: ${status} (${entry.requests} request(s), ~$${entry.estimatedCost.toFixed(4)})${override}${reason}`);
|
|
2882
2999
|
}
|
|
2883
3000
|
if (result.repo) {
|
|
2884
3001
|
const head = result.repo.sourceHead ? ` at ${result.repo.sourceHead.slice(0, 8)}${result.repo.sourceDirty ? " (dirty)" : ""}` : "";
|
|
@@ -2915,8 +3032,15 @@ export function estimateSubmitText(result, lenses) {
|
|
|
2915
3032
|
return lines.join("\n");
|
|
2916
3033
|
}
|
|
2917
3034
|
export function modelsText(entries, opts) {
|
|
3035
|
+
const endpoints = opts.endpoints ?? {};
|
|
2918
3036
|
const lines = [
|
|
2919
3037
|
`Batch models on OpenRouter (${entries.length}, cheapest first).`,
|
|
3038
|
+
// The catalog over-reports: it returns a `:batch` id for models whose
|
|
3039
|
+
// Batch API refuses the job, with nothing in the entry to tell them
|
|
3040
|
+
// apart (#141). Say so before the table, not after it.
|
|
3041
|
+
"Advisory: this is the catalog's list of :batch ids, not a list of working batch endpoints. Some ids are refused at submit " +
|
|
3042
|
+
"(\"does not have a :batch endpoint\"), at no cost. Rows tagged [no batch endpoint …] or [batch OK …] carry what this " +
|
|
3043
|
+
"repository's own submits found; an untagged row has not been tried here.",
|
|
2920
3044
|
"",
|
|
2921
3045
|
"id | $/M in | $/M out | ctx | max out | structured | coding idx",
|
|
2922
3046
|
];
|
|
@@ -2936,12 +3060,21 @@ export function modelsText(entries, opts) {
|
|
|
2936
3060
|
const out = entry.maxCompletionTokens ? `${(entry.maxCompletionTokens / 1024).toFixed(0)}k` : "?";
|
|
2937
3061
|
const tag = entry.id === opts.defaultModel ? " (default)" : "";
|
|
2938
3062
|
const exp = entry.expirationDate ? " [deprecated]" : "";
|
|
2939
|
-
|
|
3063
|
+
const record = endpoints[entry.id];
|
|
3064
|
+
const seen = record
|
|
3065
|
+
? record.status === "rejected"
|
|
3066
|
+
? ` [no batch endpoint, refused ${record.at.slice(0, 10)}]`
|
|
3067
|
+
: ` [batch OK ${record.at.slice(0, 10)}]`
|
|
3068
|
+
: "";
|
|
3069
|
+
lines.push(`${entry.id}${tag}${exp}${seen} | ${entry.inputPerM.toFixed(3)} | ${entry.outputPerM.toFixed(3)} | ${ctx} | ${out} | ${structured} | ${coding}`);
|
|
2940
3070
|
}
|
|
2941
3071
|
if (opts.benchmarks?.meta.as_of) {
|
|
2942
3072
|
lines.push("", `Benchmarks: Artificial Analysis coding index (as of ${String(opts.benchmarks.meta.as_of)}).`);
|
|
2943
3073
|
}
|
|
2944
|
-
lines.push("", "
|
|
3074
|
+
lines.push("", "Choose with the model parameter (--model= on Pi) for one run, lens_models (--lens-model=LENS:ID) per lens, or the model key in " +
|
|
3075
|
+
".codecarto/broadside/config.yaml for the repository. Higher coding index ≠ better scout: precision, context, structured-output " +
|
|
3076
|
+
"support, and whether the model spends its output budget reasoning (see reasoning: in config.yaml) matter most here. " +
|
|
3077
|
+
"A refused submit costs nothing, so probe an untried model on one lens first.");
|
|
2945
3078
|
return lines.join("\n");
|
|
2946
3079
|
}
|
|
2947
3080
|
/** One line of a batch's error field, whatever shape the provider gave it. */
|
|
@@ -2954,6 +3087,15 @@ function describeBatchError(error) {
|
|
|
2954
3087
|
const message = error.message;
|
|
2955
3088
|
if (typeof message === "string" && message)
|
|
2956
3089
|
return message.slice(0, 300);
|
|
3090
|
+
// OpenRouter wraps a submit refusal as `{ error: { message } }`.
|
|
3091
|
+
const nested = error.error;
|
|
3092
|
+
if (nested && typeof nested === "object") {
|
|
3093
|
+
const inner = nested.message;
|
|
3094
|
+
if (typeof inner === "string" && inner)
|
|
3095
|
+
return inner.slice(0, 300);
|
|
3096
|
+
}
|
|
3097
|
+
if (typeof nested === "string" && nested)
|
|
3098
|
+
return nested.slice(0, 300);
|
|
2957
3099
|
try {
|
|
2958
3100
|
return JSON.stringify(error).slice(0, 300);
|
|
2959
3101
|
}
|
|
@@ -2963,6 +3105,30 @@ function describeBatchError(error) {
|
|
|
2963
3105
|
}
|
|
2964
3106
|
return String(error);
|
|
2965
3107
|
}
|
|
3108
|
+
/**
|
|
3109
|
+
* A provider refusal plus what to do about it, for the two refusals a batch
|
|
3110
|
+
* run meets in practice and cannot fix by itself (#141):
|
|
3111
|
+
*
|
|
3112
|
+
* - `Model '<id>' does not have a :batch endpoint.` — the catalog advertises a
|
|
3113
|
+
* `:batch` id that OpenRouter runs no batch endpoint for. Nothing in the
|
|
3114
|
+
* catalog distinguishes these; the `models` action marks ids this
|
|
3115
|
+
* repository has seen refused.
|
|
3116
|
+
* - `job-submission-count … in use: 16, quota: 16` — the per-account limit
|
|
3117
|
+
* on concurrent batch jobs. Broad-Side submits one job per lens, so a few
|
|
3118
|
+
* runs in flight on the same key fill it; the refusal costs nothing.
|
|
3119
|
+
*/
|
|
3120
|
+
export function explainBatchError(error) {
|
|
3121
|
+
const message = describeBatchError(error);
|
|
3122
|
+
if (!message)
|
|
3123
|
+
return null;
|
|
3124
|
+
if (NO_BATCH_ENDPOINT_RE.test(message)) {
|
|
3125
|
+
return `${message} — the catalog lists this id, but OpenRouter runs no batch endpoint for it. Nothing was charged; pick another model (the models action marks ids this repository has seen refused).`;
|
|
3126
|
+
}
|
|
3127
|
+
if (BATCH_QUOTA_RE.test(message)) {
|
|
3128
|
+
return `${message} — OpenRouter's per-account limit on concurrent batch jobs is full. Broad-Side submits one job per lens, so a few runs in flight on this key (in any repository) fill it. Nothing was charged; collect or wait out the runs in flight, then re-submit.`;
|
|
3129
|
+
}
|
|
3130
|
+
return message;
|
|
3131
|
+
}
|
|
2966
3132
|
export function collectResultText(result) {
|
|
2967
3133
|
const lines = [
|
|
2968
3134
|
`Broad-Side run ${result.runId}: ${result.status}`,
|
|
@@ -18,11 +18,15 @@ export interface BroadsideFlags {
|
|
|
18
18
|
waitSeconds?: number;
|
|
19
19
|
/** For collect: the run to collect instead of the most recent (#268). */
|
|
20
20
|
runId?: string;
|
|
21
|
+
/** For submit: the run's batch model, replacing config.yaml's (#141). */
|
|
22
|
+
model?: string;
|
|
23
|
+
/** For submit: per-lens model overrides, layered over config.yaml's (#141). */
|
|
24
|
+
lensModels?: Partial<Record<BroadsideLensId, string>>;
|
|
21
25
|
benchmarks: boolean;
|
|
22
26
|
unknown: string[];
|
|
23
27
|
/** Set on an invalid combination. The caller surfaces it as an error. */
|
|
24
28
|
error?: string;
|
|
25
29
|
}
|
|
26
30
|
/** Every token the completer offers, in the order it offers them. */
|
|
27
|
-
export declare const KNOWN_BROADSIDE_TOKENS: readonly ["submit", "collect", "status", "models", "architecture", "api", "security", "defect", "conventions", "porting", "--incremental", "--no-incremental", "--max-cost=", "--wait=", "--run=", "--no-synthesis", "--no-triage", "--no-retry-truncated", "--benchmarks"];
|
|
31
|
+
export declare const KNOWN_BROADSIDE_TOKENS: readonly ["submit", "collect", "status", "models", "architecture", "api", "security", "defect", "conventions", "porting", "--incremental", "--no-incremental", "--max-cost=", "--wait=", "--run=", "--model=", "--lens-model=", "--no-synthesis", "--no-triage", "--no-retry-truncated", "--benchmarks"];
|
|
28
32
|
export declare function parseBroadsideFlags(args: string): BroadsideFlags;
|
|
@@ -14,6 +14,11 @@
|
|
|
14
14
|
// --max-cost=N --no-retry-truncated
|
|
15
15
|
// --wait=SECONDS --benchmarks (models only)
|
|
16
16
|
// --run=ID (collect only: an older run, as listed by status)
|
|
17
|
+
// --model=ID (submit only: the run's batch model, as listed by models)
|
|
18
|
+
// --lens-model=LENS:ID (submit only, repeatable: one lens on its own model)
|
|
19
|
+
//
|
|
20
|
+
// A model id itself contains a colon (`vendor/name:batch`), so --lens-model
|
|
21
|
+
// splits on the first colon only: `security:deepseek/deepseek-v4-pro:batch`.
|
|
17
22
|
//
|
|
18
23
|
// --incremental has a spelled-out negative because the value is tri-state:
|
|
19
24
|
// absent defers to config.yaml, so a repository that set `incremental: true`
|
|
@@ -36,6 +41,8 @@ export const KNOWN_BROADSIDE_TOKENS = [
|
|
|
36
41
|
"--max-cost=",
|
|
37
42
|
"--wait=",
|
|
38
43
|
"--run=",
|
|
44
|
+
"--model=",
|
|
45
|
+
"--lens-model=",
|
|
39
46
|
"--no-synthesis",
|
|
40
47
|
"--no-triage",
|
|
41
48
|
"--no-retry-truncated",
|
|
@@ -115,6 +122,32 @@ export function parseBroadsideFlags(args) {
|
|
|
115
122
|
result.runId = value || undefined;
|
|
116
123
|
continue;
|
|
117
124
|
}
|
|
125
|
+
if (token.startsWith("--model=")) {
|
|
126
|
+
const value = token.slice("--model=".length).trim();
|
|
127
|
+
// An empty value is a mistyped selection, not "use the default":
|
|
128
|
+
// the command is about to spend money on whichever model wins.
|
|
129
|
+
if (!value)
|
|
130
|
+
result.error ??= "--model= needs an OpenRouter batch model id (see /codecarto-broadside models).";
|
|
131
|
+
result.model = value || undefined;
|
|
132
|
+
continue;
|
|
133
|
+
}
|
|
134
|
+
if (token.startsWith("--lens-model=")) {
|
|
135
|
+
const value = token.slice("--lens-model=".length).trim();
|
|
136
|
+
const colon = value.indexOf(":");
|
|
137
|
+
const lensId = colon > 0 ? value.slice(0, colon).trim() : "";
|
|
138
|
+
const modelId = colon > 0 ? value.slice(colon + 1).trim() : "";
|
|
139
|
+
if (!lensId || !modelId) {
|
|
140
|
+
result.error ??= `--lens-model needs LENS:MODEL, e.g. --lens-model=security:vendor/name:batch (got "${value}").`;
|
|
141
|
+
}
|
|
142
|
+
else if (!BROADSIDE_LENS_IDS.includes(lensId)) {
|
|
143
|
+
result.error ??= `--lens-model: unknown lens "${lensId}". Lenses: ${BROADSIDE_LENS_IDS.join(", ")}.`;
|
|
144
|
+
}
|
|
145
|
+
else {
|
|
146
|
+
result.lensModels ??= {};
|
|
147
|
+
result.lensModels[lensId] = modelId;
|
|
148
|
+
}
|
|
149
|
+
continue;
|
|
150
|
+
}
|
|
118
151
|
result.unknown.push(token);
|
|
119
152
|
}
|
|
120
153
|
// Flags that only mean something for one action are refused rather than
|
|
@@ -137,5 +170,11 @@ export function parseBroadsideFlags(args) {
|
|
|
137
170
|
if (result.runId !== undefined && result.action !== "collect") {
|
|
138
171
|
result.error ??= `--run is only meaningful for collect (got action "${result.action}").`;
|
|
139
172
|
}
|
|
173
|
+
if (result.model !== undefined && result.action !== "submit") {
|
|
174
|
+
result.error ??= `--model is only meaningful for submit (got action "${result.action}").`;
|
|
175
|
+
}
|
|
176
|
+
if (result.lensModels !== undefined && result.action !== "submit") {
|
|
177
|
+
result.error ??= `--lens-model is only meaningful for submit (got action "${result.action}").`;
|
|
178
|
+
}
|
|
140
179
|
return result;
|
|
141
180
|
}
|
|
@@ -1000,7 +1000,7 @@ export default function codeCartographerExtension(pi) {
|
|
|
1000
1000
|
},
|
|
1001
1001
|
});
|
|
1002
1002
|
pi.registerCommand("codecarto-broadside", {
|
|
1003
|
-
description: "Batch reconnaissance (Broad-Side): /codecarto-broadside [submit|collect|status|models] [lenses…] [flags]",
|
|
1003
|
+
description: "Batch reconnaissance (Broad-Side): /codecarto-broadside [submit|collect|status|models] [lenses…] [--model=ID] [--lens-model=LENS:ID] [flags]",
|
|
1004
1004
|
getArgumentCompletions: (prefix) => {
|
|
1005
1005
|
const items = KNOWN_BROADSIDE_TOKENS
|
|
1006
1006
|
.filter((value) => value.startsWith(prefix))
|
|
@@ -1077,10 +1077,10 @@ export default function codeCartographerExtension(pi) {
|
|
|
1077
1077
|
if (flags.action === "models") {
|
|
1078
1078
|
notifyCtx(ctx, "Fetching the OpenRouter batch-model catalog…", "info");
|
|
1079
1079
|
try {
|
|
1080
|
-
const { entries, benchmarks } = await listBatchModels(broadsideDir, config, apiKey, {
|
|
1080
|
+
const { entries, benchmarks, endpoints } = await listBatchModels(broadsideDir, config, apiKey, {
|
|
1081
1081
|
includeBenchmarks: flags.benchmarks,
|
|
1082
1082
|
});
|
|
1083
|
-
finish(modelsText(entries, { benchmarks, defaultModel: config.model }).split("\n"), `Broad-Side: ${entries.length} batch model${entries.length === 1 ? "" : "s"} listed`);
|
|
1083
|
+
finish(modelsText(entries, { benchmarks, defaultModel: config.model, endpoints }).split("\n"), `Broad-Side: ${entries.length} batch model${entries.length === 1 ? "" : "s"} listed`);
|
|
1084
1084
|
}
|
|
1085
1085
|
catch (error) {
|
|
1086
1086
|
notifyCtx(ctx, `Model catalog lookup failed: ${error instanceof Error ? error.message : String(error)}`, "error");
|
|
@@ -1117,7 +1117,11 @@ export default function codeCartographerExtension(pi) {
|
|
|
1117
1117
|
try {
|
|
1118
1118
|
submit = await runBroadsideSubmit(ctx.cwd, apiKey, {
|
|
1119
1119
|
lenses,
|
|
1120
|
-
model
|
|
1120
|
+
// --model= and --lens-model= select for this run; the file's
|
|
1121
|
+
// values are the fallback, and core pre-flights either the
|
|
1122
|
+
// same way (#141).
|
|
1123
|
+
model: flags.model ?? config.model,
|
|
1124
|
+
lensModels: flags.lensModels,
|
|
1121
1125
|
maxCost: flags.maxCost ?? config.maxCost,
|
|
1122
1126
|
// `??`, not `||`: --no-incremental parses to false and must beat a
|
|
1123
1127
|
// config-set true, exactly as MCP's `incremental: false` does (#163).
|
|
@@ -1079,15 +1079,19 @@ export async function handleBroadside(args) {
|
|
|
1079
1079
|
const retryTruncated = args.retry_truncated ?? config.retryTruncated;
|
|
1080
1080
|
const incremental = args.incremental ?? config.incremental;
|
|
1081
1081
|
if (action === "models") {
|
|
1082
|
-
const { entries, benchmarks } = await listBatchModels(broadsideDirFor(cwd), config, apiKey, {
|
|
1082
|
+
const { entries, benchmarks, endpoints } = await listBatchModels(broadsideDirFor(cwd), config, apiKey, {
|
|
1083
1083
|
includeBenchmarks: args.include_benchmarks === true,
|
|
1084
1084
|
}).catch((error) => {
|
|
1085
1085
|
throw new McpError(ErrorCode.InvalidRequest, error instanceof Error ? error.message : String(error));
|
|
1086
1086
|
});
|
|
1087
|
-
return textResult(modelsText(entries, { benchmarks, defaultModel: config.model }), {
|
|
1087
|
+
return textResult(modelsText(entries, { benchmarks, defaultModel: config.model, endpoints }), {
|
|
1088
1088
|
models: entries,
|
|
1089
1089
|
defaultModel: config.model,
|
|
1090
1090
|
benchmarkMeta: benchmarks?.meta ?? null,
|
|
1091
|
+
// The catalog is advisory (#141): what this repository's submits
|
|
1092
|
+
// learned about each id's batch endpoint rides alongside it.
|
|
1093
|
+
catalogAdvisory: true,
|
|
1094
|
+
endpoints,
|
|
1091
1095
|
});
|
|
1092
1096
|
}
|
|
1093
1097
|
if (action === "submit") {
|
|
@@ -1105,9 +1109,34 @@ export async function handleBroadside(args) {
|
|
|
1105
1109
|
// An explicit 0 is "no limit" (#231); absent falls back to config.yaml,
|
|
1106
1110
|
// whose own default is BROADSIDE_DEFAULT_MAX_COST.
|
|
1107
1111
|
const maxCost = typeof args.max_cost === "number" && args.max_cost >= 0 ? args.max_cost : config.maxCost;
|
|
1112
|
+
// Model selection for one run (#141): `model` replaces the run default,
|
|
1113
|
+
// `lens_models` layers per-lens overrides over config.yaml's. Both are
|
|
1114
|
+
// pre-flighted by core exactly like the file's values — priced from the
|
|
1115
|
+
// catalog, refused without structured-output support, clamped to the
|
|
1116
|
+
// model's ceiling — so a wrong id fails before anything is submitted.
|
|
1117
|
+
const model = typeof args.model === "string" && args.model.trim() ? args.model.trim() : config.model;
|
|
1118
|
+
if (args.model !== undefined && !(typeof args.model === "string" && args.model.trim())) {
|
|
1119
|
+
throw new McpError(ErrorCode.InvalidParams, "model must be a non-empty OpenRouter batch model id (see action 'models').");
|
|
1120
|
+
}
|
|
1121
|
+
const lensModels = {};
|
|
1122
|
+
if (args.lens_models !== undefined) {
|
|
1123
|
+
if (!args.lens_models || typeof args.lens_models !== "object" || Array.isArray(args.lens_models)) {
|
|
1124
|
+
throw new McpError(ErrorCode.InvalidParams, "lens_models must be an object mapping lens ids to batch model ids.");
|
|
1125
|
+
}
|
|
1126
|
+
for (const [lensId, value] of Object.entries(args.lens_models)) {
|
|
1127
|
+
if (!BROADSIDE_LENS_IDS.includes(lensId)) {
|
|
1128
|
+
throw new McpError(ErrorCode.InvalidParams, `lens_models: unknown lens "${lensId}". Valid: ${BROADSIDE_LENS_IDS.join(", ")}`);
|
|
1129
|
+
}
|
|
1130
|
+
if (typeof value !== "string" || !value.trim()) {
|
|
1131
|
+
throw new McpError(ErrorCode.InvalidParams, `lens_models.${lensId} must be a non-empty OpenRouter batch model id.`);
|
|
1132
|
+
}
|
|
1133
|
+
lensModels[lensId] = value.trim();
|
|
1134
|
+
}
|
|
1135
|
+
}
|
|
1108
1136
|
const result = await runBroadsideSubmit(cwd, apiKey, {
|
|
1109
1137
|
lenses,
|
|
1110
|
-
model
|
|
1138
|
+
model,
|
|
1139
|
+
lensModels,
|
|
1111
1140
|
maxCost,
|
|
1112
1141
|
force: args.force === true,
|
|
1113
1142
|
incremental,
|
|
@@ -1516,6 +1545,15 @@ const TOOLS = [
|
|
|
1516
1545
|
type: "boolean",
|
|
1517
1546
|
description: "For action 'models': annotate each model with its Artificial Analysis coding index (extra API call; default false).",
|
|
1518
1547
|
},
|
|
1548
|
+
model: {
|
|
1549
|
+
type: "string",
|
|
1550
|
+
description: "For submit: the OpenRouter batch model for this run (an id ending in :batch, as listed by action 'models'). Falls back to model in .codecarto/broadside/config.yaml, then the shipped default. Pre-flighted like the configured model: priced from the live catalog, refused without structured-output support, clamped to its completion ceiling. The models listing is advisory — some catalog ids have no batch endpoint and are refused at submit, at no cost; the listing tags ids this repository has already seen accepted or refused.",
|
|
1551
|
+
},
|
|
1552
|
+
lens_models: {
|
|
1553
|
+
type: "object",
|
|
1554
|
+
additionalProperties: { type: "string" },
|
|
1555
|
+
description: "For submit: per-lens model overrides for this run, e.g. {\"security\": \"deepseek/deepseek-v4-pro-0813:batch\"}. Keys are lens ids; a lens named here runs on that model, others on `model`. Layered over lens_models in .codecarto/broadside/config.yaml (a lens set in both takes the parameter's). Each override is priced, capability-checked, and clamped individually, and the estimate breaks cost out per lens.",
|
|
1556
|
+
},
|
|
1519
1557
|
},
|
|
1520
1558
|
required: ["cwd", "action"],
|
|
1521
1559
|
},
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "codecartographer-pi",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.22.0",
|
|
4
4
|
"mcpName": "io.github.HuginnIndustries/codecartographer",
|
|
5
5
|
"description": "Turn an unfamiliar codebase into a validated reimplementation spec, then synthesize confirmed specs and a product vision into a traceable plan.",
|
|
6
6
|
"type": "module",
|