codecartographer-pi 0.21.0 → 0.22.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -108,7 +108,14 @@ architecture map. Each override is priced, capability-checked, and clamped like
108
108
  the default, the submit estimate breaks cost out per lens, and `run-meta.json`
109
109
  records which lens ran on what. No stronger default is shipped: which model is
110
110
  worth the money depends on the repository and the budget, so compare with the
111
- `models` action and decide.
111
+ `models` action and decide — for one run with the `model` and `lens_models`
112
+ parameters (Pi: `--model=ID`, `--lens-model=LENS:ID`), or here for the
113
+ repository. The `models` listing is advisory: OpenRouter's catalog returns a
114
+ `:batch` id for some models its Batch API refuses (`does not have a :batch
115
+ endpoint`), at no cost, and nothing in the catalog tells them apart. The
116
+ listing tags the ids this repository's own submits have seen accepted or
117
+ refused (`broadside/batch-endpoints.json`), and a refused lens says why in the
118
+ submit report.
112
119
 
113
120
  Every run knob — `incremental`, `retry_truncated`, `include_synthesis`,
114
121
  `include_triage`, `wait_seconds` — also has a repository default under the same
@@ -54,10 +54,18 @@
54
54
  # Nothing in the catalog distinguishes them. Every Anthropic and OpenAI
55
55
  # batch id tried so far is rejected this way; Google and DeepSeek work.
56
56
  # A rejected batch costs nothing, so probe a candidate on one lens before
57
- # relying on it.
57
+ # relying on it: `codecarto_broadside {action: "submit", lenses:
58
+ # ["architecture"], model: "<id>"}` (Pi: `/codecarto-broadside architecture
59
+ # --model=<id>`). Submits remember the answer in batch-endpoints.json next
60
+ # to this file, and the `models` listing tags each id accordingly.
58
61
  # 2. A reasoning-capable model spends its output budget thinking, and the
59
62
  # thinking is billed at the full output rate. See `reasoning:` below.
60
63
  #
64
+ # The same two keys are submit parameters for one run — `model` and
65
+ # `lens_models` on codecarto_broadside, `--model=ID` and `--lens-model=LENS:ID`
66
+ # on /codecarto-broadside — and a lens set both there and here takes the
67
+ # parameter's.
68
+ #
61
69
  # lens_models:
62
70
  # security: deepseek/deepseek-v4-pro-0813:batch
63
71
  # defect: deepseek/deepseek-v4-pro-0813:batch
@@ -3,4 +3,4 @@
3
3
  # workspace's framework-owned files (GUIDE.md, templates/, workflow/ pipelines
4
4
  # and VALIDATE.md) predate the running release. Written at release time and
5
5
  # copied verbatim by init — never edit by hand.
6
- scaffold_version: 0.21.0
6
+ scaffold_version: 0.22.0
package/README.md CHANGED
@@ -398,9 +398,9 @@ codecarto_broadside {cwd, action: "collect"} # poll, save, synt
398
398
 
399
399
  Submit and collect are separate because batch jobs routinely take tens of minutes; collect is resumable and picks up whatever is still in flight (`wait_seconds: 0`, the default, polls once and returns). Submit prices the run from the collected file sizes against the model's live per-token pricing (cached 24h) and refuses when the estimate exceeds `max_cost` — $1.00 unless the config or the call sets another value, `0` for no limit — unless `force: true` is passed — a pre-flight estimate, not a runtime stop. Actual spend lands in each run's `run-meta.json`.
400
400
 
401
- Repository defaults live in `.codecarto/broadside/config.yaml` (`model`, `api_key`, `default_lenses`, `max_cost`, `pricing` overrides, `lens_models`, `incremental`, `retry_truncated`, `include_synthesis`, `include_triage`, `wait_seconds`); an explicit tool parameter always wins. `lens_models` routes individual lenses to their own batch model — a stronger model changes security and defect findings far more than it changes an architecture map — and each override is priced, capability-checked, and clamped exactly like the default, with the estimate broken out per lens so a mixed-model run cannot be approved without seeing which lens costs what. CodeCartographer ships no stronger default: which model earns its price depends on your repository and budget, so compare candidates with the `models` action and choose. `codecarto_skill {cwd, name: "broadside"}` returns the reading guide for a completed run, and unlike post-pipeline skills it is not gated on a finished pipeline.
401
+ Repository defaults live in `.codecarto/broadside/config.yaml` (`model`, `api_key`, `default_lenses`, `max_cost`, `pricing` overrides, `lens_models`, `incremental`, `retry_truncated`, `include_synthesis`, `include_triage`, `wait_seconds`); an explicit tool parameter always wins. `lens_models` routes individual lenses to their own batch model — a stronger model changes security and defect findings far more than it changes an architecture map — and each override is priced, capability-checked, and clamped exactly like the default, with the estimate broken out per lens so a mixed-model run cannot be approved without seeing which lens costs what. CodeCartographer ships no stronger default: which model earns its price depends on your repository and budget, so compare candidates with the `models` action and choose — for one run with the `model` and `lens_models` parameters (Pi: `--model=ID`, `--lens-model=LENS:ID`), or for the repository in `config.yaml`. The `models` listing is advisory: OpenRouter's catalog returns a `:batch` id for some models its Batch API then refuses (`does not have a :batch endpoint`), at no cost, and nothing in the catalog tells them apart — so the listing tags the ids this repository's own submits have seen accepted or refused, and a refused lens says why in the submit report. `codecarto_skill {cwd, name: "broadside"}` returns the reading guide for a completed run, and unlike post-pipeline skills it is not gated on a finished pipeline.
402
402
 
403
- On the Pi extension the same run is `/codecarto-broadside [submit|collect|status|models] [lenses…]`, with tab-completion for actions and lens names and live per-lens progress while batches poll. The two surfaces differ in one deliberate place: MCP cannot ask a human, so it refuses a run over `max_cost` until you pass `force`; Pi shows the per-lens breakdown and asks, and your approval *is* the force flag. Neither surface takes an API key as a command argument — a key typed into a slash command lands in the session transcript.
403
+ On the Pi extension the same run is `/codecarto-broadside [submit|collect|status|models] [lenses…] [--model=ID] [--lens-model=LENS:ID]`, with tab-completion for actions, lens names, and flags and live per-lens progress while batches poll. The two surfaces differ in one deliberate place: MCP cannot ask a human, so it refuses a run over `max_cost` until you pass `force`; Pi shows the per-lens breakdown and asks, and your approval *is* the force flag. Neither surface takes an API key as a command argument — a key typed into a slash command lands in the session transcript.
404
404
 
405
405
  Broad-Side needs runtime code, so firing a run is an executable-surface feature: Pi and MCP have it, the pure drop-in template does not (it carries only the reading guide).
406
406
 
@@ -82,6 +82,16 @@ Two more economies worth knowing:
82
82
  conventions is usually a better trade than raising the model for everything.
83
83
  Overrides are priced and capability-checked individually, and the estimate
84
84
  breaks cost out per lens.
85
+ - `model` and `lens_models` are also submit parameters (Pi: `--model=ID`,
86
+ `--lens-model=LENS:ID`), for one run without editing the file. Choose from
87
+ `action: "models"`, and read that listing as advisory: OpenRouter's catalog
88
+ returns a `:batch` id for some models its Batch API then refuses (`does not
89
+ have a :batch endpoint`) — free, reported on the lens with the reason, and
90
+ remembered, so the listing tags ids this repository has seen accepted or
91
+ refused. Probe an untried model on one lens before a six-lens run. A
92
+ `job-submission-count` refusal is the account's concurrent-job quota (one
93
+ job per lens fills it fast across runs); collect or wait out what is in
94
+ flight, then re-submit.
85
95
 
86
96
  ## Reading a run
87
97
 
@@ -11,6 +11,13 @@ export declare const BROADSIDE_OUTPUT_PRICE_PER_M = 1.875;
11
11
  export declare const BROADSIDE_MODELS_URL = "https://openrouter.ai/api/v1/models";
12
12
  export declare const BROADSIDE_BENCHMARKS_URL = "https://openrouter.ai/api/v1/benchmarks";
13
13
  export declare const BROADSIDE_CATALOG_CACHE_FILE = "model-catalog.json";
14
+ /**
15
+ * What this repository's own submits learned about batch endpoints: which
16
+ * `:batch` ids OpenRouter accepted a job for and which it refused with
17
+ * "does not have a :batch endpoint". The catalog cannot tell the two apart
18
+ * (#141), so the `models` action annotates its rows from this file.
19
+ */
20
+ export declare const BROADSIDE_ENDPOINTS_FILE = "batch-endpoints.json";
14
21
  export declare const BROADSIDE_CATALOG_CACHE_TTL_MS: number;
15
22
  export declare const BROADSIDE_LENS_IDS: readonly ["architecture", "api", "security", "defect", "conventions", "porting"];
16
23
  export type BroadsideLensId = (typeof BROADSIDE_LENS_IDS)[number];
@@ -539,6 +546,26 @@ export declare function loadBroadsideConfig(broadsideDir: string): Promise<Broad
539
546
  export declare function defaultBroadsideConfig(): BroadsideConfig;
540
547
  /** The catalog cache schema this build writes; a file from another is not read. */
541
548
  export declare const BROADSIDE_CATALOG_CACHE_SCHEMA = 3;
549
+ /** One model's most recent submit outcome, as remembered in {@link BROADSIDE_ENDPOINTS_FILE}. */
550
+ export type BatchEndpointRecord = {
551
+ status: "accepted" | "rejected";
552
+ /** ISO timestamp of the submit that produced this record. */
553
+ at: string;
554
+ /** The provider's refusal, for a rejected endpoint. */
555
+ error?: string;
556
+ };
557
+ export declare function readBatchEndpoints(broadsideDir: string): Promise<Record<string, BatchEndpointRecord>>;
558
+ /**
559
+ * Remember what a submit learned about each model it posted to. An accepted
560
+ * job proves the endpoint exists; a "does not have a :batch endpoint"
561
+ * refusal proves it does not. Any other rejection (quota, malformed request,
562
+ * auth) says nothing about the endpoint and leaves the record alone.
563
+ */
564
+ export declare function recordBatchEndpoints(broadsideDir: string, outcomes: Array<{
565
+ model: string;
566
+ batchId: string;
567
+ error?: unknown;
568
+ }>): Promise<void>;
542
569
  export declare function builtInCatalogEntry(model: string): CatalogEntry | null;
543
570
  export declare function builtInPricing(model: string): ModelPricing | null;
544
571
  export declare function resolveCatalogEntry(broadsideDir: string, config: BroadsideConfig, model: string, apiKey: string, fetcher?: FetchLike): Promise<BroadsideCatalogResult>;
@@ -552,6 +579,8 @@ export declare function listBatchModels(broadsideDir: string, config: BroadsideC
552
579
  source: string;
553
580
  benchmarks: CodingBenchmarks | null;
554
581
  defaultModel: string;
582
+ /** This repository's remembered submit outcomes per model, from {@link BROADSIDE_ENDPOINTS_FILE}. */
583
+ endpoints: Record<string, BatchEndpointRecord>;
555
584
  }>;
556
585
  export type FetchLike = (url: string, init: Record<string, unknown>) => Promise<Response>;
557
586
  export declare function submitBatch(batchRequests: BatchRequest[], apiKey: string, fetcher?: FetchLike, model?: string): Promise<{
@@ -602,6 +631,12 @@ export declare function runBroadsideSubmit(cwd: string, apiKey: string, opts?: {
602
631
  lenses?: BroadsideLensId[];
603
632
  fetcher?: FetchLike;
604
633
  model?: string;
634
+ /**
635
+ * Per-lens model overrides for this run, layered over config.yaml's
636
+ * `lens_models`: a lens named here runs on this model, a lens named only
637
+ * in the file runs on the file's, and the rest run on `model` (#141).
638
+ */
639
+ lensModels?: Partial<Record<BroadsideLensId, string>>;
605
640
  /** Approximate run expense limit in USD; 0 means no limit. */
606
641
  maxCost?: number;
607
642
  /** Submit even when the estimate exceeds maxCost. */
@@ -673,7 +708,21 @@ export declare function estimateSubmitText(result: BroadsideSubmitResult, lenses
673
708
  export declare function modelsText(entries: CatalogEntry[], opts: {
674
709
  benchmarks: CodingBenchmarks | null;
675
710
  defaultModel: string;
711
+ endpoints?: Record<string, BatchEndpointRecord>;
676
712
  }): string;
713
+ /**
714
+ * A provider refusal plus what to do about it, for the two refusals a batch
715
+ * run meets in practice and cannot fix by itself (#141):
716
+ *
717
+ * - `Model '<id>' does not have a :batch endpoint.` — the catalog advertises a
718
+ * `:batch` id that OpenRouter runs no batch endpoint for. Nothing in the
719
+ * catalog distinguishes these; the `models` action marks ids this
720
+ * repository has seen refused.
721
+ * - `job-submission-count … in use: 16, quota: 16` — the per-account limit
722
+ * on concurrent batch jobs. Broad-Side submits one job per lens, so a few
723
+ * runs in flight on the same key fill it; the refusal costs nothing.
724
+ */
725
+ export declare function explainBatchError(error: unknown): string | null;
677
726
  export declare function collectResultText(result: BroadsideCollectResult): string;
678
727
  export declare function statusText(state: BroadsideStateFile): string;
679
728
  export {};
@@ -65,6 +65,13 @@ export const BROADSIDE_OUTPUT_PRICE_PER_M = 1.875;
65
65
  export const BROADSIDE_MODELS_URL = "https://openrouter.ai/api/v1/models";
66
66
  export const BROADSIDE_BENCHMARKS_URL = "https://openrouter.ai/api/v1/benchmarks";
67
67
  export const BROADSIDE_CATALOG_CACHE_FILE = "model-catalog.json";
68
+ /**
69
+ * What this repository's own submits learned about batch endpoints: which
70
+ * `:batch` ids OpenRouter accepted a job for and which it refused with
71
+ * "does not have a :batch endpoint". The catalog cannot tell the two apart
72
+ * (#141), so the `models` action annotates its rows from this file.
73
+ */
74
+ export const BROADSIDE_ENDPOINTS_FILE = "batch-endpoints.json";
68
75
  export const BROADSIDE_CATALOG_CACHE_TTL_MS = 24 * 60 * 60 * 1000;
69
76
  export const BROADSIDE_LENS_IDS = [
70
77
  "architecture",
@@ -1621,6 +1628,67 @@ async function writeCatalogCache(broadsideDir, cache) {
1621
1628
  await mkdir(broadsideDir, { recursive: true });
1622
1629
  await writeFile(join(broadsideDir, BROADSIDE_CATALOG_CACHE_FILE), `${JSON.stringify(cache, null, "\t")}\n`, "utf8");
1623
1630
  }
1631
+ const BROADSIDE_ENDPOINTS_SCHEMA = 1;
1632
+ export async function readBatchEndpoints(broadsideDir) {
1633
+ const path = join(broadsideDir, BROADSIDE_ENDPOINTS_FILE);
1634
+ if (!(await pathExists(path)))
1635
+ return {};
1636
+ try {
1637
+ const parsed = JSON.parse(await readFile(path, "utf8"));
1638
+ if (!parsed || typeof parsed !== "object" || parsed.schema_version !== BROADSIDE_ENDPOINTS_SCHEMA)
1639
+ return {};
1640
+ if (!parsed.models || typeof parsed.models !== "object")
1641
+ return {};
1642
+ const out = {};
1643
+ for (const [model, record] of Object.entries(parsed.models)) {
1644
+ if (!record || typeof record !== "object")
1645
+ continue;
1646
+ if (record.status !== "accepted" && record.status !== "rejected")
1647
+ continue;
1648
+ if (typeof record.at !== "string")
1649
+ continue;
1650
+ out[model] = { status: record.status, at: record.at, ...(typeof record.error === "string" && { error: record.error }) };
1651
+ }
1652
+ return out;
1653
+ }
1654
+ catch {
1655
+ // An unreadable memory is an empty one: it only annotates a listing.
1656
+ return {};
1657
+ }
1658
+ }
1659
+ /**
1660
+ * The refusal OpenRouter returns for a catalog id that has no batch endpoint
1661
+ * behind it. Matched loosely: the message is the only signal there is.
1662
+ */
1663
+ const NO_BATCH_ENDPOINT_RE = /does not have a :batch endpoint/i;
1664
+ /** The refusal for a full per-account concurrent batch-job quota. */
1665
+ const BATCH_QUOTA_RE = /job-submission-count/i;
1666
+ /**
1667
+ * Remember what a submit learned about each model it posted to. An accepted
1668
+ * job proves the endpoint exists; a "does not have a :batch endpoint"
1669
+ * refusal proves it does not. Any other rejection (quota, malformed request,
1670
+ * auth) says nothing about the endpoint and leaves the record alone.
1671
+ */
1672
+ export async function recordBatchEndpoints(broadsideDir, outcomes) {
1673
+ const at = new Date().toISOString();
1674
+ const updates = {};
1675
+ for (const { model, batchId, error } of outcomes) {
1676
+ if (batchId) {
1677
+ updates[model] = { status: "accepted", at };
1678
+ continue;
1679
+ }
1680
+ const message = describeBatchError(error);
1681
+ if (message && NO_BATCH_ENDPOINT_RE.test(message)) {
1682
+ updates[model] = { status: "rejected", at, error: message };
1683
+ }
1684
+ }
1685
+ if (Object.keys(updates).length === 0)
1686
+ return;
1687
+ const models = { ...(await readBatchEndpoints(broadsideDir)), ...updates };
1688
+ await mkdir(broadsideDir, { recursive: true });
1689
+ const file = { schema_version: BROADSIDE_ENDPOINTS_SCHEMA, models };
1690
+ await atomicWriteFile(join(broadsideDir, BROADSIDE_ENDPOINTS_FILE), `${JSON.stringify(file, null, "\t")}\n`);
1691
+ }
1624
1692
  function parseCatalogEntry(raw) {
1625
1693
  const id = String(raw.id ?? "");
1626
1694
  if (!id)
@@ -1843,7 +1911,8 @@ export async function listBatchModels(broadsideDir, config, apiKey, opts = {}) {
1843
1911
  cache.models[entry.id] = { ...entry, fetched_at: fetchedAt };
1844
1912
  await writeCatalogCache(broadsideDir, cache);
1845
1913
  const benchmarks = opts.includeBenchmarks ? await fetchCodingBenchmarks(apiKey, fetcher) : null;
1846
- return { entries, source: "live", benchmarks, defaultModel: config.model };
1914
+ const endpoints = await readBatchEndpoints(broadsideDir);
1915
+ return { entries, source: "live", benchmarks, defaultModel: config.model, endpoints };
1847
1916
  }
1848
1917
  export async function submitBatch(batchRequests, apiKey, fetcher = fetch, model = BROADSIDE_MODEL) {
1849
1918
  // The OpenRouter batch endpoint stream-parses the body and requires
@@ -2001,7 +2070,8 @@ export async function runBroadsideSubmit(cwd, apiKey, opts = {}) {
2001
2070
  throw new Error(`Broad-Side found no ${info.language} source files to scan (detected from ${info.manifest?.path ?? "the file counts"}; ` +
2002
2071
  `the lenses look for ${info.sourceExts.join(", ")}). Nothing was submitted.`);
2003
2072
  }
2004
- const modelForLens = (lensId) => config.lensModels[lensId] ?? model;
2073
+ const lensModels = { ...config.lensModels, ...opts.lensModels };
2074
+ const modelForLens = (lensId) => lensModels[lensId] ?? model;
2005
2075
  const resolved = new Map();
2006
2076
  for (const candidate of new Set([model, ...lensIds.map(modelForLens)])) {
2007
2077
  const catalog = await resolveCatalogEntry(broadsideDir, config, candidate, apiKey, opts.fetcher);
@@ -2215,6 +2285,12 @@ export async function runBroadsideSubmit(cwd, apiKey, opts = {}) {
2215
2285
  }
2216
2286
  await Promise.allSettled(submissions);
2217
2287
  await persistBroadsideRun(broadsideDir, run);
2288
+ // What the provider just said about each model's batch endpoint outlives
2289
+ // the run: the `models` action reads it back (#141).
2290
+ await recordBatchEndpoints(broadsideDir, lensIds
2291
+ .map((lensId) => run.batches[lensId])
2292
+ .filter((entry) => Boolean(entry) && entry.status !== "skipped")
2293
+ .map((entry) => ({ model: entry.model ?? model, batchId: entry.batchId, error: entry.error })));
2218
2294
  // Persist the exact request bodies so collect can re-submit a truncated
2219
2295
  // slice (bumped output cap) without re-walking the repo (#133). The run
2220
2296
  // dir is created here rather than waiting for collect so a crash between
@@ -2520,7 +2596,19 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
2520
2596
  truncatedCount += truncated;
2521
2597
  totalCost += cost ?? 0;
2522
2598
  await writeFile(join(runDir, `raw-${lensId}.json`), `${JSON.stringify(batch, null, "\t")}\n`, "utf8");
2523
- lensOutcomes[lensId] = { status, cost: entry.cost, resultCount: entry.resultCount, truncated };
2599
+ // A batch can complete with every request failed — the account's
2600
+ // concurrent-job quota filling after acceptance does exactly this.
2601
+ // The per-request errors are on disk as `<id>.error.json`, but a
2602
+ // lens reporting "completed, 0 result(s)" with the reason buried
2603
+ // there read as an empty repository rather than a refused run.
2604
+ const results = Array.isArray(batch.results) ? batch.results : [];
2605
+ const failed = results.filter((r) => r.error && extractContent(r) === null);
2606
+ const allFailed = stored.length === 0 && failed.length > 0
2607
+ ? `all ${failed.length} request(s) failed: ${explainBatchError(failed[0].error)}`
2608
+ : null;
2609
+ if (allFailed)
2610
+ entry.error = allFailed;
2611
+ lensOutcomes[lensId] = { status, cost: entry.cost, resultCount: entry.resultCount, truncated, ...(allFailed && { error: allFailed }) };
2524
2612
  }
2525
2613
  else {
2526
2614
  // Every non-completed outcome still has to reach the report.
@@ -2532,7 +2620,7 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
2532
2620
  // indistinguishable in the output from one that was never requested.
2533
2621
  if (batch.error)
2534
2622
  entry.error = batch.error;
2535
- const error = describeBatchError(batch.error);
2623
+ const error = explainBatchError(batch.error);
2536
2624
  lensOutcomes[lensId] = { status, cost: entry.cost, resultCount: entry.resultCount, ...(error && { error }) };
2537
2625
  }
2538
2626
  await persistBroadsideRun(broadsideDir, run);
@@ -2540,9 +2628,19 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
2540
2628
  // #133: re-submit truncated slices once with a bumped output cap. Batch
2541
2629
  // requests are pure, so re-running is always safe; the aim is to recover
2542
2630
  // coverage the first pass lost to a max_tokens cutoff, not to loop forever.
2631
+ //
2632
+ // All bumped requests for one model go out as ONE batch, and the batches
2633
+ // (one per model, since a batch carries a single model) are polled
2634
+ // together against the shared deadline. Each truncated slice used to be
2635
+ // submitted and polled to terminal before the next was submitted, so a
2636
+ // model that truncated 11 of 13 slices turned a five-minute collect into
2637
+ // eleven sequential round trips — the serialization #136 removed from the
2638
+ // lens pass, still present here (#206). Grouping also keeps the retry to
2639
+ // one job per model against OpenRouter's 16-concurrent-job quota.
2543
2640
  let retriedCount = 0;
2544
2641
  if (opts.retryTruncated !== false && truncatedCount > 0) {
2545
2642
  const requestsByCustomId = await loadStoredRequests(runDir);
2643
+ const byModel = new Map();
2546
2644
  for (const stored of allLensResults) {
2547
2645
  if (!stored.truncated)
2548
2646
  continue;
@@ -2559,42 +2657,57 @@ export async function runBroadsideCollect(cwd, apiKey, opts = {}) {
2559
2657
  const bumpedMax = lensCap ? Math.min(previousMax * 2, lensCap) : previousMax * 2;
2560
2658
  if (bumpedMax <= previousMax)
2561
2659
  continue; // already at the ceiling
2562
- const bumped = {
2563
- ...original,
2564
- body: { ...original.body, max_tokens: bumpedMax },
2565
- };
2660
+ const group = byModel.get(lensModel) ?? { requests: [], slices: new Map() };
2661
+ group.requests.push({ ...original, body: { ...original.body, max_tokens: bumpedMax } });
2662
+ group.slices.set(stored.customId, stored);
2663
+ byModel.set(lensModel, group);
2664
+ }
2665
+ // Submit every group, then poll whatever was accepted, together.
2666
+ const submitted = [];
2667
+ for (const [model, group] of byModel) {
2566
2668
  try {
2567
- const { batchId, error } = await submitBatch([bumped], apiKey, opts.fetcher, lensModel);
2568
- if (error)
2669
+ const { batchId, error } = await submitBatch(group.requests, apiKey, opts.fetcher, model);
2670
+ if (!error && batchId)
2671
+ submitted.push({ model, batchId });
2672
+ }
2673
+ catch {
2674
+ // A retry batch that fails to submit leaves its slices' original
2675
+ // truncated results in place — nothing is lost.
2676
+ }
2677
+ }
2678
+ const polled = await pollBatchesConcurrently(submitted.map(({ model, batchId }) => ({ lensId: `retry:${model}`, batchId })), apiKey, {
2679
+ // Share the caller's deadline. Each of these polls used to start a
2680
+ // fresh 25-minute budget, so `wait_seconds` bounded only the lens
2681
+ // poll and a collect could run for the caller's budget plus fifty
2682
+ // minutes.
2683
+ deadlineMs: Math.max(0, deadline - Date.now()),
2684
+ fetcher: opts.fetcher,
2685
+ onStatus: opts.onStatus,
2686
+ });
2687
+ for (const { model, batchId } of submitted) {
2688
+ const batch = polled.get(batchId);
2689
+ if (!batch || batch.status !== "completed")
2690
+ continue;
2691
+ const group = byModel.get(model);
2692
+ const usage = (batch.usage ?? {});
2693
+ totalCost += typeof usage.cost === "number" ? usage.cost : 0;
2694
+ const results = Array.isArray(batch.results) ? batch.results : [];
2695
+ for (const result of results) {
2696
+ const stored = group.slices.get(String(result.custom_id ?? ""));
2697
+ if (!stored)
2569
2698
  continue;
2570
- const batch = await pollBatchUntilTerminal(batchId, apiKey, {
2571
- // Share the caller's deadline. Each of these polls used to
2572
- // start a fresh 25-minute budget, so `wait_seconds` bounded
2573
- // only the lens poll and a collect could run for the caller's
2574
- // budget plus fifty minutes.
2575
- deadlineMs: Math.max(0, deadline - Date.now()),
2576
- onStatus: (status, counts) => opts.onStatus?.(`${stored.lensId}:retry`, status, counts),
2577
- fetcher: opts.fetcher,
2578
- });
2579
- if (batch.status !== "completed")
2699
+ const content = extractContent(result);
2700
+ if (content === null)
2580
2701
  continue;
2581
- const results = Array.isArray(batch.results) ? batch.results : [];
2582
- const content = results.length > 0 ? extractContent(results[0]) : null;
2583
- if (content === null || parseLensJson(content) === null)
2584
- continue; // still no good
2585
- const usage = (batch.usage ?? {});
2586
- totalCost += typeof usage.cost === "number" ? usage.cost : 0;
2587
2702
  const parsed = parseLensJson(content);
2703
+ if (parsed === null)
2704
+ continue; // still no good
2588
2705
  await writeFile(join(runDir, `${sanitizeId(stored.customId)}.json`), `${JSON.stringify(parsed, null, "\t")}\n`, "utf8");
2589
2706
  await writeFile(join(runDir, `${sanitizeId(stored.customId)}.md`), renderFindingsMarkdown(content), "utf8");
2590
2707
  stored.content = content;
2591
2708
  stored.truncated = false;
2592
2709
  retriedCount += 1;
2593
2710
  }
2594
- catch {
2595
- // A retry that fails to submit/poll leaves the original
2596
- // truncated result in place — nothing is lost.
2597
- }
2598
2711
  }
2599
2712
  truncatedCount = allLensResults.filter((s) => s.truncated).length;
2600
2713
  for (const [lensId, outcome] of Object.entries(lensOutcomes)) {
@@ -2878,7 +2991,11 @@ export function estimateSubmitText(result, lenses) {
2878
2991
  continue;
2879
2992
  const status = entry.batchId ? `batch ${entry.batchId}` : entry.status;
2880
2993
  const override = entry.model ? ` on ${entry.model}` : "";
2881
- lines.push(` ${lens.name}: ${status} (${entry.requests} request(s), ~$${entry.estimatedCost.toFixed(4)})${override}`);
2994
+ // A rejected lens says why: the message is the only way to tell a
2995
+ // catalog id with no batch endpoint from a full job quota, and both
2996
+ // used to read as a bare "rejected".
2997
+ const reason = !entry.batchId && entry.error ? ` — ${explainBatchError(entry.error)}` : "";
2998
+ lines.push(` ${lens.name}: ${status} (${entry.requests} request(s), ~$${entry.estimatedCost.toFixed(4)})${override}${reason}`);
2882
2999
  }
2883
3000
  if (result.repo) {
2884
3001
  const head = result.repo.sourceHead ? ` at ${result.repo.sourceHead.slice(0, 8)}${result.repo.sourceDirty ? " (dirty)" : ""}` : "";
@@ -2915,8 +3032,15 @@ export function estimateSubmitText(result, lenses) {
2915
3032
  return lines.join("\n");
2916
3033
  }
2917
3034
  export function modelsText(entries, opts) {
3035
+ const endpoints = opts.endpoints ?? {};
2918
3036
  const lines = [
2919
3037
  `Batch models on OpenRouter (${entries.length}, cheapest first).`,
3038
+ // The catalog over-reports: it returns a `:batch` id for models whose
3039
+ // Batch API refuses the job, with nothing in the entry to tell them
3040
+ // apart (#141). Say so before the table, not after it.
3041
+ "Advisory: this is the catalog's list of :batch ids, not a list of working batch endpoints. Some ids are refused at submit " +
3042
+ "(\"does not have a :batch endpoint\"), at no cost. Rows tagged [no batch endpoint …] or [batch OK …] carry what this " +
3043
+ "repository's own submits found; an untagged row has not been tried here.",
2920
3044
  "",
2921
3045
  "id | $/M in | $/M out | ctx | max out | structured | coding idx",
2922
3046
  ];
@@ -2936,12 +3060,21 @@ export function modelsText(entries, opts) {
2936
3060
  const out = entry.maxCompletionTokens ? `${(entry.maxCompletionTokens / 1024).toFixed(0)}k` : "?";
2937
3061
  const tag = entry.id === opts.defaultModel ? " (default)" : "";
2938
3062
  const exp = entry.expirationDate ? " [deprecated]" : "";
2939
- lines.push(`${entry.id}${tag}${exp} | ${entry.inputPerM.toFixed(3)} | ${entry.outputPerM.toFixed(3)} | ${ctx} | ${out} | ${structured} | ${coding}`);
3063
+ const record = endpoints[entry.id];
3064
+ const seen = record
3065
+ ? record.status === "rejected"
3066
+ ? ` [no batch endpoint, refused ${record.at.slice(0, 10)}]`
3067
+ : ` [batch OK ${record.at.slice(0, 10)}]`
3068
+ : "";
3069
+ lines.push(`${entry.id}${tag}${exp}${seen} | ${entry.inputPerM.toFixed(3)} | ${entry.outputPerM.toFixed(3)} | ${ctx} | ${out} | ${structured} | ${coding}`);
2940
3070
  }
2941
3071
  if (opts.benchmarks?.meta.as_of) {
2942
3072
  lines.push("", `Benchmarks: Artificial Analysis coding index (as of ${String(opts.benchmarks.meta.as_of)}).`);
2943
3073
  }
2944
- lines.push("", "Set the batch model in .codecarto/broadside/config.yaml (model key). Higher coding index ≠ better scout: precision, context, and structured-output support matter most here.");
3074
+ lines.push("", "Choose with the model parameter (--model= on Pi) for one run, lens_models (--lens-model=LENS:ID) per lens, or the model key in " +
3075
+ ".codecarto/broadside/config.yaml for the repository. Higher coding index ≠ better scout: precision, context, structured-output " +
3076
+ "support, and whether the model spends its output budget reasoning (see reasoning: in config.yaml) matter most here. " +
3077
+ "A refused submit costs nothing, so probe an untried model on one lens first.");
2945
3078
  return lines.join("\n");
2946
3079
  }
2947
3080
  /** One line of a batch's error field, whatever shape the provider gave it. */
@@ -2954,6 +3087,15 @@ function describeBatchError(error) {
2954
3087
  const message = error.message;
2955
3088
  if (typeof message === "string" && message)
2956
3089
  return message.slice(0, 300);
3090
+ // OpenRouter wraps a submit refusal as `{ error: { message } }`.
3091
+ const nested = error.error;
3092
+ if (nested && typeof nested === "object") {
3093
+ const inner = nested.message;
3094
+ if (typeof inner === "string" && inner)
3095
+ return inner.slice(0, 300);
3096
+ }
3097
+ if (typeof nested === "string" && nested)
3098
+ return nested.slice(0, 300);
2957
3099
  try {
2958
3100
  return JSON.stringify(error).slice(0, 300);
2959
3101
  }
@@ -2963,6 +3105,30 @@ function describeBatchError(error) {
2963
3105
  }
2964
3106
  return String(error);
2965
3107
  }
3108
+ /**
3109
+ * A provider refusal plus what to do about it, for the two refusals a batch
3110
+ * run meets in practice and cannot fix by itself (#141):
3111
+ *
3112
+ * - `Model '<id>' does not have a :batch endpoint.` — the catalog advertises a
3113
+ * `:batch` id that OpenRouter runs no batch endpoint for. Nothing in the
3114
+ * catalog distinguishes these; the `models` action marks ids this
3115
+ * repository has seen refused.
3116
+ * - `job-submission-count … in use: 16, quota: 16` — the per-account limit
3117
+ * on concurrent batch jobs. Broad-Side submits one job per lens, so a few
3118
+ * runs in flight on the same key fill it; the refusal costs nothing.
3119
+ */
3120
+ export function explainBatchError(error) {
3121
+ const message = describeBatchError(error);
3122
+ if (!message)
3123
+ return null;
3124
+ if (NO_BATCH_ENDPOINT_RE.test(message)) {
3125
+ return `${message} — the catalog lists this id, but OpenRouter runs no batch endpoint for it. Nothing was charged; pick another model (the models action marks ids this repository has seen refused).`;
3126
+ }
3127
+ if (BATCH_QUOTA_RE.test(message)) {
3128
+ return `${message} — OpenRouter's per-account limit on concurrent batch jobs is full. Broad-Side submits one job per lens, so a few runs in flight on this key (in any repository) fill it. Nothing was charged; collect or wait out the runs in flight, then re-submit.`;
3129
+ }
3130
+ return message;
3131
+ }
2966
3132
  export function collectResultText(result) {
2967
3133
  const lines = [
2968
3134
  `Broad-Side run ${result.runId}: ${result.status}`,
@@ -18,11 +18,15 @@ export interface BroadsideFlags {
18
18
  waitSeconds?: number;
19
19
  /** For collect: the run to collect instead of the most recent (#268). */
20
20
  runId?: string;
21
+ /** For submit: the run's batch model, replacing config.yaml's (#141). */
22
+ model?: string;
23
+ /** For submit: per-lens model overrides, layered over config.yaml's (#141). */
24
+ lensModels?: Partial<Record<BroadsideLensId, string>>;
21
25
  benchmarks: boolean;
22
26
  unknown: string[];
23
27
  /** Set on an invalid combination. The caller surfaces it as an error. */
24
28
  error?: string;
25
29
  }
26
30
  /** Every token the completer offers, in the order it offers them. */
27
- export declare const KNOWN_BROADSIDE_TOKENS: readonly ["submit", "collect", "status", "models", "architecture", "api", "security", "defect", "conventions", "porting", "--incremental", "--no-incremental", "--max-cost=", "--wait=", "--run=", "--no-synthesis", "--no-triage", "--no-retry-truncated", "--benchmarks"];
31
+ export declare const KNOWN_BROADSIDE_TOKENS: readonly ["submit", "collect", "status", "models", "architecture", "api", "security", "defect", "conventions", "porting", "--incremental", "--no-incremental", "--max-cost=", "--wait=", "--run=", "--model=", "--lens-model=", "--no-synthesis", "--no-triage", "--no-retry-truncated", "--benchmarks"];
28
32
  export declare function parseBroadsideFlags(args: string): BroadsideFlags;
@@ -14,6 +14,11 @@
14
14
  // --max-cost=N --no-retry-truncated
15
15
  // --wait=SECONDS --benchmarks (models only)
16
16
  // --run=ID (collect only: an older run, as listed by status)
17
+ // --model=ID (submit only: the run's batch model, as listed by models)
18
+ // --lens-model=LENS:ID (submit only, repeatable: one lens on its own model)
19
+ //
20
+ // A model id itself contains a colon (`vendor/name:batch`), so --lens-model
21
+ // splits on the first colon only: `security:deepseek/deepseek-v4-pro:batch`.
17
22
  //
18
23
  // --incremental has a spelled-out negative because the value is tri-state:
19
24
  // absent defers to config.yaml, so a repository that set `incremental: true`
@@ -36,6 +41,8 @@ export const KNOWN_BROADSIDE_TOKENS = [
36
41
  "--max-cost=",
37
42
  "--wait=",
38
43
  "--run=",
44
+ "--model=",
45
+ "--lens-model=",
39
46
  "--no-synthesis",
40
47
  "--no-triage",
41
48
  "--no-retry-truncated",
@@ -115,6 +122,32 @@ export function parseBroadsideFlags(args) {
115
122
  result.runId = value || undefined;
116
123
  continue;
117
124
  }
125
+ if (token.startsWith("--model=")) {
126
+ const value = token.slice("--model=".length).trim();
127
+ // An empty value is a mistyped selection, not "use the default":
128
+ // the command is about to spend money on whichever model wins.
129
+ if (!value)
130
+ result.error ??= "--model= needs an OpenRouter batch model id (see /codecarto-broadside models).";
131
+ result.model = value || undefined;
132
+ continue;
133
+ }
134
+ if (token.startsWith("--lens-model=")) {
135
+ const value = token.slice("--lens-model=".length).trim();
136
+ const colon = value.indexOf(":");
137
+ const lensId = colon > 0 ? value.slice(0, colon).trim() : "";
138
+ const modelId = colon > 0 ? value.slice(colon + 1).trim() : "";
139
+ if (!lensId || !modelId) {
140
+ result.error ??= `--lens-model needs LENS:MODEL, e.g. --lens-model=security:vendor/name:batch (got "${value}").`;
141
+ }
142
+ else if (!BROADSIDE_LENS_IDS.includes(lensId)) {
143
+ result.error ??= `--lens-model: unknown lens "${lensId}". Lenses: ${BROADSIDE_LENS_IDS.join(", ")}.`;
144
+ }
145
+ else {
146
+ result.lensModels ??= {};
147
+ result.lensModels[lensId] = modelId;
148
+ }
149
+ continue;
150
+ }
118
151
  result.unknown.push(token);
119
152
  }
120
153
  // Flags that only mean something for one action are refused rather than
@@ -137,5 +170,11 @@ export function parseBroadsideFlags(args) {
137
170
  if (result.runId !== undefined && result.action !== "collect") {
138
171
  result.error ??= `--run is only meaningful for collect (got action "${result.action}").`;
139
172
  }
173
+ if (result.model !== undefined && result.action !== "submit") {
174
+ result.error ??= `--model is only meaningful for submit (got action "${result.action}").`;
175
+ }
176
+ if (result.lensModels !== undefined && result.action !== "submit") {
177
+ result.error ??= `--lens-model is only meaningful for submit (got action "${result.action}").`;
178
+ }
140
179
  return result;
141
180
  }
@@ -1000,7 +1000,7 @@ export default function codeCartographerExtension(pi) {
1000
1000
  },
1001
1001
  });
1002
1002
  pi.registerCommand("codecarto-broadside", {
1003
- description: "Batch reconnaissance (Broad-Side): /codecarto-broadside [submit|collect|status|models] [lenses…] [flags]",
1003
+ description: "Batch reconnaissance (Broad-Side): /codecarto-broadside [submit|collect|status|models] [lenses…] [--model=ID] [--lens-model=LENS:ID] [flags]",
1004
1004
  getArgumentCompletions: (prefix) => {
1005
1005
  const items = KNOWN_BROADSIDE_TOKENS
1006
1006
  .filter((value) => value.startsWith(prefix))
@@ -1077,10 +1077,10 @@ export default function codeCartographerExtension(pi) {
1077
1077
  if (flags.action === "models") {
1078
1078
  notifyCtx(ctx, "Fetching the OpenRouter batch-model catalog…", "info");
1079
1079
  try {
1080
- const { entries, benchmarks } = await listBatchModels(broadsideDir, config, apiKey, {
1080
+ const { entries, benchmarks, endpoints } = await listBatchModels(broadsideDir, config, apiKey, {
1081
1081
  includeBenchmarks: flags.benchmarks,
1082
1082
  });
1083
- finish(modelsText(entries, { benchmarks, defaultModel: config.model }).split("\n"), `Broad-Side: ${entries.length} batch model${entries.length === 1 ? "" : "s"} listed`);
1083
+ finish(modelsText(entries, { benchmarks, defaultModel: config.model, endpoints }).split("\n"), `Broad-Side: ${entries.length} batch model${entries.length === 1 ? "" : "s"} listed`);
1084
1084
  }
1085
1085
  catch (error) {
1086
1086
  notifyCtx(ctx, `Model catalog lookup failed: ${error instanceof Error ? error.message : String(error)}`, "error");
@@ -1117,7 +1117,11 @@ export default function codeCartographerExtension(pi) {
1117
1117
  try {
1118
1118
  submit = await runBroadsideSubmit(ctx.cwd, apiKey, {
1119
1119
  lenses,
1120
- model: config.model,
1120
+ // --model= and --lens-model= select for this run; the file's
1121
+ // values are the fallback, and core pre-flights either the
1122
+ // same way (#141).
1123
+ model: flags.model ?? config.model,
1124
+ lensModels: flags.lensModels,
1121
1125
  maxCost: flags.maxCost ?? config.maxCost,
1122
1126
  // `??`, not `||`: --no-incremental parses to false and must beat a
1123
1127
  // config-set true, exactly as MCP's `incremental: false` does (#163).
@@ -245,6 +245,8 @@ export declare function handleBroadside(args: {
245
245
  force?: boolean;
246
246
  include_benchmarks?: boolean;
247
247
  incremental?: boolean;
248
+ model?: string;
249
+ lens_models?: Record<string, string>;
248
250
  }): Promise<{
249
251
  content: {
250
252
  type: "text";
@@ -1079,15 +1079,19 @@ export async function handleBroadside(args) {
1079
1079
  const retryTruncated = args.retry_truncated ?? config.retryTruncated;
1080
1080
  const incremental = args.incremental ?? config.incremental;
1081
1081
  if (action === "models") {
1082
- const { entries, benchmarks } = await listBatchModels(broadsideDirFor(cwd), config, apiKey, {
1082
+ const { entries, benchmarks, endpoints } = await listBatchModels(broadsideDirFor(cwd), config, apiKey, {
1083
1083
  includeBenchmarks: args.include_benchmarks === true,
1084
1084
  }).catch((error) => {
1085
1085
  throw new McpError(ErrorCode.InvalidRequest, error instanceof Error ? error.message : String(error));
1086
1086
  });
1087
- return textResult(modelsText(entries, { benchmarks, defaultModel: config.model }), {
1087
+ return textResult(modelsText(entries, { benchmarks, defaultModel: config.model, endpoints }), {
1088
1088
  models: entries,
1089
1089
  defaultModel: config.model,
1090
1090
  benchmarkMeta: benchmarks?.meta ?? null,
1091
+ // The catalog is advisory (#141): what this repository's submits
1092
+ // learned about each id's batch endpoint rides alongside it.
1093
+ catalogAdvisory: true,
1094
+ endpoints,
1091
1095
  });
1092
1096
  }
1093
1097
  if (action === "submit") {
@@ -1105,9 +1109,34 @@ export async function handleBroadside(args) {
1105
1109
  // An explicit 0 is "no limit" (#231); absent falls back to config.yaml,
1106
1110
  // whose own default is BROADSIDE_DEFAULT_MAX_COST.
1107
1111
  const maxCost = typeof args.max_cost === "number" && args.max_cost >= 0 ? args.max_cost : config.maxCost;
1112
+ // Model selection for one run (#141): `model` replaces the run default,
1113
+ // `lens_models` layers per-lens overrides over config.yaml's. Both are
1114
+ // pre-flighted by core exactly like the file's values — priced from the
1115
+ // catalog, refused without structured-output support, clamped to the
1116
+ // model's ceiling — so a wrong id fails before anything is submitted.
1117
+ const model = typeof args.model === "string" && args.model.trim() ? args.model.trim() : config.model;
1118
+ if (args.model !== undefined && !(typeof args.model === "string" && args.model.trim())) {
1119
+ throw new McpError(ErrorCode.InvalidParams, "model must be a non-empty OpenRouter batch model id (see action 'models').");
1120
+ }
1121
+ const lensModels = {};
1122
+ if (args.lens_models !== undefined) {
1123
+ if (!args.lens_models || typeof args.lens_models !== "object" || Array.isArray(args.lens_models)) {
1124
+ throw new McpError(ErrorCode.InvalidParams, "lens_models must be an object mapping lens ids to batch model ids.");
1125
+ }
1126
+ for (const [lensId, value] of Object.entries(args.lens_models)) {
1127
+ if (!BROADSIDE_LENS_IDS.includes(lensId)) {
1128
+ throw new McpError(ErrorCode.InvalidParams, `lens_models: unknown lens "${lensId}". Valid: ${BROADSIDE_LENS_IDS.join(", ")}`);
1129
+ }
1130
+ if (typeof value !== "string" || !value.trim()) {
1131
+ throw new McpError(ErrorCode.InvalidParams, `lens_models.${lensId} must be a non-empty OpenRouter batch model id.`);
1132
+ }
1133
+ lensModels[lensId] = value.trim();
1134
+ }
1135
+ }
1108
1136
  const result = await runBroadsideSubmit(cwd, apiKey, {
1109
1137
  lenses,
1110
- model: config.model,
1138
+ model,
1139
+ lensModels,
1111
1140
  maxCost,
1112
1141
  force: args.force === true,
1113
1142
  incremental,
@@ -1516,6 +1545,15 @@ const TOOLS = [
1516
1545
  type: "boolean",
1517
1546
  description: "For action 'models': annotate each model with its Artificial Analysis coding index (extra API call; default false).",
1518
1547
  },
1548
+ model: {
1549
+ type: "string",
1550
+ description: "For submit: the OpenRouter batch model for this run (an id ending in :batch, as listed by action 'models'). Falls back to model in .codecarto/broadside/config.yaml, then the shipped default. Pre-flighted like the configured model: priced from the live catalog, refused without structured-output support, clamped to its completion ceiling. The models listing is advisory — some catalog ids have no batch endpoint and are refused at submit, at no cost; the listing tags ids this repository has already seen accepted or refused.",
1551
+ },
1552
+ lens_models: {
1553
+ type: "object",
1554
+ additionalProperties: { type: "string" },
1555
+ description: "For submit: per-lens model overrides for this run, e.g. {\"security\": \"deepseek/deepseek-v4-pro-0813:batch\"}. Keys are lens ids; a lens named here runs on that model, others on `model`. Layered over lens_models in .codecarto/broadside/config.yaml (a lens set in both takes the parameter's). Each override is priced, capability-checked, and clamped individually, and the estimate breaks cost out per lens.",
1556
+ },
1519
1557
  },
1520
1558
  required: ["cwd", "action"],
1521
1559
  },
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "codecartographer-pi",
3
- "version": "0.21.0",
3
+ "version": "0.22.0",
4
4
  "mcpName": "io.github.HuginnIndustries/codecartographer",
5
5
  "description": "Turn an unfamiliar codebase into a validated reimplementation spec, then synthesize confirmed specs and a product vision into a traceable plan.",
6
6
  "type": "module",