codecartographer-pi 0.18.0 → 0.19.1

package/.codecarto/GUIDE.md

@@ -215,6 +215,13 @@ post_pipeline:
 
 `kind` is one of: `needs-runtime-test`, `needs-maintainer-decision`, `needs-spec-ruling`, `defer-to-phase`, `needs-fixture-capture`, or a post-pipeline work kind such as `spike` or `amendment`. Every new `carry_forward.target_phase` must be an ID in the active pipeline. Every `post_pipeline` entry requires a stable ID. Open questions should carry a stable `id` (e.g. `q-loadconfig-ambiguity`); if omitted, the framework auto-assigns one. When a later phase resolves an open question, list its id in `open_question_closures` to remove it from all phases. The downstream phase records resolved carry-forward IDs in `carry_forward_closures`; completion removes those entries atomically.
 
+**Closing a routed item does not settle the question it came from.** A phase routinely registers an open question and routes one of its candidate answers onward in the same handoff; addressing the routed item later is not the same as answering the question. Two optional fields make that distinction enforceable:
+
+- A `carry_forward` entry may name `derives_from: <open-question-id>`, meaning "this routed item is one candidate answer to that question." Fill it in the handoff that registers the question, when both entries are in front of you. Completion then refuses a `carry_forward_closures` entry for that item while its question is still open and is not closed by the same handoff, naming both ids. Close the question with evidence in that handoff, or leave the item routed and give the finding an unsettled action (`verify at runtime`). Entirely opt-in: a `carry_forward` entry without `derives_from` closes exactly as it always has.
+- An `open_question_closures` entry may be written as `{ id, evidence }` instead of a bare id, where `evidence` names what settled the question. For a question whose `kind` is `needs-runtime-test`, non-empty `evidence` is **required** — such a question closes on a spike report or an observation against the running system, not on another read of the same source — and that requirement holds however the closure is written, so a bare id no longer closes a runtime question. Questions of every other kind are unaffected, and a bare id stays valid for them. The requirement follows the scaffold: a workspace at scaffold version 0.19.0 or newer has the closure refused, while an older scaffold — which never documented the rule — gets a non-gating note, and refreshing the scaffold opts it in.
+
+Completed phases' declared coverage gaps travel too: the `Skipped scope` and `Known blind spots` bullets of every completed phase's `## Coverage and limits` section are surfaced in the next phase's orchestrator duties. A finding that lands inside one of those gaps must either close it with cited new evidence of its own or inherit its uncertainty.
+
 After the pipeline completes, the handoff channel closes with it. Post-pipeline resolutions — an open question answered on evidence, a finished `post_pipeline` backlog item — are applied with an **amendment**: write `scratch/amendments/<slug>.yaml` (see `templates/amendment.yaml`) and run `codecarto_amend` (MCP) or `/codecarto-amend <slug>` (Pi). It updates `workflow/status.yaml` under the same lock completion uses and writes an amendment closeout plus THREAD_LOG entry. Never hand-edit `status.yaml` for this; amendments are refused while the pipeline is still running, so the two channels cannot race.
 
 ## Phase Selection Logic

package/.codecarto/broadside/SKILL.md

@@ -125,6 +125,13 @@ from OpenRouter's runtime cost tracking: it predicts from file sizes before
 spend, it does not stop a batch mid-flight. Actual spend appears in
 `run-meta.json` after collect.
 
+Budget for the **ceiling, not the estimate**. OpenRouter charges the worst
+case when it accepts a batch — every request's full `max_tokens` — and
+refunds the unused output as each request settles. A six-lens scan of this
+repository held $0.6972 and settled at $0.3806, so the balance a run needs to
+*start* is roughly double what it ends up costing. The submit estimate sits
+between the two: above what the run should settle at, below what it can hold.
+
 ## Resilience notes
 
 - **Truncation is spoken.** A lens output whose JSON does not parse — even

package/.codecarto/broadside/config.yaml

@@ -68,8 +68,8 @@
 # lookup fails (offline, private model) or you want to assert a ceiling.
 #
 # pricing:
-#   input_per_m: 0.1875
-#   output_per_m: 0.9375
+#   input_per_m: 0.375
+#   output_per_m: 1.875
 # ---------------------------------------------------------------------------
 # Run defaults. Each key below mirrors a codecarto_broadside parameter of the
 # same name and sets this repository's default for it; an explicit parameter on

package/.codecarto/templates/phase-handoff.yaml

@@ -4,9 +4,28 @@ schema_version: 1
 phase_id: <phase-id>
 owner_notes: []
 open_questions: []
+# Items a specific later phase closes. Every entry needs a target_phase naming a
+# downstream phase in the active pipeline. Optional derives_from names the
+# open_questions id this item is one candidate answer to:
+#   - id: mech-CF3
+#     kind: defer-to-phase
+#     target_phase: defect-scan-semantic
+#     derives_from: q-logit-bias-root-cause
+#     description: <the routed item>
+# Fill derives_from in the handoff that registers the question — that is the one
+# moment both entries are in front of you. Closing the routed item later does not
+# settle the question it came from, and completion refuses such a closure while
+# the question is still open and unclosed by the same handoff.
 carry_forward: []
 carry_forward_closures: []
-# Resolved open questions: their IDs are removed from all phases.
+# Resolved open questions: their IDs are removed from all phases. Either a bare
+# id, or an object naming the evidence that settled it:
+#   - q-loadconfig-ambiguity
+#   - id: q-logit-bias-root-cause
+#     evidence: <the spike report or runtime observation that answered it>
+# A question of kind needs-runtime-test requires non-empty evidence — it closes
+# on runtime evidence, not on another read of the same source. Refused from
+# scaffold version 0.19.0; older scaffolds get a non-gating note instead.
 open_question_closures: []
 # Work after the active pipeline: spikes, amendments, deltas, maintainer rulings,
 # or opinionated reruns. Every entry requires a stable id.

package/.codecarto/workflow/scaffold-version.yaml

@@ -3,4 +3,4 @@
 # workspace's framework-owned files (GUIDE.md, templates/, workflow/ pipelines
 # and VALIDATE.md) predate the running release. Written at release time and
 # copied verbatim by init — never edit by hand.
-scaffold_version: 0.18.0
+scaffold_version: 0.19.1

package/agent-skill/codecartographer/references/handoff-contract.md

@@ -17,7 +17,7 @@ owner_notes: []                 # 2-3 durable observations; appended to the phas
 open_questions: []              # genuinely unknown, no later phase will close them
 carry_forward: []               # deferred to a specific later phase in this pipeline
 carry_forward_closures: []      # ids of carry_forward entries this phase resolved
-open_question_closures: []      # ids of open questions this phase resolved, removed everywhere
+open_question_closures: []      # open questions this phase resolved, removed everywhere; bare id or {id, evidence}
 post_pipeline: []               # work after the pipeline; every entry needs a stable id
 decisions: []                   # choices made beyond what the prompt specified; completion appends them to DECISIONS.md
 proposed_conventions: []        # patterns proposed for promotion; completion stages them in CONVENTIONS.md
@@ -41,7 +41,7 @@ Omitted arrays default to empty. A malformed collection fails completion without
   deferred_reason: Distinguishing them needs a runtime probe this phase cannot run.
 ```
 
-`carry_forward` entries add `target_phase`:
+`carry_forward` entries add `target_phase`, and optionally `derives_from`:
 
 ```yaml
 - id: arch-CF2
@@ -49,10 +49,18 @@ Omitted arrays default to empty. A malformed collection fails completion without
   target_phase: protocols
   description: MCP endpoints listed by name only; schemas not extracted.
   deferred_reason: Wire-format extraction is the protocols phase's rubric.
+
+- id: mech-CF3
+  kind: defer-to-phase
+  target_phase: defect-scan-semantic
+  derives_from: q-logit-bias-root-cause   # optional: the open question this is one candidate answer to
+  description: The client sends logit_bias as a map; the documented shape is an array.
 ```
 
 Allowed `kind` values: `needs-runtime-test`, `needs-maintainer-decision`, `needs-spec-ruling`, `defer-to-phase`, `needs-fixture-capture`.
 
+`derives_from` names an `open_questions` id. Fill it in the handoff that registers the question — a phase that routes a candidate answer onward usually writes both entries at once, which is the one moment both are in front of you. It is optional and additive: a handoff that omits it behaves exactly as before.
+
 `proposed_conventions` entries (optional; omitted defaults to empty):
 
 ```yaml
@@ -81,6 +89,26 @@ Completion then removes the entry atomically. Resolving an open question works t
 
 Re-deferring instead of closing means writing a fresh `carry_forward` entry naming a later `target_phase`.
 
+### Closing a routed item does not settle the question it came from
+
+Addressing what was routed to you is not the same as answering the question that produced it. Two rules make that enforceable:
+
+- **A closure whose `derives_from` question is still open is refused.** If the item you are closing declares `derives_from: <question-id>`, that question is still in `status.yaml`, and this same handoff does not close it, completion refuses and names both ids. Either close the question here with the evidence that settles it, or leave the item routed and give the finding an unsettled action (`verify at runtime`) so it inherits the question's uncertainty. This rule is entirely opt-in — an entry without `derives_from` closes as it always has.
+- **A `needs-runtime-test` question closes on runtime evidence.** Write the closure as an object and say where that evidence lives:
+
+```yaml
+open_question_closures:
+  - q-loadconfig-ambiguity                   # bare id: still valid for any other kind
+  - id: q-logit-bias-root-cause
+    evidence: scratch/spikes/logit-bias.md — probe against llama-server b4321
+```
+
+Non-empty `evidence` is required when the question's `kind` is `needs-runtime-test`, whether the closure is written as a bare id or as an object. It is checked for presence, not judged — a spike report or an observation against the running system is what belongs there, and another read of the same source is not. Questions of every other kind close on a bare id exactly as before.
+
+The requirement is scoped to the scaffold that documents it. A workspace whose `workflow/scaffold-version.yaml` is 0.19.0 or newer has completion refuse such a closure; an older or unversioned scaffold — whose own templates never stated the rule — gets a non-gating `NOTE:` instead, so an in-flight run written against the older contract cannot be stopped by a rule it was never told. Refreshing the scaffold (`codecarto_refresh_scaffold` on MCP, `/codecarto-refresh-scaffold` on Pi) opts a workspace in.
+
+Upstream coverage gaps travel the same way, without gating: the `Skipped scope` and `Known blind spots` bullets of every completed phase's `## Coverage and limits` section appear in your phase prompt's orchestrator duties. A finding of yours inside one of those gaps must either close it with cited new evidence or inherit its uncertainty.
+
 ## The failure this prevents
 
 Before completion required a handoff, a phase could finish with empty state and no signal. A real seven-phase run documented five cross-phase routings in its report prose, wrote no handoffs, and completed all seven phases with `carry_forward: []` throughout. Every downstream phase's routed-item intake was empty. The findings survived only because each phase happened to re-read the previous phase's full markdown.

package/dist/core/broadside.d.ts

@@ -6,8 +6,8 @@ export declare const BROADSIDE_SKILL_NAME = "broadside";
 export declare const BROADSIDE_STATE_FILE = "state.json";
 export declare const BROADSIDE_CONFIG_FILE = "config.yaml";
 export declare const BROADSIDE_STATE_SCHEMA_VERSION = 1;
-export declare const BROADSIDE_INPUT_PRICE_PER_M = 0.1875;
-export declare const BROADSIDE_OUTPUT_PRICE_PER_M = 0.9375;
+export declare const BROADSIDE_INPUT_PRICE_PER_M = 0.375;
+export declare const BROADSIDE_OUTPUT_PRICE_PER_M = 1.875;
 export declare const BROADSIDE_MODELS_URL = "https://openrouter.ai/api/v1/models";
 export declare const BROADSIDE_BENCHMARKS_URL = "https://openrouter.ai/api/v1/benchmarks";
 export declare const BROADSIDE_CATALOG_CACHE_FILE = "model-catalog.json";
@@ -49,7 +49,13 @@ export type CodingBenchmarks = {
 export type BroadsideCatalogResult = {
     model: string;
     source: "built-in" | "config" | "live" | "cache";
-    entry: CatalogEntry | null;
+    /**
+     * Always resolved. `resolveCatalogEntry` either returns an entry — from
+     * config, cache, the live catalog, or the compile-time fallback — or throws
+     * naming the model it could not price. This was declared nullable, which is
+     * the only reason the single consumer needed a non-null assertion to read it.
+     */
+    entry: CatalogEntry;
     benchmarks?: CodingBenchmarks;
 };
 export type JsonSchemaDef = {
@@ -115,6 +121,8 @@ export type BroadsideSynthesisEntry = {
     batchId?: string;
     status: "pending" | "submitted" | "completed" | "failed";
     cost?: number;
+    /** Why the pass was retired, when the batch reported one. */
+    error?: string;
 };
 /** One triage item — a scouting lead turned into a work-order entry. */
 export type TriageItem = {
@@ -220,6 +228,8 @@ export type BroadsideEstimate = {
     /** Set when incremental scouting found a baseline to diff against. */
     baseHead: string | null;
     sourceDirty: boolean;
+    /** Whether a requested incremental run actually narrowed this estimate. */
+    incremental: BroadsideIncrementalOutcome;
     /** The provider's completion ceiling, when the catalog advertises one. */
     outputCap?: number;
 };
@@ -227,6 +237,22 @@ export type BroadsideEstimate = {
 export declare class BroadsideCancelledError extends Error {
     constructor(message?: string);
 }
+/**
+ * Whether incremental scouting actually narrowed the run.
+ *
+ * A request for incremental falls back to a full scan whenever there is nothing
+ * to diff against, and that fallback costs real money — the caller asked for the
+ * cheap mode and gets the expensive one. It must be reported, not inferred from
+ * the request counts.
+ */
+export type BroadsideIncrementalOutcome = {
+    requested: boolean;
+    applied: boolean;
+    /** The commit the run diffed against, when one was found. */
+    baseHead: string | null;
+    /** Why a requested incremental run did not apply. */
+    reason?: "dirty-worktree" | "no-baseline" | "diff-failed";
+};
 export type BroadsideSubmitResult = {
     runId: string;
     outputDir: string;
@@ -242,6 +268,7 @@ export type BroadsideSubmitResult = {
         supportsStructuredOutputs?: boolean;
         expirationDate?: string | null;
     };
+    incremental: BroadsideIncrementalOutcome;
 };
 export type BroadsideCollectResult = {
     runId: string;
@@ -287,7 +314,22 @@ export declare function listLenses(): LensDefinition[];
 export declare function collectRepoInfo(targetDir: string): Promise<RepoInfo>;
 export declare function gatherSlices(targetDir: string, lens: LensDefinition, info: RepoInfo): Promise<FileSlice[]>;
 export declare function buildBatchRequest(lens: LensDefinition, info: RepoInfo, slice: FileSlice, index: number, sliceCount: number, model?: string, maxTokensOverride?: number): BatchRequest;
-export declare function estimateCost(lens: LensDefinition, slices: FileSlice[], pricing: ModelPricing, maxTokensOverride?: number): {
+/**
+ * Pre-flight cost estimate for one lens.
+ *
+ * Every slice is its own batch request, so both halves scale with the slice
+ * count. The output half used to be a single `maxTokens * 0.75` for the whole
+ * lens no matter how many requests it sent — on a repository that sliced into
+ * 13 modules that budgeted one request's output and shipped thirteen, and a
+ * live run came in at roughly 3x its estimate. Since this number is what
+ * `max_cost` binds against, under-counting it lets a run outspend the cap the
+ * user set.
+ *
+ * @param info - Repo info, when the caller has it: lets the estimate include
+ *   the system prompt and JSON schema each request carries. Omitted, the
+ *   estimate covers slice content only, which is what the old signature did.
+ */
+export declare function estimateCost(lens: LensDefinition, slices: FileSlice[], pricing: ModelPricing, maxTokensOverride?: number, info?: RepoInfo): {
     inputTokens: number;
     outputTokens: number;
     cost: number;
@@ -313,7 +355,40 @@ export declare function readBroadsideSkill(cwd: string): Promise<{
 }>;
 export declare function defaultBroadsideState(): BroadsideStateFile;
 export declare function loadBroadsideState(broadsideDir: string): Promise<BroadsideStateFile>;
+/**
+ * Overwrite `state.json` wholesale with `state`.
+ *
+ * Prefer {@link persistBroadsideRun} anywhere a live operation is recording its
+ * own progress — this entry point replaces the file, so any run a concurrent
+ * process recorded in the meantime is erased. It remains the right call for
+ * seeding a fresh workspace and for test fixtures, where "make the file exactly
+ * this" is the intent.
+ */
 export declare function saveBroadsideState(broadsideDir: string, state: BroadsideStateFile): Promise<void>;
+/**
+ * Read-modify-write `state.json` under a lock.
+ *
+ * The lock is held only for the read-modify-write, never for the surrounding
+ * operation: a `collect` can poll for the better part of an hour, and holding
+ * the lock across that would push every concurrent caller past the 5s lock
+ * timeout.
+ */
+export declare function updateBroadsideStateAtomically(broadsideDir: string, mutate: (state: BroadsideStateFile) => void | Promise<void>): Promise<BroadsideStateFile>;
+/**
+ * Record one run's current shape, merged into whatever is on disk *now*.
+ *
+ * Broad-Side operations are long-lived and hold their state in memory while
+ * they poll. Writing that snapshot back wholesale silently erased any run a
+ * concurrent operation had recorded since it was loaded, orphaning that run's
+ * paid results on disk — present as files, invisible to `list`, and unreachable
+ * by `collect`, which finds its run by position in `state.runs`. Observed live:
+ * a submit at 23:35 was erased by a collect that had loaded state before it and
+ * wrote back at 00:08.
+ *
+ * Merging by run id also self-heals: a run erased by an older writer is
+ * restored the next time its own operation checkpoints.
+ */
+export declare function persistBroadsideRun(broadsideDir: string, run: BroadsideRun): Promise<BroadsideStateFile>;
 export declare function loadBroadsideConfig(broadsideDir: string): Promise<BroadsideConfig>;
 export declare function builtInCatalogEntry(model: string): CatalogEntry | null;
 export declare function builtInPricing(model: string): ModelPricing | null;
@@ -336,6 +411,14 @@ export declare function submitBatch(batchRequests: BatchRequest[], apiKey: strin
     error?: unknown;
 }>;
 export declare function fetchBatch(batchId: string, apiKey: string, fetcher?: FetchLike): Promise<Record<string, unknown>>;
+/**
+ * Batch statuses that will never produce a result.
+ *
+ * Deliberately excludes the synthetic `timeout` this module returns when a poll
+ * budget expires: that batch is still running server-side and has already been
+ * charged, so callers must come back for it rather than retire it.
+ */
+export declare const BROADSIDE_DEAD_BATCH_STATUSES: string[];
 export declare function pollBatchUntilTerminal(batchId: string, apiKey: string, opts?: {
     deadlineMs?: number;
     onStatus?: (status: string, counts: Record<string, unknown>) => void;
@@ -398,6 +481,18 @@ export type StoredLensResult = {
  */
 export declare function parseLensJson(content: string): unknown | null;
 export declare function saveLensResults(runDir: string, lensId: BroadsideLensId, batch: Record<string, unknown>): Promise<StoredLensResult[]>;
+/**
+ * Rebuild lens results from what a previous collect already wrote to disk.
+ *
+ * The post-passes are gated on having lens findings in hand, and a collect
+ * only holds the ones *it* polled. When an earlier collect saved every lens
+ * and then died before synthesis and triage ran — the batch window is long
+ * and a poll can easily be interrupted — the next collect finds every lens
+ * already terminal, skips them all, and would otherwise reach the post-pass
+ * gate with nothing to hand it. Reading the saved results back is what makes
+ * "a resumed collect can finish whichever is still pending" true.
+ */
+export declare function loadSavedLensResults(runDir: string, lenses: BroadsideLensId[]): Promise<StoredLensResult[]>;
 export declare function runBroadsideCollect(cwd: string, apiKey: string, opts?: {
     waitMs?: number;
     includeSynthesis?: boolean;