baldart 4.30.1 → 4.31.0

package/CHANGELOG.md

@@ -5,6 +5,19 @@ All notable changes to BALDART will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [4.31.0] - 2026-06-11
+
+**`new2`: the residual ledger self-corrects before returning — no more duplicate-of-done follow-ups, no more N defers for one external action.** A real `new2` run (FEAT-0022 epic, 3 cards) surfaced two over-report classes in the offline-safe residual ledger that the skill was absorbing **by hand** every run: (1) **4 of 8 follow-ups were false-open** — scope-expansion residuals deferred early in the batch but satisfied LATER by another card's commit / a final-review fix, which only `integrateCrossCard()` retracts; a residual closed by any *other* in-batch path stayed falsely-open (and left a best-effort, uncommitted follow-up YAML in the worktree). (2) **3 follow-ups for ONE physical action** — one migration's remote `db:push` re-raised per-card AND batch-wide by the final review → three near-identical owner-gated cards. Both were caught only by the skill's manual per-residual disk grep + consolidation (load-bearing, repeated every run). This release moves that work into the workflow, once, deterministically:
+- **Ledger self-correction (`reconcileLedgerAgainstHead`)** — before returning, re-verify every still-pending code/doc residual (`scope-expansion`/`out-of-scope`/`unresolved`/`file-diff-violation`/`merge-artifact-skipped`/`out-of-ownership`, `materialized` true OR false) against the worktree HEAD via one read-only agent, and retract only the ones it can back with a `file:line` proof. **Conservative by design** — default KEEP: a false-open residual is recoverable downstream (the skill re-checks disk), but a wrong retract silently drops real work (F-029). Owner-gated / not-a-code-defect / policy-deferred are EXTERNAL actions a commit cannot close → never auto-retracted here. Telemetry `ledger_reconciled`.
+- **Owner-gated dedup (`dedupOwnerGatedResiduals`)** — collapse owner-gated / not-a-code-defect residuals that share one action key (migration filename · `db:push`/`db:check-sync` · deploy · secret · DNS), keeping **one per distinct real card** (the skill marks each card DONE only after ITS follow-up exists) and dropping only batch-level duplicates (residual `card` is a finding id → no DONE-linkage); a batch-level residual with no matching per-card entry is a genuinely-new action and is kept. Telemetry `owner_gated_deduped`.
+
+The skill's Step 5.1 disk reconciliation **still runs** — it is now a safety net over a pre-cleaned ledger (the self-correction is conservative; F-040 worktree-not-merged still applies), not the sole defence. Same recurring shape as prior `new2` fixes: the splice existed in ONE location (`integrateCrossCard`), the resolution-detection was needed in ALL paths that close a residual. **MINOR** (additive capability + observability on the EXPERIMENTAL `new2` surface; no behavior regression — the guards/policies are unchanged and the retract is proof-gated + conservative; no `baldart.config.yml` key, so the schema-change propagation rule does not apply; no change to `/new`).
+
+### Added
+
+- **`framework/.claude/workflows/new2.js`** — `reconcileLedgerAgainstHead()` (new `Reconcile` phase, conservative proof-gated retract of already-satisfied residuals) + `dedupOwnerGatedResiduals()` (collapse duplicate external actions), both run right before `finalReturn`. New telemetry fields `ledger_reconciled` + `owner_gated_deduped`; new counters wired through `buildTelemetry`.
+- **`framework/.claude/skills/new2/SKILL.md`** — documents that `residuals[]` now arrives pre-cleaned (Step 5.1 reframed as a safety net) and records the two new telemetry fields in the A/B step.
+
 ## [4.30.1] - 2026-06-11
 
 **`new2`: stop spending opus on mechanical ops steps — explicit per-step model overrides.** Three `general-purpose` agents had no `model:` override, so they inherited the session's main-loop model (opus) for work that needs none. The Merge step is a deterministic OPS/GIT executor (git merge + YAML status reconciliation + grep-based epic closure + leave-and-report hygiene gates) whose correctness-critical checks (F-029 forcedDone guard, F-040 deferred guard) are enforced in JS AFTER it returns — independent of the agent's reasoning → **sonnet**. The per-card Codex review agent is a pure DRIVER (runs the companion, strips `[codex]` traces, maps findings) — the review intelligence is Codex, run externally → **haiku**. The post-merge Production Readiness checklist is non-blocking report-not-execute → **sonnet**. The Pre-flight agent (DAG + ownership map + idempotency — it grounds the whole batch) intentionally **stays opus**. **PATCH** (cost optimization on the EXPERIMENTAL `new2` surface; no behavior change — the deterministic guards/policies are unchanged; no config key).

package/VERSION

@@ -1 +1 @@
-4.30.1
+4.31.0

package/framework/.claude/skills/new2/SKILL.md

@@ -159,6 +159,14 @@ returns when the batch is done. It returns:
   flag is **advisory only** — `true` means the workflow *attempted* a write (possibly
   into a worktree that never merged), not that a card exists on disk in the main repo.
   **You (the skill) must reconcile EVERY residual against the main-repo disk** (Step 5.1).
+  **v4.31.0 — the workflow now self-corrects this ledger before returning**: it re-verifies
+  every still-pending code/doc residual against the worktree HEAD and **retracts the ones a
+  later in-batch commit already satisfied** (telemetry `ledger_reconciled`), and **collapses
+  duplicate owner-gated residuals that map to one external action** (e.g. one migration's
+  `db:push` re-raised per-card + batch-wide → telemetry `owner_gated_deduped`). So `residuals[]`
+  arrives pre-cleaned of false-open and duplicate entries; your Step 5.1 disk reconciliation is
+  now a **safety net** over a clean ledger (it still runs — the self-correction is conservative
+  and F-040 worktree-not-merged still applies), not the sole defence.
 - `degraded` / `degradationReasons` — the batch stopped early under a sustained
   outage (or another degradation). The batch is NOT complete; it must be resumed.
 - `telemetry` — the Phase-8 record (`variant:"new2"`).
@@ -234,4 +242,9 @@ returns when the batch is done. It returns:
    too (count of residuals the pre-final-review Cross-Card Integration Pass implemented in-batch —
    out-of-ownership-within-batch + outage retries — instead of leaving as follow-ups); with
    `deferral_breakdown` it shows how many deferrals were genuinely undeferrable vs absorbed in-batch.
+   Keep `ledger_reconciled` + `owner_gated_deduped` (v4.31.0) too — they quantify the ledger
+   self-correction: `ledger_reconciled` > 0 means the workflow retracted residuals a later commit had
+   already satisfied (work the skill used to suppress by hand; a persistently high value signals
+   deferrals resolving too late — order the dependent card earlier), and `owner_gated_deduped` > 0
+   means N defers were collapsed to one external action.
    Do NOT re-summarise the cards — the workflow already did.

package/framework/.claude/workflows/new2.js

@@ -9,6 +9,7 @@ export const meta = {
     { title: 'Final', detail: 'cross-batch final review (delegates to new-final-review)' },
     { title: 'Merge', detail: 'integrity-gated auto-merge to trunk via git.merge_strategy + cleanup' },
     { title: 'Production', detail: 'post-merge production-readiness checklist (Phase 7, non-blocking)' },
+    { title: 'Reconcile', detail: 'ledger self-correction: retract residuals already satisfied in HEAD + dedup owner-gated external actions (v4.31.0)' },
   ],
 }
 
@@ -71,6 +72,8 @@ const cardMayEdit = {}       // v4.30.0 — per-card MAY-EDIT, for the cross-car
 const perCardResults = []
 let prodReadiness = null
 let integratedCount = 0      // v4.30.0 — residuals resolved by the Cross-Card Integration Pass
+let ledgerReconciled = 0     // v4.31.0 — residuals retracted because already-satisfied in HEAD (resolved by a later commit, not by integrateCrossCard)
+let ownerGatedDeduped = 0    // v4.31.0 — duplicate owner-gated residuals collapsed (N defers → 1 external action)
 let degraded = false
 const degradationReasons = []
 
@@ -1062,6 +1065,111 @@ if (mergeResult && mergeResult.merged) {
   ledger(firstCard, 'phase7-production', 'SKIPPED', 'not merged')
 }
 
+// ───────────────────────────────────────────────────────────────────────────
+// Ledger self-correction (v4.31.0) — a residual deferred earlier in the batch can be satisfied
+// LATER by another card's commit, the Cross-Card Integration Pass, or the final-review resolve.
+// Only integrateCrossCard() splices ITS OWN resolutions; a residual closed by any OTHER in-batch
+// path stays falsely-open in the offline-safe ledger (and materialiseFollowup left a best-effort,
+// uncommitted follow-up YAML in the worktree). The skill then has to grep main-repo disk per
+// residual to suppress the false-open ones — load-bearing, manual, repeated every run. Move that
+// verification HERE, once, deterministically: re-check every still-pending code/doc residual
+// against the worktree HEAD and retract only the ones an agent can PROVE are already satisfied.
+// CONSERVATIVE: default KEEP — a false-open residual is recoverable downstream (the skill re-checks
+// disk), but a wrong retract silently drops real work (F-029). Owner-gated / not-a-code-defect /
+// policy-deferred are EXTERNAL actions a commit cannot close → never auto-retracted here.
+// ───────────────────────────────────────────────────────────────────────────
+const VERIFIABLE_RETRACT = new Set(['scope-expansion', 'out-of-scope', 'unresolved', 'file-diff-violation', 'merge-artifact-skipped', 'out-of-ownership'])
+async function reconcileLedgerAgainstHead() {
+  if (degraded || !sharedCtx || !sharedCtx.worktreePath) return
+  // materialized:true is NOT proof a residual is correctly-open — its best-effort worktree YAML may
+  // be for work since closed, which would mint a duplicate-of-done card in the main repo. So verify
+  // BOTH materialized true and false, restricted to classes a later commit could actually close.
+  const candidates = residuals.filter((r) => !r.integrated && VERIFIABLE_RETRACT.has(r.deferralClass || r.kind))
+  if (!candidates.length) return
+  phase('Reconcile')
+  let verdict = null
+  try {
+    verdict = await agentSafe(
+      `Read-only verification (you are general-purpose ops; ROLE BOUNDARY: git read commands only — you NEVER edit, write, or delete any file). cd into the worktree ${sharedCtx.worktreePath}; its HEAD holds every committed card of the batch.\n\n` +
+        `For each residual below, decide whether the gap it describes is ALREADY SATISFIED in the worktree HEAD — a later card's commit, a cross-card fix, or a final-review fix may have closed it after it was deferred. Inspect the actual code/docs/tests in HEAD; do NOT trust the residual text.\n` +
+        `CONSERVATIVE CONTRACT: set resolved only for indices you can back with the exact file:line in HEAD that satisfies the gap. If you are not certain, OMIT the index (a still-open residual is safely re-checked downstream; a wrong 'resolved' silently drops real work).\n\n` +
+        `Residuals (index · card · class · evidence):\n` +
+        candidates.map((r, i) => `  [${i}] ${r.card} · ${r.deferralClass || r.kind} · ${r.evidence}`).join('\n') +
+        `\n\nReturn: { resolved: [ { index, proof } ] } — only indices already satisfied in HEAD, each with a file:line proof.`,
+      { label: 'ledger-reconcile', phase: 'Reconcile', agentType: 'general-purpose', model: 'sonnet',
+        schema: { type: 'object', required: ['resolved'], additionalProperties: true, properties: { resolved: { type: 'array', items: { type: 'object', required: ['index'], additionalProperties: true, properties: { index: { type: 'number' }, proof: { type: 'string' } } } } } } }
+    )
+  } catch (e) { if (e && e.transientExhausted) noteDegraded('outage'); return }
+  const satisfied = ((verdict && verdict.resolved) || []).map((x) => x.index).filter((i) => Number.isInteger(i) && i >= 0 && i < candidates.length)
+  if (!satisfied.length) { ledger(firstCard, 'ledger-reconcile', 'CLEAN', `${candidates.length} pending residual(s); none verifiably already-satisfied in HEAD`); return }
+  const retracted = []
+  for (const i of satisfied) {
+    const r = candidates[i]
+    const idx = residuals.indexOf(r)
+    if (idx < 0) continue // already spliced (duplicate index)
+    residuals.splice(idx, 1)
+    for (let j = residualFollowups.length - 1; j >= 0; j--) {
+      if (residualFollowups[j].card === r.card && residualFollowups[j].kind === r.kind) residualFollowups.splice(j, 1)
+    }
+    retracted.push(`${r.card}/${r.deferralClass || r.kind}`)
+    ledgerReconciled++
+  }
+  ledger(firstCard, 'ledger-reconcile', 'RETRACTED', `${retracted.length} residual(s) already-satisfied in HEAD → retracted (no duplicate-of-done card): ${retracted.join(', ')}`)
+}
+
+// ───────────────────────────────────────────────────────────────────────────
+// Owner-gated dedup (v4.31.0) — several deferred ACs across the batch can map to ONE physical
+// external action (e.g. one migration's remote `db:push` re-raised per-card AND batch-wide by the
+// final review). Minting N follow-ups for one action is redundant. Collapse owner-gated /
+// not-a-code-defect residuals that share an action key — but KEEP one per distinct REAL card (the
+// skill marks each card DONE only after ITS follow-up exists) and drop only batch-level duplicates
+// (residual.card is a finding id, not a backlog card → no DONE-linkage to preserve). A batch-level
+// owner-gated residual with NO matching real-card entry is a genuinely-new action → kept untouched.
+// ───────────────────────────────────────────────────────────────────────────
+function ownerGatedActionKey(r) {
+  const hay = `${r.evidence || ''} ${(r.remedyFiles || []).join(' ')}`
+  const mig = hay.match(/(\d{14}_[a-z0-9_]+\.sql)/i)
+  if (mig) return 'migration:' + mig[1].toLowerCase()
+  if (/\bdb:push\b|\bdb:check-sync\b|remote db push|migration.*(deploy|remote|push)/i.test(hay)) return 'db-migration-deploy'
+  if (/\bdeploy(ment)?\b/i.test(hay)) return 'deploy'
+  if (/\bsecret\b/i.test(hay)) return 'secret'
+  if (/\bDNS\b|\bdomain\b/i.test(hay)) return 'dns'
+  return null // unknown action → never dedup (avoid collapsing two genuinely-distinct externals)
+}
+function dedupOwnerGatedResiduals() {
+  const realCard = new Set(cardIds)
+  const groups = {}
+  for (const r of residuals) {
+    if (r.deferralClass !== 'owner-gated' && r.deferralClass !== 'not-a-code-defect') continue
+    const k = ownerGatedActionKey(r)
+    if (!k) continue
+    ;(groups[k] = groups[k] || []).push(r)
+  }
+  for (const k of Object.keys(groups)) {
+    const g = groups[k]
+    if (g.length < 2) continue
+    if (!g.some((r) => realCard.has(r.card))) continue // all batch-level (new action) — keep them
+    const seenCard = new Set()
+    for (const r of g) {
+      const isReal = realCard.has(r.card)
+      const isDup = isReal ? seenCard.has(r.card) : true // keep one per real card; every batch-level entry is a dup of the per-card tracking
+      if (isReal) seenCard.add(r.card)
+      if (!isDup) continue
+      const idx = residuals.indexOf(r)
+      if (idx < 0) continue
+      residuals.splice(idx, 1)
+      for (let j = residualFollowups.length - 1; j >= 0; j--) {
+        if (residualFollowups[j].card === r.card && residualFollowups[j].kind === r.kind) residualFollowups.splice(j, 1)
+      }
+      ownerGatedDeduped++
+    }
+  }
+  if (ownerGatedDeduped) ledger(firstCard, 'owner-gated-dedup', 'COLLAPSED', `${ownerGatedDeduped} duplicate owner-gated residual(s) collapsed (multiple defers → one external action)`)
+}
+
+await reconcileLedgerAgainstHead()
+dedupOwnerGatedResiduals()
+
 return finalReturn({ fatal: false })
 
 // ───────────────────────────────────────────────────────────────────────────
@@ -1105,6 +1213,13 @@ function buildTelemetry() {
     // v4.30.0 — residuals the Cross-Card Integration Pass implemented in-batch (out-of-ownership
     // within the batch union + outage retries) instead of leaving as follow-ups to manage later.
     cross_card_integrated: integratedCount,
+    // v4.31.0 — residuals retracted because a later in-batch commit already satisfied them (verified
+    // against the worktree HEAD, conservative). A non-zero value means the ledger self-corrected what
+    // the skill used to suppress by hand; a persistently high value signals deferrals that resolve too
+    // late (consider ordering the dependent card earlier).
+    ledger_reconciled: ledgerReconciled,
+    // v4.31.0 — duplicate owner-gated residuals collapsed to one external action (e.g. one db:push).
+    owner_gated_deduped: ownerGatedDeduped,
     // followups_on_disk is filled by the SKILL after it materialises pending residuals.
     followups_materialized_in_workflow: residuals.filter((x) => x.materialized).length,
     resolve_invocations: resolvedSignatures.size,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "baldart",
-  "version": "4.30.1",
+  "version": "4.31.0",
   "description": "Claude Agent Framework - Reusable framework for coordinating AI agents and humans in software projects",
   "bin": {
     "baldart": "./bin/baldart.js"