@fraction12/deepclean 0.1.0-alpha.1 → 0.1.0-alpha.2

package/CHANGELOG.md

@@ -2,6 +2,13 @@
 
 ## Unreleased
 
+## 0.1.0-alpha.2 - 2026-05-27
+
+- Changed `scan` and CI-style scans to request Codex synthesis by default after local evidence collection, with `--evidence-only` as the deterministic-only escape hatch.
+- Added synthesis attempt ledgers with validation checks, failure records for malformed provider output, and final candidate ID alignment after ranking.
+- Included `.deepclean/synthesis/` in doctor/status/prune retention so synthesis artifacts are visible, validated, and cleaned up with the rest of a run.
+- Refined the public site hero and motion treatment after UAT.
+
 ## 0.1.0-alpha.1 - 2026-05-27
 
 - Added semantic feature mapping with `.deepclean/features/` artifacts, `deepclean map`, scan feature counts, and first-pass package script, TS/JS, Python, test-suite, route/component/module, and config feature records.

package/README.md

@@ -30,33 +30,34 @@ deepclean next
 deepclean plan candidate-001
 ```
 
-To include local Codex synthesis:
+`deepclean scan` collects local evidence first, then runs Codex synthesis by default. Use evidence-only mode when you only want deterministic local analysis:
 
 ```bash
-deepclean scan --synthesize --json
+deepclean scan --evidence-only --json
 deepclean report
-deepclean plan theme-001
 ```
 
 Global flags work before or after the command:
 
 ```bash
-deepclean --root ./some-repo scan --synthesize
-deepclean scan --root ./some-repo --synthesize
+deepclean --root ./some-repo scan
+deepclean scan --root ./some-repo --evidence-only
 ```
 
+Older examples may include `deepclean scan --synthesize`; that flag still works, but it is no longer required. Plain `deepclean scan` is the normal synthesized path. Use `--evidence-only`, `--offline`, or `--local-only` when a run must avoid provider execution.
+
 ## Workflow
 
 ```bash
 deepclean init
 deepclean map
 deepclean scan
-deepclean scan --synthesize
 deepclean report
 deepclean cluster
 deepclean plan theme-001 --format codex
 deepclean next
 deepclean show <candidate-id>
+deepclean explain <candidate-or-finding-id>
 deepclean triage <candidate-id> --status ignored --note "intentional boundary"
 deepclean handoff <candidate-id> --format codex
 ```
@@ -68,6 +69,7 @@ Deepclean writes durable local artifacts under `.deepclean/`:
 - `runs/` - scan metadata
 - `features/` - semantic feature/work-unit maps
 - `evidence/` - raw local evidence records
+- `synthesis/` - provider attempt ledgers, prompt manifests, and candidate validation results
 - `candidates/` - cleanup candidates
 - `clusters/` - related cleanup themes
 - `reports/` - Markdown and JSON reports
@@ -84,12 +86,13 @@ Core commands support `--json` for automation:
 ```bash
 deepclean scan --json
 deepclean map --json
-deepclean scan --synthesize --json
+deepclean scan --evidence-only --json
 deepclean report --json
 deepclean cluster --json
 deepclean plan theme-001 --json
 deepclean next --json
 deepclean show candidate-001 --json
+deepclean explain candidate-001 --json
 deepclean handoff candidate-001 --json
 ```
 
@@ -104,7 +107,7 @@ Useful global flags:
 
 ## Local Evidence
 
-Deepclean runs local evidence first and optional model synthesis second. The built-in evidence layer includes:
+Deepclean runs local evidence first and model synthesis second unless evidence-only or local-only mode is selected. The built-in evidence layer includes:
 
 - semantic feature mapping for package scripts, TS/JS modules/routes/components, Python modules, test suites, and config files
 - file metrics
@@ -122,7 +125,9 @@ For TS/JS projects using NodeNext-style source imports, Deepclean resolves emitt
 
 ## Codex Synthesis
 
-`deepclean scan --synthesize` runs the local `codex` CLI in read-only mode over the collected evidence bundle. The model is asked to return strict JSON, and candidates without valid evidence IDs are rejected.
+`deepclean scan` runs the local `codex` CLI in read-only mode over the collected evidence bundle by default. The model is asked to return strict JSON, and candidates are validated before they are persisted: cited evidence IDs must exist, file paths must be anchored by cited evidence, line ranges must be sane, and optional quotes must match source. Rejected drafts stay in the synthesis attempt ledger as diagnostics rather than becoming open findings.
+
+Use `deepclean explain <candidate-or-finding-id>` to inspect why a candidate exists, which evidence supports it, which validation checks passed, and what fix-readiness guidance was attached.
 
 Synthesis uses a built-in reviewer pack so runs do not depend on arbitrary local agent skills. The current pack looks for architecture boundaries, conceptual duplication, dependency graph risk, testability gaps, domain language drift, agent-sized cleanup slices, and weak findings that should be rejected.
 
@@ -137,7 +142,7 @@ Reviewer packs can be configured in `.deepclean/config.json`:
 }
 ```
 
-Source samples are redacted from the synthesis prompt by default. Use `--allow-source-in-model` only when the target repository and provider configuration make that acceptable.
+Source samples are redacted from the synthesis prompt by default. Use `--allow-source-in-model` only when the target repository and provider configuration make that acceptable. Use `--evidence-only`, `--offline`, or `--local-only` when no provider should run.
 
 See [Privacy And Trust](docs/privacy-and-trust.md), [Reviewer References](docs/reviewer-references.md), and [Troubleshooting](docs/troubleshooting.md) before using synthesis on private repos.
 

package/dist/cli.js

@@ -17,7 +17,7 @@ import { LockContentionError, lockRecoveryCommand, readLockStatuses, recoverStal
 import { buildCandidatePlan, buildClusterPlan } from "./plans.js";
 import { classifyRevalidation } from "./revalidation.js";
 import { buildHandoff, buildReportRecord, renderMarkdownReport, renderMarkdownReportWithClusters, } from "./reporting.js";
-import { ensureState, latestRunId, readConfig, readCandidates, readFindings, readLatestCandidates, readLatestClusters, readLatestEvidence, readLatestFeatures, readLifecycleEvents, resolveStatePaths, updateLatestCandidates, writeCandidates, writeCandidateObservations, writeCiRun, writeClusters, writeEvidence, writeFeatures, writeFindings, writeFixAttempt, writeHandoff, writeLifecycleEvents, writePlan, writeReport, writeRetentionManifest, writeRevalidation, writeRun, writeTriage, } from "./state.js";
+import { ensureState, latestRunId, readConfig, readCandidates, readFindings, readLatestCandidates, readLatestClusters, readLatestEvidence, readLatestFeatures, readLatestSynthesisAttempt, readLifecycleEvents, resolveStatePaths, updateLatestCandidates, writeCandidates, writeCandidateObservations, writeCiRun, writeClusters, writeEvidence, writeFeatures, writeFindings, writeFixAttempt, writeHandoff, writeLifecycleEvents, writePlan, writeReport, writeRetentionManifest, writeRevalidation, writeRun, writeSynthesisAttempt, writeTriage, } from "./state.js";
 import { candidateStatuses, schemaVersion, } from "./types.js";
 import { timestampId } from "./ids.js";
 import { synthesizeWithCodex } from "./synthesis.js";
@@ -35,6 +35,7 @@ const commands = [
     "list",
     "findings",
     "show",
+    "explain",
     "history",
     "revalidate",
     "unlock",
@@ -60,7 +61,8 @@ Commands:
   ci                           Run non-interactive scan and policy gates for CI
   map                          Write semantic feature records without producing candidates
   scan                         Collect local evidence and generate candidates
-    --synthesize               Run local Codex synthesis over evidence
+    --synthesize               Run local Codex synthesis over evidence (default)
+    --evidence-only            Skip synthesis and produce local evidence candidates only
     --allow-source-in-model    Include source samples in Codex prompt
     --offline                  Skip provider calls and network-style analyzers
     --local-only               Alias for --offline
@@ -87,6 +89,8 @@ Commands:
   list                         List findings with shared filters
   findings                     Alias for list
   show <candidate-or-theme>    Show one candidate or cleanup theme with evidence
+  explain <candidate-or-finding>
+                               Explain evidence, validation, and fix-readiness for a finding
   history <finding-or-candidate-id>
                                Show lifecycle history for a finding
   revalidate <finding-id|candidate-id|all>
@@ -177,6 +181,8 @@ export async function main(argv, cwd = process.cwd()) {
                 return await listCommand(context);
             case "show":
                 return await showCommand(context);
+            case "explain":
+                return await explainCommand(context);
             case "history":
                 return await historyCommand(context);
             case "revalidate":
@@ -696,16 +702,26 @@ async function executeFeatureMap(context) {
 }
 async function ciCommand(context) {
     const requireSynthesis = flagBoolean(context.parsed.flags, "require-synthesis");
-    if (requireSynthesis && !flagBoolean(context.parsed.flags, "synthesize")) {
+    const config = await ensureState(context.paths);
+    if (requireSynthesis && synthesisDisabledByPolicy(context, config)) {
         const diagnostic = {
             level: "error",
             code: "ci_synthesis_required",
-            message: "CI policy requires synthesis; rerun with --synthesize and a configured provider.",
+            message: "CI policy requires synthesis; rerun without evidence-only/local-only flags and with a configured provider.",
         };
         emit(context.json, fail("ci", "ci_synthesis_required", diagnostic.message, [diagnostic]));
         return 2;
     }
-    const scan = await executeScan(context, { synthesize: flagBoolean(context.parsed.flags, "synthesize") });
+    const scan = await executeScan(context, {});
+    const synthesisFailure = requireSynthesis ? requiredSynthesisFailure(scan) : undefined;
+    if (synthesisFailure) {
+        const diagnostics = [
+            synthesisFailure,
+            ...scan.diagnostics.filter((diagnostic) => !sameSynthesisFailure(diagnostic, synthesisFailure)),
+        ];
+        emit(context.json, fail("ci", "ci_synthesis_failed", synthesisFailure.message, diagnostics));
+        return 2;
+    }
     const policy = ciPolicyFromFlags(context);
     const gate = evaluateCiPolicy(scan.data.candidates, policy);
     const createdAt = new Date().toISOString();
@@ -773,14 +789,13 @@ async function executeScan(context, options) {
     const evidence = markDirtyTreeEvidence(adapterResult.evidence, scope);
     const completedAt = new Date().toISOString();
     const localCandidates = candidatesFromEvidence(runId, evidence, completedAt, config.candidateCaps, verificationProfile);
-    const synthesisRequested = options.synthesize ?? (flagBoolean(context.parsed.flags, "synthesize")
-        || config.reviewSynthesis.enabled);
+    const synthesisRequested = options.synthesize ?? true;
     const runtime = providerRuntimeControls(context, config);
     if (synthesisRequested && runtime.offline) {
         adapterResult.diagnostics.push({
             level: "info",
             code: "synthesis_skipped_by_policy",
-            message: "Provider synthesis was skipped because offline/local-only mode is active.",
+            message: "Provider synthesis was skipped because evidence-only/offline/local-only mode is active.",
             adapter: "codex-synthesis",
         });
     }
@@ -814,6 +829,9 @@ async function executeScan(context, options) {
     const clusters = buildClusters(runId, candidates, evidence, completedAt, config.clusters);
     await writeFeatures(context.paths, runId, features);
     await writeEvidence(context.paths, runId, evidence);
+    if (synthesisResult.attempt) {
+        await writeSynthesisAttempt(context.paths, remapSynthesisAttemptCandidateIds(synthesisResult.attempt, candidates));
+    }
     await writeCandidates(context.paths, runId, candidates);
     await writeFindings(context.paths, identity.findings);
     await writeCandidateObservations(context.paths, runId, identity.observations);
@@ -835,6 +853,9 @@ async function executeScan(context, options) {
             requested: shouldSynthesize,
             provider: shouldSynthesize ? runtime.provider : undefined,
             candidateCount: synthesisResult.candidates.length,
+            attemptId: synthesisResult.attempt?.id,
+            acceptedCandidateCount: synthesisResult.attempt?.acceptedCandidateCount,
+            rejectedCandidateCount: synthesisResult.attempt?.rejectedCandidateCount,
             runtime: providerRuntimeSummary(runtime),
         },
         scope,
@@ -851,6 +872,9 @@ async function executeScan(context, options) {
         synthesis: {
             requested: shouldSynthesize,
             candidateCount: synthesisResult.candidates.length,
+            attemptId: synthesisResult.attempt?.id,
+            acceptedCandidateCount: synthesisResult.attempt?.acceptedCandidateCount,
+            rejectedCandidateCount: synthesisResult.attempt?.rejectedCandidateCount,
             runtime: providerRuntimeSummary(runtime),
         },
         candidates,
@@ -860,6 +884,20 @@ async function executeScan(context, options) {
     };
     return { runId, diagnostics, data };
 }
+function remapSynthesisAttemptCandidateIds(attempt, candidates) {
+    const candidateIdByValidationId = new Map(candidates
+        .filter((candidate) => candidate.provenance.source === "model-synthesis")
+        .flatMap((candidate) => candidate.provenance.validationId
+        ? [[candidate.provenance.validationId, candidate.id]]
+        : []));
+    return {
+        ...attempt,
+        validations: attempt.validations.map((validation) => ({
+            ...validation,
+            candidateId: candidateIdByValidationId.get(validation.id),
+        })),
+    };
+}
 async function reportCommand(context) {
     const { candidates, evidence, runId } = await latestState(context.paths);
     const config = await ensureState(context.paths);
@@ -966,6 +1004,79 @@ async function showCommand(context) {
     }
     return 0;
 }
+async function explainCommand(context) {
+    const id = requireCandidateId(context);
+    const { candidates, evidence } = await latestState(context.paths);
+    const attempt = await readLatestSynthesisAttempt(context.paths);
+    const candidate = candidates.find((item) => item.id === id || item.findingId === id);
+    if (!candidate) {
+        emit(context.json, fail("explain", "candidate_not_found", `Candidate or finding not found: ${id}`));
+        return 1;
+    }
+    const supportingEvidence = evidenceForIds(evidence, candidate.evidenceIds);
+    const validation = validationForCandidate(candidate, attempt);
+    const diagnostics = validation?.diagnostics ?? [];
+    const explanation = {
+        candidate,
+        evidence: supportingEvidence,
+        synthesisAttempt: attempt ? {
+            id: attempt.id,
+            runId: attempt.runId,
+            provider: attempt.provider,
+            model: attempt.model,
+            promptVersion: attempt.promptVersion,
+            promptBytes: attempt.promptBytes,
+            rawCandidateCount: attempt.rawCandidateCount,
+            acceptedCandidateCount: attempt.acceptedCandidateCount,
+            rejectedCandidateCount: attempt.rejectedCandidateCount,
+            evidenceManifest: attempt.evidenceManifest,
+        } : undefined,
+        validation,
+        fixReadiness: candidate.fixReadiness,
+        verification: candidate.verification,
+        diagnostics,
+    };
+    emit(context.json, ok("explain", explanation, diagnostics));
+    if (!context.json && !context.quiet) {
+        printCandidate(candidate);
+        console.log("");
+        console.log("Why this exists:");
+        console.log(`  ${candidate.whyItMatters}`);
+        console.log("");
+        console.log("Evidence:");
+        for (const record of supportingEvidence) {
+            console.log(`  ${record.id} ${record.kind}: ${record.summary}`);
+            for (const file of record.files) {
+                console.log(`    ${formatFileRef(file)}`);
+            }
+        }
+        if (validation) {
+            console.log("");
+            console.log(`Validation: ${validation.status} (${validation.id})`);
+            if (validation.diagnostics.length === 0) {
+                console.log("  All cited evidence IDs, file paths, line ranges, and quotes passed validation.");
+            }
+            else {
+                for (const diagnostic of validation.diagnostics) {
+                    console.log(`  ${diagnostic.code}: ${diagnostic.message}`);
+                }
+            }
+        }
+        if (candidate.fixReadiness) {
+            console.log("");
+            console.log("Fix readiness:");
+            console.log(`  scope: ${candidate.fixReadiness.minimumFixScope}`);
+            console.log(`  regression: ${candidate.fixReadiness.suggestedRegressionTest}`);
+            console.log(`  test gap: ${candidate.fixReadiness.whyCurrentTestsMissIt}`);
+            for (const reason of candidate.fixReadiness.confidenceDowngradeReasons) {
+                console.log(`  confidence note: ${reason}`);
+            }
+        }
+        console.log("");
+        console.log(`Verification: ${candidate.verification.join("; ") || "n/a"}`);
+    }
+    return 0;
+}
 async function historyCommand(context) {
     const id = requireCandidateId(context);
     const runId = flagString(context.parsed.flags, "run");
@@ -1456,6 +1567,22 @@ function evidenceForIds(evidence, ids) {
     const wanted = new Set(ids);
     return evidence.filter((item) => wanted.has(item.id));
 }
+function validationForCandidate(candidate, attempt) {
+    const validationId = candidate.provenance.validationId;
+    if (!attempt || !validationId) {
+        return undefined;
+    }
+    return attempt.validations.find((validation) => validation.id === validationId);
+}
+function formatFileRef(file) {
+    if (file.startLine !== undefined && file.endLine !== undefined) {
+        return `${file.path}:${file.startLine}-${file.endLine}`;
+    }
+    if (file.startLine !== undefined) {
+        return `${file.path}:${file.startLine}`;
+    }
+    return file.path;
+}
 function printDiagnostics(diagnostics) {
     for (const diagnostic of diagnostics) {
         console.log(`${diagnostic.level}: ${diagnostic.code}: ${diagnostic.message}`);
@@ -1489,6 +1616,7 @@ async function missingStateDirectories(paths) {
         ["locks", paths.locksDir],
         ["retention", paths.retentionDir],
         ["fixes", paths.fixesDir],
+        ["synthesis", paths.synthesisDir],
     ];
     const missing = [];
     for (const [name, dir] of expected) {
@@ -1805,6 +1933,7 @@ async function buildRetentionManifest(context) {
         [context.paths.candidatesDir, "json"],
         [context.paths.clustersDir, "json"],
         [context.paths.observationsDir, "json"],
+        [context.paths.synthesisDir, "json"],
     ]) {
         const files = await filesWithExtension(dir, extension);
         for (const file of files) {
@@ -2096,6 +2225,7 @@ function providerRuntimeControls(context, config) {
         ?? config.reviewSynthesis.privacyMode;
     const offline = flagBoolean(context.parsed.flags, "offline")
         || flagBoolean(context.parsed.flags, "local-only")
+        || flagBoolean(context.parsed.flags, "evidence-only")
         || config.reviewSynthesis.offline
         || privacyMode === "local-only";
     const excerptBudget = numberFlag(context, "excerpt-budget") ?? config.reviewSynthesis.excerptBudget;
@@ -2125,6 +2255,46 @@ function providerRuntimeControls(context, config) {
     }
     return runtime;
 }
+function synthesisDisabledByPolicy(context, config) {
+    const privacyMode = privacyModeFromFlag(flagString(context.parsed.flags, "privacy-mode"))
+        ?? config.reviewSynthesis.privacyMode;
+    return flagBoolean(context.parsed.flags, "offline")
+        || flagBoolean(context.parsed.flags, "local-only")
+        || flagBoolean(context.parsed.flags, "evidence-only")
+        || config.reviewSynthesis.offline
+        || privacyMode === "local-only";
+}
+const requiredSynthesisFailureCodes = new Set([
+    "codex_provider_unavailable",
+    "codex_synthesis_timeout",
+    "codex_synthesis_failed",
+    "codex_synthesis_error",
+]);
+function requiredSynthesisFailure(scan) {
+    if (!scan.data.synthesis.requested) {
+        return {
+            level: "error",
+            code: "ci_synthesis_required",
+            message: "CI policy requires synthesis, but the scan did not run provider synthesis.",
+            adapter: "codex-synthesis",
+        };
+    }
+    const diagnostic = scan.diagnostics.find((item) => (item.adapter === "codex-synthesis"
+        && requiredSynthesisFailureCodes.has(item.code)));
+    if (!diagnostic) {
+        return undefined;
+    }
+    return {
+        ...diagnostic,
+        level: "error",
+        message: `CI policy requires synthesis, but provider synthesis failed: ${diagnostic.message}`,
+    };
+}
+function sameSynthesisFailure(diagnostic, failure) {
+    return diagnostic.adapter === failure.adapter
+        && diagnostic.code === failure.code
+        && requiredSynthesisFailureCodes.has(diagnostic.code);
+}
 function providerRuntimeSummary(runtime) {
     return {
         provider: runtime.provider,
@@ -2193,6 +2363,7 @@ async function stateArtifactCounts(paths) {
         ["locks", paths.locksDir],
         ["retention", paths.retentionDir],
         ["fixes", paths.fixesDir],
+        ["synthesis", paths.synthesisDir],
     ];
     const counts = {};
     for (const [name, dir] of dirs) {