@ngocsangairvds/vsaf 3.2.13 → 3.2.15

package/tools/vds-scripts/audit_orchestrator/docs/PHASE_64_TO_92_HISTORICAL_SPEC.md

@@ -1,1772 +0,0 @@
-# Audit Orchestrator - Historical Specs (Phases 64-92)
-
-This document details the completed functional requirements, designs, and tasks from Phase 64 up through 92.
-
-### Phase 88 FR Traceability (Closure)
-
-- FR-177/FR-178: Direct crawl attachments and linked fallback attachments are chunked and appended to corpus with deterministic lineage metadata before retrieval refresh.
-- FR-179: Mobile/native source extensions are included in bounded code chunking coverage with max file-count/size and skip-dir guards preserved.
-- FR-180: `attachment_conversion_summary` is emitted in extract-docs project payload with status/format/converter diagnostics.
-- FR-180 follow-up: extensionless draw.io/XML payloads are inferred to `.xml` and counted in format diagnostics instead of `(unknown)`.
-- FR-180 follow-up: `sync_metadata.artifacts[]` persists attachment `resource_metadata` (`source_format`, `converter`, `discovery`, size/timing) so Postgres state diagnostics can inspect conversion quality without rerunning extraction.
-- FR-181: Real-run multi-format probes (Miniapp + DOCX-heavy + PDF-heavy) verify conversion/chunk inclusion across `.pdf/.docx/.xlsx/.xls`.
-- FR-181 follow-up: HTML conversion fallback preserves readable Vietnamese content when external HTML conversion emits mojibake text.
-- FR-182: Retrieval snapshot stale-pruning + dedupe semantics are preserved and regression-hardened, including empty-chunk refresh behavior.
-- FR-183: Corpus diagnostics expose deterministic source/discovery composition through `docs_chunk_summary` and `project_chunk_summary`.
-- FR-184: AGENTS/spec/docs/guides/skill sync gate remains green via `validate-spec-sync` with aligned version/status/test baseline, backed by durable repo evidence and CI guard (`scripts/verify_phase_88.py`, `tests/unit/scripts/test_phase88_verification_artifacts.py`).
-
-
-### NFRs for Phase 86
-
-- **NFR-T1: Non-regression** - Preserve Phase 84 routing payload and Phase 85 readiness artifacts.
-- **NFR-T2: Telemetry completeness** - Emit deterministic diagnostics for all effectiveness controls.
-- **NFR-T3: Bounded behavior** - Keep depth/context policies under timeout and budget guardrails.
-
-### NFRs for Phase 87
-
-- **NFR-U1: Selector determinism** - URL/page-ID/storage-key selectors for project parsing must resolve to one canonical project identity with explicit diagnostics.
-- **NFR-U2: Direct-project UX** - Commands intended for individual project parsing should not require registry-root indirection.
-- **NFR-U3: Documentation parity** - Docs/guides/skills must reflect implemented Phase 87 parsing contracts in the same release.
-- **NFR-U4: Legacy elimination clarity** - Deprecated parsing paths must be removed with explicit migration diagnostics, not hidden fallback.
-
-### NFRs for Phase 88
-
-- **NFR-V1: Corpus completeness visibility** - Extract/retrieval artifacts must surface deterministic counts for confluence pages, linked pages, attachments, and code chunks.
-- **NFR-V2: Format diagnostics** - Attachment conversion/chunking diagnostics must include format-level and converter-level summaries with explicit failure samples.
-- **NFR-V3: Coverage boundedness** - Code extension expansion must remain bounded by existing max file-count/size/skip-dir constraints.
-- **NFR-V4: Non-regression** - Phase 84..87 behavior/contracts remain stable while Phase 88 enhancements are added.
-
-### NFRs for Phase 89
-
-- **NFR-W1: Score accuracy** - Repo-type-aware N/A marking must not alter scoring for rows that are genuinely applicable; only structurally inapplicable rows are marked N/A.
-- **NFR-W2: Performance non-regression** - Caching and parallelism changes must not degrade correctness or determinism of evaluation outputs.
-- **NFR-W3: Bounded concurrency** - Parallel row evaluation must respect LLM backend rate limits and remain configurable via CLI options.
-- **NFR-W4: Cache safety** - Model digest TTL cache and evidence retrieval cache must invalidate correctly on content changes; stale cache must never produce incorrect embeddings or evidence.
-- **NFR-W5: Non-regression** - Phase 84..88 behavior/contracts remain stable while Phase 89 enhancements are added.
-- **NFR-W6: Skill efficiency** - Optional-skill rows must avoid discovery/automation churn and prefer minimal high-yield skill reads only when required by evidence gaps.
-- **NFR-W7: Telemetry parity** - Skill usage diagnostics must be complete for every row (including timeout/degraded rows) and consistent across tool-first/prompt-backend traces.
-- **NFR-W8: Evidence-target compliance visibility** - Docs/code target execution compliance must be measurable and must always include explicit audited skip reasons when a target path is not executed.
-
-### NFRs for Phase 90
-
-- **NFR-X1: Hierarchy determinism** - Department, project, Bitbucket project, and repo keys must be deterministic across parse/extract/state commands.
-- **NFR-X2: Hard-cut semantics** - Deprecated flat selector aliases (especially Bitbucket project key -> project resolution) must be removed, not silently preserved.
-- **NFR-X3: Migration safety** - Schema/data migration must be idempotent and auditable, with explicit counts for migrated and quarantined records.
-- **NFR-X4: Non-regression** - Phase 84..89 workflow, retrieval, and quality contracts remain stable while hierarchy contracts are introduced.
-
-### NFRs for Phase 91
-
-- **NFR-Y1: CLI behavioral non-regression** - CLI decomposition must preserve all existing command behavior, option parsing, and output formatting without any user-visible regression.
-- **NFR-Y2: Module isolation** - Each CLI submodule must be independently importable without circular dependencies or shared mutable state.
-- **NFR-Y3: Error specificity** - Hardened exception handlers must catch the narrowest applicable exception type and include structured context (error type, source module, recovery action).
-- **NFR-Y4: Scoring determinism** - Profile-based weight adjustment, gating conditions, and coverage tracking must produce deterministic outputs for identical inputs.
-- **NFR-Y5: Non-regression** - Phase 84..90 workflow, retrieval, hierarchy, and quality contracts remain stable while Phase 91 enhancements are added.
-
-## Phase 89 Functional Requirements — Unified Audit Quality + Skill Effectiveness + Performance Optimization (Package W: FR-185..FR-206)
-
-### High Priority — Score Quality
-
-#### FR-185: Repo-Type-Aware Checklist Scoping
-
-The workflow must classify the target repository by type (frontend, backend, infrastructure, fullstack) based on heuristics (file extensions, framework markers, directory structure) and mark structurally inapplicable checklist rows as N/A with a deterministic reason code.
-
-**Acceptance Criteria:**
-- AC-185.1: Repository classification heuristic assigns one of `frontend`, `backend`, `infrastructure`, `fullstack` based on language/framework markers in the repo file tree.
-- AC-185.2: A configurable scoping policy maps repo types to sets of checklist row IDs (or check-category patterns) that are structurally inapplicable (e.g., Docker/Helm/CI-CD/DB-schema/Grafana rows for frontend repos).
-- AC-185.3: Rows matched by the scoping policy are marked `status=N/A` with `reason_code=repo_type_excluded` and `repo_type=<detected>` in the checklist output, without invoking LLM evaluation.
-- AC-185.4: N/A rows are excluded from aggregate score denominator so they do not deflate the pass rate.
-- AC-185.5: A `--repo-type {auto|frontend|backend|infra|fullstack}` CLI option allows manual override of the detected type. Default is `auto`.
-- AC-185.6: Scoping diagnostics are emitted in runtime logs and checklist metadata (`repo_type_detected`, `rows_scoped_na`, `scoping_policy_version`).
-
-### Medium Priority — Performance
-
-#### FR-186: Skip Ollama Probe in Lexical-Only Mode
-
-When `retrieval_mode=lexical` (lexical-only), the embedding pipeline must skip the Ollama connectivity probe (`embed_texts()` test call) that currently executes unconditionally in `vector_index.py`.
-
-**Acceptance Criteria:**
-- AC-186.1: `vector_index.py` initialization skips `embed_texts()` probe call when `lexical_only=True` or `retrieval_mode=lexical`.
-- AC-186.2: No Ollama network calls occur during extract-docs when retrieval mode is lexical-only.
-- AC-186.3: Hybrid and vector modes continue to execute the probe as before (non-regression).
-
-#### FR-187: Model Digest TTL Cache
-
-The embedding pipeline must cache the model digest returned by Ollama `/api/show` with a configurable TTL (default: 5 minutes) to avoid redundant network calls across sequential snapshot operations within the same extract-docs run.
-
-**Acceptance Criteria:**
-- AC-187.1: `_fetch_model_digest` result is cached in-memory with a configurable TTL (default 300s).
-- AC-187.2: Subsequent calls within TTL return the cached digest without network I/O.
-- AC-187.3: Cache is invalidated when TTL expires or when the model name changes.
-- AC-187.4: Cache hit/miss is logged at DEBUG level for diagnostics.
-
-#### FR-188: Parallel Row Evaluation Within Batches
-
-The workflow batch executor must support concurrent row evaluation, dispatching up to N rows (configurable, default: 1 for backward compatibility) to the LLM backend simultaneously within a single batch.
-
-**Acceptance Criteria:**
-- AC-188.1: A `--row-concurrency <N>` CLI option controls the maximum number of concurrent row evaluations per batch. Default is `1` (sequential, backward compatible).
-- AC-188.2: When `row-concurrency > 1`, rows within a batch are dispatched concurrently using asyncio tasks with a semaphore-bounded concurrency limit.
-- AC-188.3: Row-level checkpointing, diagnostics, and error handling remain correct under concurrent execution (no shared mutable state corruption).
-- AC-188.4: Batch-level timing diagnostics report both wall-clock and cumulative row times for concurrency efficiency measurement.
-- AC-188.5: Rate-limit errors from the LLM backend trigger exponential backoff per-row without failing the entire batch.
-
-#### FR-189: Evidence Retrieval Cache Within Batches
-
-The evidence retrieval layer must cache retrieved evidence chunks within a batch scope so that identical evidence pages appearing in top-N results for multiple rows are fetched and parsed only once.
-
-**Acceptance Criteria:**
-- AC-189.1: A batch-scoped evidence cache (keyed by chunk ID or document ID) stores retrieved evidence content.
-- AC-189.2: Subsequent rows within the same batch reuse cached evidence instead of re-fetching from Postgres or disk.
-- AC-189.3: Cache is cleared between batches to bound memory usage.
-- AC-189.4: Cache hit/miss counts are emitted in batch timing diagnostics.
-
-### Low Priority — Robustness
-
-#### FR-190: Configurable Default Batch Size
-
-The workflow must support a configurable default batch size higher than the current default of 5, with a recommended default of 8.
-
-**Acceptance Criteria:**
-- AC-190.1: Default batch size is changed from 5 to 8.
-- AC-190.2: `--batch-size` CLI option continues to override the default.
-- AC-190.3: No behavioral regression when explicit `--batch-size 5` is passed.
-
-#### FR-191: Non-Retryable Confluence Error Marker Expansion
-
-The Confluence crawl/fetch error classifier must include "not permitted" as a non-retryable error marker in addition to existing markers (403, "forbidden", "unauthorized").
-
-**Acceptance Criteria:**
-- AC-191.1: The non-retryable error classifier regex/set includes `"not permitted"` (case-insensitive).
-- AC-191.2: Pages returning "not permitted" errors are marked as permanently failed after the first attempt, skipping retry delays.
-- AC-191.3: Existing non-retryable markers continue to function (non-regression).
-
-#### FR-192: Permission-Denied Page Visibility in Doctor Crawl-Status
-
-The `doctor crawl-status` diagnostic command must surface pages that failed with permission-denied errors as a distinct category in its output.
-
-**Acceptance Criteria:**
-- AC-192.1: `crawl-status` output includes a `permission_denied_pages` section listing pages with 403/forbidden/unauthorized/not-permitted errors.
-- AC-192.2: Count of permission-denied pages is included in the summary totals.
-- AC-192.3: Each entry includes `page_id`, `title` (if available), and `error_message`.
-
-#### FR-193: Streamed Post-Processing Pipeline
-
-The workflow post-processing phase (Excel export, Postgres upload, report generation) must begin processing completed batches as they finish, rather than waiting for all rows to complete before starting export.
-
-**Acceptance Criteria:**
-- AC-193.1: Post-processing pipeline accepts incremental batch results and begins export/upload for completed batches while remaining batches are still evaluating.
-- AC-193.2: Final report assembly waits for all batches and post-processing tasks to complete before producing the aggregate checklist artifact.
-- AC-193.3: Post-processing timing diagnostics report overlap savings vs sequential baseline.
-- AC-193.4: When `row-concurrency=1` and `batch-size=5` (legacy defaults), behavior is equivalent to current sequential post-processing (non-regression).
-
-### High Priority — Skill Effectiveness + Telemetry Parity
-
-#### FR-194: Optional Skill Hard-Gate for `skills_needed=false`
-
-Rows interpreted as `skills_needed=false` must default to strict skill suppression so optional skill calls do not add avoidable latency or discovery-only churn.
-
-**Acceptance Criteria:**
-- AC-194.1: For `skills_needed=false`, prompt-tool skill discovery calls (`list_skills`, `load_skill`) are disabled by default.
-- AC-194.2: `run_skill_script` is disabled by default for `skills_needed=false`.
-- AC-194.3: At most one fallback `read_skill_resource` is allowed only when docs/code evidence targets are not yet grounded and a deterministic gap predicate is met.
-- AC-194.4: Suppression mode and fallback trigger reason are emitted in retrieval trace and runtime logs.
-
-#### FR-195: Minimal Effective Skill Sequence for `skills_needed=true`
-
-Rows interpreted as `skills_needed=true` must follow a minimal execution-first skill sequence that prioritizes high-yield skill resources over discovery loops.
-
-**Acceptance Criteria:**
-- AC-195.1: The first skill action for `skills_needed=true` is execution-class (`read_skill_resource` preferred) against objective-ranked skills.
-- AC-195.2: `list_skills` is only permitted when available-skill metadata is absent in current context.
-- AC-195.3: `run_skill_script` is allowed only for explicitly script-oriented objectives (for example `analyze_tooling_automation_guidance`) or explicit operator override.
-- AC-195.4: If no effective skill call is produced, policy-corrective retry is attempted once with narrowed skill focus and explicit diagnostics.
-
-#### FR-196: Low-Yield Skill Churn Guard (Including Execution Calls)
-
-The non-progress guard must detect and stop low-yield skill churn even when execution calls are present but ineffective.
-
-**Acceptance Criteria:**
-- AC-196.1: Guard policy evaluates skill effective-rate and requirement-coverage contribution, not only raw execution-call presence.
-- AC-196.2: Rows with high skill-call volume and low effective-rate trigger `skill_low_yield_churn` stop reason (or equivalent deterministic reason code).
-- AC-196.3: Guard output includes observed skill call counts, effective counts, and effective-rate in row diagnostics.
-- AC-196.4: Corrective guidance constrains subsequent retries to minimal allowed skill paths.
-
-#### FR-197: Unified Skill Telemetry Contract and Degraded-Row Completeness
-
-Skill telemetry must be unified and complete across tool-first and prompt-backend traces, including timeout/degraded rows.
-
-**Acceptance Criteria:**
-- AC-197.1: `row_skill_policy_effectiveness` is emitted for every checklist row, including timeout/degraded paths.
-- AC-197.2: `skill_tool_usage` merges all observed skill interactions with explicit source attribution (`tool_first_loop` vs `prompt_tool_telemetry`) and no double counting.
-- AC-197.3: `tool_call_summary` falls back to timeout/degraded progress snapshot when main loop trace is unavailable so counters are not silently zeroed.
-- AC-197.4: Runtime diagnostics include coverage counters for missing/complete skill-policy events.
-
-#### FR-198: Evidence-Target Execution Compliance Gate
-
-When requirement interpretation declares docs/code targets, execution traces must include at least one concrete action per target or an explicit audited skip reason.
-
-**Acceptance Criteria:**
-- AC-198.1: If `docs_target=true`, row trace must include at least one docs path action (`search_docs` or `read_doc_chunk`) or an explicit skip reason code.
-- AC-198.2: If `code_target=true`, row trace must include at least one code path action (`search_code` or `read_code_file`) or an explicit skip reason code.
-- AC-198.3: Checklist/retrieval diagnostics include aggregate compliance metrics (`docs_target_compliance`, `code_target_compliance`) for the run.
-- AC-198.4: Rows lacking both execution and audited skip reason are marked with deterministic quality warning diagnostics.
-
-#### FR-199: Skill Recommendation Quality Loop
-
-Skill recommendation ranking must prioritize measured effectiveness and requirement objective fit instead of broad fallback expansion.
-
-**Acceptance Criteria:**
-- AC-199.1: Recommendation ranking combines objective priority with recent effective-rate signals by skill and objective family.
-- AC-199.2: Broad fallback expansion is deferred until top-ranked skills fail to produce effective results.
-- AC-199.3: Recommendation diagnostics expose chosen skills and ranking rationale in row trace metadata.
-
-#### FR-200: Default Skill Controls for Canonical Checklist Runs
-
-Canonical checklist runs must default to a minimal, high-signal skill profile with explicit operator overrides.
-
-**Acceptance Criteria:**
-- AC-200.1: Default canonical checklist mode uses minimal skill policy (`skills_allow_script_execution=false` and constrained allowlist).
-- AC-200.2: Operator controls allow mode override (for example minimal/balanced/full) without changing baseline deterministic defaults.
-- AC-200.3: Per-row skill budget limits are deterministic and surfaced in diagnostics.
-- AC-200.4: Legacy behavior remains available through explicit override flags/env settings for backward compatibility.
-
-### High Priority — Evidence Quality + Observability (From Log Analysis)
-
-#### FR-201: Doc Search Tool Bias for Non-Code Categories
-
-The agentic tool selection must bias toward `search_docs` for checklist categories that are primarily documentation-oriented (governance, process, compliance, architecture) rather than defaulting to code-first search patterns.
-
-**Acceptance Criteria:**
-- AC-201.1: A category-to-tool-bias mapping classifies checklist categories as `docs_primary`, `code_primary`, or `balanced` based on category semantics.
-- AC-201.2: For `docs_primary` categories, the agentic system prompt instructs the LLM to prioritize `search_docs` and `read_doc_chunk` before code tools.
-- AC-201.3: `search_docs` call rate for `docs_primary` rows increases from baseline (~18%) to ≥60% without degrading code-category row quality.
-- AC-201.4: Tool bias policy is emitted in retrieval trace metadata per row for diagnostics.
-
-#### FR-202: Evidence Anti-Concentration (Diversity Penalty)
-
-The evidence retrieval pipeline must prevent the same source file or document from dominating top-k results across multiple rows within a run, ensuring diverse evidence grounding.
-
-**Acceptance Criteria:**
-- AC-202.1: A diversity penalty is applied during evidence ranking when the same document/file ID appears in top-k results for more than a configurable threshold of rows (default: 30% of evaluated rows).
-- AC-202.2: The penalty reduces the ranking score of over-represented sources, promoting alternative evidence into top-k positions.
-- AC-202.3: Anti-concentration metrics (`top_source_concentration_ratio`, `unique_sources_per_row_avg`) are emitted in run-level retrieval diagnostics.
-- AC-202.4: Diversity penalty is configurable and can be disabled via `--no-evidence-diversity` CLI flag for backward compatibility.
-
-#### FR-203: Evidence Truncation Mitigation
-
-The evidence context pipeline must use smarter ranking before truncation to ensure the most relevant evidence survives the `max_evidence_chars` budget, rather than relying on naive position-based truncation.
-
-**Acceptance Criteria:**
-- AC-203.1: Evidence chunks are ranked by relevance score before applying the `max_evidence_chars` budget cut, so highest-relevance chunks are preserved.
-- AC-203.2: Default `max_evidence_chars` is increased from 8000 to 12000 for canonical checklist runs (operator-overridable).
-- AC-203.3: Truncation diagnostics include `pre_truncation_chunk_count`, `post_truncation_chunk_count`, and `relevance_score_range` for truncated vs retained chunks.
-- AC-203.4: Rows where truncation removes >50% of evidence chunks by count are flagged with `evidence_truncation_severe` quality warning.
-
-#### FR-204: Unverified Reference Corrective Retry
-
-Rows where more than 50% of cited evidence references are unverified (not grounded in retrieved evidence) must trigger a single corrective retry with narrowed evidence focus.
-
-**Acceptance Criteria:**
-- AC-204.1: After initial row evaluation, if `unverified_ref_ratio > 0.5`, one corrective retry is dispatched with explicit instruction to ground citations in available evidence.
-- AC-204.2: Corrective retry uses the same evidence context but adds explicit grounding constraint in the LLM prompt.
-- AC-204.3: Retry outcome replaces the original evaluation only if the `unverified_ref_ratio` improves; otherwise the original is preserved.
-- AC-204.4: Retry diagnostics include `original_unverified_ratio`, `retry_unverified_ratio`, and `retry_accepted` boolean in row trace metadata.
-
-#### FR-205: Evaluation Stability Metric and Cross-Repo Divergence Flag
-
-The workflow must compute an evaluation stability metric for checklist rows and flag rows with high cross-repo divergence (>50% score variance when the same checklist is applied to repos within the same project).
-
-**Acceptance Criteria:**
-- AC-205.1: When `--stability-runs N` is used (N ≥ 2), per-row stability is computed as `1 - (std_dev / max_possible_range)` and emitted in checklist metadata.
-- AC-205.2: Rows with stability < 0.5 are flagged with `low_stability` quality warning in checklist output.
-- AC-205.3: When multi-repo project runs are available, cross-repo divergence is computed for each check ID and rows with >50% score divergence across repos are flagged with `cross_repo_divergent` marker.
-- AC-205.4: Stability and divergence metrics are included in the run-level quality summary for trend monitoring.
-
-#### FR-206: LLM Token Usage Instrumentation
-
-Runtime log events for LLM calls must include token usage metrics (prompt tokens, completion tokens, total tokens) for cost analysis and budget monitoring.
-
-**Acceptance Criteria:**
-- AC-206.1: Every LLM call event in runtime logs includes `prompt_tokens`, `completion_tokens`, and `total_tokens` fields when available from the LLM backend response.
-- AC-206.2: Run-level summary includes aggregate token usage (`total_prompt_tokens`, `total_completion_tokens`, `total_tokens`) across all LLM calls.
-- AC-206.3: Per-row token usage is emitted in row diagnostics for cost attribution.
-- AC-206.4: Token metrics gracefully degrade to `null` when the LLM backend does not return usage metadata (no errors or crashes).
-
-#### FR-206A: Bitbucket-Native Repo Identifier Precedence
-
-Repository identity resolution must prefer Bitbucket-native identifiers over project-prefixed fallback keys for deterministic repo-scoped retrieval and workflow execution.
-
-**Acceptance Criteria:**
-- AC-206A.1: `repo_storage_key` derivation prefers `bitbucket_project_key + bitbucket_repo_key` when available, then explicit `repo_storage_key`, then project-prefixed fallback.
-- AC-206A.2: `parse-repo`, `workflow`, and `analyze --repo-path` resolve the same canonical repo key for the same repository.
-- AC-206A.3: State-side loader/upsert paths apply the same precedence so persisted repo identity remains consistent across `sync repos`, `extract-docs`, and readiness checks.
-- AC-206A.4: Seed repo disambiguation prefers Bitbucket-native identity when multiple equal-rank matches exist.
-- AC-206A.5: State manifest/loaders must not retain synthetic `bitbucket.local/projects/local/...` placeholders in project mapping; local placeholders are dropped and non-local placeholders are canonicalized to the project Bitbucket origin when available.
-
-#### FR-206B: Retrieval Snapshot Compatibility for Repo-Key Migration
-
-Repo-key migration from legacy project-prefixed identifiers to Bitbucket-native `local-*` keys must preserve retrieval snapshot reuse and avoid unnecessary full re-embedding.
-
-**Acceptance Criteria:**
-- AC-206B.1: Retrieval refresh planning for repo scope must resolve previous snapshots in deterministic order: exact repo key, legacy repo alias key, then project-scope fallback.
-- AC-206B.2: When chunk IDs change but chunk content hash is unchanged, planner marks chunks as reusable and avoids re-embedding those chunks.
-- AC-206B.3: Snapshot embedding copy path supports reuse by `content_hash` (not only `chunk_id`) for unchanged chunks.
-- AC-206B.4: Legacy non-canonical model-digest values (for example old Ollama modelfile payloads) do not force full re-embedding, while canonical hash digest mismatches still force full re-embedding.
-
-## Phase 90 Functional Requirements — Hierarchy Model Hard-Cut (Package X: FR-207..FR-216)
-
-### High Priority — Canonical Hierarchy Contract
-
-#### FR-207: Department-First Registry Model
-
-Registry parsing must produce a first-class department hierarchy where projects are explicit children of departments, not only a flat project list with optional department metadata.
-
-**Acceptance Criteria:**
-- AC-207.1: Registry payload includes `departments[]` with deterministic `department_storage_key`, `department_name`, `department_page_id`, and `projects[]`.
-- AC-207.2: Every discovered project belongs to exactly one department node in the payload (or explicit `department=unassigned` bucket with audited reason).
-- AC-207.3: Project discovery retains Confluence ancestry path so department assignment is derived from tree structure rather than title-only heuristics.
-- AC-207.4: Department assignment is persisted in centralized state and surfaced consistently across `parse-registry`, `parse-project`, and `state` commands.
-
-#### FR-208: Explicit Project Membership and Identity
-
-Project identity must be canonicalized independently from Bitbucket project aliases, with deterministic project-level selectors and zero cross-project alias contamination.
-
-**Acceptance Criteria:**
-- AC-208.1: Canonical project selectors are limited to `project_storage_key`, Confluence page ID, and project title/page URL aliases.
-- AC-208.2: Bitbucket project keys must not be accepted as project selectors in `parse-project`.
-- AC-208.3: Selector collisions produce deterministic error diagnostics rather than auto-selection.
-- AC-208.4: Real selector sanity checks verify `parse-project --project MAP|WEBAPPV2|SUPERAPP` no longer resolves implicitly to Mini app.
-
-#### FR-209: Bitbucket Project as First-Class Entity
-
-Each project must expose nested Bitbucket-project groupings, with repositories modeled under their owning Bitbucket project.
-
-**Acceptance Criteria:**
-- AC-209.1: Project payload includes `bitbucket_projects[]` where each group has canonical `bitbucket_project_key` and `repos[]`.
-- AC-209.2: `repos[]` under each Bitbucket project must carry deterministic `repo_storage_key`, `bitbucket_repo_key/repo_slug`, and lineage metadata.
-- AC-209.3: Flat `project.bitbucket[]` output is removed from canonical parse contracts (hard-cut), with all repo consumers switched to grouped structure.
-- AC-209.4: Grouping supports both `/projects/<KEY>/repos/<slug>` and `/scm/<key>/<slug>.git` URL forms.
-
-#### FR-210: Bitbucket-Native Repo Identity Hard-Cut
-
-Repository keys must be Bitbucket-native by default and reject ambiguous fallback identity as canonical outputs.
-
-**Acceptance Criteria:**
-- AC-210.1: Canonical repo identity format is `<bitbucket_project_key>:<bitbucket_repo_key>` whenever Bitbucket URL/metadata is available.
-- AC-210.2: Project-prefixed fallback keys remain only as migration aliases, not canonical emit keys for new parse/extract outputs.
-- AC-210.3: Entries missing both `bitbucket_project_key` and `repo_url` are quarantined with deterministic diagnostics instead of silently promoted.
-- AC-210.4: `parse-repo --all-repos` reports grouped repos with explicit native keys and migration alias (if present).
-
-### Medium Priority — State + CLI Migration
-
-#### FR-211: Parse Command Contract Isolation
-
-Parse commands must align to hierarchy boundaries and avoid cross-layer selector mixing.
-
-**Acceptance Criteria:**
-- AC-211.1: `parse-project` resolves project identity only (no Bitbucket project alias acceptance).
-- AC-211.2: `parse-repo` supports filtering by explicit `--bitbucket-project <KEY>` within a selected project.
-- AC-211.3: A project-level summary includes department and Bitbucket-project counts for operator diagnostics.
-- AC-211.4: Error messages include deterministic selector guidance (project selector vs Bitbucket-project selector).
-
-#### FR-212: Hierarchy Schema Migration in Postgres State
-
-Centralized state must evolve to persist hierarchy entities and relations explicitly.
-
-**Acceptance Criteria:**
-- AC-212.1: New state entities include `departments` and `bitbucket_projects` (or equivalent normalized relation) with FK links to projects/repos.
-- AC-212.2: Migration is idempotent and records migrated row counts and unresolved records.
-- AC-212.3: Existing project/repo records are backfilled into hierarchy tables using deterministic mapping rules.
-- AC-212.4: Migration diagnostics are available through a state/doctor command for operator verification.
-
-#### FR-213: Extract-Docs Hierarchy Persistence
-
-`extract-docs` must persist grouped Bitbucket metadata and department/project hierarchy as first-class artifacts.
-
-**Acceptance Criteria:**
-- AC-213.1: Extracted project artifacts persist `department_storage_key` and grouped `bitbucket_projects[]`.
-- AC-213.2: Project-level Bitbucket links and repo-level links are both retained with clear link-role metadata.
-- AC-213.3: Synthetic/local placeholder contamination is sanitized before grouping and persistence.
-- AC-213.4: Retrieval snapshot refresh consumes grouped repo metadata without flattening regressions.
-
-#### FR-214: Hierarchy Readiness and Doctor Diagnostics
-
-Readiness and doctor flows must surface hierarchy completeness/quality explicitly.
-
-**Acceptance Criteria:**
-- AC-214.1: `state readiness` includes `hierarchy_ready` and counters (`departments`, `projects`, `bitbucket_projects`, `repos`).
-- AC-214.2: `doctor crawl-status` (or dedicated hierarchy audit command) reports hierarchy anomalies: unassigned projects, ungrouped repos, selector collisions.
-- AC-214.3: Diagnostics include deterministic samples for remediation with bounded output size.
-- AC-214.4: Workflow preflight fails fast when hierarchy completeness is below configured minimums.
-
-### Low Priority — Rollout and Verification
-
-#### FR-215: Deterministic Hierarchy Keys and Config Boundary
-
-Hierarchy normalization must be deterministic and follow existing sensitive/non-sensitive config boundaries.
-
-**Acceptance Criteria:**
-- AC-215.1: Key normalization for department/project/bitbucket-project/repo is centralized and reused across parser/state/CLI.
-- AC-215.2: Non-sensitive hierarchy tuning stays in `~/.vds/config/*.toml`; sensitive tokens/credentials remain in `~/.vds/.env`.
-- AC-215.3: Deterministic defaults are documented for reuse in future project onboarding to avoid remapping rework.
-
-#### FR-216: Phase 90 Real-Run Verification Gates
-
-Phase 90 closure includes real-run verification proving hierarchy correctness on production Confluence/Bitbucket mappings.
-
-**Acceptance Criteria:**
-- AC-216.1: Real run on root registry (`pageId=88716675`) persists department/project/bitbucket-project/repo hierarchy with no cross-project contamination.
-- AC-216.2: Real run on Mini app (`pageId=88718882`) produces grouped Bitbucket projects (`MAP`, `WEBAPPV2`, `SUPERAPP`) with correct repo assignments.
-- AC-216.3: Selector isolation checks pass (`parse-project --project MAP` fails with guidance; project selector still resolves correctly).
-- AC-216.4: Spec/docs/tasks alignment checks remain green after hierarchy contract migration.
-- AC-216.5: Project-scoped `doctor hierarchy-status` and `state readiness` report `anomaly_count=0` and `hierarchy_ready=true` for closure dataset `3-2-1-4-mini-app`.
-
-#### FR-31 (Legacy): Row Consistency Corrective Retry
-
-When a row evaluation response contains reasoning/evidence contradictions, the evaluator must run one corrective retry with contradiction-specific instructions.
-
-**Acceptance Criteria:**
-- AC-31.1: Contradiction detection is based on reasoning text versus cited evidence references.
-- AC-31.2: One corrective retry is attempted when contradiction confidence exceeds threshold.
-- AC-31.3: Retry behavior is controlled by `--row-consistency-retry-mode {off|auto|always}`.
-- AC-31.4: Retry diagnostics include original contradiction marker and whether retry was accepted.
-
-## Phase 91 Functional Requirements — CLI Modularization + Scoring Completeness + Error Handling Hardening (Package Y: FR-217..FR-227)
-
-### High Priority — CLI Decomposition
-
-#### FR-217: CLI Command-Group Submodule Extraction
-
-The monolithic `cli.py` (18,957 lines) must be decomposed into dedicated command-group submodules, each containing the command handlers for a single functional area.
-
-**Acceptance Criteria:**
-- AC-217.1: Command handlers for workflow, parse, state, doctor, sync, extract, and analyze are extracted into dedicated submodule files (`workflow_cmds.py`, `parse_cmds.py`, `state_cmds.py`, `doctor_cmds.py`, `sync_cmds.py`, `extract_cmds.py`, `analyze_cmds.py`).
-- AC-217.2: Each submodule is independently importable with no circular dependencies on `cli.py` or other submodules.
-- AC-217.3: All existing CLI tests pass without modification against the decomposed structure.
-
-#### FR-218: CLI Shared Infrastructure Module
-
-Shared CLI infrastructure (global options, output formatting, callback helpers, error context builders) must be extracted into a common module reusable by all command-group submodules.
-
-**Acceptance Criteria:**
-- AC-218.1: A `cli_common.py` module provides global option decorators, JSON/table output formatters, error context helpers, and callback infrastructure.
-- AC-218.2: All command-group submodules import shared infrastructure from `cli_common.py` without duplication.
-- AC-218.3: The residual `cli.py` is a thin Typer router (<500 lines) that wires submodule sub-apps.
-
-#### FR-219: CLI Decomposition Regression Gate
-
-The CLI decomposition must be verified for behavioral non-regression and import isolation.
-
-**Acceptance Criteria:**
-- AC-219.1: A submodule import isolation test validates each command-group submodule is importable independently.
-- AC-219.2: CLI entry-point smoke tests cover every registered command group through the thin router.
-- AC-219.3: Full existing CLI test suite passes green with zero behavioral regression.
-
-### Medium Priority — Scoring Completeness
-
-#### FR-220: Profile-Based Weight Adjustment
-
-Checklist row scoring must apply profile-based weight adjustments sourced from the active audit profile configuration.
-
-**Acceptance Criteria:**
-- AC-220.1: `checklist_generator.py` reads weight scaling factors from the active audit profile.
-- AC-220.2: Different profile configurations produce measurably different weight-adjusted scores for the same row.
-- AC-220.3: Default profile produces weights identical to current behavior (backward compatibility).
-
-#### FR-221: Score Gating Conditions
-
-Checklist row scoring must support configurable gating conditions that cap or floor scores based on evidence quality thresholds.
-
-**Acceptance Criteria:**
-- AC-221.1: A gating condition model defines score cap/floor rules based on configurable predicates (e.g., minimum evidence count, minimum confidence).
-- AC-221.2: When a gating condition triggers, the capped/floored score and trigger reason are recorded in row metadata.
-- AC-221.3: Gating conditions are opt-in and default to no gating (backward compatibility).
-
-#### FR-222: Row Coverage Tracking
-
-Checklist row evaluation must track and report corpus-hit coverage ratio diagnostics.
-
-**Acceptance Criteria:**
-- AC-222.1: Coverage ratio (evidence chunks used / evidence chunks available) is computed per row.
-- AC-222.2: Coverage diagnostics are emitted in checklist metadata per row.
-- AC-222.3: Aggregate coverage statistics are included in workflow summary.
-
-### Medium Priority — Error Handling
-
-#### FR-223: Exception Audit and Classification
-
-Broad exception catches across the codebase must be audited and classified as intentional recovery vs. error-swallowing.
-
-**Acceptance Criteria:**
-- AC-223.1: An annotated exception audit identifies all `except Exception` and bare `except:` statements in the top 5 largest source files.
-- AC-223.2: Each catch is classified as intentional-recovery (justified), error-swallowing (must fix), or boundary-guard (acceptable at system boundary).
-- AC-223.3: The audit is recorded as a durable artifact for tracking remediation progress.
-
-#### FR-224: Specific Exception Type Replacement
-
-Error-swallowing broad exception catches must be replaced with specific exception types in the top source files.
-
-**Acceptance Criteria:**
-- AC-224.1: Error-swallowing `except Exception` catches in CLI submodules and `state/store.py` are replaced with specific exception types.
-- AC-224.2: Error-swallowing `except Exception` catches in `engine/llm_row_evaluator.py`, `agents/agentic_investigator.py`, and `state/__init__.py` are replaced with specific exception types.
-- AC-224.3: Broad exception catches reduced by ≥50% in top 5 largest source files compared to pre-Phase-91 baseline.
-
-#### FR-225: Structured Error Logging for Hardened Handlers
-
-All hardened exception handlers must emit structured error logs with error type, context, and recovery action.
-
-**Acceptance Criteria:**
-- AC-225.1: Hardened exception handlers emit structured log entries with `error_type`, `source_module`, `context`, and `recovery_action` fields.
-- AC-225.2: Structured error logs are machine-parseable via runtime log infrastructure.
-- AC-225.3: Regression tests verify error handling paths emit expected structured log fields.
-
-### Medium Priority — Skill Fallback
-
-#### FR-226: Broad Fallback Skill Expansion
-
-When top-ranked skills fail or produce insufficient evidence, the skill selection pipeline must expand to effectiveness-ranked fallback candidates.
-
-**Acceptance Criteria:**
-- AC-226.1: Skill fallback triggers when the primary skill read returns insufficient evidence or errors.
-- AC-226.2: Fallback candidates are ranked by historical effectiveness scores from skill telemetry.
-- AC-226.3: Fallback telemetry records trigger count, candidate ranking, and success rate per row.
-- AC-226.4: Fallback depth is bounded (max 2 candidates) to prevent unbounded churn.
-
-### Low Priority — Baseline Sync
-
-#### FR-227: Phase 91 Test Baseline and Spec Sync
-
-Spec/docs/AGENTS must be updated with Phase 91 version, status, and test baseline.
-
-**Acceptance Criteria:**
-- AC-227.1: Test baseline in spec files matches current `pytest tests --collect-only` count.
-- AC-227.2: `validate-spec-sync` and `check-alignment` pass green after all Phase 91 changes.
-- AC-227.3: AGENTS.md version/status/test baseline reflects Phase 91 scope.
-
-### Package Z - Intelligent Preparation + Metadata Completeness (Phase 92)
-
-> **Clean-break implementation**: Phase 92 is a hard-cut modernization. Legacy full-scan, force-reembed, and opt-out refresh paths are **removed**, not preserved. All new metadata fields are **required**, not Optional.
-
-#### High Priority — Incremental Preparation (Clean Break)
-
-##### FR-228: DiffDetector-Only Code Chunking
-
-Extract-docs code chunking must use `DiffDetector` as the **only** code file discovery path, replacing the legacy `_iter_repo_code_files()` full-scan (`cli_impl.py:902-926`). The `_CODE_CORPUS_MAX_FILES_PER_REPO` constant (`cli_impl.py:853`) is removed.
-
-**Acceptance Criteria:**
-- AC-228.1: `_build_repo_code_chunk_records` (`cli_impl.py:928`) is rewritten to use `DiffDetector(repo_root).get_changes_since_commit(head_commit_sha)` as the file discovery path.
-- AC-228.2: `head_commit_sha` is stored in a dedicated required TEXT column in `audit_state.repos` and persisted after every successful code chunking run.
-- AC-228.3: On first extraction (no stored `head_commit_sha`), DiffDetector diffs against an empty tree (`git diff --name-status 4b825dc642cb6eb9a060e54bf8d69288fbee4904...HEAD`) to enumerate all tracked files.
-- AC-228.4: The legacy `_iter_repo_code_files()` function (`cli_impl.py:902-926`) and `_CODE_CORPUS_MAX_FILES_PER_REPO` constant (`cli_impl.py:853`) are **deleted**.
-- AC-228.5: Unchanged file chunk records are carried forward from the previous snapshot (lookup from Postgres `corpus_chunks`), not re-chunked.
-- AC-228.6: Chunking diagnostics include `diff_based: true`, `skipped_unchanged_count`, `changed_file_count`, and `head_commit_sha` in `code_chunking` stats.
-
-##### FR-229: Snapshot-Level Early Exit (Mandatory)
-
-Retrieval snapshot refresh must short-circuit when the computed content hash of incoming chunks matches the latest snapshot's content hash. There is no opt-out — early exit is the only path when content is unchanged.
-
-**Acceptance Criteria:**
-- AC-229.1: `plan_retrieval_refresh` (`state/store.py:4858`) compares incoming aggregate content hash against latest `retrieval_snapshots.content_hash` for the same `snapshot_key` before planning chunk-level diffs.
-- AC-229.2: When content hashes match, plan returns `early_exit: true` with zero changed chunks; `refresh_project_index` (`evidence/vector_index.py:43`) short-circuits before embedding client instantiation (line 100).
-- AC-229.3: Early-exit diagnostics are logged with `retrieval_refresh_early_exit` event including `snapshot_key`, `content_hash`, and elapsed time.
-- AC-229.4: Early-exit completion time is < 100ms for prepared repos with no content changes.
-
-##### FR-230: Document-Level Chunking Skip (Version-Gated)
-
-Document chunking must track per-document Confluence page version and skip re-chunking for unchanged pages. The `version_tag` column (`store.py:789`) is made NOT NULL and populated for all documents.
-
-**Acceptance Criteria:**
-- AC-230.1: `corpus_documents.version_tag` (`store.py:789`) is changed from `TEXT NULL` to `TEXT NOT NULL DEFAULT ''` and populated with Confluence `page.version.number` (for Confluence docs) or git commit SHA (for code files).
-- AC-230.2: Before chunking a document, the pipeline queries `corpus_documents.version_tag` by (`source_type`, `source_uri`) and skips chunking if versions match (`page_id` may be used as a Confluence convenience key only).
-- AC-230.3: Skipped documents reuse existing `corpus_chunks` rows without re-processing.
-- AC-230.4: Chunking diagnostics include `docs_skipped_unchanged_count` and `docs_rechunked_count`.
-- AC-230.5: Migration/backfill for existing NULL/empty versions is deterministic: legacy rows are backfilled to `''` before NOT NULL enforcement, then overwritten with authoritative version values on next successful refresh.
-
-##### FR-234: Legacy Code Removal (Clean Break)
-
-Legacy full-scan, force-reembed, opt-out refresh, and redundant CLI flag code paths are **deleted** from the codebase.
-
-**Acceptance Criteria:**
-- AC-234.1: `force_full_reembed` parameter is removed from `refresh_project_index` (`vector_index.py:56`), `plan_retrieval_refresh` (`store.py:4868`), and all 8+ caller sites in `extract_cmds.py`.
-- AC-234.2: The `effective_force_full` computation block (`store.py:4945-5005`) is replaced with: if embedding model changed, treat all chunks as changed (simpler, same outcome).
-- AC-234.3: `_iter_repo_code_files()` (`cli_impl.py:902-926`) and `_CODE_CORPUS_MAX_FILES_PER_REPO` constant (`cli_impl.py:853`) are **deleted**.
-- AC-234.4: `--refresh-retrieval-snapshot/--no-refresh-retrieval-snapshot` CLI flag (`extract_cmds.py:424-431`) and all downstream skip logic (`extract_cmds.py:2110-2119`, `2394-2415`) are **deleted**.
-- AC-234.5: `--incremental/--no-incremental` CLI flag (`extract_cmds.py:187-191`) and all downstream conditional code are **deleted**.
-- AC-234.6: `--crawl-mode` CLI flag (`extract_cmds.py:116-120`) and locked validation (`extract_cmds.py:595`) are **deleted** — `bulk` is hardcoded.
-- AC-234.7: `--emit-corpus` CLI flag (`extract_cmds.py:121-127`) and auto-force logic (`extract_cmds.py:609-615`) are **deleted** — corpus emission is always on.
-- AC-234.8: `--fetch-policy` CLI flag (`extract_cmds.py:159-163`), `--cache-mode` flag (`extract_cmds.py:154-157`), and their validation blocks (`extract_cmds.py:625-630`) are **deleted** — `if-changed` and `read-write` are hardcoded.
-- AC-234.9: TOML config `_apply_default_*` overrides for removed options (`extract_cmds.py:548-591`) are cleaned up to remove dead keys.
-- AC-234.10: `force-refresh` fetch policy no longer triggers `force_full_reembed` side effect (`extract_cmds.py:2419`) because both concepts are gone.
-
-#### Medium Priority — Metadata Completeness (Required Fields)
-
-##### FR-231: Confluence Metadata Population Completeness (Required Fields)
-
-Confluence metadata fields (`page_id`, `space_key`, `page_title`, `source_uri`) must flow through the entire extraction pipeline as **required** fields. Legacy aliases (`confluence_url`, `source_url`) must map to canonical `source_uri`. `source_uri` in `corpus_documents` (`store.py:787`) is made NOT NULL.
-
-**Acceptance Criteria:**
-- AC-231.1: `corpus_documents.source_uri` (`store.py:787`) is changed to `TEXT NOT NULL` — every document must have a traceable source URI.
-- AC-231.2: Confluence-sourced chunks must include `space_key` and canonical `source_uri` in chunk metadata (legacy aliases normalize to `source_uri` at ingestion boundaries).
-- AC-231.3: `CorpusChunk` model (`chunking.py:55-63`) is extended with required first-class fields: `source_type: str`, `source_uri: str`, `version_tag: str`, `content_hash: str` — not buried in `metadata` dict.
-- AC-231.4: Code file chunks include `commit_sha` in metadata and `version_tag` set to commit SHA.
-- AC-231.5: A verification query confirms zero NULL `page_id` for Confluence-sourced `corpus_documents` and zero NULL `source_uri` for all `corpus_documents`.
-- AC-231.6: Migration/backfill for existing NULL `source_uri` is deterministic before NOT NULL enforcement (for example `unknown://legacy/<document_id>`), then corrected to authoritative URIs on subsequent refresh.
-
-##### FR-232: Extract-Docs CLI Simplification (Clean Break)
-
-Extract-docs CLI is aggressively simplified: redundant flags, manual override modes, and stale-data paths are **removed**. The system always checks for changes, always refreshes snapshots, and always produces fresh data.
-
-**Removed CLI options and modes:**
-
-| Option | Current Location | Values Removed | Reason |
-|--------|-----------------|----------------|--------|
-| `--fetch-policy` | `extract_cmds.py:159-163` | `force-refresh`, `offline` | System always checks for changes; no manual staleness override |
-| `--cache-mode` | `extract_cmds.py:154-157` | `off`, `read`, `refresh` | Only `read-write` is valid; `off` defeats caching, `read` produces stale data, `refresh` is redundant |
-| `--refresh-retrieval-snapshot` | `extract_cmds.py:424-431` | Entire flag removed | Always on during `--execute`; no opt-out |
-| `--incremental` | `extract_cmds.py:187-190` | Entire flag removed | Always incremental; no opt-out |
-| `--crawl-mode` | `extract_cmds.py:116-120` | Entire flag removed | Already locked to `bulk` only (line 595); option is dead code |
-| `--emit-corpus` | `extract_cmds.py:121-127` | Entire flag removed | Auto-forced True when refresh is True (lines 609-615); always True in clean-break |
-
-**Retained CLI options:**
-
-| Option | Values | Reason |
-|--------|--------|--------|
-| `--verify-cache-integrity` | Boolean | Optional diagnostic, keep |
-| `--include-bitbucket` | Boolean (default True) | Controls Bitbucket credential requirement, keep |
-| `--include-linked-confluence-pages` | Boolean (default True) | Controls link crawl depth, keep |
-
-**Acceptance Criteria:**
-- AC-232.1: `--fetch-policy` is removed from CLI and validation logic; canonical behavior `if-changed` is hardcoded.
-- AC-232.2: `--cache-mode` is removed from CLI and validation logic; canonical behavior `read-write` is hardcoded.
-- AC-232.3: Removed flags (`--refresh-retrieval-snapshot`, `--incremental`, `--crawl-mode`, `--emit-corpus`, `--fetch-policy`, `--cache-mode`) no longer appear in `--help` output.
-- AC-232.4: TOML config overrides for removed options (`~/.vds/config/extract-docs.toml` keys `cache_mode`, `sync_policy`, `fetch_policy`, `refresh_retrieval_snapshot`, `incremental`, `crawl_mode`, `emit_corpus`) are removed from `_apply_default_*` calls (lines 548-591).
-- AC-232.5: Running `extract-docs --execute` without any flags always produces fresh snapshots with incremental chunking, read-write caching, and retrieval refresh.
-- AC-232.6: Documentation, AGENTS.md, and automation scripts/runbooks are updated to reflect the simplified CLI surface (no removed flag references remain).
-
-##### FR-233: Agent Instruction Template Citation Update
-
-Agent system prompt instructions must direct evaluating agents to cite `page_id` and canonical `source_uri` when recording evidence references from Confluence-sourced evidence (`source_url` accepted as compatibility alias in prompts/tool hints).
-
-**Acceptance Criteria:**
-- AC-233.1: `build_agentic_system_prompt` (`llm/prompts/instruction_templates.py:179`) or `_grounding_rules` include an instruction to cite `page_id` and canonical `source_uri` in evidence references.
-- AC-233.2: The instruction specifies using `ref_type='confluence_page'` for Confluence-sourced evidence.
-- AC-233.3: `record_evidence_refs` tool usage hint (`instruction_templates.py:81`) is updated to include `page_id` and canonical `source_uri` field guidance (`source_url` alias accepted).
-
-### Non-Functional Requirements (Phase 92)
-
-#### NFR-Y7: Incremental Refresh Performance
-
-Incremental extract-docs reruns must meet defined performance targets for prepared repos.
-
-**Acceptance Criteria:**
-- AC-Y7.1: No-change rerun of extract-docs completes in < 30 seconds for prepared repos (snapshot early exit + diff-based skip).
-- AC-Y7.2: Changed-file-only rerun completes in < 50% of full extraction retrieval-refresh time (`performance.retrieval_refresh_duration_ms`; fallback to total duration when refresh timing is unavailable).
-- AC-Y7.3: Performance metrics are emitted in extract-docs `--json-only` output for verification.
-- AC-Y7.4: No-change fast-path coverage includes linked-page resolution bypass, cache-first markdown reuse for unchanged Confluence pages, and canonical Bitbucket repo-identity deduplication to minimize fixed pre-refresh overhead.
-
-
-
-## Package T - Evaluation Effectiveness Hardening (Phase 86)
-
-### Design Decisions
-
-- **Decision 61**: Preserve Phase 84/85 contracts; Phase 86 remains additive hardening.
-- **Decision 62**: Docs/source balance is intent-aware with explicit docs utilization or audited skip reason.
-- **Decision 63**: Investigation depth is adaptive and bounded.
-- **Decision 64**: Infra/deployment ranking is intent-sensitive.
-- **Decision 65**: Silent synthesis failures are disallowed; explicit failure diagnostics required.
-- **Decision 66**: Published evidence refs are precision-filtered to final cited refs.
-- **Decision 67**: Evidence-context budgeting is adaptive with truncation diagnostics.
-- **Decision 68**: Recommendation completeness and verification coverage are quality gates.
-- **Decision 69**: For skills_needed=false rows with strong grounding, prompt-stage tools are suppressed (skills_optional_suppressed_grounded) and synthesis is constrained to existing evidence context/refs with explicit suppression diagnostics.
-
-## Package U - Individual Project Parsing + Docs/Guides/Skills Sync + Legacy Cleanup (Phase 87, Complete)
-
-### Design Implementation
-
-- `parse-project` and `parse-repo` accept direct project-page selectors (`pageId` URL and display URL forms) via selector normalization and alias matching.
-- `parse-registry` emits explicit project-page-scope guidance diagnostics when no projects are discovered.
-- Parse payloads expose deterministic repository coverage summaries (`total/ready/skipped/failed` + category breakdown).
-- AGENTS/docs/guides/skills references are synchronized with the hard-cut preparation order and direct parsing workflow.
-- Parse-path legacy silent behavior (project-page selector ambiguity) is replaced by explicit operator guidance.
-
-### Design Decisions
-
-- **Decision 70**: Introduce selector-normalization pipeline for project URL -> page_id -> canonical project_storage_key resolution.
-- **Decision 71**: Add explicit command-level mode diagnostics to distinguish `registry-root` parsing vs `project-page` parsing.
-- **Decision 72**: Add deterministic repo-coverage summary block to parse outputs for direct runbook usage.
-- **Decision 73**: Expand Package U scope to `FR-169..FR-176` and bind docs/guides/skills updates as release gates.
-- **Decision 74**: Keep AGENTS workflow runbook snippets aligned with direct-project selector contracts.
-- **Decision 75**: Enforce spec/docs/guides/skills sync validation before Phase 87 closure.
-- **Decision 76**: Remove obsolete parse fallback branches (hard-cut) and emit deterministic replacement guidance.
-
-## Package V - Corpus Completeness + Format Coverage (Phase 88, Complete)
-
-### Design Implementation
-
-- Expand repo code chunk extension allowlist for mobile/native ecosystems so canonical code corpus includes Flutter/mobile platform source files when present.
-- Append converted attachments from both direct crawl and linked fallback flows into project corpus prior to Postgres retrieval snapshot refresh.
-- Preserve chunk dedupe semantics (chunk-id + content-hash) and stale-pruning behavior in Postgres state refresh.
-- Emit deterministic chunk diagnostics in extract/retrieval outputs for:
-  - source-type distribution (`confluence_page`/`confluence_attachment`/`code_file`),
-  - attachment format distribution (`xlsx`, `pdf`, `docx`, etc.),
-  - converter usage and conversion failures.
-- Resolve extensionless attachment formats before conversion using media-type metadata and byte signatures (`png/jpeg/pdf/json/xml`) so draw.io `mxfile` payloads are classified and chunked.
-- For HTML inputs, treat external converter output as best-effort: if mojibake signatures are detected, auto-fallback to deterministic HTML parser output.
-- Emit deterministic attachment conversion summary in per-project extract output with:
-  - status totals (`found`/`partial`/`error`/`unknown`),
-  - `format_counts`, `converter_counts`, `format_status_counts`,
-  - bounded `failure_samples` for debugging.
-- Persist per-resource attachment diagnostics in `sync_metadata.artifacts[].resource_metadata` (including `source_format`, converter identity, discovery path, size, and conversion timing) for state-first triage and readiness analysis.
-- Ensure generated code chunk records carry explicit lineage `discovery=repo_chunking` at top-level + metadata.
-
-### Design Decisions
-
-- **Decision 77**: Treat corpus completeness as a first-class readiness quality dimension, not an implicit side effect.
-- **Decision 78**: Attachments are chunked based on converted markdown text with strict lineage propagation into metadata.
-- **Decision 79**: Code extension expansion remains bounded by existing file-count/size/skip-dir controls to avoid runtime blow-ups.
-- **Decision 80**: Format-level diagnostics are mandatory so operators can validate retrieval corpus quality before row analysis.
-- **Decision 81**: Attachment conversion quality is reported in extraction payloads, not inferred from corpus rows only.
-- **Decision 82**: Code chunk discovery lineage must be explicit to avoid ambiguous `unknown` diagnostics in downstream readiness/reporting.
-- **Decision 83**: Extensionless attachments are inferred before conversion to reduce false `(unknown)` partials and improve retrieval corpus completeness.
-- **Decision 84**: External HTML conversion quality is validated heuristically; mojibake output is rejected in favor of deterministic fallback conversion.
-- **Decision 85**: Attachment conversion diagnostics must be persisted in `sync_metadata` per artifact (not only aggregated summaries) so future runs can reuse and inspect quality telemetry directly from Postgres state.
-
-## Package W - Unified Audit Quality + Skill Effectiveness + Performance Optimization (Phase 89, Complete)
-
-### Design Implementation
-
-Observed run-quality baseline driving this package:
-- Miniapp-be run (`miniapp-be-e2e-20260226-171208`) showed optional-skill churn on `CL-028` (`12` skill calls, `3` effective) despite `skills_needed=false`.
-- One required-skill row (`CL-047`) missed skill-policy event coverage due timeout/degraded path.
-- Docs-target execution compliance remained low (`12/51`) versus code-target compliance (`48/55`), requiring explicit execution/skip controls.
-
-#### Repo-Type-Aware Checklist Scoping (FR-185)
-
-- Introduce a `RepoTypeClassifier` component that detects repo type from file-tree heuristics:
-  - `frontend`: presence of `package.json` + framework markers (React/Vue/Angular/Flutter) + absence of backend markers (Dockerfile, Helm, server frameworks).
-  - `backend`: presence of server framework files (Spring, Django, FastAPI, Go main), Dockerfiles, database migration scripts.
-  - `infrastructure`: primarily Terraform, Ansible, Helm charts, CI/CD pipelines.
-  - `fullstack`: significant presence of both frontend and backend markers.
-- Scoping policy is a declarative mapping (configurable YAML/dict) from repo type to sets of check IDs or category patterns that are structurally N/A.
-  - Default policy for `frontend`: N/A for Docker/Helm/CI-CD pipeline/DB schema/Grafana dashboard/monitoring-infra categories.
-  - Default policy for `backend`/`fullstack`: no rows scoped out (all applicable).
-  - Default policy for `infrastructure`: N/A for frontend-specific UI/UX/accessibility categories.
-- N/A rows bypass LLM evaluation entirely and are injected into checklist output with `status=N/A`, `reason_code=repo_type_excluded`.
-- Aggregate score denominator excludes N/A rows so pass rate reflects only applicable rows.
-- CLI `--repo-type` option allows explicit override with `auto` default.
-
-#### Repo Identifier Consistency (FR-206A Follow-up)
-
-- Repo identity derivation uses a deterministic precedence chain:
-  - `bitbucket_project_key + bitbucket_repo_key` (or values parsed from `repo_url`) as primary.
-  - Explicit `repo_storage_key` when primary Bitbucket identity is unavailable.
-  - `project_storage_key + repo_slug` fallback only as last resort for non-Bitbucket entries.
-- CLI resolver paths (`parse-repo`, workflow repo-path inference, analyze preflight repo resolution) and state loader paths (`_repo_storage_key_from_entry`) must share this precedence contract.
-- Seed selector tie-breakers prefer entries with Bitbucket-native identity to prevent fallback project-prefixed keys from winning ambiguous matches.
-- Existing legacy project-prefixed keys remain readable for backward state compatibility but are no longer preferred for new run resolution.
-- State loaders sanitize synthetic `bitbucket.local/projects/local/...` placeholders during artifact recording and manifest hydration to prevent cross-project mapping contamination; non-local placeholder repo links are canonicalized to the discovered Bitbucket origin when possible.
-
-#### Retrieval Snapshot Migration Compatibility (FR-206B Follow-up)
-
-- Repo-scoped retrieval refresh resolves previous snapshot baseline with deterministic precedence:
-  - exact repo key snapshot (`repo_key == selected repo key`);
-  - legacy alias snapshot derived from `local-*` key (`<project>-<slug>`);
-  - project-scope fallback snapshot (`repo_key IS NULL`) only when repo baselines are unavailable.
-- Changed-chunk planning reuses prior snapshot hashes in two modes:
-  - direct match (`chunk_id` + `content_hash`);
-  - migration-safe match (`content_hash` only) when chunk IDs drift after repo-key normalization.
-- Embedding copy SQL for unchanged chunks matches either `chunk_id` or `content_hash`, so reused vectors are preserved across identifier migrations without re-embedding unchanged text.
-- Model-digest compatibility gate treats legacy non-canonical digest payloads as compatible to prevent forced full re-embed churn from historical snapshot metadata; canonical hash digest mismatches still trigger `force_full_reembed`.
-
-#### Embedding Pipeline Optimization (FR-186, FR-187)
-
-- Guard Ollama probe (`embed_texts()` connectivity check) behind `lexical_only` flag check at `vector_index.py` initialization.
-- Wrap `_fetch_model_digest` with an in-memory LRU/TTL cache (default TTL: 300s). Use `functools.lru_cache` with a timestamp wrapper or `cachetools.TTLCache`.
-- Cache key: model name string. Cache invalidation: TTL expiry or model name change.
-
-#### Concurrent Row Evaluation (FR-188)
-
-- Extend `BatchExecutor` to accept `row_concurrency` parameter.
-- When `row_concurrency > 1`, dispatch row evaluation coroutines via `asyncio.Semaphore(row_concurrency)` within the batch loop.
-- Row-level state mutations (checkpoint writes, diagnostics accumulation) use per-row isolation; batch-level aggregation occurs after all rows complete.
-- Rate-limit errors (429, 529) trigger per-row exponential backoff with jitter; other errors fail the individual row without blocking siblings.
-- Batch timing diagnostics emit `wall_clock_ms`, `cumulative_row_ms`, `concurrency_efficiency_ratio`.
-
-**Detailed Architecture for Parallel Row Evaluation:**
-
-1. **Concurrency primitive**: Use `asyncio.Semaphore(N)` where N is `--row-concurrency` (default: 1 for backward compatibility). Rows within `_process_batch()` are dispatched via `asyncio.gather(*[sem_limited_evaluate(row) for row in batch_rows])`.
-
-2. **Sync-to-async bridge**: The current synchronous `match_row()` evaluation function is wrapped with `asyncio.to_thread(match_row, row, ...)` to avoid blocking the event loop. Each thread gets its own call context.
-
-3. **Connection pooling**: Replace the single `psycopg.connect()` used for checkpoint/state writes with `psycopg_pool.ConnectionPool(min_size=1, max_size=row_concurrency+1)`. The pool is initialized once per workflow run and passed to the batch executor. Each concurrent row evaluation acquires a connection from the pool for checkpoint writes and releases it after completion.
-
-4. **Per-row telemetry isolation**: `_last_prompt_tool_telemetry` and other mutable row-level diagnostics accumulators are isolated per-row (thread-local or explicit per-row dict). Batch-level aggregation merges per-row results after `asyncio.gather()` completes. No shared mutable state between concurrent row evaluations.
-
-5. **Phased implementation plan**:
-   - **Phase 1 (Foundation)**: Add `--row-concurrency` CLI option, `asyncio.Semaphore` dispatch in `_process_batch()`, `asyncio.to_thread()` wrapper for `match_row()`, per-row diagnostics isolation. Default concurrency=1 preserves sequential behavior.
-   - **Phase 2 (Connection pool)**: Replace `psycopg.connect()` with `psycopg_pool.ConnectionPool`. Add pool health diagnostics (acquired/idle/waiting counts) to batch timing output.
-   - **Phase 3 (Hardening)**: Add per-row exponential backoff for rate-limit errors (429/529) with configurable max retries. Add concurrency efficiency benchmarks for miniapp-core/miniapp-be at concurrency=2 and concurrency=3. Verify checkpoint correctness under concurrent writes.
-
-6. **CLI option**: `--row-concurrency <N>` (default: 1). Validate N ≥ 1. When N=1, execution path is identical to current sequential behavior (non-regression).
-
-#### Evidence Retrieval Cache (FR-189)
-
-- Introduce `BatchEvidenceCache` (dict keyed by chunk/document ID) initialized at batch start, cleared at batch end.
-- Evidence retrieval path checks cache before Postgres/disk fetch; on miss, fetches and stores.
-- Cache hit/miss counters are accumulated in batch timing diagnostics.
-
-#### Batch Size Default (FR-190)
-
-- Change default `batch_size` from 5 to 8 in CLI option definition.
-- No other behavioral changes; `--batch-size` override continues to work.
-
-#### Non-Retryable Error Expansion (FR-191)
-
-- Add `"not permitted"` to the non-retryable error marker set in the Confluence fetch error classifier (case-insensitive match).
-- Existing markers (`403`, `"forbidden"`, `"unauthorized"`) remain unchanged.
-
-#### Permission-Denied Crawl Visibility (FR-192)
-
-- Extend `doctor crawl-status` output model to include a `permission_denied_pages` list.
-- Filter state-persisted crawl errors matching permission-denied patterns into this dedicated section.
-- Include count in summary totals alongside existing success/error/pending counts.
-
-#### Streamed Post-Processing (FR-193)
-
-- Refactor post-processing pipeline to accept batch results incrementally.
-- Start Excel row append + Postgres row upload as each batch completes, using an asyncio producer/consumer pattern.
-- Final report assembly (aggregate checklist, metadata, delta) waits for all batches and all post-processing tasks.
-- Timing diagnostics report `post_processing_overlap_ms` (time saved vs sequential baseline).
-
-#### Optional Skill Hard-Gate + Minimal Sequence Policy (FR-194, FR-195, FR-200)
-
-- Extend prompt toolset filtering so `skills_needed=false` defaults to strict skill suppression:
-  - disable `list_skills`, `load_skill`, `run_skill_script`,
-  - permit fallback `read_skill_resource` only under deterministic evidence-gap predicate.
-- For `skills_needed=true`, enforce execution-first path:
-  - objective-ranked skill selection,
-  - direct `read_skill_resource` before discovery expansion,
-  - `run_skill_script` allowed only for script-oriented objectives or explicit override.
-- Add explicit skill operating modes (`minimal`, `balanced`, `full`) with canonical checklist defaulting to `minimal`.
-- Keep legacy behavior available through explicit operator override flags/env settings.
-
-#### Low-Yield Skill Churn Guard (FR-196)
-
-- Upgrade non-progress guard to evaluate:
-  - skill effective-rate,
-  - requirement-coverage contribution of skill turns,
-  - high-volume low-yield execution patterns (not only discovery-only churn).
-- Emit deterministic churn reason code (`skill_low_yield_churn`) and constrain retries to narrower skill plan.
-- Include skill counts/effective-rate directly in row diagnostics for verification.
-
-#### Unified Skill Telemetry Completeness (FR-197)
-
-- Unify `tool_first_loop` and `prompt_tool_telemetry` skill observations into one `skill_tool_usage` surface with source attribution.
-- Emit `row_skill_policy_effectiveness` for every row, including timeout/degraded rows.
-- Preserve timeout/degraded counter integrity in projection (`tool_call_summary` fallback to timeout progress snapshot).
-- Add run-level diagnostics for skill-policy event coverage completeness.
-
-#### Evidence-Target Execution Compliance (FR-198)
-
-- Add compliance gate in row quality layer:
-  - docs target requires docs-path execution or explicit audited skip reason,
-  - code target requires code-path execution or explicit audited skip reason.
-- Store aggregate run metrics (`docs_target_compliance`, `code_target_compliance`) for trend monitoring.
-- Flag rows that miss both execution and audited skip reason with deterministic quality-warning diagnostics.
-
-#### Effectiveness-Driven Skill Ranking (FR-199)
-
-- Extend skill recommendation ranking with measured effectiveness signals by objective family and repo type.
-- Delay broad fallback skill expansion until top-ranked candidates fail to produce effective contributions.
-- Persist ranking rationale in row trace metadata for operator debugging and policy tuning.
-
-#### Doc Search Tool Bias for Non-Code Categories (FR-201)
-
-- Introduce a category-to-tool-bias mapping that classifies checklist categories into `docs_primary`, `code_primary`, or `balanced` based on category semantics.
-- For `docs_primary` categories (governance, process, compliance, architecture, documentation quality), the agentic system prompt includes explicit instruction to prioritize `search_docs` and `read_doc_chunk` before code-oriented tools.
-- For `code_primary` categories (code quality, testing, security implementation), current code-first behavior is preserved.
-- For `balanced` categories, no bias is applied (tool selection remains LLM-driven).
-- Tool bias policy is emitted per-row in retrieval trace metadata (`tool_bias_policy: docs_primary|code_primary|balanced`).
-- Target: increase `search_docs` call rate for `docs_primary` rows from ~18% baseline to ≥60%.
-
-#### Evidence Anti-Concentration — Diversity Penalty (FR-202)
-
-- Add a diversity penalty mechanism to the evidence ranking pipeline that reduces ranking scores for source documents/files that appear in top-k results across more than a configurable threshold of rows (default: 30%).
-- The penalty is applied at the run level: a `source_frequency_map` is maintained as rows are evaluated, and sources exceeding the threshold receive a multiplicative score penalty (default: 0.7x).
-- Anti-concentration metrics (`top_source_concentration_ratio`, `unique_sources_per_row_avg`) are computed at run completion and emitted in run-level retrieval diagnostics.
-- A `--no-evidence-diversity` CLI flag disables the penalty for backward compatibility and A/B comparison.
-
-#### Evidence Truncation Mitigation (FR-203)
-
-- Refactor evidence context assembly to rank chunks by relevance score before applying the `max_evidence_chars` budget, ensuring highest-relevance chunks survive truncation.
-- Increase default `max_evidence_chars` from 8000 to 12000 for canonical checklist runs (existing `--max-evidence-chars` CLI option continues to work as override).
-- Emit enhanced truncation diagnostics: `pre_truncation_chunk_count`, `post_truncation_chunk_count`, `relevance_score_range` (min/max of truncated vs retained).
-- Flag rows where truncation removes >50% of chunks with `evidence_truncation_severe` quality warning in row diagnostics.
-
-#### Unverified Reference Corrective Retry (FR-204)
-
-- After initial row evaluation, compute `unverified_ref_ratio` from evidence verification results.
-- When `unverified_ref_ratio > 0.5`, dispatch one corrective retry with the same evidence context but an explicit grounding constraint in the LLM prompt (instruction to cite only from provided evidence chunks).
-- The retry result replaces the original only if `unverified_ref_ratio` improves; otherwise the original evaluation is preserved.
-- Retry diagnostics (`original_unverified_ratio`, `retry_unverified_ratio`, `retry_accepted`) are recorded in row trace metadata.
-- This mechanism is independent of the existing `--row-consistency-retry-mode` (which handles reasoning/evidence contradiction); both can fire on the same row.
-
-#### Evaluation Stability Metric (FR-205)
-
-- When `--stability-runs N` (N ≥ 2) is used, compute per-row stability as `1 - (std_dev / max_possible_range)` across repeated evaluations.
-- Rows with stability < 0.5 receive `low_stability` quality warning in checklist output.
-- When multi-repo project runs are available (same checklist applied to multiple repos in same project), compute cross-repo divergence per check ID. Rows with >50% score divergence are flagged with `cross_repo_divergent` marker.
-- Stability and divergence metrics are included in run-level quality summary (`stability_summary`, `cross_repo_divergence_summary`).
-
-#### LLM Token Usage Instrumentation (FR-206)
-
-- Extend the LLM call wrapper to capture token usage from backend responses (`prompt_tokens`, `completion_tokens`, `total_tokens`).
-- Emit token usage in every LLM call runtime log event under `token_usage` key.
-- Compute run-level aggregate token metrics (`total_prompt_tokens`, `total_completion_tokens`, `total_tokens`) and emit in workflow summary.
-- Emit per-row token usage in row diagnostics for cost attribution by row/batch.
-- Token metrics gracefully degrade to `null` when backend does not return usage metadata (pydanticai model response may not always include usage for all providers).
-
-### Design Decisions
-
-- **Decision 83**: Repo-type classification is heuristic-based (file-tree analysis), not LLM-based, for determinism and zero-cost execution.
-- **Decision 84**: Checklist scoping policy is declarative and configurable, not hardcoded, to support evolving checklist templates.
-- **Decision 85**: N/A rows are excluded from score denominator to avoid penalizing repos for structurally inapplicable checks.
-- **Decision 86**: Parallel row evaluation uses asyncio semaphore (not thread pool) to stay within the existing async runtime model.
-- **Decision 87**: Evidence cache scope is per-batch (not per-run) to bound memory usage proportional to batch size.
-- **Decision 88**: Post-processing streaming uses producer/consumer pattern for clean separation of evaluation and export concerns.
-- **Decision 89**: Default batch size increase (5→8) is conservative; operators can tune higher via CLI option.
-- **Decision 90**: Model digest TTL cache is in-memory only; no disk persistence needed for a per-run optimization.
-- **Decision 91**: Optional skill paths are fail-closed by default in canonical checklist mode; fallback skill use is evidence-gap driven, not advisory-only.
-- **Decision 92**: Required skill rows prioritize execution-first minimal sequence over discovery-first sequence to reduce latency and churn.
-- **Decision 93**: Skill churn detection must account for low-yield execution loops, not only discovery-without-execution loops.
-- **Decision 94**: Skill telemetry completeness is a first-class quality contract, including timeout/degraded row coverage.
-- **Decision 95**: Target compliance enforcement (docs/code execution or audited skip reason) is required for deterministic row-quality diagnostics.
-- **Decision 96**: Skill ranking favors measured effectiveness over static fallback priority when runtime evidence is available.
-- **Decision 97**: Canonical checklist defaults prioritize deterministic minimal skill usage with explicit operator opt-in for broader skill behavior.
-- **Decision 98**: Doc search tool bias uses category-level classification (not per-row LLM reasoning) for deterministic and zero-overhead tool routing.
-- **Decision 99**: Evidence anti-concentration uses run-level source frequency tracking with multiplicative penalty, not per-batch, to catch cross-batch concentration patterns.
-- **Decision 100**: Evidence truncation mitigation prioritizes relevance-ranked preservation over position-based truncation to maximize signal per evidence budget.
-- **Decision 101**: Unverified reference corrective retry is independent of consistency retry (FR-31) — both can fire on the same row for different quality signals.
-- **Decision 102**: Evaluation stability metric reuses existing `--stability-runs` infrastructure and adds cross-repo divergence as an optional quality dimension.
-- **Decision 103**: LLM token instrumentation is best-effort (graceful null) because not all protocol backends guarantee usage metadata in every response.
-
-## Package X - Hierarchy Model Hard-Cut (Phase 90, Complete)
-
-### Design Implementation
-
-Current implementation state (Feb 27, 2026):
-- Implemented: `parse-project` selector domain now rejects Bitbucket project keys with boundary-aware diagnostics.
-- Implemented: `parse-repo` supports `--bitbucket-project` scoping and emits grouped Bitbucket project summaries.
-- Implemented: `parse-registry` output now includes deterministic `departments[]` summary with nested compact project entries.
-- Implemented: Confluence ancestry metadata and deterministic lineage-derived department assignment are persisted through parser -> state flows.
-- Implemented: normalized hierarchy schema entities (`departments`, `bitbucket_projects`) with FK links and idempotent backfill/mapping diagnostics.
-- Implemented: `state readiness` emits hierarchy counters, `hierarchy_ready`, and bounded anomaly samples.
-- Implemented: `doctor hierarchy-status` emits bounded hierarchy anomaly diagnostics for operator triage.
-- Closure status: project-scoped hierarchy diagnostics now report `anomaly_count=0` and `hierarchy_ready=true` for the closure dataset, with deterministic dedupe/normalization behavior validated in unit tests and real CLI runs.
-
-#### Canonical Hierarchy Graph (FR-207..FR-210)
-
-- Introduce explicit hierarchy entities:
-  - `DepartmentNode` -> `ProjectNode` -> `BitbucketProjectNode` -> `RepoNode`.
-- Registry parser records Confluence ancestry and assigns each project to one canonical department bucket.
-- Project payload contract migrates from flat `bitbucket[]` to grouped `bitbucket_projects[]`, each containing `repos[]`.
-- Repo canonical identity emits Bitbucket-native keys (`<bitbucket_project_key>:<bitbucket_repo_key>`) when URL/metadata is available.
-
-#### Parse Contract Hard-Cut (FR-208, FR-211)
-
-- `parse-project` selector domain is restricted to project selectors only:
-  - `project_storage_key`,
-  - Confluence page ID/URL aliases,
-  - canonical project titles.
-- Bitbucket project keys are removed from project selector aliases to eliminate collisions (`MAP|WEBAPPV2|SUPERAPP` -> Mini app contamination class).
-- `parse-repo` adds explicit Bitbucket-project scoping (`--bitbucket-project`) inside a selected project.
-- Error diagnostics become boundary-aware: project selector mismatch vs bitbucket-project selector mismatch.
-
-#### State Schema Evolution (FR-212, FR-213)
-
-- Add normalized hierarchy persistence:
-  - `departments` table/entity for canonical department nodes,
-  - `bitbucket_projects` table/entity scoped by project,
-  - deterministic FK links from repos to `bitbucket_projects`.
-- Migration/backfill pipeline:
-  - derive department membership from stored project ancestry/tree signals,
-  - derive Bitbucket project key from URL + existing metadata fields,
-  - quarantine unresolved rows with explicit diagnostics.
-- Extract-docs state artifact recording persists grouped hierarchy payloads directly (no flattening rewrite stage).
-
-#### Diagnostics and Readiness Gates (FR-214..FR-216)
-
-- Extend readiness payloads with hierarchy counters and `hierarchy_ready` flag.
-- Add hierarchy anomaly diagnostics:
-  - unassigned projects,
-  - ungrouped repos,
-  - selector collision candidates,
-  - quarantined unresolved repo links.
-- Closure verification requires real-run proofs on:
-  - root registry `pageId=88716675`,
-  - Mini app page `pageId=88718882`,
-  - selector isolation checks (`parse-project --project MAP` must fail with guidance).
-
-## Package Y - CLI Modularization + Scoring Completeness + Error Handling Hardening (Phase 91)
-
-Current implementation status:
-- Implemented: FR-217..FR-227 (all Phase 91 requirements complete).
-
-### CLI Decomposition (FR-217..FR-219)
-
-The monolithic `cli.py` decomposition is complete:
-
-- **Command-group submodules (implemented)**: `workflow_cmds.py`, `parse_cmds.py`, `state_cmds.py`, `doctor_cmds.py`, `sync_cmds.py`, `extract_cmds.py`, `analyze_cmds.py`, `misc_cmds.py`.
-- **Shared infrastructure module (complete)**: `cli_common.py` centralizes shared primitives (`app`, `sync_app`, `doctor_app`, `state_app`, `_cli_state`, `console`, `logger`, DSN/shared-env constants) and command modules consume shared infrastructure without monolithic fallbacks.
-- **Thin router target (complete)**: `cli.py` is now a thin router shim (12 lines) that delegates to `cli_impl.py`.
-
-Import isolation is verified by deterministic wiring coverage in `tests/unit/test_cli_decomposition_fr219.py` (23 passing tests).
-
-### Checklist Scoring Completeness (FR-220..FR-222)
-
-Three deferred scoring features in `checklist_generator.py` are implemented:
-
-- **Profile-based weight adjustment** (FR-220): Weight scaling factors sourced from the active audit profile, applied during row scoring. Default profile preserves current behavior.
-- **Score gating conditions** (FR-221): Configurable cap/floor rules triggered by evidence quality predicates (min evidence count, min confidence) with persisted `gating_applied`/`gating_reason` metadata in row artifacts. Default config is non-gating for backward-compatible scoring.
-- **Coverage tracking** (FR-222): Per-row coverage factor is persisted in row score breakdown; aggregate coverage diagnostics are emitted in `workflow-summary.json`.
-
-### Error Handling Hardening (FR-223..FR-225)
-
-This package area is complete:
-
-- **Exception audit** (FR-223): implemented; durable artifacts are available at `reports/verification/phase91_error_handling/phase91-exception-audit-20260227/` and finalize snapshot `.../phase91-exception-audit-20260227-finalize/`.
-- **Specific exception replacement** (FR-224): complete; top-5-file broad-catch count reduced from 117 to 56 (52.14% reduction, gate <=58 passed).
-- **Structured error logging** (FR-225): complete; hardened handlers emit structured fields (`error_type`, `source_module`, `context`, `recovery_action`) and are validated by targeted regression suites.
-
-WP-91.3 depends on WP-91.1 completion — CLI submodules must be split before exception hardening to avoid merge conflicts in the monolithic file.
-
-### Skill Fallback Robustness (FR-226)
-
-Implemented broad fallback path:
-
-- Fallback triggers when primary skill read returns insufficient evidence or errors.
-- Candidates ranked by historical effectiveness scores from existing skill telemetry.
-- Bounded depth (max 2 fallback candidates) prevents unbounded churn.
-- Telemetry records: trigger count, candidate ranking, success rate per row.
-
-### Design Decisions
-
-- **Decision 104**: Hierarchy model is first-class and normalized; implicit hierarchy inferred from flat metadata is not accepted as canonical.
-- **Decision 105**: Phase 90 is a hard-cut migration for parse contracts; ambiguous compatibility aliases are removed, not silently preserved.
-- **Decision 106**: Bitbucket-project grouping is canonical output shape for repos; flat project-level repo arrays are deprecated from parse outputs.
-- **Decision 107**: Project selector and Bitbucket-project selector domains must remain isolated to prevent cross-scope mis-resolution.
-- **Decision 108**: Repo identity precedence remains Bitbucket-native first; fallback keys are migration aliases only.
-- **Decision 109**: Schema/data migration must be idempotent and produce auditable diagnostics for migrated and quarantined records.
-- **Decision 110**: Hierarchy readiness is promoted to a preflight quality gate alongside docs/code chunk readiness.
-- **Decision 111**: Non-sensitive hierarchy tuning remains TOML-driven (`~/.vds/config/*`), while credentials stay in `~/.vds/.env`.
-- **Decision 112**: Real-run verification on production Confluence/Bitbucket mappings is mandatory before Phase 90 closure.
-- **Decision 113**: CLI decomposition is complete for Phase 91: `cli.py` remains a thin router and command-group handlers live in dedicated modules.
-- **Decision 114**: CLI submodules are importable and consume shared primitives from `cli_common.py`; regression gates enforce import isolation and command registration parity.
-- **Decision 115**: Profile-based weight adjustment defaults to identity scaling (no change) for backward compatibility; explicit profile config required to activate.
-- **Decision 116**: Error handling hardening targets error-swallowing catches only; intentional-recovery and boundary-guard catches are preserved.
-- **Decision 117**: Skill fallback depth is bounded at max 2 candidates to prevent unbounded churn; effectiveness ranking reuses existing telemetry infrastructure.
-
-## Package Z - Intelligent Preparation + Metadata Completeness (Phase 92)
-
-> **Clean-break implementation**: Phase 92 removes legacy full-scan, force-reembed, and opt-out refresh paths. All new metadata fields are required. DiffDetector is the only code file discovery path. Incremental refresh is the only mode.
-
-Current implementation status:
-- In progress: FR-232/FR-233 baseline cleanup and NFR-Y7 instrumentation are implemented; FR-228..FR-231/FR-234 remain open for full closure.
-
-### DiffDetector-Only Code Chunking (FR-228)
-
-The existing `DiffDetector` (`incremental/diff_detector.py:64`) provides a complete git diff API surface:
-- `get_current_commit()` (line 122): returns HEAD SHA.
-- `get_changes_since_commit(base_commit)` (line 136): returns `DiffResult` with `FileChange` list.
-- `_parse_diff_output()` (line 216): parses `--name-status` output with rename/copy detection.
-
-**Clean-break change**: `_iter_repo_code_files()` (`cli_impl.py:902-926`) and `_CODE_CORPUS_MAX_FILES_PER_REPO` (`cli_impl.py:853`) are **deleted**. The replacement:
-1. Store `head_commit_sha` in a dedicated required TEXT column in `audit_state.repos` (persisted after every extraction).
-2. On first extraction (no stored SHA), diff against Git empty tree (`4b825dc642cb6eb9a060e54bf8d69288fbee4904...HEAD`) to enumerate all tracked files.
-3. On subsequent extractions, diff against stored SHA to get only changed/added files.
-4. Unchanged file chunk records are carried forward from previous Postgres `corpus_chunks` snapshot.
-5. No fallback to full scan — DiffDetector failure is a hard error with structured logging.
-
-### Snapshot-Level Early Exit (FR-229, Mandatory)
-
-The retrieval refresh pipeline flows: `refresh_project_index` (`evidence/vector_index.py:43`) → `state_store.plan_retrieval_refresh` (`state/store.py:4858`) → `state_store.apply_retrieval_refresh`.
-
-Early exit is inserted at the `plan_retrieval_refresh` stage:
-1. Compute aggregate `content_hash` of all incoming chunks.
-2. Compare against `retrieval_snapshots.content_hash` for the latest snapshot with matching `snapshot_key`.
-3. If hashes match, return `early_exit: true` plan with zero changed chunks.
-4. `refresh_project_index` short-circuits before embedding client instantiation (line 100-108) when early exit is signaled.
-5. **No opt-out** — early exit is the only path when content is unchanged.
-
-Target: < 100ms for no-change reruns on prepared repos.
-
-### No-Change Preparation Fast Paths (Phase 92 follow-up)
-
-To reduce fixed overhead outside retrieval-refresh early exit, extract-docs now applies additional no-change fast paths:
-1. Linked Confluence page resolution/download is skipped when resume-state indicates unchanged crawl output (`completed=true`, `pages_fetched=0`, `pages_failed=0`, `pages_skipped_unchanged>0`).
-2. No-change root/child page handling reuses prepared cached markdown entries in offline mode, avoiding redundant Confluence fetch + reconversion.
-3. Bitbucket link processing deduplicates by canonical repo identity (`bitbucket_project_key:repo_slug`) so alias URLs are not processed repeatedly.
-4. Retrieval refresh planning uses exact `snapshot_key` lineage first; project-level fallback snapshots are consulted only when the exact key is absent.
-5. Repo `head_commit_sha` is persisted eagerly after diff planning (before long embedding refresh) and repo refresh targets are deduplicated by resolved `local_code_path` to avoid duplicate alias processing.
-
-Observed effect (Merchant + Mini App verification bundles): no-change reruns improved but still exceed AC-Y7.1 for full prepared runs. Merchant bundle `merchant-20260228-011021` improved to `34207ms` best selector-scoped run; Mini App remediation bundles reduced missing repo heads from `64 -> 22` (`miniapp-head-backfill-20260228-053449`) and then `22 -> 14` (`miniapp-postfix2-20260228-060821`). Follow-up validation `miniapp-postfix4-20260228-091636` confirmed state-aware repo target ranking + no-baseline skip path removes prior large fallback re-embeds (`repo_refresh.refreshed_repo_snapshots=1`, `repo_refresh.skipped_without_repo_baseline=16`) and removed missing-row head-persist warnings; active synced repos report `active_missing_head=0` (`head-count-active.json`). Remaining aggregate gap is stale/non-repo state rows that still keep `missing_head=14`; cleanup is tracked under TSK-920.41.
-
-### Document-Level Chunking Skip (FR-230)
-
-`corpus_documents.version_tag` (`store.py:789`) is changed from `TEXT NULL` to `TEXT NOT NULL DEFAULT ''`:
-- For Confluence documents: populated with `page.version.number` from Confluence API.
-- For code files: populated with git commit SHA from DiffDetector.
-
-During document chunking:
-1. Query `version_tag` by (`source_type`, `source_uri`) for the document (`page_id` is Confluence convenience only).
-2. Compare against current Confluence page `version` from material metadata.
-3. Skip chunking if versions match; reuse existing `corpus_chunks`.
-4. After chunking, update `version_tag` to current page version.
-
-### Legacy Code Removal (FR-234)
-
-**Deleted code paths** (clean-break inventory):
-
-| Item | Location | Action |
-|------|----------|--------|
-| `_iter_repo_code_files()` full scan | `cli_impl.py:902-926` | **Delete** — replaced by DiffDetector |
-| `_CODE_CORPUS_MAX_FILES_PER_REPO` | `cli_impl.py:853` | **Delete** — DiffDetector returns only changed files |
-| `force_full_reembed` parameter | `vector_index.py:56`, `store.py:4868` | **Delete** from all signatures |
-| `force_full_reembed` callers (8 sites) | `extract_cmds.py:2419,2440,2451,2551,2565,2663,2670,2717` | **Delete** all passthrough |
-| `effective_force_full` logic | `store.py:4945-5005` | **Replace** with: model change = all chunks changed |
-| `--refresh-retrieval-snapshot` flag | `extract_cmds.py:424-431` | **Delete** — always on |
-| `refresh_retrieval_snapshot_enabled` skip | `extract_cmds.py:2110-2119`, `2394-2415` | **Delete** — always refresh |
-| `--incremental/--no-incremental` flag | `extract_cmds.py:187-191` | **Delete** — always incremental |
-| `--fetch-policy` flag | `extract_cmds.py:159-163` | **Delete** — `if-changed` hardcoded |
-| `--cache-mode` flag | `extract_cmds.py:154-157` | **Delete** — `read-write` hardcoded |
-| `--crawl-mode` flag | `extract_cmds.py:116-120` | **Delete** — dead code, locked to `bulk` at line 595 |
-| `--emit-corpus` flag | `extract_cmds.py:121-127` | **Delete** — auto-forced True, always on |
-| Fetch policy validation | `extract_cmds.py:625-630` | **Delete** — no policy to validate |
-| Cache mode validation | `extract_cmds.py:625` | **Delete** — no mode to validate |
-| TOML overrides for removed options | `extract_cmds.py:548-591` | **Clean up** — remove dead `_apply_default_*` keys |
-
-### Confluence Metadata Flow (FR-231, Required Fields)
-
-Metadata fields are promoted from optional to **required**:
-- `corpus_documents.source_uri` (`store.py:787`): changed to `TEXT NOT NULL`.
-- `CorpusChunk` model (`chunking.py:55-63`): extended with required first-class fields `source_type`, `source_uri`, `version_tag`, `content_hash` instead of burying them in `metadata` dict.
-
-Flow enforcement:
-1. **Confluence API response** → material download artifacts: `page_id`, `space_key`, `page_title`, canonical `source_uri` preserved (`confluence_url`/`source_url` normalized as aliases).
-2. **Material artifacts** → chunking stage: all four fields propagated into chunk records.
-3. **Chunk records** → `ChunkRecord` (`evidence/vector_index.py:18`): `page_id` (line 25) and `source_uri` (line 27) already exist; `space_key` and `page_title` are required metadata.
-4. **Code file chunks**: `commit_sha` added to metadata; `version_tag` set to commit SHA.
-
-### Simplified CLI Surface (FR-232, FR-234)
-
-The extract-docs CLI is aggressively simplified. The clean-break removes all manual override modes and opt-out flags:
-
-**Removed CLI options (with rationale):**
-
-| Option | Location | Why Removed |
-|--------|----------|-------------|
-| `--fetch-policy` | `extract_cmds.py:159-163` | Only `if-changed` is valid; `force-refresh` and `offline` allow stale/missing data |
-| `--cache-mode` | `extract_cmds.py:154-157` | Only `read-write` is valid; `off` defeats caching, `read` produces stale, `refresh` is redundant |
-| `--refresh-retrieval-snapshot` | `extract_cmds.py:424-431` | Always on during `--execute`; the opt-out `--no-refresh-retrieval-snapshot` allowed silently stale snapshots |
-| `--incremental` | `extract_cmds.py:187-190` | Always incremental; `--no-incremental` caused full wasteful re-processing |
-| `--crawl-mode` | `extract_cmds.py:116-120` | Dead code — already locked to `bulk` at line 595 |
-| `--emit-corpus` | `extract_cmds.py:121-127` | Dead code — auto-forced True when refresh is on (lines 609-615) |
-
-**TOML config cleanup**: `_apply_default_*` overrides (`extract_cmds.py:548-591`) for removed options are deleted; remaining TOML tuning (e.g., `recursive_depth`, `crawl_max_nodes`, `linked_page_concurrency`) is preserved.
-
-**Retained options**: `--verify-cache-integrity`, `--include-bitbucket`, `--include-linked-confluence-pages`, and operational parameters (depth, concurrency, output paths).
-
-After cleanup, `extract-docs --execute` is the only invocation pattern: always `if-changed`, always `read-write` cache, always incremental, always refresh snapshots, always emit corpus.
-
-### Agent Instruction Citation Update (FR-233)
-
-Modification target: `_grounding_rules()` and `record_evidence_refs` usage hint in `instruction_templates.py`.
-
-Add explicit citation guidance:
-- When evidence comes from Confluence documents, cite `page_id` and canonical `source_uri` (Confluence page URL); `source_url` remains an accepted alias in tool hints.
-- Use `ref_type='confluence_page'` to distinguish Confluence-sourced evidence from code-sourced evidence.
-
-### Design Decisions
-
-- **Decision 118**: DiffDetector is the **only** code file discovery path; legacy `_iter_repo_code_files()` full-scan is deleted, not kept as fallback.
-- **Decision 119**: Snapshot early exit comparison uses aggregate content hash, not per-chunk comparison, for O(1) check. No opt-out.
-- **Decision 120**: Document-level chunking skip uses Confluence page version (integer), not content hash, for simplicity and alignment with Confluence API semantics.
-- **Decision 121**: Metadata fields are promoted to required first-class fields on `CorpusChunk` and schema columns, not buried in optional metadata dicts.
-- **Decision 122**: `force_full_reembed` concept is deleted entirely; model changes are handled by treating all chunks as changed (simpler, same outcome).
-- **Decision 123**: Performance gates (NFR-Y7) are verified via extract-docs `--json-only` timing output, not separate benchmarking infrastructure.
-- **Decision 124**: First extraction (no stored `head_commit_sha`) diffs against Git empty tree to enumerate all tracked files, eliminating the need for a separate first-run code path.
-- **Decision 125**: Extract-docs CLI flags that offer only one valid value (`--fetch-policy if-changed`, `--cache-mode read-write`) are deleted entirely; the canonical behavior is hardcoded. Flags that control credential/network scope (`--include-bitbucket`, `--include-linked-confluence-pages`) and operational parameters are retained.
-- **Decision 126**: TOML config overrides (`~/.vds/config/extract-docs.toml`) for removed CLI options are cleaned up; remaining tuning keys (depth, concurrency, linked page settings) are preserved.
-- **Decision 127**: Canonical Confluence URL field is `source_uri`; `confluence_url` and `source_url` are treated as legacy aliases normalized at ingestion and prompt layers.
-
-
-
-## Relevant Completed Task Index
-
-### Phase 66/70 Foundation References
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-660.70 | FR-83, FR-84, FR-90 | Complete | Agentic analysis quality carry-forward baseline retained for downstream phases. |
-| TSK-700.20 | FR-107..FR-110 | Complete | Phase 70 convergence verification closure artifact retained. |
-
-### Phase 71/72 Requirement Understanding and Unified Evaluator
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-710.12 | FR-111..FR-114 | Complete | Requirement-understanding implementation verification completed. |
-| TSK-710.13 | FR-115 | Complete | Phase 71 verification gate closure artifact retained. |
-| TSK-710.23 | FR-111..FR-115 | Complete | Post-convergence guardrail closure retained. |
-| TSK-720.10 | FR-120 | Complete | Dynamic tool-efficiency extension implemented. |
-| TSK-720.13 | FR-120 | Complete | Phase 72 extension verification closure completed. |
-
-### Phase 75 Quality Hardening
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-750.1 | FR-125..FR-129 | Complete | Non-retryable Confluence permission handling validated. |
-| TSK-750.4 | FR-125..FR-129 | Complete | Docs-evidence hygiene for tool paths implemented and verified. |
-| TSK-750.5 | FR-125..FR-129 | Complete | Docs retrieval quality recovery implemented and verified. |
-| TSK-750.6 | FR-125..FR-129 | Complete | Prompt repair eligibility guard for auth/transport failures implemented. |
-| TSK-750.7 | FR-125..FR-129 | Complete | Retrieval-trace sidecar diagnostics enrichment completed. |
-| TSK-750.8 | FR-125..FR-129 | Complete | Tool-error evidence salvage implemented. |
-| TSK-750.9 | FR-125..FR-129 | Complete | Docs-exhausted fallback quality guard completed. |
-| TSK-750.10 | FR-125..FR-129 | Complete | Verdict contradiction guard implemented. |
-| TSK-750.11 | FR-125..FR-129 | Complete | Final-score status alignment completed. |
-| TSK-750.12 | FR-125..FR-129 | Complete | Hybrid docs-first stagnation/churn reduction completed. |
-| TSK-750.13 | FR-125..FR-129 | Complete | Pricing/SLA route override + recovery closure completed. |
-| TSK-750.14 | FR-125..FR-129 | Complete | Recovery-search relevance precision hardening completed. |
-| TSK-750.15 | FR-125..FR-129 | Complete | Relevance precision matrix verification completed. |
-| TSK-750.16 | FR-125..FR-129 | Complete | Tool-error field normalization in retrieval trace completed. |
-
-### Phase 83 Evidence-Postgres Migration
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-830.1 | FR-143..FR-147 | Complete | MatcherProtocol extraction + Postgres evidence hard-cut path. |
-| TSK-830.9 | FR-143..FR-147 | Complete | Standalone migration verification closure completed. |
-| TSK-830.10 | FR-143..FR-147 | Complete | Removed --evidence-dir from all CLI commands; removed code_evidence_pack.json and local rglob fallbacks; strictly relying on PostgresEvidenceMatcher. |
-
-## Phase 86 - Evaluation Effectiveness Hardening (Package T: FR-161..FR-168)
-
-### WP-86.1: Source Balance + Adaptive Depth
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-860.1 | FR-161 | Complete | Docs-intent utilization policy + diagnostics implemented. |
-| TSK-860.2 | FR-162 | Complete | Bounded adaptive-depth expansion/stop telemetry implemented. |
-| TSK-860.3 | FR-161, FR-162 | Complete | Unit tests for docs-balance and adaptive-depth behavior passed. |
-
-### WP-86.2: Infra Ranking + Failure Visibility
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-860.4 | FR-163 | Complete | Infra/deployment intent ranking weighting implemented. |
-| TSK-860.5 | FR-163 | Complete | Top-k ranking factor diagnostics emitted in retrieval trace. |
-| TSK-860.6 | FR-164 | Complete | Non-silent failure propagation (`failure_stage`/`reason_code`) implemented. |
-| TSK-860.7 | FR-163, FR-164 | Complete | Regression tests for ranking/failure visibility completed. |
-
-### WP-86.3: Citation Precision + Context Budgeting
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-860.8 | FR-165 | Complete | Exploratory refs vs final cited refs split implemented. |
-| TSK-860.9 | FR-165 | Complete | Citation precision filtering + dedupe implemented. |
-| TSK-860.10 | FR-166 | Complete | Adaptive evidence-context budgeting + truncation telemetry implemented. |
-| TSK-860.11 | FR-165, FR-166 | Complete | Unit tests for citation precision and truncation policy passed. |
-
-### WP-86.4: Recommendation + Verification Completeness
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-860.12 | FR-167 | Complete | Recommendation fallback for non-pass rows enforced. |
-| TSK-860.13 | FR-168 | Complete | Verification taxonomy standardized for evidence anchors. |
-| TSK-860.14 | FR-168 | Complete | Verification coverage diagnostics in row/runtime/report artifacts implemented. |
-| TSK-860.15 | FR-167, FR-168 | Complete | Unit/integration coverage for recommendation + verification completed. |
-
-### WP-86.5: Verification, Non-Regression, and Spec Sync
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-860.16 | FR-161..FR-168 | Complete | Multi-row real verification matrix published with closure reruns. |
-| TSK-860.17 | FR-148..FR-168 | Complete | Phase 84/85 contract non-regression validation completed. |
-| TSK-860.18 | FR-161..FR-168 | Complete | Spec/docs/guides/skills sync after closure completed. |
-| TSK-860.19 | FR-161, FR-164 | Complete | Grounded optional-skill suppression (`skills_needed=false`) policy implemented. |
-| TSK-860.20 | FR-161..FR-168 | Complete | CL-028 timeout-churn follow-up verification closure completed. |
-| TSK-860.21 | FR-153..FR-160 | Complete | `sync repos`/`extract-docs` status-normalization alignment (`synced`/`complete`) and readiness-consistent state persistence hardening. |
-| TSK-860.22 | FR-153..FR-160 | Complete | Retrieval snapshot freshness hardening: stale `corpus_documents`/`corpus_chunks` rows are pruned for stable snapshot keys before reuse/copy logic. |
-
-### Acceptance Gates (Phase 86)
-
-- [x] Docs-relevant rows show docs-path utilization or explicit audited skip reason.
-- [x] Adaptive depth behavior remains observable and bounded.
-- [x] Infra-intent rows prioritize infra artifacts in ranked candidates.
-- [x] Silent synthesis failure class is eliminated with explicit failure diagnostics.
-- [x] Final checklist evidence refs are precision-filtered.
-- [x] Truncation diagnostics are present with improved prevalence.
-- [x] Recommendation completeness for non-pass rows is deterministic.
-- [x] Verification coverage diagnostics are explicit and deterministic.
-- [x] Grounded `skills_needed=false` rows avoid prompt-stage tool/skill churn and avoid timeout-degraded fallback (`skills_optional_suppressed_grounded`).
-
-## Phase 87 - Individual Project Parsing + Docs/Guides/Skills Sync + Legacy Cleanup (Package U: FR-169..FR-176, Complete)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-870.1 | FR-169 | Complete | Added selector normalization utility for project URL/page-ID/storage-key canonical resolution. |
-| TSK-870.2 | FR-169, FR-170 | Complete | Extended `parse-project` and `parse-repo` to accept direct Confluence project-page selectors. |
-| TSK-870.3 | FR-171 | Complete | Added explicit `parse-registry` diagnostics when input URL is project-page scope and zero projects are discovered. |
-| TSK-870.4 | FR-172 | Complete | Added deterministic repo coverage summaries to parse payloads (`total/ready/skipped/failed` + category breakdown). |
-| TSK-870.5 | FR-169..FR-172 | Complete | Added/updated unit and CLI tests for selector normalization and parse output contracts. |
-| TSK-870.6 | FR-169..FR-172 | Complete | Verified Miniapp direct project-page parsing against centralized state (`parse-project`/`parse-repo`). |
-| TSK-870.7 | FR-173 | Complete | Updated docs/guides references to align with direct-project selector behavior and hard-cut prep order. |
-| TSK-870.8 | FR-174 | Complete | Updated skills guidance (`skills/vds-scripts-skill/SKILL.md`) to align with direct-project parse workflow. |
-| TSK-870.9 | FR-175 | Complete | Replaced silent project-page ambiguity with explicit parse-registry migration guidance diagnostics. |
-| TSK-870.10 | FR-176 | Complete | Verified closure with targeted tests, lint checks, real direct-project selector runs, and spec sync checks. |
-
-### Acceptance Gates (Phase 87)
-
-- [x] `parse-project` and `parse-repo` accept project-page URL selectors and resolve deterministic project identity.
-- [x] `parse-registry` surfaces explicit project-page guidance instead of silent empty outputs.
-- [x] Parse outputs include deterministic repo coverage counters for direct readiness interpretation.
-- [x] AGENTS/docs/guides/skills references are synchronized with implemented selector/diagnostic behavior.
-- [x] Legacy parsing fallback paths are replaced by explicit migration diagnostics for project-page selector misuse.
-
-## Phase 88 - Corpus Completeness + Format Coverage (Package V: FR-177..FR-184, Complete)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-880.1 | FR-177..FR-184 | Complete | Consolidated Phase 88 requirements/design/tasks baseline in central spec. |
-| TSK-880.2 | FR-177, FR-178 | Complete | Verified direct crawl + linked fallback attachment chunk append path in extract-docs corpus pipeline with deterministic lineage metadata. |
-| TSK-880.3 | FR-179 | Complete | Expanded bounded code chunk extension allowlist and language mapping for mobile/native repository files (`.aidl`, `.proto`, `.podspec`, `.pbxproj`, `.xcconfig`, `.plist`, `.storyboard`, `.xib`) with unit coverage in `tests/unit/test_extract_docs_code_chunking.py`. |
-| TSK-880.4 | FR-180, FR-183 | Complete | Added `attachment_conversion_summary` (status/format/converter counts + failure samples) to extract-docs project outputs; added unit coverage. |
-| TSK-880.5 | FR-181 | Complete | Verified deterministic attachment conversion/chunk diagnostics across multi-format real runs and persisted durable evidence in `reports/verification/phase88_corpus_completeness/phase88-corpus-baseline-20260226/phase88-verification-summary.json` (generated by `scripts/verify_phase_88.py`). |
-| TSK-880.6 | FR-182 | Complete | Preserved stale-pruning + dedupe semantics with regression-hardening tests in `tests/unit/state/test_vector_index_store.py` (snapshot prune on empty chunk sets and copy-path exclusion for changed chunks). |
-| TSK-880.7 | FR-177..FR-183 | Complete | Added broader corpus diagnostics coverage for `_summarize_chunk_payload` plus extract-docs `docs_chunk_summary`/`project_chunk_summary` assertions in `tests/unit/test_extract_docs_code_chunking.py` and `tests/test_cli/test_materials_cli.py`. |
-| TSK-880.8 | FR-181 | Complete | Closed PDF/DOCX dataset verification with repo-persisted baseline evidence summary (`reports/verification/phase88_corpus_completeness/phase88-corpus-baseline-20260226/phase88-verification-summary.json`) covering Miniapp + DOCX/XLS/XLSX probe + PDF/DOCX/XLSX probe. |
-| TSK-880.9 | FR-184 | Complete | Synced AGENTS/docs/guides/skills with Phase 88 contracts (`attachment_conversion_summary`, `discovery=repo_chunking`) and kept `validate-spec-sync` green after consolidation updates. |
-| TSK-880.10 | FR-177..FR-184 | Complete | Closed Phase 88 acceptance gates, synchronized requirements/design/tasks version/status/test baselines (`v4.19.1`, 4178 tests), and added durable repo evidence (`reports/verification/phase88_corpus_completeness/phase88-corpus-baseline-20260226/phase88-verification-summary.json`) with CI guard test (`tests/unit/scripts/test_phase88_verification_artifacts.py`). |
-| TSK-880.11 | FR-180, FR-181 | Complete | Extraction quality follow-up: added HTML mojibake detection fallback for external conversion and extensionless attachment format inference (media-type + byte-signature, including draw.io XML). Verified on project `88718856` real run: attachment `found/partial` improved `95/65 -> 120/40`, format diagnostics now include `.xml|found=25`, and `Tổng quan hệ thống PMT.md` Vietnamese text restored. |
-| TSK-880.12 | FR-180, FR-184 | Complete | Persisted attachment-level sync metadata diagnostics in `sync_metadata.artifacts[].resource_metadata` (`source_format`, converter, discovery, size, conversion timing). Real run verification (`88718856`) confirms Postgres state now reports `unknown_source_format_count=0` across 160 attachments and supports deterministic partial-reason triage. |
-
-### Acceptance Gates (Phase 88)
-
-- [x] Direct crawl attachments with successful conversion are chunked and persisted with `source_type=confluence_attachment`.
-- [x] Linked fallback attachments continue to be chunked with correct lineage and no duplicate churn.
-- [x] Code chunk corpus includes required mobile/native file families while respecting bounded limits.
-- [x] Extract/retrieval outputs expose deterministic format-level diagnostics for operator verification.
-- [x] Miniapp real-run verification demonstrates chunk completeness and retrieval readiness parity.
-
-## Phase 89 - Unified Audit Quality + Skill Effectiveness + Performance Optimization (Package W: FR-185..FR-206, Complete)
-
-### Work Package Dependency Map
-
-```
-WP-89.1 (Scoping)  ──┐
-WP-89.2 (Embedding) ─┤── can parallelize (independent)
-WP-89.4 (Robustness) ┘
-
-WP-89.3 (Parallel Eval) ── depends on shared-state isolation understanding; TSK-890.10 before TSK-890.12
-WP-89.5 (Skill Policy) ──→ before WP-89.7 (Verification runs require skill policy changes deployed)
-WP-89.6 (Evidence Quality) ── TSK-890.28 depends on TSK-890.1 (repo type classifier informs category context)
-WP-89.7 (Verification) ── last; depends on all implementation WPs (WP-89.1..WP-89.6)
-```
-
-### WP-89.1: Repo-Type-Aware Checklist Scoping (High Priority — Score Quality)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-890.1 | FR-185 | Complete | Implement `RepoTypeClassifier` with file-tree heuristics for frontend/backend/infra/fullstack detection. |
-| TSK-890.2 | FR-185 | Complete | Implement declarative scoping policy (repo type → N/A row IDs/categories) with default frontend exclusion set (Docker/Helm/CI-CD/DB/Grafana). |
-| TSK-890.3 | FR-185 | Complete | Wire scoping into workflow: mark matched rows as `status=N/A`, `reason_code=repo_type_excluded`, exclude from score denominator. |
-| TSK-890.4 | FR-185 | Complete | Add `--repo-type {auto|frontend|backend|infra|fullstack}` CLI option. |
-| TSK-890.5 | FR-185 | Complete | Add unit/integration tests for repo classification, scoping policy application, and N/A score exclusion. |
-| TSK-890.6 | FR-185 | Complete | Verify with miniapp-fe real run: confirm 17 backend-infra rows are marked N/A and score lifts ~12-15pp. |
-
-### WP-89.2: Embedding Pipeline Optimization (Medium Priority — Performance)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-890.7 | FR-186 | Complete | Guard `embed_texts()` probe behind `lexical_only` flag in `vector_index.py` initialization. |
-| TSK-890.8 | FR-187 | Complete | Add TTL cache (default 300s) to `_fetch_model_digest` for Ollama `/api/show` calls. |
-| TSK-890.9 | FR-186, FR-187 | Complete | Add unit tests for probe skip and digest cache hit/miss/expiry behavior. |
-
-### WP-89.3: Parallel Row Evaluation + Evidence Caching (Medium Priority — Performance)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-890.10 | FR-188 | Complete | Extend `BatchExecutor` with `row_concurrency` parameter and asyncio semaphore-bounded dispatch. |
-| TSK-890.11 | FR-188 | Complete | Add `--row-concurrency <N>` CLI option (default: 1). |
-| TSK-890.12 | FR-188 | Complete | Implement per-row rate-limit backoff and ensure checkpoint/diagnostics isolation under concurrency. |
-| TSK-890.13 | FR-189 | Complete | Implement `BatchEvidenceCache` with chunk-ID-keyed storage, batch-scoped lifecycle, and hit/miss diagnostics. |
-| TSK-890.14 | FR-188, FR-189 | Complete | Add unit/integration tests for concurrent evaluation correctness and evidence cache behavior. |
-| TSK-890.15 | FR-188 | Complete | Benchmark with miniapp-core/miniapp-be (55-row baseline): measure wall-clock reduction at concurrency=2 and concurrency=3 vs baseline. |
-
-### WP-89.4: Robustness + Post-Processing (Low Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-890.16 | FR-190 | Complete | Change default `batch_size` from 5 to 8. |
-| TSK-890.17 | FR-191 | Complete | Add `"not permitted"` to non-retryable Confluence error marker set. |
-| TSK-890.18 | FR-192 | Complete | Add `permission_denied_pages` section to `doctor crawl-status` output. |
-| TSK-890.19 | FR-193 | Complete | Refactor post-processing to incremental producer/consumer pattern for batch-level streaming. |
-| TSK-890.20 | FR-190..FR-193 | Complete | Add unit tests for batch size default, error markers, crawl-status visibility, and post-processing overlap. |
-
-### WP-89.5: Skill Policy + Telemetry + Target Compliance (High Priority — Effectiveness)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-890.21 | FR-194 | Complete | Implement strict optional-skill hard-gate for `skills_needed=false` (disable list/load/script calls by default, controlled fallback read path only). |
-| TSK-890.22 | FR-195, FR-200 | Complete | Implement minimal execution-first required-skill policy with canonical default mode `minimal` and explicit override modes. |
-| TSK-890.23 | FR-196 | Complete | Upgrade non-progress guard to detect high-volume low-yield skill churn even when execution calls are present. |
-| TSK-890.24 | FR-197 | Complete | Unify skill telemetry surfaces (tool-first + prompt telemetry), enforce per-row skill-policy event emission including timeout/degraded rows. |
-| TSK-890.25 | FR-198 | Complete | Implement docs/code target execution compliance gate with explicit audited skip-reason requirement and aggregate compliance metrics. |
-| TSK-890.26 | FR-199 | Complete | Add effectiveness-driven skill ranking/rationale diagnostics (objective fit + observed effective-rate) with deferred broad fallback. |
-| TSK-890.27 | FR-194..FR-200 | Complete | Add unit/integration tests for hard-gate behavior, required-skill sequence, low-yield churn stop, telemetry completeness, and target compliance diagnostics. |
-
-### WP-89.6: Evidence Quality + Observability (High Priority — From Log Analysis)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-890.28 | FR-201 | Complete | Implement category-to-tool-bias mapping (`docs_primary`/`code_primary`/`balanced`) and wire into agentic system prompt for `docs_primary` categories. Depends on: TSK-890.1 (repo type classifier informs category context). |
-| TSK-890.29 | FR-202 | Complete | Implement run-level `source_frequency_map` and diversity penalty (default 0.7x for sources exceeding 30% row threshold) in evidence ranking pipeline. Add `--no-evidence-diversity` CLI flag. |
-| TSK-890.30 | FR-203 | Complete | Refactor evidence context assembly to rank chunks by relevance score before budget truncation. Increase default `max_evidence_chars` to 12000. Add enhanced truncation diagnostics and `evidence_truncation_severe` warning. |
-| TSK-890.31 | FR-204 | Complete | Implement unverified reference corrective retry: compute `unverified_ref_ratio` post-evaluation, dispatch one grounding-constrained retry when >0.5, replace original only if ratio improves. |
-| TSK-890.32 | FR-205 | Complete | Extend `--stability-runs` to compute per-row stability metric and cross-repo divergence flag. Emit `low_stability` and `cross_repo_divergent` quality warnings. |
-| TSK-890.33 | FR-206 | Complete | Add token usage capture (`prompt_tokens`/`completion_tokens`/`total_tokens`) to LLM call wrapper, per-row diagnostics, and run-level summary. Graceful null for backends without usage metadata. |
-| TSK-890.34 | FR-201..FR-206 | Complete | Add unit/integration tests for tool bias policy, diversity penalty, truncation mitigation, unverified ref retry, stability metric, and token instrumentation. |
-
-### WP-89.7: Verification + Spec Sync
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-890.35 | FR-185..FR-206 | Complete | Run miniapp-fe, miniapp-be, miniapp-core with Phase 89 enhancements enabled; verify score lift, latency improvements, skill-churn reduction, and evidence quality improvements. |
-| TSK-890.36 | FR-194..FR-200 | Complete | Re-run miniapp-be thread parity check to confirm no missing `row_skill_policy_effectiveness` rows (including timeout/degraded rows). |
-| TSK-890.37 | FR-201..FR-206 | Complete | Verify doc-search tool bias rate ≥60% for docs_primary rows, evidence concentration reduction, truncation severity reduction, and token usage completeness. |
-| TSK-890.38 | FR-185..FR-206 | Complete | Sync AGENTS/docs/guides/skills with unified Phase 89 contracts (CLI options, scoping policy, skill modes, evidence quality controls, diagnostics). |
-| TSK-890.39 | FR-185..FR-206 | Complete | Close Phase 89 acceptance gates and update version/status baselines. |
-| TSK-890.40 | FR-206A | Complete | Enforce Bitbucket-native repo identifier precedence across CLI/state/seed repo resolution, add regression tests, and verify `parse-repo` + `analyze` preflight resolve `local-merchant-profile-service` for miniapp merchant-profile-service repo. |
-| TSK-890.41 | FR-206B | Complete | Add retrieval snapshot migration compatibility: legacy alias baseline resolution for `local-*` repo keys, content-hash reuse for chunk-id drift, digest-compatibility guard for legacy non-hash digests, plus real dual-source refresh verification (`chunk_count=8127`, `changed=551`, `reused=7576`, `snapshot_id=278`, `docs_chunk_ready=true`, `code_chunk_ready=true`) for `local-merchant-profile-service`. |
-| TSK-890.42 | FR-206A | Complete | Add state-side synthetic link sanitation for `bitbucket.local/projects/local/...`, canonicalize non-local placeholder repo links to project Bitbucket origin, and verify real root-registry run (`pageId=88716675`) keeps `3-2-1-4-mini-app` free of `merchant-profile-service` contamination in both `parse-project` and `state list-links`. |
-
-### Acceptance Gates (Phase 89)
-
-- [x] Frontend repos have backend-infra rows marked N/A with deterministic reason code and excluded from score denominator.
-- [x] Repo-type classification is deterministic and overridable via `--repo-type` CLI option.
-- [x] Lexical-only mode skips Ollama probe; model digest is cached with TTL across sequential snapshots.
-- [x] Parallel row evaluation produces identical results to sequential evaluation at concurrency=1.
-- [x] Evidence retrieval cache reduces redundant chunk fetches within batches with observable hit/miss diagnostics.
-- [x] Default batch size is 8; "not permitted" pages are non-retryable; permission-denied pages visible in crawl-status.
-- [x] Post-processing overlaps with batch evaluation; timing diagnostics report overlap savings.
-- [x] Optional-skill rows (`skills_needed=false`) avoid discovery/script churn by default and only use controlled fallback skill reads when evidence-gap predicate is met.
-- [x] Required-skill rows (`skills_needed=true`) execute minimal effective skill sequence without discovery-first loops and with deterministic corrective retry behavior.
-- [x] Low-yield skill churn is detected and stopped even when execution calls are present but ineffective.
-- [x] `row_skill_policy_effectiveness` coverage is 100% across evaluated rows, including timeout/degraded rows.
-- [x] Docs/code target compliance metrics are emitted and rows missing both execution and audited skip reason are explicitly flagged.
-- [x] `search_docs` call rate for `docs_primary` checklist categories is ≥60% (up from ~18% baseline).
-- [x] Evidence anti-concentration metrics are emitted and no single source dominates >30% of rows without penalty.
-- [x] Evidence truncation uses relevance-ranked preservation; rows with severe truncation (>50% chunks removed) are flagged.
-- [x] Rows with >50% unverified refs trigger corrective retry; retry diagnostics are emitted in row trace.
-- [x] Evaluation stability metric is computed when `--stability-runs ≥ 2`; low-stability rows are flagged.
-- [x] LLM token usage is emitted per-call, per-row, and per-run in runtime logs and workflow summary.
-- [x] Repo resolution prefers Bitbucket-native keys across parse/workflow/analyze/state/seed paths; legacy project-prefixed keys remain readable but are not selected as primary.
-- [x] Repo-key migration refresh uses legacy alias baselines before project fallback, preserves unchanged vectors via content-hash reuse, and avoids false full re-embed when legacy non-hash digests are encountered.
-- [x] Root-registry material refresh (`88716675`) preserves project mapping isolation: Mini app manifest/state exclude synthetic local merchant-profile-service links and do not report `bitbucket.local/projects/local/...` in project-level mapping.
-- [x] Miniapp-fe score lifts ~12-15pp from N/A scoping; miniapp-core/miniapp-be wall-clock improves with concurrency>1 and skill-churn-related latency tails are reduced.
-
-## Phase 90 - Hierarchy Model Hard-Cut (Package X: FR-207..FR-216, Complete)
-
-### Work Package Dependency Map
-
-```
-WP-90.1 (Hierarchy Model) ──┐
-WP-90.2 (State Migration)  ─┼── foundation first
-WP-90.3 (CLI Contracts)    ─┘
-
-WP-90.4 (Extract/Readiness) ── depends on WP-90.1 + WP-90.2
-WP-90.5 (Verification/Sync) ── final, depends on WP-90.1..WP-90.4
-```
-
-### WP-90.1: Canonical Hierarchy Contracts (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-900.1 | FR-207 | Complete | Added `parse-registry` `departments[]` summary output with nested compact project entries and deterministic ordering. |
-| TSK-900.2 | FR-207, FR-208 | Complete | Persisted `confluence_ancestry_path` and lineage-derived `department` assignment metadata in parser/state payloads with deterministic assignment reasons. |
-| TSK-900.3 | FR-209 | Complete | Introduced grouped `bitbucket_projects[]` contracts in parse/extract outputs with nested `repos[]` and deterministic ordering; canonical parse outputs now omit flat `bitbucket[]` arrays. |
-| TSK-900.4 | FR-210 | Complete | Canonical repo key resolution now prefers Bitbucket-native identity (`<bb_project>:<bb_repo>`) over legacy aliases in parse/workflow key paths. |
-| TSK-900.5 | FR-207..FR-210 | Complete | Added unit/CLI coverage for hierarchy modeling, grouped parse contracts, selector isolation, and quarantine diagnostics. |
-
-### WP-90.2: State Schema Migration + Backfill (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-900.6 | FR-212 | Complete | Added normalized hierarchy entities (`departments`, `bitbucket_projects`) and FK links on `projects.department_id` and `repos.bitbucket_project_id`. |
-| TSK-900.7 | FR-212 | Complete | Implemented idempotent hierarchy backfill (`backfill_hierarchy_entities`) during state bootstrap for existing project/repo metadata. |
-| TSK-900.8 | FR-212, FR-213 | Complete | Added explicit quarantine diagnostics for unresolved/ambiguous hierarchy mappings and surfaced counts in hierarchy diagnostics payloads. |
-| TSK-900.9 | FR-213 | Complete | Updated state record/load paths to persist and hydrate grouped hierarchy payloads (`bitbucket_projects`, `bitbucket_quarantine`). |
-| TSK-900.10 | FR-212..FR-213 | Complete | Added regression coverage for hierarchy schema/backfill/integrity behavior in state store and loader tests. |
-
-### WP-90.3: Parse Command Hard-Cut + Selector Isolation (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-900.11 | FR-208, FR-211 | Complete | Removed Bitbucket project key aliases from `parse-project` selector resolution (`seed.select_project` no longer matches Bitbucket project keys). |
-| TSK-900.12 | FR-211 | Complete | Added explicit Bitbucket-project scoping support to `parse-repo` via `--bitbucket-project <KEY>`. |
-| TSK-900.13 | FR-208, FR-211 | Complete | Added deterministic selector diagnostics for project-vs-Bitbucket-project mismatch classes (actionable guidance + known Bitbucket project keys). |
-| TSK-900.14 | FR-209..FR-211 | Complete | Parse outputs now include department context and grouped Bitbucket project/repo summaries (`bitbucket_projects`, `project_bitbucket_projects`). |
-| TSK-900.15 | FR-208..FR-211 | Complete | Added CLI tests proving `parse-project --project MAP|WEBAPPV2|SUPERAPP` no longer maps to Mini app and emits guidance. |
-
-### WP-90.4: Extract-Docs + Readiness Hierarchy Enforcement (Medium Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-900.16 | FR-213 | Complete | `extract-docs` now emits grouped Bitbucket project artifacts and persists project hierarchy metadata in state artifact payloads. |
-| TSK-900.17 | FR-213, FR-214 | Complete | Hierarchy pipeline retains placeholder/sanitation protections and surfaces quarantined entries with bounded samples. |
-| TSK-900.18 | FR-214 | Complete | Added hierarchy diagnostics in `state readiness` (`hierarchy_ready`, department/project/bitbucket-project/repo counters, anomaly summaries with bounded samples). |
-| TSK-900.19 | FR-214 | Complete | Added `doctor hierarchy-status` command with bounded anomaly samples and deterministic hierarchy counters/status. |
-| TSK-900.20 | FR-214..FR-215 | Complete | Added/validated hierarchy readiness gate coverage (project-scoped diagnostics + schema auto-repair tests) and config-boundary coverage via extract-config CLI tests (`test_materials_cli.py::test_extract_docs_dry_run_includes_extract_config_metadata`). |
-
-### WP-90.5: Real-Run Verification + Spec/Docs Sync (Closure)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-900.21 | FR-216 | Complete | Root-registry live verification rerun succeeded (`parse-registry --registry-page 88716675`) with deterministic hierarchy payload persistence (`reports/verification/phase90_hierarchy/phase90-realrun-closure-20260227-101757/`). |
-| TSK-900.22 | FR-216 | Complete | Mini app live verification succeeded (`parse-project --project 3-2-1-4-mini-app` + grouped Bitbucket projects `MAP`,`SUPERAPP`,`WEBAPPV2` and scoped repo extraction for all three groups). |
-| TSK-900.23 | FR-216 | Complete | Selector isolation validated in real run: `parse-project --project MAP` fails with deterministic guidance to use `parse-repo --bitbucket-project`. |
-| TSK-900.24 | FR-215..FR-216 | Complete | Published durable real-run artifacts (`summary.json`, per-command stdout/stderr/exit, `metrics.json`, `README.md`) in `reports/verification/phase90_hierarchy/phase90-realrun-closure-20260227-101757/`. |
-| TSK-900.25 | FR-207..FR-216 | Complete | Synced spec/tasks/design + AGENTS to v4.20.5 and revalidated alignment via `vds-audit --json-only validate-spec-sync` (`valid=true`). |
-| TSK-900.26 | FR-207..FR-216 | Complete | Closed Phase 90 acceptance gates with closure evidence bundle `phase90-realrun-closure-20260227-101757` (`doctor hierarchy-status` + `state readiness` project-scoped anomaly count = 0). |
-
-### Acceptance Gates (Phase 90)
-
-- [x] `parse-registry` emits `departments[]` summary with nested projects and deterministic ordering.
-- [x] `parse-project` selector domain is isolated from Bitbucket project aliases; ambiguous alias mapping class is eliminated.
-- [x] Project payload contracts expose grouped `bitbucket_projects[]` with nested repos.
-- [x] Flat project-level `bitbucket[]` arrays are removed from canonical parse outputs (strict hard-cut compatibility cleanup).
-- [x] State schema migration/backfill is idempotent with explicit migrated/quarantined diagnostics.
-- [x] Confluence ancestry metadata is persisted and used for deterministic department assignment diagnostics.
-- [x] `state readiness` exposes hierarchy completeness and anomaly counters.
-- [x] doctor diagnostics expose hierarchy completeness and anomaly counters (`doctor hierarchy-status`).
-- [x] Real root-registry + Mini app runs validate hierarchy correctness and no cross-project contamination at closure thresholds (project-scoped anomaly count = 0 for closure dataset).
-- [x] Spec/docs/tasks/AGENTS alignment checks remain green after hierarchy migration.
-
-## Phase 91 - CLI Modularization + Scoring Completeness + Error Handling Hardening (Package Y: FR-217..FR-227, Complete)
-
-### Work Package Dependency Map
-
-```
-WP-91.1 (CLI Decomposition) ─────── foundation first (HIGH)
-          │
-WP-91.2 (Scoring Completeness) ──── independent (MEDIUM)
-          │
-WP-91.3 (Error Handling) ────────── depends on WP-91.1 (cli.py must be decomposed first)
-          │
-WP-91.4 (Skill Fallback) ────────── independent (MEDIUM)
-          │
-WP-91.5 (Baseline Sync) ─────────── final, depends on WP-91.1..WP-91.4 (LOW)
-```
-
-### WP-91.1: CLI Modular Decomposition (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-910.1 | FR-217 | Done | Created 8 command-group submodules (`workflow_cmds.py`, `parse_cmds.py`, `state_cmds.py`, `doctor_cmds.py`, `sync_cmds.py`, `extract_cmds.py`, `analyze_cmds.py`, `misc_cmds.py`) extracting command handlers from `cli.py`. |
-| TSK-910.2 | FR-218 | Done | Completed `cli_common.py` shared infrastructure extraction and wired command modules to consume centralized shared primitives. |
-| TSK-910.3 | FR-217, FR-218 | Done | Completed thin-router conversion: `cli.py` reduced to 12-line router shim delegating to `cli_impl.py`. |
-| TSK-910.4 | FR-219 | Done | Added command execution smoke coverage (`--help`) for root/groups/subcommands and verified deterministic router registration. |
-| TSK-910.5 | FR-219 | Done | Decomposition regression coverage validated with import + registration + help execution (`tests/unit/test_cli_decomposition_fr219.py`: 23 passing). |
-
-### WP-91.2: Checklist Scoring Completeness (Medium Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-910.6 | FR-220 | Done | Implemented profile-based weight adjustment in `checklist_generator.py` using audit profile configuration. |
-| TSK-910.7 | FR-221 | Done | Added configurable gating conditions for score caps in checklist row scoring. |
-| TSK-910.8 | FR-222 | Done | Added coverage tracking to checklist row evaluation with corpus-hit ratio diagnostics. |
-| TSK-910.9 | FR-220..FR-222 | Done | Added unit tests for weight adjustment, gating conditions, and coverage tracking with deterministic assertions. |
-
-### WP-91.3: Error Handling Hardening (Medium Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-910.10 | FR-223 | Done | Produced durable exception audit artifact at `reports/verification/phase91_error_handling/phase91-exception-audit-20260227/` with line-level classification (`intentional-recovery` / `boundary-guard` / `error-swallowing`). |
-| TSK-910.11 | FR-224 | Done | Hardened high-risk modules and finalized broad-catch reduction in top-5 files to 56 from 117 (52.14% reduction). |
-| TSK-910.12 | FR-224 | Done | Completed recoverable-fallback contracts across `engine/llm_row_evaluator.py`, `agents/agentic_investigator.py`, `state/store.py`, and CLI modules. |
-| TSK-910.13 | FR-225 | Done | Structured error logging fields (`error_type`, `source_module`, `context`, `recovery_action`) emitted consistently across hardened handlers. |
-| TSK-910.14 | FR-223..FR-225 | Done | Hardened-path regression suites pass (`tests/unit/state/test_state_cmds.py`, `tests/unit/state/test_vector_index_store.py`, `tests/unit/test_misc_cmds_fr224_225_hardening.py`, agent skill-policy suites). |
-
-### WP-91.4: Skill Fallback Robustness (Medium Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-910.15 | FR-226 | Done | Implemented broad fallback skill expansion path with effectiveness-driven ranking when top-ranked skills fail. |
-| TSK-910.16 | FR-226 | Done | Added fallback skill telemetry (fallback trigger count, success rate, ranked candidates) to skill diagnostics. |
-| TSK-910.17 | FR-226 | Done | Added unit and integration tests for skill fallback scenarios including multi-candidate ranking and effectiveness-driven selection. |
-
-### WP-91.5: Test Baseline Alignment + Spec Sync (Low Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-910.18 | FR-227 | Done | Updated spec/docs/AGENTS test baseline to 4546 and aligned Phase 91 status metadata across authoritative files. |
-| TSK-910.19 | FR-227 | Done | Revalidated green gates with `vds-audit --json-only validate-spec-sync`, `create_spec.py validate`, and `create_spec.py check-alignment`. |
-| TSK-910.20 | FR-220, FR-221 | Done | Closed scoring wiring gap: `generate_checklist_from_result` now derives profile multipliers/gating from `RepositoryAuditResult.profile_info`, and row metadata persists `gating_applied` + `gating_reason`. |
-| TSK-910.21 | FR-222 | Done | Closed workflow summary gap: `workflow-summary.json` now includes normalized `synthesis` payload and aggregate `coverage` diagnostics derived from checklist rows. |
-| TSK-910.22 | FR-227 | Done | Revalidated targeted Phase 91 regression gate after follow-up (`500 passed`) and refreshed spec/docs/AGENTS baseline to 4562 collected tests. |
-
-### Acceptance Gates (Phase 91)
-
-- [x] `cli.py` reduced to <500 lines (thin router only); all command logic lives in dedicated submodules.
-- [x] All command groups extracted to dedicated submodules with no behavioral regression (existing CLI tests pass).
-- [x] Profile-based weight adjustment produces different scores for different profile configurations.
-- [x] Score gating conditions and coverage tracking emit deterministic diagnostics in checklist metadata.
-- [x] Broad exception catches reduced by ≥50% in top 5 largest source files (measured by `rg 'except Exception' --count`).
-- [x] Structured error logging emitted for all hardened exception handlers with error type, context, and recovery action.
-- [x] Skill fallback path produces measurable fallback telemetry (trigger count, candidate ranking, success rate).
-- [x] Spec/docs/AGENTS sync remains green with aligned version/status/test baseline after all Phase 91 changes.
-- [x] `workflow-summary.json` includes normalized `synthesis` fields and aggregate coverage diagnostics.
-
-## Phase 92 - Intelligent Preparation + Metadata Completeness (Package Z: FR-228..FR-234, NFR-Y7)
-
-> **Clean-break implementation**: Legacy full-scan, force-reembed, and opt-out refresh paths are removed, not preserved. All new metadata fields are required. DiffDetector is the only code file discovery path.
-
-### Work Package Dependency Map
-
-```
-WP-92.1 (Legacy Code Removal) ─────── HIGH, foundation first (unblocks WP-92.2/92.3)
-          │
-WP-92.2 (DiffDetector-Only Chunking) ─ HIGH, depends on WP-92.1
-          │
-WP-92.3 (Snapshot Early Exit) ──────── HIGH, depends on WP-92.1 (force_full_reembed removed)
-          │
-WP-92.4 (Doc Chunking Skip) ────────── HIGH, depends on WP-92.2 (code-file version_tag uses commit SHA)
-          │
-WP-92.5 (Metadata Completeness) ────── MEDIUM, independent
-          │
-WP-92.6 (CLI Simplification) ──────── MEDIUM, depends on WP-92.1 (flags removed)
-          │
-WP-92.7 (Agent Citation Update) ────── MEDIUM, depends on WP-92.5 (metadata fields must flow first)
-          │
-WP-92.8 (Baseline Sync) ───────────── LOW, final, depends on WP-92.1..WP-92.7
-```
-
-### WP-92.1: Legacy Code Removal — Clean Break (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.1 | FR-234 | Done | Delete `_iter_repo_code_files()` (`cli_impl.py:902-926`) and `_CODE_CORPUS_MAX_FILES_PER_REPO` constant (`cli_impl.py:853`). |
-| TSK-920.2 | FR-234 | Done | Remove `force_full_reembed` from `refresh_project_index` (`vector_index.py:56`), `plan_retrieval_refresh` (`store.py:4868`), caller sites in `extract_cmds.py`, plus state/CLI API payload surfaces (`state_cmds.py`, `cli_impl.py`, snapshot plan/result models) to avoid dangling contracts. |
-| TSK-920.3 | FR-234 | Done | Replace `effective_force_full` computation block (`store.py:4945-5005`) with simplified logic: if embedding model changed, treat all chunks as changed. |
-| TSK-920.4 | FR-234 | Done | Remove `--refresh-retrieval-snapshot/--no-refresh-retrieval-snapshot` CLI flag (`extract_cmds.py:424-431`) and all downstream skip logic (`extract_cmds.py:2110-2119`, `2394-2415`). Retrieval refresh is always on during `--execute`. |
-| TSK-920.5 | FR-234 | Done | Remove `--incremental/--no-incremental` CLI flag (`extract_cmds.py:187-191`) and all downstream conditional code. |
-| TSK-920.6 | FR-234 | Done | Remove `--crawl-mode` CLI flag (`extract_cmds.py:116-120`) and locked validation (`extract_cmds.py:595`) — `bulk` is hardcoded. |
-| TSK-920.7 | FR-234 | Done | Remove `--emit-corpus` CLI flag (`extract_cmds.py:121-127`) and auto-force logic (`extract_cmds.py:609-615`) — corpus emission is always on. |
-| TSK-920.8 | FR-234 | Done | Remove `--fetch-policy` CLI flag (`extract_cmds.py:159-163`) and validation block (`extract_cmds.py:625-630`) — `if-changed` is hardcoded. |
-| TSK-920.9 | FR-234 | Done | Remove `--cache-mode` CLI flag (`extract_cmds.py:154-157`) and validation block (`extract_cmds.py:625`) — `read-write` is hardcoded. |
-| TSK-920.10 | FR-234 | Done | Clean up TOML config parsing/overrides in both `extract_cmds.py` and shared parser paths (`cli_impl.py`) — remove dead keys for `cache_mode`, `sync_policy`, `fetch_policy`, `refresh_retrieval_snapshot`, `incremental`, `crawl_mode`, `emit_corpus`. |
-| TSK-920.11 | FR-234 | Done | Add regression tests verifying all removed flags no longer appear in `--help` output, and `extract-docs --execute` behavior is preserved with always-on refresh, always-incremental, always-`if-changed`, always-`read-write` semantics. |
-
-### WP-92.2: DiffDetector-Only Code Chunking (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.12 | FR-228 | Done | Add `head_commit_sha` required TEXT column to `audit_state.repos` (single canonical storage path). |
-| TSK-920.13 | FR-228 | Done | Rewrite `_build_repo_code_chunk_records` (`cli_impl.py:928`) to use `DiffDetector(repo_root)` as the only file discovery path. On first extraction (no stored SHA), diff against Git empty tree (`4b825dc642cb6eb9a060e54bf8d69288fbee4904...HEAD`). |
-| TSK-920.14 | FR-228 | Done | Carry forward unchanged file chunk records from previous Postgres `corpus_chunks` snapshot instead of re-chunking. |
-| TSK-920.15 | FR-228 | Done | Persist current HEAD commit SHA as `head_commit_sha` in repo state after successful code chunk refresh. |
-| TSK-920.16 | FR-228 | Done | Update `extract_cmds.py:2611-2621` caller loop to pass stored `head_commit_sha` from repo state to `_build_repo_code_chunk_records`. |
-| TSK-920.17 | FR-228 | Done | Emit chunking diagnostics: `diff_based: true`, `skipped_unchanged_count`, `changed_file_count`, `head_commit_sha` in `code_chunking` stats within extract-docs `--json-only` output. |
-| TSK-920.18 | FR-228 | Done | Add unit tests for DiffDetector-only integration: mock git diff output, verify file filtering, verify first-extraction empty-tree diff, verify chunk carry-forward, verify diagnostics emission. |
-
-### WP-92.3: Snapshot-Level Early Exit (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.19 | FR-229 | Done | Add early-exit check in `plan_retrieval_refresh` (`state/store.py:4858`): compute aggregate content hash of incoming chunks, compare against latest `retrieval_snapshots.content_hash` for matching `snapshot_key`. Return `early_exit: true` plan when hashes match. |
-| TSK-920.20 | FR-229 | Done | Short-circuit `refresh_project_index` (`evidence/vector_index.py:43`) when `early_exit: true` — skip embedding client instantiation (line 100), probe (line 110-114), and `apply_retrieval_refresh` call. |
-| TSK-920.21 | FR-229 | Done | Add `retrieval_refresh_early_exit` structured log event with `snapshot_key`, `content_hash`, and elapsed time. |
-| TSK-920.22 | FR-229 | Done | Handle early-exit return payload in `extract_cmds.py:2648-2664` — emit `skipped: true` in diagnostics. |
-| TSK-920.23 | FR-229, NFR-Y7 | Done | Added deterministic early-exit latency metric (`early_exit_duration_ms`) in retrieval refresh payload, optimized pre-plan early-exit path to use minimal chunk projection, and added gate/test coverage enforcing `<100ms` no-change early-exit (`evaluate_phase92_early_exit_latency`, `verify_phase_92_performance.py`, and unit/CLI propagation tests). |
-
-### WP-92.4: Document-Level Chunking Skip (High Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.24 | FR-230 | Done | Change `corpus_documents.version_tag` (`store.py:789`) from `TEXT NULL` to `TEXT NOT NULL DEFAULT ''`. Add migration for existing NULL rows. |
-| TSK-920.25 | FR-230 | Done | Populate `version_tag` with Confluence `page.version.number` during doc ingestion and with git commit SHA for code files from DiffDetector path. |
-| TSK-920.26 | FR-230 | Done | Add version comparison logic in chunking pipeline keyed by (`source_type`, `source_uri`) with `page_id` as Confluence convenience key; skip chunking if unchanged. |
-| TSK-920.27 | FR-230 | Done | Emit chunking diagnostics: `docs_skipped_unchanged_count` and `docs_rechunked_count` in extract-docs output. |
-| TSK-920.28 | FR-230 | Done | Add unit tests for version comparison logic, skip behavior, version_tag population, and diagnostics emission. |
-
-### WP-92.5: Confluence Metadata Population Completeness (Medium Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.29 | FR-231 | Done | Change `corpus_documents.source_uri` (`store.py:787`) to `TEXT NOT NULL`. Add migration for existing NULL rows. |
-| TSK-920.30 | FR-231 | Done | Extend `CorpusChunk` model (`chunking.py:55-63`) with required first-class fields: `source_type: str`, `source_uri: str`, `version_tag: str`, `content_hash: str`. |
-| TSK-920.31 | FR-231 | Done | Fix `space_key` and canonical `source_uri` propagation gaps (normalize `confluence_url`/`source_url` aliases at ingestion boundaries) so chunk metadata is complete. |
-| TSK-920.32 | FR-231 | Done | Add `commit_sha` to code file chunk metadata (`cli_impl.py:976-990`) and set `version_tag` to commit SHA. |
-| TSK-920.33 | FR-231 | Done | Enforced `ck_audit_state_corpus_documents_confluence_page_id_not_blank`, added migration/backfill for legacy rows, and added metadata completeness tests for Confluence `page_id` extraction + schema constraint presence. |
-
-### WP-92.6: CLI Surface Simplification (Medium Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.34 | FR-232 | Done | Verified `extract-docs --execute` without flags produces fresh snapshots with always-`if-changed`, always-`read-write`, always-incremental, always-refresh semantics. |
-| TSK-920.35 | FR-232 | Done | Updated CLAUDE.md, SKILL.md, vds-cli-reference.md, cli-commands.md to reflect simplified CLI surface: removed `--fetch-policy`, `--cache-mode`, `--refresh-retrieval-snapshot`, `--incremental`, `--crawl-mode`, `--emit-corpus` flags from extract-docs documentation. |
-
-### WP-92.7: Agent Instruction Template Citation Update (Medium Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.36 | FR-233 | Done | Verified `_grounding_rules()` in `instruction_templates.py` already includes Confluence citation guidance: cite `page_id` and canonical `source_uri` (`source_url` alias accepted), use `ref_type='confluence_page'` for Confluence-sourced evidence. |
-| TSK-920.37 | FR-233 | Done | Verified `record_evidence_refs` usage hint (`instruction_templates.py:81`) already includes `page_id` and canonical `source_uri` field guidance (`source_url` alias accepted). |
-| TSK-920.38 | FR-233 | Done | Verified unit tests exist verifying instruction template output includes Confluence citation guidance when tools are available. |
-
-### WP-92.8: Test Baseline Alignment + Spec Sync (Low Priority)
-
-| Task ID | FR Coverage | Status | Notes |
-|--------|-------------|--------|-------|
-| TSK-920.39 | Cross-FR | Done | Added deterministic `performance` timing block to extract-docs `--json-only` output, implemented `performance_gates` evaluator + `scripts/verify_phase_92_performance.py`, and added automated unit tests for `<30s` no-change + `<50%` changed-only gate logic. Changed/full ratio now evaluates `performance.retrieval_refresh_duration_ms` (fallback `total_duration_ms`) to isolate incremental refresh cost from fixed orchestration overhead. Real-run `uv` verification bundles: `reports/verification/phase92_performance/20260227-233402` (Product Catalog: pass) and `reports/verification/phase92_performance/merchant-20260228-011021` (Merchant Profile: changed-only pass with real delta, no-change total still >30s). |
-| TSK-920.40 | FR-228..FR-234 | Done | Updated spec/docs/AGENTS version to v4.22.0, Phase 92 status to Complete, test baseline to 4562. SKILL.md, vds-cli-reference.md, cli-commands.md versions aligned. |
-| TSK-920.41 | Cross-FR | Done | Reduced no-change rerun total wall-clock for large prepared projects. Implemented follow-ups: (1) retrieval plan prioritizes exact `snapshot_key` lineage over cross-key snapshots, (2) eager `head_commit_sha` persistence during repo chunk planning, (3) repo-target dedupe by resolved `local_code_path` with native-identity preference, (4) canonical per-repo snapshot key composition, (5) cross-server linked-page dedupe fix for page-id collisions, (6) strict pre-plan lineage requirement (no model-name-only early-exit), (7) state-aware repo target ranking (`audit_state.repos` + existing repo snapshot lineage), (8) skip repo snapshot refresh when there are no repo code chunks and no repo snapshot baseline, and (9) no-change fast paths for linked-page materialization + Bitbucket metadata cache reuse + repo head-unchanged chunk-build skip. Validation artifacts: `reports/verification/phase92_performance/miniapp-head-backfill-20260228-053449` (`missing_head` `64 -> 22`), `.../miniapp-postfix2-20260228-060821` (`missing_head` `22 -> 14`), `.../miniapp-postfix4-20260228-091636` (`active_missing_head=0` for active synced repos), and closure rerun bundle `reports/verification/phase92_performance/tsk-920-41-final2-20260228-110559` (`no-change-pass2.performance.total_duration_ms=14529`, `retrieval_refresh_duration_ms=2212`, `no_change_pass=true`). Remaining aggregate `missing_head=14` rows are stale/non-repo aliases (`...:project`, legacy `node_modules`, alias rows) and are outside active repo readiness scope. |
-
-### Acceptance Gates (Phase 92)
-
-- [x] `_iter_repo_code_files()` and `_CODE_CORPUS_MAX_FILES_PER_REPO` deleted from codebase.
-- [x] `force_full_reembed` parameter deleted from all function signatures and caller sites.
-- [x] `--refresh-retrieval-snapshot`, `--incremental`, `--fetch-policy`, `--cache-mode`, `--crawl-mode`, `--emit-corpus` flags deleted from CLI.
-- [x] DiffDetector is the only code file discovery path; first extraction uses empty-tree diff.
-- [x] `head_commit_sha` persisted in repo state after every successful code chunking run.
-- [x] Snapshot early exit short-circuits retrieval refresh in < 100ms when content hash is unchanged.
-- [x] Document-level chunking skip uses `version_tag` (NOT NULL) to avoid re-chunking unchanged pages.
-- [x] `CorpusChunk` model has required first-class `source_type`, `source_uri`, `version_tag`, `content_hash` fields.
-- [x] `corpus_documents.source_uri` is NOT NULL; zero NULL `page_id` for Confluence-sourced rows.
-- [x] Agent instruction templates cite `page_id` and canonical `source_uri` (`source_url` alias accepted) with `ref_type='confluence_page'`.
-- [x] No-change extract-docs rerun completes in < 30 seconds for prepared repos (NFR-Y7). Product Catalog baseline passes (`20260227-233402/no-change-warm.json=21277ms`) and Mini app closure rerun passes on heavy prepared corpus (`tsk-920-41-final2-20260228-110559/no-change-pass2.json=14529ms`).
-- [x] Changed-file-only rerun stays < 50% of full-run retrieval refresh time (NFR-Y7 AC-Y7.2) with real code delta.
-- [x] TOML config overrides for removed options are cleaned up.
-- [x] Spec/docs/AGENTS sync remains green with aligned version/status/test baseline.
-
-
-
-## Older Version History
-| v4.22.0 | 2026-02-27 | Phase 92 clean-break planning: DiffDetector-only code chunking (FR-228), snapshot early exit (FR-229), document chunking skip (FR-230), required metadata fields (FR-231), CLI simplification (FR-232), agent citation update (FR-233), legacy code removal (FR-234), performance targets (NFR-Y7). Clean-break: no backward compat, required fields, legacy deletion. |
-| v4.21.3 | 2026-02-27 | Phase 91 follow-up closure: fixed profile-info-driven checklist scoring wiring, persisted gating metadata, restored workflow summary synthesis block, added aggregate coverage diagnostics, resolved test isolation leaks, and validated targeted regression bundle (`500 passed`). |
-| v4.21.2 | 2026-02-27 | Phase 91 closure: CLI decomposition completed (`cli.py` thin router), backend arbitration regression fixed, state/parse/extract/analyze decomposition compatibility hooks restored, hardening tests green, and exception-audit gate passed (117 -> 56 broad catches, 52.14% reduction). |
-| v4.21.1 | 2026-02-27 | Phase 88 extraction-quality follow-up: HTML external-conversion mojibake fallback + extensionless attachment format inference (`.xml` draw.io payloads) verified on real project run (`88718856`) with improved attachment conversion quality (`120 found / 40 partial`). |
-| v4.20.5 | 2026-02-27 | Phase 90 closure finalized: hierarchy diagnostics now use deterministic repo canonicalization/deduping + local-placeholder exclusion, live closure bundle `phase90-realrun-closure-20260227-101757` reports `anomaly_count=0` and `hierarchy_ready=true`, and spec/docs/AGENTS sync revalidated. |
-| v4.20.4 | 2026-02-27 | Phase 90 follow-up closure progress: fixed state schema auto-repair for missing hierarchy tables, scoped readiness/doctor hierarchy diagnostics to selected project to remove cross-project contamination, and added live verification artifacts (`phase90-realrun-final-20260227-093658`). |
-| v4.20.3 | 2026-02-27 | Phase 90 spec consistency sync: marked implemented hierarchy schema/backfill + ancestry/grouped-contract/quarantine tasks complete, removed conflicting “migration pending” posture, and kept strict hard-cut + real-run closure gates open. |
-| v4.20.2 | 2026-02-26 | Phase 90 in-progress follow-up: `parse-registry` `departments[]` summary added with deterministic nested projects, `doctor hierarchy-status` command added, and docs/guides/skills synchronization updated for hierarchy diagnostics and Bitbucket-project scoping. |
-| v4.20.1 | 2026-02-26 | Phase 90 in-progress implementation update: selector isolation hard-cut completed (`parse-project` vs Bitbucket project key), `parse-repo --bitbucket-project` added, grouped Bitbucket summary parse outputs added, and `state readiness` hierarchy diagnostics landed with bounded anomaly samples. |
-| v4.20.0 | 2026-02-26 | Phase 90 planning kickoff: hierarchy model hard-cut (`FR-207..FR-216`) for explicit Department -> Project -> Bitbucket Project -> Repo contracts, selector isolation, state schema migration, and real-run closure gates. |
-| v4.19.2 | 2026-02-26 | Phase 89 follow-up hardening: sanitize synthetic `bitbucket.local/projects/local/...` state links, canonicalize non-local placeholder repo URLs to project Bitbucket origin, and verify root-registry mini-app mapping isolation (`88716675`). |
-| v4.19.1 | 2026-02-26 | Phase 88 closure retained and Phase 89 planning unified to FR-185..FR-200 (score quality + skill effectiveness + telemetry parity + performance) with synchronized spec baselines (4178 tests). |
-| v4.18.13 | 2026-02-26 | Phase 88 follow-up: code chunk discovery lineage normalization + attachment conversion summary diagnostics + miniapp real-run verification artifact |
-| v4.18.12 | 2026-02-26 | Phase 88 kickoff: corpus completeness + format coverage planning and task activation |
-| v4.18.11 | 2026-02-26 | Phase 87 implementation closure: direct project-page parsing, parse diagnostics, repo coverage summaries, docs/guides/skills sync |
-| v4.18.10 | 2026-02-26 | Phase 87 scope expansion: docs/guides/skills sync + legacy parsing cleanup planning |
-| v4.18.8 | 2026-02-26 | Phase 86 follow-up closure + spec consolidation alignment |
-| v4.18.7 | 2026-02-25 | Phase 86 semantic gap closure baseline |
-