agentic-orchestrator 0.1.6 → 0.1.8

package/spec-files/outstanding/agentic_orchestrator_planning_review_quality_spec.md

@@ -14,6 +14,7 @@
 ### 0.1 Required Standards
 
 All implementation MUST preserve:
+
 - deterministic tool behavior and normalized error envelopes
 - MCP/in-process transport parity for all new tools
 - policy/schema-governed behavior changes (no hidden defaults)
@@ -23,6 +24,7 @@ All implementation MUST preserve:
 ### 0.2 Required Upstream Inputs
 
 Implementing agents MUST read:
+
 - `spec-files/outstanding/agentic_orchestrator_quality_adoption_execution_spec.md`
 - `apps/control-plane/src/application/services/plan-service.ts`
 - `apps/control-plane/src/core/collisions.ts`
@@ -47,6 +49,7 @@ Implementing agents MUST read:
 ### 0.3 Feature Scope
 
 This spec implements:
+
 - **Q1** Semantic Collision Engine
 - **Q2** Plan Feasibility Scoring
 - **Q7** Prompt Contract Test Harness
@@ -54,6 +57,7 @@ This spec implements:
 - **Q9** Auto-Generated Review Briefs
 
 Out of scope:
+
 - ML-based probabilistic heuristics
 - autonomous prompt rewriting
 - silent gate bypasses without policy and audit metadata
@@ -65,41 +69,49 @@ Out of scope:
 The prior draft was directionally useful, but not implementation-safe against the current architecture. Key issues:
 
 1. **Semantic collision data model was underspecified**
+
 - Problem: Q1 demanded semantic conflicts (OpenAPI/event/db intent) but provided no canonical plan-level fields to compare deterministically.
 - Impact: impossible to implement reliably in `plan.submit` without brittle repo parsing or heuristic guessing.
 - Correction: add explicit `plan.semantic_intent` contract to `plan.schema.json`.
 
 2. **Policy changes were not wired to current schema constraints**
+
 - Problem: root `policy.schema.json` is strict (`additionalProperties: false`). New sections cannot be added by policy text alone.
 - Impact: runtime load fails if sections are added without schema/default updates.
 - Correction: update defaults + full policy schema + init-policy generation paths.
 
 3. **Tool-surface proposal was incomplete**
+
 - Problem: adding tools without corresponding updates to constants, kernel handler registration, catalog, RBAC, and behavior metadata causes contract/runtime drift.
 - Impact: `validate:mcp-contracts` and runtime authorization failures.
 - Correction: include full tool lifecycle wiring as normative work.
 
 4. **Flaky workflow lacked a mutating governance path**
+
 - Problem: draft provided read-only flake report but no deterministic mutation interface for quarantine owner/reason/expiry.
 - Impact: “workflow” cannot be executed via orchestrator contracts.
 - Correction: add explicit quarantine upsert/clear contracts and audit metadata.
 
 5. **Flaky model lacked deterministic test-level input source**
+
 - Problem: current gate evidence includes step-level execution, not standardized per-test outcomes.
 - Impact: per-test flaky scoring cannot be computed reliably.
 - Correction: extend gate parser model to optionally ingest structured test results (e.g., junit_xml / vitest_json).
 
 6. **Review brief integration path mismatched actual dashboard architecture**
+
 - Problem: draft targeted non-existent `packages/web-dashboard/src/components/ReviewPanel.tsx`.
 - Impact: implementation ambiguity and likely dead-end edits.
 - Correction: wire through existing `aop-client.ts`, `types.ts`, and `app/page.tsx`.
 
 7. **Plan feasibility scoring lacked deterministic formula contract**
+
 - Problem: scoring factors were named but not formally defined.
 - Impact: non-repeatable behavior and test fragility.
 - Correction: define exact component formulas and rounding rules.
 
 8. **No rollout/failure-mode safety for quality gates**
+
 - Problem: missing `report_only` vs `enforce` semantics for early rollout.
 - Impact: high risk of accidental throughput regression.
 - Correction: add explicit enforcement modes with default `report_only`.
@@ -143,7 +155,7 @@ innovation:
 planning:
   feasibility:
     enabled: false
-    enforcement_mode: report_only   # report_only | enforce
+    enforcement_mode: report_only # report_only | enforce
     minimum_score: 70
     minimum_component_score: 0.45
     weights:
@@ -176,6 +188,7 @@ reporting:
 ```
 
 Normative behavior:
+
 - all flags default `false` (opt-in rollout)
 - `weights` MUST sum to 1.0 exactly after normalization (schema + runtime validation)
 - `enforcement_mode=report_only` logs score metadata but never blocks plan acceptance
@@ -214,6 +227,7 @@ Add optional deterministic semantic declarations:
 ```
 
 Rules:
+
 - absence of `semantic_intent` is valid; engine treats as empty intent set
 - deterministic compare uses normalized lowercase for `method` and stable key ordering
 
@@ -229,6 +243,7 @@ Add/extend runtime artifacts:
 ```
 
 State frontmatter extension (`state.schema.json`):
+
 - `collisions.semantic` array
 - `planning_quality.feasibility_score` number (0-100)
 - `planning_quality.feasibility_breakdown` object
@@ -239,18 +254,23 @@ State frontmatter extension (`state.schema.json`):
 ### New Tools
 
 1. `plan.score` (read-only)
+
 - returns feasibility score + deterministic breakdown + reasons
 
 2. `gate.flaky_report_get` (read-only)
+
 - returns flaky suspects, quarantine entries, expiry warnings
 
 3. `gate.flaky_quarantine_upsert` (mutating)
+
 - upsert quarantine entry (`test_key`, `owner`, `reason`, `expires_at`)
 
 4. `gate.flaky_quarantine_clear` (mutating)
+
 - clear quarantine entry by `test_key`
 
 5. `report.feature_review_brief` (read-only)
+
 - returns parsed review brief for feature
 
 ### Extended Existing Tool Contracts
@@ -272,6 +292,7 @@ State frontmatter extension (`state.schema.json`):
 ## 3.5 Error Code Additions
 
 Add to `apps/control-plane/src/core/error-codes.ts`:
+
 - `semantic_collision_detected`
 - `plan_feasibility_below_threshold`
 - `plan_feasibility_config_invalid`
@@ -287,9 +308,11 @@ Add to `apps/control-plane/src/core/error-codes.ts`:
 ## PRQ0: Contract Scaffolding and Policy Wiring
 
 ### Goals
+
 - land all schema/default/tool scaffolding first to avoid mid-phase drift
 
 ### File Targets
+
 - `agentic/orchestrator/defaults/policy.defaults.yaml`
 - `agentic/orchestrator/schemas/policy.schema.json`
 - `agentic/orchestrator/schemas/plan.schema.json`
@@ -304,10 +327,12 @@ Add to `apps/control-plane/src/core/error-codes.ts`:
 - `apps/control-plane/src/cli/init-command-handler.ts` (full policy template alignment)
 
 ### Mandatory Checks
+
 - `npm run validate:mcp-contracts`
 - `npm run validate:architecture`
 
 Exit criteria:
+
 - kernel loads composed policy with new fields
 - no contract-validator drift
 
@@ -323,9 +348,11 @@ Implement semantic comparisons as an additive layer over existing syntactic coll
 
 1. Normalize candidate + accepted plan semantic intents.
 2. Compare candidate against each accepted plan:
+
 - OpenAPI conflict if same `operation_id` OR same `path+method` and incompatible `change` class.
 - Event conflict if same `event_name` and incompatible `version/change` pair.
 - DB intent conflict if same `entity` and conflicting operations (`drop_column` vs `rename_column`, etc.).
+
 3. Emit stable sorted conflicts and deterministic fingerprint.
 4. Merge with existing `files/areas/contracts` collision result.
 
@@ -351,6 +378,7 @@ Implement semantic comparisons as an additive layer over existing syntactic coll
 - extend transport parity assertions in `apps/control-plane/test/mcp.spec.ts`
 
 Exit criteria:
+
 - same inputs produce byte-identical `semantic_conflicts` ordering
 - plan blocking/rejection behavior unchanged except added semantic details
 
@@ -367,6 +395,7 @@ Add deterministic score computation service and optional admission enforcement.
 `score = round(100 * sum(weight_i * component_i))`
 
 Components (0.0-1.0):
+
 - `scope_realism`: bounded function of planned file count, allowed-area breadth, and summary specificity.
 - `test_sufficiency`: acceptance criteria count and presence of verification mapping (`gate_profile`, `gate_targets`, `verification_overrides`).
 - `dependency_completeness`: dependency declarations resolvable from index state.
@@ -374,6 +403,7 @@ Components (0.0-1.0):
 - `lock_contention_risk`: penalty when required lock resources are currently held and blocked queue depth is high.
 
 Normalization rules:
+
 - all component values clamp to `[0,1]`
 - weights normalize to sum 1.0 at runtime
 - reasons array order is lexical and deterministic
@@ -401,6 +431,7 @@ Normalization rules:
 - extend `apps/control-plane/test/mcp.spec.ts` for `plan.score`
 
 Exit criteria:
+
 - repeated scoring over identical inputs yields identical score and reasons
 - enforcement only active when explicitly configured
 
@@ -415,15 +446,19 @@ Treat prompt behavior as deterministic source contract tests in CI.
 ### Contract model
 
 Fixture set:
+
 - `agentic/orchestrator/prompt-contract/fixtures/*.json`
 
 Invariant set:
+
 - `agentic/orchestrator/prompt-contract/invariants/*.json`
 
 Harness:
+
 - `apps/control-plane/scripts/test-prompt-contracts.mjs`
 
 Output artifact:
+
 - `.aop/runtime/prompt-contract/latest.json`
 
 ### Invariants (minimum)
@@ -441,7 +476,7 @@ Output artifact:
   - `agentic/orchestrator/prompts/`
   - `agentic/orchestrator/prompt-contract/`
   - `apps/control-plane/src/supervisor/worker-decision-loop.ts`
-  change.
+    change.
 
 ### Implementation Targets
 
@@ -457,6 +492,7 @@ Output artifact:
 - failure snapshot tests for invariant diagnostics
 
 Exit criteria:
+
 - invariant failures provide actionable role + fixture + invariant id diagnostics
 - harness runtime is deterministic and independent of live provider calls
 
@@ -471,12 +507,16 @@ Introduce deterministic flaky scoring over historical test outcomes and governed
 ### Data pipeline
 
 1. Gate execution emits structured test results (optional parser path):
+
 - extend `gates.yaml` parser config with `parsers.tests`
 - supported initial parsers: `none`, `junit_xml`, `vitest_json`
+
 2. `GateService` ingests latest run outcomes after each gate run.
 3. `FlakyGateService` updates `.aop/analytics/flaky-tests.json` with rolling window stats.
 4. Suspect score computed deterministically:
+
 - `suspect_score = failure_rate_weight * failure_rate + alternation_weight * alternation_rate`
+
 5. Quarantine controls applied only with explicit metadata and expiry.
 
 ### Quarantine rules
@@ -532,6 +572,7 @@ Introduce deterministic flaky scoring over historical test outcomes and governed
 - extend `apps/control-plane/test/mcp.spec.ts` for new tools and parity
 
 Exit criteria:
+
 - suspect list deterministic for identical history
 - quarantine lifecycle enforced with owner/reason/expiry constraints
 - required-gate integrity preserved by default
@@ -590,6 +631,7 @@ Generate deterministic review briefs from canonical runtime artifacts; no free-f
 - add UI rendering test under web-dashboard workspace for brief display
 
 Exit criteria:
+
 - brief generation is deterministic and schema-valid
 - dashboard shows brief without regressions to existing detail panel behavior
 
@@ -624,21 +666,27 @@ Exit criteria:
 ## 5. Core Tradeoffs
 
 1. **Semantic precision vs planner burden**
+
 - richer `semantic_intent` improves detection but increases plan authoring complexity.
 
 2. **Early plan quality gates vs throughput**
+
 - enforcing feasibility reduces churn but can block progress if thresholds are too strict.
 
 3. **Flaky noise reduction vs masking regressions**
+
 - quarantine reduces false alarms but can hide real defects if governance is weak.
 
 4. **Strict prompt contracts vs iteration speed**
+
 - strong invariants prevent regressions but add maintenance overhead when role behavior evolves.
 
 5. **Comprehensive review briefs vs generation cost**
+
 - richer summaries help reviewers but add I/O/compute per `ready_to_merge` transition.
 
 6. **New tool count vs API surface simplicity**
+
 - dedicated tools improve explicitness and RBAC isolation but increase contract maintenance load.
 
 ---
@@ -646,22 +694,27 @@ Exit criteria:
 ## 6. Milestones and Implementation Sequence
 
 ### Milestone A (Week 1)
+
 - PRQ0 scaffolding complete
 - contract validators green
 
 ### Milestone B (Week 2)
+
 - Q1 semantic collisions + tests complete
 - Q2 feasibility scoring in `report_only`
 
 ### Milestone C (Week 3)
+
 - Q7 prompt contract harness enforced in CI
 - Q8 flaky ingestion and reporting complete
 
 ### Milestone D (Week 4)
+
 - Q8 quarantine workflow complete
 - Q9 review brief generation + dashboard integration complete
 
 ### Milestone E (Week 5)
+
 - full stabilization, parity tests, docs, and rollout playbook
 
 ---
@@ -722,9 +775,9 @@ Exit criteria:
 ## 9. Definition of Done
 
 A completed milestone MUST include:
+
 - implemented file inventory mapped to the phase above
 - schema/catalog/RBAC/tool-doc updates
 - unit + integration + parity tests
 - validator and test command summaries
 - `spec-files/progress.md` update with completed tasks, residual tasks, and risks
-