agentic-orchestrator 0.1.6 → 0.1.8

package/spec-files/outstanding/agentic_orchestrator_provider_auth_bootstrap_spec.md

@@ -0,0 +1,384 @@
+# Feature Spec: Provider Auth Bootstrap UX for Local CLI and API-Backed Providers (AOP)
+
+> **Purpose of this document**: Define implementation-ready changes for `aop init` and provider selection so local CLI providers (for example Codex/Claude Code) do not require API-key env wiring, while API-backed providers get guided env validation and credential bootstrap.
+
+**Version:** 1.0  
+**Date:** 2026-03-04  
+**Status:** Draft  
+**Roadmap Mapping:** M41
+
+---
+
+## 0. Standards and Dependencies
+
+### 0.1 Required Standards
+
+All implementation MUST preserve:
+
+- deterministic provider selection behavior
+- normalized error envelopes (`{ ok: false, error: { code, message, details } }`)
+- no credential values written to logs, CLI payloads, or exceptions
+- existing CLI command names and flags (`aop init`, `aop run`, `--provider-config-env`)
+- backward compatibility for existing `agents.yaml` + env workflows
+- Nx + Vitest + lint/typecheck quality gates
+
+### 0.2 Upstream Inputs
+
+Implementing agents MUST read:
+
+- `apps/control-plane/src/cli/init-command-handler.ts`
+- `apps/control-plane/src/cli/help-command-handler.ts`
+- `apps/control-plane/src/cli/cli-argument-parser.ts`
+- `apps/control-plane/src/cli/types.ts`
+- `apps/control-plane/src/providers/providers.ts`
+- `apps/control-plane/src/interfaces/cli/bootstrap.ts`
+- `apps/control-plane/src/core/kernel.ts`
+- `apps/control-plane/test/init-wizard.spec.ts`
+- `apps/control-plane/test/providers.spec.ts`
+- `agentic/orchestrator/schemas/agents.schema.json`
+- `README.md`
+
+---
+
+## 1. Problem Statement
+
+### 1.1 Current Gaps
+
+1. `aop init` always emits `runtime.provider_config_env: AOP_PROVIDER_CONFIG_ENV` and does not branch by provider auth model.
+2. `resolveProviderSelection()` currently treats `codex` and `claude` as auth-required, even though current execution path uses local CLI commands.
+3. The init flow does not validate whether a user-provided provider env var exists.
+4. There is no guided fallback to capture missing credentials during init.
+5. Current semantics around `AOP_PROVIDER_CONFIG_ENV` are confusing in CLI-first workflows.
+
+### 1.2 User-Requested Behavior
+
+The target UX must implement this sequence:
+
+1. init asks whether the user will use a local CLI provider (Codex/Claude Code/etc.)
+2. if yes, init does not ask for `provider_config_env`
+3. if no, init asks for provider and env var name
+4. if env var missing, init asks for key input and stores it in `AOP_PROVIDER_CONFIG_ENV`
+
+---
+
+## 2. Objectives
+
+### 2.1 Must-Have Outcomes
+
+1. Interactive init has an explicit local-CLI vs API-backed auth branch.
+2. Local-CLI path generates valid config without `provider_config_env`.
+3. API-backed path validates configured env var availability.
+4. Missing-env fallback writes credential into repo-local `.env` under `AOP_PROVIDER_CONFIG_ENV`.
+5. Runtime provider selection supports this fallback without breaking existing env-name indirection setups.
+6. `codex` and `claude` flows no longer hard-fail for missing provider auth env vars.
+
+### 2.2 Non-Goals
+
+- no change to orchestration algorithm/state machine
+- no remote secret manager integration in this phase
+- no encryption-at-rest feature for local `.env`
+- no dashboard credential editor in this phase
+
+---
+
+## 3. UX Specification (Normative)
+
+### 3.1 Init Decision Tree
+
+New prompt early in interactive init:
+
+- `Will you use a local agent CLI (codex/claude-code/kiro-cli/copilot)? [yes/no]` (default: `yes`)
+
+Branch rules:
+
+- **If `yes`**:
+  - ask default provider and model as today
+  - do **not** ask for provider config env var
+  - do not write `runtime.provider_config_env` in generated `agents.yaml`
+- **If `no`**:
+  - ask default provider and model
+  - ask: `Provider config env var name` (example defaults by provider: `GEMINI_API_KEY`, `OPENAI_API_KEY`, etc.)
+  - validate env var exists in effective env context
+  - if missing: prompt for key and persist to `.env` as `AOP_PROVIDER_CONFIG_ENV=<pasted_key>`
+  - set generated `runtime.provider_config_env: AOP_PROVIDER_CONFIG_ENV`
+
+### 3.2 Env Validation Rules
+
+Env var existence check MUST include:
+
+1. current process env (`runtime.env`)
+2. repo-root `.env` entries (if file exists)
+
+A value is “existing” only when non-empty after trim.
+
+### 3.3 Missing Env Fallback Rules
+
+If requested env var is missing:
+
+1. init must print actionable message: requested env var not found
+2. init must prompt: `Paste provider key to store in AOP_PROVIDER_CONFIG_ENV`
+3. empty input is rejected and re-prompted
+4. key is persisted to `.env` as `AOP_PROVIDER_CONFIG_ENV=<key>`
+5. return payload includes non-sensitive note in `next_steps` that `.env` was updated
+
+No printed output may include key value.
+
+### 3.4 `--auto` Mode
+
+`aop init --auto` remains non-interactive and defaults to local-CLI style output:
+
+- provider/model defaults unchanged
+- no `provider_config_env` emitted
+- no secret prompts
+
+---
+
+## 4. Runtime Resolution Contract
+
+### 4.1 Provider Auth Classes
+
+Introduce explicit auth class sets in `providers.ts`:
+
+- `LOCAL_CLI_PROVIDERS = {'codex','claude','kiro-cli','copilot','custom'}`
+- `CREDENTIAL_REQUIRED_PROVIDERS = {'gemini'}` (initial set; future providers may be added)
+
+### 4.2 New Credential Resolution Algorithm
+
+`resolveProviderSelection()` MUST resolve credential reference using this precedence:
+
+1. CLI `--provider-config-env <NAME>` => if `env[NAME]` exists, use it.
+2. `agents.yaml runtime.provider_config_env` => if `env[NAME]` exists, use it.
+3. `env.AOP_PROVIDER_CONFIG_ENV` fallback:
+   - if value looks like env-var name (`^[A-Z_][A-Z0-9_]*$`) and `env[value]` exists, treat as legacy indirection and use `env[value]`
+   - else treat value itself as direct credential string (new bootstrap behavior)
+4. if provider is in `CREDENTIAL_REQUIRED_PROVIDERS` and no credential resolved, throw `PROVIDER_AUTH_MISSING`
+
+### 4.3 Backward Compatibility Guarantees
+
+All of the following must continue to work:
+
+1. Existing config where `provider_config_env: SOME_KEY` and `SOME_KEY` exists in env
+2. Existing env indirection usage with `AOP_PROVIDER_CONFIG_ENV=SOME_KEY` and `SOME_KEY` exists
+3. New bootstrap usage with `AOP_PROVIDER_CONFIG_ENV=<actual_key>`
+
+---
+
+## 5. Configuration Contract Changes
+
+### 5.1 Generated `agents.yaml` Shape
+
+Local CLI branch example:
+
+```yaml
+version: 1
+roles:
+  planner:
+    system_prompt_path: config/agentic/orchestrator/prompts/planner.system.md
+  builder:
+    system_prompt_path: config/agentic/orchestrator/prompts/builder.system.md
+  qa:
+    system_prompt_path: config/agentic/orchestrator/prompts/qa.system.md
+missing_prompt_behavior: ignore
+runtime:
+  default_provider: codex
+  default_model: local-default
+  role_provider_overrides: {}
+```
+
+API-backed branch with fallback example:
+
+```yaml
+runtime:
+  default_provider: gemini
+  default_model: gemini-default
+  provider_config_env: AOP_PROVIDER_CONFIG_ENV
+  role_provider_overrides: {}
+```
+
+### 5.2 Schema/Docs Clarification
+
+Update `agents.schema.json` field description for `runtime.provider_config_env` to clarify:
+
+- optional for local CLI providers
+- when set, should be an env var name
+- runtime also supports `AOP_PROVIDER_CONFIG_ENV` fallback behavior
+
+---
+
+## 6. Secret Persistence Contract
+
+### 6.1 Storage Location
+
+Persist init-captured fallback key in repo root:
+
+- `.env` file at `<repoRoot>/.env`
+- key: `AOP_PROVIDER_CONFIG_ENV`
+
+### 6.2 Write Semantics
+
+Env write helper must be deterministic and idempotent:
+
+1. if `.env` missing, create it
+2. if `AOP_PROVIDER_CONFIG_ENV=` exists, replace value
+3. else append new line
+4. preserve unrelated entries
+
+### 6.3 Security Requirements
+
+- do not emit key values in logs, exceptions, test snapshots, or payloads
+- avoid including raw key in `validation_warnings`/`next_steps`
+- recommend users rotate/move secret to external manager in docs (non-blocking guidance)
+
+---
+
+## 7. File-Level Implementation Plan
+
+### M41-M1: Init Wizard Branching + Env Bootstrap
+
+#### AUTH-T-001: Extend Wizard Config Model
+
+**Files:**
+
+- `apps/control-plane/src/cli/init-command-handler.ts`
+
+**Changes:**
+
+- add wizard auth branch metadata (local CLI vs API-backed)
+- add provider env prompt only in API-backed branch
+- add env existence validation helper (process env + `.env`)
+- add fallback key capture + `.env` persistence helper
+- conditionally render `provider_config_env` in generated `agents.yaml`
+
+#### AUTH-T-002: Add `.env` Read/Write Utility
+
+**Files (recommended):**
+
+- `apps/control-plane/src/cli/env-file.ts` (new)
+- `apps/control-plane/src/cli/init-command-handler.ts`
+
+**Changes:**
+
+- parse minimal dotenv key/value format needed for this feature
+- implement upsert for `AOP_PROVIDER_CONFIG_ENV`
+
+### M41-M2: Provider Resolution Semantics Update
+
+#### AUTH-T-003: Reclassify Provider Auth Requirements
+
+**Files:**
+
+- `apps/control-plane/src/providers/providers.ts`
+
+**Changes:**
+
+- replace current auth-required set to include only credential-required providers
+- ensure local CLI providers do not throw `PROVIDER_AUTH_MISSING` when key is absent
+
+#### AUTH-T-004: Add Backward-Compatible AOP Fallback Parser
+
+**Files:**
+
+- `apps/control-plane/src/providers/providers.ts`
+
+**Changes:**
+
+- implement precedence and dual-mode parsing from Section 4.2
+- maintain output fields `provider_config_env` and `provider_config_ref`
+
+### M41-M3: Documentation and Help Updates
+
+#### AUTH-T-005: Update CLI Help + README
+
+**Files:**
+
+- `README.md`
+- `apps/control-plane/src/cli/help-command-handler.ts`
+- `agentic/orchestrator/schemas/agents.schema.json`
+
+**Changes:**
+
+- document init auth branching and fallback behavior
+- clarify `--provider-config-env` applies to API-backed provider auth
+- clarify local CLI providers generally use local CLI login state
+
+---
+
+## 8. Test Plan
+
+### 8.1 Init Wizard Tests (`init-wizard.spec.ts`)
+
+Add/adjust tests:
+
+1. local-CLI branch does not ask for provider env var and generated `agents.yaml` omits `provider_config_env`
+2. API-backed branch with existing env var writes that env var name to `agents.yaml`
+3. API-backed branch with missing env var prompts for key and writes `.env` entry `AOP_PROVIDER_CONFIG_ENV=...`
+4. fallback branch sets `runtime.provider_config_env: AOP_PROVIDER_CONFIG_ENV`
+5. `--auto` path remains non-interactive and emits no provider config env
+6. `.env` upsert replaces existing `AOP_PROVIDER_CONFIG_ENV` value without duplicating lines
+
+### 8.2 Provider Resolver Tests (`providers.spec.ts`)
+
+Add/adjust tests:
+
+1. `codex` without credentials no longer throws `PROVIDER_AUTH_MISSING`
+2. `claude` without credentials no longer throws `PROVIDER_AUTH_MISSING`
+3. credential-required provider (initially `gemini`) still throws when unresolved
+4. `AOP_PROVIDER_CONFIG_ENV=<KEY_VALUE>` direct fallback resolves as credential
+5. legacy indirection `AOP_PROVIDER_CONFIG_ENV=SOME_ENV` with `SOME_ENV=value` still resolves
+6. CLI/env/config precedence remains deterministic
+
+### 8.3 Regression/Contract Tests
+
+1. `npm run nx -- run control-plane:test`
+2. `npm run nx -- run control-plane:build`
+3. `npm run validate:mcp-contracts`
+4. targeted manual smoke:
+   - `aop init` local CLI path
+   - `aop init` API-backed path with missing env var + key paste
+   - `aop run` using resulting config
+
+---
+
+## 9. Acceptance Criteria
+
+- [ ] `aop init` asks whether user will use a local CLI provider
+- [ ] local CLI path does not prompt for `provider_config_env`
+- [ ] API-backed path prompts for provider then env var name
+- [ ] init validates env var existence
+- [ ] if missing, init captures key and stores it in `.env` as `AOP_PROVIDER_CONFIG_ENV`
+- [ ] generated config references `AOP_PROVIDER_CONFIG_ENV` in fallback path
+- [ ] codex/claude local CLI runs do not fail on missing provider auth env var
+- [ ] gemini/API-backed runs still enforce credential presence
+- [ ] docs and schema descriptions reflect updated behavior
+- [ ] full test/build/contract validation passes
+
+---
+
+## 10. Risks and Mitigations
+
+### Risk 1: Plaintext Secret in `.env`
+
+- **Mitigation:** keep `.env` gitignored, avoid logging values, document rotation path.
+
+### Risk 2: Breaking Existing Indirection Behavior
+
+- **Mitigation:** keep legacy pointer interpretation in `AOP_PROVIDER_CONFIG_ENV` when it resolves to an existing env-var name.
+
+### Risk 3: User Confusion About Local vs API Mode
+
+- **Mitigation:** explicit init prompt text + README examples for both branches.
+
+### Risk 4: Credential Leaks in Tests
+
+- **Mitigation:** use placeholder test values and assert only presence/non-presence, never full output dumps.
+
+---
+
+## 11. Rollout and Migration
+
+1. Ship init + resolver changes together to avoid mismatched behavior.
+2. Existing repos require no migration; current `agents.yaml` remains valid.
+3. On first `aop init --force`, local-CLI users get cleaner `agents.yaml` without `provider_config_env`.
+4. Add release note callout:
+   - “Codex/Claude local CLI providers no longer require provider auth env by default.”
+   - “`AOP_PROVIDER_CONFIG_ENV` now supports direct credential fallback for init bootstrap.”

package/spec-files/outstanding/agentic_orchestrator_quality_adoption_execution_spec.md

@@ -22,6 +22,7 @@ The original monolithic execution document covered too many orthogonal concerns
 - `spec-files/agentic_orchestrator_observability_integrity_diagnostics_spec.md`
 
 Implements:
+
 - Q3 Deterministic Replay + Time-Travel Debugger
 - Q4 Tamper-Evident Evidence Chain
 - Q11 `aop doctor` Readiness + Drift Scanner
@@ -31,6 +32,7 @@ Implements:
 - `spec-files/agentic_orchestrator_planning_review_quality_spec.md`
 
 Implements:
+
 - Q1 Semantic Collision Engine
 - Q2 Plan Feasibility Scoring
 - Q7 Prompt Contract Test Harness
@@ -42,6 +44,7 @@ Implements:
 - `spec-files/agentic_orchestrator_enterprise_governance_dashboard_spec.md`
 
 Implements:
+
 - Q5 Compliance Policy Packs + Control Export
 - Q6 Dashboard AuthN/AuthZ + SSO + Audit Federation
 
@@ -50,6 +53,7 @@ Implements:
 - `spec-files/agentic_orchestrator_knowledge_canary_spec.md`
 
 Implements:
+
 - Q10 Cross-Feature Knowledge Graph
 - Q12 Progressive Merge Guardrails (Canary Verification)
 
@@ -57,20 +61,20 @@ Implements:
 
 ## 3. Feature Outcome Matrix
 
-| ID | Feature | Primary Outcome | Detailed Spec |
-|---|---|---|---|
-| Q1 | Semantic Collision Engine | Fewer hidden integration conflicts before build/QA | `agentic_orchestrator_planning_review_quality_spec.md` |
-| Q2 | Plan Feasibility Scoring | Early detection of weak plans | `agentic_orchestrator_planning_review_quality_spec.md` |
-| Q3 | Replay + Time-Travel | Faster incident RCA | `agentic_orchestrator_observability_integrity_diagnostics_spec.md` |
-| Q4 | Tamper-Evident Evidence | Stronger audit trust | `agentic_orchestrator_observability_integrity_diagnostics_spec.md` |
-| Q5 | Compliance Packs + Export | Faster enterprise onboarding | `agentic_orchestrator_enterprise_governance_dashboard_spec.md` |
-| Q6 | Dashboard Auth + SSO + Audit | Production-safe dashboard operation | `agentic_orchestrator_enterprise_governance_dashboard_spec.md` |
-| Q7 | Prompt Contract Tests | Prompt regression prevention | `agentic_orchestrator_planning_review_quality_spec.md` |
-| Q8 | Flaky Intelligence | Higher gate signal quality | `agentic_orchestrator_planning_review_quality_spec.md` |
-| Q9 | Review Briefs | Faster/high-quality human review | `agentic_orchestrator_planning_review_quality_spec.md` |
-| Q10 | Knowledge Graph | Better cross-feature reuse | `agentic_orchestrator_knowledge_canary_spec.md` |
-| Q11 | `aop doctor` | Faster readiness/drift triage | `agentic_orchestrator_observability_integrity_diagnostics_spec.md` |
-| Q12 | Canary Merge Verification | Safer merge promotion | `agentic_orchestrator_knowledge_canary_spec.md` |
+| ID  | Feature                      | Primary Outcome                                    | Detailed Spec                                                      |
+| --- | ---------------------------- | -------------------------------------------------- | ------------------------------------------------------------------ |
+| Q1  | Semantic Collision Engine    | Fewer hidden integration conflicts before build/QA | `agentic_orchestrator_planning_review_quality_spec.md`             |
+| Q2  | Plan Feasibility Scoring     | Early detection of weak plans                      | `agentic_orchestrator_planning_review_quality_spec.md`             |
+| Q3  | Replay + Time-Travel         | Faster incident RCA                                | `agentic_orchestrator_observability_integrity_diagnostics_spec.md` |
+| Q4  | Tamper-Evident Evidence      | Stronger audit trust                               | `agentic_orchestrator_observability_integrity_diagnostics_spec.md` |
+| Q5  | Compliance Packs + Export    | Faster enterprise onboarding                       | `agentic_orchestrator_enterprise_governance_dashboard_spec.md`     |
+| Q6  | Dashboard Auth + SSO + Audit | Production-safe dashboard operation                | `agentic_orchestrator_enterprise_governance_dashboard_spec.md`     |
+| Q7  | Prompt Contract Tests        | Prompt regression prevention                       | `agentic_orchestrator_planning_review_quality_spec.md`             |
+| Q8  | Flaky Intelligence           | Higher gate signal quality                         | `agentic_orchestrator_planning_review_quality_spec.md`             |
+| Q9  | Review Briefs                | Faster/high-quality human review                   | `agentic_orchestrator_planning_review_quality_spec.md`             |
+| Q10 | Knowledge Graph              | Better cross-feature reuse                         | `agentic_orchestrator_knowledge_canary_spec.md`                    |
+| Q11 | `aop doctor`                 | Faster readiness/drift triage                      | `agentic_orchestrator_observability_integrity_diagnostics_spec.md` |
+| Q12 | Canary Merge Verification    | Safer merge promotion                              | `agentic_orchestrator_knowledge_canary_spec.md`                    |
 
 ---
 
@@ -79,6 +83,7 @@ Implements:
 ### 4.1 Shared Constraints (Normative)
 
 All split specs MUST preserve:
+
 - deterministic MCP contract behavior
 - explicit user approval for merge promotion
 - policy/schema-driven validation
@@ -107,6 +112,7 @@ innovation:
 ### 4.3 Tool Contract Integration
 
 Any new tool introduced in a child spec MUST include:
+
 - catalog entry
 - input/output schemas
 - RBAC entry
@@ -120,40 +126,48 @@ Any new tool introduced in a child spec MUST include:
 ### M33 (must complete first)
 
 From `observability_integrity_diagnostics_spec`:
+
 - replay timeline
 - evidence chain verification
 - doctor scanner
 
 Dependency rationale:
+
 - provides baseline observability needed to debug and harden later milestones
 
 ### M34-M36
 
 From `planning_review_quality_spec`:
+
 - semantic collisions + feasibility scoring
 - prompt contracts
 - flaky intelligence
 - review briefs
 
 Dependency rationale:
+
 - quality-admission and review optimizations depend on M33 observability artifacts
 
 ### M37
 
 From `enterprise_governance_dashboard_spec`:
+
 - compliance export
 - dashboard auth/authz/audit federation
 
 Dependency rationale:
+
 - enterprise controls consume stable evidence/reporting from prior milestones
 
 ### M38
 
 From `knowledge_canary_spec`:
+
 - knowledge graph retrieval
 - canary merge verification
 
 Dependency rationale:
+
 - canary and graph quality improve with historical artifacts generated by M33-M37
 
 ---
@@ -192,6 +206,7 @@ For each milestone:
 ## 8. Definition of Done (Master)
 
 The split initiative is complete only when:
+
 1. Q1-Q12 are delivered via their child specs.
 2. all feature flags and schema/tool contracts are integrated and validated.
 3. no regressions occur in existing deterministic behavior.