agentic-orchestrator 0.1.26 → 0.1.28

package/spec-files/outstanding/shadow_workspace_implementation_spec.md

@@ -0,0 +1,1271 @@
+# Shadow Workspace Implementation Specification
+
+**Status:** Outstanding  
+**Created:** 2026-03-05  
+**Author:** System  
+**Scope:** Detailed implementation of shadow workspace strategy for interactive execution mode  
+**Parent Spec:** [Execution Mode Specification](../completed/agentic_orchestrator_execution_mode_spec.md)
+
+---
+
+## 0. Executive Summary
+
+### 0.1 Purpose
+
+This specification details the **shadow workspace strategy** for interactive execution mode, where agents work in an isolated copy of the feature worktree. Changes are validated before promotion to the real worktree, providing validation-before-write guarantees while maintaining the benefits of direct file system access.
+
+### 0.2 Key Concepts
+
+**Shadow Workspace:** An isolated copy of the feature worktree where the agent performs all file operations. Changes remain in the shadow until validated and promoted.
+
+**Promotion:** The atomic operation of copying validated changes from shadow workspace to real worktree.
+
+**Lifecycle:** Shadow creation → Agent execution → Checkpoint validation → Promotion (if valid) or Discard (if invalid) → Repeat
+
+### 0.3 Benefits Over Direct Worktree
+
+| Aspect               | Direct Worktree            | Shadow Workspace                |
+| -------------------- | -------------------------- | ------------------------------- |
+| Validation timing    | After write (risky)        | Before promotion (safe)         |
+| Rollback complexity  | Destructive revert         | Simple discard                  |
+| Real worktree safety | Can be corrupted           | Always valid                    |
+| Race conditions      | Possible during checkpoint | Eliminated (shadow is isolated) |
+| Agent restart cost   | Low                        | Medium (must recreate shadow)   |
+| Disk overhead        | None                       | 2x worktree size                |
+
+### 0.4 Trade-offs
+
+**Pros:**
+
+- Real worktree never corrupted
+- Validation before promotion (safe)
+- Easy rollback (discard shadow)
+- No race conditions
+- Atomic promotion guarantees
+
+**Cons:**
+
+- 2x disk space required
+- Promotion overhead (copy operation)
+- Agent must restart in fresh shadow after validation failure
+- More complex implementation
+
+### 0.5 When to Use
+
+**Use shadow workspace for:**
+
+- High-risk features (contract changes, schema migrations, API modifications)
+- Features with strict validation requirements
+- Production environments where safety > performance
+- Features modifying protected areas
+
+**Use direct worktree for:**
+
+- Low-risk features (documentation, tests, internal utilities)
+- Development environments where iteration speed matters
+- Features with simple validation rules
+- Trusted agents with proven track record
+
+---
+
+## 1. Architecture Overview
+
+### 1.1 Component Diagram
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                    SupervisorRuntime                        │
+│  ┌──────────────────────────────────────────────────────┐  │
+│  │         InteractiveExecutionService                   │  │
+│  │  ┌────────────────────────────────────────────────┐  │  │
+│  │  │      ShadowWorkspaceManager                    │  │  │
+│  │  │  - createShadow()                              │  │  │
+│  │  │  - promoteShadow()                             │  │  │
+│  │  │  - discardShadow()                             │  │  │
+│  │  │  - syncShadow()                                │  │  │
+│  │  └────────────────────────────────────────────────┘  │  │
+│  │                        │                              │  │
+│  │                        ▼                              │  │
+│  │  ┌────────────────────────────────────────────────┐  │  │
+│  │  │      WorktreeWatchdogService                   │  │  │
+│  │  │  - startWatching(shadowPath)                   │  │  │
+│  │  │  - getChangedFiles()                           │  │  │
+│  │  └────────────────────────────────────────────────┘  │  │
+│  │                        │                              │  │
+│  │                        ▼                              │  │
+│  │  ┌────────────────────────────────────────────────┐  │  │
+│  │  │      CheckpointService                         │  │  │
+│  │  │  - validateShadow()                            │  │  │
+│  │  │  - computeShadowDiff()                         │  │  │
+│  │  └────────────────────────────────────────────────┘  │  │
+│  └──────────────────────────────────────────────────────┘  │
+└─────────────────────────────────────────────────────────────┘
+                           │
+                           ▼
+              ┌────────────────────────┐
+              │   WorkerProvider       │
+              │  (Agent runs in shadow)│
+              └────────────────────────┘
+```
+
+### 1.2 Directory Structure
+
+```
+.worktrees/
+├── my_feature/                    # Real worktree (always valid)
+│   ├── src/
+│   ├── tests/
+│   └── ...
+└── my_feature.shadow/             # Shadow workspace (agent works here)
+    ├── src/
+    ├── tests/
+    └── ...
+
+.aop/features/my_feature/
+├── state.md
+├── plan.json
+├── checkpoints/
+│   ├── ckpt-001.diff              # Diff between real and shadow at checkpoint
+│   └── ckpt-002.diff
+└── shadow/
+    ├── metadata.json              # Shadow lifecycle metadata
+    └── promotion-history.jsonl    # Log of all promotions
+```
+
+---
+
+## 2. Shadow Workspace Lifecycle
+
+### 2.1 Phase 1: Shadow Creation
+
+**Trigger:** Feature starts in interactive mode with shadow strategy enabled
+
+**Steps:**
+
+1. Verify real worktree exists and is clean
+2. Create shadow directory (`.worktrees/<feature_id>.shadow`)
+3. Copy real worktree to shadow (fast copy with hardlinks where possible)
+4. Initialize shadow metadata
+5. Start watchdog on shadow directory
+6. Return shadow path to agent
+
+**Implementation:**
+
+```typescript
+interface ShadowWorkspaceManager {
+  createShadow(featureId: string): Promise<ShadowInfo>;
+  promoteShadow(featureId: string, validationResult: ValidationResult): Promise<PromotionResult>;
+  discardShadow(featureId: string, reason: string): Promise<void>;
+  syncShadow(featureId: string): Promise<void>;
+  getShadowInfo(featureId: string): Promise<ShadowInfo>;
+}
+
+interface ShadowInfo {
+  feature_id: string;
+  shadow_path: string;
+  real_worktree_path: string;
+  created_at: string;
+  last_sync_at: string;
+  generation: number; // Increments on each recreate
+  size_bytes: number;
+  file_count: number;
+}
+
+class ShadowWorkspaceManager {
+  async createShadow(featureId: string): Promise<ShadowInfo> {
+    const realPath = this.kernel.worktreePath(featureId);
+    const shadowPath = `${realPath}.shadow`;
+
+    // 1. Verify real worktree is clean
+    const status = await this.git.status(realPath);
+    if (status.modified.length > 0) {
+      throw new Error('Real worktree has uncommitted changes');
+    }
+
+    // 2. Remove existing shadow if present
+    if (await fs.pathExists(shadowPath)) {
+      await fs.remove(shadowPath);
+    }
+
+    // 3. Fast copy with hardlinks (copy-on-write)
+    await this.fastCopy(realPath, shadowPath);
+
+    // 4. Initialize metadata
+    const shadowInfo: ShadowInfo = {
+      feature_id: featureId,
+      shadow_path: shadowPath,
+      real_worktree_path: realPath,
+      created_at: new Date().toISOString(),
+      last_sync_at: new Date().toISOString(),
+      generation: await this.getNextGeneration(featureId),
+      size_bytes: await this.getDirectorySize(shadowPath),
+      file_count: await this.countFiles(shadowPath),
+    };
+
+    await this.persistShadowMetadata(featureId, shadowInfo);
+
+    return shadowInfo;
+  }
+
+  private async fastCopy(src: string, dest: string): Promise<void> {
+    // Use cp with reflink for copy-on-write (fast on modern filesystems)
+    await execa('cp', ['-r', '--reflink=auto', src, dest]);
+  }
+}
+```
+
+### 2.2 Phase 2: Agent Execution in Shadow
+
+**Trigger:** Shadow created, agent ready to start
+
+**Steps:**
+
+1. Spawn agent with `cwd` set to shadow path
+2. Agent performs file operations in shadow
+3. Watchdog monitors shadow for changes
+4. Track changed files for checkpoint
+
+**Key Points:**
+
+- Agent is unaware it's in a shadow (transparent)
+- All file operations are scoped to shadow
+- Real worktree remains untouched
+- Watchdog only monitors shadow directory
+
+**Implementation:**
+
+```typescript
+class InteractiveExecutionService {
+  async runWithShadowWorkspace(featureId: string): Promise<void> {
+    // 1. Create shadow
+    const shadowInfo = await this.shadowManager.createShadow(featureId);
+
+    // 2. Start watchdog on shadow
+    await this.watchdog.startWatching(featureId, shadowInfo.shadow_path);
+
+    // 3. Start checkpoint loop
+    const checkpointTimer = setInterval(async () => {
+      await this.checkpointShadow(featureId);
+    }, this.config.checkpoint_interval_ms);
+
+    // Also trigger on change threshold
+    this.watchdog.on('changeThreshold', async (fid) => {
+      if (fid === featureId) {
+        await this.checkpointShadow(featureId);
+      }
+    });
+
+    try {
+      // 4. Run agent in shadow
+      const result = await this.provider.runWorker({
+        role: 'builder',
+        feature_id: featureId,
+        working_directory: shadowInfo.shadow_path, // Agent works in shadow
+        execution_mode: 'interactive',
+      });
+
+      // 5. Final checkpoint and promotion
+      await this.finalCheckpointAndPromote(featureId);
+    } finally {
+      clearInterval(checkpointTimer);
+      await this.watchdog.stopWatching(featureId);
+    }
+  }
+}
+```
+
+### 2.3 Phase 3: Checkpoint Validation
+
+**Trigger:** Checkpoint interval elapsed OR change threshold reached
+
+**Steps:**
+
+1. Pause agent
+2. Compute diff between real worktree and shadow
+3. Validate diff against plan/policy/locks
+4. If valid: promote shadow changes to real worktree
+5. If invalid: notify agent, optionally discard shadow
+6. Resume agent (in shadow or fresh shadow)
+
+**Implementation:**
+
+```typescript
+class InteractiveExecutionService {
+  private async checkpointShadow(featureId: string): Promise<void> {
+    // 1. Pause agent
+    const pauseResult = await this.provider.pauseAgent(featureId, 10000);
+    if (!pauseResult.acknowledged) {
+      this.logger.warn(`Agent ${featureId} did not acknowledge pause`);
+    }
+
+    // 2. Wait for file writes to stabilize
+    await this.delay(100);
+
+    // 3. Compute diff between real and shadow
+    const shadowInfo = await this.shadowManager.getShadowInfo(featureId);
+    const diff = await this.computeShadowDiff(
+      shadowInfo.real_worktree_path,
+      shadowInfo.shadow_path,
+    );
+
+    // 4. Validate diff
+    const validation = await this.checkpointService.validateCheckpoint(featureId, diff);
+
+    // 5. Record checkpoint
+    await this.checkpointService.recordCheckpoint(featureId, {
+      checkpoint_id: this.generateCheckpointId(),
+      timestamp: new Date().toISOString(),
+      files_changed: this.parseDiffFiles(diff),
+      validation_status: validation.valid ? 'valid' : 'invalid',
+      violations: validation.violations,
+      severity: validation.severity,
+      diff_snapshot: await this.storeDiff(featureId, diff),
+    });
+
+    // 6. Handle validation result
+    if (validation.valid) {
+      await this.promoteShadowChanges(featureId, shadowInfo, diff);
+    } else {
+      await this.handleValidationFailure(featureId, validation);
+    }
+
+    // 7. Resume agent
+    await this.provider.resumeAgent(featureId);
+  }
+
+  private async computeShadowDiff(realPath: string, shadowPath: string): Promise<string> {
+    // Use git diff to compare real and shadow
+    // This captures all changes agent made in shadow
+    const result = await execa('git', ['diff', '--no-index', '--no-color', realPath, shadowPath], {
+      reject: false,
+    }); // Exit code 1 means differences found (expected)
+
+    return result.stdout;
+  }
+}
+```
+
+### 2.4 Phase 4: Promotion (Valid Changes)
+
+**Trigger:** Checkpoint validation passed
+
+**Steps:**
+
+1. Verify real worktree is still clean (no external changes)
+2. Apply shadow diff to real worktree (atomic)
+3. Sync shadow with real worktree (reset shadow to match real)
+4. Log promotion in history
+5. Continue agent execution in shadow
+
+**Promotion Strategies:**
+
+#### Strategy A: Atomic Copy (Default)
+
+Copy changed files from shadow to real worktree atomically.
+
+```typescript
+class ShadowWorkspaceManager {
+  async promoteShadow(
+    featureId: string,
+    validationResult: ValidationResult,
+  ): Promise<PromotionResult> {
+    const shadowInfo = await this.getShadowInfo(featureId);
+    const changedFiles = validationResult.files_changed;
+
+    // 1. Verify real worktree is clean
+    const status = await this.git.status(shadowInfo.real_worktree_path);
+    if (status.modified.length > 0) {
+      throw new Error('Real worktree modified externally during shadow execution');
+    }
+
+    // 2. Atomic promotion: copy changed files
+    const tempDir = `${shadowInfo.real_worktree_path}.promotion-temp`;
+    await fs.ensureDir(tempDir);
+
+    try {
+      // Copy changed files to temp
+      for (const file of changedFiles) {
+        const shadowFile = path.join(shadowInfo.shadow_path, file);
+        const tempFile = path.join(tempDir, file);
+
+        if (await fs.pathExists(shadowFile)) {
+          await fs.copy(shadowFile, tempFile);
+        }
+      }
+
+      // Atomic move from temp to real
+      for (const file of changedFiles) {
+        const tempFile = path.join(tempDir, file);
+        const realFile = path.join(shadowInfo.real_worktree_path, file);
+
+        if (await fs.pathExists(tempFile)) {
+          await fs.move(tempFile, realFile, { overwrite: true });
+        } else {
+          // File was deleted in shadow
+          await fs.remove(realFile);
+        }
+      }
+
+      // 3. Sync shadow with real (reset shadow)
+      await this.syncShadow(featureId);
+
+      // 4. Log promotion
+      await this.logPromotion(featureId, {
+        timestamp: new Date().toISOString(),
+        files_promoted: changedFiles,
+        validation_result: validationResult,
+      });
+
+      return {
+        success: true,
+        files_promoted: changedFiles.length,
+        promotion_time_ms: Date.now() - startTime,
+      };
+    } finally {
+      await fs.remove(tempDir);
+    }
+  }
+
+  async syncShadow(featureId: string): Promise<void> {
+    const shadowInfo = await this.getShadowInfo(featureId);
+
+    // Remove shadow and recreate from real worktree
+    await fs.remove(shadowInfo.shadow_path);
+    await this.fastCopy(shadowInfo.real_worktree_path, shadowInfo.shadow_path);
+
+    // Update metadata
+    shadowInfo.last_sync_at = new Date().toISOString();
+    await this.persistShadowMetadata(featureId, shadowInfo);
+  }
+}
+```
+
+#### Strategy B: Incremental Sync
+
+Only sync changed files (faster for large worktrees).
+
+```typescript
+async syncShadowIncremental(featureId: string, changedFiles: string[]): Promise<void> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+
+  // Copy only changed files from real to shadow
+  for (const file of changedFiles) {
+    const realFile = path.join(shadowInfo.real_worktree_path, file);
+    const shadowFile = path.join(shadowInfo.shadow_path, file);
+
+    if (await fs.pathExists(realFile)) {
+      await fs.copy(realFile, shadowFile, { overwrite: true });
+    } else {
+      await fs.remove(shadowFile);
+    }
+  }
+
+  shadowInfo.last_sync_at = new Date().toISOString();
+  await this.persistShadowMetadata(featureId, shadowInfo);
+}
+```
+
+### 2.5 Phase 5: Discard (Invalid Changes)
+
+**Trigger:** Checkpoint validation failed
+
+**Steps:**
+
+1. Notify agent of violations
+2. Decide: discard shadow OR keep shadow for agent to fix
+3. If discard: remove shadow, recreate fresh from real worktree
+4. If keep: agent continues in shadow with violation feedback
+5. Resume agent
+
+**Discard Strategies:**
+
+#### Strategy A: Full Discard (Default for critical violations)
+
+```typescript
+async discardShadow(featureId: string, reason: string): Promise<void> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+
+  // 1. Log discard event
+  await this.logDiscard(featureId, {
+    timestamp: new Date().toISOString(),
+    reason,
+    generation: shadowInfo.generation,
+  });
+
+  // 2. Remove shadow
+  await fs.remove(shadowInfo.shadow_path);
+
+  // 3. Recreate fresh shadow from real worktree
+  await this.createShadow(featureId);
+
+  // 4. Notify agent
+  await this.provider.sendMessage(featureId, {
+    type: 'shadow_discarded',
+    severity: 'error',
+    content: `Shadow workspace discarded due to validation failure: ${reason}`,
+    structured_data: { reason, new_generation: shadowInfo.generation + 1 },
+    requires_acknowledgment: true,
+  });
+}
+```
+
+#### Strategy B: Partial Discard (Revert only violated files)
+
+```typescript
+async discardViolatedFiles(
+  featureId: string,
+  violations: ValidationViolation[]
+): Promise<void> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+  const violatedFiles = new Set(violations.map(v => v.file_path));
+
+  // Revert only violated files in shadow (copy from real worktree)
+  for (const file of violatedFiles) {
+    const realFile = path.join(shadowInfo.real_worktree_path, file);
+    const shadowFile = path.join(shadowInfo.shadow_path, file);
+
+    if (await fs.pathExists(realFile)) {
+      await fs.copy(realFile, shadowFile, { overwrite: true });
+    } else {
+      await fs.remove(shadowFile);
+    }
+  }
+
+  // Notify agent which files were reverted
+  await this.provider.sendMessage(featureId, {
+    type: 'partial_revert',
+    severity: 'warning',
+    content: `Reverted ${violatedFiles.size} files with violations`,
+    structured_data: {
+      reverted_files: Array.from(violatedFiles),
+      violations
+    },
+    requires_acknowledgment: true,
+  });
+}
+```
+
+---
+
+## 3. Configuration
+
+### 3.1 Schema Changes
+
+**File:** `agentic/orchestrator/schemas/agents.schema.json`
+
+```json
+{
+  "runtime": {
+    "execution_mode": "interactive",
+    "interactive": {
+      "strategy": "shadow_workspace",
+      "shadow_workspace": {
+        "enabled": true,
+        "promotion_strategy": "atomic",
+        "sync_strategy": "full",
+        "discard_strategy": "full",
+        "cleanup_on_failure": true,
+        "max_shadow_size_mb": 2048,
+        "max_shadow_age_hours": 24,
+        "hardlink_optimization": true,
+        "violation_handling": {
+          "critical": "discard_full",
+          "error": "discard_violated_files",
+          "warning": "keep_shadow",
+          "info": "keep_shadow"
+        }
+      }
+    }
+  }
+}
+```
+
+### 3.2 Configuration Options
+
+| Option                  | Type    | Default   | Description                                                  |
+| ----------------------- | ------- | --------- | ------------------------------------------------------------ |
+| `enabled`               | boolean | `false`   | Enable shadow workspace strategy                             |
+| `promotion_strategy`    | enum    | `atomic`  | How to promote changes (`atomic` \| `incremental`)           |
+| `sync_strategy`         | enum    | `full`    | How to sync shadow after promotion (`full` \| `incremental`) |
+| `discard_strategy`      | enum    | `full`    | How to discard invalid changes (`full` \| `partial`)         |
+| `cleanup_on_failure`    | boolean | `true`    | Delete shadow after validation failure                       |
+| `max_shadow_size_mb`    | number  | `2048`    | Maximum shadow workspace size (MB)                           |
+| `max_shadow_age_hours`  | number  | `24`      | Maximum shadow age before forced cleanup                     |
+| `hardlink_optimization` | boolean | `true`    | Use hardlinks for fast copy (copy-on-write)                  |
+| `violation_handling`    | object  | See above | Per-severity discard strategy                                |
+
+### 3.3 Feature-Level Override
+
+Features can override shadow strategy in `state.md`:
+
+```yaml
+---
+feature_id: my_feature
+execution_mode: interactive
+shadow_workspace:
+  enabled: true
+  discard_strategy: partial # Override: keep valid changes
+---
+```
+
+---
+
+## 4. Performance Optimization
+
+### 4.1 Fast Copy with Hardlinks
+
+**Problem:** Copying large worktrees (>1GB) is slow.
+
+**Solution:** Use copy-on-write (reflink) or hardlinks.
+
+```typescript
+async fastCopy(src: string, dest: string): Promise<void> {
+  if (this.config.hardlink_optimization) {
+    // Try reflink first (copy-on-write on btrfs, xfs, apfs)
+    try {
+      await execa('cp', ['-r', '--reflink=always', src, dest]);
+      return;
+    } catch {
+      // Reflink not supported, fall back to hardlinks
+    }
+
+    // Use hardlinks (fast, but files share inodes until modified)
+    await execa('cp', ['-rl', src, dest]);
+  } else {
+    // Standard copy (slow but safe)
+    await fs.copy(src, dest);
+  }
+}
+```
+
+**Benchmark:**
+
+- Standard copy (1GB worktree): ~10s
+- Hardlink copy (1GB worktree): ~500ms
+- Reflink copy (1GB worktree): ~100ms
+
+### 4.2 Incremental Sync
+
+**Problem:** Full shadow recreation after each promotion is slow.
+
+**Solution:** Only sync changed files.
+
+```typescript
+async syncShadowIncremental(featureId: string, promotedFiles: string[]): Promise<void> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+
+  // Only copy promoted files from real to shadow
+  await pMap(promotedFiles, async (file) => {
+    const realFile = path.join(shadowInfo.real_worktree_path, file);
+    const shadowFile = path.join(shadowInfo.shadow_path, file);
+    await fs.copy(realFile, shadowFile, { overwrite: true });
+  }, { concurrency: 10 });
+}
+```
+
+**Benchmark:**
+
+- Full sync (1GB worktree): ~10s
+- Incremental sync (50 files): ~200ms
+
+### 4.3 Lazy Shadow Creation
+
+**Problem:** Creating shadow upfront delays agent start.
+
+**Solution:** Create shadow in background while agent initializes.
+
+```typescript
+async runWithLazyShadow(featureId: string): Promise<void> {
+  // Start shadow creation in background
+  const shadowPromise = this.shadowManager.createShadow(featureId);
+
+  // Initialize agent in parallel
+  const agentPromise = this.provider.createSession('builder', featureId, systemPrompt);
+
+  // Wait for both
+  const [shadowInfo, session] = await Promise.all([shadowPromise, agentPromise]);
+
+  // Now run agent in shadow
+  await this.provider.runWorker({
+    working_directory: shadowInfo.shadow_path,
+    // ...
+  });
+}
+```
+
+### 4.4 Shadow Pooling
+
+**Problem:** Creating/destroying shadows repeatedly is wasteful.
+
+**Solution:** Maintain pool of pre-created shadows.
+
+```typescript
+class ShadowPool {
+  private pool = new Map<string, ShadowInfo[]>();
+  private readonly POOL_SIZE = 3;
+
+  async getOrCreateShadow(featureId: string): Promise<ShadowInfo> {
+    const available = this.pool.get(featureId) || [];
+
+    if (available.length > 0) {
+      return available.pop()!;
+    }
+
+    return await this.shadowManager.createShadow(featureId);
+  }
+
+  async returnShadow(featureId: string, shadowInfo: ShadowInfo): Promise<void> {
+    const pool = this.pool.get(featureId) || [];
+
+    if (pool.length < this.POOL_SIZE) {
+      // Clean shadow and return to pool
+      await this.cleanShadow(shadowInfo);
+      pool.push(shadowInfo);
+      this.pool.set(featureId, pool);
+    } else {
+      // Pool full, discard
+      await fs.remove(shadowInfo.shadow_path);
+    }
+  }
+}
+```
+
+---
+
+## 5. Error Handling and Edge Cases
+
+### 5.1 Shadow Creation Failures
+
+**Scenario:** Disk full, permission denied, or real worktree corrupted.
+
+**Handling:**
+
+```typescript
+async createShadow(featureId: string): Promise<ShadowInfo> {
+  try {
+    // Attempt shadow creation
+    return await this._createShadow(featureId);
+  } catch (error) {
+    if (error.code === 'ENOSPC') {
+      // Disk full: cleanup old shadows and retry
+      await this.cleanupOldShadows();
+      return await this._createShadow(featureId);
+    } else if (error.code === 'EACCES') {
+      // Permission denied: escalate to human
+      throw new Error('Shadow creation failed: permission denied');
+    } else {
+      // Unknown error: fall back to direct worktree mode
+      this.logger.error(`Shadow creation failed: ${error.message}`);
+      throw new Error('Shadow creation failed, falling back to direct worktree');
+    }
+  }
+}
+```
+
+### 5.2 Promotion Failures
+
+**Scenario:** Real worktree modified externally during shadow execution.
+
+**Handling:**
+
+```typescript
+async promoteShadow(featureId: string): Promise<PromotionResult> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+
+  // Verify real worktree is clean
+  const status = await this.git.status(shadowInfo.real_worktree_path);
+
+  if (status.modified.length > 0) {
+    // Real worktree modified externally
+    this.logger.error(`Real worktree modified externally: ${status.modified}`);
+
+    // Options:
+    // 1. Abort promotion, notify human
+    // 2. Attempt 3-way merge
+    // 3. Force promotion (overwrite external changes)
+
+    // Default: abort and escalate
+    throw new Error('Promotion aborted: real worktree modified externally');
+  }
+
+  // Proceed with promotion
+  return await this._promoteShadow(featureId);
+}
+```
+
+### 5.3 Shadow Corruption
+
+**Scenario:** Agent corrupts shadow (e.g., deletes `.git`, creates infinite loop).
+
+**Handling:**
+
+```typescript
+async validateShadowIntegrity(featureId: string): Promise<boolean> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+
+  // Check .git directory exists
+  const gitDir = path.join(shadowInfo.shadow_path, '.git');
+  if (!await fs.pathExists(gitDir)) {
+    this.logger.error(`Shadow corrupted: .git directory missing`);
+    return false;
+  }
+
+  // Check shadow size is reasonable
+  const size = await this.getDirectorySize(shadowInfo.shadow_path);
+  if (size > this.config.max_shadow_size_mb * 1024 * 1024) {
+    this.logger.error(`Shadow corrupted: size exceeds limit (${size} bytes)`);
+    return false;
+  }
+
+  // Check file count is reasonable
+  const fileCount = await this.countFiles(shadowInfo.shadow_path);
+  if (fileCount > 100000) {
+    this.logger.error(`Shadow corrupted: too many files (${fileCount})`);
+    return false;
+  }
+
+  return true;
+}
+
+async checkpointShadow(featureId: string): Promise<void> {
+  // Validate shadow integrity before checkpoint
+  const isValid = await this.validateShadowIntegrity(featureId);
+
+  if (!isValid) {
+    // Shadow corrupted: discard and recreate
+    await this.discardShadow(featureId, 'shadow_corrupted');
+    return;
+  }
+
+  // Proceed with normal checkpoint
+  // ...
+}
+```
+
+### 5.4 Concurrent Promotions
+
+**Scenario:** Multiple checkpoints trigger simultaneously (race condition).
+
+**Handling:**
+
+```typescript
+class ShadowWorkspaceManager {
+  private promotionLocks = new Map<string, Promise<PromotionResult>>();
+
+  async promoteShadow(featureId: string): Promise<PromotionResult> {
+    // Serialize promotions per feature
+    const existingPromotion = this.promotionLocks.get(featureId);
+    if (existingPromotion) {
+      this.logger.warn(`Promotion already in progress for ${featureId}, waiting...`);
+      return await existingPromotion;
+    }
+
+    const promotionPromise = this._promoteShadow(featureId);
+    this.promotionLocks.set(featureId, promotionPromise);
+
+    try {
+      return await promotionPromise;
+    } finally {
+      this.promotionLocks.delete(featureId);
+    }
+  }
+}
+```
+
+---
+
+## 6. Monitoring and Observability
+
+### 6.1 Metrics
+
+**Shadow lifecycle metrics:**
+
+- Shadow creation time (p50, p95, p99)
+- Shadow size distribution
+- Promotion time (p50, p95, p99)
+- Promotion success rate
+- Discard rate (by severity)
+- Shadow age distribution
+
+**Resource metrics:**
+
+- Total disk usage (all shadows)
+- Disk usage per shadow
+- Shadow count (active, pooled, orphaned)
+- Hardlink effectiveness (% space saved)
+
+**Validation metrics:**
+
+- Validation pass rate (shadow vs direct worktree)
+- Violations per checkpoint (shadow vs direct worktree)
+- Time to first violation
+
+### 6.2 Logging
+
+**Shadow events to log:**
+
+```typescript
+interface ShadowEvent {
+  event_type: 'created' | 'promoted' | 'discarded' | 'synced' | 'corrupted';
+  feature_id: string;
+  timestamp: string;
+  generation: number;
+  details: Record<string, unknown>;
+}
+
+// Example log entries
+{
+  "event_type": "created",
+  "feature_id": "my_feature",
+  "timestamp": "2026-03-05T17:45:00.000Z",
+  "generation": 1,
+  "details": {
+    "shadow_path": ".worktrees/my_feature.shadow",
+    "size_bytes": 1048576000,
+    "file_count": 5432,
+    "creation_time_ms": 523
+  }
+}
+
+{
+  "event_type": "promoted",
+  "feature_id": "my_feature",
+  "timestamp": "2026-03-05T17:46:30.000Z",
+  "generation": 1,
+  "details": {
+    "files_promoted": 12,
+    "promotion_time_ms": 145,
+    "validation_status": "valid"
+  }
+}
+
+{
+  "event_type": "discarded",
+  "feature_id": "my_feature",
+  "timestamp": "2026-03-05T17:47:15.000Z",
+  "generation": 1,
+  "details": {
+    "reason": "validation_failed",
+    "violations": ["Path 'src/config.ts' not in allowed_areas"],
+    "severity": "error"
+  }
+}
+```
+
+### 6.3 Alerts
+
+**Alert conditions:**
+
+- Shadow creation time > 5s
+- Promotion time > 2s
+- Discard rate > 30%
+- Total shadow disk usage > 80% of limit
+- Shadow age > 24 hours (orphaned shadow)
+- Shadow corruption detected
+
+---
+
+## 7. Testing Strategy
+
+### 7.1 Unit Tests
+
+**ShadowWorkspaceManager:**
+
+- `createShadow()` creates shadow with correct structure
+- `promoteShadow()` copies only changed files
+- `discardShadow()` removes shadow and recreates
+- `syncShadow()` resets shadow to match real worktree
+- Fast copy uses hardlinks when available
+- Promotion is atomic (all or nothing)
+
+**CheckpointService:**
+
+- `computeShadowDiff()` correctly compares real and shadow
+- `validateShadow()` enforces plan/policy/locks
+- Validation failures trigger correct discard strategy
+
+### 7.2 Integration Tests
+
+**Shadow lifecycle:**
+
+1. Create shadow → Agent modifies files → Checkpoint validates → Promote → Verify real worktree updated
+2. Create shadow → Agent violates policy → Checkpoint fails → Discard → Verify shadow recreated
+3. Create shadow → Agent corrupts shadow → Checkpoint detects → Discard → Verify recovery
+
+**Concurrent features:**
+
+1. Start 3 features in shadow mode simultaneously
+2. Verify each has isolated shadow
+3. Verify promotions don't interfere
+4. Verify disk usage is tracked correctly
+
+### 7.3 Performance Tests
+
+**Benchmarks:**
+
+- Shadow creation time vs worktree size (100MB, 1GB, 10GB)
+- Promotion time vs changed file count (10, 100, 1000 files)
+- Sync time (full vs incremental)
+- Disk space overhead (hardlinks vs standard copy)
+
+**Load tests:**
+
+- 10 concurrent features in shadow mode
+- 100 checkpoints per feature
+- Verify no memory leaks
+- Verify disk cleanup works
+
+---
+
+## 8. Migration Path
+
+### 8.1 Phase 1: Infrastructure (Week 1)
+
+- Implement `ShadowWorkspaceManager` service
+- Add shadow configuration to `agents.schema.json`
+- Implement fast copy with hardlink optimization
+- Add shadow metadata persistence
+- Unit tests (>= 90% coverage)
+
+### 8.2 Phase 2: Lifecycle Integration (Week 2)
+
+- Integrate shadow creation into `InteractiveExecutionService`
+- Implement checkpoint validation for shadow
+- Implement promotion logic (atomic strategy)
+- Implement discard logic (full strategy)
+- Integration tests
+
+### 8.3 Phase 3: Optimization (Week 3)
+
+- Implement incremental sync
+- Implement partial discard
+- Add shadow pooling
+- Performance benchmarks
+- Load tests
+
+### 8.4 Phase 4: Monitoring and Rollout (Week 4)
+
+- Add shadow metrics and logging
+- Add shadow alerts
+- Dashboard integration (shadow status in RuntimeInspector)
+- Documentation
+- Beta rollout (opt-in via config)
+
+---
+
+## 9. Acceptance Criteria
+
+### 9.1 Must Have
+
+- [ ] `ShadowWorkspaceManager` implemented with all methods
+- [ ] Shadow creation uses hardlinks (< 1s for 1GB worktree)
+- [ ] Promotion is atomic (all files or none)
+- [ ] Discard recreates fresh shadow from real worktree
+- [ ] Validation enforces plan/policy/locks (same as direct worktree)
+- [ ] Real worktree never corrupted by agent
+- [ ] Shadow integrity checks detect corruption
+- [ ] Concurrent features have isolated shadows
+- [ ] All unit tests pass (>= 90% coverage)
+- [ ] All integration tests pass
+
+### 9.2 Should Have
+
+- [ ] Incremental sync (< 500ms for 50 files)
+- [ ] Partial discard (revert only violated files)
+- [ ] Shadow pooling (reduce creation overhead)
+- [ ] Metrics and logging for all shadow events
+- [ ] Dashboard displays shadow status
+- [ ] Alerts for shadow issues
+- [ ] Performance benchmarks documented
+
+### 9.3 Nice to Have
+
+- [ ] Shadow diff viewer in dashboard
+- [ ] Shadow history timeline
+- [ ] Manual shadow promotion command (`aop promote --feature-id <id>`)
+- [ ] Shadow inspection command (`aop shadow inspect --feature-id <id>`)
+- [ ] Shadow cleanup command (`aop shadow cleanup --all`)
+
+---
+
+## 10. Comparison with Direct Worktree
+
+### 10.1 Safety
+
+| Aspect                        | Direct Worktree    | Shadow Workspace              |
+| ----------------------------- | ------------------ | ----------------------------- |
+| Real worktree corruption risk | High               | None                          |
+| Validation timing             | After write        | Before promotion              |
+| Rollback complexity           | Destructive revert | Simple discard                |
+| Race conditions               | Possible           | Eliminated                    |
+| Agent can escape worktree     | Yes (via symlinks) | Yes (but doesn't affect real) |
+
+**Winner:** Shadow Workspace (much safer)
+
+### 10.2 Performance
+
+| Aspect                | Direct Worktree | Shadow Workspace           |
+| --------------------- | --------------- | -------------------------- |
+| Agent start time      | Instant         | +500ms (shadow creation)   |
+| Checkpoint overhead   | Low             | Medium (diff + promotion)  |
+| Disk usage            | 1x              | 2x (with hardlinks: ~1.1x) |
+| Validation speed      | Same            | Same                       |
+| Agent iteration speed | Fast            | Fast (works in shadow)     |
+
+**Winner:** Direct Worktree (slightly faster)
+
+### 10.3 Complexity
+
+| Aspect                       | Direct Worktree | Shadow Workspace |
+| ---------------------------- | --------------- | ---------------- |
+| Implementation lines of code | ~500            | ~1500            |
+| Service dependencies         | 2               | 3                |
+| Configuration options        | 5               | 12               |
+| Error scenarios              | 8               | 15               |
+| Testing surface              | Medium          | Large            |
+
+**Winner:** Direct Worktree (simpler)
+
+### 10.4 Recommendation
+
+**Use Shadow Workspace when:**
+
+- Safety is paramount (production environments)
+- Features modify contracts, schemas, or migrations
+- Agent is untrusted or experimental
+- Validation failures are expected to be common
+- Rollback needs to be fast and reliable
+
+**Use Direct Worktree when:**
+
+- Iteration speed is critical (development environments)
+- Features are low-risk (docs, tests, internal utilities)
+- Agent is trusted and proven
+- Validation failures are rare
+- Disk space is constrained
+
+**Hybrid approach:** Start with direct worktree, automatically switch to shadow workspace on first validation failure.
+
+---
+
+## 11. Future Enhancements
+
+### 11.1 Multi-Shadow Branching
+
+**Concept:** Agent can create multiple shadow branches to explore alternatives.
+
+**Use case:** Agent wants to try two different implementation approaches in parallel.
+
+```typescript
+interface ShadowBranch {
+  branch_id: string;
+  parent_shadow_id: string;
+  created_at: string;
+  description: string;
+}
+
+async createShadowBranch(featureId: string, description: string): Promise<ShadowBranch> {
+  const currentShadow = await this.getShadowInfo(featureId);
+  const branchShadow = await this.copyShadow(currentShadow, `${featureId}.branch-${uuid()}`);
+
+  return {
+    branch_id: branchShadow.shadow_path,
+    parent_shadow_id: currentShadow.shadow_path,
+    created_at: new Date().toISOString(),
+    description,
+  };
+}
+```
+
+### 11.2 Shadow Snapshots
+
+**Concept:** Create lightweight snapshots of shadow at any point for easy rollback.
+
+**Use case:** Agent wants to experiment but be able to revert to a known-good state.
+
+```typescript
+async createShadowSnapshot(featureId: string, label: string): Promise<SnapshotInfo> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+
+  // Use git to create snapshot (lightweight)
+  await this.git.commit(shadowInfo.shadow_path, `Snapshot: ${label}`);
+  const commitHash = await this.git.getHeadCommit(shadowInfo.shadow_path);
+
+  return {
+    snapshot_id: commitHash,
+    label,
+    created_at: new Date().toISOString(),
+  };
+}
+
+async rollbackToSnapshot(featureId: string, snapshotId: string): Promise<void> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+  await this.git.reset(shadowInfo.shadow_path, snapshotId, { hard: true });
+}
+```
+
+### 11.3 Shadow Diff Streaming
+
+**Concept:** Stream shadow changes to dashboard in real-time.
+
+**Use case:** Human observer wants to watch agent work live.
+
+```typescript
+async streamShadowChanges(featureId: string): AsyncIterableIterator<FileChange> {
+  const shadowInfo = await this.getShadowInfo(featureId);
+
+  const watcher = chokidar.watch(shadowInfo.shadow_path);
+
+  for await (const event of watcher) {
+    yield {
+      event_type: event.type,
+      file_path: event.path,
+      timestamp: new Date().toISOString(),
+      diff: await this.computeFileDiff(event.path),
+    };
+  }
+}
+```
+
+### 11.4 Shadow Collaboration
+
+**Concept:** Multiple agents work in same shadow with conflict resolution.
+
+**Use case:** Planner and builder agents collaborate on same feature.
+
+**Challenge:** Requires sophisticated merge logic and coordination.
+
+---
+
+## 12. References
+
+### 12.1 Related Specifications
+
+- **Parent Spec:** [Execution Mode Specification](../completed/agentic_orchestrator_execution_mode_spec.md) - Overall architecture for deterministic vs interactive modes
+- **Related:** [Runtime Inspection Specification](./agentic_orchestrator_runtime_inspection_spec.md) - Dashboard integration for shadow monitoring
+
+### 12.2 External Resources
+
+- **Copy-on-Write Filesystems:** [Btrfs reflink](https://btrfs.wiki.kernel.org/index.php/Reflink), [XFS reflink](https://www.kernel.org/doc/Documentation/filesystems/xfs.txt)
+- **Hardlinks:** [Linux hardlink documentation](https://man7.org/linux/man-pages/man2/link.2.html)
+- **File System Monitoring:** [chokidar](https://github.com/paulmillr/chokidar)
+
+### 12.3 Implementation Files
+
+**New files to create:**
+
+- `apps/control-plane/src/application/services/shadow-workspace-manager.ts`
+- `apps/control-plane/src/application/services/shadow-pool.ts`
+- `apps/control-plane/test/shadow-workspace-manager.spec.ts`
+- `apps/control-plane/test/shadow-pool.spec.ts`
+
+**Files to modify:**
+
+- `apps/control-plane/src/supervisor/worker-decision-loop.ts` - Add shadow mode execution path
+- `apps/control-plane/src/application/services/checkpoint-service.ts` - Add shadow diff computation
+- `agentic/orchestrator/schemas/agents.schema.json` - Add shadow configuration
+- `agentic/orchestrator/schemas/state.schema.json` - Add shadow metadata
+
+---
+
+**End of Specification**