agentic-orchestrator 0.1.28 → 0.2.0

package/docs/superpowers/plans/2026-03-29-organizer-sidecar-runtime-loop.md

@@ -0,0 +1,1289 @@
+# Organizer Sidecar Runtime Loop Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Implement the polling-based organizer sidecar service that invokes the LLM to analyze feature specs and writes the ordering artifact to disk (Layer 1 of the ordering agent).
+
+**Architecture:** A standalone `OrganizerSidecarService` class polls on a timer, collects spec paths from execution control and runtime state, calls `provider.runWorker()` with the organizer role, validates the JSON response against the AJV schema, and writes the artifact atomically. Wired into `SupervisorRuntime` alongside the enrollment service.
+
+**Tech Stack:** TypeScript, Vitest, AJV (JSON Schema draft-2020-12), Node.js fs (atomic writes)
+
+**Spec:** `docs/superpowers/specs/2026-03-29-organizer-sidecar-runtime-loop-design.md`
+
+---
+
+## File Map
+
+| File                                                             | Action | Responsibility                                                                  |
+| :--------------------------------------------------------------- | :----- | :------------------------------------------------------------------------------ |
+| `apps/control-plane/src/supervisor/organizer-sidecar-service.ts` | Create | Polling loop, spec collection, LLM dispatch, response parsing, artifact writing |
+| `apps/control-plane/src/supervisor/runtime.ts`                   | Modify | Wire sidecar into lifecycle, add `organizer` to `resolveRoleSessionId()`        |
+| `config/agentic/orchestrator/prompts/organizer.system.md`        | Modify | Add dependency detection signals section                                        |
+| `apps/control-plane/test/organizer-sidecar-service.spec.ts`      | Create | Unit tests for the sidecar service                                              |
+
+---
+
+### Task 1: Enhance Organizer System Prompt
+
+**Files:**
+
+- Modify: `config/agentic/orchestrator/prompts/organizer.system.md`
+
+- [ ] **Step 1: Add dependency detection signals section**
+
+Insert the following between the "Analysis discipline" section and "Dependency classification" section in `config/agentic/orchestrator/prompts/organizer.system.md`:
+
+```markdown
+## Dependency detection signals
+
+### Explicit signals (highest confidence)
+
+- `Depends On:` field in spec frontmatter or body — authoritative, never ignore
+- Spec A references spec B by feature ID (e.g. "requires feature-xyz to be merged first")
+- Spec B references spec A — if spec B says "builds on feature-xyz" or "extends the work in feature-xyz," that means B depends on A
+- Spec explicitly lists file paths that another spec also explicitly lists — when both specs declare they will modify the same file(s), the one that creates/restructures goes first
+- "After X is complete" / "Once X lands" language in either direction
+
+### Structural signals (medium confidence — classify as `inferred_dependency`)
+
+- Two specs modify the same file paths — the one that creates/restructures should go first
+- Spec A adds an interface that spec B consumes (import/type dependency)
+- Spec A creates a database table/schema that spec B queries
+- Spec A introduces a configuration key that spec B reads
+
+### Semantic signals (lower confidence — classify carefully)
+
+- Specs in the same domain area (e.g. both touch auth) don't necessarily depend on each other — only flag if one's changes would conflict with or build upon the other's
+- Shared terminology alone is not a dependency — look for actual data/control flow relationships
+- If two specs could be implemented in either order without merge conflicts or behavioral issues, they are independent
+
+### Not dependencies
+
+- Specs that happen to be in the same directory but touch different concerns
+- Specs that share a programming language or framework
+- Specs that were submitted in the same request (ordering within a request is the organizer's job to determine, not assume)
+```
+
+- [ ] **Step 2: Verify the prompt file is well-formed**
+
+Read the file back and confirm the sections flow: "Organizer objective" → "Analysis discipline" → "Dependency detection signals" → "Dependency classification" → "Cycle detection" → "Output format" → "Constraints".
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add config/agentic/orchestrator/prompts/organizer.system.md
+git commit -m "feat: add dependency detection signals to organizer system prompt"
+```
+
+---
+
+### Task 2: Implement OrganizerSidecarService
+
+**Files:**
+
+- Create: `apps/control-plane/src/supervisor/organizer-sidecar-service.ts`
+
+- [ ] **Step 1: Create the service file with types and constructor**
+
+Create `apps/control-plane/src/supervisor/organizer-sidecar-service.ts`:
+
+```typescript
+import { readFile, writeFile, rename } from 'node:fs/promises';
+import type { ValidateFunction } from 'ajv';
+import { stableHash, nowIso } from '../core/fs.js';
+import type { ExecutionRequestItem } from '../core/execution-control.js';
+import type {
+  OrganizerOrderingArtifact,
+  SupervisorKernelPort,
+  SupervisorRuntimeState,
+} from './types.js';
+import type { WorkerProvider, WorkerRunOutput } from '../providers/providers.js';
+
+export interface OrganizerSidecarDependencies {
+  provider: WorkerProvider;
+  organizerSessionId: () => string | null;
+  kernel: SupervisorKernelPort;
+  state: SupervisorRuntimeState;
+  artifactPath: string;
+  schemaValidator: ValidateFunction;
+  pollIntervalMs?: number;
+  maxSpecContentBytes?: number;
+  artifactTtlMs?: number;
+}
+
+interface SpecEntry {
+  feature_id: string;
+  spec_path: string;
+  status: string;
+  content: string;
+}
+
+const DEFAULT_POLL_INTERVAL_MS = 10_000;
+const DEFAULT_MAX_SPEC_CONTENT_BYTES = 102_400;
+const DEFAULT_ARTIFACT_TTL_MS = 120_000;
+
+export class OrganizerSidecarService {
+  private readonly provider: WorkerProvider;
+  private readonly organizerSessionId: () => string | null;
+  private readonly kernel: SupervisorKernelPort;
+  private readonly state: SupervisorRuntimeState;
+  private readonly artifactPath: string;
+  private readonly schemaValidator: ValidateFunction;
+  private readonly pollIntervalMs: number;
+  private readonly maxSpecContentBytes: number;
+  private readonly artifactTtlMs: number;
+
+  private lastFingerprint: string | null = null;
+  private timer: NodeJS.Timeout | null = null;
+  private reconcileInFlight: Promise<void> | null = null;
+  private started = false;
+
+  constructor(deps: OrganizerSidecarDependencies) {
+    this.provider = deps.provider;
+    this.organizerSessionId = deps.organizerSessionId;
+    this.kernel = deps.kernel;
+    this.state = deps.state;
+    this.artifactPath = deps.artifactPath;
+    this.schemaValidator = deps.schemaValidator;
+    this.pollIntervalMs = deps.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS;
+    this.maxSpecContentBytes = deps.maxSpecContentBytes ?? DEFAULT_MAX_SPEC_CONTENT_BYTES;
+    this.artifactTtlMs = deps.artifactTtlMs ?? DEFAULT_ARTIFACT_TTL_MS;
+  }
+
+  start(): void {
+    if (this.started) {
+      return;
+    }
+    this.started = true;
+    this.timer = setInterval(() => {
+      void this.reconcile();
+    }, this.pollIntervalMs);
+    if (typeof this.timer.unref === 'function') {
+      this.timer.unref();
+    }
+  }
+
+  async stop(): Promise<void> {
+    this.started = false;
+    if (this.timer) {
+      clearInterval(this.timer);
+      this.timer = null;
+    }
+    if (this.reconcileInFlight !== null) {
+      await this.reconcileInFlight;
+    }
+  }
+
+  private async reconcile(): Promise<void> {
+    if (this.reconcileInFlight !== null) {
+      await this.reconcileInFlight;
+      return;
+    }
+    this.reconcileInFlight = this.performReconcile()
+      .catch((error: unknown) => {
+        const message = error instanceof Error ? error.message : String(error);
+        console.warn(`[organizer-sidecar] reconcile failed: ${message}`);
+      })
+      .finally(() => {
+        this.reconcileInFlight = null;
+      });
+    await this.reconcileInFlight;
+  }
+
+  private async performReconcile(): Promise<void> {
+    const sessionId = this.organizerSessionId();
+    if (!sessionId || sessionId === 'unknown' || sessionId === 'unassigned') {
+      return;
+    }
+
+    const specs = await this.collectSpecs();
+    if (specs.length === 0) {
+      return;
+    }
+
+    const requestIds = await this.collectRequestIds();
+    const fingerprint = this.computeFingerprint(specs, requestIds);
+    if (fingerprint === this.lastFingerprint) {
+      return;
+    }
+
+    const { prompt, partial } = this.buildPrompt(specs);
+    const artifact = await this.dispatchAndParse(prompt, partial, fingerprint, requestIds);
+    if (!artifact) {
+      return;
+    }
+
+    await this.writeArtifactAtomically(artifact);
+    this.lastFingerprint = fingerprint;
+  }
+
+  private async collectSpecs(): Promise<SpecEntry[]> {
+    const specMap = new Map<string, { spec_path: string; status: string }>();
+
+    // Source 1: execution control (pending requests)
+    try {
+      const pending = await this.kernel.drainPendingExecutionRequests(
+        this.state.runId,
+        this.state.ownerInstanceId,
+      );
+      for (const request of pending.data.items) {
+        if (request.status !== 'pending') {
+          continue;
+        }
+        for (const feature of request.features) {
+          if (!specMap.has(feature.feature_id)) {
+            specMap.set(feature.feature_id, {
+              spec_path: feature.spec_path,
+              status: 'pending',
+            });
+          }
+        }
+      }
+    } catch {
+      console.warn('[organizer-sidecar] failed to read execution control');
+    }
+
+    // Source 2: active/queued features from runtime state
+    for (const featureId of this.state.sessionsByFeature.keys()) {
+      if (!specMap.has(featureId)) {
+        await this.addFeatureFromState(specMap, featureId);
+      }
+    }
+    for (const queued of this.state.queue) {
+      if (!specMap.has(queued.feature_id)) {
+        await this.addFeatureFromState(specMap, queued.feature_id);
+      }
+    }
+
+    // Read spec file contents
+    const entries: SpecEntry[] = [];
+    for (const [featureId, meta] of specMap) {
+      try {
+        const content = await readFile(meta.spec_path, 'utf8');
+        entries.push({
+          feature_id: featureId,
+          spec_path: meta.spec_path,
+          status: meta.status,
+          content,
+        });
+      } catch {
+        console.warn(`[organizer-sidecar] spec file missing for ${featureId}: ${meta.spec_path}`);
+      }
+    }
+
+    return entries;
+  }
+
+  private async addFeatureFromState(
+    specMap: Map<string, { spec_path: string; status: string }>,
+    featureId: string,
+  ): Promise<void> {
+    try {
+      const state = await this.kernel.readState(featureId);
+      const fm = state.frontMatter;
+      const specPath = typeof fm.spec_path === 'string' ? fm.spec_path : null;
+      const status = typeof fm.status === 'string' ? fm.status : 'unknown';
+      if (specPath) {
+        specMap.set(featureId, { spec_path: specPath, status });
+      }
+    } catch {
+      // Feature state unreadable — skip
+    }
+  }
+
+  private async collectRequestIds(): Promise<string[]> {
+    try {
+      const pending = await this.kernel.drainPendingExecutionRequests(
+        this.state.runId,
+        this.state.ownerInstanceId,
+      );
+      return pending.data.items
+        .filter((item) => item.status === 'pending')
+        .map((item) => item.request_id)
+        .sort();
+    } catch {
+      return [];
+    }
+  }
+
+  private computeFingerprint(specs: SpecEntry[], requestIds: string[]): string {
+    return stableHash(
+      JSON.stringify({
+        featureIds: specs.map((s) => s.feature_id).sort(),
+        specPaths: specs.map((s) => s.spec_path).sort(),
+        requestIds,
+      }),
+    );
+  }
+
+  private buildPrompt(specs: SpecEntry[]): { prompt: string; partial: boolean } {
+    let totalBytes = 0;
+    let partial = false;
+    const included: SpecEntry[] = [];
+
+    for (const spec of specs) {
+      const specBytes = Buffer.byteLength(spec.content, 'utf8');
+      if (totalBytes + specBytes > this.maxSpecContentBytes) {
+        partial = true;
+        console.warn(
+          `[organizer-sidecar] spec content limit reached at ${included.length}/${specs.length} features`,
+        );
+        break;
+      }
+      totalBytes += specBytes;
+      included.push(spec);
+    }
+
+    const sections = included.map(
+      (spec) =>
+        `--- FEATURE: ${spec.feature_id} ---\nspec_path: ${spec.spec_path}\nstatus: ${spec.status}\n\n${spec.content}`,
+    );
+
+    const prompt = [
+      'Analyze the following feature specs and produce an ordering artifact.',
+      '',
+      `Active instance: ${this.state.ownerInstanceId}`,
+      `Total features: ${included.length}`,
+      '',
+      ...sections,
+      '',
+      '---',
+      '',
+      'Produce a single JSON object with the ordering analysis. No markdown fences.',
+    ].join('\n');
+
+    return { prompt, partial };
+  }
+
+  private async dispatchAndParse(
+    prompt: string,
+    partial: boolean,
+    fingerprint: string,
+    requestIds: string[],
+  ): Promise<OrganizerOrderingArtifact | null> {
+    const output = await this.invokeOrganizer(prompt);
+    const firstAttempt = this.parseAndValidate(output, partial, fingerprint, requestIds);
+    if (firstAttempt) {
+      return firstAttempt;
+    }
+
+    // Retry once with correction
+    const errors = this.schemaValidator.errors;
+    const errorSummary = errors
+      ? errors.map((e) => `${e.instancePath} ${e.message}`).join('; ')
+      : 'invalid JSON structure';
+    const correctionPrompt = `Your previous response was invalid: ${errorSummary}. Please produce the ordering JSON again with the same specs.`;
+
+    const retryOutput = await this.invokeOrganizer(correctionPrompt);
+    const secondAttempt = this.parseAndValidate(retryOutput, partial, fingerprint, requestIds);
+    if (secondAttempt) {
+      return secondAttempt;
+    }
+
+    console.warn('[organizer-sidecar] artifact validation failed after retry, skipping write');
+    return null;
+  }
+
+  private async invokeOrganizer(instructions: string): Promise<WorkerRunOutput> {
+    return await this.provider.runWorker({
+      role: 'organizer',
+      feature_id: 'global',
+      instructions,
+    });
+  }
+
+  private parseAndValidate(
+    output: WorkerRunOutput,
+    partial: boolean,
+    fingerprint: string,
+    requestIds: string[],
+  ): OrganizerOrderingArtifact | null {
+    const text = this.extractText(output);
+    if (!text) {
+      return null;
+    }
+
+    let parsed: Record<string, unknown>;
+    try {
+      parsed = JSON.parse(text) as Record<string, unknown>;
+    } catch {
+      // Try extracting JSON from embedded text
+      const firstBrace = text.indexOf('{');
+      const lastBrace = text.lastIndexOf('}');
+      if (firstBrace === -1 || lastBrace <= firstBrace) {
+        return null;
+      }
+      try {
+        parsed = JSON.parse(text.slice(firstBrace, lastBrace + 1)) as Record<string, unknown>;
+      } catch {
+        return null;
+      }
+    }
+
+    // Unwrap { type: "ORDERING", ordering: { ... } } envelope
+    const ordering =
+      parsed.type === 'ORDERING' && parsed.ordering && typeof parsed.ordering === 'object'
+        ? (parsed.ordering as Record<string, unknown>)
+        : parsed;
+
+    const artifact: OrganizerOrderingArtifact = {
+      version: 1,
+      generated_at: nowIso(),
+      expires_at: new Date(Date.now() + this.artifactTtlMs).toISOString(),
+      completion_status: partial ? 'partial' : 'complete',
+      backlog_fingerprint: fingerprint,
+      analyzed_requests: requestIds,
+      dependency_graph: (ordering.dependency_graph as Record<string, unknown>) ?? {},
+      ready_set: (ordering.ready_set as string[]) ?? [],
+      deferred_set:
+        (ordering.deferred_set as Array<{
+          feature_id: string;
+          blocked_by: string[];
+          reason: string;
+        }>) ?? [],
+      blocked_set: (ordering.blocked_set as Array<{ feature_id: string; reason: string }>) ?? [],
+    } as OrganizerOrderingArtifact;
+
+    const valid = this.schemaValidator(artifact);
+    if (!valid) {
+      return null;
+    }
+
+    return artifact;
+  }
+
+  private extractText(output: WorkerRunOutput): string | null {
+    // Try common provider output text fields
+    if (typeof output.content === 'string' && output.content.length > 0) {
+      return output.content;
+    }
+    if (typeof output.text === 'string' && output.text.length > 0) {
+      return output.text;
+    }
+    if (typeof output.response === 'string' && output.response.length > 0) {
+      return output.response;
+    }
+    // Fallback: check outputs array (structured provider envelope)
+    if (Array.isArray(output.outputs)) {
+      for (const item of output.outputs) {
+        const record = item as Record<string, unknown>;
+        if (typeof record.content === 'string' && record.content.length > 0) {
+          return record.content;
+        }
+      }
+    }
+    // Last resort: stringify the entire output
+    try {
+      const str = JSON.stringify(output);
+      if (str.length > 2) {
+        return str;
+      }
+    } catch {
+      // empty
+    }
+    return null;
+  }
+
+  private async writeArtifactAtomically(artifact: OrganizerOrderingArtifact): Promise<void> {
+    const tmpPath = this.artifactPath + '.tmp';
+    await writeFile(tmpPath, JSON.stringify(artifact, null, 2), 'utf8');
+    await rename(tmpPath, this.artifactPath);
+  }
+}
+```
+
+- [ ] **Step 2: Verify the file compiles**
+
+Run: `npx tsc --noEmit`
+Expected: Clean (no errors in the new file).
+
+- [ ] **Step 3: Commit**
+
+```bash
+git add apps/control-plane/src/supervisor/organizer-sidecar-service.ts
+git commit -m "feat: implement OrganizerSidecarService with polling loop and artifact writer"
+```
+
+---
+
+### Task 3: Write Unit Tests for OrganizerSidecarService
+
+**Files:**
+
+- Create: `apps/control-plane/test/organizer-sidecar-service.spec.ts`
+
+- [ ] **Step 1: Create the test file with helpers and mocks**
+
+Create `apps/control-plane/test/organizer-sidecar-service.spec.ts`:
+
+```typescript
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import { mkdtemp, rm, readFile, writeFile, mkdir } from 'node:fs/promises';
+import { existsSync } from 'node:fs';
+import { tmpdir } from 'node:os';
+import { join } from 'node:path';
+import Ajv2020Import from 'ajv/dist/2020.js';
+import type { ValidateFunction } from 'ajv';
+import { OrganizerSidecarService } from '../src/supervisor/organizer-sidecar-service.js';
+import type { OrganizerOrderingArtifact, SupervisorRuntimeState } from '../src/supervisor/types.js';
+import type { WorkerProvider, WorkerRunOutput } from '../src/providers/providers.js';
+import artifactSchema from '../../../agentic/orchestrator/schemas/organizer-ordering-artifact.schema.json';
+
+function createAjvValidator(): ValidateFunction {
+  const Ajv2020Ctor = Ajv2020Import as unknown as new (opts: Record<string, unknown>) => {
+    compile(schema: Record<string, unknown>): ValidateFunction;
+    addFormat(name: string, format: { type?: string; validate: (v: string) => boolean }): void;
+  };
+  const ajv = new Ajv2020Ctor({ allErrors: true, strict: false });
+  ajv.addFormat('date-time', {
+    type: 'string',
+    validate: (v: string) => !Number.isNaN(Date.parse(v)),
+  });
+  return ajv.compile(artifactSchema as unknown as Record<string, unknown>);
+}
+
+function makeValidOrderingResponse(): WorkerRunOutput {
+  return {
+    content: JSON.stringify({
+      type: 'ORDERING',
+      ordering: {
+        dependency_graph: {
+          feat_a: { depends_on: [], classification: 'no_dependency_found' },
+          feat_b: { depends_on: ['feat_a'], classification: 'explicit_dependency' },
+        },
+        ready_set: ['feat_a'],
+        deferred_set: [{ feature_id: 'feat_b', blocked_by: ['feat_a'], reason: 'explicit dep' }],
+        blocked_set: [],
+      },
+    }),
+  };
+}
+
+function makeMalformedResponse(): WorkerRunOutput {
+  return { content: '{ not valid json' };
+}
+
+function makeInvalidClassificationResponse(): WorkerRunOutput {
+  return {
+    content: JSON.stringify({
+      type: 'ORDERING',
+      ordering: {
+        dependency_graph: {
+          feat_a: { depends_on: [], classification: 'INVALID_ENUM_VALUE' },
+        },
+        ready_set: ['feat_a'],
+        deferred_set: [],
+        blocked_set: [],
+      },
+    }),
+  };
+}
+
+let tempDir: string;
+let artifactPath: string;
+let specDir: string;
+let validator: ValidateFunction;
+
+function makeState(overrides: Partial<SupervisorRuntimeState> = {}): SupervisorRuntimeState {
+  return {
+    runId: 'run:test',
+    ownerInstanceId: 'supervisor:test',
+    orchestratorSessionId: 'orch-1',
+    reconcilerSessionId: null,
+    organizerSessionId: 'org-1',
+    sessionsByFeature: new Map(),
+    queue: [],
+    runMetadata: {},
+    ...overrides,
+  };
+}
+
+function makeKernel(specPaths: Record<string, string> = {}) {
+  return {
+    drainPendingExecutionRequests: vi.fn().mockResolvedValue({
+      data: {
+        items: [
+          {
+            request_id: 'req_1',
+            request_type: 'add_features',
+            status: 'pending',
+            requested_at: new Date().toISOString(),
+            processed_at: null,
+            requested_by: 'cli:add',
+            operation_id: 'op:1',
+            features: Object.entries(specPaths).map(([featureId, specPath]) => ({
+              feature_id: featureId,
+              spec_path: specPath,
+            })),
+            progress: null,
+            outcome: null,
+          },
+        ],
+      },
+    }),
+    readState: vi.fn().mockImplementation(async (featureId: string) => ({
+      frontMatter: {
+        status: 'active',
+        spec_path: specPaths[featureId] ?? `/tmp/missing/${featureId}.md`,
+      },
+      body: '',
+    })),
+    getRuntimeSessions: vi.fn().mockResolvedValue({}),
+    getRepoRoot: vi.fn().mockReturnValue(tempDir),
+    getAgentsConfig: vi.fn().mockReturnValue({ roles: { organizer: {} } }),
+    getPolicySnapshot: vi.fn().mockReturnValue({}),
+    getAdaptersConfig: vi.fn().mockReturnValue({}),
+    ensureLoaded: vi.fn().mockResolvedValue(undefined),
+    renewRunLease: vi.fn(),
+    releaseRunLease: vi.fn(),
+    markExecutionRequestProcessed: vi.fn(),
+    updateExecutionRequestProgress: vi.fn(),
+    repoDiff: vi.fn(),
+    worktreePath: vi.fn(),
+    featurePath: vi.fn(),
+    readState_: vi.fn(),
+    validatePatchDiff: vi.fn(),
+    updateState: vi.fn(),
+    checkBudget: vi.fn(),
+  };
+}
+
+function makeProvider(responses: WorkerRunOutput[]): WorkerProvider {
+  let callIndex = 0;
+  return {
+    mode: 'stub' as const,
+    selection: {
+      provider: 'test',
+      model: 'test-model',
+      provider_config_ref: 'test-ref',
+    },
+    createSession: vi
+      .fn()
+      .mockResolvedValue({
+        session_id: 'org-1',
+        role: 'organizer',
+        feature_id: 'global',
+        system_prompt_loaded: true,
+      }),
+    closeSession: vi.fn().mockResolvedValue({ closed: true }),
+    reattachSession: vi.fn().mockResolvedValue(true),
+    runWorker: vi.fn().mockImplementation(async () => {
+      const response = responses[callIndex] ?? responses[responses.length - 1];
+      callIndex++;
+      return response;
+    }),
+  } as unknown as WorkerProvider;
+}
+
+beforeEach(async () => {
+  tempDir = await mkdtemp(join(tmpdir(), 'organizer-sidecar-test-'));
+  artifactPath = join(tempDir, 'organizer-ordering.json');
+  specDir = join(tempDir, 'specs');
+  await mkdir(specDir, { recursive: true });
+  validator = createAjvValidator();
+});
+
+afterEach(async () => {
+  await rm(tempDir, { recursive: true, force: true });
+});
+```
+
+- [ ] **Step 2: Add test — backlog unchanged, no LLM call**
+
+Append to the test file:
+
+```typescript
+describe('OrganizerSidecarService', () => {
+  it('GIVEN_backlog_unchanged_WHEN_tick_fires_THEN_no_runWorker_call', async () => {
+    const specPath = join(specDir, 'feat_a.md');
+    await writeFile(specPath, '# Feature A spec', 'utf8');
+    const kernel = makeKernel({ feat_a: specPath });
+    const provider = makeProvider([makeValidOrderingResponse()]);
+    const state = makeState();
+
+    const service = new OrganizerSidecarService({
+      provider,
+      organizerSessionId: () => 'org-1',
+      kernel: kernel as any,
+      state,
+      artifactPath,
+      schemaValidator: validator,
+      pollIntervalMs: 100_000,
+    });
+
+    // First call — triggers LLM
+    await (service as any).performReconcile();
+    expect(provider.runWorker).toHaveBeenCalledTimes(1);
+
+    // Second call — fingerprint unchanged, no LLM call
+    await (service as any).performReconcile();
+    expect(provider.runWorker).toHaveBeenCalledTimes(1);
+  });
+```
+
+- [ ] **Step 3: Add test — new pending request triggers LLM**
+
+```typescript
+it('GIVEN_new_pending_request_WHEN_tick_fires_THEN_sends_spec_content_to_organizer', async () => {
+  const specPath = join(specDir, 'feat_a.md');
+  await writeFile(specPath, '# Feature A\nImplement widget system', 'utf8');
+  const kernel = makeKernel({ feat_a: specPath });
+  const provider = makeProvider([makeValidOrderingResponse()]);
+  const state = makeState();
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+  });
+
+  await (service as any).performReconcile();
+
+  expect(provider.runWorker).toHaveBeenCalledTimes(1);
+  const call = (provider.runWorker as any).mock.calls[0][0];
+  expect(call.role).toBe('organizer');
+  expect(call.feature_id).toBe('global');
+  expect(call.instructions).toContain('--- FEATURE: feat_a ---');
+  expect(call.instructions).toContain('# Feature A');
+  expect(call.instructions).toContain('Implement widget system');
+});
+```
+
+- [ ] **Step 4: Add test — valid response writes artifact atomically**
+
+```typescript
+it('GIVEN_valid_LLM_response_WHEN_parsed_THEN_writes_artifact_atomically', async () => {
+  const specPath = join(specDir, 'feat_a.md');
+  await writeFile(specPath, '# Feature A spec', 'utf8');
+  const kernel = makeKernel({ feat_a: specPath });
+  const provider = makeProvider([makeValidOrderingResponse()]);
+  const state = makeState();
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+  });
+
+  await (service as any).performReconcile();
+
+  expect(existsSync(artifactPath)).toBe(true);
+  const written = JSON.parse(await readFile(artifactPath, 'utf8')) as OrganizerOrderingArtifact;
+  expect(written.version).toBe(1);
+  expect(written.completion_status).toBe('complete');
+  expect(written.ready_set).toEqual(['feat_a']);
+  expect(written.deferred_set).toHaveLength(1);
+  expect(written.deferred_set[0].feature_id).toBe('feat_b');
+  expect(written.dependency_graph.feat_a.classification).toBe('no_dependency_found');
+  expect(written.dependency_graph.feat_b.classification).toBe('explicit_dependency');
+  // Verify temp file is cleaned up
+  expect(existsSync(artifactPath + '.tmp')).toBe(false);
+});
+```
+
+- [ ] **Step 5: Add test — malformed response triggers retry**
+
+```typescript
+it('GIVEN_malformed_LLM_response_WHEN_first_attempt_fails_THEN_retries_with_correction', async () => {
+  const specPath = join(specDir, 'feat_a.md');
+  await writeFile(specPath, '# Feature A spec', 'utf8');
+  const kernel = makeKernel({ feat_a: specPath });
+  // First call returns malformed, retry returns valid
+  const provider = makeProvider([makeMalformedResponse(), makeValidOrderingResponse()]);
+  const state = makeState();
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+  });
+
+  await (service as any).performReconcile();
+
+  expect(provider.runWorker).toHaveBeenCalledTimes(2);
+  const retryCall = (provider.runWorker as any).mock.calls[1][0];
+  expect(retryCall.instructions).toContain('Your previous response was invalid');
+  expect(existsSync(artifactPath)).toBe(true);
+});
+```
+
+- [ ] **Step 6: Add test — both attempts fail, no artifact written**
+
+```typescript
+it('GIVEN_malformed_response_on_retry_WHEN_second_attempt_fails_THEN_logs_warning_and_skips', async () => {
+  const specPath = join(specDir, 'feat_a.md');
+  await writeFile(specPath, '# Feature A spec', 'utf8');
+  const kernel = makeKernel({ feat_a: specPath });
+  const provider = makeProvider([makeMalformedResponse(), makeMalformedResponse()]);
+  const state = makeState();
+  const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+  });
+
+  await (service as any).performReconcile();
+
+  expect(provider.runWorker).toHaveBeenCalledTimes(2);
+  expect(existsSync(artifactPath)).toBe(false);
+  expect(warnSpy).toHaveBeenCalledWith(
+    expect.stringContaining('artifact validation failed after retry'),
+  );
+  warnSpy.mockRestore();
+});
+```
+
+- [ ] **Step 7: Add test — no organizer session skips gracefully**
+
+```typescript
+it('GIVEN_no_organizer_session_WHEN_tick_fires_THEN_skips_gracefully', async () => {
+  const kernel = makeKernel({});
+  const provider = makeProvider([makeValidOrderingResponse()]);
+  const state = makeState();
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => null,
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+  });
+
+  await (service as any).performReconcile();
+
+  expect(provider.runWorker).not.toHaveBeenCalled();
+  expect(existsSync(artifactPath)).toBe(false);
+});
+```
+
+- [ ] **Step 8: Add test — missing spec file skips feature with warning**
+
+```typescript
+it('GIVEN_spec_file_missing_WHEN_collecting_specs_THEN_skips_that_feature', async () => {
+  const kernel = makeKernel({ feat_a: '/nonexistent/path/spec.md' });
+  const provider = makeProvider([makeValidOrderingResponse()]);
+  const state = makeState();
+  const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+  });
+
+  await (service as any).performReconcile();
+
+  // No specs collected, so no LLM call
+  expect(provider.runWorker).not.toHaveBeenCalled();
+  expect(warnSpy).toHaveBeenCalledWith(expect.stringContaining('spec file missing for feat_a'));
+  warnSpy.mockRestore();
+});
+```
+
+- [ ] **Step 9: Add test — size guard produces partial artifact**
+
+```typescript
+it('GIVEN_spec_content_exceeds_limit_WHEN_building_prompt_THEN_produces_partial_artifact', async () => {
+  const specPathA = join(specDir, 'feat_a.md');
+  const specPathB = join(specDir, 'feat_b.md');
+  // Write large spec content
+  await writeFile(specPathA, 'A'.repeat(500), 'utf8');
+  await writeFile(specPathB, 'B'.repeat(500), 'utf8');
+  const kernel = makeKernel({ feat_a: specPathA, feat_b: specPathB });
+  const provider = makeProvider([makeValidOrderingResponse()]);
+  const state = makeState();
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+    maxSpecContentBytes: 600, // Only room for ~1 spec
+  });
+
+  await (service as any).performReconcile();
+
+  expect(existsSync(artifactPath)).toBe(true);
+  const written = JSON.parse(await readFile(artifactPath, 'utf8')) as OrganizerOrderingArtifact;
+  expect(written.completion_status).toBe('partial');
+});
+```
+
+- [ ] **Step 10: Add test — stop awaits in-flight reconcile**
+
+```typescript
+it('GIVEN_service_started_WHEN_stop_called_THEN_awaits_inflight_reconcile', async () => {
+  const specPath = join(specDir, 'feat_a.md');
+  await writeFile(specPath, '# Feature A spec', 'utf8');
+  const kernel = makeKernel({ feat_a: specPath });
+  // Slow provider response
+  const provider = makeProvider([]);
+  (provider.runWorker as any).mockImplementation(async () => {
+    await new Promise((resolve) => setTimeout(resolve, 100));
+    return makeValidOrderingResponse();
+  });
+  const state = makeState();
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+    pollIntervalMs: 50,
+  });
+
+  service.start();
+  // Wait for at least one tick
+  await new Promise((resolve) => setTimeout(resolve, 80));
+  await service.stop();
+
+  // After stop, if reconcile was in-flight it should have completed
+  // (no assertion on artifact because timing is non-deterministic,
+  // but stop() should resolve without hanging)
+});
+```
+
+- [ ] **Step 11: Add test — schema validation passes for valid artifact**
+
+```typescript
+it('GIVEN_valid_artifact_WHEN_schema_validation_runs_THEN_passes', async () => {
+  const specPath = join(specDir, 'feat_a.md');
+  await writeFile(specPath, '# Feature A spec', 'utf8');
+  const kernel = makeKernel({ feat_a: specPath });
+  const provider = makeProvider([makeValidOrderingResponse()]);
+  const state = makeState();
+
+  const service = new OrganizerSidecarService({
+    provider,
+    organizerSessionId: () => 'org-1',
+    kernel: kernel as any,
+    state,
+    artifactPath,
+    schemaValidator: validator,
+  });
+
+  await (service as any).performReconcile();
+
+  expect(existsSync(artifactPath)).toBe(true);
+  const written = JSON.parse(await readFile(artifactPath, 'utf8'));
+  expect(validator(written)).toBe(true);
+});
+```
+
+- [ ] **Step 12: Add test — invalid classification enum is rejected by schema**
+
+```typescript
+  it('GIVEN_invalid_classification_enum_WHEN_schema_validation_runs_THEN_rejects', async () => {
+    const specPath = join(specDir, 'feat_a.md');
+    await writeFile(specPath, '# Feature A spec', 'utf8');
+    const kernel = makeKernel({ feat_a: specPath });
+    // Both attempts return invalid classification
+    const provider = makeProvider([
+      makeInvalidClassificationResponse(),
+      makeInvalidClassificationResponse(),
+    ]);
+    const state = makeState();
+
+    const service = new OrganizerSidecarService({
+      provider,
+      organizerSessionId: () => 'org-1',
+      kernel: kernel as any,
+      state,
+      artifactPath,
+      schemaValidator: validator,
+    });
+
+    const warnSpy = vi.spyOn(console, 'warn').mockImplementation(() => {});
+    await (service as any).performReconcile();
+
+    expect(existsSync(artifactPath)).toBe(false);
+    warnSpy.mockRestore();
+  });
+});
+```
+
+- [ ] **Step 13: Run the tests**
+
+Run: `npx vitest run apps/control-plane/test/organizer-sidecar-service.spec.ts`
+Expected: All 11 tests pass.
+
+- [ ] **Step 14: Commit**
+
+```bash
+git add apps/control-plane/test/organizer-sidecar-service.spec.ts
+git commit -m "test: add unit tests for OrganizerSidecarService"
+```
+
+---
+
+### Task 4: Wire Sidecar into SupervisorRuntime
+
+**Files:**
+
+- Modify: `apps/control-plane/src/supervisor/runtime.ts:937-966` (`resolveRoleSessionId`)
+- Modify: `apps/control-plane/src/supervisor/runtime.ts:243-267` (class fields)
+- Modify: `apps/control-plane/src/supervisor/runtime.ts:515-523` (scheduler wiring)
+- Modify: `apps/control-plane/src/supervisor/runtime.ts:633-640` (`start`/`stop`)
+
+- [ ] **Step 1: Add import for OrganizerSidecarService**
+
+In `apps/control-plane/src/supervisor/runtime.ts`, add to the imports (near line 25 where `OrganizerAgentEnrollmentScheduler` is imported):
+
+```typescript
+import { OrganizerSidecarService } from './organizer-sidecar-service.js';
+```
+
+Also add the AJV import for schema compilation:
+
+```typescript
+import Ajv2020Import from 'ajv/dist/2020.js';
+import type { ValidateFunction } from 'ajv';
+```
+
+And add a filesystem import for reading the schema:
+
+```typescript
+import { readFileSync } from 'node:fs';
+import { fileURLToPath } from 'node:url';
+```
+
+- [ ] **Step 2: Add organizer sidecar field to SupervisorRuntime class**
+
+After the `executionEnrollmentService` field declaration (~line 263), add:
+
+```typescript
+  private readonly organizerSidecar: OrganizerSidecarService | null;
+```
+
+- [ ] **Step 3: Instantiate sidecar in the constructor**
+
+After the `executionEnrollmentService` instantiation (~line 530), add the sidecar creation:
+
+```typescript
+if (organizerEnabled) {
+  const sidecarSchemaPath = pathLayout.organizerOrderingSchemaPath();
+  const sidecarValidator = this.compileOrganizerSchema(sidecarSchemaPath);
+  this.organizerSidecar = new OrganizerSidecarService({
+    provider: this.provider,
+    organizerSessionId: () => this.state.organizerSessionId ?? null,
+    kernel: this.kernel,
+    state: this.state,
+    artifactPath: pathLayout.organizerOrderingPath(this.state.ownerInstanceId),
+    schemaValidator: sidecarValidator,
+  });
+} else {
+  this.organizerSidecar = null;
+}
+```
+
+Add a private helper method to compile the schema:
+
+```typescript
+  private compileOrganizerSchema(schemaPath: string): ValidateFunction {
+    const Ajv2020Ctor = Ajv2020Import as unknown as new (
+      opts: Record<string, unknown>,
+    ) => { compile(schema: Record<string, unknown>): ValidateFunction; addFormat(name: string, format: { type?: string; validate: (v: string) => boolean }): void };
+    const ajv = new Ajv2020Ctor({ allErrors: true, strict: false });
+    ajv.addFormat('date-time', {
+      type: 'string',
+      validate: (v: string) => !Number.isNaN(Date.parse(v)),
+    });
+    const schema = JSON.parse(readFileSync(schemaPath, 'utf8')) as Record<string, unknown>;
+    return ajv.compile(schema);
+  }
+```
+
+- [ ] **Step 4: Add `organizerOrderingSchemaPath()` to PathLayout**
+
+In `apps/control-plane/src/core/path-layout.ts`, after `organizerOrderingPath()`:
+
+```typescript
+  organizerOrderingSchemaPath(): string {
+    return path.join(this.legacyOrchestratorRoot, 'schemas', 'organizer-ordering-artifact.schema.json');
+  }
+```
+
+- [ ] **Step 5: Add organizer to resolveRoleSessionId**
+
+In `apps/control-plane/src/supervisor/runtime.ts`, in the `resolveRoleSessionId` method (~line 942), add after the reconciler case:
+
+```typescript
+if (role === 'organizer') {
+  return this.state.organizerSessionId ?? null;
+}
+```
+
+- [ ] **Step 6: Wire start/stop into run lifecycle**
+
+Modify the `start()` method (~line 633):
+
+```typescript
+  async start(features: FeatureInput[]): Promise<Record<string, unknown>> {
+    await this.executionEnrollmentService.start();
+    this.organizerSidecar?.start();
+    try {
+      return await this.runCoordinator.start(features);
+    } finally {
+      await this.organizerSidecar?.stop();
+      await this.executionEnrollmentService.stop();
+    }
+  }
+```
+
+- [ ] **Step 7: Verify compilation**
+
+Run: `npx tsc --noEmit`
+Expected: Clean.
+
+- [ ] **Step 8: Run full test suite**
+
+Run: `npx vitest run`
+Expected: All tests pass (including new organizer-sidecar-service tests and existing supervisor tests).
+
+- [ ] **Step 9: Commit**
+
+```bash
+git add apps/control-plane/src/supervisor/runtime.ts apps/control-plane/src/supervisor/organizer-sidecar-service.ts apps/control-plane/src/core/path-layout.ts
+git commit -m "feat: wire OrganizerSidecarService into supervisor runtime lifecycle"
+```
+
+---
+
+### Task 5: Update Existing Tests for Runtime Changes
+
+**Files:**
+
+- Modify: `apps/control-plane/test/supervisor.unit.spec.ts`
+
+- [ ] **Step 1: Check if supervisor tests need updates**
+
+Run: `npx vitest run apps/control-plane/test/supervisor.unit.spec.ts`
+
+If tests fail due to the new `organizerSidecar` field or `resolveRoleSessionId` change, proceed to fix them. If all pass, skip to the commit step.
+
+- [ ] **Step 2: Fix any failing supervisor tests**
+
+Common fixes needed:
+
+- If `resolveRoleSessionId` tests exist, add an `organizer` role test case
+- If runtime constructor tests exist, the mock may need `organizerOrderingSchemaPath` on `PathLayout`
+
+- [ ] **Step 3: Run the full test suite**
+
+Run: `npx vitest run`
+Expected: All tests pass.
+
+- [ ] **Step 4: Run lint**
+
+Run: `npm run lint`
+Expected: Clean.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add apps/control-plane/test/supervisor.unit.spec.ts
+git commit -m "test: update supervisor tests for organizer sidecar wiring"
+```
+
+---
+
+### Task 6: Final Verification & Progress Update
+
+**Files:**
+
+- Modify: `spec-files/progress.md`
+
+- [ ] **Step 1: Run full test suite**
+
+Run: `npx vitest run`
+Expected: All tests pass, >=90% coverage.
+
+- [ ] **Step 2: Run typecheck**
+
+Run: `npx tsc --noEmit`
+Expected: Clean.
+
+- [ ] **Step 3: Run lint**
+
+Run: `npm run lint`
+Expected: Clean.
+
+- [ ] **Step 4: Verify organizer is opt-in**
+
+Confirm that when `agents.yaml` does NOT have an organizer role, `organizerSidecar` is `null` and no `OrganizerSidecarService` is created. Check this by reading the conditional in `runtime.ts`.
+
+- [ ] **Step 5: Update progress.md**
+
+Append entries to `spec-files/progress.md`:
+
+```markdown
+✅ **Entry N — Organizer System Prompt Enhancement**
+
+- **Goal:** Add dependency detection signals section to teach the LLM how to identify dependencies.
+- **Changes made:**
+  - `config/agentic/orchestrator/prompts/organizer.system.md` — added "Dependency detection signals" section with explicit, structural, semantic, and not-dependency subsections.
+- **Result:** Prompt file well-formed. `npm run lint` ✅.
+
+✅ **Entry N+1 — OrganizerSidecarService Implementation**
+
+- **Goal:** Implement Layer 1 runtime loop that polls for backlog changes, dispatches specs to organizer LLM, validates response, and writes ordering artifact atomically.
+- **Changes made:**
+  - `apps/control-plane/src/supervisor/organizer-sidecar-service.ts` — new service with polling loop, spec collection from execution control + runtime state, prompt construction with size guard, response parsing with JSON extraction and AJV validation, retry on failure, atomic file write.
+  - `apps/control-plane/test/organizer-sidecar-service.spec.ts` — 11 tests covering: fingerprint skip, LLM dispatch, artifact write, retry on malformed, persistent failure, no session, missing spec, size guard, stop lifecycle, schema validation pass, invalid enum rejection.
+- **Result:** `npm test` ✅. `npm run lint` ✅.
+
+✅ **Entry N+2 — Wire Sidecar into Supervisor Runtime**
+
+- **Goal:** Integrate OrganizerSidecarService into the supervisor lifecycle so it starts/stops with the run.
+- **Changes made:**
+  - `apps/control-plane/src/supervisor/runtime.ts` — added sidecar field, conditional instantiation when organizer role configured, `start()`/`stop()` lifecycle wiring, `organizer` case in `resolveRoleSessionId()`, AJV schema compilation.
+  - `apps/control-plane/src/core/path-layout.ts` — added `organizerOrderingSchemaPath()` method.
+- **Result:** `npm test` ✅. `npm run lint` ✅. `npm run typecheck` ✅.
+```
+
+- [ ] **Step 6: Commit progress update**
+
+```bash
+git add spec-files/progress.md
+git commit -m "docs: update progress.md with organizer sidecar entries"
+```
+
+---
+
+## Task Dependency Graph
+
+```
+Task 1 (system prompt) ─────────────────────────────────┐
+Task 2 (service implementation) ──┬── Task 4 (wiring) ──┤
+Task 3 (unit tests) ─────────────┘        │             │
+                                   Task 5 (test fixes) ─┤
+                                                         │
+                                   Task 6 (verification) ┘
+```
+
+- Tasks 1, 2, 3 are independent and can run in parallel.
+- Task 4 depends on Task 2 (needs the service class to exist).
+- Task 5 depends on Task 4 (needs the wiring to exist to know what breaks).
+- Task 6 depends on all prior tasks.