mia-code 0.2.0 → 0.3.0

package/contextual_research/PDE-generalization--caefee82-efb1-4dbb-8733-691b01581464--260130/DESIGN.gemini.md

@@ -0,0 +1,242 @@
+# Design Proposal: Integrating a Prompt Decomposition Engine (PDE) into `mia-code`
+
+**Document Purpose:** This document outlines a detailed design for integrating a Prompt Decomposition Engine (PDE) into the `mia-code` CLI tool. The objective is to evolve `mia-code` from a direct prompt-response wrapper into an intelligent, plan-driven agentic orchestrator, aligning with the principles of structured thinking and ceremonial development.
+
+## 1. Analysis of Current `mia-code` Architecture
+
+Based on a review of the `/a/src/mia-code` repository, the current architecture is as follows:
+
+*   **Technology:** A Node.js application written in TypeScript, using `commander.js` for CLI command parsing.
+*   **Core Functionality:** It serves as a user-friendly terminal interface that wraps the Gemini and Claude CLI tools in their headless modes.
+*   **Execution Flow:**
+    1.  The user runs `mia-code chat` or `mia-code prompt`.
+    2.  `src/cli.ts` and `src/index.ts` parse the command and manage the interactive session.
+    3.  User input is passed to the `runGeminiHeadless` function in `src/geminiHeadless.ts`.
+    4.  This function `spawn`s the appropriate backend CLI (e.g., `gemini`) as a child process, passing the user's raw prompt and other flags.
+    5.  It captures and parses the `stream-json` or `json` output from the backend, optionally running it through a `unifier` for ceremonial interpretation.
+*   **Limitation:** The current design sends the user's prompt directly to the backend LLM. It lacks a mechanism to decompose complex, multi-step user intents, leading to potential failures, incomplete responses, or a lack of transparency for the user.
+
+## 2. Proposed Architectural Vision: PDE Integration
+
+We propose embedding the PDE directly into the `mia-code` CLI's command execution lifecycle. This transforms `mia-code` into a "thinking" client that plans its work before executing it.
+
+### High-Level Workflow:
+
+```mermaid
+sequenceDiagram
+    participant User
+    participant MiaCodeCLI as `mia-code` CLI
+    participant PDE as Prompt Decomposition Engine (API)
+    participant Gemini as Gemini/Claude Backend
+
+    User->>+MiaCodeCLI: mia-code chat --pde "Complex multi-step request"
+    MiaCodeCLI->>+PDE: Decompose(request)
+    PDE-->>-MiaCodeCLI: Return Structured Plan (ExecutionPlan)
+    MiaCodeCLI->>User: Display Ceremonial Plan for Approval<br/>(EAST, SOUTH, WEST, NORTH stages)
+    User->>+MiaCodeCLI: "y" (approves plan)
+    
+    loop For Each Step in Plan
+        MiaCodeCLI->>+Gemini: Execute Step N (prompt from plan)
+        Gemini-->>-MiaCodeCLI: Return Step N Result
+        MiaCodeCLI->>User: Display Step N Result/Status
+    end
+    
+    MiaCodeCLI-->>-User: Final Summary
+```
+
+## 3. Detailed Technical Design
+
+This section details the specific code-level changes and additions required to implement the PDE integration.
+
+### 3.1. New `src/pde/` Module
+
+A new directory will be created at `/a/src/mia-code/src/pde/` to encapsulate all PDE-related logic.
+
+*   **`pde-types.ts`:** This file will define the TypeScript interfaces for the data structures exchanged with the PDE, primarily the `ExecutionPlan`.
+
+    ```typescript
+    // /a/src/mia-code/src/pde/pde-types.ts
+
+    export type MedicineWheelDirection = "EAST" | "SOUTH" | "WEST" | "NORTH";
+
+    export interface PdeTask {
+      id: string;
+      description: string;
+      agentCommand: string; // e.g., "gemini" or "claude"
+      prompt: string;       // The specific prompt for this step
+      expectedOutputs: string[];
+      dependencies: string[];
+    }
+
+    export interface PdeStage {
+      stageId: string;
+      direction: MedicineWheelDirection;
+      directionDescription: string; // e.g., "Vision & Inquiry"
+      tasks: PdeTask[];
+    }
+
+    export interface ExecutionPlan {
+      workflowId: string;
+      overallIntention: string;
+      stages: PdeStage[];
+    }
+    ```
+
+*   **`pde-client.ts`:** This client will handle communication with the PDE API. Initially, it can be a mock that returns a hardcoded plan for testing.
+
+    ```typescript
+    // /a/src/mia-code/src/pde/pde-client.ts
+    import { ExecutionPlan } from "./pde-types";
+
+    export async function decomposePrompt(prompt: string): Promise<ExecutionPlan> {
+      // In the future, this will make an HTTP call to the PDE API
+      // For now, return a mock plan for development
+      console.log(`[PDE Client] Decomposing: "${prompt}"`);
+      // ... Mock implementation
+      // Example:
+      const mockPlan: ExecutionPlan = { /* ... structured plan ... */ };
+      return mockPlan;
+    }
+    ```
+
+### 3.2. CLI Command Enhancements (`src/cli.ts`)
+
+1.  **Add `--pde` Flag:** The `chat` command will be updated to include a `--pde` flag to activate the planning workflow.
+
+    ```typescript
+    // in src/cli.ts, inside the 'chat' command action
+    await runInteractiveCli({
+      // ... existing options
+      usePde: opts.pde // new option
+    });
+    ```
+
+2.  **Add `decompose` Command:** A new command will be added for users who only want to see the plan without executing it.
+
+    ```typescript
+    // in src/index.ts, add a new command
+    program
+      .command("decompose <text>")
+      .description("Decompose a complex prompt into an executable plan using PDE")
+      .action(async (text) => {
+        const plan = await decomposePrompt(text);
+        displayPlan(plan); // A new rendering function
+      });
+    ```
+
+### 3.3. Core Logic Modification (`src/index.ts` & a new `pde-executor.ts`)
+
+The primary change will be in `runInteractiveCli`. It needs a new conditional path for PDE-enabled prompts.
+
+1.  **Modify `runInteractiveCli`:**
+
+    ```typescript
+    // in src/cli.ts
+    // ... inside processInput function
+    if (config.usePde) { // Assume config is updated with the flag
+        console.log(chalk.magenta("🧬 Decomposing prompt via PDE..."));
+        const plan = await decomposePrompt(trimmed);
+        
+        displayPlan(plan); // Render the ceremonial plan
+        
+        const { enquirer } = await import('enquirer');
+        const { confirm } = await enquirer.prompt({
+            type: 'confirm',
+            name: 'confirm',
+            message: 'Proceed with execution?'
+        });
+
+        if (confirm) {
+            await executePdePlan(plan, config, rl);
+        } else {
+            console.log(chalk.yellow("Execution cancelled."));
+        }
+    } else {
+        // ... existing runGeminiHeadless logic
+    }
+    ```
+
+2.  **Create `pde-executor.ts`:** To keep the logic clean, a new file will handle the step-by-step execution.
+
+    ```typescript
+    // /a/src/mia-code/src/pde-executor.ts
+    import { ExecutionPlan, PdeStage } from "./pde/pde-types";
+    import { runGeminiHeadless } from "./geminiHeadless";
+    // ... other imports
+
+    export async function executePdePlan(plan: ExecutionPlan, config: MiaCodeConfig, rl: Interface) {
+        console.log("\n" + chalk.bold.green("🚀 Starting Execution..."));
+        const context: Record<string, any> = {}; // To store results of steps
+
+        for (const stage of plan.stages) {
+            displayStageHeader(stage);
+            for (const task of stage.tasks) {
+                // ... logic to check if dependencies are met from `context`
+                
+                console.log(chalk.dim(`  - Executing task: ${task.description}`));
+                const result = await runGeminiHeadless({
+                    prompt: task.prompt,
+                    config,
+                    // Session management needs to be adapted for multi-step execution
+                });
+
+                // Store result in context for dependent tasks
+                context[task.id] = result; 
+                
+                // ... render task result
+            }
+        }
+        console.log("\n" + chalk.bold.green("✅ Plan execution complete."));
+    }
+    ```
+
+### 3.4. Ceremonial UI Integration (`src/formatting.ts`)
+
+New functions will be added to render the plan in a way that reflects its ceremonial structure.
+
+```typescript
+// /a/src/mia-code/src/formatting.ts
+
+const directionColors = {
+    EAST: chalk.hex('#FFD700'),  // Gold for Vision
+    SOUTH: chalk.hex('#32CD32'), // Green for Growth
+    WEST: chalk.hex('#4682B4'),  // SteelBlue for Integration
+    NORTH: chalk.hex('#FFFFFF')   // White for Wisdom
+};
+
+export function displayPlan(plan: ExecutionPlan) {
+    console.log(`\n${chalk.bold.magenta('🧬 Decomposed Plan:')} ${plan.overallIntention}`);
+    for (const stage of plan.stages) {
+        const color = directionColors[stage.direction] || chalk.white;
+        console.log(color(`\n${stage.direction}: ${stage.directionDescription}`));
+        for (const task of stage.tasks) {
+            console.log(chalk.dim(`  - [ ] ${task.description}`));
+            if (task.dependencies.length > 0) {
+                console.log(chalk.dim(`        (depends on: ${task.dependencies.join(', ')})`));
+            }
+        }
+    }
+    console.log("");
+}
+```
+
+## 4. Phased Implementation Roadmap
+
+1.  **Phase 1: Foundation & Mocking**
+    *   Implement the `pde/` directory with types and a mocked `pde-client`.
+    *   Implement the UI rendering functions in `formatting.ts`.
+    *   Add the `decompose` command to `cli.ts` to test the plan rendering.
+
+2.  **Phase 2: Core Execution Logic**
+    *   Add the `--pde` flag and the conditional logic in `cli.ts`.
+    *   Implement the `pde-executor.ts` file with the core loop for step-by-step execution.
+    *   Adapt session handling in `geminiHeadless.ts` to work across multiple calls within a single plan.
+
+3.  **Phase 3: Live Integration & Refinement**
+    *   Replace the mocked `pde-client` with a real HTTP client to call the live PDE API.
+    *   Implement robust error handling, dependency management, and state tracking in the `pde-executor`.
+    *   Conduct user testing to refine the CLI's interactive experience for plan approval and execution feedback.
+
+## 5. Conclusion
+
+Integrating the Prompt Decomposition Engine into `mia-code` represents a significant architectural evolution. It elevates the tool from a simple command-line wrapper to a sophisticated agentic assistant that brings transparency, control, and resilience to complex development tasks. By grounding this technical design in the project's ceremonial philosophy, we create a tool that is not only more powerful but also more intuitive and aligned with its intended purpose.

package/contextual_research/PDE-generalization--caefee82-efb1-4dbb-8733-691b01581464--260130/INDEX.md

@@ -0,0 +1,45 @@
+# PDE Generalization Literature Index
+
+This document serves as an organized index and guide to academic literature relevant to the Prompt Decomposition Engine (PDE) project. Its purpose is to assist other LLMs in quickly understanding the available research, its relevance to PDE's architecture and implementation, and its contribution to the overall literature survey.
+
+The PDE aims to decompose complex, multi-intent prompts into actionable workflows, addressing limitations in current agentic systems. This literature base supports the development of PDE's five progressive layers: Intent Extraction & Classification, Dependency Graph Construction, Medicine Wheel Direction Assignment, Workflow Template Generation, and Execution Plan with Checkpoints, as well as its academic grounding.
+
+## Processed Literature References:
+
+Here is an overview of the academic papers that have been processed and converted into Markdown, along with their key relevance to the PDE project:
+
+1.  **[Decomposed Prompting: A Modular Approach For Solving Complex Tasks](sources/footnote_1_12_decomposed_prompting.md)**
+    *   **Original Source:** `https://arxiv.org/pdf/2210.02406.pdf`
+    *   **Relevance to PDE:** Directly relevant to the core concept of PDE, especially Layer 1 (Intent Extraction) and Layer 4 (Workflow Template Generation). This paper explores decomposing complex tasks into simpler sub-tasks via prompting, utilizing a shared library of LLMs. It highlights advantages like modularity, debuggability, and extensibility, which are foundational principles for PDE's layered architecture and prompt chaining approach. It also discusses hierarchical and recursive decomposition, which aligns with PDE's dependency graph construction.
+
+2.  **[Navigating Complexity: Orchestrated Problem Solving with Multi-Agent LLMs](sources/footnote_1_2_navigating_complexity.md)**
+    *   **Original Source:** `http://arxiv.org/pdf/2402.16713.pdf`
+    *   **Relevance to PDE:** Highly relevant to PDE's overall architecture and Layer 4 (Workflow Template Generation). This paper proposes leveraging decomposition and an orchestrating LLM to tackle complex and vague problems in multi-agent systems. It details how an orchestrating LLM can decompose problems, ask follow-up questions, and assign sub-problems to specialized agents, directly mirroring PDE's goal of transforming raw user input into executable agent workflows within a multi-agent context.
+
+3.  **[LLM Multi-Agent Systems: Challenges and Open Problems](sources/footnote_1_1_coordination_challenges.md)**
+    *   **Original Source:** `https://arxiv.org/pdf/2402.03578.pdf`
+    *   **Relevance to PDE:** Provides critical context for PDE's operational environment, specifically informing Layer 2 (Dependency Graph Construction), Layer 5 (Execution Plan with Checkpoints), and the "Multi-Agent Systems Challenges" section of the literature survey. It discusses challenges in multi-agent systems such as task allocation, robust reasoning, managing complex context, and memory management. Understanding these challenges is essential for designing a resilient and effective PDE.
+
+4.  **[MARCO: Multi-Agent Real-time Chat Orchestration](sources/footnote_1_28_guardrails_multi_agent.md)**
+    *   **Original Source:** `https://arxiv.org/pdf/2410.21784.pdf`
+    *   **Relevance to PDE:** Very relevant to Layer 5 (Execution Plan with Checkpoints) and the "Multi-Agent Systems Challenges" section, particularly concerning guardrails and error recovery. This paper introduces a framework for automating tasks in multi-agent LLM systems, emphasizing robust guardrails to steer LLM behavior, validate outputs, and recover from errors. These concepts are directly applicable to building reliable execution plans and checkpointing mechanisms within PDE.
+
+5.  **[TOWARDS HIERARCHICAL MULTI-AGENT WORKFLOWS FOR ZERO-SHOT PROMPT OPTIMIZATION](sources/footnote_1_34_hierarchical_multi_agent.md)**
+    *   **Original Source:** `https://arxiv.org/pdf/2405.20252.pdf`
+    *   **Relevance to PDE:** Directly relevant to Prompt Engineering for Decomposition and Workflow Orchestration, impacting Layer 1 (Intent Extraction) and Layer 4 (Workflow Template Generation). This paper explores giving LLMs the freedom to design optimal prompts hierarchically (CEO → Manager → Worker workflow). This zero-shot, task-agnostic, and query-specific approach to prompt optimization offers insights into how PDE can dynamically generate and refine prompts for its various decomposition layers.
+
+6.  **[Agent-Oriented Planning in Multi-Agent Systems](sources/footnote_1_19_hugginggpt_planning.md)**
+    *   **Original Source:** `https://arxiv.org/pdf/2410.02189v2.pdf`
+    *   **Relevance to PDE:** Highly relevant to Layer 2 (Dependency Graph Construction) and Layer 4 (Workflow Template Generation), as well as general Multi-Agent Systems Challenges. This paper identifies design principles for agent-oriented planning (solvability, completeness, non-redundancy) and proposes AOP, a framework for fast task decomposition and allocation with reward model-based evaluation and feedback loops. This directly informs how PDE can structure its sub-task assignments and ensure effective multi-agent collaboration.
+
+7.  **[Open Intent Extraction from Natural Language Interactions (Extended Abstract)](sources/footnote_1_5_open_intent_extraction.md)**
+    *   **Original Source:** `https://www.ijcai.org/proceedings/2021/0663.pdf`
+    *   **Relevance to PDE:** Directly relevant to Layer 1 (Intent Extraction & Classification) and the "Intent Recognition & NLU" section of the literature survey. This paper introduces OPINE, a domain-agnostic approach for open intent discovery, framing it as a sequence tagging task. Understanding how to extract explicit and implicit user intents from diverse natural language inputs is fundamental to PDE's initial layer of decomposing prompts.
+
+8.  **[Agents Under Siege: Breaking Pragmatic Multi-Agent LLM Systems with Optimized Prompt Attacks](sources/2504.00218v2.md)**
+    *   **Original Source:** `https://arxiv.org/pdf/2504.00218v2.pdf`
+    *   **Relevance to PDE:** Provides critical context for the "Communication Vulnerabilities" within "Multi-Agent Systems Challenges" (Section 3.5) of PDE's literature survey. This paper discusses adversarial attacks and prompt propagation in multi-agent LLM systems, highlighting vulnerabilities in inter-agent communication. While not directly about constructive prompt decomposition, understanding these attack vectors can inform the design of robust and secure PDE implementations, particularly regarding prompt sanitization and communication protocols.
+
+9.  **[Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](sources/2505.00212v3.md)**
+    *   **Original Source:** `https://arxiv.org/pdf/2505.00212v3.pdf`
+    *   **Relevance to PDE:** Highly relevant and foundational for Layer 5 (Execution Plan with Checkpoints) and the "Failure Attribution" section (Section 3.5) of PDE's literature survey. This paper directly addresses automated failure attribution in multi-agent LLM systems, including identifying agents and steps responsible for task failures. This knowledge is crucial for developing PDE's capabilities for progress tracking, precise failure diagnosis, and effective recovery mechanisms.