coaiajs 0.3.0 → 0.3.2

package/articles/reviews/observability-ai-systems-review.md

@@ -1,176 +0,0 @@
-# Literature Review: Observability in AI Systems
-
-> Literature review for the CoAiA.js project — OpenTelemetry for LLMs, Langfuse, LangSmith, and Weights & Biases examined through the trace/span/generation model for AI application monitoring.
-
-## Abstract
-
-Observability for AI systems—particularly LLM-powered applications—has evolved rapidly from ad-hoc logging to structured tracing frameworks that capture the full lifecycle of model interactions. This literature review examines the current state of AI observability through four lenses: OpenTelemetry as the emerging standard for vendor-neutral instrumentation, Langfuse as the leading open-source LLM observability platform, LangSmith as the LangChain-native monitoring solution, and Weights & Biases (W&B Weave) as the experiment-tracking platform extending into production monitoring. We analyze the convergence on a trace/span/generation data model, evaluate each platform's approach to evaluation and scoring, and identify the gap that structural tension methodology fills: transforming observability from a problem-detection tool into a creative advancement instrument.
-
-## Introduction
-
-Traditional software observability rests on three pillars: logs (event records), metrics (aggregated measurements), and traces (request-scoped execution paths). These pillars, standardized through OpenTelemetry, have proven sufficient for deterministic systems where identical inputs produce identical outputs and where "correct behavior" is well-defined [1][2].
-
-LLM-powered applications violate both assumptions. Identical prompts can produce different outputs across invocations (non-determinism), and "correct behavior" is often subjective—dependent on context, user intent, and quality standards that resist formal specification. This fundamental shift requires observability approaches that capture not just operational health but **semantic quality**: what was generated, whether it was helpful, and how it compares to desired standards [3].
-
-The 2024–2025 period has seen the rapid emergence of LLM-specific observability platforms that extend the traditional three pillars with a fourth: **evaluation**—structured assessment of output quality through automated scoring, human annotation, and LLM-as-judge techniques.
-
-## Review
-
-### OpenTelemetry: The Foundation Layer
-
-**Overview:** OpenTelemetry (OTel) is a CNCF (Cloud Native Computing Foundation) project that provides vendor-neutral APIs, SDKs, and tools for generating and collecting telemetry data. It has become the de facto standard for observability instrumentation in cloud-native applications [1][4].
-
-**LLM Extensions:** The OpenTelemetry community has developed semantic conventions for LLM operations, defining standard attribute names for:
-- Model name and version
-- Token counts (input, output, total)
-- Prompt and completion content (with optional redaction)
-- Temperature and other generation parameters
-- Cost attribution
-
-These conventions enable LLM telemetry to flow through existing OTel infrastructure (collectors, exporters, backends) without requiring LLM-specific tooling for data transport [4][5].
-
-**Strengths:**
-- Vendor-neutral: data can be routed to any compatible backend
-- Mature ecosystem: collectors, exporters, and instrumentation libraries for all major languages
-- Standards-based: semantic conventions ensure consistent data representation
-- Composable: LLM traces can be correlated with infrastructure traces for full-stack visibility
-
-**Limitations:**
-- Generic by design: LLM-specific concepts (prompt management, evaluation workflows, cost optimization) are not addressed
-- No built-in evaluation: OTel collects data but does not analyze quality
-- Configuration complexity: setting up collectors, exporters, and backends requires significant infrastructure knowledge [6]
-
-**Literature Position:** OTel is best understood as a foundation layer—it provides the instrumentation and transport upon which LLM-specific platforms build. Several LLM observability platforms (Langfuse, LangSmith) now support OTel data ingestion, enabling teams to use standard OTel instrumentation while routing LLM-specific data to specialized analysis platforms.
-
-### Langfuse: Open-Source LLM Observability
-
-**Overview:** Langfuse is an open-source (MIT-licensed) LLM observability platform providing trace-based monitoring, prompt management, evaluation workflows, and cost analytics. It can be self-hosted as a single Docker container or accessed through managed cloud hosting [7][8].
-
-**Data Model:** Langfuse implements a hierarchical trace model:
-- **Traces** represent end-to-end executions (user request → final response)
-- **Spans** represent logical subdivisions (retrieval, prompt construction, generation)
-- **Generations** represent specific LLM completions with full prompt/completion capture
-- **Scores** represent quality assessments attached at any level of the hierarchy
-
-**Evaluation Capabilities:**
-- LLM-as-Judge: automated quality scoring using a judge model
-- Annotation Queues: human review workflows with structured scoring
-- Dataset Management: curated evaluation sets for regression testing
-- Prompt Experimentation: A/B testing with tracked performance metrics [9]
-
-**Literature Assessment:** Langfuse occupies a unique position in the observability landscape: open-source, self-hostable, and increasingly feature-complete. Its MIT license addresses the data sovereignty concerns that prevent many organizations from using cloud-only platforms. As of 2025, all major features (including formerly "Pro" capabilities like LLM-as-judge) are fully open source [10].
-
-**Key Publication:** Langfuse's blog post on OpenTelemetry integration (October 2024) is a significant technical document, demonstrating how LLM-specific observability can be built on standardized instrumentation rather than proprietary data collection [4].
-
-### LangSmith: LangChain-Native Monitoring
-
-**Overview:** LangSmith is the observability platform developed by LangChain Inc., designed for deep integration with LangChain and LangGraph applications. It provides tracing, evaluation, dataset management, and prompt monitoring with tight coupling to the LangChain execution model [11].
-
-**Key Features:**
-- Automatic tracing of LangChain chains, agents, and tools
-- End-to-end OpenTelemetry support (announced 2025)
-- Evaluation workflows with human annotation and automated scoring
-- Prompt hub for versioning and sharing prompt templates
-- Regression detection through dataset-based testing
-
-**Strengths:**
-- Deepest integration with LangChain/LangGraph ecosystem
-- Most comprehensive evaluation workflow (A/B testing, regression gating, win-rate analysis)
-- Strong enterprise features (access control, compliance, audit trails)
-- OTel bridge enables integration with broader observability infrastructure [11]
-
-**Limitations:**
-- Primarily a cloud service (limited self-hosting options)
-- Tightly coupled to LangChain ecosystem
-- Proprietary—dependency on LangChain Inc.'s business model
-
-**Literature Assessment:** LangSmith is the most feature-rich LLM observability platform, particularly for teams already invested in the LangChain ecosystem. Its adoption of OpenTelemetry standards signals a convergence in the industry toward standardized instrumentation. However, its cloud-first model and ecosystem coupling create vendor lock-in concerns.
-
-### Weights & Biases (W&B Weave): Experiment Tracking to Production
-
-**Overview:** Weights & Biases, originally an experiment tracking platform for ML research, has extended into LLM application monitoring through its Weave product. Weave provides tracing, evaluation, and comparison capabilities specifically for LLM applications [12][13].
-
-**Key Features:**
-- Side-by-side experiment comparison (prompts, models, parameters)
-- Automatic token and cost tracking
-- Trace-based debugging with full input/output capture
-- Integration with research workflows (MLflow compatibility)
-- Real-time dashboards with custom metrics
-
-**Strengths:**
-- Research-to-production continuity: teams using W&B for ML research can extend to LLM monitoring
-- Deep experiment comparison capabilities
-- Strong visualization tools
-- Broad model and framework support
-
-**Limitations:**
-- Primarily cloud-hosted
-- Research-oriented UX may not suit operations teams
-- Less LLM-specific than Langfuse or LangSmith
-- Pricing can be significant for high-volume production use [13]
-
-**Literature Assessment:** W&B Weave is best suited for teams that bridge research and production—organizations that experiment with different models, prompts, and architectures and need to carry insights from experimentation into production monitoring.
-
-### Emerging Platforms
-
-Beyond the four primary platforms, the literature notes several emerging tools:
-
-- **Arize Phoenix:** Focus on drift detection and production monitoring with strong visualization
-- **Helicone:** No-code observability with particular strength in cost analytics and caching optimization
-- **Maxim AI:** Agent-specific tracing with support for diverse agent architectures
-- **VictoriaMetrics + OTel:** Infrastructure-first approach integrating AI agent observability into existing monitoring stacks [14][15]
-
-### Convergence Patterns
-
-The literature reveals several convergent trends:
-
-1. **Trace/span/generation as standard model.** All platforms have converged on this hierarchical data model, whether they call it "traces" (Langfuse, LangSmith) or "runs" (W&B).
-
-2. **OpenTelemetry as foundation.** Langfuse, LangSmith, and emerging platforms increasingly support OTel data ingestion, suggesting convergence on standardized instrumentation.
-
-3. **Evaluation as core capability.** All platforms have moved beyond passive observation to active evaluation—automated scoring, human annotation, and regression testing.
-
-4. **Cost attribution as first-class concern.** LLM applications have direct, per-request costs that traditional applications lack. All platforms now track and attribute costs.
-
-5. **Self-hosting demand.** Data sovereignty requirements are driving open-source and self-hostable solutions, with Langfuse leading this trend.
-
-### The Structural Tension Observability Gap
-
-The literature reveals a gap in current observability thinking: all platforms treat observability as **problem-detection**—finding errors, identifying bottlenecks, detecting drift. None frame observability as **creative advancement**—measuring progress toward a desired quality standard while maintaining honest current reality assessment.
-
-This gap is precisely where structural tension methodology contributes. By mapping:
-- Traces → STC progressions (how is the agent advancing toward its creative goal?)
-- Scores → MMOT evaluations (honest assessment of output quality against Elements of Performance)
-- Reality updates → observation data (what does the observability data tell us about current reality?)
-
-...observability transforms from defensive monitoring into a creative instrument that drives purposeful advancement.
-
-## Implications for CoAiA.js
-
-1. **Langfuse as primary observability backend.** Open-source, self-hostable, and MIT-licensed—aligned with CoAiA.js's zero-dependency philosophy.
-2. **OpenTelemetry instrumentation.** Standard OTel spans for all agent operations, enabling routing to any compatible backend.
-3. **MMOT-as-score integration.** Self-evaluation scores flow to Langfuse as structured quality signals.
-4. **Cost-aware agents.** Token and cost data from Langfuse informs agent decision-making (model selection, context management).
-5. **Creative-orientation observability.** Dashboards frame metrics as tension indicators (how far is current reality from desired outcome?) rather than problem indicators (what's broken?).
-
-## Conclusion
-
-AI observability has matured rapidly from ad-hoc logging to structured, platform-supported monitoring with evaluation capabilities. The convergence on the trace/span/generation model, OpenTelemetry standardization, and active evaluation workflows provides a solid foundation. The remaining gap—transforming observability from problem-detection to creative advancement—is addressed by integrating structural tension methodology with the observability data model, a contribution that CoAiA.js implements through MMOT-as-score and creative-orientation dashboards.
-
-## References
-
-1. OpenTelemetry. "What is OpenTelemetry?" https://opentelemetry.io/docs/what-is-opentelemetry/
-2. Glukhov.org. (2024). "Observability for LLM Systems." https://www.glukhov.org/observability/observability-for-llm-systems/
-3. Elysiate. (2025). "LLM Observability: Monitoring, Tracing, and Cost Control." https://www.elysiate.com/blog/llm-observability-monitoring-langsmith-helicone-2025
-4. Langfuse. (2024). "OpenTelemetry for LLM Observability." https://langfuse.com/blog/2024-10-opentelemetry-for-llm-observability
-5. Grafana. (2024). "LLM Observability with OpenTelemetry and Grafana Cloud." https://grafana.com/blog/a-complete-guide-to-llm-observability-with-opentelemetry-and-grafana-cloud/
-6. Spanora AI. (2025). "OpenTelemetry LLM Monitoring." https://spanora.ai/blog/opentelemetry-llm-monitoring
-7. Langfuse GitHub. https://github.com/langfuse/langfuse
-8. Mayol, M. (2025). "Langfuse: The Definitive LLM Observability Platform." https://marcmayol.com/blog/en/langfuse_the_definitive_observability_platform_for_llms_in_2025/
-9. Pondhouse Data. "Langfuse: The Open Source Observability Platform." https://www.pondhouse-data.com/blog/langfuse-observability-platform
-10. It's FOSS. (2025). "LLM Analytics Platform Langfuse Goes Open Source." https://itsfoss.com/news/langfuse-open-source/
-11. LangChain Blog. (2025). "End-to-End OpenTelemetry Support in LangSmith." https://blog.langchain.com/end-to-end-opentelemetry-langsmith/
-12. AI Multiple Research. (2024). "LLM Observability Tools: Weights & Biases, LangSmith." https://research.aimultiple.com/llm-observability/
-13. GetMaxim. (2025). "Top AI Observability Tools in 2025." https://www.getmaxim.ai/articles/top-ai-observability-tools-in-2025-the-ultimate-guide/
-14. VictoriaMetrics. (2025). "AI Agents Observability with OpenTelemetry." https://victoriametrics.com/blog/ai-agents-observability/
-15. DEV Community. (2025). "LLM Observability with OpenTelemetry: A Practical Guide." https://dev.to/kartikdudeja21/llm-observability-with-opentelemetry-a-practical-guide-3clo

package/articles/reviews/prompt-engineering-decomposition-review.md

@@ -1,184 +0,0 @@
-# Literature Review: Prompt Engineering and Decomposition Methodologies
-
-> Literature review for the CoAiA.js project — From Chain-of-Thought to Tree-of-Thought, Self-Consistency, and PDE: a review of how prompt decomposition strategies address complexity in LLM applications.
-
-## Abstract
-
-Prompt engineering has evolved from simple instruction formatting to a sophisticated discipline encompassing reasoning strategies, decomposition techniques, and quality-assurance methodologies. This literature review examines the major prompt decomposition approaches—Chain-of-Thought (CoT), Tree-of-Thought (ToT), Self-Consistency, Decomposed Prompting (DecomP), Plan-and-Solve, and Prompt Decomposition Engineering (PDE)—analyzing their theoretical foundations, empirical effectiveness, and complementary relationships. We identify a critical distinction in the literature between strategies that improve *reasoning quality* (CoT, ToT, Self-Consistency) and strategies that improve *input completeness* (DecomP, PDE), arguing that both categories are necessary for robust AI agent systems. The review situates PDE's unique contribution—extracting implicit intents from hedging language and organizing actions into Four Directions stacks—within this broader landscape.
-
-## Introduction
-
-The quality of LLM output is profoundly shaped by input construction. A well-structured prompt can transform a mediocre model into an effective problem-solver, while a poorly structured prompt can cause a capable model to miss critical requirements, hallucinate details, or produce superficially correct but substantively incomplete results [1][2].
-
-The prompt engineering community has responded with an expanding toolkit of strategies, each addressing different aspects of the prompting challenge. This review organizes these strategies into a taxonomy based on what they optimize, examines their empirical support, and identifies the gaps that remain.
-
-## Review
-
-### Category 1: Reasoning Enhancement Strategies
-
-These strategies improve the quality of the model's reasoning process on a well-specified input.
-
-#### Chain-of-Thought Prompting (CoT)
-
-**Foundational Work:** Wei et al. (2022) demonstrated that prompting LLMs to show intermediate reasoning steps ("Let's think step by step") dramatically improves performance on multi-step problems, achieving state-of-the-art results on arithmetic, commonsense, and symbolic reasoning benchmarks [3].
-
-**Variants:**
-- **Zero-Shot CoT:** Simply adding "Let's think step by step" to any prompt (Kojima et al., 2022)
-- **Few-Shot CoT:** Providing examples with explicit reasoning chains
-- **Auto-CoT:** Automatically generating diverse reasoning demonstrations (Zhang et al., 2022)
-
-**Mechanism:** CoT works by encouraging the model to allocate computation to intermediate steps rather than jumping directly to an answer. The intermediate steps serve as a "scratchpad" that keeps the model's reasoning on track [4].
-
-**Empirical Support:** Strong and consistent. CoT improves performance on virtually all multi-step reasoning tasks, with gains proportional to task complexity. The technique is most effective for models above ~100B parameters.
-
-**Limitations:** CoT improves reasoning fidelity but does not address input completeness. If the prompt contains multiple intents, CoT may reason carefully about a subset while ignoring others. The model's selection of which intents to address remains uncontrolled [5].
-
-#### Tree-of-Thought (ToT)
-
-**Foundational Work:** Yao et al. (2023) extended CoT from linear reasoning to branching exploration, modeling the reasoning process as a tree where each node represents a partial solution and branches represent alternative reasoning paths [6].
-
-**Mechanism:** ToT prompts the model to:
-1. Generate multiple possible next steps at each reasoning node
-2. Evaluate each possibility using a value function (often the model itself)
-3. Prune unpromising branches
-4. Continue exploration along promising paths
-
-**Empirical Support:** ToT significantly outperforms CoT on problems requiring exploration—creative tasks, strategic planning, and scenarios with multiple valid approaches. The Game of 24 benchmark showed ToT solving 74% of problems vs. CoT's 4% [6].
-
-**Limitations:** ToT is computationally expensive (multiple model calls per reasoning step), introduces additional latency, and is most beneficial for problems where exploration is genuinely valuable. For straightforward sequential tasks, CoT is sufficient and more efficient.
-
-**Literature Context:** ToT represents a shift from prompt engineering as input formatting to prompt engineering as search algorithm design. The model becomes a component in a larger reasoning architecture rather than a standalone solution.
-
-#### Self-Consistency
-
-**Foundational Work:** Wang et al. (2022) proposed sampling multiple independent reasoning paths and selecting the most consistent answer—a technique analogous to ensemble methods in traditional ML [7].
-
-**Mechanism:** The model generates N different reasoning chains for the same prompt, each potentially following a different path. The final answer is determined by majority vote across the N chains.
-
-**Empirical Support:** Self-Consistency consistently improves upon single-chain CoT, particularly on problems where the reasoning path is ambiguous or where the model's initial reasoning may be led astray by surface-level patterns.
-
-**Limitations:** Computational cost scales linearly with N (number of sampled chains). The technique addresses reasoning reliability but not input completeness—all N chains reason about the same (potentially incomplete) interpretation of the input.
-
-### Category 2: Task Structure Strategies
-
-These strategies address the organization and decomposition of the task itself.
-
-#### Decomposed Prompting (DecomP)
-
-**Foundational Work:** Khot et al. (2022) proposed decomposing complex questions into simpler sub-questions, each handled by specialized sub-prompts. The key innovation is that different sub-questions can be routed to different models or tools based on their requirements [8].
-
-**Mechanism:**
-1. A decomposer module breaks the complex question into sub-questions
-2. Each sub-question is routed to an appropriate handler (LLM, calculator, retriever)
-3. Sub-answers are composed into a final answer
-
-**Empirical Support:** DecomP improves performance on multi-hop reasoning, compositional generalization, and long-context tasks by reducing the complexity that any single model call must handle.
-
-**Limitations:** DecomP assumes the input question is well-specified—it decomposes the *answer strategy*, not the *question itself*. If the original question contains implicit or hedged intents, DecomP will miss them.
-
-#### Plan-and-Solve (PS)
-
-**Foundational Work:** Wang et al. (2023) proposed a two-stage approach: first devise a plan for solving the problem, then execute the plan step by step [9].
-
-**Mechanism:**
-1. The model generates a plan: "To solve this, I need to: (1) extract data, (2) calculate totals, (3) compare results"
-2. The model executes each plan step sequentially
-
-**Empirical Support:** PS improves performance on math word problems and multi-step reasoning tasks by making the execution strategy explicit before execution begins.
-
-**Limitations:** The plan is only as good as the model's understanding of the input. If the input contains multiple intents, the plan may address only the salient ones.
-
-#### Skeleton-of-Thought (SoT)
-
-**Foundational Work:** Ning et al. (2023) proposed generating a "skeleton" answer structure first, then filling in each section in parallel, dramatically reducing latency for long-form generation.
-
-**Mechanism:**
-1. Generate a skeleton: section headers or key points
-2. Fill each section independently (can be parallelized)
-3. Compose the final answer
-
-**Empirical Support:** SoT reduces latency by 2-3x with comparable or improved quality for long-form outputs.
-
-**Relevance to Decomposition:** SoT is relevant because it demonstrates that explicit structural decomposition before generation improves both speed and quality—a principle that PDE extends from output structure to input analysis.
-
-### Category 3: Input Completeness Strategies
-
-This category, which PDE anchors, addresses the fidelity of intent extraction from complex inputs.
-
-#### Prompt Decomposition Engineering (PDE)
-
-**Foundational Context:** PDE emerges from the observation that Categories 1 and 2 both assume the model correctly identifies all intents in the input—an assumption that fails for complex, multi-intent, hedging-rich prompts.
-
-**Mechanism:**
-1. **Intent Extraction:** The complex prompt is analyzed for primary intents (explicit, unhedged), secondary intents (hedged, conditional), and implicit intents (logically entailed but unstated)
-2. **Dependency Mapping:** Extracted intents are analyzed for prerequisite relationships
-3. **Four Directions Organization:** Intents are organized into East (vision), South (analysis), West (implementation), North (evaluation) stacks
-
-**Key Innovation:** PDE's treatment of hedging language as intent signal rather than uncertainty noise. When a user says "maybe also add Husky," PDE captures this as a secondary intent with medium confidence—ensuring it is not dropped during execution.
-
-**Theoretical Foundation:** PDE draws from speech act theory (Austin, 1962; Searle, 1969) in treating utterances as actions with varying illocutionary force. "Maybe also configure X" is an indirect speech act—its surface form (suggestion) differs from its underlying intent (request). PDE's hedging detection operationalizes this insight [10].
-
-**Empirical Support:** While PDE is newer and less empirically studied than CoT or ToT, its value proposition is validated by the well-documented phenomenon of "prompt drift"—the tendency for LLMs to lose track of secondary intents in complex prompts, a finding consistently reproduced in agent evaluation benchmarks.
-
-### Comparative Analysis
-
-| Strategy | Optimizes | Input Requirements | Computational Cost | Maturity |
-|----------|-----------|-------------------|-------------------|----------|
-| CoT | Reasoning accuracy | Well-specified question | Low (1 extra pass) | High |
-| ToT | Solution exploration | Well-specified problem | High (N branches × M steps) | Medium |
-| Self-Consistency | Reasoning reliability | Well-specified question | Medium (N samples) | High |
-| DecomP | Complexity management | Well-specified question | Medium (sub-prompts) | Medium |
-| Plan-and-Solve | Execution strategy | Well-specified problem | Low (plan + execute) | Medium |
-| SoT | Generation speed | Clear output structure | Low (skeleton + fill) | Medium |
-| **PDE** | **Input completeness** | **Any complexity** | **Low (one analysis pass)** | **Early** |
-
-### Complementary Relationships
-
-The strategies are not competing but complementary, addressing different stages of the prompt-to-output pipeline:
-
-```
-User Input → [PDE: ensure completeness] → [PS: plan strategy] → 
-[CoT/ToT: reason carefully] → [Self-Consistency: verify] → Output
-```
-
-PDE operates at the earliest stage—ensuring that all intents are captured before any reasoning strategy is applied. Its output feeds into Plan-and-Solve (which creates execution plans for each extracted intent), Chain-of-Thought (which reasons carefully about each step), and Self-Consistency (which verifies the reasoning).
-
-### Reflexion and Self-Improvement
-
-Shinn et al. (2023) introduced Reflexion—prompting agents to reflect on their own outputs and identify errors for correction. This is relevant to the decomposition landscape because it addresses a failure mode that all prior strategies share: what happens when the model executes correctly but misinterpreted the input?
-
-Reflexion operates as a post-hoc correction mechanism. PDE operates as a pre-hoc prevention mechanism. Both are valuable; their combination provides the strongest completeness guarantee [11].
-
-### Meta-Prompting and Prompt Chaining
-
-The literature also notes the importance of prompt chaining—linking multiple prompt responses into explicit multi-stage workflows. This pattern is the operational foundation for PDE: the decomposition analysis is one prompt, and each extracted intent is executed through subsequent prompts, potentially with different reasoning strategies applied to each.
-
-## Implications for CoAiA.js
-
-1. **PDE as session entry point.** Every CoAiA.js session begins with PDE decomposition, ensuring completeness before reasoning begins.
-2. **Strategy composition.** After PDE extracts intents, each intent can use the appropriate reasoning strategy (CoT for sequential tasks, ToT for creative tasks).
-3. **Four Directions alignment.** PDE's Four Directions output stack maps directly to CoAiA.js's directional agent orchestration.
-4. **MMOT as Reflexion.** The MMOT evaluation cycle serves as the post-hoc completeness verification that Reflexion provides, checking whether all PDE-extracted intents were addressed.
-5. **Hedging-aware agents.** CoAiA.js agents treat hedged requests as real requests with reduced confidence, preventing the intent loss that plagues naive prompt processing.
-
-## Conclusion
-
-The prompt decomposition literature reveals a rich but incomplete landscape. Reasoning enhancement strategies (CoT, ToT, Self-Consistency) and task structure strategies (DecomP, PS, SoT) have achieved significant advances in the quality and organization of LLM outputs. However, they share a common blind spot: the assumption that the input prompt has been correctly and completely understood. PDE addresses this blind spot by operating on the input itself—extracting all intents (including implicit and hedged ones), mapping dependencies, and organizing them for systematic execution. The combination of input completeness (PDE) with reasoning quality (CoT/ToT) and post-hoc verification (Reflexion/MMOT) provides the most robust foundation for complex AI agent operations.
-
-## References
-
-1. LearnPrompting.org. (2024). "Advanced Decomposition Techniques for Improved Prompting." https://learnprompting.org/docs/advanced/decomposition/introduction
-2. Coupler.io. (2024). "Advanced Prompting Techniques for Complex AI Reasoning." https://blog.coupler.io/advanced-prompting-techniques/
-3. Wei, J., et al. (2022). "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." *NeurIPS 2022*.
-4. Toward Data Science. (2024). "Advanced Prompt Engineering: Chain of Thought." https://towardsdatascience.com/advanced-prompt-engineering-chain-of-thought-cot-8d8b090bf699/
-5. CalmOps. (2024). "Prompt Engineering Patterns: CoT, ReAct, and ToT." https://calmops.com/ai/prompt-engineering-patterns-cot-react-tot/
-6. Yao, S., et al. (2023). "Tree of Thoughts: Deliberate Problem Solving with Large Language Models." *NeurIPS 2023*.
-7. Wang, X., et al. (2022). "Self-Consistency Improves Chain of Thought Reasoning in Language Models." *ICLR 2023*.
-8. Khot, T., et al. (2022). "Decomposed Prompting: A Modular Approach for Solving Complex Tasks." *ICLR 2023*.
-9. Wang, L., et al. (2023). "Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning." *ACL 2023*.
-10. Searle, J. R. (1969). *Speech Acts: An Essay in the Philosophy of Language*. Cambridge University Press.
-11. Shinn, N., et al. (2023). "Reflexion: Language Agents with Verbal Reinforcement Learning." *NeurIPS 2023*.
-12. Oxen.ai. (2024). "The Prompt Report Part 2." https://ghost.oxen.ai/the-prompt-report-part-2-thought-generation-tree-of-thought-and-decomposition-prompting/
-13. Exploratio Journal. (2024). "Zooming-in On Prompting: A Comparative Study." https://exploratiojournal.com/zooming-in-on-prompting/
-14. Metric Coders. (2024). "Complex Reasoning with Chain-of-Thought, Tree-of-Thought, and More." https://www.metriccoders.com/post/beyond-the-basics-unleashing-complex-reasoning-with-chain-of-thought-tree-of-thought-and-more
-15. Fedotov, I. (2024). "Advanced Prompt Engineering Techniques: From Tree of Thoughts to Multimodal AI." https://ilyafedotov.com/advanced-prompt-engineering-techniques-from-tree-of-thoughts-to-multimodal-ai/

package/articles/surveys/agent-orchestration-survey.md

@@ -1,186 +0,0 @@
-# Survey of Multi-Agent Orchestration Patterns: AutoGen, CrewAI, LangGraph, and MCP
-
-> Academic survey for the CoAiA.js project — How AI agents coordinate, delegate, and share context across leading orchestration frameworks and the emergent role of MCP as a universal interoperability layer.
-
-## Abstract
-
-The multi-agent AI ecosystem has fragmented into distinct orchestration paradigms: role-based teams (CrewAI), graph-based state machines (LangGraph), conversation-driven negotiation (AutoGen/AG2), and protocol-standardized tool invocation (MCP). This survey examines the architectural patterns, coordination mechanisms, and operational trade-offs of these four approaches to multi-agent orchestration. We analyze each framework's model for agent specialization, inter-agent communication, context sharing, and error recovery. Our findings reveal a convergence toward hybrid architectures where MCP provides the universal tool interface, while framework-specific patterns handle agent coordination logic. The survey situates CoAiA.js's Four Directions orchestration model within this landscape, demonstrating how structural tension methodology provides an alternative to both role-based and graph-based coordination that naturally prevents the oscillatory patterns common in complex multi-agent workflows.
-
-## Introduction
-
-The transition from single-agent to multi-agent AI systems introduces coordination challenges that mirror those in distributed computing and organizational management: how do agents divide labor, share context, handle failures, and maintain coherent progress toward shared goals? [1]
-
-The 2024–2025 period has seen rapid proliferation of orchestration frameworks, each embodying different assumptions about how agents should interact. This survey examines the four dominant paradigms, identifies their structural properties, and evaluates their implications for the design of CoAiA.js's multi-agent capabilities.
-
-## Survey of Orchestration Patterns
-
-### 1. Role-Based Team Orchestration: CrewAI
-
-**Architecture:** Agents are defined as team members with explicit roles (Researcher, Writer, Reviewer), goals, tools, and behavioral constraints. A "crew" is assembled from these role-defined agents and given a task sequence to execute [2][3].
-
-**Coordination Model:**
-```python
-researcher = Agent(role="Senior Researcher", goal="Find comprehensive information", tools=[search, scrape])
-writer = Agent(role="Technical Writer", goal="Create clear documentation", tools=[write, format])
-crew = Crew(agents=[researcher, writer], tasks=[research_task, writing_task], process=Process.sequential)
-```
-
-**Communication Pattern:** Primarily sequential task handoff—the output of one agent becomes the input of the next. Limited support for parallel execution and inter-agent negotiation.
-
-**Strengths:**
-- Intuitive mental model (mirrors human team organization)
-- Fastest prototyping and lowest learning curve
-- Effective for linear business workflows
-- Clean separation of concerns through role specialization
-
-**Weaknesses:**
-- Limited control flow: branching, looping, and conditional logic are constrained
-- Susceptible to "agent loops" where agents repeatedly invoke each other without progress
-- Low determinism: identical inputs can produce different execution paths
-- Debugging is challenging: agent decisions are opaque [4][5]
-
-**Structural Tension Analysis:** CrewAI operates without explicit structural tension. Goals are stated but current reality is not formally tracked. This makes CrewAI susceptible to the oscillation pattern: agents work toward goals, partially achieve them, lose momentum, and re-engage when prompted.
-
-### 2. Graph-Based State Machine Orchestration: LangGraph
-
-**Architecture:** Workflows are modeled as directed graphs where nodes represent processing steps (which may involve agents) and edges represent transitions between steps. State is explicitly managed and passed between nodes [6][7].
-
-**Coordination Model:**
-```python
-graph = StateGraph(AgentState)
-graph.add_node("research", research_node)
-graph.add_node("write", write_node)
-graph.add_node("review", review_node)
-graph.add_edge("research", "write")
-graph.add_conditional_edges("review", should_revise, {"revise": "write", "approve": END})
-```
-
-**Communication Pattern:** State-passing through a shared state object. Each node reads from and writes to the state, enabling complex branching, looping, and conditional logic.
-
-**Strengths:**
-- Highest control and determinism among frameworks
-- Explicit branching, looping, error recovery, and conditional logic
-- Production-grade reliability (~94% accuracy in benchmarks)
-- Full visibility into execution state at every point
-- Natural fit for complex, multi-step workflows with failure recovery [8]
-
-**Weaknesses:**
-- Steepest learning curve: requires understanding of state machines and graph theory
-- Most verbose code for simple workflows
-- Tightly coupled to LangChain ecosystem
-- Over-engineering risk for straightforward sequential tasks
-
-**Structural Tension Analysis:** LangGraph's state machine model can encode structural tension: the state object can carry desired outcome and current reality, and conditional edges can check tension resolution. However, this requires explicit implementation—it is not a native feature.
-
-### 3. Conversation-Driven Orchestration: AutoGen (AG2)
-
-**Architecture:** Agents communicate through structured conversations—rounds of messages where each agent contributes based on its perspective. Coordination emerges from the conversation rather than being imposed by a workflow graph [9][10].
-
-**Coordination Model:**
-```python
-assistant = AssistantAgent("assistant", llm_config=llm_config)
-critic = AssistantAgent("critic", system_message="Critique code quality")
-user_proxy = UserProxyAgent("user", code_execution_config={"work_dir": "coding"})
-groupchat = GroupChat(agents=[user_proxy, assistant, critic], messages=[], max_round=12)
-```
-
-**Communication Pattern:** Round-robin or dynamic turn-taking in group chats. Agents see the full conversation history and contribute based on their system messages and observed context.
-
-**Strengths:**
-- Most natural model for collaborative reasoning and negotiation
-- Flexible: agent composition can change dynamically
-- Strong for human-in-the-loop workflows
-- Effective for research, brainstorming, and iterative refinement
-- AG2 (successor) adds enterprise reliability features [11]
-
-**Weaknesses:**
-- Least deterministic: conversation dynamics are inherently unpredictable
-- Context overflow risk: long conversations exceed context windows
-- Debugging difficulty: tracing which conversational turn caused which outcome
-- Performance unpredictability: conversation length varies significantly
-
-**Structural Tension Analysis:** AutoGen's conversational model naturally surfaces tension—critics identify gaps between desired and actual output. However, this tension is implicit in conversation rather than explicitly tracked, making it difficult to measure advancement vs. oscillation.
-
-### 4. Protocol-Standardized Tool Invocation: MCP
-
-**Architecture:** MCP is not an orchestration framework but a **communication protocol** that standardizes how agents invoke tools. Unlike CrewAI, LangGraph, and AutoGen, MCP does not prescribe coordination patterns—it provides the infrastructure layer upon which coordination is built [12][13].
-
-**Coordination Model:** MCP defines tool discovery (tools/list), tool invocation (tools/call), resource access (resources/read), and prompt provision (prompts/get) through JSON-RPC 2.0 messages over stdio or HTTP transport.
-
-**Strengths:**
-- Universal: any framework can use MCP for tool invocation
-- Dynamic tool discovery: agents learn available capabilities at runtime
-- Human-in-the-loop: built-in approval gates for sensitive operations
-- Framework-agnostic: works with CrewAI, LangGraph, AutoGen, or custom systems
-
-**Weaknesses:**
-- Not an orchestration framework: does not handle agent coordination, context sharing, or workflow management
-- Young ecosystem: tooling and community still developing
-- Potential overhead: JSON-RPC adds latency compared to direct function calls
-
-**Structural Tension Analysis:** MCP provides the tool interface through which structural tension charts can be created, read, and updated. MCP servers can expose STC operations as tools, making structural tension accessible to any MCP-compliant agent.
-
-### Comparative Analysis
-
-| Dimension | CrewAI | LangGraph | AutoGen/AG2 | MCP |
-|-----------|--------|-----------|-------------|-----|
-| **Paradigm** | Role-based teams | Graph state machine | Conversation | Protocol |
-| **Control** | Low–Medium | Very High | Medium | N/A (infra) |
-| **Determinism** | Low | High | Low | N/A |
-| **Learning Curve** | Low | High | Medium | Low |
-| **Best For** | Prototyping, business | Production, complex | Research, negotiation | Tool interop |
-| **STC Support** | None native | Possible via state | Implicit in conversation | Tool interface |
-| **Oscillation Risk** | High | Low (explicit control) | Medium | N/A |
-
-### Emerging Pattern: Hybrid Architectures
-
-The most effective multi-agent systems in 2025 combine multiple paradigms:
-
-1. **MCP as universal tool layer.** All external tool invocations go through MCP, providing standardized access, observability, and security.
-2. **LangGraph for critical workflows.** Mission-critical, multi-step processes use graph-based orchestration for determinism and error recovery.
-3. **CrewAI for team simulation.** Business workflows with clear role divisions use role-based orchestration for intuitive design.
-4. **AutoGen for collaborative reasoning.** Research, brainstorming, and negotiation tasks use conversational orchestration for emergent insights.
-
-### The Four Directions Alternative
-
-CoAiA.js proposes a fifth paradigm: **directional orchestration** based on the Medicine Wheel framework:
-
-- **East Agent:** Vision articulation and creative ideation
-- **South Agent:** Analysis, research, and prompt decomposition (PDE)
-- **West Agent:** Implementation and iterative building
-- **North Agent:** Evaluation (MMOT), documentation, and wisdom synthesis
-
-This model is structurally distinct from role-based, graph-based, or conversation-based orchestration:
-
-- Unlike CrewAI roles, directional agents represent **phases of creative process**, not job descriptions
-- Unlike LangGraph graphs, the Four Directions form a **cycle**, not a directed acyclic graph
-- Unlike AutoGen conversations, handoffs between directions are **structurally motivated** by the completion of each phase's contribution
-- Structural tension charts provide the coordination mechanism: each direction advances the chart, and the tension between desired outcome and current reality drives the cycle forward
-
-## Implications for CoAiA.js
-
-1. **MCP as tool layer.** CoAiA.js uses MCP for all external tool invocations, ensuring framework-agnostic tool access.
-2. **Four Directions as orchestration.** Agent coordination follows the directional cycle, not role-based teams or conversation rounds.
-3. **Structural tension as coordination.** The STC's tension measurement drives agent scheduling—the direction with the highest contribution potential receives attention.
-4. **Hybrid compatibility.** CoAiA.js's directional agents can operate within LangGraph workflows, CrewAI teams, or AutoGen conversations when integration with external frameworks is needed.
-5. **Oscillation prevention.** The structural tension chart's advancing structure prevents the oscillation that plagues role-based and conversation-based systems.
-
-## Conclusion
-
-Multi-agent orchestration in 2025 is characterized by framework proliferation and emerging hybridization. Each paradigm—role-based, graph-based, conversational, and protocol-based—offers distinct trade-offs in control, determinism, and expressiveness. CoAiA.js's Four Directions orchestration model adds a fifth paradigm rooted in structural tension methodology, providing cyclical, balanced coordination that naturally resists oscillation. The convergence on MCP as a universal tool interface layer suggests that orchestration frameworks will increasingly specialize in coordination logic while delegating tool invocation to the protocol layer.
-
-## References
-
-1. DataCamp. (2024). "CrewAI vs LangGraph vs AutoGen: Choosing the Right Multi-Agent Framework." https://www.datacamp.com/tutorial/crewai-vs-langgraph-vs-autogen
-2. TutorialQ. (2025). "CrewAI vs LangGraph vs AutoGen." https://tutorialq.com/agents/frameworks/crewai-vs-langgraph-vs-autogen
-3. Youngju.dev. (2026). "Comparing LLM Agent Frameworks." https://www.youngju.dev/blog/llm/2026-03-09-llm-agent-framework-autogen-crewai-langgraph-comparison.en
-4. Amplework. (2025). "LangGraph vs AutoGen vs CrewAI." https://www.amplework.com/blog/langgraph-vs-autogen-vs-crewai-multi-agent-framework/
-5. Braincuber. (2025). "CrewAI vs AutoGen vs LangGraph: Framework Comparison." https://www.braincuber.com/blog/crewai-vs-autogen-vs-langgraph-multi-agent-framework-comparison
-6. Meta Intelligence. (2025). "The Complete Guide to AI Agent Development." https://www.meta-intelligence.tech/en/insight-ai-agent-frameworks
-7. SoftwareSeni. (2025). "Navigating the Multi-Agent Framework Landscape." https://www.softwareseni.com/navigating-the-multi-agent-framework-landscape-from-crewai-to-langgraph-to-autogen-and-beyond/
-8. Infinite Lambda. (2025). "CrewAI, AutoGen, Vertex AI, and LangGraph Comparison." https://infinitelambda.com/compare-crewai-autogen-vertexai-langgraph/
-9. Datagrom. (2025). "Top 3 Trending Agentic AI Frameworks." https://www.datagrom.com/data-science-machine-learning-ai-blog/langgraph-vs-autogen-vs-crewai-comparison-agentic-ai-frameworks
-10. Tagline Infotech. (2025). "Choosing the Right AI Agent Framework." https://taglineinfotech.com/blog/langgraph-vs-crewai-vs-autogen/
-11. AG2 (AutoGen successor). https://github.com/ag2ai/ag2
-12. Anthropic. (2024). "Introducing the Model Context Protocol." https://www.anthropic.com/news/model-context-protocol
-13. Model Context Protocol Specification. (2025). https://modelcontextprotocol.io/specification/2025-03-26