cognitive-core 0.0.2 → 0.1.2

package/docs/RESEARCH-team-trajectory-learning.md

@@ -0,0 +1,553 @@
+# Research Synthesis: Learning from Multi-Agent Team Trajectories
+
+## Context
+
+This document synthesizes recent research (2025+) relevant to the design of team-aware learning in cognitive-core. The core problem: given a **graph of interconnected agent trajectories** from a multi-agent team execution, how do we extract learning signals when (a) the primary analysis mechanism is LLM-based semantic reasoning, and (b) the data can be extremely large and diverse, especially for long-horizon tasks?
+
+The research clusters into five themes, each addressing a piece of this problem.
+
+---
+
+## Theme 1: LLM-Based Credit Assignment Across Agents
+
+The central question for team learning is credit assignment: when a team succeeds or fails, which agents and which interactions were responsible? Several recent papers demonstrate that LLMs can perform this decomposition through semantic reasoning, without hand-designed reward functions.
+
+### Key Papers
+
+**LLM-MCA / LLM-TACA** (AAMAS 2025)
+- Reformulates multi-agent credit assignment as sequence improvement recognition + attribution
+- A centralized LLM reward-critic numerically decomposes environment reward based on individualized agent contributions
+- LLM-TACA extension performs explicit task assignment by passing intermediary goals to each agent
+- Generates trajectory datasets annotated with per-agent reward at each timestep
+- *Relevance*: This is the closest existing work to what our `TeamTrajectoryAnalyzer` needs to do — take a team trajectory graph, use an LLM to assess each member's contribution and each interaction's value
+- Source: https://arxiv.org/abs/2502.16863
+
+**RICOL — Retrospective In-Context Learning for Efficient Credit Assignment** (NeurIPS 2025)
+- Uses in-context learning to evaluate actions in a trajectory, converting sparse rewards into dense advantage signals
+- Achieves equivalent accuracy to Monte Carlo rollouts with ~100x fewer environment samples
+- Key finding: retrospective LLM analysis is sample-efficient enough to be practical at scale
+- *Relevance*: Validates that post-hoc LLM-based credit assignment is practical. For our system, this means we can analyze team trajectories after execution rather than requiring online instrumentation
+- Source: https://arxiv.org/abs/2502.19607
+
+**Speaking the Language of Teamwork** (Feb 2025)
+- LLM generates dense, agent-specific rewards based on natural language task descriptions and team goals
+- Learns a potential-based reward function over multiple queries, reducing impact of ranking errors
+- Evaluates each agent's contribution to overall task completion using natural language reasoning
+- *Relevance*: The natural-language reward generation aligns with our LLM-based semantic analysis approach — credit is expressed and reasoned about in language, not computed from numeric features
+- Source: https://arxiv.org/abs/2502.03723
+
+**CollabUIAgents — Credit Re-Assignment for Multi-Agent Teams** (Feb 2025)
+- Multi-agent RL framework with LLM-based process rewards rather than environment-specific rewards
+- 7B-parameter system achieves results on par with or exceeding strong closed-source models
+- Cross-environment generalizability — learned credit signals transfer across task domains
+- *Relevance*: Suggests that coordination patterns learned from one domain (e.g., web development) can transfer to another (e.g., data pipeline), which is important for team playbook generalization
+- Source: https://arxiv.org/abs/2502.14496
+
+**MAGRPO — Multi-Agent Group Relative Policy Optimization** (NeurIPS 2025 Workshop)
+- Models LLM collaboration as a Dec-POMDP (Decentralized Partially Observable MDP)
+- Proposes centralized group-relative advantages for joint optimization with decentralized execution
+- Agents roll out trajectories in groups; relative advantages computed by comparing final rewards within each group
+- *Relevance*: The Dec-POMDP formalization provides theoretical grounding for our team trajectory graph — each agent has partial observability of the team state, and the interaction edges are the communication channel
+- Source: https://arxiv.org/abs/2508.04652
+
+### Synthesis: What This Means for Our Design
+
+The literature converges on a clear pattern: **LLMs can serve as the credit assignment function for multi-agent teams, reasoning about contributions in natural language**. This eliminates the need for hand-designed reward functions or environment-specific metrics. The key architectural implication:
+
+```
+TeamTrajectoryAnalyzer does NOT compute credit from metrics alone.
+Instead:
+1. Structural metrics (latency, message counts, bottlenecks) provide CONTEXT
+2. An LLM reasons about credit using the metrics + trajectory content
+3. The LLM produces per-agent and per-interaction attribution scores
+   with natural language explanations
+```
+
+The RICOL result (100x sample efficiency of retrospective LLM analysis vs. Monte Carlo) validates that this is practical even for large trajectory datasets.
+
+---
+
+## Theme 2: Trajectory Compression for Scale
+
+Long-horizon multi-agent tasks can produce enormous trajectory data. An agent solving a complex task might generate hundreds of steps; a team of 5 agents produces 5x that, plus interaction overhead. LLM-based semantic analysis cannot process all of this raw data. Two complementary strategies emerge from the literature.
+
+### Strategy A: Learned Compression Before Analysis
+
+**ACON — Optimizing Context Compression for Long-Horizon Agents** (Oct 2025)
+- Unified framework compressing both environment observations and interaction histories
+- Uses a *compression guideline optimization* pipeline: given paired trajectories where full context succeeds but compressed context fails, a capable LLM analyzes the failure cause and updates compression guidelines in natural language
+- Gradient-free — works with closed-source models
+- Reduces peak tokens by 26-54% while preserving 95%+ accuracy
+- The optimized compressor can be distilled into smaller models
+- *Relevance*: The learned compression guidelines could be specialized for team trajectories — learning what information to preserve at interaction boundaries vs. within individual agent reasoning chains
+- Source: https://arxiv.org/abs/2510.00615
+
+**AgentFold — Proactive Context Management** (Oct 2025)
+- Treats context as a "dynamic cognitive workspace" rather than a passive log
+- At each step, agent executes a learned "folding" operation at multiple scales:
+  - Granular condensations preserving fine-grained details
+  - Deep consolidations abstracting entire multi-step sub-tasks
+- Implements a "perceive-reason-fold-act" loop where context curation is an explicit, learned step
+- A 30B model with folding outperforms 355B and 671B baselines without it
+- *Relevance*: The multi-scale folding maps well to our hierarchy: individual steps can be granularly condensed, entire sub-tasks deeply abstracted, and interaction boundaries kept at full fidelity
+- Source: https://arxiv.org/abs/2510.24699
+
+**Context-Folding via Branch/Return** (Oct 2025)
+- Introduces two special actions: `branch` and `return`
+- Agent branches into a sub-trajectory for a subtask; upon `return`, intermediate steps collapse into a self-chosen summary
+- FoldPO: an RL framework with process rewards for learning when/how to fold
+- With a 32K active token budget and max 10 branches (theoretical 327K tokens), achieves strong results on SWE-Bench Verified
+- Active context is 10x smaller than baselines
+- *Relevance*: The branch/return mechanism directly mirrors how sub-tasks within a team trajectory could be recursively summarized. The RL-trained folding policy learns which parts are critical vs. safely abstractable
+- Source: https://huggingface.co/papers/2510.11967
+
+**AgentDiet — Trajectory Reduction** (Sep 2025)
+- Through manual inspection of top-performing agent trajectories, found that *token waste is pervasive*
+- Categorized three typical waste scenarios where tokens can be removed or compressed
+- Achieves comparable performance (-1.0% to +2.0%) with significant step reduction (57.2 to 43.9 average steps)
+- **Key finding: removing low-quality context can actually *improve* performance**
+- *Relevance*: Validates aggressive compression. Much of what agents produce is noise — removing it before team-level analysis is not just acceptable, it's beneficial
+- Source: https://arxiv.org/abs/2509.23586
+
+**PAACE — Plan-Aware Automated Agent Context Engineering** (Dec 2025)
+- Introduces next-k-task relevance modeling: compression is aware of upcoming tasks, not just past history
+- Plan-structure analysis determines what to preserve based on the agent's plan
+- Function-preserving compression ensures compressed context retains functional equivalence
+- *Relevance*: For team trajectories, compression should be plan-aware at two levels: each agent's individual plan, and the team's overall coordination plan. Information relevant to cross-agent interactions should be preserved even if it's low-relevance within a single agent's trajectory
+- Source: https://arxiv.org/abs/2512.16970
+
+**The Complexity Trap** (Aug 2025)
+- Challenges the assumption that sophisticated semantic compression always beats simpler methods
+- Finds that LLM-based summarization can cause "trajectory elongation" — agents persist on unproductive paths because summaries lose important failure signals
+- Simple observation masking can be equally effective in many cases
+- Summary-generation costs 5-7% of per-instance cost
+- *Relevance*: Cautionary finding. For our compression pipeline, simpler methods (e.g., dropping verbose tool output, truncating repeated patterns) should be tried before LLM-based summarization. Hybrid approach is likely optimal
+- Source: https://arxiv.org/abs/2508.21433
+
+### Strategy B: Identify Critical Steps, Ignore the Rest
+
+**Verified Critical Step Optimization (CSO)** (Feb 2026)
+- Identifies verified critical steps where alternative actions demonstrably flip task outcomes
+- **Only 16% of trajectory steps are critical**
+- An 8B model trained on just critical steps matches GPT-4.1 on GAIA and XBench-DeepSearch
+- *Relevance*: We don't need to analyze entire team trajectories — we need to find the ~16% of steps and interactions that actually determined the outcome. This dramatically reduces the scope of expensive LLM analysis
+- Source: https://arxiv.org/abs/2602.03412
+
+**AgentPRM — Process Reward Models for Agents** (Nov 2025)
+- Constructs process reward models that evaluate each step based on:
+  - **Promise**: proximity to goal (forward-looking)
+  - **Progress**: incremental contribution (backward-looking)
+- Captures sequential dependencies between decisions
+- Unlike reasoning PRMs that score correctness, agent PRMs evaluate contribution and progress
+- *Relevance*: Could be applied per-step within each agent's trajectory and per-interaction across the team graph. The promise/progress dual scoring is particularly useful for interaction edges: "did this delegation bring the team closer to the goal?"
+- Source: https://arxiv.org/abs/2511.08325
+
+**MT-GRPO — Turn-Level Credit Assignment** (May 2025)
+- Fine-grained turn-level credit assignment for multi-turn agent training
+- Designs turn-level verifiable rewards for both intermediate and final turns
+- Avoids merging signals into a single sparse trajectory-level reward
+- *Relevance*: Validates dense per-step rewards over sparse per-trajectory rewards. Our team analyzer should produce per-interaction credit, not just per-agent credit
+- Source: https://arxiv.org/abs/2505.11821
+
+**iStar — Implicit Step Rewards** (Sep 2025)
+- Jointly optimizes an implicit PRM with the policy model via a trajectory-based DPO objective
+- No additional rollouts or explicit step labels needed
+- General credit-assignment strategy integrating with standard RL algorithms
+- *Relevance*: A potential approach for learning per-step rewards without manual annotation — the reward model learns implicitly from trajectory outcome comparisons
+- Source: https://arxiv.org/abs/2509.19199
+
+### Synthesis: What This Means for Our Design
+
+The literature strongly supports a **two-phase approach to handling scale**:
+
+1. **Compress first**: Use plan-aware compression (PAACE) or learned folding (AgentFold, Context-Folding) to reduce individual trajectories before team-level analysis. Preserve interaction boundary steps in full while folding internal reasoning. AgentDiet validates that aggressive compression is safe.
+
+2. **Find critical moments**: CSO's finding that only ~16% of steps matter is powerful. Before expensive LLM analysis, identify critical steps (those where outcomes diverge) and critical interactions (those that changed the trajectory of downstream agents). Focus semantic analysis on these.
+
+Together, these reduce the data an LLM needs to analyze by roughly 10x (compression) × 5x (critical step filtering) = **~50x reduction** before semantic analysis begins.
+
+```
+Raw team trajectory: ~5 agents × ~100 steps × ~500 tokens/step = 250K tokens
+After per-agent compression (10x): ~25K tokens
+After critical step filtering (~16%): ~4K tokens
+→ LLM semantic analysis operates on ~4K tokens, not 250K
+```
+
+---
+
+## Theme 3: Graph-Based Trajectory Analysis
+
+Our team trajectory is a graph of interconnected individual trajectories. Several papers explore graph-based representations of agent execution data.
+
+### Key Papers
+
+**From Experience to Strategy: Trainable Graph Memory** (Nov 2025)
+- Abstracts raw trajectories into structured decision paths in a state machine
+- Distills state-machine paths into high-level, human-interpretable "meta-cognitions"
+- Uses RL-based weight optimization to estimate empirical utility of each meta-cognition based on reward feedback
+- Dynamically integrates optimized strategies via meta-cognitive prompting
+- *Relevance*: The state-machine abstraction over trajectories could represent our team interaction graph. Each agent's trajectory becomes a set of states, and interactions between agents become transitions in a team-level state machine. The meta-cognition distillation maps to our team playbook extraction
+- Source: https://arxiv.org/abs/2511.07800
+
+**Reasoning LLM Behavior Analysis via Graph Perspective** (May 2025)
+- Traces reasoning processes from a graph perspective
+- Goes beyond the token level to focus on semantic organization: clusters raw reasoning tokens into coherent logic steps and maps their inter-dependencies as a directed graph
+- Enables structural analysis of reasoning quality through graph metrics (connectivity, branching factor, cycle detection)
+- *Relevance*: The methodology — cluster raw steps into semantic units, then analyze the dependency graph — is directly applicable to team trajectory analysis. Individual agent steps cluster into subtasks, and the interaction graph connects subtasks across agents
+- Source: https://arxiv.org/abs/2505.13890
+
+**AgentArmor — Structured Graph Abstraction of Execution Traces** (Nov 2025)
+- Abstracts runtime execution traces into Program Dependence Graphs (PDGs) incorporating:
+  - Control dependencies (what triggered what)
+  - Data dependencies (what data flowed where)
+  - Data flow edges (the content that moved between components)
+- Hooks into agent runtime to capture traces, then generates PDGs
+- *Relevance*: The PDG abstraction — capturing both control and data dependencies — maps well to our interaction graph. A delegation is a control dependency; the delegated task content is a data dependency. This dual-edge representation is richer than our current single-edge `AgentInteraction` type
+- Source: https://arxiv.org/abs/2508.01249
+
+**Semantic Dynamics Analysis** (Feb 2026)
+- Proposes the *Structural Context Model* for formally analyzing LLM agents from the perspective of context structure
+- Introduces *Semantic Dynamics Analysis* — a method for analyzing functional roles and relative contributions of different context parts
+- Facilitates identification and abstraction of reusable context patterns
+- Up to 32 percentage points improvement in success rate when reusable patterns are applied
+- *Relevance*: The semantic dynamics framework could analyze roles and contributions of different segments within our team trajectory graph — identifying which agent communications, reasoning steps, or actions are most functionally significant
+- Source: https://arxiv.org/abs/2602.08276
+
+### Synthesis: What This Means for Our Design
+
+The graph-based analysis literature suggests our interaction graph should capture **two types of edges**, not one:
+
+1. **Control edges**: "Agent A triggered Agent B to do X" (delegations, signals, escalations)
+2. **Data edges**: "Agent A passed information Y to Agent B" (message content, task descriptions, results)
+
+This dual-edge representation enables richer analysis:
+- Control-only edges with no data = empty delegation (agent was told to act but given no useful context)
+- Data-only edges with no control = passive information sharing (useful context was available but didn't trigger action)
+- Both = well-structured coordination
+
+The state-machine abstraction from "Experience to Strategy" is particularly compelling: rather than analyzing the raw interaction graph, we could first abstract it into a team-level state machine where states represent coordination phases (planning, parallel execution, integration, verification) and transitions represent phase changes triggered by interactions.
+
+---
+
+## Theme 4: Experience Distillation into Reusable Knowledge
+
+Once we've analyzed team trajectories and assigned credit, we need to extract reusable coordination patterns — the team playbooks. Several papers address how to distill raw execution experience into abstract, reusable knowledge.
+
+### Key Papers
+
+**EvolveR — Self-Evolving Agents through Experience-Driven Lifecycle** (Oct 2025)
+- Two-stage lifecycle:
+  1. *Offline Self-Distillation*: Trajectories are synthesized into a structured repository of abstract, reusable strategic principles via semantic deduplication, integration, and quality control guided by dynamic metrics
+  2. *Online Interaction*: Agent retrieves distilled principles to guide decision-making
+- Contrasts with agents that discard experiences or retrieve raw un-distilled trajectories
+- Key insight: **raw trajectory retrieval is inferior to distilled principle retrieval** — abstraction improves both retrieval quality and downstream performance
+- *Relevance*: Directly applicable. Our team playbook extraction pipeline should follow the same pattern: distill team trajectory batches into abstract coordination principles, not just store and retrieve raw team trajectories
+- Source: https://arxiv.org/abs/2510.16079
+
+**Lessons Learned — Multi-Agent Framework for Learning and Improvement** (IBM, NeurIPS 2025)
+- Agents learn from each other's successes and failures through a "lesson" mechanism
+- Three-phase process: lesson solicitation → banking → selection
+- A "lesson" is knowledge produced by one agent and passed to others — abstract enough to be reusable, concrete enough to be actionable
+- Team of small LLMs with lessons outperforms much larger single LLMs and other multi-LLM collaboration methods
+- *Relevance*: The lesson abstraction level — between raw trajectory data and high-level principles — is a useful intermediate representation. Our system could extract "team lessons" from trajectory analysis before distilling them into playbooks
+- Source: IBM Research
+
+**Memento — Fine-tuning Agents without Fine-tuning LLMs** (Aug 2025)
+- Memory-augmented MDP with a neural case-selection policy
+- Stores past trajectories as episodic memory in a growing Case Bank
+- Planner retrieves relevant cases and selects execution plans; results are stored back, creating a feedback loop
+- Achieves 87.88% Pass@3 on GAIA without any gradient updates to the LLM
+- *Relevance*: The Case Bank architecture could store team-level execution patterns. The key insight: you don't need to fine-tune models to learn from experience — retrieval from a well-curated case bank is sufficient
+- Source: https://arxiv.org/abs/2508.16153
+
+**ELL — Experience-Driven Lifelong Learning** (Aug 2025)
+- Meta-cognitive learning architecture that explicitly reflects on successes and failures across task trajectories
+- Extracts actionable lessons and integrates them into future behavior via in-context learning or knowledge distillation
+- Introduces a benchmark for lifelong learning evaluation
+- *Relevance*: The meta-cognitive reflection process — explicitly asking "what worked, what didn't, and why" — is the core of our team playbook refinement loop
+- Source: https://arxiv.org/abs/2508.19005
+
+**Structured Agent Distillation** (May 2025)
+- Segments trajectories into `{[REASON]}` and `{[ACT]}` spans, applying segment-specific losses
+- Unlike standard token-level distillation, this preserves both reasoning fidelity and action consistency
+- Outperforms token-level and imitation learning baselines
+- *Relevance*: The segment-specific treatment could be extended to team trajectories with three segment types: reasoning spans, action spans, and **coordination spans** (inter-agent communication). Each type may need different distillation/extraction logic
+- Source: https://arxiv.org/abs/2505.13820
+
+### Synthesis: What This Means for Our Design
+
+The distillation literature suggests a **three-level knowledge hierarchy**:
+
+```
+Level 1: Raw team trajectory graph
+  ↓ (compression + critical step identification)
+Level 2: Team lessons — concrete, contextualized insights
+  e.g., "In this web auth task, delegating OAuth implementation to a
+  specialist while the planner handled routing reduced total time by 40%"
+  ↓ (abstraction + deduplication across multiple team executions)
+Level 3: Team playbooks — abstract, reusable coordination patterns
+  e.g., "For tasks requiring specialist knowledge, delegate to a
+  domain-expert executor rather than having the planner attempt it directly.
+  Correlated with 35% efficiency improvement across 8 team executions."
+```
+
+EvolveR's key finding — that distilled principles outperform raw trajectory retrieval — argues strongly for investing in the Level 2 → Level 3 abstraction step rather than just storing and retrieving raw team experiences. The Lessons Learned framework from IBM provides a concrete mechanism for the Level 1 → Level 2 step: structured solicitation of lessons from each team member's perspective, then banking them for cross-team reuse.
+
+---
+
+## Theme 5: Hierarchical Summarization for Long-Horizon Tasks
+
+Long-horizon team tasks require multi-level summarization — individual steps within a sub-task, sub-tasks within an agent's trajectory, agent trajectories within the team execution.
+
+### Key Papers
+
+**HiAgent — Hierarchical Working Memory Management** (ACL 2025)
+- Uses subgoals as memory chunks to manage working memory hierarchically
+- Agents formulate subgoals before actions and proactively replace previous subgoals with summarized observations
+- Only retains action-observation pairs relevant to the current subgoal
+- Achieves 2x success rate increase and 3.8 fewer average steps
+- *Relevance*: The subgoal-based hierarchical chunking provides a natural summarization boundary for team trajectories. Each agent's subgoal hierarchy forms a tree; the team trajectory is a forest of these trees connected by interaction edges
+- Source: https://aclanthology.org/2025.acl-long.1575/
+
+**Focus — Active Context Compression** (Jan 2026)
+- Agent actively prunes its own history during a single task
+- Preserves learnings in a structured "knowledge block"
+- Creates a "sawtooth" context pattern: context grows during exploration and collapses during consolidation
+- 22.7% net token savings despite summary generation overhead
+- *Relevance*: The sawtooth pattern and knowledge block abstraction could be applied per-agent before merging at the team level. Each agent's "knowledge block" at the end of a consolidation phase becomes a natural unit for team-level analysis
+- Source: https://arxiv.org/abs/2601.07190
+
+**ReSum — Context Summarization for Long-Horizon Search** (Sep 2025)
+- Converts growing interaction histories into compact "reasoning states" via periodic summarization
+- Proposes ReSum-GRPO: GRPO with segmented trajectory training and advantage broadcasting
+- With 1K training samples, achieves strong results on BrowseComp
+- *Relevance*: The "reasoning state" concept could serve as the unit of exchange in our team trajectory graph — at interaction boundaries, each agent's current state is captured as a compact reasoning state rather than the full history
+- Source: https://arxiv.org/abs/2509.13313
+
+**U-Fold — Dynamic Intent-Aware Context Folding** (Jan 2026)
+- Extends context folding to handle evolving intents
+- Unlike methods that periodically summarize and discard history, U-Fold retains full history and dynamically extracts intent-relevant data on demand
+- Avoids information loss from premature summarization
+- *Relevance*: For team-level analysis, the "intent" changes depending on what learning signal we're extracting (credit assignment vs. pattern extraction vs. anti-pattern detection). U-Fold's approach suggests keeping the full trajectory available and extracting intent-specific views rather than committing to a single compression
+- Source: https://arxiv.org/abs/2601.18285
+
+### Synthesis: What This Means for Our Design
+
+The hierarchical summarization literature points to a **multi-resolution representation** of team trajectories:
+
+```
+Resolution 1 (full): Complete step-by-step data for each agent
+  → Used for: detailed debugging, fine-grained credit assignment
+  → Stored: raw trajectory files
+
+Resolution 2 (subgoal): Agent trajectories chunked by subgoal, with
+  internal steps summarized into knowledge blocks
+  → Used for: interaction analysis, coordination pattern extraction
+  → Stored: compressed trajectory with subgoal boundaries
+
+Resolution 3 (agent summary): Each agent's trajectory collapsed into
+  a reasoning state — goals, key decisions, outcome
+  → Used for: team-level credit assignment, playbook extraction
+  → Stored: team trajectory summary
+
+Resolution 4 (team summary): Entire team execution as a narrative —
+  who did what, how they coordinated, what worked/failed
+  → Used for: team playbook matching, meta-learning
+  → Stored: team experience record
+```
+
+U-Fold's insight is important: rather than committing to one resolution upfront, keep the full data and extract the appropriate resolution on demand based on the analysis intent. This means our storage strategy should preserve the full trajectories, with compressed views generated lazily during analysis.
+
+---
+
+## Proposed Knowledge Extraction Pipeline
+
+Drawing from all five themes, the team trajectory knowledge extraction pipeline:
+
+```
+┌──────────────────────────────────────────────────────────────────┐
+│ PHASE 1: INGEST + TAG                                           │
+│                                                                  │
+│ Input: Raw team trajectory graph                                 │
+│   - Individual trajectories with runtime interaction tags        │
+│   - Interaction edges built from matching interactionId tags     │
+│                                                                  │
+│ Output: TeamTrajectory with validated interaction graph           │
+│                                                                  │
+│ Informed by: Runtime tagging (our design)                        │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│ PHASE 2: COMPRESS                                                │
+│                                                                  │
+│ Per-agent trajectory compression:                                │
+│   - Fold internal reasoning into subgoal-level summaries         │
+│     (HiAgent, AgentFold, Context-Folding)                        │
+│   - Preserve interaction boundary steps at full fidelity         │
+│   - Remove token waste: verbose tool output, repeated attempts   │
+│     (AgentDiet)                                                  │
+│   - Plan-aware: preserve info relevant to upcoming interactions  │
+│     (PAACE)                                                      │
+│                                                                  │
+│ Target: ~10x reduction in per-agent trajectory size              │
+│                                                                  │
+│ Caution: Simple methods first (The Complexity Trap).             │
+│ LLM-based summarization only where structural methods fail.      │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│ PHASE 3: IDENTIFY CRITICAL MOMENTS                               │
+│                                                                  │
+│ Critical step identification:                                    │
+│   - CSO-style: find steps where alternative actions would have   │
+│     flipped the outcome (~16% of steps)                          │
+│   - AgentPRM-style: score each step on promise + progress        │
+│                                                                  │
+│ Critical interaction identification:                             │
+│   - Which interactions changed downstream agent behavior?        │
+│   - Which delegations led to successful/failed subtasks?         │
+│   - Which signals triggered meaningful state changes?            │
+│   - Which messages were redundant/unnecessary?                   │
+│                                                                  │
+│ Target: Focus downstream LLM analysis on ~16% of steps           │
+│ and the interaction edges connecting them                        │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│ PHASE 4: LLM-BASED SEMANTIC ANALYSIS                             │
+│                                                                  │
+│ Now operating on compressed, focused data (~50x smaller):        │
+│                                                                  │
+│ 4a. Per-agent credit assignment (RICOL-style)                    │
+│   - Retrospective analysis of each agent's trajectory            │
+│   - Dense per-step attribution with natural language reasoning   │
+│                                                                  │
+│ 4b. Per-interaction credit assignment (LLM-MCA-style)            │
+│   - For each interaction edge: was it helpful, neutral, harmful? │
+│   - Cross-trajectory causal analysis                             │
+│   - Natural language explanation of interaction value            │
+│                                                                  │
+│ 4c. Team-level outcome analysis                                  │
+│   - Holistic assessment: what coordination patterns drove the    │
+│     outcome?                                                     │
+│   - Comparison to expected patterns (from team.yaml)             │
+│   - Identification of emergent coordination (not declared but    │
+│     observed)                                                    │
+│                                                                  │
+│ Output: Annotated team trajectory with credit scores +           │
+│         natural language analysis                                │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│ PHASE 5: EXTRACT LESSONS (Level 2 knowledge)                     │
+│                                                                  │
+│ From the annotated trajectory, extract concrete lessons          │
+│ (Lessons Learned framework, IBM):                                │
+│                                                                  │
+│   - What coordination patterns worked? Why?                      │
+│   - What coordination patterns failed? Why?                      │
+│   - Were there unexpected interactions (emergent coordination)?  │
+│   - What would the team do differently?                          │
+│   - Per-role lessons: what did each role learn?                   │
+│                                                                  │
+│ Output: TeamLessons — contextualized, concrete insights          │
+│         attached to specific trajectory evidence                 │
+└──────────────────────────┬───────────────────────────────────────┘
+                           │
+                           ▼
+┌──────────────────────────────────────────────────────────────────┐
+│ PHASE 6: DISTILL PLAYBOOKS (Level 3 knowledge)                   │
+│                                                                  │
+│ Across batches of team lessons (5+ team executions):             │
+│                                                                  │
+│ 6a. Abstract into principles (EvolveR-style)                     │
+│   - Semantic deduplication across team lessons                   │
+│   - Integration of recurring patterns into abstract principles   │
+│   - Quality control via success/failure correlation              │
+│                                                                  │
+│ 6b. Extract interaction graph patterns                           │
+│   - Common subgraph motifs across successful teams               │
+│   - Anti-pattern subgraphs from failed teams                     │
+│   - Role interaction templates                                   │
+│                                                                  │
+│ 6c. Build/update team playbooks                                  │
+│   - Composition guidance (roles, topology, spawn rules)          │
+│   - Coordination guidance (channels, interaction patterns,       │
+│     delegation strategy)                                         │
+│   - Verification criteria (success/failure indicators)           │
+│                                                                  │
+│ 6d. State-machine abstraction (Experience to Strategy)           │
+│   - Abstract interaction graph into coordination phases          │
+│   - Encode phase transitions as meta-cognitive strategies        │
+│                                                                  │
+│ Output: TeamPlaybooks — abstract, reusable coordination patterns │
+└──────────────────────────────────────────────────────────────────┘
+```
+
+### Scale Estimates
+
+For a typical long-horizon team task:
+
+| Stage | Data Size | Reduction |
+|-------|-----------|-----------|
+| Raw team trajectory | ~250K tokens (5 agents × 100 steps × 500 tok/step) | — |
+| After Phase 2 (compression) | ~25K tokens | 10x |
+| After Phase 3 (critical steps) | ~4K tokens | 6x |
+| Phase 4 LLM analysis input | ~4K tokens + ~2K prompt | — |
+| Phase 5 lessons output | ~1K tokens per lesson, ~5 lessons | — |
+| Phase 6 playbook output | ~2K tokens per playbook | — |
+
+This makes LLM-based semantic analysis tractable even for long-horizon multi-agent tasks. The LLM sees ~4K tokens of focused, high-signal data rather than ~250K tokens of raw trajectories.
+
+---
+
+## Key References
+
+### Credit Assignment
+- LLM-MCA / LLM-TACA (AAMAS 2025) — https://arxiv.org/abs/2502.16863
+- RICOL (NeurIPS 2025) — https://arxiv.org/abs/2502.19607
+- Speaking the Language of Teamwork (Feb 2025) — https://arxiv.org/abs/2502.03723
+- CollabUIAgents (Feb 2025) — https://arxiv.org/abs/2502.14496
+- MAGRPO (NeurIPS 2025 Workshop) — https://arxiv.org/abs/2508.04652
+
+### Trajectory Compression
+- ACON (Oct 2025) — https://arxiv.org/abs/2510.00615
+- AgentFold (Oct 2025) — https://arxiv.org/abs/2510.24699
+- Context-Folding (Oct 2025) — https://huggingface.co/papers/2510.11967
+- AgentDiet (Sep 2025) — https://arxiv.org/abs/2509.23586
+- PAACE (Dec 2025) — https://arxiv.org/abs/2512.16970
+- The Complexity Trap (Aug 2025) — https://arxiv.org/abs/2508.21433
+
+### Critical Step Identification
+- Verified Critical Step Optimization (Feb 2026) — https://arxiv.org/abs/2602.03412
+- AgentPRM (Nov 2025) — https://arxiv.org/abs/2511.08325
+- MT-GRPO (May 2025) — https://arxiv.org/abs/2505.11821
+- iStar (Sep 2025) — https://arxiv.org/abs/2509.19199
+
+### Graph-Based Analysis
+- From Experience to Strategy (Nov 2025) — https://arxiv.org/abs/2511.07800
+- Reasoning via Graph Perspective (May 2025) — https://arxiv.org/abs/2505.13890
+- AgentArmor (Nov 2025) — https://arxiv.org/abs/2508.01249
+- Semantic Dynamics Analysis (Feb 2026) — https://arxiv.org/abs/2602.08276
+
+### Experience Distillation
+- EvolveR (Oct 2025) — https://arxiv.org/abs/2510.16079
+- Lessons Learned (IBM, NeurIPS 2025)
+- Memento (Aug 2025) — https://arxiv.org/abs/2508.16153
+- ELL (Aug 2025) — https://arxiv.org/abs/2508.19005
+- Structured Agent Distillation (May 2025) — https://arxiv.org/abs/2505.13820
+
+### Hierarchical Summarization
+- HiAgent (ACL 2025) — https://aclanthology.org/2025.acl-long.1575/
+- Focus (Jan 2026) — https://arxiv.org/abs/2601.07190
+- ReSum (Sep 2025) — https://arxiv.org/abs/2509.13313
+- U-Fold (Jan 2026) — https://arxiv.org/abs/2601.18285
+
+### Surveys
+- Self-Evolving AI Agents (Aug 2025) — https://arxiv.org/abs/2508.07407
+- Multi-Agent Collaboration Mechanisms (Jan 2025) — https://arxiv.org/abs/2501.06322
+- Process Reward Models (Oct 2025) — https://arxiv.org/abs/2510.08049