massgen 0.0.3__py3-none-any.whl

massgen/tests/__init__.py

@@ -0,0 +1,10 @@
+"""
+MassGen Tests
+
+Test suite for MassGen functionality including:
+- Basic agent functionality
+- Tool handling
+- Orchestrator coordination
+- Multi-agent scenarios
+- Frontend displays
+"""

massgen/tests/multi_turn_conversation_design.md

@@ -0,0 +1,214 @@
+# Multi-Turn Conversation Design for MassGen Orchestrator
+
+## Overview
+
+This document outlines the design approach for implementing multi-turn conversations in the MassGen orchestrator, based on the proven approach used in MassGen v0.0.1.
+
+## Current State
+
+The current orchestrator has **partial multi-turn support**:
+- ✅ Accepts conversation history through `chat(messages)` interface
+- ✅ Maintains conversation history at orchestrator level
+- ❌ **Limited**: Only processes the last user message for coordination
+- ❌ **CLI Issue**: Multi-agent mode with history bypasses coordination UI display
+
+## V0.0.1 Approach Analysis
+
+### Key Innovation: Dynamic Context Reconstruction
+
+V0.0.1 implements multi-turn conversations through **dynamic message reconstruction** rather than persistent conversation state:
+
+```python
+def work_on_task(self, task: TaskInput) -> List[Dict[str, str]]:
+    # Initialize working messages
+    working_status, working_messages, all_tools = self._get_curr_messages_and_tools(task)
+    
+    while curr_round < self.max_rounds and self.state.status == "working":
+        # Process messages...
+        
+        # When agents need to restart due to updates:
+        if renew_conversation:
+            # Rebuild conversation with latest context
+            working_status, working_messages, all_tools = self._get_curr_messages_and_tools(task)
+```
+
+### Core Principles
+
+1. **Dynamic Context Generation**: Agents don't maintain persistent conversations - they regenerate context each time they restart
+2. **Fresh State on Updates**: When other agents provide new answers, the conversation context is rebuilt with latest information
+3. **Multi-layered Context**: Conversations include:
+   - System instructions
+   - Original task/question  
+   - Current answers from all agents
+   - Voting information (when applicable)
+
+### Context Reconstruction Method
+
+```python
+def _get_curr_messages_and_tools(self, task: TaskInput):
+    """Get the current messages and tools for the agent."""
+    working_status, user_input = self._get_task_input(task)  # Includes latest agent answers
+    working_messages = self._get_task_input_messages(user_input)  # System + user messages
+    all_tools = self._get_available_tools()  # Current tool set
+    return working_status, working_messages, all_tools
+```
+
+## Proposed Implementation
+
+### 1. Orchestrator-Level Conversation Management
+
+```python
+class Orchestrator:
+    def __init__(self, ...):
+        self.conversation_history: List[Dict[str, Any]] = []  # User conversation
+        self.current_coordination_context: Optional[str] = None
+    
+    async def chat(self, messages: List[Dict[str, Any]], ...):
+        # Extract full conversation context
+        conversation_context = self._build_conversation_context(messages)
+        
+        # Start coordination with full context
+        async for chunk in self._coordinate_agents(conversation_context):
+            yield chunk
+```
+
+### 2. Context-Aware Agent Coordination
+
+```python
+async def _coordinate_agents(self, conversation_context: Dict[str, Any]):
+    """Coordinate agents with full conversation context."""
+    
+    # Build enriched task context including:
+    # - Original conversation history  
+    # - Current user message
+    # - Existing agent answers (if any)
+    
+    for agent_id in self.agents:
+        # Each agent gets full context when starting/restarting
+        agent_context = self._build_agent_context(
+            conversation_history=conversation_context['history'],
+            current_task=conversation_context['current_message'], 
+            agent_answers=self._get_current_answers(),
+            voting_state=self._get_voting_state()
+        )
+        
+        # Agent processes with full context
+        await self._stream_agent_execution(agent_id, agent_context)
+```
+
+### 3. Dynamic Context Rebuilding
+
+```python
+def _build_agent_context(self, conversation_history, current_task, agent_answers, voting_state):
+    """Build agent context dynamically based on current state."""
+    
+    # Format conversation history for agent context
+    history_context = self._format_conversation_history(conversation_history)
+    
+    # Format current coordination state
+    coordination_context = self.message_templates.build_coordination_context(
+        current_task=current_task,
+        conversation_history=history_context,
+        agent_answers=agent_answers,
+        voting_state=voting_state
+    )
+    
+    return {
+        "system_message": self.message_templates.system_message_with_context(history_context),
+        "user_message": coordination_context,
+        "tools": self.workflow_tools
+    }
+```
+
+### 4. Message Template Updates
+
+```python
+class MessageTemplates:
+    def build_coordination_context(self, current_task, conversation_history, agent_answers, voting_state):
+        """Build coordination context including conversation history."""
+        
+        context_parts = []
+        
+        # Add conversation history if present
+        if conversation_history:
+            context_parts.append(f"""
+<CONVERSATION_HISTORY>
+{self._format_conversation_for_agent(conversation_history)}
+</CONVERSATION_HISTORY>
+""")
+        
+        # Add current task
+        context_parts.append(f"""
+<CURRENT_MESSAGE>
+{current_task}
+</CURRENT_MESSAGE>
+""")
+        
+        # Add agent answers if any exist
+        if agent_answers:
+            context_parts.append(f"""
+<CURRENT_ANSWERS>
+{self._format_agent_answers(agent_answers)}
+</CURRENT_ANSWERS>
+""")
+        
+        return "\n".join(context_parts)
+```
+
+## Implementation Benefits
+
+### 1. **True Multi-Turn Support**
+- Agents understand full conversation context, not just current message
+- Natural conversation flow maintained across coordination rounds
+- Context-aware responses that reference previous exchanges
+
+### 2. **Dynamic State Management**
+- Agents always work with latest information from all sources
+- No stale conversation state issues
+- Clean restart mechanism when coordination state changes
+
+### 3. **Scalable Architecture**
+- Conversation history managed centrally at orchestrator level
+- Agents remain stateless - context provided on-demand
+- Easy to extend for different conversation patterns
+
+### 4. **Backward Compatibility**
+- Existing single-turn usage patterns continue to work
+- Gradual migration path for CLI and frontend improvements
+
+## Implementation Priority
+
+This approach addresses multiple TODO items:
+
+- **HIGH PRIORITY**: Support chat with an orchestrator (core multi-agent functionality)
+- **MEDIUM PRIORITY**: Fix CLI multi-turn conversation display in multi-agent mode
+- **MEDIUM PRIORITY**: Port missing features from v0.0.1
+
+## Next Steps
+
+1. **Phase 1**: Update message templates to support conversation context
+2. **Phase 2**: Modify orchestrator coordination to pass full context to agents  
+3. **Phase 3**: Update CLI to properly display coordination with conversation history
+4. **Phase 4**: Add conversation management utilities and testing
+
+## Technical Notes
+
+### Context Size Management
+- Monitor conversation history length to prevent token limit issues
+- Implement conversation truncation strategies for very long histories
+- Consider conversation summarization for extended sessions
+
+### Performance Considerations
+- Context rebuilding is lightweight (no persistent state management)
+- Memory usage scales with conversation length, not coordination complexity
+- Caching opportunities for repeated context elements
+
+### Testing Strategy
+- Unit tests for context building methods
+- Integration tests for multi-turn coordination scenarios
+- CLI testing with conversation history of various lengths
+- Edge case testing (empty history, very long conversations, etc.)
+
+---
+
+*This design document is based on analysis of MassGen v0.0.1's proven multi-turn approach and adapted for the current async streaming architecture.*

massgen/tests/multiturn_llm_input_analysis.md

@@ -0,0 +1,189 @@
+# Multi-Turn LLM Input Analysis - MassGen
+
+## Overview
+
+This document shows the exact input structure sent to LLMs during multi-turn conversations in MassGen, demonstrating how conversation context is built and passed to agents.
+
+## Context Building Progression
+
+### Turn 1: Initial Question (No History)
+
+**Context Size:** 568 characters total
+- System Message: 389 chars
+- User Message: 179 chars  
+- Tools: 2 (new_answer, vote)
+
+**System Message:**
+```
+You are evaluating answers from multiple agents for final response to a message. Does the best CURRENT ANSWER address the ORIGINAL MESSAGE?
+
+If YES, use the `vote` tool to record your vote and skip the `new_answer` tool.
+Otherwise, do additional work first, then use the `new_answer` tool to record a better answer to the ORIGINAL MESSAGE. Make sure you actually call one of the two tools.
+```
+
+**User Message Structure:**
+```
+<ORIGINAL MESSAGE> What are the main benefits of renewable energy? <END OF ORIGINAL MESSAGE>
+
+<CURRENT ANSWERS from the agents>
+(no answers available yet)
+<END OF CURRENT ANSWERS>
+```
+
+**Key Features:**
+- ❌ No conversation history section
+- ✅ Original message clearly marked
+- ✅ Empty current answers section
+- ❌ Standard system message (no context awareness)
+
+---
+
+### Turn 2: Follow-up with History
+
+**Context Size:** 1,152 characters total (+103% from Turn 1)
+- System Message: 574 chars (+47%)
+- User Message: 578 chars (+223%)
+- Tools: 2 (same)
+
+**System Message (Enhanced):**
+```
+You are evaluating answers from multiple agents for final response to a message. Does the best CURRENT ANSWER address the ORIGINAL MESSAGE?
+
+If YES, use the `vote` tool to record your vote and skip the `new_answer` tool.
+Otherwise, do additional work first, then use the `new_answer` tool to record a better answer to the ORIGINAL MESSAGE. Make sure you actually call one of the two tools.
+            
+IMPORTANT: You are responding to the latest message in an ongoing conversation. Consider the full conversation context when evaluating answers and providing your response.
+```
+
+**User Message Structure:**
+```
+<CONVERSATION_HISTORY>
+User: What are the main benefits of renewable energy?
+Assistant: Renewable energy offers several key benefits including environmental sustainability, economic advantages, and energy security. It reduces greenhouse gas emissions, creates jobs, and decreases dependence on fossil fuel imports.
+<END OF CONVERSATION_HISTORY>
+
+<ORIGINAL MESSAGE> What about the challenges and limitations? <END OF ORIGINAL MESSAGE>
+
+<CURRENT ANSWERS from the agents>
+<agent1> Key benefits include environmental and economic advantages. <end of agent1>
+<END OF CURRENT ANSWERS>
+```
+
+**Key Features:**
+- ✅ **Conversation history section** with previous exchange
+- ✅ Original message (current question)
+- ✅ Agent answers from coordination
+- ✅ **Context-aware system message**
+
+---
+
+### Turn 3: Extended Conversation
+
+**Context Size:** 1,252 characters total (+120% from Turn 1)
+- System Message: 574 chars (same as Turn 2)
+- User Message: 678 chars (+279% from Turn 1)
+- Tools: 2 (same)
+
+**User Message Structure:**
+```
+<CONVERSATION_HISTORY>
+User: What are the main benefits of renewable energy?
+Assistant: Renewable energy offers environmental, economic, and energy security benefits.
+User: What about the challenges and limitations?
+Assistant: Main challenges include high upfront costs, intermittency issues, and infrastructure requirements.
+<END OF CONVERSATION_HISTORY>
+
+<ORIGINAL MESSAGE> How can governments support the transition? <END OF ORIGINAL MESSAGE>
+
+<CURRENT ANSWERS from the agents>
+<agent2> Benefits include environmental and economic advantages. <end of agent2>
+<agent1> Challenges include costs, intermittency, and infrastructure needs. <end of agent1>
+<END OF CURRENT ANSWERS>
+```
+
+**Key Features:**
+- ✅ **Full conversation history** (2 previous exchanges)
+- ✅ Original message (current question)
+- ✅ **Multiple agent answers** from coordination
+- ✅ Context-aware system message
+- ✅ **Progressive context building**
+
+## Context Growth Analysis
+
+### Size Progression
+```
+Turn 1: 568 chars  (baseline)
+Turn 2: 1,152 chars (+103% growth)
+Turn 3: 1,252 chars (+120% growth)
+```
+
+### Context Elements by Turn
+| Element | Turn 1 | Turn 2 | Turn 3 |
+|---------|--------|--------|--------|
+| CONVERSATION_HISTORY | ❌ | ✅ (1 exchange) | ✅ (2 exchanges) |
+| ORIGINAL MESSAGE | ✅ | ✅ | ✅ |
+| CURRENT ANSWERS | ✅ (empty) | ✅ (1 agent) | ✅ (2 agents) |
+| Context-aware system | ❌ | ✅ | ✅ |
+
+## Key Implementation Insights
+
+### 1. **Dynamic Context Reconstruction**
+- Each turn rebuilds the complete context from scratch (v0.0.1 approach)
+- No persistent conversation state in agents
+- Context includes conversation history + current coordination state
+
+### 2. **Conversation History Format**
+```
+<CONVERSATION_HISTORY>
+User: [previous question]
+Assistant: [previous response]
+User: [another question]
+Assistant: [another response]
+<END OF CONVERSATION_HISTORY>
+```
+
+### 3. **System Message Enhancement**
+- Turn 1: Standard evaluation prompt
+- Turn 2+: Enhanced with context awareness note:
+  ```
+  IMPORTANT: You are responding to the latest message in an ongoing conversation. 
+  Consider the full conversation context when evaluating answers and providing your response.
+  ```
+
+### 4. **Multi-layered Context**
+Each agent receives:
+1. **Conversation History**: Previous user-assistant exchanges
+2. **Original Message**: Current question being coordinated
+3. **Current Answers**: Existing answers from other agents in this coordination round
+4. **Tools**: Standard MassGen workflow tools (vote, new_answer)
+
+## Benefits of This Approach
+
+### ✅ **True Multi-Turn Support**
+- Agents understand full conversation context, not just current message
+- Natural conversation flow maintained across coordination rounds
+- Context-aware responses that reference previous exchanges
+
+### ✅ **Scalable Context Management**
+- Context size grows linearly with conversation length
+- Clean separation between conversation history and coordination state
+- Memory-efficient (no persistent agent state)
+
+### ✅ **Robust State Management**
+- Each coordination round starts with fresh, complete context
+- No issues with stale or inconsistent conversation state
+- Easy to debug and understand exactly what agents receive
+
+## Testing and Validation
+
+The implementation includes comprehensive tests:
+
+1. **`test_message_context_building.py`**: Shows exact message structure without API calls
+2. **`test_multiturn_llm_input.py`**: Captures actual LLM calls during coordination with debug backend
+3. **`test_multiturn_conversation.py`**: End-to-end multi-turn conversation testing
+
+Run these tests to see the exact LLM inputs and validate the context building behavior.
+
+---
+
+*This analysis confirms that MassGen's multi-turn conversation support properly implements the proven v0.0.1 dynamic context reconstruction approach, adapted for the current async streaming architecture.*

massgen/tests/test_case_studies.md

@@ -0,0 +1,113 @@
+# MassGen Case Study Test Commands
+
+This document contains commands to test all the case studies from `docs/case_studies/` using the three agents default configuration.
+
+## Quick Commands
+
+All tests use the `three_agents_default.yaml` configuration with:
+- **Gemini 2.5 Flash** (web search enabled)
+- **GPT-4o-mini** (web search + code interpreter)  
+- **Grok 3 mini** (web search with citations)
+
+### 1. Collaborative Creative Writing
+```bash
+# From project root:
+python massgen/cli.py --config massgen/configs/three_agents_default.yaml "Write a short story about a robot who discovers music."
+
+# From tests directory:
+python ../cli.py --config ../configs/three_agents_default.yaml "Write a short story about a robot who discovers music."
+```
+**Original:** gpt-4o, gemini-2.5-flash, grok-3-mini  
+**Current:** gemini2.5flash, 4omini, grok3mini with builtin tools  
+
+### 2. AI News Synthesis
+```bash
+# From project root:
+python massgen/cli.py --config massgen/configs/three_agents_default.yaml "find big AI news this week"
+
+# From tests directory:
+python ../cli.py --config ../configs/three_agents_default.yaml "find big AI news this week"
+```
+**Original:** gpt-4.1, gemini-2.5-flash, grok-3-mini  
+**Current:** gemini2.5flash, 4omini, grok3mini with web search  
+
+### 3. Grok HLE Cost Estimation
+```bash
+# From project root:
+python massgen/cli.py --config massgen/configs/three_agents_default.yaml "How much does it cost to run HLE benchmark with Grok-4"
+
+# From tests directory:
+python ../cli.py --config ../configs/three_agents_default.yaml "How much does it cost to run HLE benchmark with Grok-4"
+```
+**Original:** gpt-4o, gemini-2.5-flash, grok-3-mini  
+**Current:** gemini2.5flash, 4omini, grok3mini with web search  
+
+### 4. IMO 2025 Winner
+```bash
+# From project root:
+python massgen/cli.py --config massgen/configs/three_agents_default.yaml "Which AI won IMO 2025?"
+
+# From tests directory:
+python ../cli.py --config ../configs/three_agents_default.yaml "Which AI won IMO 2025?"
+```
+**Original:** gemini-2.5-flash, gpt-4.1 (2 agents)  
+**Current:** gemini2.5flash, 4omini, grok3mini (3 agents with web search)  
+
+### 5. Stockholm Travel Guide
+```bash
+# From project root:
+python massgen/cli.py --config massgen/configs/three_agents_default.yaml "what's best to do in Stockholm in October 2025"
+
+# From tests directory:
+python ../cli.py --config ../configs/three_agents_default.yaml "what's best to do in Stockholm in October 2025"
+```
+**Original:** gemini-2.5-flash, gpt-4o (2 agents)  
+**Current:** gemini2.5flash, 4omini, grok3mini with web search for current info
+
+## Configuration Details
+
+The `three_agents_default.yaml` configuration provides:
+
+### Agent Capabilities
+- **gemini2.5flash**: Gemini 2.5 Flash with web search
+- **4omini**: GPT-4o-mini with web search + code interpreter  
+- **grok3mini**: Grok 3 mini with web search and citations
+
+### UI Features
+- Rich terminal display with enhanced visualization
+- Real-time coordination updates
+- Logging enabled for debugging
+
+### Custom Queries
+```bash
+# Use for any question with the three agents setup:
+python massgen/cli.py --config massgen/configs/three_agents_default.yaml "your question here"
+```
+
+## Running All Tests
+
+Use the interactive test script:
+```bash
+# From project root:
+./massgen/tests/test_case_studies.sh
+
+# From tests directory:
+./test_case_studies.sh
+```
+
+## Requirements
+
+- **OpenAI API Key:** Set `OPENAI_API_KEY` environment variable (for GPT-4o-mini)
+- **Gemini API Key:** Set `GOOGLE_API_KEY` environment variable (for Gemini 2.5 Flash)
+- **Grok API Key:** Set `XAI_API_KEY` environment variable (for Grok 3 mini)
+
+## Notes
+
+- All tests now use the unified `three_agents_default.yaml` configuration
+- Combines three different model providers for diverse perspectives
+- Built-in tools (web search, code execution) available across agents
+- Rich terminal UI provides enhanced visualization and real-time updates
+- Each agent brings unique strengths:
+  - Gemini: Advanced reasoning with web search
+  - GPT-4o-mini: Cost-effective with code execution
+  - Grok: Real-time information with citations