massgen 0.1.0a3__py3-none-any.whl → 0.1.1__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of massgen might be problematic. Click here for more details.

Files changed (111) hide show
  1. massgen/__init__.py +1 -1
  2. massgen/agent_config.py +17 -0
  3. massgen/api_params_handler/_api_params_handler_base.py +1 -0
  4. massgen/api_params_handler/_chat_completions_api_params_handler.py +8 -1
  5. massgen/api_params_handler/_claude_api_params_handler.py +8 -1
  6. massgen/api_params_handler/_gemini_api_params_handler.py +73 -0
  7. massgen/api_params_handler/_response_api_params_handler.py +8 -1
  8. massgen/backend/base.py +31 -0
  9. massgen/backend/{base_with_mcp.py → base_with_custom_tool_and_mcp.py} +282 -11
  10. massgen/backend/chat_completions.py +182 -92
  11. massgen/backend/claude.py +115 -18
  12. massgen/backend/claude_code.py +378 -14
  13. massgen/backend/docs/CLAUDE_API_RESEARCH.md +3 -3
  14. massgen/backend/gemini.py +1275 -1607
  15. massgen/backend/gemini_mcp_manager.py +545 -0
  16. massgen/backend/gemini_trackers.py +344 -0
  17. massgen/backend/gemini_utils.py +43 -0
  18. massgen/backend/response.py +129 -70
  19. massgen/cli.py +577 -110
  20. massgen/config_builder.py +376 -27
  21. massgen/configs/README.md +111 -80
  22. massgen/configs/basic/multi/three_agents_default.yaml +1 -1
  23. massgen/configs/basic/single/single_agent.yaml +1 -1
  24. massgen/configs/providers/openai/gpt5_nano.yaml +3 -3
  25. massgen/configs/tools/custom_tools/claude_code_custom_tool_example.yaml +32 -0
  26. massgen/configs/tools/custom_tools/claude_code_custom_tool_example_no_path.yaml +28 -0
  27. massgen/configs/tools/custom_tools/claude_code_custom_tool_with_mcp_example.yaml +40 -0
  28. massgen/configs/tools/custom_tools/claude_code_custom_tool_with_wrong_mcp_example.yaml +38 -0
  29. massgen/configs/tools/custom_tools/claude_code_wrong_custom_tool_with_mcp_example.yaml +38 -0
  30. massgen/configs/tools/custom_tools/claude_custom_tool_example.yaml +24 -0
  31. massgen/configs/tools/custom_tools/claude_custom_tool_example_no_path.yaml +22 -0
  32. massgen/configs/tools/custom_tools/claude_custom_tool_with_mcp_example.yaml +35 -0
  33. massgen/configs/tools/custom_tools/claude_custom_tool_with_wrong_mcp_example.yaml +33 -0
  34. massgen/configs/tools/custom_tools/claude_wrong_custom_tool_with_mcp_example.yaml +33 -0
  35. massgen/configs/tools/custom_tools/gemini_custom_tool_example.yaml +24 -0
  36. massgen/configs/tools/custom_tools/gemini_custom_tool_example_no_path.yaml +22 -0
  37. massgen/configs/tools/custom_tools/gemini_custom_tool_with_mcp_example.yaml +35 -0
  38. massgen/configs/tools/custom_tools/gemini_custom_tool_with_wrong_mcp_example.yaml +33 -0
  39. massgen/configs/tools/custom_tools/gemini_wrong_custom_tool_with_mcp_example.yaml +33 -0
  40. massgen/configs/tools/custom_tools/github_issue_market_analysis.yaml +94 -0
  41. massgen/configs/tools/custom_tools/gpt5_nano_custom_tool_example.yaml +24 -0
  42. massgen/configs/tools/custom_tools/gpt5_nano_custom_tool_example_no_path.yaml +22 -0
  43. massgen/configs/tools/custom_tools/gpt5_nano_custom_tool_with_mcp_example.yaml +35 -0
  44. massgen/configs/tools/custom_tools/gpt5_nano_custom_tool_with_wrong_mcp_example.yaml +33 -0
  45. massgen/configs/tools/custom_tools/gpt5_nano_wrong_custom_tool_with_mcp_example.yaml +33 -0
  46. massgen/configs/tools/custom_tools/gpt_oss_custom_tool_example.yaml +25 -0
  47. massgen/configs/tools/custom_tools/gpt_oss_custom_tool_example_no_path.yaml +23 -0
  48. massgen/configs/tools/custom_tools/gpt_oss_custom_tool_with_mcp_example.yaml +34 -0
  49. massgen/configs/tools/custom_tools/gpt_oss_custom_tool_with_wrong_mcp_example.yaml +34 -0
  50. massgen/configs/tools/custom_tools/gpt_oss_wrong_custom_tool_with_mcp_example.yaml +34 -0
  51. massgen/configs/tools/custom_tools/grok3_mini_custom_tool_example.yaml +24 -0
  52. massgen/configs/tools/custom_tools/grok3_mini_custom_tool_example_no_path.yaml +22 -0
  53. massgen/configs/tools/custom_tools/grok3_mini_custom_tool_with_mcp_example.yaml +35 -0
  54. massgen/configs/tools/custom_tools/grok3_mini_custom_tool_with_wrong_mcp_example.yaml +33 -0
  55. massgen/configs/tools/custom_tools/grok3_mini_wrong_custom_tool_with_mcp_example.yaml +33 -0
  56. massgen/configs/tools/custom_tools/qwen_api_custom_tool_example.yaml +25 -0
  57. massgen/configs/tools/custom_tools/qwen_api_custom_tool_example_no_path.yaml +23 -0
  58. massgen/configs/tools/custom_tools/qwen_api_custom_tool_with_mcp_example.yaml +36 -0
  59. massgen/configs/tools/custom_tools/qwen_api_custom_tool_with_wrong_mcp_example.yaml +34 -0
  60. massgen/configs/tools/custom_tools/qwen_api_wrong_custom_tool_with_mcp_example.yaml +34 -0
  61. massgen/configs/tools/custom_tools/qwen_local_custom_tool_example.yaml +24 -0
  62. massgen/configs/tools/custom_tools/qwen_local_custom_tool_example_no_path.yaml +22 -0
  63. massgen/configs/tools/custom_tools/qwen_local_custom_tool_with_mcp_example.yaml +35 -0
  64. massgen/configs/tools/custom_tools/qwen_local_custom_tool_with_wrong_mcp_example.yaml +33 -0
  65. massgen/configs/tools/custom_tools/qwen_local_wrong_custom_tool_with_mcp_example.yaml +33 -0
  66. massgen/configs/tools/filesystem/claude_code_context_sharing.yaml +1 -1
  67. massgen/configs/voting/gemini_gpt_voting_sensitivity.yaml +67 -0
  68. massgen/formatter/_chat_completions_formatter.py +104 -0
  69. massgen/formatter/_claude_formatter.py +120 -0
  70. massgen/formatter/_gemini_formatter.py +448 -0
  71. massgen/formatter/_response_formatter.py +88 -0
  72. massgen/frontend/coordination_ui.py +4 -2
  73. massgen/logger_config.py +35 -3
  74. massgen/message_templates.py +56 -6
  75. massgen/orchestrator.py +179 -10
  76. massgen/stream_chunk/base.py +3 -0
  77. massgen/tests/custom_tools_example.py +392 -0
  78. massgen/tests/mcp_test_server.py +17 -7
  79. massgen/tests/test_config_builder.py +423 -0
  80. massgen/tests/test_custom_tools.py +401 -0
  81. massgen/tests/test_tools.py +127 -0
  82. massgen/tool/README.md +935 -0
  83. massgen/tool/__init__.py +39 -0
  84. massgen/tool/_async_helpers.py +70 -0
  85. massgen/tool/_basic/__init__.py +8 -0
  86. massgen/tool/_basic/_two_num_tool.py +24 -0
  87. massgen/tool/_code_executors/__init__.py +10 -0
  88. massgen/tool/_code_executors/_python_executor.py +74 -0
  89. massgen/tool/_code_executors/_shell_executor.py +61 -0
  90. massgen/tool/_exceptions.py +39 -0
  91. massgen/tool/_file_handlers/__init__.py +10 -0
  92. massgen/tool/_file_handlers/_file_operations.py +218 -0
  93. massgen/tool/_manager.py +634 -0
  94. massgen/tool/_registered_tool.py +88 -0
  95. massgen/tool/_result.py +66 -0
  96. massgen/tool/_self_evolution/_github_issue_analyzer.py +369 -0
  97. massgen/tool/docs/builtin_tools.md +681 -0
  98. massgen/tool/docs/exceptions.md +794 -0
  99. massgen/tool/docs/execution_results.md +691 -0
  100. massgen/tool/docs/manager.md +887 -0
  101. massgen/tool/docs/workflow_toolkits.md +529 -0
  102. massgen/tool/workflow_toolkits/__init__.py +57 -0
  103. massgen/tool/workflow_toolkits/base.py +55 -0
  104. massgen/tool/workflow_toolkits/new_answer.py +126 -0
  105. massgen/tool/workflow_toolkits/vote.py +167 -0
  106. {massgen-0.1.0a3.dist-info → massgen-0.1.1.dist-info}/METADATA +89 -131
  107. {massgen-0.1.0a3.dist-info → massgen-0.1.1.dist-info}/RECORD +111 -36
  108. {massgen-0.1.0a3.dist-info → massgen-0.1.1.dist-info}/WHEEL +0 -0
  109. {massgen-0.1.0a3.dist-info → massgen-0.1.1.dist-info}/entry_points.txt +0 -0
  110. {massgen-0.1.0a3.dist-info → massgen-0.1.1.dist-info}/licenses/LICENSE +0 -0
  111. {massgen-0.1.0a3.dist-info → massgen-0.1.1.dist-info}/top_level.txt +0 -0
massgen/tool/README.md ADDED
@@ -0,0 +1,935 @@
1
+ # MassGen Tool System
2
+
3
+ ## Overview
4
+
5
+ **What is the Tool System?**
6
+ The MassGen Tool System is a flexible, extensible framework for managing and executing tools that AI agents can use during task solving. Think of it as a plugin system that allows agents to interact with code execution, file operations, and custom workflows in a standardized, type-safe way.
7
+
8
+ **What does this module do?**
9
+ The Tool System provides a unified interface for:
10
+
11
+ - Registering tools (custom functions)
12
+ - Managing tool categories
13
+ - Generating JSON schemas from Python functions automatically
14
+ - Executing tools asynchronously with streaming support
15
+ - Extending tool schemas dynamically with Pydantic models
16
+ - Handling multimodal outputs (text, images, audio)
17
+
18
+ **Why do you need it?**
19
+ Instead of hardcoding capabilities into each AI agent, the Tool System enables:
20
+
21
+ - **Type Safety**: Automatic schema generation from function signatures
22
+ - **Extensibility**: Easy addition of custom tools without modifying core code
23
+ - **Categorization**: Organize tools by purpose and enable/disable groups
24
+ - **Streaming Results**: Support for long-running operations with progress updates
25
+ - **Multimodal Support**: Handle text, images, and audio outputs seamlessly
26
+
27
+ This module makes it easy to give AI agents new capabilities while maintaining consistency and safety.
28
+
29
+ ## Architecture
30
+
31
+ ```mermaid
32
+ graph TB
33
+ subgraph Tool Registration
34
+ A[Python Function] -->|Introspection| B[Schema Extractor]
35
+ B -->|JSON Schema| C[RegisteredToolEntry]
36
+ D[Category Config] --> E[ToolCategory]
37
+ end
38
+
39
+ subgraph Tool Management
40
+ F[ToolManager] -->|Stores| C
41
+ F -->|Organizes| E
42
+ F -->|Generates| G[Tool Schemas]
43
+ end
44
+
45
+ subgraph Tool Execution
46
+ H[Tool Request] --> F
47
+ F -->|Routes| I[Base Function]
48
+ I -->|Returns| J[ExecutionResult]
49
+ J -->|Wraps| K[Async Generator]
50
+ K -->|Streams| L[Agent]
51
+ end
52
+
53
+ subgraph Extension System
54
+ M[Pydantic Model] -->|Extends| C
55
+ C -->|Merges| G
56
+ end
57
+
58
+ classDef registration fill:#e1f5fe,stroke:#0288d1,stroke-width:2px
59
+ classDef management fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
60
+ classDef execution fill:#e8f5e9,stroke:#388e3c,stroke-width:2px
61
+ classDef extension fill:#fff3e0,stroke:#f57c00,stroke-width:2px
62
+
63
+ class A,B,C,D,E registration
64
+ class F,G management
65
+ class H,I,J,K,L execution
66
+ class M extension
67
+ ```
68
+
69
+ ## Features
70
+
71
+ ### What You Can Do
72
+
73
+ - **Register Any Function**: Turn any Python function into an agent tool automatically
74
+ - **Schema Generation**: Automatic JSON schema from docstrings and type hints
75
+ - **Category Management**: Group tools into categories
76
+ - **Preset Arguments**: Hide complex configuration from agents while controlling behavior
77
+ - **Schema Extension**: Dynamically add new parameters using Pydantic models
78
+ - **Streaming Support**: Handle long-running operations with progress updates
79
+ - **Multimodal Output**: Return text, images, or audio content
80
+ - **Post-Processing**: Transform tool results before returning to agents
81
+
82
+ ### Tool Registration Methods
83
+
84
+ - **Function Registration**: Direct function objects or callables
85
+ - **Path-Based Loading**: Load from Python files or module names
86
+ - **Built-in Tools**: Pre-configured tools for common tasks
87
+ - **Custom Tools**: User-defined functions with automatic schema extraction
88
+
89
+ ## Quick Start
90
+
91
+ ### Basic Usage
92
+
93
+ #### 1. Create and Register a Simple Tool
94
+
95
+ ```python
96
+ from massgen.tool import ToolManager, ExecutionResult, TextContent
97
+
98
+ # Initialize the tool manager
99
+ manager = ToolManager()
100
+
101
+ # Define a simple tool function
102
+ async def calculate_sum(a: int, b: int) -> ExecutionResult:
103
+ """Add two numbers together.
104
+
105
+ Args:
106
+ a: First number
107
+ b: Second number
108
+
109
+ Returns:
110
+ ExecutionResult containing the sum
111
+ """
112
+ result = a + b
113
+ return ExecutionResult(
114
+ output_blocks=[TextContent(data=f"The sum is: {result}")]
115
+ )
116
+
117
+ # Register the tool
118
+ manager.add_tool_function(func=calculate_sum)
119
+
120
+ # Get schemas for all tools
121
+ schemas = manager.fetch_tool_schemas()
122
+ print(f"Registered tools: {[s['function']['name'] for s in schemas]}")
123
+ ```
124
+
125
+ #### 2. Execute a Tool
126
+
127
+ ```python
128
+ async def execute_example():
129
+ # Prepare tool request
130
+ tool_request = {
131
+ "name": "custom_tool__calculate_sum", # Prefixed with custom_tool__
132
+ "input": {"a": 5, "b": 3}
133
+ }
134
+
135
+ # Execute and get results
136
+ async for result in manager.execute_tool(tool_request):
137
+ print(result.output_blocks[0].data)
138
+ # Output: "The sum is: 8"
139
+ ```
140
+
141
+ #### 3. Use Tool Categories
142
+
143
+ ```python
144
+ # Create a category for math tools
145
+ manager.setup_category(
146
+ category_name="math",
147
+ description="Mathematical operations",
148
+ enabled=True, # Enable by default
149
+ usage_hints="Use these tools for calculations and numeric analysis"
150
+ )
151
+
152
+ # Register tool in category
153
+ manager.add_tool_function(
154
+ func=calculate_sum,
155
+ category="math"
156
+ )
157
+
158
+ # Disable all math tools at once
159
+ manager.modify_categories(["math"], enabled=False)
160
+
161
+ # Get schemas (math tools won't be included)
162
+ schemas = manager.fetch_tool_schemas()
163
+ ```
164
+
165
+ ## Core Components
166
+
167
+ ### ToolManager
168
+
169
+ The central class for tool registration and execution.
170
+
171
+ **Key Methods:**
172
+
173
+ - `add_tool_function()`: Register a tool
174
+ - `delete_tool_function()`: Remove a tool
175
+ - `setup_category()`: Create a tool category
176
+ - `modify_categories()`: Enable/disable categories
177
+ - `fetch_tool_schemas()`: Get JSON schemas for active tools
178
+ - `execute_tool()`: Execute a tool and stream results
179
+ - `apply_extension_model()`: Add schema extensions
180
+
181
+ **Example:**
182
+
183
+ ```python
184
+ manager = ToolManager()
185
+
186
+ # Register a tool from a file
187
+ manager.add_tool_function(
188
+ path="massgen/tool/_code_executors/_python_executor.py",
189
+ func="run_python_script",
190
+ category="code_execution",
191
+ description="Execute Python code in a sandbox"
192
+ )
193
+
194
+ # Register with preset arguments
195
+ manager.add_tool_function(
196
+ func=my_api_call,
197
+ preset_args={"api_key": "secret", "timeout": 30} # Hidden from schema
198
+ )
199
+ ```
200
+
201
+ ### RegisteredToolEntry
202
+
203
+ Data model for registered tools containing:
204
+
205
+ - `tool_name`: Unique identifier
206
+ - `category`: Tool category
207
+ - `base_function`: The callable function
208
+ - `schema_def`: JSON schema
209
+ - `preset_params`: Hidden preset arguments
210
+ - `extension_model`: Optional Pydantic extension
211
+ - `post_processor`: Optional result transformer
212
+
213
+ ### ExecutionResult
214
+
215
+ Container for tool execution outputs:
216
+
217
+ ```python
218
+ from massgen.tool import ExecutionResult, TextContent, ImageContent
219
+
220
+ # Simple text result
221
+ result = ExecutionResult(
222
+ output_blocks=[TextContent(data="Operation completed")]
223
+ )
224
+
225
+ # Multimodal result with image
226
+ result = ExecutionResult(
227
+ output_blocks=[
228
+ TextContent(data="Generated image:"),
229
+ ImageContent(data="base64_encoded_image_data")
230
+ ],
231
+ meta_info={"model": "dalle-3", "size": "1024x1024"}
232
+ )
233
+
234
+ # Streaming result
235
+ result = ExecutionResult(
236
+ output_blocks=[TextContent(data="Processing...")],
237
+ is_streaming=True,
238
+ is_final=False
239
+ )
240
+ ```
241
+
242
+ ## Built-in Tools
243
+
244
+ ### Code Executors
245
+
246
+ #### run_python_script
247
+
248
+ Execute Python code in an isolated subprocess.
249
+
250
+ ```python
251
+ from massgen.tool import run_python_script
252
+
253
+ result = await run_python_script(
254
+ source_code="""
255
+ print("Hello from Python!")
256
+ result = 2 + 2
257
+ print(f"Result: {result}")
258
+ """,
259
+ max_duration=60.0 # Timeout in seconds
260
+ )
261
+ ```
262
+
263
+ #### run_shell_script
264
+
265
+ Execute shell commands safely.
266
+
267
+ ```python
268
+ from massgen.tool import run_shell_script
269
+
270
+ result = await run_shell_script(
271
+ command="ls -la /tmp",
272
+ max_duration=30.0
273
+ )
274
+ ```
275
+
276
+ ### File Handlers
277
+
278
+ #### read_file_content
279
+
280
+ Read files with optional line range.
281
+
282
+ ```python
283
+ from massgen.tool import read_file_content
284
+
285
+ # Read entire file
286
+ result = await read_file_content(
287
+ target_path="/path/to/file.txt"
288
+ )
289
+
290
+ # Read specific line range
291
+ result = await read_file_content(
292
+ target_path="/path/to/file.txt",
293
+ line_range=[10, 20] # Lines 10-20
294
+ )
295
+
296
+ # Read last 100 lines
297
+ result = await read_file_content(
298
+ target_path="/path/to/file.txt",
299
+ line_range=[-100, -1] # Negative indices from end
300
+ )
301
+ ```
302
+
303
+ #### save_file_content
304
+
305
+ Write content to files.
306
+
307
+ ```python
308
+ from massgen.tool import save_file_content
309
+
310
+ result = await save_file_content(
311
+ target_path="/path/to/output.txt",
312
+ file_content="Hello, World!",
313
+ create_dirs=True # Create parent directories
314
+ )
315
+ ```
316
+
317
+ #### append_file_content
318
+
319
+ Append or insert content into files.
320
+
321
+ ```python
322
+ from massgen.tool import append_file_content
323
+
324
+ # Append to end
325
+ result = await append_file_content(
326
+ target_path="/path/to/log.txt",
327
+ additional_content="New log entry\n"
328
+ )
329
+
330
+ # Insert at specific line
331
+ result = await append_file_content(
332
+ target_path="/path/to/file.txt",
333
+ additional_content="Inserted line\n",
334
+ line_position=5 # Insert at line 5
335
+ )
336
+ ```
337
+
338
+ ### Workflow Toolkits
339
+
340
+ #### NewAnswerToolkit
341
+
342
+ Allows agents to submit new answers during coordination.
343
+
344
+ ```python
345
+ from massgen.tool.workflow_toolkits import NewAnswerToolkit
346
+
347
+ toolkit = NewAnswerToolkit()
348
+ tools = toolkit.get_tools(config={
349
+ "api_format": "chat_completions",
350
+ "enable_workflow_tools": True
351
+ })
352
+ ```
353
+
354
+ #### VoteToolkit
355
+
356
+ Enables agents to vote for other agents' answers.
357
+
358
+ ```python
359
+ from massgen.tool.workflow_toolkits import VoteToolkit
360
+
361
+ toolkit = VoteToolkit(valid_agent_ids=["agent1", "agent2", "agent3"])
362
+ tools = toolkit.get_tools(config={
363
+ "api_format": "chat_completions",
364
+ "enable_workflow_tools": True
365
+ })
366
+ ```
367
+
368
+ ## Advanced Features
369
+
370
+ ### Schema Extension
371
+
372
+ Dynamically add parameters to tool schemas using Pydantic models.
373
+
374
+ ```python
375
+ from pydantic import BaseModel, Field
376
+ from massgen.tool import ToolManager
377
+
378
+ class AdvancedParams(BaseModel):
379
+ """Additional parameters for advanced users."""
380
+ debug: bool = Field(default=False, description="Enable debug output")
381
+ retry_count: int = Field(default=3, description="Number of retries")
382
+
383
+ manager = ToolManager()
384
+ manager.add_tool_function(func=my_tool)
385
+
386
+ # Extend the schema
387
+ manager.apply_extension_model(
388
+ tool_name="custom_tool__my_tool",
389
+ model_class=AdvancedParams
390
+ )
391
+
392
+ # Schema now includes debug and retry_count parameters
393
+ ```
394
+
395
+ ### Preset Arguments
396
+
397
+ Hide configuration from agents while controlling behavior.
398
+
399
+ ```python
400
+ # Register with preset arguments
401
+ manager.add_tool_function(
402
+ func=api_call_function,
403
+ preset_args={
404
+ "api_key": os.getenv("API_KEY"),
405
+ "base_url": "https://api.example.com",
406
+ "timeout": 30
407
+ }
408
+ )
409
+
410
+ # Agents only see and provide user-facing parameters
411
+ # api_key, base_url, timeout are automatically added during execution
412
+ ```
413
+
414
+ ### Post-Processing
415
+
416
+ Transform tool results before returning to agents.
417
+
418
+ ```python
419
+ def cleanup_output(request: dict, result: ExecutionResult) -> ExecutionResult:
420
+ """Remove sensitive information from output."""
421
+ for block in result.output_blocks:
422
+ if isinstance(block, TextContent):
423
+ # Redact API keys, tokens, etc.
424
+ block.data = re.sub(r'api_key=\w+', 'api_key=***', block.data)
425
+ return result
426
+
427
+ manager.add_tool_function(
428
+ func=my_api_tool,
429
+ post_processor=cleanup_output
430
+ )
431
+ ```
432
+
433
+ ### Streaming Results
434
+
435
+ Support long-running operations with progress updates.
436
+
437
+ ```python
438
+ from typing import AsyncGenerator
439
+ from massgen.tool import ExecutionResult, TextContent
440
+
441
+ async def long_running_task() -> AsyncGenerator[ExecutionResult, None]:
442
+ """Tool that streams progress updates."""
443
+
444
+ # Initial status
445
+ yield ExecutionResult(
446
+ output_blocks=[TextContent(data="Starting task...")],
447
+ is_streaming=True,
448
+ is_final=False
449
+ )
450
+
451
+ # Progress updates
452
+ for i in range(1, 6):
453
+ await asyncio.sleep(1)
454
+ yield ExecutionResult(
455
+ output_blocks=[TextContent(data=f"Progress: {i*20}%")],
456
+ is_streaming=True,
457
+ is_final=False
458
+ )
459
+
460
+ # Final result
461
+ yield ExecutionResult(
462
+ output_blocks=[TextContent(data="Task completed!")],
463
+ is_streaming=True,
464
+ is_final=True
465
+ )
466
+
467
+ # Register and execute
468
+ manager.add_tool_function(func=long_running_task)
469
+
470
+ async for result in manager.execute_tool({"name": "custom_tool__long_running_task", "input": {}}):
471
+ print(result.output_blocks[0].data)
472
+ ```
473
+
474
+ ### Tool Loading Strategies
475
+
476
+ #### Load from Built-in Tools
477
+
478
+ ```python
479
+ # Load by function name (auto-discovered)
480
+ manager.add_tool_function(func="run_python_script")
481
+ ```
482
+
483
+ #### Load from File Path
484
+
485
+ ```python
486
+ # Load from specific file
487
+ manager.add_tool_function(
488
+ path="my_tools/data_processor.py",
489
+ func="process_data"
490
+ )
491
+
492
+ # Load main/first function from file
493
+ manager.add_tool_function(
494
+ path="my_tools/analyzer.py" # Loads 'main' or first public function
495
+ )
496
+ ```
497
+
498
+ #### Load from Module
499
+
500
+ ```python
501
+ # Load from module name
502
+ manager.add_tool_function(
503
+ path="massgen.tool._code_executors",
504
+ func="run_python_script"
505
+ )
506
+ ```
507
+
508
+ ## Configuration Integration
509
+
510
+ ### Usage with MassGen Agents
511
+
512
+ The tool system integrates seamlessly with MassGen's agent backends:
513
+
514
+ ```yaml
515
+ # In agent configuration
516
+ agents:
517
+ - id: "coder_agent"
518
+ backend:
519
+ type: "claude_code"
520
+ model: "claude-sonnet-4"
521
+
522
+ # Built-in tools are auto-configured
523
+ # Custom tools can be added via ToolManager
524
+
525
+ allowed_tools:
526
+ - "custom_tool__data_analyzer"
527
+ - "custom_tool__file_processor"
528
+
529
+ exclude_tools:
530
+ - "custom_tool__run_shell_script" # Disable specific tools
531
+ ```
532
+
533
+ ### Custom Tool Registration
534
+
535
+ ```python
536
+ from massgen.tool import ToolManager
537
+
538
+ def setup_custom_tools(manager: ToolManager):
539
+ """Set up custom tools for an agent."""
540
+
541
+ # Create category
542
+ manager.setup_category(
543
+ category_name="data_processing",
544
+ description="Tools for data analysis and transformation",
545
+ enabled=True
546
+ )
547
+
548
+ # Register custom tools
549
+ manager.add_tool_function(
550
+ path="my_tools/analyzer.py",
551
+ func="analyze_dataset",
552
+ category="data_processing",
553
+ preset_args={"max_rows": 10000}
554
+ )
555
+
556
+ manager.add_tool_function(
557
+ path="my_tools/visualizer.py",
558
+ func="create_chart",
559
+ category="data_processing"
560
+ )
561
+
562
+ return manager
563
+ ```
564
+
565
+ ## Tool Schema Format
566
+
567
+ Tools are exposed to AI agents using JSON Schema format compatible with function calling APIs:
568
+
569
+ ```json
570
+ {
571
+ "type": "function",
572
+ "function": {
573
+ "name": "custom_tool__calculate_statistics",
574
+ "description": "Calculate statistical measures for a dataset.\n\nComputes mean, median, standard deviation, and other metrics.",
575
+ "parameters": {
576
+ "type": "object",
577
+ "properties": {
578
+ "data": {
579
+ "type": "array",
580
+ "items": {"type": "number"},
581
+ "description": "List of numeric values to analyze"
582
+ },
583
+ "metrics": {
584
+ "type": "array",
585
+ "items": {"type": "string"},
586
+ "description": "Statistical metrics to compute (mean, median, std, etc.)"
587
+ },
588
+ "round_to": {
589
+ "type": "integer",
590
+ "description": "Number of decimal places for results",
591
+ "default": 2
592
+ }
593
+ },
594
+ "required": ["data"]
595
+ }
596
+ }
597
+ }
598
+ ```
599
+
600
+ ### Schema Generation
601
+
602
+ Schemas are automatically generated from:
603
+
604
+ - **Function signature**: Parameter names and types
605
+ - **Type hints**: Python type annotations
606
+ - **Docstrings**: Google-style docstrings for descriptions
607
+ - **Default values**: Optional parameters with defaults
608
+ - **Pydantic models**: Type-safe data structures
609
+
610
+ Example:
611
+
612
+ ```python
613
+ from typing import List, Optional
614
+
615
+ async def analyze_text(
616
+ text: str,
617
+ language: str = "english",
618
+ metrics: Optional[List[str]] = None
619
+ ) -> ExecutionResult:
620
+ """Analyze text and extract linguistic metrics.
621
+
622
+ Performs natural language analysis including sentiment,
623
+ readability, and keyword extraction.
624
+
625
+ Args:
626
+ text: The text content to analyze
627
+ language: Language code for analysis (default: english)
628
+ metrics: Specific metrics to compute (optional)
629
+
630
+ Returns:
631
+ ExecutionResult with analysis results
632
+ """
633
+ # Implementation
634
+ ...
635
+
636
+ # Schema automatically includes:
637
+ # - text: required string parameter
638
+ # - language: optional string with default "english"
639
+ # - metrics: optional array of strings
640
+ # - Description from docstring
641
+ ```
642
+
643
+ ## Error Handling
644
+
645
+ ### Exception Types
646
+
647
+ ```python
648
+ from massgen.tool._exceptions import (
649
+ ToolException, # Base exception
650
+ InvalidToolArgumentsException, # Invalid arguments
651
+ ToolNotFoundException, # Tool not found
652
+ ToolExecutionException, # Execution failed
653
+ CategoryNotFoundException # Category not found
654
+ )
655
+
656
+ # Usage
657
+ try:
658
+ await manager.execute_tool(tool_request)
659
+ except ToolNotFoundException as e:
660
+ print(f"Tool not found: {e.tool_name}")
661
+ except ToolExecutionException as e:
662
+ print(f"Execution failed: {e.error_details}")
663
+ except InvalidToolArgumentsException as e:
664
+ print(f"Invalid arguments: {e.error_msg}")
665
+ ```
666
+
667
+ ### Execution Error Handling
668
+
669
+ Tool execution errors are returned as ExecutionResult:
670
+
671
+ ```python
672
+ async for result in manager.execute_tool({"name": "unknown_tool", "input": {}}):
673
+ # Returns error as result, doesn't raise exception
674
+ print(result.output_blocks[0].data)
675
+ # Output: "ToolNotFound: No tool named 'unknown_tool' exists"
676
+ ```
677
+
678
+ ### Best Practices
679
+
680
+ ```python
681
+ async def safe_tool_execution():
682
+ """Safe pattern for tool execution."""
683
+
684
+ # 1. Check if tool exists
685
+ schemas = manager.fetch_tool_schemas()
686
+ available_tools = [s['function']['name'] for s in schemas]
687
+
688
+ if "custom_tool__my_tool" not in available_tools:
689
+ print("Tool not available")
690
+ return
691
+
692
+ # 2. Validate input before execution
693
+ tool_request = {
694
+ "name": "custom_tool__my_tool",
695
+ "input": validate_input(user_input)
696
+ }
697
+
698
+ # 3. Handle execution results
699
+ try:
700
+ async for result in manager.execute_tool(tool_request):
701
+ if result.was_interrupted:
702
+ print("Tool was cancelled")
703
+ break
704
+
705
+ if result.is_final:
706
+ # Process final result
707
+ process_result(result)
708
+ except Exception as e:
709
+ print(f"Unexpected error: {e}")
710
+ ```
711
+
712
+ ## Documentation
713
+
714
+ For detailed information on specific components:
715
+
716
+ - **[Tool Manager](docs/manager.md)**: Complete ToolManager API reference
717
+ - **[Execution Results](docs/execution_results.md)**: ExecutionResult and content types
718
+ - **[Built-in Tools](docs/builtin_tools.md)**: Detailed guide to built-in tools
719
+ - **[Workflow Toolkits](docs/workflow_toolkits.md)**: NewAnswer and Vote toolkits
720
+ - **[Exceptions](docs/exceptions.md)**: Exception classes and handling
721
+
722
+ ## Troubleshooting
723
+
724
+ ### Common Issues
725
+
726
+ **Tool Not Found**
727
+
728
+ ```
729
+ ToolNotFound: No tool named 'my_tool' exists
730
+ ```
731
+
732
+ - Verify tool was registered with correct name
733
+ - Check if category is enabled
734
+ - Custom tools are prefixed with `custom_tool__`
735
+
736
+ **Schema Generation Fails**
737
+
738
+ ```
739
+ TypeError: cannot create schema for function with **kwargs
740
+ ```
741
+
742
+ - Set `allow_var_kwargs=True` when registering
743
+ - Or remove `**kwargs` from function signature
744
+
745
+ **Execution Timeout**
746
+
747
+ ```python
748
+ # For code execution tools, adjust timeout
749
+ result = await run_python_script(
750
+ source_code=code,
751
+ max_duration=300 # 5 minutes
752
+ )
753
+ ```
754
+
755
+ **Category Conflicts**
756
+
757
+ ```
758
+ ValueError: Category 'tools' already exists or is reserved
759
+ ```
760
+
761
+ - Use unique category names
762
+ - Avoid reserved name "default"
763
+
764
+ ### Debug Mode
765
+
766
+ Enable detailed logging:
767
+
768
+ ```python
769
+ import logging
770
+
771
+ # Enable debug logging for tool system
772
+ logging.getLogger('massgen.tool').setLevel(logging.DEBUG)
773
+
774
+ # See tool registration and execution details
775
+ manager.add_tool_function(func=my_tool)
776
+ ```
777
+
778
+ ## Examples
779
+
780
+ ### Example 1: Data Processing Tool
781
+
782
+ ```python
783
+ from typing import List, Dict
784
+ from massgen.tool import ToolManager, ExecutionResult, TextContent
785
+ import json
786
+
787
+ async def process_json_data(
788
+ data: List[Dict],
789
+ filter_key: str,
790
+ filter_value: str
791
+ ) -> ExecutionResult:
792
+ """Filter JSON data by key-value pair.
793
+
794
+ Args:
795
+ data: List of dictionaries to filter
796
+ filter_key: Key to filter on
797
+ filter_value: Value to match
798
+
799
+ Returns:
800
+ ExecutionResult with filtered data
801
+ """
802
+ filtered = [item for item in data if item.get(filter_key) == filter_value]
803
+
804
+ return ExecutionResult(
805
+ output_blocks=[
806
+ TextContent(data=f"Found {len(filtered)} matching items:\n{json.dumps(filtered, indent=2)}")
807
+ ],
808
+ meta_info={"original_count": len(data), "filtered_count": len(filtered)}
809
+ )
810
+
811
+ # Setup
812
+ manager = ToolManager()
813
+ manager.setup_category("data", "Data processing tools", enabled=True)
814
+ manager.add_tool_function(func=process_json_data, category="data")
815
+ ```
816
+
817
+ ### Example 2: Multi-Category System
818
+
819
+ ```python
820
+ # Create specialized categories
821
+ categories = {
822
+ "analysis": ("Data analysis and statistics", True),
823
+ "visualization": ("Chart and graph generation", True),
824
+ "export": ("Data export and formatting", False) # Disabled by default
825
+ }
826
+
827
+ for name, (desc, enabled) in categories.items():
828
+ manager.setup_category(name, desc, enabled=enabled)
829
+
830
+ # Register tools in categories
831
+ manager.add_tool_function(func=calculate_stats, category="analysis")
832
+ manager.add_tool_function(func=create_chart, category="visualization")
833
+ manager.add_tool_function(func=export_csv, category="export")
834
+
835
+ # Enable export tools when needed
836
+ manager.modify_categories(["export"], enabled=True)
837
+ ```
838
+
839
+ ### Example 3: Streaming Progress Tool
840
+
841
+ ```python
842
+ from typing import AsyncGenerator
843
+ import asyncio
844
+
845
+ async def batch_processor(
846
+ items: List[str],
847
+ operation: str
848
+ ) -> AsyncGenerator[ExecutionResult, None]:
849
+ """Process items in batches with progress updates.
850
+
851
+ Args:
852
+ items: List of items to process
853
+ operation: Operation to perform on each item
854
+ """
855
+ total = len(items)
856
+
857
+ for i, item in enumerate(items, 1):
858
+ # Simulate processing
859
+ await asyncio.sleep(0.5)
860
+
861
+ # Yield progress
862
+ progress = int((i / total) * 100)
863
+ yield ExecutionResult(
864
+ output_blocks=[
865
+ TextContent(data=f"Processing {i}/{total} ({progress}%): {item}")
866
+ ],
867
+ is_streaming=True,
868
+ is_final=(i == total)
869
+ )
870
+
871
+ # Execute with streaming
872
+ async for result in manager.execute_tool({
873
+ "name": "custom_tool__batch_processor",
874
+ "input": {"items": ["a", "b", "c"], "operation": "analyze"}
875
+ }):
876
+ print(result.output_blocks[0].data)
877
+ ```
878
+
879
+ ## Best Practices
880
+
881
+ ### Tool Design
882
+
883
+ 1. **Single Responsibility**: Each tool should do one thing well
884
+ 2. **Clear Documentation**: Use detailed docstrings with parameter descriptions
885
+ 3. **Type Hints**: Always include type annotations for automatic schema generation
886
+ 4. **Error Handling**: Return errors as ExecutionResult, don't raise exceptions
887
+ 5. **Async First**: Use async functions for I/O-bound operations
888
+
889
+ ### Security
890
+
891
+ 1. **Input Validation**: Validate all tool arguments
892
+ 2. **Sandboxing**: Use code executors for untrusted code
893
+ 3. **Preset Arguments**: Hide sensitive config (API keys, credentials)
894
+ 4. **Timeouts**: Set reasonable timeouts for all operations
895
+ 5. **Resource Limits**: Prevent resource exhaustion (memory, CPU)
896
+
897
+ ### Performance
898
+
899
+ 1. **Lazy Loading**: Load tools only when needed
900
+ 2. **Streaming**: Use generators for large outputs
901
+ 3. **Caching**: Cache expensive computations
902
+ 4. **Async Execution**: Don't block the event loop
903
+ 5. **Clean Resources**: Always clean up in finally blocks
904
+
905
+ ### Integration
906
+
907
+ 1. **Consistent Naming**: Use clear, descriptive tool names
908
+ 2. **Category Organization**: Group related tools
909
+ 3. **Schema Clarity**: Provide clear parameter descriptions
910
+ 4. **Version Compatibility**: Document required dependencies
911
+ 5. **Testing**: Test tools independently before registration
912
+
913
+ ## Integration with MassGen
914
+
915
+ The Tool System is deeply integrated with MassGen's multi-agent framework:
916
+
917
+ ```python
918
+ from massgen.orchestrator import Orchestrator
919
+ from massgen.tool import ToolManager
920
+
921
+ # Tools are automatically available to agents
922
+ orchestrator = Orchestrator(config_path="config.yaml")
923
+
924
+ # Agents discover and use tools during task execution
925
+ result = await orchestrator.run_task("Analyze the dataset and create visualizations")
926
+
927
+ # Tool execution is logged and tracked
928
+ # Results are shared between agents during coordination
929
+ ```
930
+
931
+ This integration allows agents to dynamically discover and use tools while the orchestrator manages execution, logging, and result coordination.
932
+
933
+ ---
934
+
935
+ **Made with the MassGen Tool System**