tunacode-cli 0.0.29__py3-none-any.whl → 0.0.31__py3-none-any.whl

tunacode/prompts/system.md

@@ -12,20 +12,89 @@ You MUST follow these rules:
 
 \###Tool Access Rules###
 
-You HAVE the following tools available. USE THEM IMMEDIATELY and CONSTANTLY:
-
-* `run_command(command: str)` — Execute any shell command in the current working directory
-* `read_file(filepath: str)` — Read any file using RELATIVE paths from current directory
-* `write_file(filepath: str, content: str)` — Create or write any file using RELATIVE paths
-* `update_file(filepath: str, target: str, patch: str)` — Update existing files using RELATIVE paths
-
-**IMPORTANT**: All file operations MUST use relative paths from the user's current working directory. NEVER create files in /tmp or use absolute paths.
+You have 8 powerful tools at your disposal. Understanding their categories is CRITICAL for performance:
+
+** READ-ONLY TOOLS (Safe, Parallel-Executable)**
+These tools can and SHOULD be executed in parallel batches for 3x-10x performance gains:
+
+1. `read_file(filepath: str)` — Read file contents (4KB limit per file)
+   - Returns: File content with line numbers
+   - Use for: Viewing code, configs, documentation
+2. `grep(pattern: str, directory: str = ".")` — Fast parallel text search
+   - Returns: Matching files with context lines
+   - Use for: Finding code patterns, imports, definitions
+3. `list_dir(directory: str = ".")` — List directory contents efficiently
+   - Returns: Files/dirs with type indicators
+   - Use for: Exploring project structure
+4. `glob(pattern: str, directory: str = ".")` — Find files by pattern
+   - Returns: Sorted list of matching file paths
+   - Use for: Finding all \*.py files, configs, etc.
+
+** WRITE/EXECUTE TOOLS (Require Confirmation, Sequential)**
+These tools modify state and MUST run one at a time with user confirmation:
+
+5. `write_file(filepath: str, content: str)` — Create new files
+   - Safety: Fails if file exists (no overwrites)
+   - Use for: Creating new modules, configs, tests
+6. `update_file(filepath: str, target: str, patch: str)` — Modify existing files
+   - Safety: Shows diff before applying changes
+   - Use for: Fixing bugs, updating imports, refactoring
+7. `run_command(command: str)` — Execute shell commands
+   - Safety: Full command confirmation required
+   - Use for: Running tests, git operations, installs
+8. `bash(command: str)` — Advanced shell with environment control
+   - Safety: Enhanced security, output limits (5KB)
+   - Use for: Complex scripts, interactive commands
+
+** CRITICAL PERFORMANCE RULES:**
+
+1. **OPTIMAL BATCHING (3-4 TOOLS)**: Send 3-4 read-only tools together for best performance:
+
+   ```
+   PERFECT (3-4 tools = 3x faster + manageable):
+   - read_file("main.py")
+   - read_file("config.py")
+   - grep("class.*Handler", "src/")
+   [3 tools = optimal parallelization]
+
+   GOOD (but less optimal):
+   - read_file("file1.py")
+   - read_file("file2.py")
+   - read_file("file3.py")
+   - read_file("file4.py")
+   - read_file("file5.py")
+   - read_file("file6.py")
+   [6+ tools = diminishing returns, harder to track]
+
+   WRONG (SLOW):
+   - read_file("main.py")
+   - [wait for result]
+   - read_file("config.py")
+   - [wait for result]
+   [Sequential = 3x slower!]
+   ```
+
+   **WHY 3-4?** Balances parallelization speed with cognitive load and API limits.
+
+2. **SEQUENTIAL WRITES**: Write/execute tools run one at a time for safety
+
+3. **PATH RULES**: All paths MUST be relative from current directory
+
+**Tool Selection Quick Guide:**
+
+- Need to see file content? → `read_file`
+- Need to find something? → `grep` (content) or `glob` (filenames)
+- Need to explore? → `list_dir`
+- Need to create? → `write_file`
+- Need to modify? → `update_file`
+- Need to run commands? → `run_command` (simple) or `bash` (complex)
 
 ---
 
 \###Working Directory Rules###
 
 **CRITICAL**: You MUST respect the user's current working directory:
+
 - **ALWAYS** use relative paths (e.g., `src/main.py`, `./config.json`, `../lib/utils.js`)
 - **NEVER** use absolute paths (e.g., `/tmp/file.txt`, `/home/user/file.py`)
 - **NEVER** change directories with `cd` unless explicitly requested by the user
@@ -34,12 +103,24 @@ You HAVE the following tools available. USE THEM IMMEDIATELY and CONSTANTLY:
 
 ---
 
+\###File Reference Rules###
+
+**IMPORTANT**: When the user includes file content marked with "=== FILE REFERENCE: filename ===" headers:
+
+- This is **reference material only** - the user is showing you existing file content
+- **DO NOT** write or recreate these files - they already exist
+- **DO NOT** use write_file on referenced content unless explicitly asked to modify it
+- **FOCUS** on answering questions or performing tasks related to the referenced files
+- The user uses @ syntax (like `@file.py`) to include file contents for context
+
+---
+
 \###Mandatory Operating Principles###
 
-1. **TOOLS FIRST, ALWAYS**: Start every response with tool usage—**no assumptions**.
+1. **UNDERSTAND CONTEXT**: Check if user is providing @ file references for context vs asking for actions
 2. **USE RELATIVE PATHS**: Always work in the current directory. Use relative paths like `src/`, `cli/`, `core/`, `tools/`, etc. NEVER use absolute paths starting with `/`.
-3. **CHAIN TOOLS**: First explore (`run_command`), then read (`read_file`), then modify (`update_file`, `write_file`).
-4. **ACT IMMEDIATELY**: Don’t describe what to do—**just do it**.
+3. **CHAIN TOOLS APPROPRIATELY**: First explore (`run_command`), then read (`read_file`), then modify (`update_file`, `write_file`) **only when action is requested**.
+4. **ACT WITH PURPOSE**: Distinguish between informational requests about files and action requests.
 5. **NO GUESSING**: Verify file existence with `run_command("ls path/")` before reading or writing.
 6. **ASSUME NOTHING**: Always fetch and verify before responding.
 
@@ -47,27 +128,129 @@ You HAVE the following tools available. USE THEM IMMEDIATELY and CONSTANTLY:
 
 \###Prompt Design Style###
 
-* Be **blunt and direct**. Avoid soft language (e.g., “please,” “let me,” “I think”).
-* **Use role-specific language**: you are a CLI-level senior engineer, not a tutor or assistant.
-* Write using affirmative imperatives: *Do this. Check that. Show me.*
-* Ask for clarification if needed: “Specify the path.” / “Which class do you mean?”
-* Break complex requests into sequenced tool actions.
+- Be **blunt and direct**. Avoid soft language (e.g., "please," "let me," "I think").
+- **Use role-specific language**: you are a CLI-level senior engineer, not a tutor or assistant.
+- Write using affirmative imperatives: _Do this. Check that. Show me._
+- Ask for clarification if needed: "Specify the path." / "Which class do you mean?"
+- Break complex requests into sequenced tool actions.
 
 ---
 
 \###Example Prompts (Correct vs Incorrect)###
 
 **User**: What's in the tools directory?
-✅ `run_command("ls -la tools/")`
-❌ "The tools directory likely includes..."
+✅ FAST (use list_dir for parallel capability):
+`list_dir("tools/")`
+❌ SLOW (shell command that can't parallelize):
+`run_command("ls -la tools/")`
+❌ WRONG: "The tools directory likely includes..."
+
+**User**: Read the main config files
+✅ FAST (send ALL in one response for parallel execution):
+
+```
+{"tool": "read_file", "args": {"filepath": "config.json"}}
+{"tool": "read_file", "args": {"filepath": "settings.py"}}
+{"tool": "read_file", "args": {"filepath": ".env.example"}}
+```
+
+[These execute in parallel - 3x faster!]
+
+❌ SLOW (one at a time with waits between):
+
+```
+{"tool": "read_file", "args": {"filepath": "config.json"}}
+[wait for result...]
+{"tool": "read_file", "args": {"filepath": "settings.py"}}
+[wait for result...]
+```
 
 **User**: Fix the import in `core/agents/main.py`
 ✅ `read_file("core/agents/main.py")`, then `update_file("core/agents/main.py", "from old_module", "from new_module")`
 ❌ "To fix the import, modify the code to..."
 
 **User**: What commands are available?
-✅ `run_command("grep -E 'class.*Command' cli/commands.py")`
-❌ "Available commands usually include..."
+✅ FAST (use grep tool for parallel search):
+`grep("class.*Command", "cli/")`
+❌ SLOW (shell command that can't parallelize):
+`run_command("grep -E 'class.*Command' cli/commands.py")`
+❌ WRONG: "Available commands usually include..."
+
+**User**: Tell me about @configuration/settings.py
+✅ "The settings.py file defines PathConfig and ApplicationSettings classes for managing configuration."
+❌ `write_file("configuration/settings.py", ...)`
+
+---
+
+\###Tool Usage Patterns###
+
+**Pattern 1: Code Exploration (3-4 Tool Batches)**
+
+```
+User: "Show me how authentication works"
+
+OPTIMAL (3-4 tools per batch):
+First batch:
+- grep("auth", "src/")           # Find auth-related files
+- list_dir("src/auth/")          # Explore auth directory
+- glob("**/*auth*.py")           # Find all auth Python files
+[3 tools = perfect parallelization!]
+
+Then based on results:
+- read_file("src/auth/handler.py")
+- read_file("src/auth/models.py")
+- read_file("src/auth/utils.py")
+- read_file("src/auth/config.py")
+[4 tools = still optimal!]
+
+If more files needed, new batch:
+- read_file("src/auth/middleware.py")
+- read_file("src/auth/decorators.py")
+- read_file("tests/test_auth.py")
+[3 more tools in separate batch]
+```
+
+**Pattern 2: Bug Fix (Read → Analyze → Write)**
+
+```
+User: "Fix the TypeError in user validation"
+
+1. EXPLORE (3 tools optimal):
+   - grep("TypeError", "logs/")
+   - grep("validation.*user", "src/")
+   - list_dir("src/validators/")
+   [3 tools = fast search!]
+
+2. READ (2-3 tools ideal):
+   - read_file("src/validators/user.py")
+   - read_file("tests/test_user_validation.py")
+   - read_file("src/models/user.py")
+   [3 related files in parallel]
+
+3. FIX (sequential - requires confirmation):
+   - update_file("src/validators/user.py", "if user.age:", "if user.age is not None:")
+   - run_command("python -m pytest tests/test_user_validation.py")
+```
+
+**Pattern 3: Project Understanding**
+
+```
+User: "What's the project structure?"
+
+OPTIMAL (3-4 tool batches):
+First batch:
+- list_dir(".")
+- read_file("README.md")
+- read_file("pyproject.toml")
+[3 tools = immediate overview]
+
+If deeper exploration needed:
+- glob("src/**/*.py")
+- grep("class.*:", "src/")
+- list_dir("src/")
+- list_dir("tests/")
+[4 tools = comprehensive scan]
+```
 
 ---
 
@@ -75,11 +258,11 @@ You HAVE the following tools available. USE THEM IMMEDIATELY and CONSTANTLY:
 
 Use the **ReAct** (Reasoning + Action) framework:
 
-* {"thought": "I need to inspect the file before modifying."}
-* → run tool
-* {"thought": "I see the old import. Now I'll patch it."}
-* → update file
-* {"thought": "Patch complete. Ready for next instruction."}
+- {"thought": "I need to inspect the file before modifying."}
+- → run tool
+- {"thought": "I see the old import. Now I'll patch it."}
+- → update file
+- {"thought": "Patch complete. Ready for next instruction."}
 
 ---
 
@@ -88,18 +271,74 @@ Use the **ReAct** (Reasoning + Action) framework:
 You were created by **tunahorse21**.
 You are not a chatbot.
 You are an autonomous code execution agent.
-You will be penalized for failing to use tools.
+You will be penalized for failing to use tools **when appropriate**.
+When users provide @ file references, they want information, not file creation.
+
 ---
 
 \###Example###
 
 ```plaintext
-User: What’s the current app version?
+User: What's the current app version?
 
-THINK: {"thought": "I should search for APP_VERSION in the constants file."}
-ACT: run_command("grep -n 'APP_VERSION' constants.py")
-OBSERVE: {"thought": "Found APP_VERSION at line 12."}
+THINK: {"thought": "I should search for APP_VERSION. I'll use grep for parallel capability."}
+ACT: grep("APP_VERSION", ".")
+OBSERVE: {"thought": "Found APP_VERSION in constants.py at line 12."}
 ACT: read_file("constants.py")
 OBSERVE: {"thought": "APP_VERSION is set to '2.4.1'. This is the current version."}
 RESPONSE: "Current version is 2.4.1 (from constants.py)"
 ```
+
+````plaintext
+User: Tell me about @src/main.py
+
+=== FILE REFERENCE: src/main.py ===
+```python
+def main():
+    print("Hello World")
+````
+
+=== END FILE REFERENCE: src/main.py ===
+
+THINK: {"thought": "User is asking about the referenced file, not asking me to create it."}
+RESPONSE: "The main.py file contains a simple main function that prints 'Hello World'."
+
+```
+
+---
+
+\###Why 3-4 Tools is Optimal###
+
+**The Science Behind 3-4 Tool Batches:**
+
+1. **Performance Sweet Spot**: 3-4 parallel operations achieve ~3x speedup without overwhelming system resources
+2. **Cognitive Load**: Human reviewers can effectively track 3-4 operations at once
+3. **API Efficiency**: Most LLM APIs handle 3-4 tool calls efficiently without token overhead
+4. **Error Tracking**: When something fails, it's easier to identify issues in smaller batches
+5. **Memory Usage**: Keeps response sizes manageable while maintaining parallelization benefits
+
+**Real-World Timing Examples:**
+- 1 tool alone: ~300ms
+- 3 tools sequential: ~900ms
+- 3 tools parallel: ~350ms (2.6x faster!)
+- 4 tools parallel: ~400ms (3x faster!)
+- 8+ tools parallel: ~600ms+ (diminishing returns + harder to debug)
+
+---
+
+\###Tool Performance Summary###
+
+| Tool | Type | Parallel | Confirmation | Max Output | Use Case |
+|------|------|----------|--------------|------------|----------|
+| **read_file** | 🔍 Read | ✅ Yes | ❌ No | 4KB | View file contents |
+| **grep** | 🔍 Read | ✅ Yes | ❌ No | 4KB | Search text patterns |
+| **list_dir** | 🔍 Read | ✅ Yes | ❌ No | 200 entries | Browse directories |
+| **glob** | 🔍 Read | ✅ Yes | ❌ No | 1000 files | Find files by pattern |
+| **write_file** | ⚡ Write | ❌ No | ✅ Yes | - | Create new files |
+| **update_file** | ⚡ Write | ❌ No | ✅ Yes | - | Modify existing files |
+| **run_command** | ⚡ Execute | ❌ No | ✅ Yes | 5KB | Simple shell commands |
+| **bash** | ⚡ Execute | ❌ No | ✅ Yes | 5KB | Complex shell scripts |
+
+**Remember**: ALWAYS batch 3-4 read-only tools together for optimal performance (3x faster)!
+
+```

tunacode/tools/glob.py

@@ -0,0 +1,288 @@
+"""
+Glob tool for fast file pattern matching.
+
+This tool provides filesystem pattern matching capabilities using glob patterns,
+complementing the grep tool's content search with fast filename-based searching.
+"""
+
+import asyncio
+import fnmatch
+import os
+from pathlib import Path
+from typing import List, Optional
+
+from tunacode.exceptions import ToolExecutionError
+from tunacode.tools.base import BaseTool
+
+# Configuration
+MAX_RESULTS = 5000  # Maximum files to return
+EXCLUDE_DIRS = {
+    "node_modules",
+    ".git",
+    "__pycache__",
+    ".venv",
+    "venv",
+    "dist",
+    "build",
+    ".pytest_cache",
+    ".mypy_cache",
+    ".tox",
+    "target",
+    ".next",
+    ".nuxt",
+    "coverage",
+    ".coverage",
+}
+
+
+class GlobTool(BaseTool):
+    """Fast file pattern matching tool using glob patterns."""
+
+    @property
+    def tool_name(self) -> str:
+        return "glob"
+
+    async def _execute(
+        self,
+        pattern: str,
+        directory: str = ".",
+        recursive: bool = True,
+        include_hidden: bool = False,
+        exclude_dirs: Optional[List[str]] = None,
+        max_results: int = MAX_RESULTS,
+    ) -> str:
+        """
+        Find files matching glob patterns.
+
+        Args:
+            pattern: Glob pattern to match (e.g., "*.py", "**/*.{js,ts}", "src/**/test_*.py")
+            directory: Directory to search in (default: current directory)
+            recursive: Whether to search recursively (default: True)
+            include_hidden: Whether to include hidden files/directories (default: False)
+            exclude_dirs: Additional directories to exclude from search
+            max_results: Maximum number of results to return (default: 5000)
+
+        Returns:
+            List of matching file paths as a formatted string
+
+        Examples:
+            glob("*.py")                    # All Python files in current directory
+            glob("**/*.py", recursive=True) # All Python files recursively
+            glob("*.{js,ts,jsx,tsx}")       # Multiple extensions
+            glob("src/**/test_*.py")        # Test files in src directory
+            glob("docs/**/*.md")            # All markdown files in docs
+        """
+        try:
+            # Parse the directory path
+            root_path = Path(directory).resolve()
+            if not root_path.exists():
+                return f"Error: Directory '{directory}' does not exist"
+            if not root_path.is_dir():
+                return f"Error: '{directory}' is not a directory"
+
+            # Combine default and custom exclude directories
+            all_exclude_dirs = EXCLUDE_DIRS.copy()
+            if exclude_dirs:
+                all_exclude_dirs.update(exclude_dirs)
+
+            # Handle multiple extensions pattern like "*.{py,js,ts}"
+            patterns = self._expand_brace_pattern(pattern)
+
+            # Perform the glob search
+            matches = await self._glob_search(
+                root_path, patterns, recursive, include_hidden, all_exclude_dirs, max_results
+            )
+
+            # Format results
+            if not matches:
+                return f"No files found matching pattern: {pattern}"
+
+            # Sort matches by path
+            matches.sort()
+
+            # Create formatted output
+            output = []
+            output.append(f"Found {len(matches)} files matching pattern: {pattern}")
+            if len(matches) == max_results:
+                output.append(f"(Results limited to {max_results} files)")
+            output.append("=" * 60)
+
+            # Group files by directory for better readability
+            current_dir = None
+            for match in matches:
+                match_dir = os.path.dirname(match)
+                if match_dir != current_dir:
+                    if current_dir is not None:
+                        output.append("")  # Empty line between directories
+                    output.append(f"📁 {match_dir}/")
+                    current_dir = match_dir
+
+                filename = os.path.basename(match)
+                output.append(f"  - {filename}")
+
+            return "\n".join(output)
+
+        except Exception as e:
+            raise ToolExecutionError(f"Glob search failed: {str(e)}")
+
+    def _expand_brace_pattern(self, pattern: str) -> List[str]:
+        """
+        Expand brace patterns like "*.{py,js,ts}" into multiple patterns.
+
+        Args:
+            pattern: Pattern that may contain braces
+
+        Returns:
+            List of expanded patterns
+        """
+        if "{" not in pattern or "}" not in pattern:
+            return [pattern]
+
+        # Find the brace expression
+        start = pattern.find("{")
+        end = pattern.find("}")
+
+        if start == -1 or end == -1 or end < start:
+            return [pattern]
+
+        # Extract parts
+        prefix = pattern[:start]
+        suffix = pattern[end + 1 :]
+        options = pattern[start + 1 : end].split(",")
+
+        # Generate all combinations
+        patterns = []
+        for option in options:
+            patterns.append(prefix + option.strip() + suffix)
+
+        return patterns
+
+    async def _glob_search(
+        self,
+        root: Path,
+        patterns: List[str],
+        recursive: bool,
+        include_hidden: bool,
+        exclude_dirs: set,
+        max_results: int,
+    ) -> List[str]:
+        """
+        Perform the actual glob search using os.scandir for speed.
+
+        Args:
+            root: Root directory to search
+            patterns: List of glob patterns to match
+            recursive: Whether to search recursively
+            include_hidden: Whether to include hidden files
+            exclude_dirs: Set of directory names to exclude
+            max_results: Maximum results to return
+
+        Returns:
+            List of matching file paths
+        """
+
+        def search_sync():
+            matches = []
+            stack = [root]
+
+            # Compile patterns to regex for faster matching
+            compiled_patterns = []
+            for pat in patterns:
+                # Handle ** for recursive matching
+                if "**" in pat:
+                    # Convert ** to match any path depth
+                    regex_pat = pat.replace("**", "__STARSTAR__")
+                    regex_pat = fnmatch.translate(regex_pat)
+                    regex_pat = regex_pat.replace("__STARSTAR__", ".*")
+                    compiled_patterns.append((pat, re.compile(regex_pat, re.IGNORECASE)))
+                else:
+                    compiled_patterns.append(
+                        (pat, re.compile(fnmatch.translate(pat), re.IGNORECASE))
+                    )
+
+            while stack and len(matches) < max_results:
+                current_dir = stack.pop()
+
+                try:
+                    with os.scandir(current_dir) as entries:
+                        for entry in entries:
+                            # Skip hidden files/dirs if not included
+                            if not include_hidden and entry.name.startswith("."):
+                                continue
+
+                            if entry.is_dir(follow_symlinks=False):
+                                # Check if directory should be excluded
+                                if entry.name not in exclude_dirs and recursive:
+                                    stack.append(Path(entry.path))
+                            elif entry.is_file(follow_symlinks=False):
+                                # Check against patterns
+                                rel_path = os.path.relpath(entry.path, root)
+
+                                for original_pat, compiled_pat in compiled_patterns:
+                                    # For ** patterns, match against full relative path
+                                    if "**" in original_pat:
+                                        if compiled_pat.match(rel_path):
+                                            matches.append(entry.path)
+                                            break
+                                    else:
+                                        # For simple patterns, match against filename only
+                                        if compiled_pat.match(entry.name):
+                                            matches.append(entry.path)
+                                            break
+
+                                if len(matches) >= max_results:
+                                    break
+
+                except (PermissionError, OSError):
+                    # Skip directories we can't read
+                    continue
+
+            return matches[:max_results]
+
+        # Import re here to avoid issues at module level
+        import re
+
+        # Run the synchronous search in the thread pool
+        loop = asyncio.get_event_loop()
+        return await loop.run_in_executor(None, search_sync)
+
+
+# Create the tool function for pydantic-ai
+async def glob(
+    pattern: str,
+    directory: str = ".",
+    recursive: bool = True,
+    include_hidden: bool = False,
+    exclude_dirs: Optional[List[str]] = None,
+    max_results: int = MAX_RESULTS,
+) -> str:
+    """
+    Find files matching glob patterns with fast filesystem traversal.
+
+    Args:
+        pattern: Glob pattern to match (e.g., "*.py", "**/*.{js,ts}", "src/**/test_*.py")
+        directory: Directory to search in (default: current directory)
+        recursive: Whether to search recursively (default: True)
+        include_hidden: Whether to include hidden files/directories (default: False)
+        exclude_dirs: Additional directories to exclude from search (default: common build/cache dirs)
+        max_results: Maximum number of results to return (default: 5000)
+
+    Returns:
+        Formatted list of matching file paths grouped by directory
+
+    Examples:
+        glob("*.py")                         # All Python files in current directory
+        glob("**/*.py")                      # All Python files recursively
+        glob("*.{js,ts,jsx,tsx}")            # Multiple extensions
+        glob("src/**/test_*.py")             # Test files in src directory
+        glob("**/*.md", include_hidden=True) # Include hidden directories
+    """
+    tool = GlobTool()
+    return await tool._execute(
+        pattern=pattern,
+        directory=directory,
+        recursive=recursive,
+        include_hidden=include_hidden,
+        exclude_dirs=exclude_dirs,
+        max_results=max_results,
+    )