shotgun-sh 0.2.6.dev5__tar.gz → 0.2.7.dev2__tar.gz

shotgun_sh-0.2.7.dev2/PKG-INFO

@@ -0,0 +1,126 @@
+Metadata-Version: 2.4
+Name: shotgun-sh
+Version: 0.2.7.dev2
+Summary: AI-powered research, planning, and task management CLI tool
+Project-URL: Homepage, https://shotgun.sh/
+Project-URL: Repository, https://github.com/shotgun-sh/shotgun
+Project-URL: Issues, https://github.com/shotgun-sh/shotgun-alpha/issues
+Project-URL: Discord, https://discord.gg/5RmY6J2N7s
+Author-email: "Proofs.io" <hello@proofs.io>
+License: MIT
+License-File: LICENSE
+Keywords: agent,ai,cli,llm,planning,productivity,pydantic-ai,research,task-management
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Utilities
+Requires-Python: >=3.11
+Requires-Dist: anthropic>=0.39.0
+Requires-Dist: genai-prices>=0.0.27
+Requires-Dist: httpx>=0.27.0
+Requires-Dist: jinja2>=3.1.0
+Requires-Dist: kuzu>=0.7.0
+Requires-Dist: logfire[pydantic-ai]>=2.0.0
+Requires-Dist: openai>=1.0.0
+Requires-Dist: packaging>=23.0
+Requires-Dist: posthog>=3.0.0
+Requires-Dist: pydantic-ai>=0.0.14
+Requires-Dist: rich>=13.0.0
+Requires-Dist: sentencepiece>=0.2.0
+Requires-Dist: sentry-sdk[pure-eval]>=2.0.0
+Requires-Dist: tenacity>=8.0.0
+Requires-Dist: textual-dev>=1.7.0
+Requires-Dist: textual>=6.1.0
+Requires-Dist: tiktoken>=0.7.0
+Requires-Dist: tree-sitter-go>=0.23.0
+Requires-Dist: tree-sitter-javascript>=0.23.0
+Requires-Dist: tree-sitter-python>=0.23.0
+Requires-Dist: tree-sitter-rust>=0.23.0
+Requires-Dist: tree-sitter-typescript>=0.23.0
+Requires-Dist: tree-sitter>=0.21.0
+Requires-Dist: typer>=0.12.0
+Requires-Dist: watchdog>=4.0.0
+Provides-Extra: dev
+Requires-Dist: commitizen>=3.13.0; extra == 'dev'
+Requires-Dist: lefthook>=1.12.0; extra == 'dev'
+Requires-Dist: mypy>=1.11.0; extra == 'dev'
+Requires-Dist: ruff>=0.6.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Shotgun
+
+**Spec-Driven Development for AI Code Generation**
+
+Shotgun is a CLI tool that turns work with AI code-gen tools from "I want to build X" into: **research → specs → plans → tasks → implementation**. It reads your entire codebase, coordinates AI agents to do the heavy lifting, and exports clean artifacts in the agents.md format so your code-gen tools actually know what they're building.
+
+🌐 **Learn more at [shotgun.sh](https://shotgun.sh/)**
+
+## Features
+
+### 📊 Complete Codebase Understanding
+
+Before writing a single line, Shotgun reads all of it. Your patterns. Your dependencies. Your technical debt. Whether you're adding features, onboarding devs, planning migrations, or refactoring - Shotgun knows what you're working with.
+
+### 🔄 Five Modes. One Journey. Zero Gaps.
+
+**Research** (what exists) → **Specify** (what to build) → **Plan** (how to build) → **Tasks** (break it down) → **Export** (to any tool)
+
+Not another chatbot. A complete workflow where each mode feeds the next.
+
+### ➡️ Export to agents.md
+
+Outputs plug into many code-generation tools including Codex, Cursor, Warp, Devin, opencode, Jules, and more.
+
+### 📝 Specs That Don't Die in Slack
+
+Every research finding, every architectural decision, every "here's why we didn't use that library" - captured as markdown in your repo. Version controlled. Searchable.
+
+## Installation
+
+### Using pipx (Recommended)
+
+```bash
+pipx install shotgun-sh
+```
+
+**Why pipx?** It installs Shotgun in an isolated environment, preventing dependency conflicts with your other Python projects.
+
+### Using pip
+
+```bash
+pip install shotgun-sh
+```
+
+## Quick Start
+
+```bash
+# Research your codebase or a topic
+shotgun research "What is our authentication flow?"
+
+# Generate specifications
+shotgun spec "Add OAuth2 authentication"
+
+# Create an implementation plan
+shotgun plan "Build user dashboard"
+
+# Break down into tasks
+shotgun tasks "Implement payment system"
+
+# Export to agents.md format for your code-gen tools
+shotgun export
+```
+
+## Support
+
+Have questions? Join our community on **[Discord](https://discord.gg/5RmY6J2N7s)**
+
+---
+
+**License:** MIT
+**Python:** 3.11+

shotgun_sh-0.2.7.dev2/README_PYPI.md

@@ -0,0 +1,71 @@
+# Shotgun
+
+**Spec-Driven Development for AI Code Generation**
+
+Shotgun is a CLI tool that turns work with AI code-gen tools from "I want to build X" into: **research → specs → plans → tasks → implementation**. It reads your entire codebase, coordinates AI agents to do the heavy lifting, and exports clean artifacts in the agents.md format so your code-gen tools actually know what they're building.
+
+🌐 **Learn more at [shotgun.sh](https://shotgun.sh/)**
+
+## Features
+
+### 📊 Complete Codebase Understanding
+
+Before writing a single line, Shotgun reads all of it. Your patterns. Your dependencies. Your technical debt. Whether you're adding features, onboarding devs, planning migrations, or refactoring - Shotgun knows what you're working with.
+
+### 🔄 Five Modes. One Journey. Zero Gaps.
+
+**Research** (what exists) → **Specify** (what to build) → **Plan** (how to build) → **Tasks** (break it down) → **Export** (to any tool)
+
+Not another chatbot. A complete workflow where each mode feeds the next.
+
+### ➡️ Export to agents.md
+
+Outputs plug into many code-generation tools including Codex, Cursor, Warp, Devin, opencode, Jules, and more.
+
+### 📝 Specs That Don't Die in Slack
+
+Every research finding, every architectural decision, every "here's why we didn't use that library" - captured as markdown in your repo. Version controlled. Searchable.
+
+## Installation
+
+### Using pipx (Recommended)
+
+```bash
+pipx install shotgun-sh
+```
+
+**Why pipx?** It installs Shotgun in an isolated environment, preventing dependency conflicts with your other Python projects.
+
+### Using pip
+
+```bash
+pip install shotgun-sh
+```
+
+## Quick Start
+
+```bash
+# Research your codebase or a topic
+shotgun research "What is our authentication flow?"
+
+# Generate specifications
+shotgun spec "Add OAuth2 authentication"
+
+# Create an implementation plan
+shotgun plan "Build user dashboard"
+
+# Break down into tasks
+shotgun tasks "Implement payment system"
+
+# Export to agents.md format for your code-gen tools
+shotgun export
+```
+
+## Support
+
+Have questions? Join our community on **[Discord](https://discord.gg/5RmY6J2N7s)**
+
+---
+
+**License:** MIT
+**Python:** 3.11+

{shotgun_sh-0.2.6.dev5 → shotgun_sh-0.2.7.dev2}/pyproject.toml

@@ -1,8 +1,8 @@
 [project]
 name = "shotgun-sh"
-version = "0.2.6.dev5"
+version = "0.2.7.dev2"
 description = "AI-powered research, planning, and task management CLI tool"
-readme = "README.md"
+readme = "README_PYPI.md"
 license = { text = "MIT" }
 authors = [
     { name = "Proofs.io", email = "hello@proofs.io" }
@@ -46,6 +46,7 @@ dependencies = [
     "sentencepiece>=0.2.0",
     "packaging>=23.0",
     "genai-prices>=0.0.27",
+    "tenacity>=8.0.0",
 ]
 
 [project.urls]
@@ -76,6 +77,7 @@ packages = ["src/shotgun"]
 include = [
     "/src",
     "/README.md",
+    "/README_PYPI.md",
     "/LICENSE",
     "/pyproject.toml"
 ]

{shotgun_sh-0.2.6.dev5 → shotgun_sh-0.2.7.dev2}/src/shotgun/agents/agent_manager.py

@@ -1,10 +1,21 @@
 """Agent manager for coordinating multiple AI agents with shared message history."""
 
+import json
 import logging
 from collections.abc import AsyncIterable, Sequence
 from dataclasses import dataclass, field, is_dataclass, replace
+from pathlib import Path
 from typing import TYPE_CHECKING, Any, cast
 
+import logfire
+from tenacity import (
+    before_sleep_log,
+    retry,
+    retry_if_exception,
+    stop_after_attempt,
+    wait_exponential,
+)
+
 if TYPE_CHECKING:
     from shotgun.agents.conversation_history import ConversationState
 
@@ -52,6 +63,35 @@ from .tasks import create_tasks_agent
 logger = logging.getLogger(__name__)
 
 
+def _is_retryable_error(exception: BaseException) -> bool:
+    """Check if exception should trigger a retry.
+
+    Args:
+        exception: The exception to check.
+
+    Returns:
+        True if the exception is a transient error that should be retried.
+    """
+    # ValueError for truncated/incomplete JSON
+    if isinstance(exception, ValueError):
+        error_str = str(exception)
+        return "EOF while parsing" in error_str or (
+            "JSON" in error_str and "parsing" in error_str
+        )
+
+    # API errors (overload, rate limits)
+    exception_name = type(exception).__name__
+    if "APIStatusError" in exception_name:
+        error_str = str(exception)
+        return "overload" in error_str.lower() or "rate" in error_str.lower()
+
+    # Network errors
+    if "ConnectionError" in exception_name or "TimeoutError" in exception_name:
+        return True
+
+    return False
+
+
 class MessageHistoryUpdated(Message):
     """Event posted when the message history is updated."""
 
@@ -265,6 +305,49 @@ class AgentManager(Widget):
                 f"Invalid agent type: {agent_type}. Must be one of: {', '.join(e.value for e in AgentType)}"
             ) from None
 
+    @retry(
+        stop=stop_after_attempt(3),
+        wait=wait_exponential(multiplier=1, min=1, max=8),
+        retry=retry_if_exception(_is_retryable_error),
+        before_sleep=before_sleep_log(logger, logging.WARNING),
+        reraise=True,
+    )
+    async def _run_agent_with_retry(
+        self,
+        agent: Agent[AgentDeps, AgentResponse],
+        prompt: str | None,
+        deps: AgentDeps,
+        usage_limits: UsageLimits | None,
+        message_history: list[ModelMessage],
+        event_stream_handler: Any,
+        **kwargs: Any,
+    ) -> AgentRunResult[AgentResponse]:
+        """Run agent with automatic retry on transient errors.
+
+        Args:
+            agent: The agent to run.
+            prompt: Optional prompt to send to the agent.
+            deps: Agent dependencies.
+            usage_limits: Optional usage limits.
+            message_history: Message history to provide to agent.
+            event_stream_handler: Event handler for streaming.
+            **kwargs: Additional keyword arguments.
+
+        Returns:
+            The agent run result.
+
+        Raises:
+            Various exceptions if all retries fail.
+        """
+        return await agent.run(
+            prompt,
+            deps=deps,
+            usage_limits=usage_limits,
+            message_history=message_history,
+            event_stream_handler=event_stream_handler,
+            **kwargs,
+        )
+
     async def run(
         self,
         prompt: str | None = None,
@@ -391,8 +474,9 @@ class AgentManager(Widget):
         )
 
         try:
-            result: AgentRunResult[AgentResponse] = await self.current_agent.run(
-                prompt,
+            result: AgentRunResult[AgentResponse] = await self._run_agent_with_retry(
+                agent=self.current_agent,
+                prompt=prompt,
                 deps=deps,
                 usage_limits=usage_limits,
                 message_history=message_history,
@@ -401,18 +485,93 @@ class AgentManager(Widget):
                 else None,
                 **kwargs,
             )
+        except ValueError as e:
+            # Handle truncated/incomplete JSON in tool calls specifically
+            error_str = str(e)
+            if "EOF while parsing" in error_str or (
+                "JSON" in error_str and "parsing" in error_str
+            ):
+                logger.error(
+                    "Tool call with truncated/incomplete JSON arguments detected",
+                    extra={
+                        "agent_mode": self._current_agent_type.value,
+                        "model_name": model_name,
+                        "error": error_str,
+                    },
+                )
+                logfire.error(
+                    "Tool call with truncated JSON arguments",
+                    agent_mode=self._current_agent_type.value,
+                    model_name=model_name,
+                    error=error_str,
+                )
+                # Add helpful hint message for the user
+                self.ui_message_history.append(
+                    HintMessage(
+                        message="⚠️ The agent attempted an operation with arguments that were too large (truncated JSON). "
+                        "Try breaking your request into smaller steps or more focused contracts."
+                    )
+                )
+                self._post_messages_updated()
+            # Re-raise to maintain error visibility
+            raise
+        except Exception as e:
+            # Log the error with full stack trace to shotgun.log and Logfire
+            logger.exception(
+                "Agent execution failed",
+                extra={
+                    "agent_mode": self._current_agent_type.value,
+                    "model_name": model_name,
+                    "error_type": type(e).__name__,
+                },
+            )
+            logfire.exception(
+                "Agent execution failed",
+                agent_mode=self._current_agent_type.value,
+                model_name=model_name,
+                error_type=type(e).__name__,
+            )
+            # Re-raise to let TUI handle user messaging
+            raise
         finally:
             self._stream_state = None
 
         # Agent ALWAYS returns AgentResponse with structured output
         agent_response = result.output
-        logger.debug("Agent returned structured AgentResponse")
+        logger.debug(
+            "Agent returned structured AgentResponse",
+            extra={
+                "has_response": agent_response.response is not None,
+                "response_length": len(agent_response.response)
+                if agent_response.response
+                else 0,
+                "response_preview": agent_response.response[:100] + "..."
+                if agent_response.response and len(agent_response.response) > 100
+                else agent_response.response or "(empty)",
+                "has_clarifying_questions": bool(agent_response.clarifying_questions),
+                "num_clarifying_questions": len(agent_response.clarifying_questions)
+                if agent_response.clarifying_questions
+                else 0,
+            },
+        )
 
         # Always add the agent's response messages to maintain conversation history
         self.ui_message_history = original_messages + cast(
             list[ModelRequest | ModelResponse | HintMessage], result.new_messages()
         )
 
+        # Get file operations early so we can use them for contextual messages
+        file_operations = deps.file_tracker.operations.copy()
+        self.recently_change_files = file_operations
+
+        logger.debug(
+            "File operations tracked",
+            extra={
+                "num_file_operations": len(file_operations),
+                "operation_files": [Path(op.file_path).name for op in file_operations],
+            },
+        )
+
         # Check if there are clarifying questions
         if agent_response.clarifying_questions:
             logger.info(
@@ -459,27 +618,93 @@ class AgentManager(Widget):
                         response_text=agent_response.response,
                     )
                 )
+
+            # Post UI update with hint messages and file operations
+            logger.debug(
+                "Posting UI update for Q&A mode with hint messages and file operations"
+            )
+            self._post_messages_updated(file_operations)
         else:
-            # No clarifying questions - just show the response if present
+            # No clarifying questions - show the response or a default success message
             if agent_response.response and agent_response.response.strip():
+                logger.debug(
+                    "Adding agent response as hint",
+                    extra={
+                        "response_preview": agent_response.response[:100] + "..."
+                        if len(agent_response.response) > 100
+                        else agent_response.response,
+                        "has_file_operations": len(file_operations) > 0,
+                    },
+                )
                 self.ui_message_history.append(
                     HintMessage(message=agent_response.response)
                 )
+            else:
+                # Fallback: response is empty or whitespace
+                logger.debug(
+                    "Agent response was empty, using fallback completion message",
+                    extra={"has_file_operations": len(file_operations) > 0},
+                )
+                # Show contextual message based on whether files were modified
+                if file_operations:
+                    self.ui_message_history.append(
+                        HintMessage(
+                            message="✅ Task completed - files have been modified"
+                        )
+                    )
+                else:
+                    self.ui_message_history.append(
+                        HintMessage(message="✅ Task completed")
+                    )
+
+            # Post UI update immediately so user sees the response without delay
+            logger.debug(
+                "Posting immediate UI update with hint message and file operations"
+            )
+            self._post_messages_updated(file_operations)
 
         # Apply compaction to persistent message history to prevent cascading growth
         all_messages = result.all_messages()
-        self.message_history = await apply_persistent_compaction(all_messages, deps)
-        usage = result.usage()
-        deps.usage_manager.add_usage(
-            usage, model_name=deps.llm_model.name, provider=deps.llm_model.provider
-        )
+        try:
+            logger.debug(
+                "Starting message history compaction",
+                extra={"message_count": len(all_messages)},
+            )
+            self.message_history = await apply_persistent_compaction(all_messages, deps)
+            logger.debug(
+                "Completed message history compaction",
+                extra={
+                    "original_count": len(all_messages),
+                    "compacted_count": len(self.message_history),
+                },
+            )
+        except Exception as e:
+            # If compaction fails, log full error with stack trace and use uncompacted messages
+            logger.error(
+                "Failed to compact message history - using uncompacted messages",
+                exc_info=True,
+                extra={
+                    "error": str(e),
+                    "message_count": len(all_messages),
+                    "agent_mode": self._current_agent_type.value,
+                },
+            )
+            # Fallback: use uncompacted messages to prevent data loss
+            self.message_history = all_messages
 
-        # Log file operations summary if any files were modified
-        file_operations = deps.file_tracker.operations.copy()
-        self.recently_change_files = file_operations
+        usage = result.usage()
+        if hasattr(deps, "llm_model") and deps.llm_model is not None:
+            deps.usage_manager.add_usage(
+                usage, model_name=deps.llm_model.name, provider=deps.llm_model.provider
+            )
+        else:
+            logger.warning(
+                "llm_model is None, skipping usage tracking",
+                extra={"agent_mode": self._current_agent_type.value},
+            )
 
-        # Post message history update (hints are now added synchronously above)
-        self._post_messages_updated(file_operations)
+        # UI updates are now posted immediately in each branch (Q&A or non-Q&A)
+        # before compaction, so no duplicate posting needed here
 
         return result
 
@@ -554,6 +779,39 @@ class AgentManager(Widget):
                     # Detect source from call stack
                     source = detect_source()
 
+                    # Log if tool call has incomplete args (for debugging truncated JSON)
+                    if isinstance(event.part.args, str):
+                        try:
+                            json.loads(event.part.args)
+                        except (json.JSONDecodeError, ValueError):
+                            args_preview = (
+                                event.part.args[:100] + "..."
+                                if len(event.part.args) > 100
+                                else event.part.args
+                            )
+                            logger.warning(
+                                "FunctionToolCallEvent received with incomplete JSON args",
+                                extra={
+                                    "tool_name": event.part.tool_name,
+                                    "tool_call_id": event.part.tool_call_id,
+                                    "args_preview": args_preview,
+                                    "args_length": len(event.part.args)
+                                    if event.part.args
+                                    else 0,
+                                    "agent_mode": self._current_agent_type.value,
+                                },
+                            )
+                            logfire.warn(
+                                "FunctionToolCallEvent received with incomplete JSON args",
+                                tool_name=event.part.tool_name,
+                                tool_call_id=event.part.tool_call_id,
+                                args_preview=args_preview,
+                                args_length=len(event.part.args)
+                                if event.part.args
+                                else 0,
+                                agent_mode=self._current_agent_type.value,
+                            )
+
                     track_event(
                         "tool_called",
                         {

{shotgun_sh-0.2.6.dev5 → shotgun_sh-0.2.7.dev2}/src/shotgun/agents/common.py

@@ -384,23 +384,48 @@ def get_agent_existing_files(agent_mode: AgentType | None = None) -> list[str]:
                     relative_path = file_path.relative_to(base_path)
                     existing_files.append(str(relative_path))
     else:
-        # For other agents, check both .md file and directory with same name
-        allowed_file = AGENT_DIRECTORIES[agent_mode]
-
-        # Check for the .md file
-        md_file_path = base_path / allowed_file
-        if md_file_path.exists():
-            existing_files.append(allowed_file)
-
-        # Check for directory with same base name (e.g., research/ for research.md)
-        base_name = allowed_file.replace(".md", "")
-        dir_path = base_path / base_name
-        if dir_path.exists() and dir_path.is_dir():
-            # List all files in the directory
-            for file_path in dir_path.rglob("*"):
-                if file_path.is_file():
-                    relative_path = file_path.relative_to(base_path)
-                    existing_files.append(str(relative_path))
+        # For other agents, check files/directories they have access to
+        allowed_paths_raw = AGENT_DIRECTORIES[agent_mode]
+
+        # Convert single Path/string to list of Paths for uniform handling
+        if isinstance(allowed_paths_raw, str):
+            # Special case: "*" means export agent (shouldn't reach here but handle it)
+            allowed_paths = (
+                [Path(allowed_paths_raw)] if allowed_paths_raw != "*" else []
+            )
+        elif isinstance(allowed_paths_raw, Path):
+            allowed_paths = [allowed_paths_raw]
+        else:
+            # Already a list
+            allowed_paths = allowed_paths_raw
+
+        # Check each allowed path
+        for allowed_path in allowed_paths:
+            allowed_str = str(allowed_path)
+
+            # Check if it's a directory (no .md suffix)
+            if not allowed_path.suffix or not allowed_str.endswith(".md"):
+                # It's a directory - list all files within it
+                dir_path = base_path / allowed_str
+                if dir_path.exists() and dir_path.is_dir():
+                    for file_path in dir_path.rglob("*"):
+                        if file_path.is_file():
+                            relative_path = file_path.relative_to(base_path)
+                            existing_files.append(str(relative_path))
+            else:
+                # It's a file - check if it exists
+                file_path = base_path / allowed_str
+                if file_path.exists():
+                    existing_files.append(allowed_str)
+
+                # Also check for associated directory (e.g., research/ for research.md)
+                base_name = allowed_str.replace(".md", "")
+                dir_path = base_path / base_name
+                if dir_path.exists() and dir_path.is_dir():
+                    for file_path in dir_path.rglob("*"):
+                        if file_path.is_file():
+                            relative_path = file_path.relative_to(base_path)
+                            existing_files.append(str(relative_path))
 
     return existing_files