drydock-cli 2.4.9__tar.gz → 2.5.1__tar.gz

{drydock_cli-2.4.9 → drydock_cli-2.5.1}/CLAUDE.md

@@ -4,12 +4,14 @@
 
 Drydock is a local CLI coding agent (fork of mistral-vibe, Apache 2.0).
 - **Repo:** https://github.com/fbobe321/drydock
-- **PyPI:** https://pypi.org/project/drydock-cli/ (v2.2.1 for v2, v3.0.0 for v3)
-- **Goal:** Best-in-class SWE-bench Verified pass rate with local LLMs
+- **PyPI:** https://pypi.org/project/drydock-cli/ (v2.5.0)
+- **Goal:** Reliable TUI coding agent with local LLMs. PRD-driven project building + SWE-bench bug fixing.
 - **Hardware:** 2x RTX 4060 Ti 16GB, Gemma 4 26B MoE (A4B) via vLLM Docker at localhost:8000
 - **Server:** remus (Ubuntu 22.04, user: bobef)
-- **Active model:** Gemma 4 26B-A4B-it-AWQ-4bit (replaced devstral-24B — 3-4x faster, better results)
-- **Active codebase:** DryDock v3 at `/data3/drydock-v3/` (clean rewrite)
+- **Active model:** Gemma 4 26B-A4B-it-AWQ-4bit (only 4B active params, ~70 tok/s)
+- **TUI results:** 88/90 PRD projects pass (98%) through the real TUI
+- **SWE-bench:** 70% file match (v3 baseline), 58% (v3 tuned — tighter prompt hurt)
+- **Priority:** TUI experience first, SWE-bench second
 
 ## Build & Test
 
@@ -36,17 +38,17 @@ curl http://localhost:8000/v1/models
 python3 -c "import ast; ast.parse(open('path/to/file.py').read())"
 ```
 
-### DryDock v2 (legacy)
+### DryDock v2 (TUI — user-facing, v2.5.0)
 
 ```bash
 # Install
-uv sync
+pip install drydock-cli==2.5.0
 
-# Run drydock CLI
-uv run drydock
+# Run TUI (the ONLY way to use drydock — no headless mode)
+drydock
 
-# Run programmatically (headless, for benchmarks)
-python3 -c "import sys; sys.path.insert(0, '.'); from drydock.cli.entrypoint import main; main()" --agent auto-approve -p "your prompt"
+# Headless mode has been REMOVED. All testing goes through the TUI.
+# Use scripts/tui_test.py (pexpect) for automated TUI testing.
 
 # Run tests
 uv run pytest tests/ -x -q --timeout=30
@@ -92,11 +94,15 @@ v3 is a from-scratch rewrite using nano-claude-code as foundation. 4 core files,
 - **Two-tier compaction.** Normal compaction truncates old tool results. Emergency compaction triggers on 400 errors (context overflow) with aggressive truncation.
 - **Provider-agnostic design.** No model-specific hacks in the agent loop. All model quirks handled in `providers.py`.
 
-### v2 Constraints (still apply to v2 codebase)
+### v2 Constraints (TUI codebase)
 - **MessageList is not a plain list.** Never `self.messages = [...]`. Use `self.messages.reset([...])`.
 - **vLLM/Mistral rejects `user` after `tool` messages.** `_sanitize_message_ordering()` runs before every LLM call. All injection must use `_inject_system_note()`.
-- **`os._exit()` in programmatic.py** — necessary because async cleanup hangs. Ensure stdout is flushed before calling.
-- **AGENTS.md is CRITICAL for devstral.** Without per-project instructions, the model loops on ls/bash. Auto-created by `_ensure_agents_md()`.
+- **Non-streaming for Gemma 4.** Streaming tool call argument accumulation produces empty/malformed JSON. `enable_streaming=False` when model is Gemma 4.
+- **Disabled tools for Gemma 4:** ask_user_question, todo, task_create, task_update, task, invoke_skill, tool_search. These cause loops or validation errors.
+- **Simplified prompt for Gemma 4.** Auto-detected via model name → uses `gemma4.md` (20 lines) instead of `cli.md` (125 lines). Complex prompts cause delegation/asking instead of coding.
+- **write_file overwrite=True by default.** Gemma 4 writes files multiple times; old default (False) caused error loops.
+- **Headless mode removed.** All interaction through TUI. Testing via pexpect (`scripts/tui_test.py`).
+- **NEVER test with headless.** The TUI has different code paths (streaming, approval, path expansion). Headless passes mask TUI bugs.
 
 ### Shared Constraints
 - **"Restate the goal" blocks tool calling.** Never ask the model to output text before its first tool call — it pre-empts the tool-calling mechanism.
@@ -252,22 +258,42 @@ v3 is a from-scratch rewrite using nano-claude-code as foundation. 4 core files,
 
 ## Key Learnings
 
-1. AGENTS.md is essential — devstral loops without it (Gemma 4 does not have this problem)
-2. "Restate the goal" blocked tool calling for months — one line
-3. Additive-only changes work best — control flow mods regress
-4. The harness matters as much as the model (Meta-Harness paper)
-5. Environment bootstrapping saves 2-4 exploratory turns
-6. Plan→Edit workflow is Mistral's design, don't fight it
-7. Circuit breaker was net negative — removed entirely
-8. Loop detection should guide, never stop
-9. Test the installed package, not source (different Python envs)
-10. Never wait for full test runs — fix issues from early results
-11. Clean rewrites beat incremental fixes — v3 (~750 lines) outperforms v2 (~5000+ lines)
-12. Model choice matters more than agent complexity — Gemma 4 MoE gives 3-4x speed + better accuracy than devstral
-13. Gemma 4 leaks thinking tokens (`<|channel>thought<channel|>`) — must filter in provider layer
-14. Two-tier compaction (normal + emergency on 400) is more robust than single-tier
-15. Provider-agnostic design enables fast model switching — v3 works with any OpenAI-compatible endpoint
-16. Docker-based model serving (vLLM) is more reliable than bare-metal — easier restarts, GPU management
+1. **Test the TUI, not headless.** Headless passes mask real bugs. TUI has different code paths (streaming, approval, render_path_prompt). ALL testing through TUI via pexpect.
+2. **Use PRDs for testing.** Concrete PRDs with expected files and `--help` give clear pass/fail. 100-project suite proved TUI works (98%).
+3. **Scaffold per model.** Weaker models need more guardrails (non-streaming, fewer tools, simpler prompts). As models improve, remove scaffolding. The code should get simpler over time.
+4. **Streaming breaks Gemma 4 tool calls.** Streaming accumulates JSON chunk-by-chunk → empty args. Non-streaming gets complete response → works perfectly. Root cause of weeks of TUI failures.
+5. **Fewer tools = better for local models.** Gemma 4 wasted 64 turns on task_create/task_update. Disabling non-essential tools eliminated the loop.
+6. **Simple prompts win.** The 125-line cli.md with phases/delegation confused Gemma 4. The 20-line gemma4.md ("ACT IMMEDIATELY") works.
+7. AGENTS.md essential for devstral (loops without it), not needed for Gemma 4
+8. Circuit breaker was net negative — removed entirely
+9. Loop detection should guide, never stop (retrospection > hardcoded nudges)
+10. Clean rewrites beat incremental fixes — v3 (~750 lines) outperforms v2 (~5000+ lines) on SWE-bench
+11. Model choice matters more than agent complexity — Gemma 4 MoE gives 3-4x speed
+12. Docker-based model serving (vLLM) is reliable — easier restarts, GPU management
+13. Auto-read on failed edit — automatically show the model the real file content
+14. Temperature bump on loops — force model to explore different paths
+15. Failed-approach accumulator — prevent re-trying strategies after pruning/compaction
+16. PRD complexity must match model capability — too many files/features causes timeout
+
+## Testing
+
+### TUI Test Harness
+- **Script:** `scripts/tui_test.py` — pexpect-based, drives the REAL TUI binary
+- **Auto-approves** tool permission prompts
+- **Detects:** write loops, API errors, tool call counts, file creation, --help pass/fail
+- **100 PRDs:** at `/data3/drydock_test_projects/` (88/90 pass = 98%)
+
+### PRD Format (what works)
+- Short (under 30 lines)
+- 4-6 files max
+- Clear package name and file list
+- Test command: `python3 -m package_name --help`
+- Stdlib only (no external deps)
+
+### What breaks PRDs
+- Too many files (8+) — model runs out of turns
+- Complex parsers (custom YAML/XML from scratch)
+- Features requiring external APIs or databases
 
 ## When Compacting
 

{drydock_cli-2.4.9 → drydock_cli-2.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: drydock-cli
-Version: 2.4.9
+Version: 2.5.1
 Summary: Drydock — nautical CLI coding agent
 Project-URL: Homepage, https://github.com/fbobe321/drydock
 Project-URL: Repository, https://github.com/fbobe321/drydock

{drydock_cli-2.4.9 → drydock_cli-2.5.1}/drydock/cli/cli.py

@@ -157,92 +157,57 @@ def run_cli(args: argparse.Namespace) -> None:
         loaded_session = load_session(args, config)
 
         stdin_prompt = get_prompt_from_stdin()
+
+        # -p flag feeds prompt into the TUI (headless mode removed)
         if args.prompt is not None:
-            # Disable tools that Gemma 4 can't handle
-            gemma_disabled = []
-            try:
-                active = config.get_active_model()
-                if "gemma" in active.name.lower():
-                    gemma_disabled = ["todo", "task_create", "task_update", "task", "invoke_skill", "tool_search"]
-            except Exception:
-                pass
-            config.disabled_tools = [*config.disabled_tools, "ask_user_question", *gemma_disabled]
-            programmatic_prompt = args.prompt or stdin_prompt
-            if not programmatic_prompt:
-                print(
-                    "Error: No prompt provided for programmatic mode", file=sys.stderr
-                )
-                sys.exit(1)
-            output_format = OutputFormat(
-                args.output if hasattr(args, "output") else "text"
-            )
+            if not hasattr(args, 'initial_prompt') or not args.initial_prompt:
+                args.initial_prompt = args.prompt or stdin_prompt
 
-            try:
-                final_response = run_programmatic(
-                    config=config,
-                    prompt=programmatic_prompt,
-                    max_turns=args.max_turns,
-                    max_price=args.max_price,
-                    output_format=output_format,
-                    previous_messages=loaded_session[0] if loaded_session else None,
-                    agent_name=initial_agent_name,
-                )
-                if final_response:
-                    print(final_response)
-                sys.exit(0)
-            except ConversationLimitException as e:
-                print(e, file=sys.stderr)
-                sys.exit(1)
-            except RuntimeError as e:
-                print(f"Error: {e}", file=sys.stderr)
-                sys.exit(1)
-        else:
-            # Disable problematic tools for models that can't handle complex schemas
-            try:
-                active = config.get_active_model()
-                if "gemma" in active.name.lower():
-                    config.disabled_tools = [
-                        *config.disabled_tools,
-                        "ask_user_question",  # Gemma sends empty {} args
-                        "todo",  # Gemma sends empty {} args
-                        "task_create",  # Gemma loops on task management instead of coding
-                        "task_update",  # Gemma sends invalid args, loops 64+ times
-                        "task",  # Subagent delegation — Gemma can't use it properly
-                        "invoke_skill",  # Gemma loads wrong skills, leaks templates
-                        "tool_search",  # Not useful for local models
-                    ]
-            except (ValueError, AttributeError):
-                pass
-
-            # Disable streaming for Gemma 4 — streaming tool call accumulation
-            # produces empty arguments (14 write_file calls → 0 files).
-            # Headless (non-streaming) works perfectly with same model.
-            use_streaming = True
-            try:
-                if "gemma" in config.get_active_model().name.lower():
-                    use_streaming = False
-            except Exception:
-                pass
-
-            agent_loop = AgentLoop(
-                config,
-                agent_name=initial_agent_name,
-                enable_streaming=use_streaming,
-                entrypoint_metadata=EntrypointMetadata(
-                    agent_entrypoint="cli",
-                    agent_version=__version__,
-                    client_name="drydock_cli",
-                    client_version=__version__,
-                ),
-            )
+        # Disable problematic tools for models that can't handle complex schemas
+        try:
+            active = config.get_active_model()
+            if "gemma" in active.name.lower():
+                config.disabled_tools = [
+                    *config.disabled_tools,
+                    "ask_user_question",
+                    "todo",
+                    "task_create",
+                    "task_update",
+                    "task",
+                    "invoke_skill",
+                    "tool_search",
+                ]
+        except (ValueError, AttributeError):
+            pass
+
+        # Disable streaming for Gemma 4 — streaming tool call accumulation
+        # produces empty arguments (14 write_file calls → 0 files)
+        use_streaming = True
+        try:
+            if "gemma" in config.get_active_model().name.lower():
+                use_streaming = False
+        except Exception:
+            pass
+
+        agent_loop = AgentLoop(
+            config,
+            agent_name=initial_agent_name,
+            enable_streaming=use_streaming,
+            entrypoint_metadata=EntrypointMetadata(
+                agent_entrypoint="cli",
+                agent_version=__version__,
+                client_name="drydock_cli",
+                client_version=__version__,
+            ),
+        )
 
-            if loaded_session:
-                _resume_previous_session(agent_loop, *loaded_session)
+        if loaded_session:
+            _resume_previous_session(agent_loop, *loaded_session)
 
-            run_textual_ui(
-                agent_loop=agent_loop,
-                initial_prompt=args.initial_prompt or stdin_prompt,
-                teleport_on_start=args.teleport,
+        run_textual_ui(
+            agent_loop=agent_loop,
+            initial_prompt=args.initial_prompt or stdin_prompt,
+            teleport_on_start=args.teleport,
             )
 
     except (KeyboardInterrupt, EOFError):

{drydock_cli-2.4.9 → drydock_cli-2.5.1}/drydock/core/agent_loop.py

@@ -1856,6 +1856,42 @@ class AgentLoop:
         except Exception:
             pass
 
+        # Inject discoverable CLI tools (built by DryDock, CLI-Anything, system)
+        try:
+            from drydock.core.config.harness_files import get_harness_files_manager
+            mgr = get_harness_files_manager()
+            user_skills_dirs = mgr.user_skills_dirs
+            tools_found = []
+            for skills_dir in user_skills_dirs:
+                if not skills_dir.is_dir():
+                    continue
+                for skill_dir in sorted(skills_dir.iterdir())[:50]:
+                    skill_md = skill_dir / "SKILL.md"
+                    if not skill_md.is_file():
+                        continue
+                    name = skill_dir.name
+                    # Only show tool-*, system-*, cli-anything-* skills
+                    if not any(name.startswith(p) for p in ("tool-", "system-", "cli-anything-")):
+                        continue
+                    # Read first line of description
+                    try:
+                        for line in skill_md.read_text(encoding="utf-8").split("\n"):
+                            if line.strip().startswith("description:"):
+                                desc = line.split(":", 1)[1].strip().strip('"').strip("'")[:80]
+                                tools_found.append(f"  {name}: {desc}")
+                                break
+                    except Exception:
+                        pass
+            if tools_found:
+                # Show max 15 to avoid bloating context
+                parts.append(
+                    f"AVAILABLE CLI TOOLS ({len(tools_found)} installed, showing first 15):\n"
+                    + "\n".join(tools_found[:15])
+                    + "\nUse via bash: cd /path && python3 -m package_name [args]"
+                )
+        except Exception:
+            pass
+
         if parts:
             self._inject_system_note("\n".join(parts))
 

{drydock_cli-2.4.9 → drydock_cli-2.5.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "drydock-cli"
-version = "2.4.9"
+version = "2.5.1"
 description = "Drydock — nautical CLI coding agent"
 readme = "README.md"
 requires-python = ">=3.12"

drydock_cli-2.5.1/scripts/discover_cli_tools.sh

@@ -0,0 +1,69 @@
+#!/bin/bash
+# Discover installed CLI tools and create DryDock skill wrappers
+# Run this to auto-detect useful CLIs on the system
+
+SKILLS_DIR="$HOME/.drydock/skills"
+mkdir -p "$SKILLS_DIR"
+
+# List of CLI tools to check for and create skill wrappers
+TOOLS=(
+    "jq|JSON processor — query and transform JSON data"
+    "yq|YAML/JSON/XML processor — like jq but for YAML"
+    "rg|ripgrep — fast regex search across files"
+    "fd|fd-find — fast file finder (better find)"
+    "bat|cat with syntax highlighting and line numbers"
+    "scc|Source code line counter by language"
+    "httpie|HTTPie — user-friendly HTTP client (http command)"
+    "tree|Directory tree viewer"
+    "pandoc|Universal document converter"
+    "ffmpeg|Audio/video processing tool"
+    "convert|ImageMagick — image conversion and manipulation"
+    "sqlite3|SQLite database CLI"
+)
+
+created=0
+for entry in "${TOOLS[@]}"; do
+    IFS='|' read -r cmd desc <<< "$entry"
+
+    # Check if command exists
+    if ! command -v "$cmd" &>/dev/null; then
+        continue
+    fi
+
+    skill_name="system-$cmd"
+    skill_dir="$SKILLS_DIR/$skill_name"
+
+    # Skip if already exists
+    [ -f "$skill_dir/SKILL.md" ] && continue
+
+    # Get help text
+    help_text=$($cmd --help 2>&1 | head -30 || echo "No help available")
+
+    mkdir -p "$skill_dir"
+    cat > "$skill_dir/SKILL.md" << EOF
+---
+name: $skill_name
+description: "$desc"
+user-invocable: false
+---
+
+# $cmd
+
+$desc
+
+## Usage
+Run via bash tool: \`$cmd [args]\`
+
+## Help Output
+\`\`\`
+$help_text
+\`\`\`
+EOF
+
+    echo "Created skill: $skill_name ($cmd)"
+    created=$((created + 1))
+done
+
+echo "---"
+echo "Discovered $created new CLI tools as skills"
+echo "Total user skills: $(ls -d "$SKILLS_DIR"/*/SKILL.md 2>/dev/null | wc -l)"