kopipasta 0.2.0__tar.gz → 0.41.0__tar.gz

kopipasta-0.41.0/PKG-INFO

@@ -0,0 +1,140 @@
+Metadata-Version: 2.1
+Name: kopipasta
+Version: 0.41.0
+Summary: A CLI tool to generate prompts with project structure and file contents
+Home-page: https://github.com/mkorpela/kopipasta
+Author: Mikko Korpela
+Author-email: mikko.korpela@gmail.com
+License: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pyperclip==1.9.0
+Requires-Dist: requests==2.32.3
+Requires-Dist: Pygments==2.18.0
+Requires-Dist: rich==13.8.1
+Requires-Dist: click==8.2.1
+Requires-Dist: prompt-toolkit==3.0.52
+
+# kopipasta
+
+[![Version](https://img.shields.io/pypi/v/kopipasta.svg)](https://pypi.python.org/pypi/kopipasta)
+[![Downloads](http://pepy.tech/badge/kopipasta)](http://pepy.tech/project/kopipasta)
+
+**kopipasta bridges the gap between your local file system and LLM context windows.**
+
+A CLI tool for taking **full, transparent control** of your prompt. No black boxes.
+
+<img src="kopipasta.jpg" alt="kopipasta" width="300">
+
+- An LLM told me that "kopi" means Coffee in some languages... and a Diffusion model then made this delicious soup.
+
+## The Philosophy: You Control the Context
+
+Many AI coding assistants use Retrieval-Augmented Generation (RAG) to automatically find what *they think* is relevant context. This is a black box. When the LLM gives a bad answer, you can't debug it because you don't know what context it was actually given.
+
+**`kopipasta` is the opposite.** I built it for myself on the principle of **explicit context control**. You are in the driver's seat. You decide *exactly* what files, functions, and snippets go into the prompt. This transparency is the key to getting reliable, debuggable results from an LLM.
+
+It's a "smart copy" command for your project, not a magic wand.
+
+## How It Works
+
+The workflow is dead simple:
+
+1.  **Gather:** Run `kopipasta` and point it at the files, directories, and URLs that matter for your task.
+2.  **Select:** The tool interactively helps you choose what to include. For large files, you can send just a snippet or even hand-pick individual functions.
+3.  **Define:** Write your instructions to the LLM in an interactive prompt directly in your terminal.
+4.  **Paste:** The final, comprehensive prompt is now on your clipboard, ready to be pasted into ChatGPT, Gemini, Claude, or your LLM of choice.
+5.  **Apply:** Inside the file selector, press `p`, paste the LLM's markdown response, and the tool will automatically patch your local files.
+
+## Installation
+
+```bash
+# Using pipx (recommended for CLI tools)
+pipx install kopipasta
+
+# Or using standard pip
+pip install kopipasta
+```
+
+## Usage
+
+`kopipasta` has two main modes: creating prompts and applying patches.
+
+### Creating a Prompt
+
+```bash
+kopipasta [options] [files_or_directories_or_urls...]
+```
+
+**Arguments:**
+
+*   `[files_or_directories_or_urls...]`: One or more paths to files, directories, or web URLs to use as the starting point for your context.
+
+**Options:**
+
+*   `-t TASK`, `--task TASK`: Provide the task description directly on the command line, skipping the editor.
+
+### Applying Patches
+
+`kopipasta` automatically injects strict instructions into your prompt, teaching the LLM how to format code for this tool.
+`kopipasta` can apply changes suggested by an LLM directly to your codebase, assuming you are in a Git repository.
+
+1.  While running `kopipasta` in the interactive file selector, press the `p` key.
+2.  Paste the entire markdown response from your LLM into the terminal prompt and submit.
+3.  The tool will find code blocks with file paths (e.g., `// FILE: src/main.py`) and immediately write those changes to your local files.
+4.  After applying, use standard Git commands like `git diff` to review the changes before staging and committing them.
+
+## Key Features
+
+*   **Total Context Control:** Interactively select files, directories, or snippets. You see everything that goes into the prompt.
+*   **Smart Dependency Analysis:** Press `d` on a Python or TypeScript/JavaScript file, and `kopipasta` will scan imports to find and add related local files to your context automatically.
+*   **Interactive Code Patcher:** Press `p` in the file selector to paste and apply LLM-suggested changes directly to your local files. Relies on your version control (like Git) for safety, enabling a fast workflow.
+*   **Built-in Search:** Press `g` to grep for text patterns inside directories to find relevant files.
+*   **Transparent & Explicit:** No hidden RAG. You know exactly what's in the prompt because you built it. This makes debugging LLM failures possible.
+*   **Web-Aware:** Pulls in content directly from URLs—perfect for API documentation.
+*   **Safety First:**
+    *   Automatically respects your `.gitignore` rules.
+    *   Detects if you're about to include secrets from a `.env` file and asks what to do.
+*   **Context-Aware:** Keeps a running total of the prompt size (in characters and estimated tokens) so you don't overload the LLM's context window.
+*   **Developer-Friendly:**
+    *   Provides a rich, interactive prompt for writing task descriptions in terminal.
+    *   Copies the final prompt directly to your clipboard.
+    *   Provides syntax highlighting during chunk selection.
+
+## Interactive Controls
+
+| Key | Action |
+| :--- | :--- |
+| `Space` | Toggle file/directory selection |
+| `s` | Toggle **Snippet Mode** (include only the first 50 lines) |
+| `d` | **Analyze Dependencies** (find and add imported files) |
+| `g` | **Grep** (search text in directory) |
+| `a` | Add all files in directory |
+| `p` | **Apply Patch** (paste LLM response) |
+| `r` | Reuse selection from previous run |
+| `Enter` | Expand/Collapse directory |
+| `q` | Quit and finalize selection |
+
+## A Real-World Example
+
+I had a bug where my `setup.py` didn't include all the dependencies from `requirements.txt`.
+
+1.  I ran `kopipasta -t "Update setup.py to read dependencies dynamically from requirements.txt" setup.py requirements.txt`.
+2.  The tool confirmed the inclusion of both files and copied the complete prompt to my clipboard.
+3.  I pasted the prompt into my LLM chat window.
+4.  I copied the LLM's response (which included a modified `setup.py` in a markdown code block).
+5.  Inside `kopipasta`, I pressed `p`, pasted the response, and my local `setup.py` was updated.
+6.  I ran `git diff` to review the changes, then tested and committed.
+
+No manual file reading, no clumsy copy-pasting, just a clean, context-rich prompt that I had full control over, and a seamless way to apply the results.

kopipasta-0.41.0/README.md

@@ -0,0 +1,112 @@
+# kopipasta
+
+[![Version](https://img.shields.io/pypi/v/kopipasta.svg)](https://pypi.python.org/pypi/kopipasta)
+[![Downloads](http://pepy.tech/badge/kopipasta)](http://pepy.tech/project/kopipasta)
+
+**kopipasta bridges the gap between your local file system and LLM context windows.**
+
+A CLI tool for taking **full, transparent control** of your prompt. No black boxes.
+
+<img src="kopipasta.jpg" alt="kopipasta" width="300">
+
+- An LLM told me that "kopi" means Coffee in some languages... and a Diffusion model then made this delicious soup.
+
+## The Philosophy: You Control the Context
+
+Many AI coding assistants use Retrieval-Augmented Generation (RAG) to automatically find what *they think* is relevant context. This is a black box. When the LLM gives a bad answer, you can't debug it because you don't know what context it was actually given.
+
+**`kopipasta` is the opposite.** I built it for myself on the principle of **explicit context control**. You are in the driver's seat. You decide *exactly* what files, functions, and snippets go into the prompt. This transparency is the key to getting reliable, debuggable results from an LLM.
+
+It's a "smart copy" command for your project, not a magic wand.
+
+## How It Works
+
+The workflow is dead simple:
+
+1.  **Gather:** Run `kopipasta` and point it at the files, directories, and URLs that matter for your task.
+2.  **Select:** The tool interactively helps you choose what to include. For large files, you can send just a snippet or even hand-pick individual functions.
+3.  **Define:** Write your instructions to the LLM in an interactive prompt directly in your terminal.
+4.  **Paste:** The final, comprehensive prompt is now on your clipboard, ready to be pasted into ChatGPT, Gemini, Claude, or your LLM of choice.
+5.  **Apply:** Inside the file selector, press `p`, paste the LLM's markdown response, and the tool will automatically patch your local files.
+
+## Installation
+
+```bash
+# Using pipx (recommended for CLI tools)
+pipx install kopipasta
+
+# Or using standard pip
+pip install kopipasta
+```
+
+## Usage
+
+`kopipasta` has two main modes: creating prompts and applying patches.
+
+### Creating a Prompt
+
+```bash
+kopipasta [options] [files_or_directories_or_urls...]
+```
+
+**Arguments:**
+
+*   `[files_or_directories_or_urls...]`: One or more paths to files, directories, or web URLs to use as the starting point for your context.
+
+**Options:**
+
+*   `-t TASK`, `--task TASK`: Provide the task description directly on the command line, skipping the editor.
+
+### Applying Patches
+
+`kopipasta` automatically injects strict instructions into your prompt, teaching the LLM how to format code for this tool.
+`kopipasta` can apply changes suggested by an LLM directly to your codebase, assuming you are in a Git repository.
+
+1.  While running `kopipasta` in the interactive file selector, press the `p` key.
+2.  Paste the entire markdown response from your LLM into the terminal prompt and submit.
+3.  The tool will find code blocks with file paths (e.g., `// FILE: src/main.py`) and immediately write those changes to your local files.
+4.  After applying, use standard Git commands like `git diff` to review the changes before staging and committing them.
+
+## Key Features
+
+*   **Total Context Control:** Interactively select files, directories, or snippets. You see everything that goes into the prompt.
+*   **Smart Dependency Analysis:** Press `d` on a Python or TypeScript/JavaScript file, and `kopipasta` will scan imports to find and add related local files to your context automatically.
+*   **Interactive Code Patcher:** Press `p` in the file selector to paste and apply LLM-suggested changes directly to your local files. Relies on your version control (like Git) for safety, enabling a fast workflow.
+*   **Built-in Search:** Press `g` to grep for text patterns inside directories to find relevant files.
+*   **Transparent & Explicit:** No hidden RAG. You know exactly what's in the prompt because you built it. This makes debugging LLM failures possible.
+*   **Web-Aware:** Pulls in content directly from URLs—perfect for API documentation.
+*   **Safety First:**
+    *   Automatically respects your `.gitignore` rules.
+    *   Detects if you're about to include secrets from a `.env` file and asks what to do.
+*   **Context-Aware:** Keeps a running total of the prompt size (in characters and estimated tokens) so you don't overload the LLM's context window.
+*   **Developer-Friendly:**
+    *   Provides a rich, interactive prompt for writing task descriptions in terminal.
+    *   Copies the final prompt directly to your clipboard.
+    *   Provides syntax highlighting during chunk selection.
+
+## Interactive Controls
+
+| Key | Action |
+| :--- | :--- |
+| `Space` | Toggle file/directory selection |
+| `s` | Toggle **Snippet Mode** (include only the first 50 lines) |
+| `d` | **Analyze Dependencies** (find and add imported files) |
+| `g` | **Grep** (search text in directory) |
+| `a` | Add all files in directory |
+| `p` | **Apply Patch** (paste LLM response) |
+| `r` | Reuse selection from previous run |
+| `Enter` | Expand/Collapse directory |
+| `q` | Quit and finalize selection |
+
+## A Real-World Example
+
+I had a bug where my `setup.py` didn't include all the dependencies from `requirements.txt`.
+
+1.  I ran `kopipasta -t "Update setup.py to read dependencies dynamically from requirements.txt" setup.py requirements.txt`.
+2.  The tool confirmed the inclusion of both files and copied the complete prompt to my clipboard.
+3.  I pasted the prompt into my LLM chat window.
+4.  I copied the LLM's response (which included a modified `setup.py` in a markdown code block).
+5.  Inside `kopipasta`, I pressed `p`, pasted the response, and my local `setup.py` was updated.
+6.  I ran `git diff` to review the changes, then tested and committed.
+
+No manual file reading, no clumsy copy-pasting, just a clean, context-rich prompt that I had full control over, and a seamless way to apply the results.

kopipasta-0.41.0/kopipasta/cache.py

@@ -0,0 +1,40 @@
+import json
+import os
+from pathlib import Path
+from typing import List, Tuple
+
+# Define FileTuple for type hinting
+FileTuple = Tuple[str, bool, List[str] | None, str]
+
+
+def get_cache_file_path() -> Path:
+    """Gets the cross-platform path to the cache file for the last selection."""
+    cache_dir = Path.home() / ".cache" / "kopipasta"
+    cache_dir.mkdir(parents=True, exist_ok=True)
+    return cache_dir / "last_selection.json"
+
+
+def save_selection_to_cache(files_to_include: List[FileTuple]):
+    """Saves the list of selected file relative paths to the cache."""
+    cache_file = get_cache_file_path()
+    relative_paths = sorted([os.path.relpath(f[0]) for f in files_to_include])
+    try:
+        with open(cache_file, "w", encoding="utf-8") as f:
+            json.dump(relative_paths, f, indent=2)
+    except IOError as e:
+        print(f"\nWarning: Could not save selection to cache: {e}")
+
+
+def load_selection_from_cache() -> List[str]:
+    """Loads the list of selected files from the cache file."""
+    cache_file = get_cache_file_path()
+    if not cache_file.exists():
+        return []
+    try:
+        with open(cache_file, "r", encoding="utf-8") as f:
+            paths = json.load(f)
+            # Filter out paths that no longer exist
+            return [p for p in paths if os.path.exists(p)]
+    except (IOError, json.JSONDecodeError) as e:
+        print(f"\nWarning: Could not load previous selection from cache: {e}")
+        return []

kopipasta-0.41.0/kopipasta/file.py

@@ -0,0 +1,322 @@
+import fnmatch
+import os
+from typing import List, Optional, Tuple, Set
+from pathlib import Path
+
+FileTuple = Tuple[str, bool, Optional[List[str]], str]
+
+# --- Caches ---
+_gitignore_cache: dict[str, list[str]] = {}
+_is_ignored_cache: dict[str, bool] = {}
+_is_binary_cache: dict[str, bool] = {}
+
+# --- Known File Extensions for is_binary ---
+# Using sets for O(1) average time complexity lookups
+TEXT_EXTENSIONS = {
+    # Code
+    ".py",
+    ".js",
+    ".ts",
+    ".jsx",
+    ".tsx",
+    ".java",
+    ".c",
+    ".cpp",
+    ".h",
+    ".hpp",
+    ".cs",
+    ".go",
+    ".rs",
+    ".sh",
+    ".bash",
+    ".ps1",
+    ".rb",
+    ".php",
+    ".swift",
+    ".kt",
+    ".kts",
+    ".scala",
+    ".pl",
+    ".pm",
+    ".tcl",
+    # Markup & Data
+    ".html",
+    ".htm",
+    ".xml",
+    ".css",
+    ".scss",
+    ".sass",
+    ".less",
+    ".json",
+    ".yaml",
+    ".yml",
+    ".toml",
+    ".ini",
+    ".cfg",
+    ".conf",
+    ".md",
+    ".txt",
+    ".rtf",
+    ".csv",
+    ".tsv",
+    ".sql",
+    ".graphql",
+    ".gql",
+    # Config & Other
+    ".gitignore",
+    ".dockerfile",
+    "dockerfile",
+    ".env",
+    ".properties",
+    ".mdx",
+}
+
+BINARY_EXTENSIONS = {
+    # Images
+    ".png",
+    ".jpg",
+    ".jpeg",
+    ".gif",
+    ".bmp",
+    ".tiff",
+    ".ico",
+    ".webp",
+    ".svg",
+    # Audio/Video
+    ".mp3",
+    ".wav",
+    ".ogg",
+    ".flac",
+    ".mp4",
+    ".avi",
+    ".mov",
+    ".wmv",
+    ".mkv",
+    # Archives
+    ".zip",
+    ".rar",
+    ".7z",
+    ".tar",
+    ".gz",
+    ".bz2",
+    ".xz",
+    # Documents
+    ".pdf",
+    ".doc",
+    ".docx",
+    ".xls",
+    ".xlsx",
+    ".ppt",
+    ".pptx",
+    ".odt",
+    # Executables & Compiled
+    ".exe",
+    ".dll",
+    ".so",
+    ".dylib",
+    ".class",
+    ".jar",
+    ".pyc",
+    ".pyd",
+    ".whl",
+    # Databases & Other
+    ".db",
+    ".sqlite",
+    ".sqlite3",
+    ".db-wal",
+    ".db-shm",
+    ".lock",
+    ".bak",
+    ".swo",
+    ".swp",
+}
+
+
+def _read_gitignore_patterns(gitignore_path: str) -> list[str]:
+    """Reads patterns from a single .gitignore file and caches them."""
+    if gitignore_path in _gitignore_cache:
+        return _gitignore_cache[gitignore_path]
+    if not os.path.isfile(gitignore_path):
+        _gitignore_cache[gitignore_path] = []
+        return []
+    patterns = []
+    try:
+        with open(gitignore_path, "r", encoding="utf-8") as f:
+            for line in f:
+                stripped_line = line.strip()
+                if stripped_line and not stripped_line.startswith("#"):
+                    patterns.append(stripped_line)
+    except IOError:
+        pass
+    _gitignore_cache[gitignore_path] = patterns
+    return patterns
+
+
+def is_ignored(
+    path: str, default_ignore_patterns: list[str], project_root: Optional[str] = None
+) -> bool:
+    """
+    Checks if a path should be ignored by splitting patterns into fast (basename)
+    and slow (full path) checks, with heavy caching and optimized inner loops.
+    """
+    path_abs = os.path.abspath(path)
+    if path_abs in _is_ignored_cache:
+        return _is_ignored_cache[path_abs]
+
+    parent_dir = os.path.dirname(path_abs)
+    if parent_dir != path_abs and _is_ignored_cache.get(parent_dir, False):
+        _is_ignored_cache[path_abs] = True
+        return True
+
+    if project_root is None:
+        project_root = os.getcwd()
+    project_root_abs = os.path.abspath(project_root)
+
+    basename_patterns, path_patterns = get_all_patterns(
+        default_ignore_patterns, path_abs, project_root_abs
+    )
+
+    # --- Step 1: Fast check for basename patterns ---
+    path_basename = os.path.basename(path_abs)
+    for pattern in basename_patterns:
+        if fnmatch.fnmatch(path_basename, pattern):
+            _is_ignored_cache[path_abs] = True
+            return True
+
+    # --- Step 2: Optimized nested check for path patterns ---
+    try:
+        path_rel_to_root = os.path.relpath(path_abs, project_root_abs)
+    except ValueError:
+        _is_ignored_cache[path_abs] = False
+        return False
+
+    # Pre-calculate all path prefixes to check, avoiding re-joins in the loop.
+    path_parts = Path(path_rel_to_root).parts
+    path_prefixes = [
+        os.path.join(*path_parts[:i]) for i in range(1, len(path_parts) + 1)
+    ]
+
+    # Pre-process patterns to remove trailing slashes once.
+    processed_path_patterns = [p.rstrip("/") for p in path_patterns]
+
+    for prefix in path_prefixes:
+        for pattern in processed_path_patterns:
+            if fnmatch.fnmatch(prefix, pattern):
+                _is_ignored_cache[path_abs] = True
+                return True
+
+    _is_ignored_cache[path_abs] = False
+    return False
+
+
+def get_all_patterns(
+    default_ignore_patterns, path_abs, project_root_abs
+) -> Tuple[Set[str], Set[str]]:
+    """
+    Gathers all applicable ignore patterns, splitting them into two sets
+    for optimized checking: one for basenames, one for full paths.
+    """
+    basename_patterns = set()
+    path_patterns = set()
+
+    for p in default_ignore_patterns:
+        if "/" in p:
+            path_patterns.add(p)
+        else:
+            basename_patterns.add(p)
+
+    search_start_dir = (
+        path_abs if os.path.isdir(path_abs) else os.path.dirname(path_abs)
+    )
+
+    current_dir = search_start_dir
+    while True:
+        gitignore_path = os.path.join(current_dir, ".gitignore")
+        patterns_from_file = _read_gitignore_patterns(gitignore_path)
+
+        if patterns_from_file:
+            gitignore_dir_rel = os.path.relpath(current_dir, project_root_abs)
+            if gitignore_dir_rel == ".":
+                gitignore_dir_rel = ""
+
+            for p in patterns_from_file:
+                if "/" in p:
+                    # Path patterns are relative to the .gitignore file's location
+                    path_patterns.add(os.path.join(gitignore_dir_rel, p.lstrip("/")))
+                else:
+                    basename_patterns.add(p)
+
+        if (
+            not current_dir.startswith(project_root_abs)
+            or current_dir == project_root_abs
+        ):
+            break
+        parent = os.path.dirname(current_dir)
+        if parent == current_dir:
+            break
+        current_dir = parent
+    return basename_patterns, path_patterns
+
+
+def read_file_contents(file_path):
+    try:
+        with open(file_path, "r", encoding="utf-8") as file:
+            return file.read()
+    except (IOError, UnicodeDecodeError) as e:
+        failure = f"Error reading {file_path}: {e}"
+        print(failure)
+        return f"<.. {failure} ..>"
+
+
+def is_binary(file_path: str) -> bool:
+    """
+    Efficiently checks if a file is binary.
+
+    The check follows a fast, multi-step process to minimize I/O:
+    1. Checks a memory cache for a previously determined result.
+    2. Checks the file extension against a list of known text file types.
+    3. Checks the file extension against a list of known binary file types.
+    4. As a last resort, reads the first 512 bytes of the file to check for
+       a null byte, a common indicator of a binary file.
+    """
+    # Step 1: Check cache first for fastest response
+    if file_path in _is_binary_cache:
+        return _is_binary_cache[file_path]
+
+    # Step 2: Fast check based on known text/binary extensions (no I/O)
+    _, extension = os.path.splitext(file_path)
+    extension = extension.lower()
+
+    if extension in TEXT_EXTENSIONS:
+        _is_binary_cache[file_path] = False
+        return False
+    if extension in BINARY_EXTENSIONS:
+        _is_binary_cache[file_path] = True
+        return True
+
+    # Step 3: Fallback to content analysis for unknown extensions
+    try:
+        with open(file_path, "rb") as file:
+            # Read a smaller chunk, 512 bytes is usually enough to find a null byte
+            chunk = file.read(512)
+            if b"\0" in chunk:
+                _is_binary_cache[file_path] = True
+                return True
+            # If no null byte, assume it's a text file
+            _is_binary_cache[file_path] = False
+            return False
+    except IOError:
+        # If we can't open it, treat it as binary to be safe
+        _is_binary_cache[file_path] = True
+        return True
+
+
+def get_human_readable_size(size):
+    for unit in ["B", "KB", "MB", "GB", "TB"]:
+        if size < 1024.0:
+            return f"{size:.2f} {unit}"
+        size /= 1024.0
+
+
+def is_large_file(file_path, threshold=102400):
+    return os.path.getsize(file_path) > threshold