promptdiff-cli 0.1.0__tar.gz

promptdiff_cli-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Raguram R
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

promptdiff_cli-0.1.0/PKG-INFO

@@ -0,0 +1,166 @@
+Metadata-Version: 2.4
+Name: promptdiff-cli
+Version: 0.1.0
+Summary: Git for prompts - version, diff, and test your LLM prompts across any model
+Author: Raguram R
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/Ragu3175/promptdiff
+Project-URL: Repository, https://github.com/Ragu3175/promptdiff
+Project-URL: Issues, https://github.com/Ragu3175/promptdiff/issues
+Keywords: llm,prompt-engineering,prompt-versioning,cli,litellm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Topic :: Software Development :: Version Control
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: fastapi>=0.110.0
+Requires-Dist: uvicorn>=0.28.0
+Requires-Dist: sqlalchemy>=2.0.0
+Requires-Dist: alembic>=1.13.0
+Requires-Dist: typer>=0.9.0
+Requires-Dist: rich>=13.7.0
+Requires-Dist: litellm>=1.30.0
+Requires-Dist: python-dotenv>=1.0.0
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0.0; extra == "dev"
+Requires-Dist: pytest-mock; extra == "dev"
+Dynamic: license-file
+
+# PromptDiff
+
+**Git for prompts.** Version your LLM prompts, diff them word-by-word, and run any version against Gemini, Groq, or any other LiteLLM-supported model — all from the terminal.
+
+```bash
+promptdiff commit summarizer "Summarize: {text}" -m "v1: baseline"
+promptdiff commit summarizer "Summarize concisely in one sentence: {text}" -m "v2: tighter constraint"
+promptdiff diff summarizer 1 2
+promptdiff run summarizer 2 --model gemini/gemini-2.5-flash
+```
+
+## Why PromptDiff
+
+If you've ever overwritten a prompt and lost the version that actually worked, or pasted two prompt drafts into a doc just to eyeball what changed — PromptDiff is the tool for that. It treats prompts as versioned, diffable artifacts the same way Git treats code, and adds one thing Git can't: running any version against a real LLM and recording exactly what it produced, so you can trace output back to the exact prompt and model that generated it.
+
+## How it's different from other prompt tools
+
+The prompt-tooling space has grown a lot — [Promptfoo](https://github.com/promptfoo/promptfoo) is a mature, YAML-config-based open-source tool with strong CI/eval integration, and platforms like PromptLayer, Langfuse, and Braintrust offer hosted, full-lifecycle prompt management with branching, RBAC, and observability.
+
+PromptDiff isn't trying to compete with those. It's deliberately smaller: a local-first CLI for a single developer iterating on prompts, with zero config files, zero hosted accounts, and zero cost beyond free-tier LLM API usage. If you outgrow it — multiple collaborators, non-engineer prompt editors, compliance requirements — those other tools are the right next step. If you're a solo developer who just wants `git commit`-style version control for prompts without standing up infrastructure, PromptDiff is built for exactly that.
+
+## Features
+
+- **Version control for prompts** — every `commit` is an immutable, numbered version (v1, v2, v3...) scoped per prompt
+- **Word-level diffs** — see exactly which words changed between versions, rendered with color in your terminal (green additions, red strikethrough removals)
+- **Multi-model runs** — execute any prompt version against multiple LLMs in a single command and compare cost, latency, and output side by side
+- **Output comparison** — `diff-output` shows what two versions actually *produced*, not just what their text looks like
+- **Free-tier friendly** — built and tested against Gemini and Groq's free tiers; rate-limit and quota failures are recorded gracefully, never crash the tool
+- **Local-first** — everything lives in a local SQLite database, no account or server required
+
+## Installation
+
+```bash
+pip install promptdiff-cli
+```
+
+(Or, to run from source — see [Development](#development) below.)
+
+## Quick start
+
+```bash
+# Create a project and switch to it
+promptdiff project create my-app
+promptdiff use my-app
+
+# Create a prompt and commit your first version
+promptdiff add summarizer
+promptdiff commit summarizer "Summarize this in one sentence: {text}" -m "v1: baseline"
+
+# Edit and commit a new version
+promptdiff commit summarizer "Summarize concisely, max 20 words: {text}" -m "v2: tighter constraint"
+
+# See version history
+promptdiff log summarizer
+
+# See exactly what changed
+promptdiff diff summarizer 1 2
+
+# Run a version against a model (needs an API key — see below)
+promptdiff run summarizer 2 --model gemini/gemini-2.5-flash
+
+# Run the same version against multiple models at once
+promptdiff run summarizer 2 --model gemini/gemini-2.5-flash --model groq/llama-3.3-70b-versatile
+
+# Compare what two versions actually produced
+promptdiff diff-output summarizer 1 2 --model gemini/gemini-2.5-flash
+```
+
+## API keys
+
+PromptDiff uses [LiteLLM](https://github.com/BerriAI/litellm) under the hood, so it works with any LiteLLM-supported provider. It's built and tested primarily against free tiers:
+
+- **Gemini** — get a free key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey)
+- **Groq** — get a free key at [console.groq.com/keys](https://console.groq.com/keys)
+
+Copy `.env.example` to `.env` in your project directory and fill in the keys you have:
+
+```
+GEMINI_API_KEY=your_key_here
+GROQ_API_KEY=your_key_here
+```
+
+## Commands
+
+| Command | What it does |
+|---|---|
+| `promptdiff project create <name>` | Create a new project |
+| `promptdiff project list` | List all projects |
+| `promptdiff use <project>` | Set the current project (so you don't need `--project` on every command) |
+| `promptdiff add <prompt>` | Add a new prompt to the current project |
+| `promptdiff commit <prompt> <content_or_file> -m "<message>"` | Commit a new version. Accepts a literal string or a path to a file |
+| `promptdiff log <prompt>` | Show version history |
+| `promptdiff diff <prompt> <v1> <v2>` | Word-level diff between two versions |
+| `promptdiff run <prompt> <version> --model <model>` | Run a version against one or more models (repeat `--model` to run against several at once) |
+| `promptdiff diff-output <prompt> <v1> <v2> [--model <model>]` | Compare what two versions actually produced. If a version was run against multiple models, `--model` is required to avoid an ambiguous comparison |
+
+Every command accepts `-p` / `--project` to override the current project for that one call.
+
+## Architecture
+
+- **Database**: SQLite, 5 tables (`projects`, `prompts`, `versions`, `runs`, `outputs`) via SQLAlchemy + Alembic migrations
+- **Diff engine**: Python's `difflib`, line-level structure with word-level precision inside changed lines
+- **Runner**: [LiteLLM](https://github.com/BerriAI/litellm) for provider-agnostic model calls; failures (rate limits, timeouts, bad keys) are recorded as failed runs rather than crashing the CLI
+- **CLI**: [Typer](https://typer.tiangolo.com/) + [Rich](https://github.com/Textualize/rich)
+
+## Development
+
+```bash
+git clone https://github.com/Ragu3175/promptdiff.git
+cd promptdiff
+python -m venv venv
+venv\Scripts\activate   # Windows
+# source venv/bin/activate   # macOS/Linux
+pip install -r requirements.txt
+pip install -e .
+pytest -v
+```
+
+## Status
+
+Core CLI is complete and tested: `commit`, `log`, `diff`, `run`, and `diff-output` all work end-to-end against real Gemini and Groq calls. A REST API layer and web UI are planned but not yet built — the CLI is fully usable on its own today.
+
+## Contributing
+
+Issues and PRs welcome. This is an early-stage solo project — if you run into something broken or have an idea, please open an issue.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+## Author
+
+Built by [Raguram R](https://github.com/Ragu3175).

promptdiff_cli-0.1.0/README.md

@@ -0,0 +1,133 @@
+# PromptDiff
+
+**Git for prompts.** Version your LLM prompts, diff them word-by-word, and run any version against Gemini, Groq, or any other LiteLLM-supported model — all from the terminal.
+
+```bash
+promptdiff commit summarizer "Summarize: {text}" -m "v1: baseline"
+promptdiff commit summarizer "Summarize concisely in one sentence: {text}" -m "v2: tighter constraint"
+promptdiff diff summarizer 1 2
+promptdiff run summarizer 2 --model gemini/gemini-2.5-flash
+```
+
+## Why PromptDiff
+
+If you've ever overwritten a prompt and lost the version that actually worked, or pasted two prompt drafts into a doc just to eyeball what changed — PromptDiff is the tool for that. It treats prompts as versioned, diffable artifacts the same way Git treats code, and adds one thing Git can't: running any version against a real LLM and recording exactly what it produced, so you can trace output back to the exact prompt and model that generated it.
+
+## How it's different from other prompt tools
+
+The prompt-tooling space has grown a lot — [Promptfoo](https://github.com/promptfoo/promptfoo) is a mature, YAML-config-based open-source tool with strong CI/eval integration, and platforms like PromptLayer, Langfuse, and Braintrust offer hosted, full-lifecycle prompt management with branching, RBAC, and observability.
+
+PromptDiff isn't trying to compete with those. It's deliberately smaller: a local-first CLI for a single developer iterating on prompts, with zero config files, zero hosted accounts, and zero cost beyond free-tier LLM API usage. If you outgrow it — multiple collaborators, non-engineer prompt editors, compliance requirements — those other tools are the right next step. If you're a solo developer who just wants `git commit`-style version control for prompts without standing up infrastructure, PromptDiff is built for exactly that.
+
+## Features
+
+- **Version control for prompts** — every `commit` is an immutable, numbered version (v1, v2, v3...) scoped per prompt
+- **Word-level diffs** — see exactly which words changed between versions, rendered with color in your terminal (green additions, red strikethrough removals)
+- **Multi-model runs** — execute any prompt version against multiple LLMs in a single command and compare cost, latency, and output side by side
+- **Output comparison** — `diff-output` shows what two versions actually *produced*, not just what their text looks like
+- **Free-tier friendly** — built and tested against Gemini and Groq's free tiers; rate-limit and quota failures are recorded gracefully, never crash the tool
+- **Local-first** — everything lives in a local SQLite database, no account or server required
+
+## Installation
+
+```bash
+pip install promptdiff-cli
+```
+
+(Or, to run from source — see [Development](#development) below.)
+
+## Quick start
+
+```bash
+# Create a project and switch to it
+promptdiff project create my-app
+promptdiff use my-app
+
+# Create a prompt and commit your first version
+promptdiff add summarizer
+promptdiff commit summarizer "Summarize this in one sentence: {text}" -m "v1: baseline"
+
+# Edit and commit a new version
+promptdiff commit summarizer "Summarize concisely, max 20 words: {text}" -m "v2: tighter constraint"
+
+# See version history
+promptdiff log summarizer
+
+# See exactly what changed
+promptdiff diff summarizer 1 2
+
+# Run a version against a model (needs an API key — see below)
+promptdiff run summarizer 2 --model gemini/gemini-2.5-flash
+
+# Run the same version against multiple models at once
+promptdiff run summarizer 2 --model gemini/gemini-2.5-flash --model groq/llama-3.3-70b-versatile
+
+# Compare what two versions actually produced
+promptdiff diff-output summarizer 1 2 --model gemini/gemini-2.5-flash
+```
+
+## API keys
+
+PromptDiff uses [LiteLLM](https://github.com/BerriAI/litellm) under the hood, so it works with any LiteLLM-supported provider. It's built and tested primarily against free tiers:
+
+- **Gemini** — get a free key at [aistudio.google.com/apikey](https://aistudio.google.com/apikey)
+- **Groq** — get a free key at [console.groq.com/keys](https://console.groq.com/keys)
+
+Copy `.env.example` to `.env` in your project directory and fill in the keys you have:
+
+```
+GEMINI_API_KEY=your_key_here
+GROQ_API_KEY=your_key_here
+```
+
+## Commands
+
+| Command | What it does |
+|---|---|
+| `promptdiff project create <name>` | Create a new project |
+| `promptdiff project list` | List all projects |
+| `promptdiff use <project>` | Set the current project (so you don't need `--project` on every command) |
+| `promptdiff add <prompt>` | Add a new prompt to the current project |
+| `promptdiff commit <prompt> <content_or_file> -m "<message>"` | Commit a new version. Accepts a literal string or a path to a file |
+| `promptdiff log <prompt>` | Show version history |
+| `promptdiff diff <prompt> <v1> <v2>` | Word-level diff between two versions |
+| `promptdiff run <prompt> <version> --model <model>` | Run a version against one or more models (repeat `--model` to run against several at once) |
+| `promptdiff diff-output <prompt> <v1> <v2> [--model <model>]` | Compare what two versions actually produced. If a version was run against multiple models, `--model` is required to avoid an ambiguous comparison |
+
+Every command accepts `-p` / `--project` to override the current project for that one call.
+
+## Architecture
+
+- **Database**: SQLite, 5 tables (`projects`, `prompts`, `versions`, `runs`, `outputs`) via SQLAlchemy + Alembic migrations
+- **Diff engine**: Python's `difflib`, line-level structure with word-level precision inside changed lines
+- **Runner**: [LiteLLM](https://github.com/BerriAI/litellm) for provider-agnostic model calls; failures (rate limits, timeouts, bad keys) are recorded as failed runs rather than crashing the CLI
+- **CLI**: [Typer](https://typer.tiangolo.com/) + [Rich](https://github.com/Textualize/rich)
+
+## Development
+
+```bash
+git clone https://github.com/Ragu3175/promptdiff.git
+cd promptdiff
+python -m venv venv
+venv\Scripts\activate   # Windows
+# source venv/bin/activate   # macOS/Linux
+pip install -r requirements.txt
+pip install -e .
+pytest -v
+```
+
+## Status
+
+Core CLI is complete and tested: `commit`, `log`, `diff`, `run`, and `diff-output` all work end-to-end against real Gemini and Groq calls. A REST API layer and web UI are planned but not yet built — the CLI is fully usable on its own today.
+
+## Contributing
+
+Issues and PRs welcome. This is an early-stage solo project — if you run into something broken or have an idea, please open an issue.
+
+## License
+
+MIT — see [LICENSE](LICENSE).
+
+## Author
+
+Built by [Raguram R](https://github.com/Ragu3175).

promptdiff_cli-0.1.0/promptdiff/__init__.py

@@ -0,0 +1 @@
+# PromptDiff package

promptdiff_cli-0.1.0/promptdiff/api/__init__.py

@@ -0,0 +1 @@
+# API package

promptdiff_cli-0.1.0/promptdiff/cli/__init__.py

@@ -0,0 +1 @@
+# CLI package

promptdiff_cli-0.1.0/promptdiff/cli/context.py

@@ -0,0 +1,52 @@
+import os
+import json
+from pathlib import Path
+from contextlib import contextmanager
+from typing import Generator
+from sqlalchemy.orm import Session
+import typer
+
+from promptdiff.db.session import get_db
+
+def get_config_dir() -> Path:
+    """Get the configuration directory path, allowing environment override for tests."""
+    env_home = os.environ.get("PROMPTDIFF_HOME")
+    if env_home:
+        return Path(env_home)
+    return Path.home() / ".promptdiff"
+
+def get_config_path() -> Path:
+    """Get the path to the config.json file."""
+    return get_config_dir() / "config.json"
+
+def resolve_project(explicit: str | None) -> str:
+    """Resolve the project name using explicit override or current project config.
+    
+    Raises typer.Exit(code=1) if no project name can be resolved.
+    """
+    if explicit:
+        return explicit
+
+    config_file = get_config_path()
+    if config_file.is_file():
+        try:
+            with open(config_file, "r") as f:
+                data = json.load(f)
+                current = data.get("current_project")
+                if current:
+                    return current
+        except Exception:
+            pass
+
+    typer.secho(
+        "Error: No project specified. Use --project or run `promptdiff use <project>` first.",
+        fg=typer.colors.RED,
+        err=True
+    )
+    raise typer.Exit(code=1)
+
+@contextmanager
+def get_cli_db() -> Generator[Session, None, None]:
+    """Yield a database session and guarantee it closes correctly."""
+    with get_db() as db:
+        yield db