promptcapsule 0.1.0__tar.gz

promptcapsule-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Udaya Nirogi
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

promptcapsule-0.1.0/PKG-INFO

@@ -0,0 +1,609 @@
+Metadata-Version: 2.4
+Name: promptcapsule
+Version: 0.1.0
+Summary: Open-source prompt compression & retrieval library
+Author-email: Udaya Nirogi <udaya@example.com>
+License: MIT
+Project-URL: Homepage, https://github.com/UdayaNirogi/promptcapsule
+Project-URL: Bug Tracker, https://github.com/UdayaNirogi/promptcapsule/issues
+Keywords: prompt,compression,retrieval,ai,llm
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.8
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0; extra == "dev"
+Requires-Dist: black>=23.0; extra == "dev"
+Requires-Dist: flake8>=6.0; extra == "dev"
+Requires-Dist: mypy>=1.0; extra == "dev"
+Requires-Dist: twine>=4.0; extra == "dev"
+Provides-Extra: vault
+Requires-Dist: boto3>=1.26; extra == "vault"
+Requires-Dist: pygithub>=1.59; extra == "vault"
+Dynamic: license-file
+
+# PromptCapsule 📦
+
+> Open-source prompt compression & retrieval library that solves a real problem for developers and AI enthusiasts.
+
+## Overview
+
+**PromptCapsule** is a generic, reusable, open-source tool for compressing and reliably reconstructing LLM prompts. It bridges the gap between short-prompt compression and long-prompt storage, giving developers an honest, transparent way to share, version-control, and manage their AI prompts.
+
+### The Problem
+
+When people talk about "prompt compression," they usually mean two fundamentally different things:
+
+| Type | What It Does | Trade-off |
+|------|-------------|-----------|
+| **Type 1: True Compression** | Mathematically limited to ~500-word prompts → 8 characters (lossless) | No external storage needed, but very limited scope |
+| **Type 2: Key-Based Retrieval** | 2,000-word prompt → 8-character hash that reconstructs the original (from any device, any account) | Requires a storage backend, but solves the real problem |
+
+**PromptCapsule** gives you **BOTH**, transparently and automatically.
+
+### The Solution
+
+PromptCapsule uses a **hybrid approach**:
+
+```
+Short prompts (<500 chars):
+  INPUT PROMPT → ZLIB COMPRESS → BASE85 ENCODE → Portable capsule string
+  (self-contained, no storage needed)
+
+Long prompts (>500 chars):
+  INPUT PROMPT → STORE IN VAULT (SQLite, GitHub Gist, S3) → Generate short hash key
+  (retrieves from configured backend)
+```
+
+Both modes are **automatic** — just compress once and the library picks the best strategy. Both are **verified** — checksums ensure byte-for-byte exact reconstruction.
+
+---
+
+## Features
+
+✨ **Hybrid Compression**
+- Short prompts: Inline zlib + Base85 compression
+- Long prompts: Vault storage with automatic fallback
+- Automatic mode selection based on size
+
+🔒 **Integrity Verification**
+- SHA256 checksums for all capsules
+- Byte-for-byte exact reconstruction guarantee
+- Verification status in decompression results
+
+🎯 **Honest Positioning**
+- Transparent about what it actually does (capsule strings, not LLM output compression)
+- Clear comparisons vs. existing tools (LLMLingua, LangChain Hub, gzip)
+- No false claims of novelty
+
+🌍 **Pluggable Backends**
+- **In-Memory**: Great for testing and prototyping
+- **SQLite**: Local storage, no external dependencies
+- **GitHub Gist**: Cloud storage, version control friendly
+- **AWS S3**: Enterprise-grade scalability (optional)
+
+⚙️ **Python 3.8+**
+- Pure Python, minimal dependencies
+- Cross-platform compatible
+- Type hints throughout
+
+---
+
+## Installation
+
+```bash
+# Basic installation (includes in-memory and SQLite backends)
+pip install promptcapsule
+
+# With cloud storage support
+pip install promptcapsule[vault]
+```
+
+### Requirements
+- Python 3.8+
+- No external dependencies for core functionality
+- Optional: `boto3` for S3 backend, `PyGithub` for GitHub Gist backend
+
+---
+
+## Quick Start
+
+### Basic Usage (Short Prompts)
+
+```python
+from promptcapsule import PromptCapsule
+
+pc = PromptCapsule()
+
+# Compress a prompt
+prompt = "You are a helpful Python coding assistant."
+capsule = pc.compress(prompt)
+# capsule: "cap_i_a1b2c3d4_K*i0?5Z7....."
+
+# Decompress anywhere, on any device/account
+result = pc.decompress(capsule)
+assert result.text == prompt
+assert result.verified is True  # Integrity verified ✓
+```
+
+### Long Prompts with Vault Backend
+
+```python
+from promptcapsule import PromptCapsule
+from promptcapsule.backends import SQLiteBackend
+
+pc = PromptCapsule()
+vault = SQLiteBackend("prompts.db")
+
+# Compress a long, carefully-crafted prompt
+long_prompt = """
+You are an expert in machine learning...
+[2000+ characters of detailed context]
+"""
+
+capsule = pc.compress(long_prompt, vault_backend=vault)
+# capsule: "cap_v_a1b2c3d4_sql_20240921_120000_0001"
+
+# Decompress later (data retrieves from SQLite)
+result = pc.decompress(capsule, vault_backend=vault)
+assert result.text == long_prompt
+assert result.verified is True
+```
+
+### Real-World Scenario: Version-Controlling Prompts
+
+```python
+import json
+from promptcapsule import PromptCapsule
+from promptcapsule.backends import SQLiteBackend
+
+pc = PromptCapsule()
+vault = SQLiteBackend("prompts.db")
+
+# Iterate on your prompts over time
+prompts_history = {
+    "v1.0": pc.compress(
+        "Generate a blog post about AI",
+        vault_backend=vault
+    ),
+    "v1.1": pc.compress(
+        "Generate a blog post about AI, focused on practical applications",
+        vault_backend=vault
+    ),
+    "v1.2": pc.compress(
+        "Generate a technical blog post about AI/ML, 2000+ words, with code examples",
+        vault_backend=vault
+    ),
+}
+
+# Save to git
+with open("prompts.json", "w") as f:
+    json.dump(prompts_history, f)
+
+# Later (or different branch), retrieve and decompress
+with open("prompts.json", "r") as f:
+    history = json.load(f)
+
+for version, capsule in history.items():
+    result = pc.decompress(capsule, vault_backend=vault)
+    print(f"{version}: {result.text[:50]}...")
+```
+
+---
+
+## How It Works
+
+### Inline Mode (Short Prompts)
+
+1. **Compress**: `prompt` → zlib (level 9) → Base85 encode → capsule
+2. **Format**: `cap_i_<8-char checksum>_<encoded data>`
+3. **Verify**: Decompress → compare checksum prefix
+4. **Result**: Portable, self-contained capsule string
+
+**Example**:
+```
+Original: "Hello world!" (12 bytes)
+Capsule:  "cap_i_2f575c63_EZxgxD/'" (28 bytes)
+Ratio:    ~2.3x (expected for very short content)
+```
+
+### Vault Mode (Long Prompts)
+
+1. **Store**: `prompt` → Save to backend (SQLite/Gist/S3) → Generate key
+2. **Format**: `cap_v_<8-char checksum>_<backend key>`
+3. **Retrieve**: On decompress → fetch from backend using key
+4. **Verify**: Compare checksum prefix with retrieved content
+
+**Example**:
+```
+Original:  "You are a senior software architect..." (2,000 chars)
+Capsule:   "cap_v_a1b2c3d4_sql_20240921_120000_0001" (41 chars)
+Ratio:     ~48x compression (key is permanent pointer to vault)
+```
+
+### Automatic Mode Selection
+
+```python
+pc = PromptCapsule()
+
+# Anything ≤ 500 bytes uses inline
+pc.compress("short prompt")  # cap_i_...
+
+# Anything > 500 bytes needs vault
+pc.compress("A" * 600)  # Error! Need vault_backend=
+
+# With vault, auto-selects best mode
+pc.compress("short prompt", vault_backend=backend)  # Still cap_i_...
+pc.compress("A" * 600, vault_backend=backend)       # cap_v_...
+```
+
+---
+
+## Backends
+
+### In-Memory Backend
+
+Perfect for testing and prototyping:
+
+```python
+from promptcapsule.backends import InMemoryBackend
+
+backend = InMemoryBackend()
+capsule = pc.compress("long prompt" * 100, vault_backend=backend)
+```
+
+### SQLite Backend
+
+Local, file-based storage:
+
+```python
+from promptcapsule.backends import SQLiteBackend
+
+backend = SQLiteBackend("prompts.db")
+# Auto-creates schema, stores prompts locally
+capsule = pc.compress("long prompt" * 100, vault_backend=backend)
+```
+
+### GitHub Gist Backend
+
+Cloud storage with version control:
+
+```python
+from promptcapsule.backends import GitHubGistBackend
+
+backend = GitHubGistBackend(token="github_pat_...")
+# Stores as private gist, returns gist ID
+capsule = pc.compress("long prompt" * 100, vault_backend=backend)
+```
+
+Requires: `pip install promptcapsule[vault]`
+
+### AWS S3 Backend
+
+Enterprise-grade cloud storage:
+
+```python
+from promptcapsule.backends import S3Backend
+
+backend = S3Backend(bucket="my-prompts", region="us-east-1")
+# Stores in S3, returns S3 key
+capsule = pc.compress("long prompt" * 100, vault_backend=backend)
+```
+
+Requires: `pip install promptcapsule[vault]`
+
+---
+
+## Command-Line Interface
+
+```bash
+# Compress a prompt
+echo "Your prompt here" | promptcapsule compress
+# Output: cap_i_a1b2c3d4_...
+
+# Decompress a capsule
+promptcapsule decompress "cap_i_a1b2c3d4_..."
+# Output: Your prompt here
+
+# With vault backend
+promptcapsule compress --vault sqlite:prompts.db < prompt.txt
+promptcapsule decompress --vault sqlite:prompts.db "cap_v_..."
+```
+
+---
+
+## Comparison with Existing Tools
+
+| Tool | What It Does | Strength | Limitation |
+|------|-------------|----------|-----------|
+| **LLMLingua** | Lossy semantic compression of prompts | High compression ratios | Approximate reconstruction, no exact guarantee |
+| **LangChain Hub** | Cloud-hosted prompt template sharing | Easy sharing, versioning | Locked into LangChain ecosystem |
+| **gzip/zlib directly** | General-purpose compression | Simple, standard | Can't store large prompts, no key-based retrieval |
+| **PromptCapsule** | Hybrid lossless + vault-based retrieval | Exact reconstruction, honest, pluggable | Requires storage backend for long prompts |
+
+---
+
+## Use Cases
+
+### 1️⃣ Share Prompts Across Accounts
+Move carefully-crafted prompts between work and personal accounts without copy-pasting:
+
+```python
+capsule = pc.compress(my_favorite_prompt)
+# Share via email, Slack, message, etc.
+# Later, paste in personal account
+result = pc.decompress(capsule)
+```
+
+### 2️⃣ Version-Control Your Prompts
+Keep prompts in git alongside your code:
+
+```
+prompts/
+  ├── article-writer-v1.0.cap
+  ├── article-writer-v1.1.cap
+  └── article-writer-v2.0.cap
+```
+
+### 3️⃣ Portable Prompt Library
+Share a repo of prompts that works on any machine, any account:
+
+```python
+prompts = load_capsule_library("prompts.json")
+for name, capsule in prompts.items():
+    result = pc.decompress(capsule, vault_backend=my_vault)
+    print(f"{name}: {result.text}")
+```
+
+### 4️⃣ Automated Prompt Management
+Build CI/CD pipelines that validate and archive prompts:
+
+```python
+# Compress before commit
+for prompt_file in glob("*.txt"):
+    with open(prompt_file) as f:
+        prompt = f.read()
+    capsule = pc.compress(prompt, vault_backend=vault)
+    save_to_metadata(capsule)
+
+# Retrieve during deployment
+capsule = load_from_metadata()
+prompt = pc.decompress(capsule, vault_backend=vault).text
+```
+
+---
+
+## API Reference
+
+### `PromptCapsule` Class
+
+```python
+class PromptCapsule:
+    def compress(
+        self,
+        text: str,
+        vault_backend: Optional[VaultBackend] = None,
+    ) -> str:
+        """
+        Compress a prompt into a capsule string.
+        
+        Args:
+            text: The prompt to compress
+            vault_backend: Backend for storing long prompts (required if > 500 bytes)
+        
+        Returns:
+            Capsule string (starts with "cap_")
+        
+        Raises:
+            ValueError: If text is empty or > 500 bytes without vault
+            TypeError: If text is not a string
+        """
+    
+    def decompress(
+        self,
+        capsule: str,
+        vault_backend: Optional[VaultBackend] = None,
+    ) -> CapsuleResult:
+        """
+        Decompress a capsule back to the original prompt.
+        
+        Args:
+            capsule: The capsule string to decompress
+            vault_backend: Backend for retrieving long prompts
+        
+        Returns:
+            CapsuleResult with:
+                - text: Original prompt
+                - verified: Integrity check passed
+                - mode: "inline" or "vault"
+                - checksum: SHA256 of original
+                - original_size: Bytes
+                - capsule_size: Bytes
+        
+        Raises:
+            ValueError: If capsule format is invalid
+            KeyError: If vault key not found
+        """
+```
+
+### `CapsuleResult` Named Tuple
+
+```python
+class CapsuleResult(NamedTuple):
+    text: str              # Original prompt
+    verified: bool         # Checksum matched
+    mode: str              # "inline" or "vault"
+    checksum: str          # SHA256 hash
+    original_size: int     # Bytes
+    capsule_size: int      # Bytes
+```
+
+### `VaultBackend` Abstract Class
+
+Implement to create custom backends:
+
+```python
+class VaultBackend:
+    def store(self, text: str, checksum: str) -> str:
+        """Store text, return a key."""
+        raise NotImplementedError
+    
+    def retrieve(self, key: str) -> str:
+        """Retrieve text by key."""
+        raise NotImplementedError
+```
+
+---
+
+## Testing
+
+Run the full regression test suite:
+
+```bash
+python run_tests.py
+```
+
+Expected output:
+```
+======================================================================
+PromptCapsule - Regression Test Suite
+======================================================================
+
+Core Functionality Tests:
+✓ Compress short prompt (inline mode)
+✓ Reject empty strings
+✓ Reject invalid types
+[... 24 more tests ...]
+
+Test Results: 27/27 passed
+======================================================================
+```
+
+---
+
+## Architecture Notes
+
+### Why This Design?
+
+1. **Hybrid approach**: Best of both worlds — inline compression for portability, vault for scale
+2. **Honest positioning**: We don't claim to be better than lossy compression at reducing LLM cost — we solve a different problem (portability + exact reconstruction)
+3. **Pluggable backends**: Future-proof; use SQLite today, S3 tomorrow, custom backend next week
+4. **Integrity by default**: Every capsule includes a checksum; verification is automatic
+5. **Zero external dependencies** for core functionality — just zlib and base64, both stdlib
+
+### Checksum Strategy
+
+- Uses SHA256 (64 hex chars)
+- Stores first 8 chars in capsule for quick verification
+- Prevents accidental corruption detection
+- Does **not** provide cryptographic authentication (future: optional HMAC signing)
+
+### Compression Levels
+
+- **Zlib level 9**: Maximum compression
+- **Base85**: Better human readability than Base64 (4-char savings per 80 bytes)
+- **Trade-off**: ~2-3x size increase for very short content (overhead of capsule format)
+
+---
+
+## Roadmap
+
+- [ ] CLI tool with full feature parity
+- [ ] HMAC signing for optional authentication
+- [ ] Async backend support
+- [ ] Compression format versioning (for future improvements)
+- [ ] Web UI for managing vaults
+- [ ] Prompt templates + variable interpolation
+- [ ] Analytics: track prompt reuse, version adoption
+
+---
+
+## Security Considerations
+
+- **Integrity**: ✅ Checksums detect corruption
+- **Authenticity**: ⚠️ No signing (roadmap)
+- **Confidentiality**: ⚠️ Vault contents transmitted/stored in plaintext (use HTTPS, encrypted S3, private gists)
+- **Access Control**: Depends on backend (GitHub: private gists, S3: IAM policies)
+
+**Recommendation**: Treat capsule strings like URLs — they're short but semantically empty. Don't rely on them for security-critical operations.
+
+---
+
+## License
+
+MIT License - see LICENSE file
+
+---
+
+## Contributing
+
+Contributions welcome! Please:
+
+1. Write tests for new features
+2. Follow PEP 8 style guide
+3. Add docstrings
+4. Update README with examples
+
+---
+
+## Citation
+
+If you use PromptCapsule in your research or project, please cite:
+
+```bibtex
+@software{promptcapsule2024,
+  title={PromptCapsule: Open-source prompt compression and retrieval library},
+  author={Nirogi, Udaya},
+  year={2024},
+  url={https://github.com/UdayaNirogi/promptcapsule}
+}
+```
+
+---
+
+## FAQ
+
+**Q: How is this different from just using a URL shortener?**
+A: URL shorteners store data on a third-party server. PromptCapsule lets you choose your own backend (SQLite locally, S3 privately, GitHub Gist for sharing, etc.). Plus, checksums guarantee integrity.
+
+**Q: Can I use this to compress LLM outputs?**
+A: No — that's a different problem (lossy compression). PromptCapsule is for *inputs* (prompts), not outputs.
+
+**Q: Is the capsule string secure?**
+A: No — treat it like a URL. The 8-char checksum prefix is for integrity, not authentication. If you need signing, that's a roadmap item.
+
+**Q: What about very old Python versions?**
+A: We support Python 3.8+. Older versions should still work (no fancy syntax), but we don't test them.
+
+**Q: Can I use multiple backends at once?**
+A: Yes! Just pass different backends to different compress/decompress calls. Each backend is independent.
+
+---
+
+## See Also
+
+- [LLMLingua](https://github.com/microsoft/LLMLingua) — Lossy prompt compression
+- [LangChain Hub](https://smith.langchain.com/) — Prompt management platform
+- [zlib Documentation](https://www.zlib.net/) — Compression format
+
+---
+
+## Feedback & Support
+
+- 📖 [Documentation](https://github.com/UdayaNirogi/promptcapsule)
+- 🐛 [Report Issues](https://github.com/UdayaNirogi/promptcapsule/issues)
+- 💬 [Discussions](https://github.com/UdayaNirogi/promptcapsule/discussions)
+- 📧 Email: udaya@example.com
+
+---
+
+**Made with ❤️ for developers who care about their prompts.**