slide-narrator 5.5.0__py3-none-any.whl

slide_narrator-5.5.0.dist-info/METADATA

@@ -0,0 +1,558 @@
+Metadata-Version: 2.4
+Name: slide-narrator
+Version: 5.5.0
+Summary: Thread and file storage components for conversational AI - the companion to Tyler AI framework
+Project-URL: Homepage, https://github.com/adamwdraper/slide
+Project-URL: Repository, https://github.com/adamwdraper/slide
+Project-URL: Bug Tracker, https://github.com/adamwdraper/slide/issues
+Author: adamwdraper
+License: MIT
+License-File: LICENSE
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.11
+Requires-Dist: aiosqlite>=0.21.0
+Requires-Dist: alembic>=1.14.1
+Requires-Dist: asyncpg>=0.30.0
+Requires-Dist: click>=8.1.8
+Requires-Dist: filetype>=1.2.0
+Requires-Dist: greenlet>=3.2.3
+Requires-Dist: pydantic>=2.10.4
+Requires-Dist: pypdf>=5.3.0
+Requires-Dist: sqlalchemy>=2.0.36
+Requires-Dist: typing-extensions>=4.12.0
+Provides-Extra: dev
+Requires-Dist: coverage>=7.6.10; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.25.2; extra == 'dev'
+Requires-Dist: pytest-cov>=6.0.0; extra == 'dev'
+Requires-Dist: pytest>=8.3.4; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# The Narrator
+
+Thread and file storage components for conversational AI - the storage foundation for the Slide ecosystem.
+
+## Overview
+
+The Narrator provides robust, production-ready storage solutions for conversational AI applications, serving as the storage layer for Tyler and other Slide components. It includes:
+
+- **ThreadStore**: Persistent storage for conversation threads with support for both in-memory and SQL backends
+- **FileStore**: Secure file storage with automatic processing for various file types  
+- **Models**: Pydantic models for threads, messages, and attachments
+- **CLI Tools**: Command-line interface for database management and setup
+
+## Features
+
+### ThreadStore
+- **Multiple Backends**: In-memory (development), SQLite (local), PostgreSQL (production)
+- **Async/Await Support**: Built for modern Python async applications
+- **Message Filtering**: Automatic handling of system vs. user messages
+- **Platform Integration**: Support for external platform references (Slack, Discord, etc.)
+- **Connection Pooling**: Production-ready database connection management
+
+### FileStore  
+- **Secure Storage**: Automatic file validation and type checking
+- **Multiple Formats**: Support for documents, images, audio, and more
+- **Content Processing**: Automatic text extraction from PDFs, image analysis
+- **Storage Limits**: Configurable file size and total storage limits
+- **Sharded Storage**: Efficient file organization to prevent directory bloat
+
+## Installation
+
+```bash
+# Using uv (recommended)
+uv add slide-narrator
+
+# Using pip (fallback)
+pip install slide-narrator
+```
+
+## Setup
+
+### Docker Setup (Recommended for Development)
+
+For local development with PostgreSQL, Narrator includes built-in Docker commands:
+
+```bash
+# One-command setup: starts PostgreSQL and initializes tables
+uv run narrator docker-setup
+
+# This will:
+# 1. Start a PostgreSQL container
+# 2. Wait for it to be ready  
+# 3. Initialize the database tables
+# 4. Show you the connection string
+
+# The database will be available at:
+# postgresql+asyncpg://narrator:narrator_dev@localhost:5432/narrator
+```
+
+To manage the Docker container:
+
+```bash
+# Stop container (preserves data)
+uv run narrator docker-stop
+
+# Stop and remove all data
+uv run narrator docker-stop --remove-volumes
+
+# Start container again
+uv run narrator docker-start
+
+# Check database status
+uv run narrator status
+```
+
+For custom configurations, the Docker commands respect environment variables:
+
+```bash
+# Use a different port
+uv run narrator docker-setup --port 5433
+
+# Or set environment variables (matching docker-compose.yml)
+export NARRATOR_DB_NAME=mydb
+export NARRATOR_DB_USER=myuser
+export NARRATOR_DB_PASSWORD=mypassword
+export NARRATOR_DB_PORT=5433
+
+# Then run docker-setup
+uv run narrator docker-setup
+
+# This will create:
+# postgresql+asyncpg://myuser:mypassword@localhost:5433/mydb
+```
+
+### Database Setup
+
+For production use with PostgreSQL or SQLite persistence, you'll need to initialize the database tables:
+
+```bash
+# Initialize database tables (PostgreSQL)
+uv run narrator init --database-url "postgresql+asyncpg://user:password@localhost/dbname"
+
+# Initialize database tables (SQLite)
+uv run narrator init --database-url "sqlite+aiosqlite:///path/to/your/database.db"
+
+# Check database status
+uv run narrator status --database-url "postgresql+asyncpg://user:password@localhost/dbname"
+```
+
+You can also use environment variables instead of passing the database URL:
+
+```bash
+# Set environment variable
+export NARRATOR_DATABASE_URL="postgresql+asyncpg://user:password@localhost/dbname"
+
+# Then run without --database-url flag
+uv run narrator init
+uv run narrator status
+```
+
+### Environment Variables
+
+Configure the narrator using environment variables:
+
+```bash
+# Database settings
+NARRATOR_DATABASE_URL="postgresql+asyncpg://user:password@localhost/dbname"
+NARRATOR_DB_POOL_SIZE=5              # Connection pool size
+NARRATOR_DB_MAX_OVERFLOW=10          # Max additional connections
+NARRATOR_DB_POOL_TIMEOUT=30          # Connection timeout (seconds)
+NARRATOR_DB_POOL_RECYCLE=300         # Connection recycle time (seconds)
+NARRATOR_DB_ECHO=false               # Enable SQL logging
+
+# File storage settings
+NARRATOR_FILE_STORAGE_PATH=/path/to/files  # Storage directory
+NARRATOR_MAX_FILE_SIZE=52428800            # 50MB max file size
+NARRATOR_MAX_STORAGE_SIZE=5368709120       # 5GB max total storage
+NARRATOR_ALLOWED_MIME_TYPES=image/jpeg,application/pdf  # Allowed file types
+
+# Logging
+NARRATOR_LOG_LEVEL=INFO              # Log level
+```
+
+## Quick Start
+
+### Basic Thread Storage
+
+```python
+import asyncio
+from narrator import ThreadStore, Thread, Message
+
+async def main():
+    # Create an in-memory store for development
+    store = await ThreadStore.create()
+    
+    # Create a thread
+    thread = Thread(title="My Conversation")
+    
+    # Add messages
+    thread.add_message(Message(role="user", content="Hello!"))
+    thread.add_message(Message(role="assistant", content="Hi there!"))
+    
+    # Save the thread
+    await store.save(thread)
+    
+    # Retrieve the thread
+    retrieved = await store.get(thread.id)
+    print(f"Thread: {retrieved.title}")
+    print(f"Messages: {len(retrieved.messages)}")
+
+asyncio.run(main())
+```
+
+### File Storage
+
+```python
+import asyncio
+from narrator import FileStore
+
+async def main():
+    # Create a file store
+    store = await FileStore.create()
+    
+    # Save a file
+    content = b"Hello, world!"
+    metadata = await store.save(content, "hello.txt", "text/plain")
+    
+    print(f"File ID: {metadata['id']}")
+    print(f"Storage path: {metadata['storage_path']}")
+    
+    # Retrieve the file
+    retrieved_content = await store.get(metadata['id'])
+    print(f"Content: {retrieved_content.decode()}")
+
+asyncio.run(main())
+```
+
+### Database Storage
+
+```python
+import asyncio
+from narrator import ThreadStore
+
+async def main():
+    # Use SQLite for persistent storage
+    store = await ThreadStore.create("sqlite+aiosqlite:///conversations.db")
+    
+    # Use PostgreSQL for production
+    # store = await ThreadStore.create("postgresql+asyncpg://user:pass@localhost/dbname")
+    
+    # The API is the same regardless of backend
+    thread = Thread(title="Persistent Conversation")
+    await store.save(thread)
+
+asyncio.run(main())
+```
+
+## Configuration
+
+### Database Configuration
+
+The Narrator supports multiple database backends:
+
+#### Memory storage (Default)
+```python
+from narrator import ThreadStore
+
+# Use factory pattern for immediate connection validation
+store = await ThreadStore.create()  # Uses memory backend
+
+# Thread operations are immediate
+thread = Thread()
+await store.save(thread)
+```
+
+Key characteristics:
+- Fastest possible performance (direct dictionary access)
+- No persistence (data is lost when program exits)
+- No setup required (works out of the box)
+- Perfect for scripts and one-off conversations
+- Great for testing and development
+
+#### PostgreSQL storage
+```python
+from narrator import ThreadStore
+
+# Use factory pattern for immediate connection validation
+db_url = "postgresql+asyncpg://user:pass@localhost/dbname"
+try:
+    store = await ThreadStore.create(db_url)
+    print("Connected to database successfully")
+except Exception as e:
+    print(f"Database connection failed: {e}")
+    # Handle connection failure appropriately
+
+# Must save threads and changes to persist
+thread = Thread()
+await store.save(thread)  # Required
+thread.add_message(message)
+await store.save(thread)  # Save changes
+
+# Always use thread.id with database storage
+result = await store.get(thread.id)
+```
+
+Key characteristics:
+- Async operations for non-blocking I/O
+- Persistent storage (data survives program restarts)
+- Cross-session support (can access threads from different processes)
+- Production-ready
+- Automatic schema management through SQLAlchemy
+- Connection validation at startup with factory pattern
+
+#### SQLite storage
+```python
+from narrator import ThreadStore
+
+# Use factory pattern for immediate connection validation
+db_url = "sqlite+aiosqlite:///path/to/db.sqlite"
+store = await ThreadStore.create(db_url)
+
+# Or use in-memory SQLite database
+store = await ThreadStore.create("sqlite+aiosqlite://")  # In-memory SQLite
+```
+
+### File Storage Configuration
+
+```python
+from narrator import FileStore
+
+# Create a FileStore instance with factory pattern
+file_store = await FileStore.create(
+    base_path="/path/to/files",  # Optional custom path
+    max_file_size=100 * 1024 * 1024,  # 100MB (optional)
+    max_storage_size=10 * 1024 * 1024 * 1024  # 10GB (optional)
+)
+
+# Or use default settings from environment variables
+file_store = await FileStore.create()
+```
+
+## Advanced Usage
+
+### Using ThreadStore and FileStore Together
+
+```python
+import asyncio
+from narrator import ThreadStore, FileStore, Thread, Message
+
+async def main():
+    # Create stores
+    thread_store = await ThreadStore.create("sqlite+aiosqlite:///main.db")
+    file_store = await FileStore.create("/path/to/files")
+    
+    # Create a thread with file attachment
+    thread = Thread(title="Document Discussion")
+    
+    # Create a message with an attachment
+    message = Message(role="user", content="Here's a document")
+    
+    # Add file content
+    pdf_content = b"..."  # Your PDF content
+    message.add_attachment(pdf_content, filename="document.pdf")
+    
+    thread.add_message(message)
+    
+    # Save thread (attachments are processed automatically)
+    await thread_store.save(thread)
+    
+    print(f"Thread saved with ID: {thread.id}")
+
+asyncio.run(main())
+```
+
+### Message Attachments
+
+Messages can include file attachments that are automatically processed:
+
+```python
+import asyncio
+from narrator import Thread, Message, Attachment, FileStore
+
+async def main():
+    file_store = await FileStore.create()
+    
+    # Create a message with an attachment
+    message = Message(role="user", content="Here's a document")
+    
+    # Add file content
+    pdf_content = b"..."  # Your PDF content
+    attachment = Attachment(filename="document.pdf", content=pdf_content)
+    message.add_attachment(attachment)
+    
+    # Process and store the attachment
+    await attachment.process_and_store(file_store)
+    
+    # The attachment now has extracted text and metadata
+    print(f"Status: {attachment.status}")
+    print(f"File ID: {attachment.file_id}")
+    if attachment.attributes:
+        print(f"Extracted text: {attachment.attributes.get('text', 'N/A')[:100]}...")
+
+asyncio.run(main())
+```
+
+### Platform Integration
+
+Threads can be linked to external platforms:
+
+```python
+import asyncio
+from narrator import Thread, ThreadStore
+
+async def main():
+    store = await ThreadStore.create()
+    
+    # Create a thread linked to Slack
+    thread = Thread(
+        title="Support Ticket #123",
+        platforms={
+            "slack": {
+                "channel": "C1234567",
+                "thread_ts": "1234567890.123"
+            }
+        }
+    )
+    
+    await store.save(thread)
+    
+    # Find threads by platform
+    slack_threads = await store.find_by_platform("slack", {"channel": "C1234567"})
+    print(f"Found {len(slack_threads)} Slack threads in channel")
+
+asyncio.run(main())
+```
+
+## Database CLI
+
+The Narrator includes a CLI tool for database management:
+
+```bash
+# Initialize database tables
+uv run narrator init --database-url "postgresql+asyncpg://user:pass@localhost/dbname"
+
+# Initialize using environment variable
+export NARRATOR_DATABASE_URL="postgresql+asyncpg://user:pass@localhost/dbname"
+uv run narrator init
+
+# Check database status
+uv run narrator status --database-url "postgresql+asyncpg://user:pass@localhost/dbname"
+
+# Check status using environment variable
+uv run narrator status
+```
+
+Available commands:
+- `uv run narrator init` - Initialize database tables
+- `uv run narrator status` - Check database connection and basic statistics
+
+## Key Design Principles
+
+1. **Factory Pattern**: Use `await ThreadStore.create()` and `await FileStore.create()` for proper initialization and connection validation
+2. **Backend Agnostic**: Same API whether using in-memory, SQLite, or PostgreSQL storage
+3. **Production Ready**: Built-in connection pooling, error handling, and health checks
+4. **Tyler Integration**: Seamlessly integrates with Tyler agents for conversation persistence
+5. **Platform Support**: Native support for external platforms like Slack, Discord, and custom integrations
+
+## API Reference
+
+### ThreadStore
+
+#### Methods
+
+- `await ThreadStore.create(database_url=None)`: Factory method to create and initialize a store
+- `await store.save(thread)`: Save a thread to storage
+- `await store.get(thread_id)`: Retrieve a thread by ID
+- `await store.delete(thread_id)`: Delete a thread
+- `await store.list(limit=100, offset=0)`: List threads with pagination
+- `await store.find_by_attributes(attributes)`: Find threads by custom attributes
+- `await store.find_by_platform(platform_name, properties)`: Find threads by platform
+- `await store.list_recent(limit=None)`: List recent threads
+
+### FileStore
+
+#### Methods
+
+- `await FileStore.create(base_path=None, ...)`: Factory method to create and validate a store
+- `await store.save(content, filename, mime_type=None)`: Save file content
+- `await store.get(file_id, storage_path=None)`: Retrieve file content
+- `await store.delete(file_id, storage_path=None)`: Delete a file
+- `await store.get_storage_size()`: Get total storage size
+- `await store.check_health()`: Check storage health
+
+### Models
+
+#### Thread
+- `id`: Unique thread identifier
+- `title`: Thread title
+- `messages`: List of messages
+- `created_at`: Creation timestamp
+- `updated_at`: Last update timestamp
+- `attributes`: Custom attributes dictionary
+- `platforms`: Platform-specific metadata
+
+#### Message
+- `id`: Unique message identifier
+- `role`: Message role (user, assistant, system, tool)
+- `content`: Message content
+- `attachments`: List of file attachments
+- `timestamp`: Message timestamp
+- `metrics`: Performance metrics
+
+#### Attachment
+- `filename`: Original filename
+- `mime_type`: File MIME type
+- `file_id`: Storage file ID
+- `storage_path`: Path in storage
+- `status`: Processing status (pending, stored, failed)
+- `attributes`: Processed content and metadata
+
+## Development
+
+### Running Tests
+
+To run the test suite locally:
+
+```bash
+# Install development dependencies
+uv sync --extra dev
+
+# Run tests with coverage
+uv run pytest tests/ --cov=narrator --cov-report=term-missing --cov-branch --cov-report=term --no-cov-on-fail -v
+
+# Run tests without coverage (faster)
+uv run pytest tests/ -v
+```
+
+### Test Requirements
+
+The test suite requires:
+- Python 3.13+
+- pytest with async support
+- Test coverage reporting
+
+## Contributing
+
+1. Fork the repository
+2. Create a feature branch
+3. Make your changes
+4. Add tests
+5. Run the test suite to ensure everything works
+6. Submit a pull request
+
+## License
+
+MIT License - see LICENSE file for details.
+
+## Support
+
+For issues and questions:
+- GitHub Issues: [Repository Issues](https://github.com/adamwdraper/the-narrator/issues)
+- Documentation: [API Reference](https://github.com/adamwdraper/the-narrator#api-reference) 

slide_narrator-5.5.0.dist-info/RECORD

@@ -0,0 +1,20 @@
+narrator/__init__.py,sha256=Yzh2UFHX_grpGr_RvFqWpUTevr06FyP_R_Z9IHpF9jI,403
+narrator/database/__init__.py,sha256=UngOnFqImCeJiMZlMasm72mC4-UnJDDvfu1MNQLkRA8,189
+narrator/database/cli.py,sha256=QvET17X5kLZ7GiOTw0b80-u4FuI-tOTu4SjAqCBkiSs,8355
+narrator/database/models.py,sha256=OyiufON_ZLNM7PUxXrUoFu1uu62SolHABwpedVnSsfk,2659
+narrator/database/storage_backend.py,sha256=NZ0SKvEL6qKgdJrxuBw5qs6k-9mdWXWE0Ad36epIn2U,27598
+narrator/database/thread_store.py,sha256=vMIPDdwuSpTyPogEUmxGcILxM_r1wxoQBUOn8XJpdqM,11301
+narrator/database/migrations/__init__.py,sha256=IqoSL8eCcbcOtn96u2_TTrNG0KN1Jn1yreDZEO4RsnM,173
+narrator/models/__init__.py,sha256=J8Rsv2lmfGR5QmUjoAPEFTSQt5TGtyrBynnp17HdZnU,179
+narrator/models/attachment.py,sha256=_DIe2F5Vd75UtKmWO7x9d69dzAxUWCOXotvzR-nGVuU,17940
+narrator/models/message.py,sha256=QVItUV75subrmVoEWFy_25khHP7fwbVaKZI8Rh5IPTI,21629
+narrator/models/thread.py,sha256=MMZfJPMMw1zdD_1Jp4NqEAvnTVpHjaFSY82eV4921PQ,19326
+narrator/storage/__init__.py,sha256=K4cxGITSQoQiw32QOWZsCBm11fwDTbsyzHGeAqcL6yY,101
+narrator/storage/file_store.py,sha256=H2F08e_DYjO-L782s1DgSctJ_2x-7EjrTFxLxV3oWuk,20062
+narrator/utils/__init__.py,sha256=P4BhLvBJbBvb8qha2tTZPlYbjCRXth_K97f4vNc77UI,109
+narrator/utils/logging.py,sha256=Hm6D4VX03e28UCkNS1pCOXnYQKHQ2nz_PvZX8h-wLgg,1807
+slide_narrator-5.5.0.dist-info/METADATA,sha256=tcd_3zqNCGJ6EIgmcZVnvyRI3ROKzn7fcKFuOw-2N9E,16494
+slide_narrator-5.5.0.dist-info/WHEEL,sha256=WLgqFyCfm_KASv4WHyYy0P3pM_m7J5L9k2skdKLirC8,87
+slide_narrator-5.5.0.dist-info/entry_points.txt,sha256=5Oa53AERvPVdrEvsdWbY85xfzAGayOqq_P4KEmf1khA,56
+slide_narrator-5.5.0.dist-info/licenses/LICENSE,sha256=g6cGasroU9sqSOjThWg14w0BMlwZhgmOQQVTiu036ks,1068
+slide_narrator-5.5.0.dist-info/RECORD,,

slide_narrator-5.5.0.dist-info/WHEEL

@@ -0,0 +1,4 @@
+Wheel-Version: 1.0
+Generator: hatchling 1.28.0
+Root-Is-Purelib: true
+Tag: py3-none-any

slide_narrator-5.5.0.dist-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+narrator = narrator.database.cli:main

slide_narrator-5.5.0.dist-info/licenses/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 Adam Draper
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.