ai-pipeline-core 0.1.7__tar.gz → 0.1.8__tar.gz

{ai_pipeline_core-0.1.7 → ai_pipeline_core-0.1.8}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: ai-pipeline-core
-Version: 0.1.7
+Version: 0.1.8
 Summary: Core utilities for AI-powered processing pipelines using prefect
 Project-URL: Homepage, https://github.com/bbarwik/ai-pipeline-core
 Project-URL: Repository, https://github.com/bbarwik/ai-pipeline-core
@@ -151,7 +151,7 @@ async def process_document(doc: Document):
     return response.parsed
 ```
 
-### Enhanced Pipeline Decorators (New in v0.1.7)
+### Enhanced Pipeline Decorators
 ```python
 from ai_pipeline_core import pipeline_flow, pipeline_task
 from ai_pipeline_core.flow import FlowOptions
@@ -182,7 +182,7 @@ async def my_pipeline(
     return DocumentList(results)
 ```
 
-### Simple Runner Utility (New in v0.1.7)
+### Simple Runner Utility
 ```python
 from ai_pipeline_core.simple_runner import run_cli, run_pipeline
 from ai_pipeline_core.flow import FlowOptions
@@ -206,7 +206,7 @@ async def main():
     )
 ```
 
-### Clean Prefect Decorators (New in v0.1.7)
+### Clean Prefect Decorators
 ```python
 # Import clean Prefect decorators without tracing
 from ai_pipeline_core.prefect import flow, task
@@ -214,12 +214,12 @@ from ai_pipeline_core.prefect import flow, task
 # Or use pipeline decorators with tracing
 from ai_pipeline_core import pipeline_flow, pipeline_task
 
-@task  # Clean Prefect task
+@task  # Clean Prefect task (supports both sync and async)
 def compute(x: int) -> int:
     return x * 2
 
-@pipeline_task(trace_level="always")  # With tracing
-def compute_traced(x: int) -> int:
+@pipeline_task(trace_level="always")  # With tracing (async only)
+async def compute_traced(x: int) -> int:
     return x * 2
 ```
 
@@ -246,12 +246,12 @@ docs = DocumentList([doc1, doc2])
 Managed AI interactions with built-in retry logic, cost tracking, and structured outputs.
 
 **Supported Models** (via LiteLLM proxy):
-- OpenAI: GPT-4, GPT-5 series
-- Anthropic: Claude 3 series
-- Google: Gemini 2.5 series
-- xAI: Grok models
-- Perplexity: Sonar models (with search capabilities)
-- And many more through LiteLLM compatibility
+- OpenAI: gpt-5
+- Anthropic: claude-4
+- Google: gemini-2.5
+- xAI: grok-3, grok-4
+- Perplexity: sonar-pro-search
+- And many more through LiteLLM compatibility. Every model from openrouter should work.
 
 ```python
 from ai_pipeline_core.llm import generate_structured, AIMessages, ModelOptions
@@ -328,13 +328,13 @@ ai_pipeline_core/
 │   └── model_options.py # Configuration models
 ├── flow/              # Prefect flow utilities
 │   ├── config.py      # Type-safe flow configuration
-│   └── options.py     # FlowOptions base class (v0.1.7)
-├── simple_runner/     # Pipeline execution utilities (v0.1.7)
+│   └── options.py     # FlowOptions base class
+├── simple_runner/     # Pipeline execution utilities
 │   ├── cli.py         # CLI interface
 │   └── simple_runner.py # Core runner logic
 ├── logging/           # Structured logging
-├── pipeline.py        # Enhanced decorators (v0.1.7)
-├── prefect.py         # Clean Prefect exports (v0.1.7)
+├── pipeline.py        # Enhanced decorators
+├── prefect.py         # Clean Prefect exports
 ├── tracing.py         # Observability decorators
 └── settings.py        # Centralized configuration
 ```
@@ -345,6 +345,7 @@ ai_pipeline_core/
 ```bash
 make test           # Run all tests
 make test-cov      # Run with coverage report
+make test-showcase # Test the showcase.py CLI example
 pytest tests/test_documents.py::TestDocument::test_creation  # Single test
 ```
 
@@ -481,6 +482,22 @@ For learning purposes, see [CLAUDE.md](CLAUDE.md) for our comprehensive coding s
 
 - [CLAUDE.md](CLAUDE.md) - Detailed coding standards and architecture guide
 
+## Examples
+
+### In This Repository
+- [showcase.py](examples/showcase.py) - Complete example demonstrating all core features including the CLI runner
+  ```bash
+  # Run the showcase example with CLI
+  python examples/showcase.py ./output --temperature 0.7 --batch-size 5
+
+  # Show help
+  python examples/showcase.py --help
+  ```
+- [showcase.jinja2](examples/showcase.jinja2) - Example Jinja2 prompt template
+
+### Real-World Application
+- [AI Documentation Writer](https://github.com/bbarwik/ai-documentation-writer) - Production-ready example showing how to build sophisticated AI pipelines for automated documentation generation. See [examples/ai-documentation-writer.md](examples/ai-documentation-writer.md) for a detailed overview.
+
 ### dependencies_docs/ Directory
 > [!NOTE]
 > The `dependencies_docs/` directory contains guides for AI assistants (like Claude Code) on how to interact with the project's external dependencies and tooling, NOT user documentation for ai-pipeline-core itself. These files are excluded from repository listings to avoid confusion.
@@ -511,29 +528,29 @@ Built with:
 - [LiteLLM](https://litellm.ai/) - LLM proxy
 - [Pydantic](https://pydantic-docs.helpmanual.io/) - Data validation
 
-## What's New in v0.1.7
+## What's New in v0.1.8
 
-### Major Additions
-- **Enhanced Pipeline Decorators**: New `pipeline_flow` and `pipeline_task` decorators combining Prefect functionality with automatic LMNR tracing
-- **FlowOptions Base Class**: Extensible configuration system for flows with type-safe inheritance
-- **Simple Runner Module**: CLI and programmatic utilities for easy pipeline execution
-- **Clean Prefect Exports**: Separate imports for Prefect decorators with and without tracing
-- **Expanded Exports**: All major components now accessible from top-level package import
+### Breaking Changes
+- **Async-Only Pipeline Decorators**: `@pipeline_flow` and `@pipeline_task` now require `async def` functions (raises TypeError for sync)
+- **Document Class Name Validation**: Document subclasses cannot start with "Test" prefix (pytest conflict prevention)
+- **FlowConfig Validation**: OUTPUT_DOCUMENT_TYPE cannot be in INPUT_DOCUMENT_TYPES (prevents circular dependencies)
+- **Temperature Field**: Added optional `temperature` field to `ModelOptions` for explicit control
 
-### API Improvements
-- Better type inference for document flows with custom options
-- Support for custom FlowOptions inheritance in pipeline flows
-- Improved error messages for invalid flow signatures
-- Enhanced document utility functions (`canonical_name_key`, `sanitize_url`)
+### Major Improvements
+- **Pipeline Module Refactoring**: Reduced from ~400 to ~150 lines with cleaner Protocol-based typing
+- **Enhanced Validation**: FlowConfig and Document classes now validate at definition time
+- **Better CLI Support**: Auto-displays help when no arguments provided, improved context management
+- **Test Suite Updates**: All tests updated to use async/await consistently
 
-### Developer Experience
-- Simplified imports - most components available from `ai_pipeline_core` directly
-- Better separation of concerns between clean Prefect and traced pipeline decorators
-- More intuitive flow configuration with `FlowOptions` inheritance
+### Documentation Updates
+- Added Document naming rules to CLAUDE.md
+- Added FlowConfig validation rules
+- Added code elegance principles section
+- Updated guide_for_ai.md to API reference format
 
 ## Stability Notice
 
-**Current Version**: 0.1.7
+**Current Version**: 0.1.8
 **Status**: Internal Preview
 **API Stability**: Unstable - Breaking changes expected
 **Recommended Use**: Learning and reference only

{ai_pipeline_core-0.1.7 → ai_pipeline_core-0.1.8}/README.md

@@ -109,7 +109,7 @@ async def process_document(doc: Document):
     return response.parsed
 ```
 
-### Enhanced Pipeline Decorators (New in v0.1.7)
+### Enhanced Pipeline Decorators
 ```python
 from ai_pipeline_core import pipeline_flow, pipeline_task
 from ai_pipeline_core.flow import FlowOptions
@@ -140,7 +140,7 @@ async def my_pipeline(
     return DocumentList(results)
 ```
 
-### Simple Runner Utility (New in v0.1.7)
+### Simple Runner Utility
 ```python
 from ai_pipeline_core.simple_runner import run_cli, run_pipeline
 from ai_pipeline_core.flow import FlowOptions
@@ -164,7 +164,7 @@ async def main():
     )
 ```
 
-### Clean Prefect Decorators (New in v0.1.7)
+### Clean Prefect Decorators
 ```python
 # Import clean Prefect decorators without tracing
 from ai_pipeline_core.prefect import flow, task
@@ -172,12 +172,12 @@ from ai_pipeline_core.prefect import flow, task
 # Or use pipeline decorators with tracing
 from ai_pipeline_core import pipeline_flow, pipeline_task
 
-@task  # Clean Prefect task
+@task  # Clean Prefect task (supports both sync and async)
 def compute(x: int) -> int:
     return x * 2
 
-@pipeline_task(trace_level="always")  # With tracing
-def compute_traced(x: int) -> int:
+@pipeline_task(trace_level="always")  # With tracing (async only)
+async def compute_traced(x: int) -> int:
     return x * 2
 ```
 
@@ -204,12 +204,12 @@ docs = DocumentList([doc1, doc2])
 Managed AI interactions with built-in retry logic, cost tracking, and structured outputs.
 
 **Supported Models** (via LiteLLM proxy):
-- OpenAI: GPT-4, GPT-5 series
-- Anthropic: Claude 3 series
-- Google: Gemini 2.5 series
-- xAI: Grok models
-- Perplexity: Sonar models (with search capabilities)
-- And many more through LiteLLM compatibility
+- OpenAI: gpt-5
+- Anthropic: claude-4
+- Google: gemini-2.5
+- xAI: grok-3, grok-4
+- Perplexity: sonar-pro-search
+- And many more through LiteLLM compatibility. Every model from openrouter should work.
 
 ```python
 from ai_pipeline_core.llm import generate_structured, AIMessages, ModelOptions
@@ -286,13 +286,13 @@ ai_pipeline_core/
 │   └── model_options.py # Configuration models
 ├── flow/              # Prefect flow utilities
 │   ├── config.py      # Type-safe flow configuration
-│   └── options.py     # FlowOptions base class (v0.1.7)
-├── simple_runner/     # Pipeline execution utilities (v0.1.7)
+│   └── options.py     # FlowOptions base class
+├── simple_runner/     # Pipeline execution utilities
 │   ├── cli.py         # CLI interface
 │   └── simple_runner.py # Core runner logic
 ├── logging/           # Structured logging
-├── pipeline.py        # Enhanced decorators (v0.1.7)
-├── prefect.py         # Clean Prefect exports (v0.1.7)
+├── pipeline.py        # Enhanced decorators
+├── prefect.py         # Clean Prefect exports
 ├── tracing.py         # Observability decorators
 └── settings.py        # Centralized configuration
 ```
@@ -303,6 +303,7 @@ ai_pipeline_core/
 ```bash
 make test           # Run all tests
 make test-cov      # Run with coverage report
+make test-showcase # Test the showcase.py CLI example
 pytest tests/test_documents.py::TestDocument::test_creation  # Single test
 ```
 
@@ -439,6 +440,22 @@ For learning purposes, see [CLAUDE.md](CLAUDE.md) for our comprehensive coding s
 
 - [CLAUDE.md](CLAUDE.md) - Detailed coding standards and architecture guide
 
+## Examples
+
+### In This Repository
+- [showcase.py](examples/showcase.py) - Complete example demonstrating all core features including the CLI runner
+  ```bash
+  # Run the showcase example with CLI
+  python examples/showcase.py ./output --temperature 0.7 --batch-size 5
+
+  # Show help
+  python examples/showcase.py --help
+  ```
+- [showcase.jinja2](examples/showcase.jinja2) - Example Jinja2 prompt template
+
+### Real-World Application
+- [AI Documentation Writer](https://github.com/bbarwik/ai-documentation-writer) - Production-ready example showing how to build sophisticated AI pipelines for automated documentation generation. See [examples/ai-documentation-writer.md](examples/ai-documentation-writer.md) for a detailed overview.
+
 ### dependencies_docs/ Directory
 > [!NOTE]
 > The `dependencies_docs/` directory contains guides for AI assistants (like Claude Code) on how to interact with the project's external dependencies and tooling, NOT user documentation for ai-pipeline-core itself. These files are excluded from repository listings to avoid confusion.
@@ -469,29 +486,29 @@ Built with:
 - [LiteLLM](https://litellm.ai/) - LLM proxy
 - [Pydantic](https://pydantic-docs.helpmanual.io/) - Data validation
 
-## What's New in v0.1.7
+## What's New in v0.1.8
 
-### Major Additions
-- **Enhanced Pipeline Decorators**: New `pipeline_flow` and `pipeline_task` decorators combining Prefect functionality with automatic LMNR tracing
-- **FlowOptions Base Class**: Extensible configuration system for flows with type-safe inheritance
-- **Simple Runner Module**: CLI and programmatic utilities for easy pipeline execution
-- **Clean Prefect Exports**: Separate imports for Prefect decorators with and without tracing
-- **Expanded Exports**: All major components now accessible from top-level package import
+### Breaking Changes
+- **Async-Only Pipeline Decorators**: `@pipeline_flow` and `@pipeline_task` now require `async def` functions (raises TypeError for sync)
+- **Document Class Name Validation**: Document subclasses cannot start with "Test" prefix (pytest conflict prevention)
+- **FlowConfig Validation**: OUTPUT_DOCUMENT_TYPE cannot be in INPUT_DOCUMENT_TYPES (prevents circular dependencies)
+- **Temperature Field**: Added optional `temperature` field to `ModelOptions` for explicit control
 
-### API Improvements
-- Better type inference for document flows with custom options
-- Support for custom FlowOptions inheritance in pipeline flows
-- Improved error messages for invalid flow signatures
-- Enhanced document utility functions (`canonical_name_key`, `sanitize_url`)
+### Major Improvements
+- **Pipeline Module Refactoring**: Reduced from ~400 to ~150 lines with cleaner Protocol-based typing
+- **Enhanced Validation**: FlowConfig and Document classes now validate at definition time
+- **Better CLI Support**: Auto-displays help when no arguments provided, improved context management
+- **Test Suite Updates**: All tests updated to use async/await consistently
 
-### Developer Experience
-- Simplified imports - most components available from `ai_pipeline_core` directly
-- Better separation of concerns between clean Prefect and traced pipeline decorators
-- More intuitive flow configuration with `FlowOptions` inheritance
+### Documentation Updates
+- Added Document naming rules to CLAUDE.md
+- Added FlowConfig validation rules
+- Added code elegance principles section
+- Updated guide_for_ai.md to API reference format
 
 ## Stability Notice
 
-**Current Version**: 0.1.7
+**Current Version**: 0.1.8
 **Status**: Internal Preview
 **API Stability**: Unstable - Breaking changes expected
 **Recommended Use**: Learning and reference only

{ai_pipeline_core-0.1.7 → ai_pipeline_core-0.1.8}/ai_pipeline_core/__init__.py

@@ -27,12 +27,12 @@ from .logging import (
 )
 from .logging import get_pipeline_logger as get_logger
 from .pipeline import pipeline_flow, pipeline_task
-from .prefect import flow, task
+from .prefect import disable_run_logger, prefect_test_harness
 from .prompt_manager import PromptManager
 from .settings import settings
 from .tracing import TraceInfo, TraceLevel, trace
 
-__version__ = "0.1.7"
+__version__ = "0.1.8"
 
 __all__ = [
     # Config/Settings
@@ -54,12 +54,12 @@ __all__ = [
     # Flow/Task
     "FlowConfig",
     "FlowOptions",
-    # Prefect decorators (clean, no tracing)
-    "task",
-    "flow",
     # Pipeline decorators (with tracing)
     "pipeline_task",
     "pipeline_flow",
+    # Prefect decorators (clean, no tracing)
+    "prefect_test_harness",
+    "disable_run_logger",
     # LLM
     "llm",
     "ModelName",

{ai_pipeline_core-0.1.7 → ai_pipeline_core-0.1.8}/ai_pipeline_core/documents/document.py

@@ -26,12 +26,27 @@ TModel = TypeVar("TModel", bound=BaseModel)
 
 
 class Document(BaseModel, ABC):
-    """Abstract base class for all documents"""
+    """Abstract base class for all documents.
+
+    Warning: Document subclasses should NOT start with 'Test' prefix as this
+    causes conflicts with pytest test discovery. Classes with 'Test' prefix
+    will be rejected at definition time.
+    """
 
     MAX_CONTENT_SIZE: ClassVar[int] = 25 * 1024 * 1024  # 25MB default
     DESCRIPTION_EXTENSION: ClassVar[str] = ".description.md"
     MARKDOWN_LIST_SEPARATOR: ClassVar[str] = "\n\n---\n\n"
 
+    def __init_subclass__(cls, **kwargs: Any) -> None:
+        """Validate subclass names to prevent pytest conflicts."""
+        super().__init_subclass__(**kwargs)
+        if cls.__name__.startswith("Test"):
+            raise TypeError(
+                f"Document subclass '{cls.__name__}' cannot start with 'Test' prefix. "
+                "This causes conflicts with pytest test discovery. "
+                "Please use a different name (e.g., 'SampleDocument', 'ExampleDocument')."
+            )
+
     def __init__(self, **data: Any) -> None:
         """Prevent direct instantiation of abstract Document class."""
         if type(self) is Document:

{ai_pipeline_core-0.1.7 → ai_pipeline_core-0.1.8}/ai_pipeline_core/flow/config.py

@@ -14,6 +14,27 @@ class FlowConfig(ABC):
     INPUT_DOCUMENT_TYPES: ClassVar[list[type[FlowDocument]]]
     OUTPUT_DOCUMENT_TYPE: ClassVar[type[FlowDocument]]
 
+    def __init_subclass__(cls, **kwargs):
+        """Validate that OUTPUT_DOCUMENT_TYPE is not in INPUT_DOCUMENT_TYPES."""
+        super().__init_subclass__(**kwargs)
+
+        # Skip validation for the abstract base class itself
+        if cls.__name__ == "FlowConfig":
+            return
+
+        # Ensure required attributes are defined
+        if not hasattr(cls, "INPUT_DOCUMENT_TYPES"):
+            raise TypeError(f"FlowConfig {cls.__name__} must define INPUT_DOCUMENT_TYPES")
+        if not hasattr(cls, "OUTPUT_DOCUMENT_TYPE"):
+            raise TypeError(f"FlowConfig {cls.__name__} must define OUTPUT_DOCUMENT_TYPE")
+
+        # Validate that output type is not in input types
+        if cls.OUTPUT_DOCUMENT_TYPE in cls.INPUT_DOCUMENT_TYPES:
+            raise TypeError(
+                f"FlowConfig {cls.__name__}: OUTPUT_DOCUMENT_TYPE "
+                f"({cls.OUTPUT_DOCUMENT_TYPE.__name__}) cannot be in INPUT_DOCUMENT_TYPES"
+            )
+
     @classmethod
     def get_input_document_types(cls) -> list[type[FlowDocument]]:
         """

{ai_pipeline_core-0.1.7 → ai_pipeline_core-0.1.8}/ai_pipeline_core/llm/model_options.py

@@ -4,6 +4,7 @@ from pydantic import BaseModel
 
 
 class ModelOptions(BaseModel):
+    temperature: float | None = None
     system_prompt: str | None = None
     search_context_size: Literal["low", "medium", "high"] | None = None
     reasoning_effort: Literal["low", "medium", "high"] | None = None
@@ -21,6 +22,9 @@ class ModelOptions(BaseModel):
             "extra_body": {},
         }
 
+        if self.temperature:
+            kwargs["temperature"] = self.temperature
+
         if self.max_completion_tokens:
             kwargs["max_completion_tokens"] = self.max_completion_tokens