TalkPipe 0.9.1__tar.gz → 0.9.2__tar.gz

{talkpipe-0.9.1 → talkpipe-0.9.2}/.github/workflows/ci-cd.yml

@@ -40,7 +40,7 @@ jobs:
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        pip install -e .[dev]
+        pip install -e ".[dev,all]"
     
     - name: Run tests with coverage
       run: |
@@ -50,7 +50,7 @@ jobs:
       if: matrix.python-version == '3.11'
       uses: codecov/codecov-action@v5
       with:
-        file: ./coverage.xml
+        files: ./coverage.xml
         flags: unittests
 
   security-scan:
@@ -68,7 +68,7 @@ jobs:
     - name: Install dependencies for security scan
       run: |
         python -m pip install --upgrade pip
-        pip install -e .[dev]
+        pip install -e ".[dev,all]"
         pip install bandit safety
     
     - name: Run Bandit security scan

{talkpipe-0.9.1 → talkpipe-0.9.2}/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 0.9.2
+### Improvements
+- Added ability for sources and segments to have multiple names in chatterlang.
+- Removed signature segments.
+- Added Anthropic as a chat provider.
+- Updated the install options so that not all the providers have to be installed. Include the provider
+you want as a parameter (e.g. pip install talkpipe[anthropic]). Current options are anthropic, openai,
+and ollama. Use "all" to install all three.
+
+## 0.9.1
+Forgot to import the lancedb module in talkpipe/__init__.py, so it wasn't registering the segments.
+
 ## 0.9.0
 ### New and Updated Segments and Sources
 - Added **set**, which simply assigns some constant to a key.

{talkpipe-0.9.1 → talkpipe-0.9.2}/Dockerfile

@@ -37,7 +37,7 @@ COPY --chown=builder:builder tests/ tests/
 RUN python3 -m pip install --user --upgrade pip setuptools wheel build
 RUN python3 -m pip install --user numpy pandas matplotlib scikit-learn scipy
 ENV SETUPTOOLS_SCM_PRETEND_VERSION_FOR_TALKPIPE=0.1.0
-RUN python3 -m pip install --user -e .[dev]
+RUN python3 -m pip install --user -e .[dev,all]
 RUN python3 -m pytest --log-cli-level=DEBUG
 RUN python3 -m build --wheel
 

{talkpipe-0.9.1 → talkpipe-0.9.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: talkpipe
-Version: 0.9.1
+Version: 0.9.2
 Summary: Python internal and external DSL for writing generative AI analytics
 Author-email: Travis Bauer <tlbauer@sandia.gov>
 Maintainer-email: Travis Bauer <tlbauer@sandia.gov>
@@ -16,32 +16,38 @@ License-File: LICENSE
 Requires-Dist: prompt_toolkit
 Requires-Dist: parsy
 Requires-Dist: pydantic
-Requires-Dist: ollama
 Requires-Dist: requests
 Requires-Dist: numpy
-Requires-Dist: numba==0.61.2
+Requires-Dist: numba==0.62.1
 Requires-Dist: python-docx
 Requires-Dist: pandas
 Requires-Dist: feedparser
 Requires-Dist: readability-lxml
 Requires-Dist: lxml
 Requires-Dist: lxml_html_clean
-Requires-Dist: openai
 Requires-Dist: fastapi[standard]
 Requires-Dist: ipywidgets
 Requires-Dist: pymongo
 Requires-Dist: umap-learn
 Requires-Dist: scikit-learn
-Requires-Dist: cryptography
 Requires-Dist: uvicorn
 Requires-Dist: whoosh
 Requires-Dist: lancedb
 Requires-Dist: deprecated
+Requires-Dist: pyyaml
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: pytest-mock; extra == "dev"
 Requires-Dist: pytest-cov; extra == "dev"
 Requires-Dist: mongomock; extra == "dev"
+Provides-Extra: ollama
+Requires-Dist: ollama; extra == "ollama"
+Provides-Extra: openai
+Requires-Dist: openai; extra == "openai"
+Provides-Extra: anthropic
+Requires-Dist: anthropic; extra == "anthropic"
+Provides-Extra: all
+Requires-Dist: talkpipe[anthropic,ollama,openai]; extra == "all"
 Dynamic: license-file
 
 <center><img src="docs/TalkPipe.png" width=500></center>
@@ -108,6 +114,17 @@ Install TalkPipe:
 pip install talkpipe
 ```
 
+For LLM support, install the provider(s) you need:
+```bash
+# Install specific providers
+pip install talkpipe[openai]    # For OpenAI
+pip install talkpipe[ollama]    # For Ollama
+pip install talkpipe[anthropic] # For Anthropic Claude
+
+# Or install all LLM providers
+pip install talkpipe[all]
+```
+
 Create a multi-turn chat function in 2 lines:
 ```python
 from talkpipe.chatterlang import compiler

{talkpipe-0.9.1 → talkpipe-0.9.2}/README.md

@@ -62,6 +62,17 @@ Install TalkPipe:
 pip install talkpipe
 ```
 
+For LLM support, install the provider(s) you need:
+```bash
+# Install specific providers
+pip install talkpipe[openai]    # For OpenAI
+pip install talkpipe[ollama]    # For Ollama
+pip install talkpipe[anthropic] # For Anthropic Claude
+
+# Or install all LLM providers
+pip install talkpipe[all]
+```
+
 Create a multi-turn chat function in 2 lines:
 ```python
 from talkpipe.chatterlang import compiler

{talkpipe-0.9.1 → talkpipe-0.9.2}/docs/quickstart.md

@@ -8,6 +8,17 @@ Welcome to TalkPipe! This guide will help you get up and running quickly with Ta
 pip install talkpipe
 ```
 
+For LLM support, install the provider(s) you need:
+```bash
+# Install specific providers
+pip install talkpipe[openai]    # For OpenAI
+pip install talkpipe[ollama]    # For Ollama
+pip install talkpipe[anthropic] # For Anthropic Claude
+
+# Or install all LLM providers
+pip install talkpipe[all]
+```
+
 ## Basic Concepts
 
 TalkPipe provides two ways to build data processing pipelines:

{talkpipe-0.9.1 → talkpipe-0.9.2}/docs/tutorials/README.md

@@ -152,7 +152,11 @@ This isn't hypothetical – it's exactly how these tutorials were designed to wo
 ### Start Running Immediately
 One of TalkPipe's core strengths is getting you from idea to working prototype faster than traditional approaches:
 
-1. **Install TalkPipe**: `pip install talkpipe`
+1. **Install TalkPipe**:
+   ```bash
+   pip install talkpipe[all]  # Includes all LLM providers
+   # Or install specific providers: talkpipe[ollama], talkpipe[openai], talkpipe[anthropic]
+   ```
 2. **Run a tutorial**: `cd Tutorial_1-Document_Indexing && ./Step_1_CreateSyntheticData.sh`
 3. **See results**: Working applications in minutes, not hours
 

{talkpipe-0.9.1 → talkpipe-0.9.2}/pyproject.toml

@@ -24,27 +24,25 @@ dependencies = [
     'prompt_toolkit',
     'parsy',
     'pydantic',
-    'ollama',
     'requests',
     'numpy',
-    'numba==0.61.2',
+    'numba==0.62.1',
     'python-docx',
     'pandas',
     'feedparser',
     'readability-lxml',
     'lxml',
     'lxml_html_clean',
-    'openai',
     'fastapi[standard]',
     'ipywidgets',
     'pymongo',
     'umap-learn',
     'scikit-learn',
-    'cryptography',
     'uvicorn',
     'whoosh',
     'lancedb',
-    'deprecated'
+    'deprecated',
+    'pyyaml'
 ]
 dynamic = ["version"]
 
@@ -55,6 +53,18 @@ dev = [
     'pytest-cov',
     'mongomock'
 ]
+ollama = [
+    'ollama'
+]
+openai = [
+    'openai'
+]
+anthropic = [
+    'anthropic'
+]
+all = [
+    'talkpipe[openai,ollama,anthropic]'
+]
 
 [project.scripts]
 chatterlang_workbench = "talkpipe.app.chatterlang_workbench:main"

{talkpipe-0.9.1 → talkpipe-0.9.2}/src/talkpipe/__init__.py

@@ -1,3 +1,7 @@
+import warnings
+warnings.filterwarnings("ignore", message=".*ColPaliEmbeddings.*has conflict with protected namespace.*")
+warnings.filterwarnings("ignore", message=".*SigLipEmbeddings.*has conflict with protected namespace.*")
+
 from talkpipe.pipe.basic import *
 from talkpipe.pipe.math import *
 from talkpipe.pipe.io import *
@@ -12,7 +16,6 @@ from talkpipe.data.mongo import *
 from talkpipe.operations.filtering import *
 from talkpipe.operations.transforms import *
 from talkpipe.operations.matrices import *
-from talkpipe.operations.signatures import *
 from talkpipe.app.chatterlang_serve import *
 from talkpipe.search.whoosh import *
 from talkpipe.search.simplevectordb import *

{talkpipe-0.9.1 → talkpipe-0.9.2}/src/talkpipe/app/chatterlang_reference_browser.py

@@ -18,24 +18,36 @@ from talkpipe.util.doc_extraction import (
 
 class TalkPipeDoc:
     """Represents a single TalkPipe component (class or function)."""
-    
-    def __init__(self, name: str, chatterlang_name: str, doc_type: str, 
-                 module: str, base_classes: List[str], docstring: str, 
+
+    def __init__(self, name: str, chatterlang_names: List[str], doc_type: str,
+                 module: str, base_classes: List[str], docstring: str,
                  parameters: Dict[str, str]):
         self.name = name
-        self.chatterlang_name = chatterlang_name
+        self.chatterlang_names = chatterlang_names  # List of all names for this component
+        self.primary_name = chatterlang_names[0]  # Primary name for display
         self.doc_type = doc_type  # 'Source', 'Segment', 'Field Segment'
         self.module = module
         self.base_classes = base_classes
         self.docstring = docstring
         self.parameters = parameters
 
+    @property
+    def chatterlang_name(self):
+        """Backward compatibility property."""
+        return self.primary_name
+
+    @property
+    def all_names_display(self):
+        """Display string showing all names."""
+        return ", ".join(self.chatterlang_names)
+
 
 class TalkPipeBrowser:
     """Interactive terminal browser for TalkPipe documentation."""
-    
+
     def __init__(self):
-        self.components: Dict[str, TalkPipeDoc] = {}
+        self.components: Dict[str, TalkPipeDoc] = {}  # Maps primary name to component
+        self.name_to_primary: Dict[str, str] = {}  # Maps any name to primary name
         self.modules: Dict[str, List[str]] = {}
         self.load_components()
     
@@ -44,23 +56,38 @@ class TalkPipeBrowser:
         return extract_parameters_dict(cls)
     
     def load_components(self):
-        """Load all components from the plugin system."""
+        """Load all components from the plugin system, grouping multiple names for the same class."""
         load_plugins()  # Ensure plugins are loaded
-        
+
+        # Group components by class to consolidate multiple names
+        class_to_names = {}
+        class_to_type = {}
+
         # Load sources
         for chatterlang_name, cls in input_registry.all.items():
-            component_info = extract_component_info(chatterlang_name, cls, "Source")
-            if component_info:
-                self._load_component_from_info(component_info)
-        
+            if cls not in class_to_names:
+                class_to_names[cls] = []
+                class_to_type[cls] = "Source"
+            class_to_names[cls].append(chatterlang_name)
+
         # Load segments
         for chatterlang_name, cls in segment_registry.all.items():
-            component_type = detect_component_type(cls, "Segment")
-            component_info = extract_component_info(chatterlang_name, cls, component_type)
+            if cls not in class_to_names:
+                class_to_names[cls] = []
+                class_to_type[cls] = detect_component_type(cls, "Segment")
+            class_to_names[cls].append(chatterlang_name)
+
+        # Create consolidated components
+        for cls, names in class_to_names.items():
+            # Sort names to ensure consistent primary name selection
+            names.sort()
+            primary_name = names[0]
+
+            component_info = extract_component_info(primary_name, cls, class_to_type[cls])
             if component_info:
-                self._load_component_from_info(component_info)
+                self._load_component_from_info(component_info, names)
     
-    def _load_component_from_info(self, component_info):
+    def _load_component_from_info(self, component_info, all_names: List[str]):
         """Load a single component from ComponentInfo into the browser."""
         try:
             # Convert parameters from ParamSpec list to dict for browser compatibility
@@ -100,20 +127,26 @@ class TalkPipeBrowser:
             # Create component
             component = TalkPipeDoc(
                 name=component_info.name,
-                chatterlang_name=component_info.chatterlang_name,
+                chatterlang_names=all_names,
                 doc_type=component_info.component_type,
                 module=component_info.module,
                 base_classes=component_info.base_classes,
                 docstring=component_info.docstring,
                 parameters=parameters
             )
-            
-            self.components[component_info.chatterlang_name] = component
+
+            # Store component under primary name
+            primary_name = all_names[0]
+            self.components[primary_name] = component
+
+            # Map all names to the primary name for lookup
+            for name in all_names:
+                self.name_to_primary[name] = primary_name
             
             # Group by module
             if component_info.module not in self.modules:
                 self.modules[component_info.module] = []
-            self.modules[component_info.module].append(component_info.chatterlang_name)
+            self.modules[component_info.module].append(primary_name)
             
         except Exception as e:
             print(f"Warning: Failed to load component {component_info.chatterlang_name}: {e}")
@@ -223,24 +256,29 @@ class TalkPipeBrowser:
                 type_icon = "🔧"
             else:
                 type_icon = "⚙️"
-            print(f"{type_icon} {comp.chatterlang_name:<20} ({comp.name})")
+            print(f"{type_icon} {comp.all_names_display:<30} ({comp.name})")
         print()
     
     def _show_component(self, component_name: str):
         """Show detailed information about a component."""
-        # Try exact match first
-        component = self.components.get(component_name)
-        
-        # If not found, try case-insensitive search
+        # Try exact match using name lookup
+        primary_name = self.name_to_primary.get(component_name)
+        component = None
+
+        if primary_name:
+            component = self.components.get(primary_name)
+
+        # If not found, try case-insensitive search in all names
         if not component:
-            matches = [name for name in self.components.keys() 
+            matches = [name for name in self.name_to_primary.keys()
                       if name.lower() == component_name.lower()]
             if matches:
-                component = self.components[matches[0]]
+                primary_name = self.name_to_primary[matches[0]]
+                component = self.components[primary_name]
         
         # If still not found, suggest similar names
         if not component:
-            similar = [name for name in self.components.keys() 
+            similar = [name for name in self.name_to_primary.keys()
                       if component_name.lower() in name.lower()]
             if similar:
                 print(f"Component '{component_name}' not found. Did you mean:")
@@ -252,7 +290,7 @@ class TalkPipeBrowser:
         
         # Display component details
         print(f"\n{'='*60}")
-        print(f"📋 {component.chatterlang_name}")
+        print(f"📋 {component.all_names_display}")
         print(f"{'='*60}")
         print(f"Class/Function: {component.name}")
         print(f"Type:           {component.doc_type}")
@@ -281,11 +319,12 @@ class TalkPipeBrowser:
         """Search for components by name or description."""
         search_lower = search_term.lower()
         matches = []
-        
+
         for comp_name, component in self.components.items():
-            # Search in chatterlang name, class name, and docstring
-            if (search_lower in comp_name.lower() or 
-                search_lower in component.name.lower() or 
+            # Search in all chatterlang names, class name, and docstring
+            name_match = any(search_lower in name.lower() for name in component.chatterlang_names)
+            if (name_match or
+                search_lower in component.name.lower() or
                 search_lower in component.docstring.lower()):
                 matches.append(component)
         
@@ -296,14 +335,14 @@ class TalkPipeBrowser:
         print(f"\nSearch Results for '{search_term}' ({len(matches)} found):")
         print("-" * 60)
         
-        for component in sorted(matches, key=lambda x: x.chatterlang_name):
+        for component in sorted(matches, key=lambda x: x.primary_name):
             if component.doc_type == "Source":
                 type_icon = "🔌"
             elif component.doc_type == "Field Segment":
                 type_icon = "🔧"
             else:
                 type_icon = "⚙️"
-            print(f"{type_icon} {component.chatterlang_name:<20} ({component.module})")
+            print(f"{type_icon} {component.all_names_display:<30} ({component.module})")
             
             # Show brief description
             if component.docstring:

{talkpipe-0.9.1 → talkpipe-0.9.2}/src/talkpipe/app/chatterlang_reference_generator.py

@@ -38,25 +38,42 @@ def sanitize_id(text: str) -> str:
 def analyze_registered_items() -> List[AnalyzedItem]:
     """
     Analyze all registered sources and segments from the plugin system.
+    Groups items with multiple chatterlang names together.
     """
     load_plugins()  # Ensure plugins are loaded
-    analyzed_items = []
-    
+
+    # Group by class to handle multiple names for the same class
+    class_to_names = {}
+
     # Process sources
     for chatterlang_name, cls in input_registry.all.items():
-        component_info = extract_component_info(chatterlang_name, cls, 'Source')
-        if component_info:
-            item = convert_component_info_to_analyzed_item(component_info)
-            analyzed_items.append(item)
-    
+        if cls not in class_to_names:
+            class_to_names[cls] = {'names': [], 'type': 'Source'}
+        class_to_names[cls]['names'].append(chatterlang_name)
+
     # Process segments
     for chatterlang_name, cls in segment_registry.all.items():
-        component_type = detect_component_type(cls, 'Segment')
-        component_info = extract_component_info(chatterlang_name, cls, component_type)
+        if cls not in class_to_names:
+            component_type = detect_component_type(cls, 'Segment')
+            class_to_names[cls] = {'names': [], 'type': component_type}
+        class_to_names[cls]['names'].append(chatterlang_name)
+
+    analyzed_items = []
+
+    # Create AnalyzedItem objects with combined names
+    for cls, info in class_to_names.items():
+        # Sort names for consistent output
+        sorted_names = sorted(info['names'])
+        primary_name = sorted_names[0]  # Use first alphabetically as primary
+
+        component_info = extract_component_info(primary_name, cls, info['type'])
         if component_info:
             item = convert_component_info_to_analyzed_item(component_info)
+            # If multiple names, combine them
+            if len(sorted_names) > 1:
+                item.chatterlang_name = ', '.join(sorted_names)
             analyzed_items.append(item)
-    
+
     return analyzed_items
 
 def convert_component_info_to_analyzed_item(component_info: ComponentInfo) -> AnalyzedItem:

talkpipe-0.9.2/src/talkpipe/app/static/unit-docs.html

@@ -0,0 +1 @@
+Documentation generation failed: Command '['/tmp/build-env-6xf3v7m7/bin/python', 'src/talkpipe/app/chatterlang_reference_generator.py', 'src', 'src/talkpipe/app/static/unit-docs.html', 'src/talkpipe/app/static/unit-docs.txt']' returned non-zero exit status 1.

talkpipe-0.9.2/src/talkpipe/app/static/unit-docs.txt

@@ -0,0 +1 @@
+Documentation generation failed: Command '['/tmp/build-env-6xf3v7m7/bin/python', 'src/talkpipe/app/chatterlang_reference_generator.py', 'src', 'src/talkpipe/app/static/unit-docs.html', 'src/talkpipe/app/static/unit-docs.txt']' returned non-zero exit status 1.

{talkpipe-0.9.1 → talkpipe-0.9.2}/src/talkpipe/chatterlang/registry.py

@@ -31,17 +31,37 @@ class Registry(Generic[T]):
 input_registry = Registry()
 segment_registry = Registry()
 
-def register_source(name: str):
-    """Decorator to register a source module in the registry. """
+def register_source(*names: str, name: str = None):
+    """Decorator to register a source module with one or more names in the registry. """
+    # Handle backward compatibility with name= keyword argument
+    if name is not None:
+        if names:
+            raise ValueError("Cannot specify both positional names and 'name' keyword argument")
+        names = (name,)
+
+    if not names:
+        raise ValueError("At least one name must be provided")
+
     def wrap(cls):
-        input_registry.register(cls, name=name)
+        for source_name in names:
+            input_registry.register(cls, name=source_name)
         return cls
     return wrap
 
-def register_segment(name: str):
-    """Decorator to register a setment module in the registry. """
+def register_segment(*names: str, name: str = None):
+    """Decorator to register a segment module with one or more names in the registry. """
+    # Handle backward compatibility with name= keyword argument
+    if name is not None:
+        if names:
+            raise ValueError("Cannot specify both positional names and 'name' keyword argument")
+        names = (name,)
+
+    if not names:
+        raise ValueError("At least one name must be provided")
+
     def wrap(cls):
-        segment_registry.register(cls, name=name)
+        for segment_name in names:
+            segment_registry.register(cls, name=segment_name)
         return cls
     return wrap
 

{talkpipe-0.9.1 → talkpipe-0.9.2}/src/talkpipe/llm/config.py

@@ -1,5 +1,5 @@
 from typing import Dict, Type, TypeVar, List
-from .prompt_adapters import AbstractLLMPromptAdapter, OllamaPromptAdapter, OpenAIPromptAdapter
+from .prompt_adapters import AbstractLLMPromptAdapter, OllamaPromptAdapter, OpenAIPromptAdapter, AnthropicPromptAdapter
 from .embedding_adapters import AbstractEmbeddingAdapter, OllamaEmbedderAdapter
 
 T_PROMPTADAPTER = TypeVar("T_PROMPTADAPTER", bound=AbstractLLMPromptAdapter)
@@ -7,7 +7,8 @@ T_EMBEDDINGADAPTER = TypeVar("T_EMBEDDINGADAPTER", bound=AbstractEmbeddingAdapte
 
 _promptAdapter:Dict[str, Type[T_PROMPTADAPTER]] = {
     "ollama": OllamaPromptAdapter,
-    "openai": OpenAIPromptAdapter
+    "openai": OpenAIPromptAdapter,
+    "anthropic": AnthropicPromptAdapter
 }
 
 def registerPromptAdapter(name:str, promptAdapter:Type[T_PROMPTADAPTER]):

{talkpipe-0.9.1 → talkpipe-0.9.2}/src/talkpipe/llm/embedding_adapters.py

@@ -1,7 +1,6 @@
 from typing import List
 
 import numpy as np
-import ollama
 
 class AbstractEmbeddingAdapter:
     """Abstract class for embedding text.
@@ -49,6 +48,13 @@ class OllamaEmbedderAdapter(AbstractEmbeddingAdapter):
         super().__init__(model, "ollama")
 
     def execute(self, text: str) -> List[float]:
+        try:
+            import ollama
+        except ImportError:
+            raise ImportError(
+                "Ollama is not installed. Please install it with: pip install talkpipe[ollama]"
+            )
+
         response = ollama.embed(
             model=self.model_name,
             input=text