amsdal_ml 0.1.4__tar.gz → 0.2.1__tar.gz

{amsdal_ml-0.1.4 → amsdal_ml-0.2.1}/.github/workflows/ci.yml

@@ -23,7 +23,7 @@ jobs:
           python license_check.py
 
   test-lint:
-    name: Run tests and check style
+    name: Run tests and check style (Python ${{ matrix.python-version }}, ${{ matrix.database-backend }})
     needs: [license-check]
     runs-on: self-hosted
     strategy:
@@ -31,6 +31,21 @@ jobs:
       fail-fast: false
       matrix:
         python-version: ["3.11", "3.12"]
+        database-backend: ["sqlite", "postgres"]
+    services:
+      postgres:
+        image: pgvector/pgvector:pg16
+        env:
+          POSTGRES_USER: postgres
+          POSTGRES_PASSWORD: example
+          POSTGRES_DB: postgres
+        ports:
+          - 5432:5432
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
     env:
       PYTHON: ${{ matrix.python-version }}
       DEPS: yes
@@ -54,9 +69,9 @@ jobs:
           hatch run sync
 
       - name: Run style checks
-        if: always()
+        if: always() && matrix.database-backend == 'sqlite'
         run: hatch run all
 
       - name: Run tests
         if: always()
-        run: hatch run cov tests/
+        run: hatch run cov tests/ -- --database_backend=${{ matrix.database-backend }}

{amsdal_ml-0.1.4 → amsdal_ml-0.2.1}/.gitignore

@@ -1,6 +1,7 @@
 .venv/
 venv/
 /warehouse
+.python-version
 
 __pycache__/
 *.py[cod]
@@ -34,3 +35,4 @@ Thumbs.db
 /models/
 /fixtures/
 /static/
+.tmp

{amsdal_ml-0.1.4 → amsdal_ml-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: amsdal_ml
-Version: 0.1.4
+Version: 0.2.1
 Summary: amsdal_ml plugin for AMSDAL Framework
 Requires-Python: >=3.11
 Requires-Dist: aiohttp==3.12.15
@@ -11,8 +11,9 @@ Requires-Dist: amsdal-utils>=0.5.4
 Requires-Dist: amsdal>=0.5.6
 Requires-Dist: mcp>=0.1
 Requires-Dist: openai==1.100.2
-Requires-Dist: pydantic-settings==2.10.1
-Requires-Dist: pydantic==2.11.7
+Requires-Dist: pydantic-settings~=2.12
+Requires-Dist: pydantic~=2.12
+Requires-Dist: pymupdf>=1.24.10
 Description-Content-Type: text/markdown
 
 # AMSDAL ML
@@ -130,6 +131,53 @@ async for chunk in agent.astream('What is semantic search?'):
     print(chunk, end='', flush=True)
 ```
 
+### 5. Functional Calling Agent with Python Tools
+
+```python
+from amsdal_ml.agents.functional_calling_agent import FunctionalCallingAgent
+from amsdal_ml.agents.python_tool import PythonTool
+from amsdal_ml.ml_models.openai_model import OpenAIModel
+
+llm = OpenAIModel()
+agent = FunctionalCallingAgent(model=llm, tools=[search_tool, render_tool])
+result = await agent.arun(user_query="Find products with price > 100", history=[])
+```
+
+### 6. Natural Language Query Retriever
+
+```python
+from amsdal_ml.ml_retrievers.query_retriever import NLQueryRetriever
+
+retriever = NLQueryRetriever(llm=llm, queryset=Product.objects.all())
+documents = await retriever.invoke("Show me red products", limit=10)
+```
+
+### 7. Document Ingestion Pipeline
+
+```python
+from amsdal_ml.ml_ingesting import ModelIngester
+from amsdal_ml.ml_ingesting.pipeline import DefaultIngestionPipeline
+from amsdal_ml.ml_ingesting.loaders.pdf_loader import PdfLoader
+from amsdal_ml.ml_ingesting.processors.text_cleaner import TextCleaner
+from amsdal_ml.ml_ingesting.splitters.token_splitter import TokenSplitter
+from amsdal_ml.ml_ingesting.embedders.openai_embedder import OpenAIEmbedder
+from amsdal_ml.ml_ingesting.stores.embedding_data import EmbeddingDataStore
+
+pipeline = DefaultIngestionPipeline(
+    loader=PdfLoader(),  # Uses pymupdf for PDF processing
+    cleaner=TextCleaner(),
+    splitter=TokenSplitter(max_tokens=800, overlap_tokens=80),
+    embedder=OpenAIEmbedder(),
+    store=EmbeddingDataStore(),
+)
+
+ingester = ModelIngester(
+    pipeline=pipeline,
+    base_tags=["document"],
+    base_metadata={"source": "pdf"},
+)
+```
+
 ## Architecture
 
 ### Core Components
@@ -139,6 +187,16 @@ async for chunk in agent.astream('What is semantic search?'):
 - **`MLRetriever`**: Semantic similarity search with tag-based filtering
 - **`Agent`**: Q&A and task-oriented agents with streaming and citations
 - **`EmbeddingModel`**: Database model storing 1536-dimensional vectors linked to source objects
+- **`PythonTool`**: Tool for executing Python functions within agents
+- **`FunctionalCallingAgent`**: Agent specialized in functional calling with configurable tools
+- **`NLQueryRetriever`**: Retriever for natural language queries on AMSDAL querysets
+- **`DefaultIngestionPipeline`**: Pipeline for document ingestion including loader, cleaner, splitter, embedder, and store
+- **`ModelIngester`**: High-level ingester for processing models with customizable pipelines and metadata
+- **`PdfLoader`**: Document loader using pymupdf for PDF processing
+- **`TextCleaner`**: Processor for cleaning and normalizing text
+- **`TokenSplitter`**: Splitter for dividing text into chunks based on token count
+- **`OpenAIEmbedder`**: Embedder for generating embeddings via OpenAI API
+- **`EmbeddingDataStore`**: Store for saving embedding data linked to source objects
 - **MCP Server/Client**: Expose retrievers as tools or consume external MCP services
 
 ### Configuration

{amsdal_ml-0.1.4 → amsdal_ml-0.2.1}/README.md

@@ -113,6 +113,53 @@ async for chunk in agent.astream('What is semantic search?'):
     print(chunk, end='', flush=True)
 ```
 
+### 5. Functional Calling Agent with Python Tools
+
+```python
+from amsdal_ml.agents.functional_calling_agent import FunctionalCallingAgent
+from amsdal_ml.agents.python_tool import PythonTool
+from amsdal_ml.ml_models.openai_model import OpenAIModel
+
+llm = OpenAIModel()
+agent = FunctionalCallingAgent(model=llm, tools=[search_tool, render_tool])
+result = await agent.arun(user_query="Find products with price > 100", history=[])
+```
+
+### 6. Natural Language Query Retriever
+
+```python
+from amsdal_ml.ml_retrievers.query_retriever import NLQueryRetriever
+
+retriever = NLQueryRetriever(llm=llm, queryset=Product.objects.all())
+documents = await retriever.invoke("Show me red products", limit=10)
+```
+
+### 7. Document Ingestion Pipeline
+
+```python
+from amsdal_ml.ml_ingesting import ModelIngester
+from amsdal_ml.ml_ingesting.pipeline import DefaultIngestionPipeline
+from amsdal_ml.ml_ingesting.loaders.pdf_loader import PdfLoader
+from amsdal_ml.ml_ingesting.processors.text_cleaner import TextCleaner
+from amsdal_ml.ml_ingesting.splitters.token_splitter import TokenSplitter
+from amsdal_ml.ml_ingesting.embedders.openai_embedder import OpenAIEmbedder
+from amsdal_ml.ml_ingesting.stores.embedding_data import EmbeddingDataStore
+
+pipeline = DefaultIngestionPipeline(
+    loader=PdfLoader(),  # Uses pymupdf for PDF processing
+    cleaner=TextCleaner(),
+    splitter=TokenSplitter(max_tokens=800, overlap_tokens=80),
+    embedder=OpenAIEmbedder(),
+    store=EmbeddingDataStore(),
+)
+
+ingester = ModelIngester(
+    pipeline=pipeline,
+    base_tags=["document"],
+    base_metadata={"source": "pdf"},
+)
+```
+
 ## Architecture
 
 ### Core Components
@@ -122,6 +169,16 @@ async for chunk in agent.astream('What is semantic search?'):
 - **`MLRetriever`**: Semantic similarity search with tag-based filtering
 - **`Agent`**: Q&A and task-oriented agents with streaming and citations
 - **`EmbeddingModel`**: Database model storing 1536-dimensional vectors linked to source objects
+- **`PythonTool`**: Tool for executing Python functions within agents
+- **`FunctionalCallingAgent`**: Agent specialized in functional calling with configurable tools
+- **`NLQueryRetriever`**: Retriever for natural language queries on AMSDAL querysets
+- **`DefaultIngestionPipeline`**: Pipeline for document ingestion including loader, cleaner, splitter, embedder, and store
+- **`ModelIngester`**: High-level ingester for processing models with customizable pipelines and metadata
+- **`PdfLoader`**: Document loader using pymupdf for PDF processing
+- **`TextCleaner`**: Processor for cleaning and normalizing text
+- **`TokenSplitter`**: Splitter for dividing text into chunks based on token count
+- **`OpenAIEmbedder`**: Embedder for generating embeddings via OpenAI API
+- **`EmbeddingDataStore`**: Store for saving embedding data linked to source objects
 - **MCP Server/Client**: Expose retrievers as tools or consume external MCP services
 
 ### Configuration