cocoindex 0.1.75__tar.gz → 0.1.76__tar.gz

{cocoindex-0.1.75 → cocoindex-0.1.76}/Cargo.lock

@@ -1297,7 +1297,7 @@ dependencies = [
 
 [[package]]
 name = "cocoindex"
-version = "0.1.75"
+version = "0.1.76"
 dependencies = [
  "anyhow",
  "async-openai",

{cocoindex-0.1.75 → cocoindex-0.1.76}/Cargo.toml

@@ -2,7 +2,7 @@
 name = "cocoindex"
 # Version used for local development is always higher than others to take precedence.
 # Will be overridden for specific release versions.
-version = "0.1.75"
+version = "0.1.76"
 edition = "2024"
 rust-version = "1.88"
 

{cocoindex-0.1.75 → cocoindex-0.1.76}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cocoindex
-Version: 0.1.75
+Version: 0.1.76
 Requires-Dist: click>=8.1.8
 Requires-Dist: rich>=14.0.0
 Requires-Dist: python-dotenv>=1.1.0
@@ -12,9 +12,12 @@ Requires-Dist: ruff ; extra == 'dev'
 Requires-Dist: mypy ; extra == 'dev'
 Requires-Dist: pre-commit ; extra == 'dev'
 Requires-Dist: sentence-transformers>=3.3.1 ; extra == 'embeddings'
+Requires-Dist: colpali-engine ; extra == 'colpali'
 Requires-Dist: sentence-transformers>=3.3.1 ; extra == 'all'
+Requires-Dist: colpali-engine ; extra == 'all'
 Provides-Extra: dev
 Provides-Extra: embeddings
+Provides-Extra: colpali
 Provides-Extra: all
 License-File: LICENSE
 Summary: With CocoIndex, users declare the transformation, CocoIndex creates & maintains an index, and keeps the derived index up to date based on source update, with minimal computation and changes.

{cocoindex-0.1.75 → cocoindex-0.1.76}/docs/docs/core/flow_methods.mdx

@@ -1,13 +1,13 @@
 ---
-title: Run a Flow
+title: Operate a Flow
 toc_max_heading_level: 4
-description: Run a CocoIndex Flow, including build / update data in the target and evaluate the flow without changing the target.
+description: Operate a CocoIndex Flow, including build / update data in the target and evaluate the flow without changing the target.
 ---
 
 import Tabs from '@theme/Tabs';
 import TabItem from '@theme/TabItem';
 
-# Run a CocoIndex Flow
+# Operate a CocoIndex Flow
 
 After a flow is defined as discussed in [Flow Definition](/docs/core/flow_def), you can start to transform data with it.
 

{cocoindex-0.1.75 → cocoindex-0.1.76}/docs/docs/ops/functions.md

@@ -188,3 +188,53 @@ Input data:
 *   `text` (*Str*, required): The text to embed.
 
 Return: *Vector[Float32, N]*, where *N* is the dimension of the embedding vector determined by the model.
+
+## ColPaliEmbedImage
+
+`ColPaliEmbedImage` embeds images using the ColPali multimodal model.
+
+ColPali (Contextual Late-interaction over Patches) uses late interaction between image patch embeddings and text token embeddings for retrieval.
+
+:::note Optional Dependency Required
+
+This function requires the `colpali-engine` library, which is an optional dependency. Install CocoIndex with:
+
+```bash
+pip install 'cocoindex[colpali]'
+```
+:::
+
+The spec takes the following fields:
+
+*   `model` (`str`): The ColPali model name to use (e.g., "vidore/colpali-v1.2")
+
+Input data:
+
+*   `img_bytes` (*Bytes*): The image data in bytes format.
+
+Return: *Vector[Vector[Float32, N]]*, where *N* is the hidden dimension determined by the model. This returns a multi-vector format with variable patches and fixed hidden dimension.
+
+## ColPaliEmbedQuery
+
+`ColPaliEmbedQuery` embeds text queries using the ColPali multimodal model.
+
+This produces query embeddings compatible with ColPali image embeddings for late interaction scoring (MaxSim).
+
+:::note Optional Dependency Required
+
+This function requires the `colpali-engine` library, which is an optional dependency. Install CocoIndex with:
+
+```bash
+pip install 'cocoindex[colpali]'
+```
+:::
+
+The spec takes the following fields:
+
+*   `model` (`str`): The ColPali model name to use (e.g., "vidore/colpali-v1.2")
+
+Input data:
+
+*   `query` (*Str*): The text query to embed.
+
+Return: *Vector[Vector[Float32, N]]*, where *N* is the hidden dimension determined by the model. This returns a multi-vector format with variable tokens and fixed hidden dimension.

cocoindex-0.1.76/docs/docs/tutorials/manage_flow_dynamically.mdx

@@ -0,0 +1,302 @@
+---
+title: Manage Flows Dynamically
+description: "Learn how to dynamically manage multiple flow instances in CocoIndex. Create parameterized data indexing pipelines, handle persistent resources, perform updates, and implement memory-efficient caching for scalable applications."
+---
+
+# Manage Flows Dynamically
+
+You write a function, a.k.a. *flow definition*, to define indexing logic.
+Sometimes you want to reuse the same flow definition for multiple *flow instances* (a.k.a. *flow*), e.g. each takes input from different sources, exports to different targets, and even with slightly different parameters for transformation logic.
+
+## States of a flow instance
+
+A flow instance has states from two aspects:
+
+*   *In-process object*, of type `cocoindex.Flow`.
+*   *Persistent resource*, including states in the [internal storage](/docs/core/basics#internal-storage) and backend resources that are owned by the flow instance.
+
+A flow instance is ultimately a persistent resource. Its in-process object is a handle to operate on it. Consider file handles and database connections.
+CocoIndex provides APIs to *open* and *close* flow instances, and *setup* and *drop* the persistent resource.
+
+## Parameterize the flow definition
+
+In the example from the [Quickstart Guide](/docs/getting_started/quickstart), we decorate the flow definition function with a `@cocoindex.flow_def(name="DemoFlow")` decorator:
+
+```python title="Example in Quickstart Guide"
+@cocoindex.flow_def(name="TextEmbedding")
+def text_embedding_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
+    ...
+```
+
+This immediately creates the in-process object of the flow instance, using the given function as the flow definition.
+This is a shortcut of:
+
+```python
+def _text_embedding_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
+    ...
+
+text_embedding_flow = cocoindex.open_flow("TextEmbedding", _text_embedding_flow)
+```
+
+Here, `cocoindex.open_flow()` is the function that creates the in-process object of the flow instance, with the given name and flow definition function.
+You can directly call it dynamically with flow name created programmatically.
+
+Oftentimes, you also want to parameterize the flow definition function.
+For example, we may have a dataclass like this to hold the parameters of the flow:
+
+```python
+@dataclass
+class TextEmbeddingFlowParameters:
+    source_path: str
+    target_table_name: str
+```
+
+And consider we have a registry of parameters for all flow instances somewhere.
+For simplicity, we use a hardcoded `dict` here, and provide a simple function to get the parameters for a given flow name.
+In reality, the source of truth may come from a configuration file, a database, etc., and the function can be replaced by your own implementation.
+
+```python
+FLOW_PARAMETERS: dict[str, TextEmbeddingFlowParameters] = {
+    "foo": TextEmbeddingFlowParameters(source_path="/path/to/foo", target_table_name="foo_embeddings"),
+    "bar": TextEmbeddingFlowParameters(source_path="/path/to/bar", target_table_name="bar_embeddings"),
+}
+
+def get_flow_parameters(name: str) -> TextEmbeddingFlowParameters:
+    return FLOW_PARAMETERS[name]
+```
+
+Then you can have a function that returns the flow definition function for the given parameters:
+
+```python
+def text_embedding_flow_def(params: TextEmbeddingFlowParameters):
+    def _flow_def(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
+        # Add a data source to read files from the specified directory
+        data_scope["documents"] = flow_builder.add_source(
+            cocoindex.sources.LocalFile(path=params.source_path))
+
+        doc_embeddings = data_scope.add_collector()
+        ...
+
+        # Export the collected data to a Postgres table, with the specified table name.
+        doc_embeddings.export(
+            "doc_embeddings",
+            cocoindex.targets.Postgres(table_name=params.target_table_name),
+            primary_key_fields=["filename", "location"],
+        )
+
+    return _flow_def
+```
+
+With this, you can open flow instances dynamically with its parameters:
+
+```python
+text_embedding_flows: dict[str, cocoindex.Flow] = {}
+
+def get_text_embedding_flow(name: str) -> cocoindex.Flow:
+    flow = text_embedding_flows.get(name)
+
+    if flow is None:
+        params = get_flow_parameters(name)
+        flow = text_embedding_flows[name] = cocoindex.open_flow(f"TextEmbedding_{name}", text_embedding_flow_def(params))
+
+    return flow
+```
+
+## Operate on the flow instances
+
+### Setup the persistent resource
+
+After you instantiated and open flow instances dynamically, before you can perform any data updates, you need to make sure the persistent resource is ready.
+You can use the `setup()` method, e.g. modify the above code to:
+
+```python
+text_embedding_flows: dict[str, cocoindex.Flow] = {}
+
+def get_text_embedding_flow(name: str) -> cocoindex.Flow:
+    flow = text_embedding_flows.get(name)
+
+    if flow is None:
+        params = get_flow_parameters(name)
+        flow = text_embedding_flows[name] = cocoindex.open_flow(f"TextEmbedding_{name}", text_embedding_flow_def(params))
+        flow.setup(report_to_stdout=True)
+
+    return flow
+```
+
+`setup()` method synchronizes the persistent resource to a state that is consistent with the in-process object. For example,
+*   If the persistent resource is not there yet, it will create the backend resources for new targets.
+*   If your flow definition changed and a new target has been added since the last time of setup, it will create the backend resources for new targets.
+*   If an existing target is removed from the flow definition, it will drop the backend resources for the removed target.
+*   If nothing changed since the last time of setup, it will be a no-op. i.e. the `setup()` method is idempotent.
+
+`setup()` takes a `report_to_stdout` parameter to control whether to print the setup progress to the standard output.
+
+`setup()` takes care of all scenarios and makes sure the persistent resource is in the right state.
+It's generally safe to call it after you open a flow instance, even if you don't know whether the persistent resource already exists.
+
+
+### Perform data updates
+
+After you make sure the persistent resource is ready, you can perform data updates using the flow.
+
+The `update()` method updates the target defined by the flow.
+
+```python
+flow.update()
+```
+
+This performs a one-time data update. After the function returns, the target is up-to-date as of the moment when the function is called. For example, we can call `update()` to update the target after the flow is setup:
+
+```python
+def update_text_embedding_index(name: str):
+    flow = get_text_embedding_flow(name)
+    flow.update()
+```
+
+You can also do a live update.
+See the [Live Updates](/docs/tutorials/live_updates) tutorial for more details.
+
+
+### Close the flow object
+
+Sometimes you don't want to hold the in-process object forever.
+You can free up the memory resources by closing the flow instances with the `close()` method.
+
+For example, the `dict` we managed above behaves like a cache to hold the flow instances.
+If a specific flow isn't used for a while, we may close it.
+The `TTLCache` from [`cachetools`](https://pypi.org/project/cachetools/) package provides exactly this functionality.
+We can rewrite the above code a little bit.
+First, we bring in necessary imports:
+
+```python
+from cachetools import cached, TTLCache
+```
+
+Then we define our own version of `TTLCache` to make it call the `close()` method when the flow instance is evicted from the cache:
+
+```python
+class MyTTLCache(TTLCache):
+    def popitem(self):
+        # Close the flow instance when it is evicted from the cache
+        key, flow = super().popitem()
+        flow.close()
+        return key, flow
+```
+
+With this, we can modify our `get_text_embedding_flow()` function to use `MyTTLCache` to cache the flow instances, instead of managing our own `dict`:
+
+```python
+@cached(cache=MyTTLCache(maxsize=20, ttl=600))
+def get_text_embedding_flow(name: str) -> cocoindex.Flow:
+    params = get_flow_parameters(name)
+    flow = cocoindex.open_flow(f"TextEmbedding_{name}", text_embedding_flow_def(params))
+    flow.setup(report_to_stdout=True)
+    return flow
+```
+
+The `@cached()` decorator from `cachetools` package automatically manages the cache for us (and it also offers thread safety!).
+Once a flow is not touched for 10 minutes, it will call the `popitem()` method, which will close the in-memory flow object.
+
+
+### Drop the persistent resource
+
+Occasionally, you may want to drop the persistent resource of a flow.
+The `drop()` method is for this purpose.
+
+```python
+def drop_text_embedding_index(name: str):
+    flow = get_text_embedding_flow(name)
+    flow.drop()
+```
+
+This will drop the persistent resource of the flow.
+The in-memory flow object is still alive, and can be reused until it's closed.
+For example, you can still call `setup()` again.
+
+## Put it all together
+
+```python
+import cocoindex
+from cachetools import cached, TTLCache
+from dataclasses import dataclass
+
+@dataclass
+class TextEmbeddingFlowParameters:
+    source_path: str
+    target_table_name: str
+
+FLOW_PARAMETERS: dict[str, TextEmbeddingFlowParameters] = {
+    "foo": TextEmbeddingFlowParameters(source_path="/path/to/foo", target_table_name="foo_embeddings"),
+    "bar": TextEmbeddingFlowParameters(source_path="/path/to/bar", target_table_name="bar_embeddings"),
+}
+
+# Placeholder to get the parameters for a given flow name. You can replace this with your own implementation.
+def get_flow_parameters(name: str) -> TextEmbeddingFlowParameters:
+    return FLOW_PARAMETERS[name]
+
+
+def text_embedding_flow_def(params: TextEmbeddingFlowParameters):
+    def _flow_def(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
+        # Add a data source to read files from a directory
+        data_scope["documents"] = flow_builder.add_source(
+            cocoindex.sources.LocalFile(path=params.source_path))
+
+        doc_embeddings = data_scope.add_collector()
+        ...
+
+        # Export the collected data to a Postgres table, with the specified table name.
+        doc_embeddings.export(
+            "doc_embeddings",
+            cocoindex.targets.Postgres(table_name=params.target_table_name),
+            primary_key_fields=["filename", "location"],
+        )
+
+    return _flow_def
+
+class MyTTLCache(TTLCache):
+    def popitem(self):
+        # Close the flow instance when it is evicted from the cache
+        key, flow = super().popitem()
+        flow.close()
+        return key, flow
+
+@cached(cache=MyTTLCache(maxsize=20, ttl=600))
+def get_text_embedding_flow(name: str) -> cocoindex.Flow:
+    params = get_flow_parameters(name)
+    flow = cocoindex.open_flow(f"TextEmbedding_{name}", text_embedding_flow_def(params))
+    flow.setup(report_to_stdout=True)
+    return flow
+
+def update_text_embedding_index(name: str):
+    flow = get_text_embedding_flow(name)
+    flow.update()
+
+def drop_text_embedding_index(name: str):
+    flow = get_text_embedding_flow(name)
+    flow.drop()
+```
+
+This provides a skeleton.
+With this, you can trigger `update_text_embedding_index()` and `drop_text_embedding_index()` from your application, e.g. a web server API.
+
+## Takeaways
+
+From this tutorial, we walked through major flow management / operation APIs provided by CocoIndex. These APIs can be categorized into three aspects:
+
+| Aspect | APIs | Description |
+|--------|------|-------------|
+| Life of in-process flow object | `open_flow()`, `Flow.close()` | Create and destroy the in-memory handle to operate on flow instances |
+| Life of persistent resource | `Flow.setup()`, `Flow.drop()` | Create and destroy the backend resources and internal storage |
+| Data updates | `Flow.update()`, `FlowLiveUpdater` | Execute the indexing logic to update targets. *Requires persistent resource to be up-to-date first.*  |
+
+
+For simplicity, we use an in-memory `dict` as source of truth for the flow parameters.
+You can replace it with your own mechanism, e.g. table from a database, a configuration file, etc.
+You can trigger these APIs from your applications specific to your use case, e.g. from a specific API endpoint of a web server.
+
+## Further readings
+
+You can see the following documents for more details:
+
+*   [CocoIndex Flow Definition: Entry Point](/docs/core/flow_def#entry-point)
+*   [Operate a Flow](/docs/core/flow_methods)

{cocoindex-0.1.75 → cocoindex-0.1.76}/docs/sidebars.ts

@@ -18,6 +18,7 @@ const sidebars: SidebarsConfig = {
       collapsed: false,
       items: [
         'tutorials/live_updates',
+        'tutorials/manage_flow_dynamically',
       ],
     },
     {

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/amazon_s3_embedding/pyproject.toml

@@ -4,7 +4,7 @@ version = "0.1.0"
 description = "Simple example for cocoindex: build embedding index based on Amazon S3 files."
 requires-python = ">=3.11"
 dependencies = [
-    "cocoindex[embeddings]>=0.1.67",
+    "cocoindex[embeddings]>=0.1.75",
     "python-dotenv>=1.0.1",
     "psycopg[binary,pool]",
 ]

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/azure_blob_embedding/pyproject.toml

@@ -4,7 +4,7 @@ version = "0.1.0"
 description = "Simple example for cocoindex: build embedding index based on Azure Blob Storage files."
 requires-python = ">=3.11"
 dependencies = [
-    "cocoindex[embeddings]>=0.1.67",
+    "cocoindex[embeddings]>=0.1.75",
     "python-dotenv>=1.0.1",
     "psycopg[binary,pool]",
 ]

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/code_embedding/pyproject.toml

@@ -4,7 +4,7 @@ version = "0.1.0"
 description = "Simple example for cocoindex: build embedding index based on source code."
 requires-python = ">=3.11"
 dependencies = [
-    "cocoindex[embeddings]>=0.1.67",
+    "cocoindex[embeddings]>=0.1.75",
     "python-dotenv>=1.0.1",
     "pgvector>=0.4.1",
     "psycopg[binary,pool]",

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/custom_output_files/pyproject.toml

@@ -3,7 +3,7 @@ name = "custom-output-files"
 version = "0.1.0"
 description = "Simple example for cocoindex: convert markdown files to HTML files and save them to a local directory."
 requires-python = ">=3.11"
-dependencies = ["cocoindex>=0.1.74", "markdown-it-py[linkify,plugins]"]
+dependencies = ["cocoindex>=0.1.75", "markdown-it-py[linkify,plugins]"]
 
 [tool.setuptools]
 packages = []

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/docs_to_knowledge_graph/pyproject.toml

@@ -3,7 +3,7 @@ name = "manuals-to-kg"
 version = "0.1.0"
 description = "Simple example for cocoindex: extract triples from files and build knowledge graph."
 requires-python = ">=3.11"
-dependencies = ["cocoindex>=0.1.67"]
+dependencies = ["cocoindex>=0.1.75"]
 
 [tool.setuptools]
 packages = []

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/face_recognition/pyproject.toml

@@ -4,7 +4,7 @@ version = "0.1.0"
 description = "Build index for papers with both metadata and content embeddings"
 requires-python = ">=3.11"
 dependencies = [
-    "cocoindex>=0.1.71",
+    "cocoindex>=0.1.75",
     "face-recognition>=1.3.0",
     "pillow>=10.0.0",
     "numpy>=1.26.0",

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/fastapi_server_docker/requirements.txt

@@ -1,4 +1,4 @@
-cocoindex[embeddings]>=0.1.63
+cocoindex[embeddings]>=0.1.75
 python-dotenv>=1.0.1
 fastapi==0.115.12
 fastapi-cli==0.0.7

{cocoindex-0.1.75 → cocoindex-0.1.76}/examples/gdrive_text_embedding/pyproject.toml

@@ -4,7 +4,7 @@ version = "0.1.0"
 description = "Simple example for cocoindex: build embedding index based on Google Drive files."
 requires-python = ">=3.11"
 dependencies = [
-    "cocoindex[embeddings]>=0.1.67",
+    "cocoindex[embeddings]>=0.1.75",
     "python-dotenv>=1.0.1",
     "psycopg[binary,pool]",
 ]

cocoindex-0.1.76/examples/image_search/.env

@@ -0,0 +1 @@
+export COCOINDEX_DATABASE_URL="postgres://cocoindex:cocoindex@localhost/cocoindex"

cocoindex-0.1.76/examples/image_search/README.md

@@ -0,0 +1,105 @@
+# Image Search with CocoIndex
+[![GitHub](https://img.shields.io/github/stars/cocoindex-io/cocoindex?color=5B5BD6)](https://github.com/cocoindex-io/cocoindex)
+
+We will build live image search and query it with natural language, using multimodal embedding models. We use CocoIndex to build real-time indexing flow. During running, you can add new files to the folder and it only processes changed files, indexing them within a minute.
+
+We appreciate a star ⭐ at [CocoIndex Github](https://github.com/cocoindex-io/cocoindex) if this is helpful.
+
+<img width="1105" alt="cover" src="https://github.com/user-attachments/assets/544fb80d-c085-4150-84b6-b6e62c4a12b9" />
+
+## Two Implementation Options
+
+This example provides two different image search implementations:
+
+### 1. CLIP-based Search (`main.py`)
+- **Model**: CLIP ViT-L/14 (OpenAI)
+- **Embedding**: Single-vector embeddings (768 dimensions)
+- **Search**: Standard cosine similarity
+
+### 2. ColPali-based Search (`colpali_main.py`)
+- **Model**: ColPali (Contextual Late-interaction over Patches)
+- **Embedding**: Multi-vector embeddings with late interaction
+- **Search**: MaxSim scoring for optimal patch-level matching
+- **Performance**: Better for document/text-in-image search
+
+## Technologies
+- CocoIndex for ETL and live update
+- **CLIP ViT-L/14** OR **ColPali** - Multimodal embedding models
+- Qdrant for Vector Storage (with multi-vector support for ColPali)
+- FastAPI for backend
+- Ollama (Optional) for generating image captions
+
+## Setup
+- [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one.
+
+- Make sure Qdrant is running
+  ```
+  docker run -d -p 6334:6334 -p 6333:6333 qdrant/qdrant
+  ```
+
+## (Optional) Run Ollama
+
+- This enables automatic image captioning
+```
+ollama pull gemma3
+ollama serve
+export OLLAMA_MODEL="gemma3"  # Optional, for caption generation
+```
+
+## Run the App
+
+### Option 1: CLIP-based Search
+- Install dependencies:
+  ```
+  pip install -e .
+  ```
+
+- Run CLIP Backend:
+  ```
+  uvicorn main:app --reload --host 0.0.0.0 --port 8000
+  ```
+
+### Option 2: ColPali-based Search
+- Install dependencies:
+  ```
+  pip install -e .
+  pip install 'cocoindex[colpali]'  # Adds ColPali support
+  ```
+
+- Configure model (optional):
+  ```sh
+  export COLPALI_MODEL="vidore/colpali-v1.2"  # Default model
+  ```
+
+- Run ColPali Backend:
+  ```
+  uvicorn colpali_main:app --reload --host 0.0.0.0 --port 8000
+  ```
+
+Note that recent Nvidia GPUs (RTX 5090) will not work with the Stable pytorch version up to 2.7.1
+
+If you get this error:
+
+```
+The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 sm_80 sm_86 sm_90 compute_37.
+```
+
+You can install the nightly pytorch build here: https://pytorch.org/get-started/locally/
+
+```sh
+pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu129
+```
+
+### Frontend (same for both)
+- Run Frontend:
+  ```
+  cd frontend
+  npm install
+  npm run dev
+  ```
+
+Go to `http://localhost:5173` to search. The frontend works with both backends identically.
+
+## Performance Notes
+- **CLIP**: Faster, good for general image-text matching
+- **ColPali**: More accurate for document images and text-heavy content, supports multi-vector late interaction for better precision