cocoindex 0.1.70__tar.gz → 0.1.72__tar.gz

{cocoindex-0.1.70 → cocoindex-0.1.72}/Cargo.lock

@@ -1297,7 +1297,7 @@ dependencies = [
 
 [[package]]
 name = "cocoindex"
-version = "0.1.70"
+version = "0.1.72"
 dependencies = [
  "anyhow",
  "async-openai",

{cocoindex-0.1.70 → cocoindex-0.1.72}/Cargo.toml

@@ -2,7 +2,7 @@
 name = "cocoindex"
 # Version used for local development is always higher than others to take precedence.
 # Will be overridden for specific release versions.
-version = "0.1.70"
+version = "0.1.72"
 edition = "2024"
 rust-version = "1.88"
 

{cocoindex-0.1.70 → cocoindex-0.1.72}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cocoindex
-Version: 0.1.70
+Version: 0.1.72
 Requires-Dist: click>=8.1.8
 Requires-Dist: rich>=14.0.0
 Requires-Dist: python-dotenv>=1.1.0
@@ -52,18 +52,18 @@ Ultra performant data transformation framework for AI, with core engine written
 ⭐ Drop a star to help us grow!
 
 <div align="center">
-    
+
 <!-- Keep these links. Translations will automatically update with the README. -->
-[Deutsch](https://readme-i18n.com/cocoindex-io/cocoindex?lang=de) | 
-[English](https://readme-i18n.com/cocoindex-io/cocoindex?lang=en) | 
-[Español](https://readme-i18n.com/cocoindex-io/cocoindex?lang=es) | 
-[français](https://readme-i18n.com/cocoindex-io/cocoindex?lang=fr) | 
-[日本語](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ja) | 
-[한국어](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ko) | 
-[Português](https://readme-i18n.com/cocoindex-io/cocoindex?lang=pt) | 
-[Русский](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ru) | 
+[Deutsch](https://readme-i18n.com/cocoindex-io/cocoindex?lang=de) |
+[English](https://readme-i18n.com/cocoindex-io/cocoindex?lang=en) |
+[Español](https://readme-i18n.com/cocoindex-io/cocoindex?lang=es) |
+[français](https://readme-i18n.com/cocoindex-io/cocoindex?lang=fr) |
+[日本語](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ja) |
+[한국어](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ko) |
+[Português](https://readme-i18n.com/cocoindex-io/cocoindex?lang=pt) |
+[Русский](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ru) |
 [中文](https://readme-i18n.com/cocoindex-io/cocoindex?lang=zh)
-    
+
 </div>
 
 </br>
@@ -208,6 +208,7 @@ It defines an index flow like this:
 | [FastAPI Server with Docker](examples/fastapi_server_docker) | Run the semantic search server in a Dockerized FastAPI setup |
 | [Product Recommendation](examples/product_recommendation) | Build real-time product recommendations with LLM and graph database|
 | [Image Search with Vision API](examples/image_search) | Generates detailed captions for images using a vision model, embeds them, enables live-updating semantic search via FastAPI and served on a React frontend|
+| [Face Recognition](examples/face_recognition) | Recognize faces in images and build embedding index |
 | [Paper Metadata](examples/paper_metadata) | Index papers in PDF files, and build metadata tables for each paper |
 
 More coming and stay tuned 👀!

{cocoindex-0.1.70 → cocoindex-0.1.72}/README.md

@@ -27,18 +27,18 @@ Ultra performant data transformation framework for AI, with core engine written
 ⭐ Drop a star to help us grow!
 
 <div align="center">
-    
+
 <!-- Keep these links. Translations will automatically update with the README. -->
-[Deutsch](https://readme-i18n.com/cocoindex-io/cocoindex?lang=de) | 
-[English](https://readme-i18n.com/cocoindex-io/cocoindex?lang=en) | 
-[Español](https://readme-i18n.com/cocoindex-io/cocoindex?lang=es) | 
-[français](https://readme-i18n.com/cocoindex-io/cocoindex?lang=fr) | 
-[日本語](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ja) | 
-[한국어](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ko) | 
-[Português](https://readme-i18n.com/cocoindex-io/cocoindex?lang=pt) | 
-[Русский](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ru) | 
+[Deutsch](https://readme-i18n.com/cocoindex-io/cocoindex?lang=de) |
+[English](https://readme-i18n.com/cocoindex-io/cocoindex?lang=en) |
+[Español](https://readme-i18n.com/cocoindex-io/cocoindex?lang=es) |
+[français](https://readme-i18n.com/cocoindex-io/cocoindex?lang=fr) |
+[日本語](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ja) |
+[한국어](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ko) |
+[Português](https://readme-i18n.com/cocoindex-io/cocoindex?lang=pt) |
+[Русский](https://readme-i18n.com/cocoindex-io/cocoindex?lang=ru) |
 [中文](https://readme-i18n.com/cocoindex-io/cocoindex?lang=zh)
-    
+
 </div>
 
 </br>
@@ -183,6 +183,7 @@ It defines an index flow like this:
 | [FastAPI Server with Docker](examples/fastapi_server_docker) | Run the semantic search server in a Dockerized FastAPI setup |
 | [Product Recommendation](examples/product_recommendation) | Build real-time product recommendations with LLM and graph database|
 | [Image Search with Vision API](examples/image_search) | Generates detailed captions for images using a vision model, embeds them, enables live-updating semantic search via FastAPI and served on a React frontend|
+| [Face Recognition](examples/face_recognition) | Recognize faces in images and build embedding index |
 | [Paper Metadata](examples/paper_metadata) | Index papers in PDF files, and build metadata tables for each paper |
 
 More coming and stay tuned 👀!

{cocoindex-0.1.70 → cocoindex-0.1.72}/docs/docs/core/custom_function.mdx

@@ -148,6 +148,17 @@ Custom functions take the following additional parameters:
     When the version is changed, the function will be re-executed even if cache is enabled.
     It's required to be set if `cache` is `True`.
 
+*   `arg_relationship: tuple[ArgRelationship, str]`: It specifies the relationship between an input argument and the output,
+    e.g. `(ArgRelationship.CHUNKS_BASE_TEXT, "content")` means the output is chunks for the text represented by the
+    input argument with name `content`.
+    This provides metadata for tools, e.g. CocoInsight.
+    Currently the following attributes are supported:
+
+    *   `ArgRelationship.CHUNKS_BASE_TEXT`:
+        The output is chunks for the text represented by the input argument. In this case, the output is expected to be a *Table*, whose each row represents a text chunk, and the first column has type *Range*, representing the range of the text chunk.
+    *   `ArgRelationship.EMBEDDING_ORIGIN_TEXT`: The output is embedding vector for the text represented by the input argument. The output is expected to be a *Vector*.
+    *   `ArgRelationship.RECTS_BASE_IMAGE`: The output is rectangles for the image represented by the input argument. The output is expected to be a *Table*, whose each row represents a rectangle, and the first column has type *Struct*, with fields `min_x`, `min_y`, `max_x`, `max_y` to represent the coordinates of the rectangle.
+
 For example:
 
 <Tabs>

{cocoindex-0.1.70 → cocoindex-0.1.72}/docs/docs/core/data_types.mdx

@@ -21,12 +21,13 @@ All you need to do is to make sure the data passed to functions and targets are
 Each type in CocoIndex type system is mapped to one or multiple types in Python.
 When you define a [custom function](/docs/core/custom_function), you need to annotate the data types of arguments and return values.
 
-*   When you pass a Python value to the engine (e.g. return values of a custom function), type annotation is required,
-    as it provides the ground truth of the data type in the flow.
+*   When you pass a Python value to the engine (e.g. return values of a custom function), a specific type annotation is required.
+    The type annotation needs to be specific in describing the target data type, as it provides the ground truth of the data type in the flow.
 
 *   When you use a Python variable to bind to an engine value (e.g. arguments of a custom function),
-    we use the type annotation as a guidance to construct the Python value.
-    Type annotation is optional for basic types and struct types, and required for table types.
+    the engine already knows the specific data type, so we don't require a specific type annotation, e.g. type annotations can be omitted, or you can use `Any` at any level.
+    When a specific type annotation is provided, it's still used as a guidance to construct the Python value with compatible type.
+    Otherwise, we will bind to a default Python type.
 
 ### Basic Types
 
@@ -54,7 +55,7 @@ This is the list of all primitive types supported by CocoIndex:
 Notes:
 
 *   For some CocoIndex types, we support multiple Python types. You can annotate with any of these Python types.
-    The first one is the default type, i.e. CocoIndex will create a value with this type when the type annotation is not provided (e.g. for arguments of a custom function).
+    The first one is the default type, i.e. CocoIndex will create a value with this type when a specific type annotation is not provided (e.g. for arguments of a custom function).
 
 *   All Python types starting with `cocoindex.` are type aliases exported by CocoIndex. They're annotated types based on certain Python types:
 
@@ -86,7 +87,7 @@ Optionally, it can have a fixed dimension. Noted as *Vector[Type]* or *Vector[Ty
 
 It supports the following Python types:
 
-*   `cocoindex.Vector[T]` or `cocoindex.Vector[T, typing.Literal[Dim]]`, e.g. `cocoindex.Vector[cocoindex.Float32]` or `cocoindex.Vector[cocoindex.Float32, 384]`
+*   `cocoindex.Vector[T]` or `cocoindex.Vector[T, typing.Literal[Dim]]`, e.g. `cocoindex.Vector[cocoindex.Float32]` or `cocoindex.Vector[cocoindex.Float32, typing.Literal[384]]`
     *   The underlying Python type is `numpy.typing.NDArray[T]` where `T` is a numpy numeric type (`numpy.int64`, `numpy.float32` or `numpy.float64`), or `list[T]` otherwise
 *   `numpy.typing.NDArray[T]` where `T` is a numpy numeric type
 *   `list[T]`
@@ -136,7 +137,7 @@ Both `Person` and `PersonTuple` are valid Struct types in CocoIndex, with identi
 Choose `dataclass` for mutable objects or when you need additional methods, and `NamedTuple` for immutable, lightweight structures.
 
 Besides, for arguments of custom functions, CocoIndex also supports using dictionaries (`dict[str, Any]`) to represent a *Struct* type.
-It's the default Python type if you don't annotate the function argument.
+It's the default Python type if you don't annotate the function argument with a specific type.
 
 ### Table Types
 
@@ -152,11 +153,16 @@ The row order of a *KTable* is not preserved.
 Type of the first column (key column) must be a [key type](#key-types).
 
 In Python, a *KTable* type is represented by `dict[K, V]`. 
-The `V` should be a *Struct* type, either a `dataclass` or `NamedTuple`, representing the value fields of each row.
+The `K` should be the type binding to a key type,
+and the `V` should be the type binding to a *Struct* type representing the value fields of each row.
+When the specific type annotation is not provided,
+the key type is bound to a tuple with its key parts when it's a *Struct* type, the value type is bound to `dict[str, Any]`.
+
+
 For example, you can use `dict[str, Person]` or `dict[str, PersonTuple]` to represent a *KTable*, with 4 columns: key (*Str*), `first_name` (*Str*), `last_name` (*Str*), `dob` (*Date*).
+It's bound to `dict[str, dict[str, Any]]` if you don't annotate the function argument with a specific type.
 
 Note that if you want to use a *Struct* as the key, you need to ensure its value in Python is immutable. For `dataclass`, annotate it with `@dataclass(frozen=True)`. For `NamedTuple`, immutability is built-in. For example:
-For example:
 
 ```python
 @dataclass(frozen=True)
@@ -170,14 +176,16 @@ class PersonKeyTuple(NamedTuple):
 ```
 
 Then you can use `dict[PersonKey, Person]` or `dict[PersonKeyTuple, PersonTuple]` to represent a KTable keyed by `PersonKey` or `PersonKeyTuple`.
+It's bound to `dict[(str, str), dict[str, Any]]` if you don't annotate the function argument with a specific type.
 
 
 #### LTable
 
 *LTable* is a *Table* type whose row order is preserved. *LTable* has no key column.
 
-In Python, a *LTable* type is represented by `list[R]`, where `R` is a dataclass representing a row.
+In Python, a *LTable* type is represented by `list[R]`, where `R` is the type binding to the *Struct* type representing the value fields of each row.
 For example, you can use `list[Person]` to represent a *LTable* with 3 columns: `first_name` (*Str*), `last_name` (*Str*), `dob` (*Date*).
+It's bound to `list[dict[str, Any]]` if you don't annotate the function argument with a specific type.
 
 ## Key Types
 

{cocoindex-0.1.70 → cocoindex-0.1.72}/docs/docs/ops/targets.md

@@ -32,6 +32,13 @@ Here's how CocoIndex data elements map to Postgres elements during export:
 For example, if you have a data collector that collects rows with fields `id`, `title`, and `embedding`, it will be exported to a Postgres table with corresponding columns.
 It should be a unique table, meaning that no other export target should export to the same table.
 
+:::warning vector type mapping to Postgres
+
+Since vectors in pgvector must have fixed dimension, we only map vectors of number types with fixed dimension (i.e. *Vector[cocoindex.Float32, N]*, *Vector[cocoindex.Float64, N]*, and *Vector[cocoindex.Int64, N]*) to `vector(N)` columns.
+For all other vector types, we map them to `jsonb` columns.
+
+:::
+
 #### Spec
 
 The spec takes the following fields:
@@ -58,6 +65,13 @@ Here's how CocoIndex data elements map to Qdrant elements during export:
 
 *Vector[Float32, N]*, *Vector[Float64, N]* and *Vector[Int64, N]* types fit into Qdrant vector.
 
+:::warning vector type mapping to Qdrant
+
+Since vectors in Qdrant must have fixed dimension, we only map vectors of number types with fixed dimension (i.e. *Vector[cocoindex.Float32, N]*, *Vector[cocoindex.Float64, N]*, and *Vector[cocoindex.Int64, N]*) to Qdrant vectors.
+For all other vector types, we map to Qdrant payload as JSON arrays.
+
+:::
+
 #### Spec
 
 The spec takes the following fields:

cocoindex-0.1.72/docs/docs/tutorials/live_updates.md

@@ -0,0 +1,156 @@
+---
+title: Live Updates
+description: "Keep your indexes up-to-date with live updates in CocoIndex."
+---
+
+# Live Updates
+
+CocoIndex is designed to keep your indexes synchronized with your data sources. This is achieved through a feature called **live updates**, which automatically detects changes in your sources and updates your indexes accordingly. This ensures that your search results and data analysis are always based on the most current information.
+
+## How Live Updates Work
+
+Live updates in CocoIndex can be triggered in two main ways:
+
+1.  **Refresh Interval:** You can configure a `refresh_interval` for any data source. CocoIndex will then periodically check the source for any new, updated, or deleted data. This is a simple and effective way to keep your index fresh, especially for sources that don't have a built-in change notification system.
+
+2.  **Change Capture Mechanisms:** Some data sources offer more sophisticated ways to track changes. For example:
+    *   **Amazon S3:** You can configure an SQS queue to receive notifications whenever a file is added, modified, or deleted in your S3 bucket. CocoIndex can listen to this queue and trigger an update instantly.
+    *   **Google Drive:** The Google Drive source can be configured to poll for recent changes, which is more efficient than a full refresh.
+
+When a change is detected, CocoIndex performs an **incremental update**. This means it only re-processes the data that has been affected by the change, without having to re-index your entire dataset. This makes the update process fast and efficient.
+
+Here's an example of how to set up a source with a `refresh_interval`:
+
+```python
+@cocoindex.flow_def(name="LiveUpdateExample")
+def live_update_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
+    # Source: local files in the 'data' directory
+    data_scope["documents"] = flow_builder.add_source(
+        cocoindex.sources.LocalFile(path="data"),
+        refresh_interval=cocoindex.timedelta(seconds=5),
+    )
+    # ...
+```
+
+By setting `refresh_interval` to 5 seconds, we're telling CocoIndex to check for changes in the `data` directory every 5 seconds.
+
+## Implementing Live Updates
+
+You can enable live updates using either the CocoIndex CLI or the Python library.
+
+### Using the CLI
+
+To start a live update process from the command line, use the `update` command with the `-L` or `--live` flag:
+
+```bash
+cocoindex update -L your_flow_definition_file.py
+```
+
+This will start a long-running process that continuously monitors your data sources for changes and updates your indexes in real-time. You can stop the process by pressing `Ctrl+C`.
+
+### Using the Python Library
+
+For more control over the live update process, you can use the `FlowLiveUpdater` class in your Python code. This is particularly useful when you want to integrate CocoIndex into a larger application.
+
+The `FlowLiveUpdater` can be used as a context manager, which automatically starts the updater when you enter the `with` block and stops it when you exit. The `wait()` method will block until the updater is aborted (e.g., by pressing `Ctrl+C`).
+
+Here's how you can use `FlowLiveUpdater` to start and manage a live update process:
+
+```python
+import cocoindex
+
+# Create a FlowLiveUpdater instance
+with cocoindex.FlowLiveUpdater(live_update_flow, cocoindex.FlowLiveUpdaterOptions(print_stats=True)) as updater:
+    print("Live updater started. Press Ctrl+C to stop.")
+    # The updater runs in the background.
+    # The wait() method blocks until the updater is stopped.
+    updater.wait()
+
+print("Live updater stopped.")
+```
+
+#### Getting Status Updates
+
+You can also get status updates from the `FlowLiveUpdater` to monitor the update process. The `next_status_updates()` method blocks until there is a new status update.
+
+```python
+import cocoindex
+
+updater = cocoindex.FlowLiveUpdater(live_update_flow)
+updater.start()
+
+while True:
+    updates = updater.next_status_updates()
+
+    if not updates.active_sources:
+        print("All sources have finished processing.")
+        break
+
+    for source_name in updates.updated_sources:
+        print(f"Source '{source_name}' has been updated.")
+
+updater.wait()
+```
+
+This allows you to react to updates in your application, for example, by notifying users or triggering downstream processes.
+
+## Example
+
+Let's walk through an example of how to set up a live update flow. For the complete, runnable code, see the [live updates example](https://github.com/cocoindex-io/cocoindex/tree/main/examples/live_updates) in the CocoIndex repository.
+
+### 1. Setting up the Source
+
+The first step is to define a source and configure a `refresh_interval`. In this example, we'll use a `LocalFile` source to monitor a directory named `data`.
+
+```python
+@cocoindex.flow_def(name="LiveUpdateExample")
+def live_update_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
+    # Source: local files in the 'data' directory
+    data_scope["documents"] = flow_builder.add_source(
+        cocoindex.sources.LocalFile(path="data"),
+        refresh_interval=cocoindex.timedelta(seconds=5),
+    )
+
+    # Collector
+    collector = data_scope.add_collector()
+    with data_scope["documents"].row() as doc:
+        collector.collect(filename=doc["filename"], content=doc["content"])
+
+    # Target: Postgres database
+    collector.export(
+        "documents_index",
+        cocoindex.targets.Postgres(),
+        primary_key_fields=["filename"]
+    )
+```
+
+By setting `refresh_interval` to 5 seconds, we're telling CocoIndex to check for changes in the `data` directory every 5 seconds.
+
+### 2. Running the Live Updater
+
+Once the flow is defined, you can use the `FlowLiveUpdater` to start the live update process.
+
+```python
+def main():
+    # Initialize CocoIndex
+    cocoindex.init()
+
+    # Setup the flow
+    live_update_flow.setup(report_to_stdout=True)
+
+    # Start the live updater
+    with cocoindex.FlowLiveUpdater(live_update_flow, cocoindex.FlowLiveUpdaterOptions(print_stats=True)) as updater:
+        print("Live updater started. Watching for changes in the 'data' directory.")
+        updater.wait()
+
+if __name__ == "__main__":
+    main()
+```
+
+The `FlowLiveUpdater` will run in the background, and the `updater.wait()` call will block until the process is stopped.
+
+## Conclusion
+
+Live updates is a powerful feature of CocoIndex that ensures your indexes are always fresh. By using a combination of refresh intervals and source-specific change capture mechanisms, you can build responsive, real-time applications that are always in sync with your data.
+
+For more detailed information on the `FlowLiveUpdater` and other live update options, please refer to the [Run a Flow documentation](https://cocoindex.io/docs/core/flow_methods#live-update).

{cocoindex-0.1.70 → cocoindex-0.1.72}/docs/sidebars.ts

@@ -12,6 +12,14 @@ const sidebars: SidebarsConfig = {
         'getting_started/installation',
       ],
     },
+    {
+      type: 'category',
+      label: 'Tutorials',
+      collapsed: false,
+      items: [
+        'tutorials/live_updates',
+      ],
+    },
     {
       type: 'category',
       label: 'CocoIndex Core',

cocoindex-0.1.72/examples/face_recognition/README.md

@@ -0,0 +1,51 @@
+# Recognize faces in images and build embedding index
+[![GitHub](https://img.shields.io/github/stars/cocoindex-io/cocoindex?color=5B5BD6)](https://github.com/cocoindex-io/cocoindex)
+
+
+In this example, we will recognize faces in images and build embedding index.
+
+We appreciate a star ⭐ at [CocoIndex Github](https://github.com/cocoindex-io/cocoindex) if this is helpful.
+
+## Steps
+### Indexing Flow
+
+1. We will ingest a list of images.
+2. For each image, we:
+   - Extract faces from the image.
+   - Compute embeddings for each face.
+3. We will export to the following tables in Postgres with PGVector:
+   - Filename, rect, embedding for each face.
+
+
+## Prerequisite
+
+1.  [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one.
+
+2.  dependencies:
+
+    ```bash
+    pip install -e .
+    ```
+
+## Run
+
+Update index, which will also setup the tables at the first time:
+
+```bash
+cocoindex update --setup main.py
+```
+
+You can also run the command with `-L`, which will watch for file changes and update the index automatically.
+
+```bash
+cocoindex update --setup -L main.py
+```
+
+## CocoInsight
+I used CocoInsight (Free beta now) to troubleshoot the index generation and understand the data lineage of the pipeline. It just connects to your local CocoIndex server, with zero pipeline data retention. Run following command to start CocoInsight:
+
+```
+cocoindex server -ci main.py
+```
+
+Then open the CocoInsight UI at [https://cocoindex.io/cocoinsight](https://cocoindex.io/cocoinsight).

cocoindex-0.1.72/examples/face_recognition/main.py

@@ -0,0 +1,120 @@
+import cocoindex
+import io
+import dataclasses
+import datetime
+import typing
+
+import face_recognition
+from PIL import Image
+import numpy as np
+
+
+@dataclasses.dataclass
+class ImageRect:
+    min_x: int
+    min_y: int
+    max_x: int
+    max_y: int
+
+
+@dataclasses.dataclass
+class FaceBase:
+    """A face in an image."""
+
+    rect: ImageRect
+    image: bytes
+
+
+MAX_IMAGE_WIDTH = 1280
+
+
+@cocoindex.op.function(
+    cache=True,
+    behavior_version=1,
+    gpu=True,
+    arg_relationship=(cocoindex.op.ArgRelationship.RECTS_BASE_IMAGE, "content"),
+)
+def extract_faces(content: bytes) -> list[FaceBase]:
+    """Extract the first pages of a PDF."""
+    orig_img = Image.open(io.BytesIO(content)).convert("RGB")
+
+    # The model is too slow on large images, so we resize them if too large.
+    if orig_img.width > MAX_IMAGE_WIDTH:
+        ratio = orig_img.width * 1.0 / MAX_IMAGE_WIDTH
+        img = orig_img.resize(
+            (MAX_IMAGE_WIDTH, int(orig_img.height / ratio)),
+            resample=Image.Resampling.BICUBIC,
+        )
+    else:
+        ratio = 1.0
+        img = orig_img
+
+    # Extract face locations.
+    locs = face_recognition.face_locations(np.array(img), model="cnn")
+
+    faces: list[FaceBase] = []
+    for min_y, max_x, max_y, min_x in locs:
+        rect = ImageRect(
+            min_x=int(min_x * ratio),
+            min_y=int(min_y * ratio),
+            max_x=int(max_x * ratio),
+            max_y=int(max_y * ratio),
+        )
+
+        # Crop the face and save it as a PNG.
+        buf = io.BytesIO()
+        orig_img.crop((rect.min_x, rect.min_y, rect.max_x, rect.max_y)).save(
+            buf, format="PNG"
+        )
+        face = buf.getvalue()
+        faces.append(FaceBase(rect, face))
+
+    return faces
+
+
+@cocoindex.op.function(cache=True, behavior_version=1, gpu=True)
+def extract_face_embedding(
+    face: bytes,
+) -> cocoindex.Vector[cocoindex.Float32]:
+    """Extract the embedding of a face."""
+    img = Image.open(io.BytesIO(face)).convert("RGB")
+    embedding = face_recognition.face_encodings(
+        np.array(img),
+        known_face_locations=[(0, img.width - 1, img.height - 1, 0)],
+    )[0]
+    return embedding
+
+
+@cocoindex.flow_def(name="FaceRecognition")
+def face_recognition_flow(
+    flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope
+) -> None:
+    """
+    Define an example flow that embeds files into a vector database.
+    """
+    data_scope["images"] = flow_builder.add_source(
+        cocoindex.sources.LocalFile(path="images", binary=True),
+        refresh_interval=datetime.timedelta(seconds=10),
+    )
+
+    face_embeddings = data_scope.add_collector()
+
+    with data_scope["images"].row() as image:
+        # Extract faces
+        image["faces"] = image["content"].transform(extract_faces)
+
+        with image["faces"].row() as face:
+            face["embedding"] = face["image"].transform(extract_face_embedding)
+
+            # Collect embeddings
+            face_embeddings.collect(
+                filename=image["filename"],
+                rect=face["rect"],
+                embedding=face["embedding"],
+            )
+
+    face_embeddings.export(
+        "face_embeddings",
+        cocoindex.targets.Postgres(),
+        primary_key_fields=["filename", "rect"],
+    )

cocoindex-0.1.72/examples/face_recognition/pyproject.toml

@@ -0,0 +1,14 @@
+[project]
+name = "cocoindex-face-recognition-example"
+version = "0.1.0"
+description = "Build index for papers with both metadata and content embeddings"
+requires-python = ">=3.11"
+dependencies = [
+    "cocoindex>=0.1.71",
+    "face-recognition>=1.3.0",
+    "pillow>=10.0.0",
+    "numpy>=1.26.0",
+]
+
+[tool.setuptools]
+packages = []

cocoindex-0.1.72/examples/live_updates/.env

@@ -0,0 +1 @@
+COCOINDEX_DATABASE_URL=postgres://cocoindex:cocoindex@localhost/cocoindex

cocoindex-0.1.72/examples/live_updates/README.md

@@ -0,0 +1,58 @@
+# Applying Live Updates to CocoIndex Flow Example
+[![GitHub](https://img.shields.io/github/stars/cocoindex-io/cocoindex?color=5B5BD6)](https://github.com/cocoindex-io/cocoindex)
+
+We appreciate a star ⭐ at [CocoIndex Github](https://github.com/cocoindex-io/cocoindex) if this is helpful.
+
+This example demonstrates how to use CocoIndex's live update feature to keep an index synchronized with a local directory.
+
+## How it Works
+
+The `main.py` script defines a CocoIndex flow that:
+
+1.  **Sources** data from a local directory named `data`. It uses a `refresh_interval` of 5 seconds to check for changes.
+2.  **Collects** the `filename` and `content` of each file.
+3.  **Exports** the collected data to a Postgres database table.
+
+The script then starts a `FlowLiveUpdater`, which runs in the background and continuously monitors the `data` directory for changes.
+
+## Running the Example
+
+1.  [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one.
+
+2. **Install the dependencies:**
+
+    ```bash
+    pip install -e .
+    ```
+
+3.  **Run the example:**
+
+    You can run the live update example in two ways:
+
+    **Option 1: Using the Python script**
+
+    This method uses CocoIndex [Library API](https://cocoindex.io/docs/core/flow_methods#library-api-2) to perform live updates.
+
+    ```bash
+    python main.py
+    ```
+
+    **Option 2: Using the CocoIndex CLI**
+
+    This method is useful for managing your indexes from the command line, through CocoIndex [CLI](https://cocoindex.io/docs/core/flow_methods#cli-2).
+
+    ```bash
+    cocoindex update main.py -L --setup
+    ```
+
+4.  **Test the live updates:**
+
+    While the script is running, you can try adding, modifying, or deleting files in the `data` directory. You will see the changes reflected in the logs as CocoIndex updates the index.
+
+## Cleaning Up
+
+To remove the database table created by this example, you can run:
+
+```bash
+cocoindex drop main.py
+```