cocoindex 0.1.74__tar.gz → 0.1.75__tar.gz

{cocoindex-0.1.74 → cocoindex-0.1.75}/.github/workflows/_doc_release.yml

@@ -11,7 +11,6 @@ jobs:
       - uses: actions/checkout@v4
       - uses: actions/setup-node@v4
         with:
-          node-version: 18
           cache: yarn
           cache-dependency-path: docs/yarn.lock
       - uses: webfactory/ssh-agent@v0.5.0

{cocoindex-0.1.74 → cocoindex-0.1.75}/.github/workflows/_test.yml

@@ -43,7 +43,7 @@ jobs:
         - name: Install Python toolchains
           run: |
             source .venv/bin/activate
-            pip install maturin pytest mypy
+            pip install maturin mypy pytest pytest-asyncio
         - name: Python build
           run: |
             source .venv/bin/activate

{cocoindex-0.1.74 → cocoindex-0.1.75}/.github/workflows/docs.yml

@@ -19,7 +19,6 @@ jobs:
       - uses: actions/checkout@v4
       - uses: actions/setup-node@v4
         with:
-          node-version: 18
           cache: yarn
           cache-dependency-path: docs/yarn.lock
       - name: Install dependencies

{cocoindex-0.1.74 → cocoindex-0.1.75}/Cargo.lock

@@ -1297,7 +1297,7 @@ dependencies = [
 
 [[package]]
 name = "cocoindex"
-version = "0.1.74"
+version = "0.1.75"
 dependencies = [
  "anyhow",
  "async-openai",

{cocoindex-0.1.74 → cocoindex-0.1.75}/Cargo.toml

@@ -2,7 +2,7 @@
 name = "cocoindex"
 # Version used for local development is always higher than others to take precedence.
 # Will be overridden for specific release versions.
-version = "0.1.74"
+version = "0.1.75"
 edition = "2024"
 rust-version = "1.88"
 

{cocoindex-0.1.74 → cocoindex-0.1.75}/PKG-INFO

@@ -1,12 +1,13 @@
 Metadata-Version: 2.4
 Name: cocoindex
-Version: 0.1.74
+Version: 0.1.75
 Requires-Dist: click>=8.1.8
 Requires-Dist: rich>=14.0.0
 Requires-Dist: python-dotenv>=1.1.0
 Requires-Dist: watchfiles>=1.1.0
 Requires-Dist: numpy>=1.23.2
 Requires-Dist: pytest ; extra == 'dev'
+Requires-Dist: pytest-asyncio ; extra == 'dev'
 Requires-Dist: ruff ; extra == 'dev'
 Requires-Dist: mypy ; extra == 'dev'
 Requires-Dist: pre-commit ; extra == 'dev'
@@ -18,7 +19,7 @@ Provides-Extra: all
 License-File: LICENSE
 Summary: With CocoIndex, users declare the transformation, CocoIndex creates & maintains an index, and keeps the derived index up to date based on source update, with minimal computation and changes.
 Author-email: CocoIndex <cocoindex.io@gmail.com>
-License: Apache-2.0
+License-Expression: Apache-2.0
 Requires-Python: >=3.11
 Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
 Project-URL: Homepage, https://cocoindex.io/
@@ -210,6 +211,7 @@ It defines an index flow like this:
 | [Image Search with Vision API](examples/image_search) | Generates detailed captions for images using a vision model, embeds them, enables live-updating semantic search via FastAPI and served on a React frontend|
 | [Face Recognition](examples/face_recognition) | Recognize faces in images and build embedding index |
 | [Paper Metadata](examples/paper_metadata) | Index papers in PDF files, and build metadata tables for each paper |
+| [Custom Output Files](examples/custom_output_files) | Convert markdown files to HTML files and save them to a local directory, using *CocoIndex Custom Targets* |
 
 More coming and stay tuned 👀!
 

{cocoindex-0.1.74 → cocoindex-0.1.75}/README.md

@@ -185,6 +185,7 @@ It defines an index flow like this:
 | [Image Search with Vision API](examples/image_search) | Generates detailed captions for images using a vision model, embeds them, enables live-updating semantic search via FastAPI and served on a React frontend|
 | [Face Recognition](examples/face_recognition) | Recognize faces in images and build embedding index |
 | [Paper Metadata](examples/paper_metadata) | Index papers in PDF files, and build metadata tables for each paper |
+| [Custom Output Files](examples/custom_output_files) | Convert markdown files to HTML files and save them to a local directory, using *CocoIndex Custom Targets* |
 
 More coming and stay tuned 👀!
 

{cocoindex-0.1.74 → cocoindex-0.1.75}/docs/docs/core/flow_def.mdx

@@ -33,23 +33,23 @@ It takes two arguments:
 *   `flow_builder`: a `FlowBuilder` object to help build the flow.
 *   `data_scope`: a `DataScope` object, representing the top-level data scope. Any data created by the flow should be added to it.
 
-Alternatively, for more flexibility (e.g. you want to do this conditionally or generate dynamic name), you can explicitly call the `cocoindex.add_flow_def()` method:
+Alternatively, for more flexibility (e.g. you want to do this conditionally or generate dynamic name), you can explicitly call the `cocoindex.open_flow()` method:
 
 ```python
 def demo_flow_def(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope):
     ...
 
 # Add the flow definition to the flow registry.
-demo_flow = cocoindex.add_flow_def("DemoFlow", demo_flow_def)
+demo_flow = cocoindex.open_flow("DemoFlow", demo_flow_def)
 ```
 
 In both cases, `demo_flow` will be an object with `cocoindex.Flow` class type.
 See [Flow Running](/docs/core/flow_methods) for more details on it.
 
-Sometimes you no longer want to keep states of the flow in memory. We provide a `cocoindex.remove_flow()` method for this purpose:
+Sometimes you no longer want to keep states of the flow in memory. We provide a `close()` method for this purpose:
 
 ```python
-cocoindex.remove_flow(demo_flow)
+demo_flow.close()
 ```
 
 After it's called, `demo_flow` becomes an invalid object, and you should not call any methods of it.

{cocoindex-0.1.74 → cocoindex-0.1.75}/docs/docs/core/flow_methods.mdx

@@ -39,7 +39,7 @@ It creates a `demo_flow` object in `cocoindex.Flow` type.
 For a flow, its persistent backends need to be ready before it can run, including:
 
 *   [Internal storage](/docs/core/basics#internal-storage) for CocoIndex.
-*   Backend entities for targets exported by the flow, e.g. a table (in relational databases), a collection (in some vector databases), etc.
+*   Backend resources for targets exported by the flow, e.g. a table (in relational databases), a collection (in some vector databases), etc.
 
 The desired state of the backends for a flow is derived based on the flow definition itself.
 CocoIndex supports two types of actions to manage the persistent backends automatically:
@@ -104,7 +104,7 @@ cocoindex.drop_all_flows(report_to_stdout=True)
 
 After dropping the flow, the in-memory `cocoindex.Flow` instance is still valid, and you can call setup methods on it again.
 
-If you want to remove the flow from the current process, you can call `cocoindex.remove_flow(demo_flow)` to do so (see [related doc](/docs/core/flow_def#entry-point)).
+If you want to remove the flow from the current process, you can call `demo_flow.close()` to do so (see [related doc](/docs/core/flow_def#entry-point)).
 
 :::
 

{cocoindex-0.1.74 → cocoindex-0.1.75}/docs/docs/custom_ops/custom_targets.mdx

@@ -114,13 +114,6 @@ This method should be implemented to:
 - Update configuration when a target spec changes
 - Clean up resources when a target is removed (`current` is `None`)
 
-:::note Best practice: Keep all actions idempotent
-
-Ideally this method should be idempotent, i.e. when calling this with the same arguments multiple times, the effect should remain the same.
-For example, if the target is a directory, it should be a no-op if we try to create it (`previous` is `None`) when the directory already exists, and also a no-op if we try to delete it (`current` is `None`) when the directory does not exist.
-
-:::
-
 #### `describe(key) -> str` (Optional)
 
 Returns a human-readable description of the target for logging and debugging purposes.
@@ -165,6 +158,18 @@ def prepare(spec: CustomTarget) -> PreparedCustomTarget:
 
 If not provided, the original spec will be passed directly to `mutate`.
 
+## Best Practices
+
+### Idempotency of Methods with Side Effects
+
+`apply_setup_change()` and `mutate()` are the two methods that are expected to produce side effects.
+We expect them to be idempotent, i.e. when calling them with the same arguments multiple times, the effect should remain the same.
+
+For example,
+- For `apply_setup_change()`, if the target is a directory, it should be a no-op if we try to create it (`previous` is `None`) when the directory already exists, and also a no-op if we try to delete it (`current` is `None`) when the directory does not exist.
+- For `mutate()`, if a mutation is a deletion, it should be a no-op if the row does not exist.
+
+This is to make sure when the system if left in an intermediate state, e.g. interrupted in the middle between a change is made and CocoIndex notes down the change is completed, the targets can still be gracefully rolled forward to the desired states after the system is resumed.
 
 ## Examples