cocoindex 0.2.2__tar.gz → 0.2.4__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cocoindex-0.2.4/.github/workflows/_test.yml +65 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/release.yml +29 -11
- {cocoindex-0.2.2 → cocoindex-0.2.4}/Cargo.lock +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/Cargo.toml +7 -2
- {cocoindex-0.2.2 → cocoindex-0.2.4}/PKG-INFO +17 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/cli.mdx +8 -2
- cocoindex-0.2.4/docs/docs/examples/examples/postgres_source.md +299 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/simple_vector_index.md +43 -21
- cocoindex-0.2.4/docs/docs/getting_started/quickstart.md +177 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ops/sources.md +21 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docusaurus.config.ts +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/components/GitHubButton/index.tsx +2 -2
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/collector.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/cover.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/description.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/embed.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/flow.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/lineage.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/price.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/postgres_source/source.png +0 -0
- cocoindex-0.2.4/docs/static/img/examples/simple_vector_index/embed.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/requirements.txt +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/pyproject.toml +2 -2
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pyproject.toml +1 -1
- cocoindex-0.2.4/examples/postgres_source/main.py +164 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/pyproject.toml +7 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/pyproject.toml +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/pyproject.toml +29 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/cli.py +22 -4
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/convert.py +37 -26
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/op.py +4 -19
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/sources.py +13 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/subprocess_exec.py +27 -3
- cocoindex-0.2.4/python/cocoindex/tests/conftest.py +38 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/user_app_loader.py +12 -10
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/value.rs +151 -194
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/analyzer.rs +9 -17
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/plan.rs +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/dumper.rs +6 -5
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/evaluator.rs +7 -7
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/indexing_status.rs +1 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/live_updater.rs +13 -9
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/row_indexer.rs +13 -14
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/source_indexer.rs +48 -46
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/factory_bases.rs +5 -2
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/split_recursively.rs +2 -2
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/interface.rs +37 -28
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/py_factory.rs +2 -2
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/shared/postgres.rs +9 -24
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/amazon_s3.rs +17 -9
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/azure_blob.rs +16 -8
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/google_drive.rs +18 -10
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/local_file.rs +16 -8
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/postgres.rs +353 -100
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/kuzu.rs +9 -12
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/neo4j.rs +2 -5
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/postgres.rs +4 -11
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/qdrant.rs +6 -5
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/shared/property_graph.rs +3 -1
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/py/convert.rs +32 -5
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/service/flows.rs +6 -5
- cocoindex-0.2.2/.github/workflows/_test.yml +0 -58
- cocoindex-0.2.2/docs/docs/getting_started/quickstart.md +0 -283
- cocoindex-0.2.2/docs/static/img/examples/simple_vector_index/embed.png +0 -0
- cocoindex-0.2.2/examples/postgres_source/main.py +0 -82
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.cargo/config.toml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.env.lib_debug +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/ISSUE_TEMPLATE//360/237/220/233-bug-report.md" +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/ISSUE_TEMPLATE//360/237/222/241-feature-request.md" +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/scripts/update_version.sh +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/CI.yml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/_doc_release.yml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/docs.yml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/format.yml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/.pre-commit-config.yaml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/CODE_OF_CONDUCT.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/CONTRIBUTING.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/LICENSE +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/dev/neo4j.yaml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/dev/postgres.yaml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/about/community.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ai/llm.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/contributing/guide.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/contributing/new_built_in_target.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/contributing/setup_dev_environment.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/basics.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/data_example.svg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/data_types.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/flow_def.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/flow_example.svg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/flow_methods.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/settings.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/custom_ops/custom_functions.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/custom_ops/custom_targets.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/academic_papers_index.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/codebase_index.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/custom_targets.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/docs_to_knowledge_graph.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/image_search.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/manual_extraction.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/multi_format_index.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/patient_form_extraction.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/photo_search.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/product_recommendation.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/index.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/getting_started/installation.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/getting_started/markdown_files.zip +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/getting_started/overview.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ops/functions.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ops/targets.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/query.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/tutorials/live_updates.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/tutorials/manage_flow_dynamically.mdx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/package.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/sidebars.ts +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/css/custom.css +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCard/index.tsx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCard/styles.module.css +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCardList/index.tsx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCardList/styles.module.css +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/Root.js +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/.nojekyll +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/docusaurus.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/abstract_chunks.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/basic_info.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/chunk_embedding.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/first_page.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/metadata.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/codebase_index/chunk.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/codebase_index/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/codebase_index/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/custom_targets/convert.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/custom_targets/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/dedupe.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/export_document.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/export_relationship.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/extract_relationship.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/relationship.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/summary.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/embedding.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/multi_modal_architecture.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/result.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/extraction.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/summary.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/colpali_architecture.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/embed.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/pages.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/extraction.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/fields.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/tomarkdown.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/photo_search/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/photo_search/extraction.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/photo_search/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/dedupe.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/export_all.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/export_product.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/export_taxonomy.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/extract_product.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/extract_taxonomy.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/neo4j.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/parse_json.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/taxonomy.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/simple_vector_index/chunk.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/simple_vector_index/cover.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/simple_vector_index/flow.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/favicon.ico +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/icon.svg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/incremental-etl.gif +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/robots.txt +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/tsconfig.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/yarn.lock +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/.env.example +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/.env.example +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/data/bizarre_animals.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/data/chunk_norris.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/Carter_welcomes_Reagan.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/Solvay_conference_1927.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/Steve_Jobs_and_Bill_Gates_(522695099).jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/einplanck3.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/.dockerignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/compose.yaml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/dockerfile +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/files/1810.04805v2.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/.env.example +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/colpali_main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/index.html +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/package-lock.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/package.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/src/App.jsx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/src/main.jsx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/src/style.css +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/vite.config.js +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/cat1.jpeg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/dog1.jpeg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/elephant1.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/giraffe.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/data/bizarre_animals.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/data/chunk_norris.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/array.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/base64.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/copy.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/glob.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/1706.03762v7.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/1810.04805v2.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/2502.06786v3.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/healthcare_industry_test_p101.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/healthcare_industry_test_p86.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/healthcare_industry_test_p9.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/restaurant_brands_international_2023.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/sweetgreen_2023.jpg +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/.env.example +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/.gitignore +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/1706.03762v7.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/1810.04805v2.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/2502.06786v3.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/2502.20346v1.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/.env.example +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_Form_David_Artificial.docx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_Form_Emily_Artificial.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_Form_Joe_Artificial.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_From_Jane_Artificial.docx +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pdf_files/1706.03762v7.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pdf_files/1810.04805v2.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pdf_files/rfc8259.pdf +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/prepare_source_data.sql +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/.env.example +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/img/cocoinsight.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/img/neo4j.png +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p1.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p2.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p3.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p4.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p5.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p6.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p7.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p8.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p9.json +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/Text_Embedding.ipynb +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/markdown_files/1706.03762v7.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/markdown_files/1810.04805v2.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/markdown_files/rfc8259.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/.env +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/README.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/main.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/markdown_files/rfc8259.md +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/__init__.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/auth_registry.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/flow.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/functions.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/index.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/lib.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/llm.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/py.typed +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/runtime.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/setting.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/setup.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/targets.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/__init__.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_convert.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_optional_database.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_transform_flow.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_typing.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_validation.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/typing.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/utils.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/validation.py +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/ruff.toml +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/duration.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/field_attrs.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/json_schema.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/schema.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/spec.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/analyzed_flow.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/exec_ctx.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/flow_builder.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/db_tracking.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/db_tracking_setup.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/memoization.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/stats.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/lib.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/lib_context.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/anthropic.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/gemini.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/litellm.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/ollama.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/openai.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/openrouter.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/vllm.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/voyage.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/embed_text.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/extract_by_llm.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/parse_json.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/test_utils.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/registration.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/registry.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sdk.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/shared/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/shared/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/shared/pattern_matcher.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/shared/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/shared/table_columns.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/prelude.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/py/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/server.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/service/error.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/service/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/settings.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/auth_registry.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/components.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/db_metadata.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/driver.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/flow_features.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/states.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/concur_control.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/db.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/fingerprint.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/immutable.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/mod.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/retryable.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/str_sanitize.rs +0 -0
- {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/yaml_ser.rs +0 -0
@@ -0,0 +1,65 @@
|
|
1
|
+
name: Run Tests
|
2
|
+
|
3
|
+
on:
|
4
|
+
workflow_call:
|
5
|
+
|
6
|
+
env:
|
7
|
+
CARGO_TERM_COLOR: always
|
8
|
+
|
9
|
+
jobs:
|
10
|
+
build-test:
|
11
|
+
strategy:
|
12
|
+
matrix:
|
13
|
+
python-version: [3.11]
|
14
|
+
platform:
|
15
|
+
- { runner: ubuntu-latest, python_exec: ".venv/bin/python" }
|
16
|
+
- { runner: ubuntu-24.04-arm, python_exec: ".venv/bin/python" }
|
17
|
+
- { runner: macos-latest, python_exec: ".venv/bin/python" }
|
18
|
+
- { runner: macos-13, python_exec: ".venv/bin/python" }
|
19
|
+
- { runner: windows-latest, python_exec: ".venv\\Scripts\\python" }
|
20
|
+
runs-on: ${{ matrix.platform.runner }}
|
21
|
+
steps:
|
22
|
+
- uses: actions/checkout@v4
|
23
|
+
|
24
|
+
- uses: actions/setup-python@v5
|
25
|
+
id: setup_python
|
26
|
+
with:
|
27
|
+
python-version: ${{ matrix.python-version }}
|
28
|
+
cache: 'pip'
|
29
|
+
|
30
|
+
- run: rustup toolchain install stable --profile minimal
|
31
|
+
- name: Rust Cache
|
32
|
+
uses: Swatinem/rust-cache@v2
|
33
|
+
with:
|
34
|
+
key: rust-${{ matrix.platform.runner }}-${{ matrix.python-version }}
|
35
|
+
- name: Rust tests
|
36
|
+
run: cargo test --verbose
|
37
|
+
|
38
|
+
- uses: actions/cache@v4
|
39
|
+
with:
|
40
|
+
path: .venv
|
41
|
+
key: pyenv-${{ matrix.platform.runner }}-${{ steps.setup_python.outputs.python-version }}-${{ hashFiles('pyproject.toml') }}
|
42
|
+
restore-keys: |
|
43
|
+
pyenv-${{ matrix.platform.runner }}-${{ steps.setup_python.outputs.python-version }}-
|
44
|
+
|
45
|
+
- name: Setup venv
|
46
|
+
run: |
|
47
|
+
python -m venv .venv
|
48
|
+
- name: Install Python toolchains
|
49
|
+
run: |
|
50
|
+
${{ matrix.platform.python_exec }} -m pip install maturin mypy pytest pytest-asyncio
|
51
|
+
- name: Python build
|
52
|
+
run: |
|
53
|
+
${{ matrix.platform.python_exec }} -m maturin develop -E all
|
54
|
+
- name: Python type check (mypy)
|
55
|
+
run: |
|
56
|
+
${{ matrix.platform.python_exec }} -m mypy python
|
57
|
+
- name: Python tests
|
58
|
+
if: ${{ !startsWith(matrix.platform.runner, 'windows') }}
|
59
|
+
run: |
|
60
|
+
${{ matrix.platform.python_exec }} -m pytest --capture=no python/cocoindex/tests
|
61
|
+
- name: Python tests (Windows cmd)
|
62
|
+
if: ${{ startsWith(matrix.platform.runner, 'windows') }}
|
63
|
+
shell: cmd # Use `cmd` to run test for Windows, as PowerShell doesn't detect exit code by `os._exit(0)` correctly.
|
64
|
+
run: |
|
65
|
+
${{ matrix.platform.python_exec }} -m pytest --capture=no python/cocoindex/tests
|
@@ -1,8 +1,7 @@
|
|
1
|
-
# This
|
2
|
-
# To update, run
|
3
|
-
#
|
4
|
-
# maturin generate-ci github
|
1
|
+
# This workflow can be triggered on tags push (automatic release) or manually on any branch.
|
5
2
|
#
|
3
|
+
# - When triggered on tags push, it will build and publishes a new version including docs.
|
4
|
+
# - When triggered manually, it's a dry-run: only build, without publishing anything.
|
6
5
|
name: release
|
7
6
|
|
8
7
|
on:
|
@@ -31,11 +30,11 @@ jobs:
|
|
31
30
|
strategy:
|
32
31
|
matrix:
|
33
32
|
platform:
|
34
|
-
- { os: linux, runner: ubuntu-
|
35
|
-
- { os: linux, runner: ubuntu-24.04, target: aarch64, container: "ghcr.io/rust-cross/manylinux_2_28-cross:aarch64" }
|
36
|
-
- { os:
|
33
|
+
- { os: linux, runner: ubuntu-latest, target: x86_64, container: "ghcr.io/rust-cross/manylinux_2_28-cross:x86_64" }
|
34
|
+
- { os: linux, runner: ubuntu-24.04-arm, target: aarch64, container: "ghcr.io/rust-cross/manylinux_2_28-cross:aarch64" }
|
35
|
+
- { os: macos, runner: macos-latest, target: aarch64 }
|
37
36
|
- { os: macos, runner: macos-13, target: x86_64 }
|
38
|
-
- { os:
|
37
|
+
- { os: windows, runner: windows-latest, target: x64 }
|
39
38
|
steps:
|
40
39
|
- uses: actions/checkout@v4
|
41
40
|
- uses: actions/download-artifact@v4
|
@@ -43,12 +42,12 @@ jobs:
|
|
43
42
|
name: Cargo.toml
|
44
43
|
- uses: actions/setup-python@v5
|
45
44
|
with:
|
46
|
-
python-version: 3.
|
45
|
+
python-version: 3.13
|
47
46
|
- name: Build wheels
|
48
47
|
uses: PyO3/maturin-action@v1
|
49
48
|
with:
|
50
49
|
target: ${{ matrix.platform.target }}
|
51
|
-
args: --release --out dist
|
50
|
+
args: --release --out dist
|
52
51
|
sccache: 'true'
|
53
52
|
manylinux: auto
|
54
53
|
container: ${{ matrix.platform.container }}
|
@@ -58,6 +57,24 @@ jobs:
|
|
58
57
|
name: wheels-${{ matrix.platform.os }}-${{ matrix.platform.target }}
|
59
58
|
path: dist
|
60
59
|
|
60
|
+
test-abi3:
|
61
|
+
runs-on: ubuntu-24.04
|
62
|
+
needs: build
|
63
|
+
strategy:
|
64
|
+
matrix:
|
65
|
+
py: ["3.11", "3.12", "3.13"]
|
66
|
+
steps:
|
67
|
+
- uses: actions/download-artifact@v4
|
68
|
+
with:
|
69
|
+
name: wheels-linux-x86_64
|
70
|
+
- uses: actions/setup-python@v5
|
71
|
+
with:
|
72
|
+
python-version: ${{ matrix.py }}
|
73
|
+
- run: python -V
|
74
|
+
- run: pip install --find-links=./ cocoindex
|
75
|
+
- run: python -c "import cocoindex, sys; print('import ok on', sys.version)"
|
76
|
+
|
77
|
+
|
61
78
|
sdist:
|
62
79
|
runs-on: ubuntu-latest
|
63
80
|
needs: [create-versioned-toml]
|
@@ -80,7 +97,7 @@ jobs:
|
|
80
97
|
release:
|
81
98
|
name: Release
|
82
99
|
runs-on: ubuntu-latest
|
83
|
-
needs: [create-versioned-toml, build, sdist]
|
100
|
+
needs: [create-versioned-toml, build, test-abi3, sdist]
|
84
101
|
permissions:
|
85
102
|
# Use to sign the release artifacts
|
86
103
|
id-token: write
|
@@ -111,5 +128,6 @@ jobs:
|
|
111
128
|
release-docs:
|
112
129
|
name: Release Docs
|
113
130
|
needs: [release]
|
131
|
+
if: ${{ startsWith(github.ref, 'refs/tags/') }}
|
114
132
|
uses: ./.github/workflows/_doc_release.yml
|
115
133
|
secrets: inherit
|
@@ -2,7 +2,7 @@
|
|
2
2
|
name = "cocoindex"
|
3
3
|
# Version used for local development is always higher than others to take precedence.
|
4
4
|
# Will be overridden for specific release versions.
|
5
|
-
version = "0.2.
|
5
|
+
version = "0.2.4"
|
6
6
|
edition = "2024"
|
7
7
|
rust-version = "1.88"
|
8
8
|
readme = "README.md"
|
@@ -16,7 +16,12 @@ name = "cocoindex_engine"
|
|
16
16
|
crate-type = ["cdylib"]
|
17
17
|
|
18
18
|
[dependencies]
|
19
|
-
pyo3 = { version = "0.25.1", features = [
|
19
|
+
pyo3 = { version = "0.25.1", features = [
|
20
|
+
"abi3-py311",
|
21
|
+
"auto-initialize",
|
22
|
+
"chrono",
|
23
|
+
"uuid",
|
24
|
+
] }
|
20
25
|
pythonize = "0.25.0"
|
21
26
|
pyo3-async-runtimes = { version = "0.25.0", features = ["tokio-runtime"] }
|
22
27
|
|
@@ -1,6 +1,21 @@
|
|
1
1
|
Metadata-Version: 2.4
|
2
2
|
Name: cocoindex
|
3
|
-
Version: 0.2.
|
3
|
+
Version: 0.2.4
|
4
|
+
Classifier: Development Status :: 3 - Alpha
|
5
|
+
Classifier: License :: OSI Approved :: Apache Software License
|
6
|
+
Classifier: Operating System :: OS Independent
|
7
|
+
Classifier: Programming Language :: Rust
|
8
|
+
Classifier: Programming Language :: Python :: 3
|
9
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
10
|
+
Classifier: Programming Language :: Python :: 3.11
|
11
|
+
Classifier: Programming Language :: Python :: 3.12
|
12
|
+
Classifier: Programming Language :: Python :: 3.13
|
13
|
+
Classifier: Programming Language :: Python :: 3.14
|
14
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
15
|
+
Classifier: Topic :: Text Processing :: Indexing
|
16
|
+
Classifier: Intended Audience :: Developers
|
17
|
+
Classifier: Natural Language :: English
|
18
|
+
Classifier: Typing :: Typed
|
4
19
|
Requires-Dist: click>=8.1.8
|
5
20
|
Requires-Dist: rich>=14.0.0
|
6
21
|
Requires-Dist: python-dotenv>=1.1.0
|
@@ -21,6 +36,7 @@ Provides-Extra: colpali
|
|
21
36
|
Provides-Extra: all
|
22
37
|
License-File: LICENSE
|
23
38
|
Summary: With CocoIndex, users declare the transformation, CocoIndex creates & maintains an index, and keeps the derived index up to date based on source update, with minimal computation and changes.
|
39
|
+
Keywords: indexing,real-time,incremental,pipeline,search,ai,etl,rag,dataflow,context-engineering
|
24
40
|
Author-email: CocoIndex <cocoindex.io@gmail.com>
|
25
41
|
License-Expression: Apache-2.0
|
26
42
|
Requires-Python: >=3.11
|
@@ -17,8 +17,13 @@ Once CocoIndex is installed, you can invoke the CLI directly using the `cocoinde
|
|
17
17
|
### APP_TARGET Format
|
18
18
|
|
19
19
|
The `APP_TARGET` can be:
|
20
|
-
1.
|
21
|
-
|
20
|
+
1. An **Python module name** that contains your flow definitions (e.g., `main`, `my_package.flows`).
|
21
|
+
You can also use `--app-dir <path>` to specify the base directory to load the module from.
|
22
|
+
|
23
|
+
2. A **path to a Python file** defining your flows (e.g., `main.py`, `path/to/my_flows.py`).
|
24
|
+
|
25
|
+
The file will be loaded as a top-level Python module, e.g. relative imports will not work as its parent package is not defined (similar to how `python main.py` works).
|
26
|
+
|
22
27
|
3. For commands that operate on a *specific flow* (like `show`, `update`, `evaluate`), you can combine the application reference with a flow name:
|
23
28
|
* `path/to/my_flows.py:MyFlow`
|
24
29
|
* `my_package.flows:MyFlow`
|
@@ -44,6 +49,7 @@ If no file is found, only existing system environment variables are used.
|
|
44
49
|
CocoIndex CLI supports the following global options:
|
45
50
|
|
46
51
|
* `--env-file <path>`: Load environment variables from a specified `.env` file. If not provided, `.env` in the current directory is loaded if it exists.
|
52
|
+
* `--app-dir <path>`: Load apps from the specified directory. It will be treated as part of `PYTHONPATH`. Default to the current directory.
|
47
53
|
* `--version`: Show the CocoIndex version and exit.
|
48
54
|
* `--help`: Show the main help message and exit.
|
49
55
|
|
@@ -0,0 +1,299 @@
|
|
1
|
+
---
|
2
|
+
title: Transform Data From Structured Source in PostgreSQL
|
3
|
+
description: Transform data from PostgreSQL table as source, transform with both AI models and non-AI data mappings, and write them into PostgreSQL/PgVector for semantic + structured search.
|
4
|
+
sidebar_class_name: hidden
|
5
|
+
slug: /examples/postgres_source
|
6
|
+
canonicalUrl: '/examples/postgres_source'
|
7
|
+
sidebar_custom_props:
|
8
|
+
image: /img/examples/postgres_source/cover.png
|
9
|
+
tags: [data-mapping, vector-index, postgres]
|
10
|
+
tags: [data-mapping, vector-index, postgres]
|
11
|
+
---
|
12
|
+
import { GitHubButton, YouTubeButton, DocumentationButton } from '../../../src/components/GitHubButton';
|
13
|
+
|
14
|
+
<GitHubButton url="https://github.com/cocoindex-io/cocoindex/tree/main/examples/postgres_source" margin="0 0 24px 0" /
|
15
|
+
>
|
16
|
+

|
17
|
+
|
18
|
+
[CocoIndex](https://github.com/cocoindex-io/cocoindex) is one framework for building **incremental** data flows across **structured and unstructured** sources. This tutorial shows how to take data from PostgreSQL table as source, transform with both AI and non-AI data mappings, and write them into a new PostgreSQL table with PgVector for semantic + structured search.
|
19
|
+
|
20
|
+
## PostgreSQL Product Indexing Flow
|
21
|
+

|
22
|
+
|
23
|
+
- Reading data from a PostgreSQL table `source_products`.
|
24
|
+
- Computing additional fields (`total_value`, `full_description`).
|
25
|
+
- Generating embeddings for semantic search.
|
26
|
+
- Storing the results in another PostgreSQL table with a vector index using pgvector
|
27
|
+
|
28
|
+
|
29
|
+
### Connect to source
|
30
|
+
|
31
|
+
`flow_builder.add_source` reads rows from `source_products`.
|
32
|
+
|
33
|
+
```python
|
34
|
+
@cocoindex.flow_def(name="PostgresProductIndexing")
|
35
|
+
def postgres_product_indexing_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope) -> None:
|
36
|
+
|
37
|
+
data_scope["products"] = flow_builder.add_source(
|
38
|
+
cocoindex.sources.Postgres(
|
39
|
+
table_name="source_products",
|
40
|
+
# Optional. Use the default CocoIndex database if not specified.
|
41
|
+
database=cocoindex.add_transient_auth_entry(
|
42
|
+
cocoindex.DatabaseConnectionSpec(
|
43
|
+
url=os.environ["SOURCE_DATABASE_URL"],
|
44
|
+
)
|
45
|
+
),
|
46
|
+
# Optional.
|
47
|
+
ordinal_column="modified_time",
|
48
|
+
notification=cocoindex.sources.PostgresNotification(),
|
49
|
+
),
|
50
|
+
)
|
51
|
+
```
|
52
|
+
This step adds source data from PostgreSQL table `source_products` to the flow as a `KTable`.
|
53
|
+
|
54
|
+

|
55
|
+
|
56
|
+
CocoIndex incrementally sync data from Postgres. When new or updated rows are found, only those rows run through the pipeline, so downstream indexes and search results reflect the latest data while unchanged rows are untouched. The following two arguments (both are optional) make this more efficient:
|
57
|
+
|
58
|
+
- `notification` enables change capture based on Postgres LISTEN/NOTIFY. Each change triggers an incremental processing on the specific row immediately.
|
59
|
+
- Regardless if `notification` is provided or not, CocoIndex still needs to scan the full table to detect changes in some scenarios (e.g. between two `update` invocation), and the `ordinal_column` provides a field that CocoIndex can use to quickly detect which row has changed without reading value columns.
|
60
|
+
|
61
|
+
Check [Postgres source](https://cocoindex.io/docs/ops/sources#postgres) for more details.
|
62
|
+
|
63
|
+
If you use the Postgres database hosted by Supabase, please click Connect on your project dashboard and find the URL there. Check [DatabaseConnectionSpec](https://cocoindex.io/docs/core/settings#databaseconnectionspec)
|
64
|
+
for more details.
|
65
|
+
|
66
|
+
## Simple Data Mapping / Transformation
|
67
|
+
|
68
|
+
Create a simple transformation to calculate the total price.
|
69
|
+
|
70
|
+
```python
|
71
|
+
@cocoindex.op.function()
|
72
|
+
def calculate_total_value(price: float, amount: int) -> float:
|
73
|
+
"""Compute total value for each product."""
|
74
|
+
return price * amount
|
75
|
+
```
|
76
|
+
|
77
|
+
Plug into the flow:
|
78
|
+
|
79
|
+
```python
|
80
|
+
with data_scope["products"].row() as product:
|
81
|
+
# Compute total value
|
82
|
+
product["total_value"] = flow_builder.transform(
|
83
|
+
calculate_total_value,
|
84
|
+
product["price"],
|
85
|
+
product["amount"],
|
86
|
+
)
|
87
|
+
```
|
88
|
+
|
89
|
+

|
90
|
+
|
91
|
+
### Data Transformation & AI Transformation
|
92
|
+
|
93
|
+
Create a custom function creates a `full_description` field by combining the product’s category, name, and description.
|
94
|
+
|
95
|
+
```python
|
96
|
+
@cocoindex.op.function()
|
97
|
+
def make_full_description(category: str, name: str, description: str) -> str:
|
98
|
+
"""Create a detailed product description for embedding."
|
99
|
+
return f"Category: {category}\nName: {name}\n\n{description}"
|
100
|
+
|
101
|
+
```
|
102
|
+
|
103
|
+
Embeddings often perform better with more context. By combining fields into a single text string, we ensure that the semantic meaning of the product is captured fully.
|
104
|
+
|
105
|
+
Now plug into the flow:
|
106
|
+
|
107
|
+
```python
|
108
|
+
with data_scope["products"].row() as product:
|
109
|
+
#.. other transformations
|
110
|
+
|
111
|
+
# Compute full description
|
112
|
+
product["full_description"] = flow_builder.transform(
|
113
|
+
make_full_description,
|
114
|
+
product["product_category"],
|
115
|
+
product["product_name"],
|
116
|
+
product["description"],
|
117
|
+
)
|
118
|
+
|
119
|
+
# Generate embeddings
|
120
|
+
product["embedding"] = product["full_description"].transform(
|
121
|
+
cocoindex.functions.SentenceTransformerEmbed(
|
122
|
+
model="sentence-transformers/all-MiniLM-L6-v2"
|
123
|
+
)
|
124
|
+
)
|
125
|
+
|
126
|
+
# Collect data
|
127
|
+
indexed_product.collect(
|
128
|
+
product_category=product["product_category"],
|
129
|
+
product_name=product["product_name"],
|
130
|
+
description=product["description"],
|
131
|
+
price=product["price"],
|
132
|
+
amount=product["amount"],
|
133
|
+
total_value=product["total_value"],
|
134
|
+
embedding=product["embedding"],
|
135
|
+
)
|
136
|
+
```
|
137
|
+
|
138
|
+
This takes each product row, and does the following:
|
139
|
+
|
140
|
+
1. builds a rich description.
|
141
|
+
|
142
|
+

|
143
|
+
|
144
|
+
2. turns it into an embedding
|
145
|
+
|
146
|
+

|
147
|
+
|
148
|
+
3. collects the embedding along with structured fields (category, name, price, etc.).
|
149
|
+
|
150
|
+

|
151
|
+
|
152
|
+
|
153
|
+
## Export
|
154
|
+
|
155
|
+
```python
|
156
|
+
indexed_product.export(
|
157
|
+
"output",
|
158
|
+
cocoindex.targets.Postgres(),
|
159
|
+
primary_key_fields=["product_category", "product_name"],
|
160
|
+
vector_indexes=[
|
161
|
+
cocoindex.VectorIndexDef(
|
162
|
+
field_name="embedding",
|
163
|
+
metric=cocoindex.VectorSimilarityMetric.COSINE_SIMILARITY,
|
164
|
+
)
|
165
|
+
],
|
166
|
+
)
|
167
|
+
```
|
168
|
+
|
169
|
+
All transformed rows are collected and exported to a new PostgreSQL table with a vector index, ready for semantic search.
|
170
|
+
|
171
|
+
|
172
|
+
## Field lineage
|
173
|
+
When the transform flow starts to getting complex, it's hard to understand how each field is derived.
|
174
|
+
CocoIndex provides a way to visualize the lineage of each field, to make it easier to trace and troubleshoot field origins and downstream dependencies.
|
175
|
+
|
176
|
+
For example, the following image shows the lineage of the `embedding` field, you can click from the final output backward all the way to the source fields, step by step.
|
177
|
+
|
178
|
+

|
179
|
+
|
180
|
+
|
181
|
+
## Running the Pipeline
|
182
|
+
|
183
|
+
1. Set up dependencies:
|
184
|
+
|
185
|
+
```bash
|
186
|
+
pip install -e .
|
187
|
+
```
|
188
|
+
|
189
|
+
2. Create the source table with sample data:
|
190
|
+
|
191
|
+
```bash
|
192
|
+
psql "postgres://cocoindex:cocoindex@localhost/cocoindex" -f ./prepare_source_data.sql
|
193
|
+
```
|
194
|
+
|
195
|
+
3. Setup tables and update the index:
|
196
|
+
|
197
|
+
```bash
|
198
|
+
cocoindex update --setup main.py
|
199
|
+
```
|
200
|
+
|
201
|
+
4. Run CocoInsight:
|
202
|
+
|
203
|
+
```bash
|
204
|
+
cocoindex server -ci main.py
|
205
|
+
```
|
206
|
+
You can walk through the project step by step in CocoInsight to see exactly how each field is constructed and what happens behind the scenes. It connects to your local CocoIndex server, with zero pipeline data retention.
|
207
|
+
|
208
|
+
|
209
|
+
## Continuous Updating
|
210
|
+
|
211
|
+
For continuous updating when the source changes, add `-L`:
|
212
|
+
|
213
|
+
```bash
|
214
|
+
cocoindex server -ci -L main.py
|
215
|
+
```
|
216
|
+
Check [live updates](https://cocoindex.io/docs/tutorials/live_updates) for more details.
|
217
|
+
|
218
|
+
## Search and Query the Index
|
219
|
+
|
220
|
+
### Query
|
221
|
+
|
222
|
+
Runs a semantic similarity search over the indexed products table, returning the top matches for a given query.
|
223
|
+
|
224
|
+
```python
|
225
|
+
def search(pool: ConnectionPool, query: str, top_k: int = 5) -> list[dict[str, Any]]:
|
226
|
+
# Get the table name, for the export target in the text_embedding_flow above.
|
227
|
+
table_name = cocoindex.utils.get_target_default_name(
|
228
|
+
postgres_product_indexing_flow, "output"
|
229
|
+
)
|
230
|
+
# Evaluate the transform flow defined above with the input query, to get the embedding.
|
231
|
+
query_vector = text_to_embedding.eval(query)
|
232
|
+
# Run the query and get the results.
|
233
|
+
with pool.connection() as conn:
|
234
|
+
register_vector(conn)
|
235
|
+
with conn.cursor(row_factory=dict_row) as cur:
|
236
|
+
cur.execute(
|
237
|
+
f"""
|
238
|
+
SELECT
|
239
|
+
product_category,
|
240
|
+
product_name,
|
241
|
+
description,
|
242
|
+
amount,
|
243
|
+
total_value,
|
244
|
+
(embedding <=> %s) AS distance
|
245
|
+
FROM {table_name}
|
246
|
+
ORDER BY distance ASC
|
247
|
+
LIMIT %s
|
248
|
+
""",
|
249
|
+
(query_vector, top_k),
|
250
|
+
)
|
251
|
+
return cur.fetchall()
|
252
|
+
```
|
253
|
+
This function
|
254
|
+
|
255
|
+
- Converts the query text into an embedding (`query_vector`).
|
256
|
+
- Compares it with each product’s stored embedding (`embedding`) using vector distance.
|
257
|
+
- Returns the closest matches, including both metadata and the similarity score (`distance`).
|
258
|
+
|
259
|
+
### Create an command-line interactive loop
|
260
|
+
|
261
|
+
```python
|
262
|
+
def _main() -> None:
|
263
|
+
# Initialize the database connection pool.
|
264
|
+
pool = ConnectionPool(os.environ["COCOINDEX_DATABASE_URL"])
|
265
|
+
# Run queries in a loop to demonstrate the query capabilities.
|
266
|
+
while True:
|
267
|
+
query = input("Enter search query (or Enter to quit): ")
|
268
|
+
if query == "":
|
269
|
+
break
|
270
|
+
# Run the query function with the database connection pool and the query.
|
271
|
+
results = search(pool, query)
|
272
|
+
print("\nSearch results:")
|
273
|
+
for result in results:
|
274
|
+
score = 1.0 - result["distance"]
|
275
|
+
print(
|
276
|
+
f"[{score:.3f}] {result['product_category']} | {result['product_name']} | {result['amount']} | {result['total_value']}"
|
277
|
+
)
|
278
|
+
print(f" {result['description']}")
|
279
|
+
print("---")
|
280
|
+
print()
|
281
|
+
|
282
|
+
if __name__ == "__main__":
|
283
|
+
load_dotenv()
|
284
|
+
cocoindex.init()
|
285
|
+
_main()
|
286
|
+
```
|
287
|
+
|
288
|
+
### Run as a Service
|
289
|
+
|
290
|
+
This [example](https://cocoindex.io/docs/examples/image_search#fast-api-application) runs as a service using Fast API.
|
291
|
+
|
292
|
+
|
293
|
+
## Why One Framework for Structured + Unstructured?
|
294
|
+
|
295
|
+
- One mental model: Treat files, APIs, and databases uniformly; AI steps are ordinary ops.
|
296
|
+
- Incremental by default: Use an ordinal column to sync only changes; no fragile glue jobs.
|
297
|
+
- Consistency: Embeddings are always derived from the exact transformed row state.
|
298
|
+
- Operational simplicity: One deployment, one lineage view, fewer moving parts.
|
299
|
+
|
@@ -71,34 +71,21 @@ with data_scope["documents"].row() as doc:
|
|
71
71
|
### Embed each chunk
|
72
72
|
|
73
73
|
```python
|
74
|
-
|
75
|
-
|
76
|
-
"""
|
77
|
-
Embed the text using a SentenceTransformer model.
|
78
|
-
This is a shared logic between indexing and querying, so extract it as a function.
|
79
|
-
"""
|
80
|
-
return text.transform(
|
74
|
+
with doc["chunks"].row() as chunk:
|
75
|
+
chunk["embedding"] = chunk["text"].transform(
|
81
76
|
cocoindex.functions.SentenceTransformerEmbed(
|
82
|
-
model="sentence-transformers/all-MiniLM-L6-v2"
|
77
|
+
model="sentence-transformers/all-MiniLM-L6-v2"
|
78
|
+
)
|
79
|
+
)
|
80
|
+
doc_embeddings.collect(filename=doc["filename"], location=chunk["location"],
|
81
|
+
text=chunk["text"], embedding=chunk["embedding"])
|
83
82
|
```
|
84
|
-

|
85
|
-
|
86
|
-
This code defines a transformation function that converts text into vector embeddings using the SentenceTransformer model.
|
87
|
-
`@cocoindex.transform_flow()` is needed to share the transformation across indexing and query.
|
88
|
-
This decorator marks this as a reusable transformation flow that can be called on specific input data from user code using `eval()`, as shown in the search function below.
|
89
83
|
|
90
84
|
The `MiniLM-L6-v2` model is a good balance of speed and quality for text embeddings, though you can swap in other SentenceTransformer models as needed.
|
91
85
|
|
92
86
|
<DocumentationButton url="https://cocoindex.io/docs/ops/functions#sentencetransformerembed" text="SentenceTransformerEmbed" margin="0 0 16px 0" />
|
93
|
-
|
94
|
-
Plug in the `text_to_embedding` function and collect the embeddings.
|
95
87
|
|
96
|
-
|
97
|
-
with doc["chunks"].row() as chunk:
|
98
|
-
chunk["embedding"] = text_to_embedding(chunk["text"])
|
99
|
-
doc_embeddings.collect(filename=doc["filename"], location=chunk["location"],
|
100
|
-
text=chunk["text"], embedding=chunk["embedding"])
|
101
|
-
```
|
88
|
+

|
102
89
|
|
103
90
|
## Export the embeddings
|
104
91
|
|
@@ -119,10 +106,32 @@ CocoIndex supports other vector databases as well, with 1-line switch.
|
|
119
106
|
|
120
107
|
## Query the index
|
121
108
|
|
109
|
+
### Define a shared flow for both indexing and querying
|
110
|
+
|
111
|
+
```python
|
112
|
+
@cocoindex.transform_flow()
|
113
|
+
def text_to_embedding(text: cocoindex.DataSlice[str]) -> cocoindex.DataSlice[list[float]]:
|
114
|
+
"""
|
115
|
+
Embed the text using a SentenceTransformer model.
|
116
|
+
This is a shared logic between indexing and querying, so extract it as a function.
|
117
|
+
"""
|
118
|
+
return text.transform(
|
119
|
+
cocoindex.functions.SentenceTransformerEmbed(
|
120
|
+
model="sentence-transformers/all-MiniLM-L6-v2"))
|
121
|
+
```
|
122
|
+
|
123
|
+
This code defines a transformation function that converts text into vector embeddings using the SentenceTransformer model.
|
124
|
+
`@cocoindex.transform_flow()` is needed to share the transformation across indexing and query.
|
125
|
+
|
126
|
+
This decorator marks this as a reusable transformation flow that can be called on specific input data from user code using `eval()`, as shown in the search function below.
|
127
|
+
|
128
|
+
### Write query
|
129
|
+
|
122
130
|
CocoIndex doesn't provide additional query interface at the moment. We can write SQL or rely on the query engine by the target storage, if any.
|
123
131
|
|
124
132
|
<DocumentationButton url="https://cocoindex.io/docs/ops/targets#postgres" text="Postgres" margin="0 0 16px 0" />
|
125
133
|
|
134
|
+
|
126
135
|
```python
|
127
136
|
def search(pool: ConnectionPool, query: str, top_k: int = 5):
|
128
137
|
table_name = cocoindex.utils.get_target_storage_default_name(text_embedding_flow, "doc_embeddings")
|
@@ -166,6 +175,19 @@ if __name__ == "__main__":
|
|
166
175
|
_main()
|
167
176
|
```
|
168
177
|
|
178
|
+
In the function above, most parts are standard query logic - you can use any libraries you like.
|
179
|
+
There're two CocoIndex-specific logic:
|
180
|
+
|
181
|
+
1. Get the table name from the export target in the `text_embedding_flow` above.
|
182
|
+
Since the table name for the `Postgres` target is not explicitly specified in the `export()` call,
|
183
|
+
CocoIndex uses a default name.
|
184
|
+
`cocoindex.utils.get_target_default_name()` is a utility function to get the default table name for this case.
|
185
|
+
|
186
|
+
2. Evaluate the transform flow defined above with the input query, to get the embedding.
|
187
|
+
It's done by the `eval()` method of the transform flow `text_to_embedding`.
|
188
|
+
The return type of this method is `NDArray[np.float32]` as declared in the `text_to_embedding()` function (`cocoindex.DataSlice[NDArray[np.float32]]`).
|
189
|
+
|
190
|
+
|
169
191
|
|
170
192
|
## Time to have fun!
|
171
193
|
- Run the following command to setup and update the index.
|