cocoindex 0.2.2__tar.gz → 0.2.4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (414) hide show
  1. cocoindex-0.2.4/.github/workflows/_test.yml +65 -0
  2. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/release.yml +29 -11
  3. {cocoindex-0.2.2 → cocoindex-0.2.4}/Cargo.lock +1 -1
  4. {cocoindex-0.2.2 → cocoindex-0.2.4}/Cargo.toml +7 -2
  5. {cocoindex-0.2.2 → cocoindex-0.2.4}/PKG-INFO +17 -1
  6. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/cli.mdx +8 -2
  7. cocoindex-0.2.4/docs/docs/examples/examples/postgres_source.md +299 -0
  8. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/simple_vector_index.md +43 -21
  9. cocoindex-0.2.4/docs/docs/getting_started/quickstart.md +177 -0
  10. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ops/sources.md +21 -0
  11. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docusaurus.config.ts +1 -1
  12. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/components/GitHubButton/index.tsx +2 -2
  13. cocoindex-0.2.4/docs/static/img/examples/postgres_source/collector.png +0 -0
  14. cocoindex-0.2.4/docs/static/img/examples/postgres_source/cover.png +0 -0
  15. cocoindex-0.2.4/docs/static/img/examples/postgres_source/description.png +0 -0
  16. cocoindex-0.2.4/docs/static/img/examples/postgres_source/embed.png +0 -0
  17. cocoindex-0.2.4/docs/static/img/examples/postgres_source/flow.png +0 -0
  18. cocoindex-0.2.4/docs/static/img/examples/postgres_source/lineage.png +0 -0
  19. cocoindex-0.2.4/docs/static/img/examples/postgres_source/price.png +0 -0
  20. cocoindex-0.2.4/docs/static/img/examples/postgres_source/source.png +0 -0
  21. cocoindex-0.2.4/docs/static/img/examples/simple_vector_index/embed.png +0 -0
  22. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/pyproject.toml +1 -1
  23. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/pyproject.toml +1 -1
  24. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/pyproject.toml +1 -1
  25. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/pyproject.toml +1 -1
  26. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/pyproject.toml +1 -1
  27. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/pyproject.toml +1 -1
  28. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/requirements.txt +1 -1
  29. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/pyproject.toml +1 -1
  30. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/pyproject.toml +2 -2
  31. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/pyproject.toml +1 -1
  32. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/pyproject.toml +1 -1
  33. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/pyproject.toml +1 -1
  34. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/pyproject.toml +1 -1
  35. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/pyproject.toml +1 -1
  36. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pyproject.toml +1 -1
  37. cocoindex-0.2.4/examples/postgres_source/main.py +164 -0
  38. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/pyproject.toml +7 -1
  39. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/pyproject.toml +1 -1
  40. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/pyproject.toml +1 -1
  41. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/pyproject.toml +1 -1
  42. {cocoindex-0.2.2 → cocoindex-0.2.4}/pyproject.toml +29 -0
  43. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/cli.py +22 -4
  44. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/convert.py +37 -26
  45. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/op.py +4 -19
  46. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/sources.py +13 -0
  47. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/subprocess_exec.py +27 -3
  48. cocoindex-0.2.4/python/cocoindex/tests/conftest.py +38 -0
  49. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/user_app_loader.py +12 -10
  50. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/value.rs +151 -194
  51. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/analyzer.rs +9 -17
  52. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/plan.rs +1 -1
  53. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/dumper.rs +6 -5
  54. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/evaluator.rs +7 -7
  55. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/indexing_status.rs +1 -1
  56. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/live_updater.rs +13 -9
  57. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/row_indexer.rs +13 -14
  58. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/source_indexer.rs +48 -46
  59. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/factory_bases.rs +5 -2
  60. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/split_recursively.rs +2 -2
  61. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/interface.rs +37 -28
  62. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/py_factory.rs +2 -2
  63. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/shared/postgres.rs +9 -24
  64. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/amazon_s3.rs +17 -9
  65. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/azure_blob.rs +16 -8
  66. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/google_drive.rs +18 -10
  67. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/local_file.rs +16 -8
  68. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/postgres.rs +353 -100
  69. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/kuzu.rs +9 -12
  70. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/neo4j.rs +2 -5
  71. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/postgres.rs +4 -11
  72. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/qdrant.rs +6 -5
  73. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/shared/property_graph.rs +3 -1
  74. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/py/convert.rs +32 -5
  75. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/service/flows.rs +6 -5
  76. cocoindex-0.2.2/.github/workflows/_test.yml +0 -58
  77. cocoindex-0.2.2/docs/docs/getting_started/quickstart.md +0 -283
  78. cocoindex-0.2.2/docs/static/img/examples/simple_vector_index/embed.png +0 -0
  79. cocoindex-0.2.2/examples/postgres_source/main.py +0 -82
  80. {cocoindex-0.2.2 → cocoindex-0.2.4}/.cargo/config.toml +0 -0
  81. {cocoindex-0.2.2 → cocoindex-0.2.4}/.env.lib_debug +0 -0
  82. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/ISSUE_TEMPLATE//360/237/220/233-bug-report.md" +0 -0
  83. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/ISSUE_TEMPLATE//360/237/222/241-feature-request.md" +0 -0
  84. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/scripts/update_version.sh +0 -0
  85. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/CI.yml +0 -0
  86. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/_doc_release.yml +0 -0
  87. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/docs.yml +0 -0
  88. {cocoindex-0.2.2 → cocoindex-0.2.4}/.github/workflows/format.yml +0 -0
  89. {cocoindex-0.2.2 → cocoindex-0.2.4}/.gitignore +0 -0
  90. {cocoindex-0.2.2 → cocoindex-0.2.4}/.pre-commit-config.yaml +0 -0
  91. {cocoindex-0.2.2 → cocoindex-0.2.4}/CODE_OF_CONDUCT.md +0 -0
  92. {cocoindex-0.2.2 → cocoindex-0.2.4}/CONTRIBUTING.md +0 -0
  93. {cocoindex-0.2.2 → cocoindex-0.2.4}/LICENSE +0 -0
  94. {cocoindex-0.2.2 → cocoindex-0.2.4}/README.md +0 -0
  95. {cocoindex-0.2.2 → cocoindex-0.2.4}/dev/neo4j.yaml +0 -0
  96. {cocoindex-0.2.2 → cocoindex-0.2.4}/dev/postgres.yaml +0 -0
  97. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/.gitignore +0 -0
  98. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/README.md +0 -0
  99. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/about/community.md +0 -0
  100. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ai/llm.mdx +0 -0
  101. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/contributing/guide.md +0 -0
  102. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/contributing/new_built_in_target.mdx +0 -0
  103. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/contributing/setup_dev_environment.md +0 -0
  104. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/basics.md +0 -0
  105. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/data_example.svg +0 -0
  106. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/data_types.mdx +0 -0
  107. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/flow_def.mdx +0 -0
  108. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/flow_example.svg +0 -0
  109. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/flow_methods.mdx +0 -0
  110. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/core/settings.mdx +0 -0
  111. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/custom_ops/custom_functions.mdx +0 -0
  112. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/custom_ops/custom_targets.mdx +0 -0
  113. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/academic_papers_index.md +0 -0
  114. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/codebase_index.md +0 -0
  115. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/custom_targets.md +0 -0
  116. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/docs_to_knowledge_graph.md +0 -0
  117. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/image_search.md +0 -0
  118. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/manual_extraction.md +0 -0
  119. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/multi_format_index.md +0 -0
  120. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/patient_form_extraction.md +0 -0
  121. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/photo_search.md +0 -0
  122. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/examples/product_recommendation.md +0 -0
  123. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/examples/index.md +0 -0
  124. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/getting_started/installation.md +0 -0
  125. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/getting_started/markdown_files.zip +0 -0
  126. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/getting_started/overview.md +0 -0
  127. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ops/functions.md +0 -0
  128. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/ops/targets.md +0 -0
  129. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/query.mdx +0 -0
  130. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/tutorials/live_updates.md +0 -0
  131. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/docs/tutorials/manage_flow_dynamically.mdx +0 -0
  132. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/package.json +0 -0
  133. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/sidebars.ts +0 -0
  134. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/css/custom.css +0 -0
  135. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCard/index.tsx +0 -0
  136. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCard/styles.module.css +0 -0
  137. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCardList/index.tsx +0 -0
  138. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/DocCardList/styles.module.css +0 -0
  139. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/src/theme/Root.js +0 -0
  140. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/.nojekyll +0 -0
  141. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/docusaurus.png +0 -0
  142. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/abstract_chunks.png +0 -0
  143. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/basic_info.png +0 -0
  144. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/chunk_embedding.png +0 -0
  145. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/cover.png +0 -0
  146. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/first_page.png +0 -0
  147. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/flow.png +0 -0
  148. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/academic_papers_index/metadata.png +0 -0
  149. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/codebase_index/chunk.png +0 -0
  150. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/codebase_index/cover.png +0 -0
  151. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/codebase_index/flow.png +0 -0
  152. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/custom_targets/convert.png +0 -0
  153. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/custom_targets/cover.png +0 -0
  154. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/cover.png +0 -0
  155. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/dedupe.png +0 -0
  156. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/export_document.png +0 -0
  157. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/export_relationship.png +0 -0
  158. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/extract_relationship.png +0 -0
  159. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/flow.png +0 -0
  160. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/relationship.png +0 -0
  161. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/docs_to_knowledge_graph/summary.png +0 -0
  162. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/cover.png +0 -0
  163. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/embedding.png +0 -0
  164. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/flow.png +0 -0
  165. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/multi_modal_architecture.png +0 -0
  166. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/image_search/result.png +0 -0
  167. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/cover.png +0 -0
  168. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/extraction.png +0 -0
  169. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/flow.png +0 -0
  170. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/manual_extraction/summary.png +0 -0
  171. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/colpali_architecture.png +0 -0
  172. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/cover.png +0 -0
  173. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/embed.png +0 -0
  174. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/flow.png +0 -0
  175. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/multi_format_index/pages.png +0 -0
  176. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/cover.png +0 -0
  177. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/extraction.png +0 -0
  178. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/fields.png +0 -0
  179. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/flow.png +0 -0
  180. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/patient_form_extraction/tomarkdown.png +0 -0
  181. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/photo_search/cover.png +0 -0
  182. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/photo_search/extraction.png +0 -0
  183. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/photo_search/flow.png +0 -0
  184. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/cover.png +0 -0
  185. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/dedupe.png +0 -0
  186. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/export_all.png +0 -0
  187. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/export_product.png +0 -0
  188. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/export_taxonomy.png +0 -0
  189. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/extract_product.png +0 -0
  190. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/extract_taxonomy.png +0 -0
  191. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/neo4j.png +0 -0
  192. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/parse_json.png +0 -0
  193. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/product_recommendation/taxonomy.png +0 -0
  194. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/simple_vector_index/chunk.png +0 -0
  195. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/simple_vector_index/cover.png +0 -0
  196. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/examples/simple_vector_index/flow.png +0 -0
  197. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/favicon.ico +0 -0
  198. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/icon.svg +0 -0
  199. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/img/incremental-etl.gif +0 -0
  200. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/static/robots.txt +0 -0
  201. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/tsconfig.json +0 -0
  202. {cocoindex-0.2.2 → cocoindex-0.2.4}/docs/yarn.lock +0 -0
  203. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/.env.example +0 -0
  204. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/.gitignore +0 -0
  205. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/README.md +0 -0
  206. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/amazon_s3_embedding/main.py +0 -0
  207. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/.env.example +0 -0
  208. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/.gitignore +0 -0
  209. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/README.md +0 -0
  210. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/azure_blob_embedding/main.py +0 -0
  211. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/.env +0 -0
  212. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/README.md +0 -0
  213. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/code_embedding/main.py +0 -0
  214. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/.env +0 -0
  215. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/.gitignore +0 -0
  216. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/README.md +0 -0
  217. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/data/bizarre_animals.md +0 -0
  218. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/data/chunk_norris.md +0 -0
  219. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/custom_output_files/main.py +0 -0
  220. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/.env +0 -0
  221. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/README.md +0 -0
  222. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/docs_to_knowledge_graph/main.py +0 -0
  223. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/.env +0 -0
  224. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/README.md +0 -0
  225. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/Carter_welcomes_Reagan.jpg +0 -0
  226. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/Solvay_conference_1927.jpg +0 -0
  227. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/Steve_Jobs_and_Bill_Gates_(522695099).jpg +0 -0
  228. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/images/einplanck3.jpg +0 -0
  229. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/face_recognition/main.py +0 -0
  230. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/.dockerignore +0 -0
  231. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/.env +0 -0
  232. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/README.md +0 -0
  233. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/compose.yaml +0 -0
  234. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/dockerfile +0 -0
  235. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/files/1810.04805v2.md +0 -0
  236. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/fastapi_server_docker/main.py +0 -0
  237. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/.env.example +0 -0
  238. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/.gitignore +0 -0
  239. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/README.md +0 -0
  240. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/gdrive_text_embedding/main.py +0 -0
  241. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/.env +0 -0
  242. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/README.md +0 -0
  243. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/colpali_main.py +0 -0
  244. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/.gitignore +0 -0
  245. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/index.html +0 -0
  246. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/package-lock.json +0 -0
  247. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/package.json +0 -0
  248. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/src/App.jsx +0 -0
  249. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/src/main.jsx +0 -0
  250. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/src/style.css +0 -0
  251. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/frontend/vite.config.js +0 -0
  252. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/cat1.jpeg +0 -0
  253. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/dog1.jpeg +0 -0
  254. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/elephant1.jpg +0 -0
  255. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/img/giraffe.jpg +0 -0
  256. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/image_search/main.py +0 -0
  257. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/.env +0 -0
  258. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/README.md +0 -0
  259. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/data/bizarre_animals.md +0 -0
  260. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/data/chunk_norris.md +0 -0
  261. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/live_updates/main.py +0 -0
  262. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/.env +0 -0
  263. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/README.md +0 -0
  264. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/main.py +0 -0
  265. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/array.pdf +0 -0
  266. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/base64.pdf +0 -0
  267. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/copy.pdf +0 -0
  268. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/manuals_llm_extraction/manuals/glob.pdf +0 -0
  269. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/.env +0 -0
  270. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/README.md +0 -0
  271. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/main.py +0 -0
  272. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/1706.03762v7.pdf +0 -0
  273. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/1810.04805v2.pdf +0 -0
  274. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/2502.06786v3.pdf +0 -0
  275. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/healthcare_industry_test_p101.jpg +0 -0
  276. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/healthcare_industry_test_p86.jpg +0 -0
  277. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/healthcare_industry_test_p9.jpg +0 -0
  278. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/restaurant_brands_international_2023.jpg +0 -0
  279. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/multi_format_indexing/source_files/sweetgreen_2023.jpg +0 -0
  280. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/.env.example +0 -0
  281. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/.gitignore +0 -0
  282. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/README.md +0 -0
  283. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/main.py +0 -0
  284. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/1706.03762v7.pdf +0 -0
  285. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/1810.04805v2.pdf +0 -0
  286. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/2502.06786v3.pdf +0 -0
  287. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/paper_metadata/papers/2502.20346v1.pdf +0 -0
  288. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/.env.example +0 -0
  289. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/README.md +0 -0
  290. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/README.md +0 -0
  291. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_Form_David_Artificial.docx +0 -0
  292. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_Form_Emily_Artificial.pdf +0 -0
  293. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_Form_Joe_Artificial.pdf +0 -0
  294. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/data/patient_forms/Patient_Intake_From_Jane_Artificial.docx +0 -0
  295. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/patient_intake_extraction/main.py +0 -0
  296. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/.env +0 -0
  297. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/README.md +0 -0
  298. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/main.py +0 -0
  299. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pdf_files/1706.03762v7.pdf +0 -0
  300. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pdf_files/1810.04805v2.pdf +0 -0
  301. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/pdf_embedding/pdf_files/rfc8259.pdf +0 -0
  302. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/.env +0 -0
  303. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/README.md +0 -0
  304. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/postgres_source/prepare_source_data.sql +0 -0
  305. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/.env.example +0 -0
  306. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/README.md +0 -0
  307. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/img/cocoinsight.png +0 -0
  308. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/img/neo4j.png +0 -0
  309. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/main.py +0 -0
  310. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p1.json +0 -0
  311. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p2.json +0 -0
  312. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p3.json +0 -0
  313. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p4.json +0 -0
  314. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p5.json +0 -0
  315. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p6.json +0 -0
  316. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p7.json +0 -0
  317. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p8.json +0 -0
  318. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/product_recommendation/products/p9.json +0 -0
  319. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/.env +0 -0
  320. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/README.md +0 -0
  321. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/Text_Embedding.ipynb +0 -0
  322. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/main.py +0 -0
  323. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/markdown_files/1706.03762v7.md +0 -0
  324. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/markdown_files/1810.04805v2.md +0 -0
  325. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding/markdown_files/rfc8259.md +0 -0
  326. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/.env +0 -0
  327. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/README.md +0 -0
  328. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/main.py +0 -0
  329. {cocoindex-0.2.2 → cocoindex-0.2.4}/examples/text_embedding_qdrant/markdown_files/rfc8259.md +0 -0
  330. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/__init__.py +0 -0
  331. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/auth_registry.py +0 -0
  332. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/flow.py +0 -0
  333. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/functions.py +0 -0
  334. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/index.py +0 -0
  335. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/lib.py +0 -0
  336. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/llm.py +0 -0
  337. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/py.typed +0 -0
  338. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/runtime.py +0 -0
  339. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/setting.py +0 -0
  340. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/setup.py +0 -0
  341. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/targets.py +0 -0
  342. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/__init__.py +0 -0
  343. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_convert.py +0 -0
  344. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_optional_database.py +0 -0
  345. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_transform_flow.py +0 -0
  346. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_typing.py +0 -0
  347. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/tests/test_validation.py +0 -0
  348. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/typing.py +0 -0
  349. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/utils.py +0 -0
  350. {cocoindex-0.2.2 → cocoindex-0.2.4}/python/cocoindex/validation.py +0 -0
  351. {cocoindex-0.2.2 → cocoindex-0.2.4}/ruff.toml +0 -0
  352. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/duration.rs +0 -0
  353. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/field_attrs.rs +0 -0
  354. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/json_schema.rs +0 -0
  355. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/mod.rs +0 -0
  356. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/schema.rs +0 -0
  357. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/base/spec.rs +0 -0
  358. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/analyzed_flow.rs +0 -0
  359. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/exec_ctx.rs +0 -0
  360. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/flow_builder.rs +0 -0
  361. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/builder/mod.rs +0 -0
  362. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/db_tracking.rs +0 -0
  363. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/db_tracking_setup.rs +0 -0
  364. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/memoization.rs +0 -0
  365. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/mod.rs +0 -0
  366. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/execution/stats.rs +0 -0
  367. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/lib.rs +0 -0
  368. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/lib_context.rs +0 -0
  369. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/anthropic.rs +0 -0
  370. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/gemini.rs +0 -0
  371. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/litellm.rs +0 -0
  372. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/mod.rs +0 -0
  373. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/ollama.rs +0 -0
  374. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/openai.rs +0 -0
  375. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/openrouter.rs +0 -0
  376. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/vllm.rs +0 -0
  377. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/llm/voyage.rs +0 -0
  378. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/embed_text.rs +0 -0
  379. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/extract_by_llm.rs +0 -0
  380. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/mod.rs +0 -0
  381. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/parse_json.rs +0 -0
  382. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/functions/test_utils.rs +0 -0
  383. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/mod.rs +0 -0
  384. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/registration.rs +0 -0
  385. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/registry.rs +0 -0
  386. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sdk.rs +0 -0
  387. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/shared/mod.rs +0 -0
  388. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/mod.rs +0 -0
  389. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/shared/mod.rs +0 -0
  390. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/sources/shared/pattern_matcher.rs +0 -0
  391. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/mod.rs +0 -0
  392. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/shared/mod.rs +0 -0
  393. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/ops/targets/shared/table_columns.rs +0 -0
  394. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/prelude.rs +0 -0
  395. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/py/mod.rs +0 -0
  396. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/server.rs +0 -0
  397. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/service/error.rs +0 -0
  398. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/service/mod.rs +0 -0
  399. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/settings.rs +0 -0
  400. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/auth_registry.rs +0 -0
  401. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/components.rs +0 -0
  402. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/db_metadata.rs +0 -0
  403. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/driver.rs +0 -0
  404. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/flow_features.rs +0 -0
  405. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/mod.rs +0 -0
  406. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/setup/states.rs +0 -0
  407. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/concur_control.rs +0 -0
  408. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/db.rs +0 -0
  409. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/fingerprint.rs +0 -0
  410. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/immutable.rs +0 -0
  411. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/mod.rs +0 -0
  412. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/retryable.rs +0 -0
  413. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/str_sanitize.rs +0 -0
  414. {cocoindex-0.2.2 → cocoindex-0.2.4}/src/utils/yaml_ser.rs +0 -0
@@ -0,0 +1,65 @@
1
+ name: Run Tests
2
+
3
+ on:
4
+ workflow_call:
5
+
6
+ env:
7
+ CARGO_TERM_COLOR: always
8
+
9
+ jobs:
10
+ build-test:
11
+ strategy:
12
+ matrix:
13
+ python-version: [3.11]
14
+ platform:
15
+ - { runner: ubuntu-latest, python_exec: ".venv/bin/python" }
16
+ - { runner: ubuntu-24.04-arm, python_exec: ".venv/bin/python" }
17
+ - { runner: macos-latest, python_exec: ".venv/bin/python" }
18
+ - { runner: macos-13, python_exec: ".venv/bin/python" }
19
+ - { runner: windows-latest, python_exec: ".venv\\Scripts\\python" }
20
+ runs-on: ${{ matrix.platform.runner }}
21
+ steps:
22
+ - uses: actions/checkout@v4
23
+
24
+ - uses: actions/setup-python@v5
25
+ id: setup_python
26
+ with:
27
+ python-version: ${{ matrix.python-version }}
28
+ cache: 'pip'
29
+
30
+ - run: rustup toolchain install stable --profile minimal
31
+ - name: Rust Cache
32
+ uses: Swatinem/rust-cache@v2
33
+ with:
34
+ key: rust-${{ matrix.platform.runner }}-${{ matrix.python-version }}
35
+ - name: Rust tests
36
+ run: cargo test --verbose
37
+
38
+ - uses: actions/cache@v4
39
+ with:
40
+ path: .venv
41
+ key: pyenv-${{ matrix.platform.runner }}-${{ steps.setup_python.outputs.python-version }}-${{ hashFiles('pyproject.toml') }}
42
+ restore-keys: |
43
+ pyenv-${{ matrix.platform.runner }}-${{ steps.setup_python.outputs.python-version }}-
44
+
45
+ - name: Setup venv
46
+ run: |
47
+ python -m venv .venv
48
+ - name: Install Python toolchains
49
+ run: |
50
+ ${{ matrix.platform.python_exec }} -m pip install maturin mypy pytest pytest-asyncio
51
+ - name: Python build
52
+ run: |
53
+ ${{ matrix.platform.python_exec }} -m maturin develop -E all
54
+ - name: Python type check (mypy)
55
+ run: |
56
+ ${{ matrix.platform.python_exec }} -m mypy python
57
+ - name: Python tests
58
+ if: ${{ !startsWith(matrix.platform.runner, 'windows') }}
59
+ run: |
60
+ ${{ matrix.platform.python_exec }} -m pytest --capture=no python/cocoindex/tests
61
+ - name: Python tests (Windows cmd)
62
+ if: ${{ startsWith(matrix.platform.runner, 'windows') }}
63
+ shell: cmd # Use `cmd` to run test for Windows, as PowerShell doesn't detect exit code by `os._exit(0)` correctly.
64
+ run: |
65
+ ${{ matrix.platform.python_exec }} -m pytest --capture=no python/cocoindex/tests
@@ -1,8 +1,7 @@
1
- # This file is autogenerated by maturin v1.8.1
2
- # To update, run
3
- #
4
- # maturin generate-ci github
1
+ # This workflow can be triggered on tags push (automatic release) or manually on any branch.
5
2
  #
3
+ # - When triggered on tags push, it will build and publishes a new version including docs.
4
+ # - When triggered manually, it's a dry-run: only build, without publishing anything.
6
5
  name: release
7
6
 
8
7
  on:
@@ -31,11 +30,11 @@ jobs:
31
30
  strategy:
32
31
  matrix:
33
32
  platform:
34
- - { os: linux, runner: ubuntu-24.04, target: x86_64, container: "ghcr.io/rust-cross/manylinux_2_28-cross:x86_64" }
35
- - { os: linux, runner: ubuntu-24.04, target: aarch64, container: "ghcr.io/rust-cross/manylinux_2_28-cross:aarch64" }
36
- - { os: windows, runner: windows-latest, target: x64 }
33
+ - { os: linux, runner: ubuntu-latest, target: x86_64, container: "ghcr.io/rust-cross/manylinux_2_28-cross:x86_64" }
34
+ - { os: linux, runner: ubuntu-24.04-arm, target: aarch64, container: "ghcr.io/rust-cross/manylinux_2_28-cross:aarch64" }
35
+ - { os: macos, runner: macos-latest, target: aarch64 }
37
36
  - { os: macos, runner: macos-13, target: x86_64 }
38
- - { os: macos, runner: macos-14, target: aarch64 }
37
+ - { os: windows, runner: windows-latest, target: x64 }
39
38
  steps:
40
39
  - uses: actions/checkout@v4
41
40
  - uses: actions/download-artifact@v4
@@ -43,12 +42,12 @@ jobs:
43
42
  name: Cargo.toml
44
43
  - uses: actions/setup-python@v5
45
44
  with:
46
- python-version: 3.x
45
+ python-version: 3.13
47
46
  - name: Build wheels
48
47
  uses: PyO3/maturin-action@v1
49
48
  with:
50
49
  target: ${{ matrix.platform.target }}
51
- args: --release --out dist --find-interpreter
50
+ args: --release --out dist
52
51
  sccache: 'true'
53
52
  manylinux: auto
54
53
  container: ${{ matrix.platform.container }}
@@ -58,6 +57,24 @@ jobs:
58
57
  name: wheels-${{ matrix.platform.os }}-${{ matrix.platform.target }}
59
58
  path: dist
60
59
 
60
+ test-abi3:
61
+ runs-on: ubuntu-24.04
62
+ needs: build
63
+ strategy:
64
+ matrix:
65
+ py: ["3.11", "3.12", "3.13"]
66
+ steps:
67
+ - uses: actions/download-artifact@v4
68
+ with:
69
+ name: wheels-linux-x86_64
70
+ - uses: actions/setup-python@v5
71
+ with:
72
+ python-version: ${{ matrix.py }}
73
+ - run: python -V
74
+ - run: pip install --find-links=./ cocoindex
75
+ - run: python -c "import cocoindex, sys; print('import ok on', sys.version)"
76
+
77
+
61
78
  sdist:
62
79
  runs-on: ubuntu-latest
63
80
  needs: [create-versioned-toml]
@@ -80,7 +97,7 @@ jobs:
80
97
  release:
81
98
  name: Release
82
99
  runs-on: ubuntu-latest
83
- needs: [create-versioned-toml, build, sdist]
100
+ needs: [create-versioned-toml, build, test-abi3, sdist]
84
101
  permissions:
85
102
  # Use to sign the release artifacts
86
103
  id-token: write
@@ -111,5 +128,6 @@ jobs:
111
128
  release-docs:
112
129
  name: Release Docs
113
130
  needs: [release]
131
+ if: ${{ startsWith(github.ref, 'refs/tags/') }}
114
132
  uses: ./.github/workflows/_doc_release.yml
115
133
  secrets: inherit
@@ -1283,7 +1283,7 @@ dependencies = [
1283
1283
 
1284
1284
  [[package]]
1285
1285
  name = "cocoindex"
1286
- version = "0.2.2"
1286
+ version = "0.2.4"
1287
1287
  dependencies = [
1288
1288
  "anyhow",
1289
1289
  "async-openai",
@@ -2,7 +2,7 @@
2
2
  name = "cocoindex"
3
3
  # Version used for local development is always higher than others to take precedence.
4
4
  # Will be overridden for specific release versions.
5
- version = "0.2.2"
5
+ version = "0.2.4"
6
6
  edition = "2024"
7
7
  rust-version = "1.88"
8
8
  readme = "README.md"
@@ -16,7 +16,12 @@ name = "cocoindex_engine"
16
16
  crate-type = ["cdylib"]
17
17
 
18
18
  [dependencies]
19
- pyo3 = { version = "0.25.1", features = ["chrono", "auto-initialize", "uuid"] }
19
+ pyo3 = { version = "0.25.1", features = [
20
+ "abi3-py311",
21
+ "auto-initialize",
22
+ "chrono",
23
+ "uuid",
24
+ ] }
20
25
  pythonize = "0.25.0"
21
26
  pyo3-async-runtimes = { version = "0.25.0", features = ["tokio-runtime"] }
22
27
 
@@ -1,6 +1,21 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: cocoindex
3
- Version: 0.2.2
3
+ Version: 0.2.4
4
+ Classifier: Development Status :: 3 - Alpha
5
+ Classifier: License :: OSI Approved :: Apache Software License
6
+ Classifier: Operating System :: OS Independent
7
+ Classifier: Programming Language :: Rust
8
+ Classifier: Programming Language :: Python :: 3
9
+ Classifier: Programming Language :: Python :: 3 :: Only
10
+ Classifier: Programming Language :: Python :: 3.11
11
+ Classifier: Programming Language :: Python :: 3.12
12
+ Classifier: Programming Language :: Python :: 3.13
13
+ Classifier: Programming Language :: Python :: 3.14
14
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
15
+ Classifier: Topic :: Text Processing :: Indexing
16
+ Classifier: Intended Audience :: Developers
17
+ Classifier: Natural Language :: English
18
+ Classifier: Typing :: Typed
4
19
  Requires-Dist: click>=8.1.8
5
20
  Requires-Dist: rich>=14.0.0
6
21
  Requires-Dist: python-dotenv>=1.1.0
@@ -21,6 +36,7 @@ Provides-Extra: colpali
21
36
  Provides-Extra: all
22
37
  License-File: LICENSE
23
38
  Summary: With CocoIndex, users declare the transformation, CocoIndex creates & maintains an index, and keeps the derived index up to date based on source update, with minimal computation and changes.
39
+ Keywords: indexing,real-time,incremental,pipeline,search,ai,etl,rag,dataflow,context-engineering
24
40
  Author-email: CocoIndex <cocoindex.io@gmail.com>
25
41
  License-Expression: Apache-2.0
26
42
  Requires-Python: >=3.11
@@ -17,8 +17,13 @@ Once CocoIndex is installed, you can invoke the CLI directly using the `cocoinde
17
17
  ### APP_TARGET Format
18
18
 
19
19
  The `APP_TARGET` can be:
20
- 1. A **path to a Python file** defining your flows (e.g., `main.py`, `path/to/my_flows.py`).
21
- 2. An **installed Python module name** that contains your flow definitions (e.g., `my_package.flows`).
20
+ 1. An **Python module name** that contains your flow definitions (e.g., `main`, `my_package.flows`).
21
+ You can also use `--app-dir <path>` to specify the base directory to load the module from.
22
+
23
+ 2. A **path to a Python file** defining your flows (e.g., `main.py`, `path/to/my_flows.py`).
24
+
25
+ The file will be loaded as a top-level Python module, e.g. relative imports will not work as its parent package is not defined (similar to how `python main.py` works).
26
+
22
27
  3. For commands that operate on a *specific flow* (like `show`, `update`, `evaluate`), you can combine the application reference with a flow name:
23
28
  * `path/to/my_flows.py:MyFlow`
24
29
  * `my_package.flows:MyFlow`
@@ -44,6 +49,7 @@ If no file is found, only existing system environment variables are used.
44
49
  CocoIndex CLI supports the following global options:
45
50
 
46
51
  * `--env-file <path>`: Load environment variables from a specified `.env` file. If not provided, `.env` in the current directory is loaded if it exists.
52
+ * `--app-dir <path>`: Load apps from the specified directory. It will be treated as part of `PYTHONPATH`. Default to the current directory.
47
53
  * `--version`: Show the CocoIndex version and exit.
48
54
  * `--help`: Show the main help message and exit.
49
55
 
@@ -0,0 +1,299 @@
1
+ ---
2
+ title: Transform Data From Structured Source in PostgreSQL
3
+ description: Transform data from PostgreSQL table as source, transform with both AI models and non-AI data mappings, and write them into PostgreSQL/PgVector for semantic + structured search.
4
+ sidebar_class_name: hidden
5
+ slug: /examples/postgres_source
6
+ canonicalUrl: '/examples/postgres_source'
7
+ sidebar_custom_props:
8
+ image: /img/examples/postgres_source/cover.png
9
+ tags: [data-mapping, vector-index, postgres]
10
+ tags: [data-mapping, vector-index, postgres]
11
+ ---
12
+ import { GitHubButton, YouTubeButton, DocumentationButton } from '../../../src/components/GitHubButton';
13
+
14
+ <GitHubButton url="https://github.com/cocoindex-io/cocoindex/tree/main/examples/postgres_source" margin="0 0 24px 0" /
15
+ >
16
+ ![PostgreSQL Product Indexing Flow](/img/examples/postgres_source/cover.png)
17
+
18
+ [CocoIndex](https://github.com/cocoindex-io/cocoindex) is one framework for building **incremental** data flows across **structured and unstructured** sources. This tutorial shows how to take data from PostgreSQL table as source, transform with both AI and non-AI data mappings, and write them into a new PostgreSQL table with PgVector for semantic + structured search.
19
+
20
+ ## PostgreSQL Product Indexing Flow
21
+ ![PostgreSQL Product Indexing Flow](/img/examples/postgres_source/flow.png)
22
+
23
+ - Reading data from a PostgreSQL table `source_products`.
24
+ - Computing additional fields (`total_value`, `full_description`).
25
+ - Generating embeddings for semantic search.
26
+ - Storing the results in another PostgreSQL table with a vector index using pgvector
27
+
28
+
29
+ ### Connect to source
30
+
31
+ `flow_builder.add_source` reads rows from `source_products`.
32
+
33
+ ```python
34
+ @cocoindex.flow_def(name="PostgresProductIndexing")
35
+ def postgres_product_indexing_flow(flow_builder: cocoindex.FlowBuilder, data_scope: cocoindex.DataScope) -> None:
36
+
37
+ data_scope["products"] = flow_builder.add_source(
38
+ cocoindex.sources.Postgres(
39
+ table_name="source_products",
40
+ # Optional. Use the default CocoIndex database if not specified.
41
+ database=cocoindex.add_transient_auth_entry(
42
+ cocoindex.DatabaseConnectionSpec(
43
+ url=os.environ["SOURCE_DATABASE_URL"],
44
+ )
45
+ ),
46
+ # Optional.
47
+ ordinal_column="modified_time",
48
+ notification=cocoindex.sources.PostgresNotification(),
49
+ ),
50
+ )
51
+ ```
52
+ This step adds source data from PostgreSQL table `source_products` to the flow as a `KTable`.
53
+
54
+ ![Add PostgreSQL Source](/img/examples/postgres_source/source.png)
55
+
56
+ CocoIndex incrementally sync data from Postgres. When new or updated rows are found, only those rows run through the pipeline, so downstream indexes and search results reflect the latest data while unchanged rows are untouched. The following two arguments (both are optional) make this more efficient:
57
+
58
+ - `notification` enables change capture based on Postgres LISTEN/NOTIFY. Each change triggers an incremental processing on the specific row immediately.
59
+ - Regardless if `notification` is provided or not, CocoIndex still needs to scan the full table to detect changes in some scenarios (e.g. between two `update` invocation), and the `ordinal_column` provides a field that CocoIndex can use to quickly detect which row has changed without reading value columns.
60
+
61
+ Check [Postgres source](https://cocoindex.io/docs/ops/sources#postgres) for more details.
62
+
63
+ If you use the Postgres database hosted by Supabase, please click Connect on your project dashboard and find the URL there. Check [DatabaseConnectionSpec](https://cocoindex.io/docs/core/settings#databaseconnectionspec)
64
+ for more details.
65
+
66
+ ## Simple Data Mapping / Transformation
67
+
68
+ Create a simple transformation to calculate the total price.
69
+
70
+ ```python
71
+ @cocoindex.op.function()
72
+ def calculate_total_value(price: float, amount: int) -> float:
73
+ """Compute total value for each product."""
74
+ return price * amount
75
+ ```
76
+
77
+ Plug into the flow:
78
+
79
+ ```python
80
+ with data_scope["products"].row() as product:
81
+ # Compute total value
82
+ product["total_value"] = flow_builder.transform(
83
+ calculate_total_value,
84
+ product["price"],
85
+ product["amount"],
86
+ )
87
+ ```
88
+
89
+ ![Calculate Total Value](/img/examples/postgres_source/price.png)
90
+
91
+ ### Data Transformation & AI Transformation
92
+
93
+ Create a custom function creates a `full_description` field by combining the product’s category, name, and description.
94
+
95
+ ```python
96
+ @cocoindex.op.function()
97
+ def make_full_description(category: str, name: str, description: str) -> str:
98
+ """Create a detailed product description for embedding."
99
+ return f"Category: {category}\nName: {name}\n\n{description}"
100
+
101
+ ```
102
+
103
+ Embeddings often perform better with more context. By combining fields into a single text string, we ensure that the semantic meaning of the product is captured fully.
104
+
105
+ Now plug into the flow:
106
+
107
+ ```python
108
+ with data_scope["products"].row() as product:
109
+ #.. other transformations
110
+
111
+ # Compute full description
112
+ product["full_description"] = flow_builder.transform(
113
+ make_full_description,
114
+ product["product_category"],
115
+ product["product_name"],
116
+ product["description"],
117
+ )
118
+
119
+ # Generate embeddings
120
+ product["embedding"] = product["full_description"].transform(
121
+ cocoindex.functions.SentenceTransformerEmbed(
122
+ model="sentence-transformers/all-MiniLM-L6-v2"
123
+ )
124
+ )
125
+
126
+ # Collect data
127
+ indexed_product.collect(
128
+ product_category=product["product_category"],
129
+ product_name=product["product_name"],
130
+ description=product["description"],
131
+ price=product["price"],
132
+ amount=product["amount"],
133
+ total_value=product["total_value"],
134
+ embedding=product["embedding"],
135
+ )
136
+ ```
137
+
138
+ This takes each product row, and does the following:
139
+
140
+ 1. builds a rich description.
141
+
142
+ ![Make Full Description](/img/examples/postgres_source/description.png)
143
+
144
+ 2. turns it into an embedding
145
+
146
+ ![Embed Full Description](/img/examples/postgres_source/embed.png)
147
+
148
+ 3. collects the embedding along with structured fields (category, name, price, etc.).
149
+
150
+ ![Collect Embedding](/img/examples/postgres_source/collector.png)
151
+
152
+
153
+ ## Export
154
+
155
+ ```python
156
+ indexed_product.export(
157
+ "output",
158
+ cocoindex.targets.Postgres(),
159
+ primary_key_fields=["product_category", "product_name"],
160
+ vector_indexes=[
161
+ cocoindex.VectorIndexDef(
162
+ field_name="embedding",
163
+ metric=cocoindex.VectorSimilarityMetric.COSINE_SIMILARITY,
164
+ )
165
+ ],
166
+ )
167
+ ```
168
+
169
+ All transformed rows are collected and exported to a new PostgreSQL table with a vector index, ready for semantic search.
170
+
171
+
172
+ ## Field lineage
173
+ When the transform flow starts to getting complex, it's hard to understand how each field is derived.
174
+ CocoIndex provides a way to visualize the lineage of each field, to make it easier to trace and troubleshoot field origins and downstream dependencies.
175
+
176
+ For example, the following image shows the lineage of the `embedding` field, you can click from the final output backward all the way to the source fields, step by step.
177
+
178
+ ![Field Lineage](/img/examples/postgres_source/lineage.png)
179
+
180
+
181
+ ## Running the Pipeline
182
+
183
+ 1. Set up dependencies:
184
+
185
+ ```bash
186
+ pip install -e .
187
+ ```
188
+
189
+ 2. Create the source table with sample data:
190
+
191
+ ```bash
192
+ psql "postgres://cocoindex:cocoindex@localhost/cocoindex" -f ./prepare_source_data.sql
193
+ ```
194
+
195
+ 3. Setup tables and update the index:
196
+
197
+ ```bash
198
+ cocoindex update --setup main.py
199
+ ```
200
+
201
+ 4. Run CocoInsight:
202
+
203
+ ```bash
204
+ cocoindex server -ci main.py
205
+ ```
206
+ You can walk through the project step by step in CocoInsight to see exactly how each field is constructed and what happens behind the scenes. It connects to your local CocoIndex server, with zero pipeline data retention.
207
+
208
+
209
+ ## Continuous Updating
210
+
211
+ For continuous updating when the source changes, add `-L`:
212
+
213
+ ```bash
214
+ cocoindex server -ci -L main.py
215
+ ```
216
+ Check [live updates](https://cocoindex.io/docs/tutorials/live_updates) for more details.
217
+
218
+ ## Search and Query the Index
219
+
220
+ ### Query
221
+
222
+ Runs a semantic similarity search over the indexed products table, returning the top matches for a given query.
223
+
224
+ ```python
225
+ def search(pool: ConnectionPool, query: str, top_k: int = 5) -> list[dict[str, Any]]:
226
+ # Get the table name, for the export target in the text_embedding_flow above.
227
+ table_name = cocoindex.utils.get_target_default_name(
228
+ postgres_product_indexing_flow, "output"
229
+ )
230
+ # Evaluate the transform flow defined above with the input query, to get the embedding.
231
+ query_vector = text_to_embedding.eval(query)
232
+ # Run the query and get the results.
233
+ with pool.connection() as conn:
234
+ register_vector(conn)
235
+ with conn.cursor(row_factory=dict_row) as cur:
236
+ cur.execute(
237
+ f"""
238
+ SELECT
239
+ product_category,
240
+ product_name,
241
+ description,
242
+ amount,
243
+ total_value,
244
+ (embedding <=> %s) AS distance
245
+ FROM {table_name}
246
+ ORDER BY distance ASC
247
+ LIMIT %s
248
+ """,
249
+ (query_vector, top_k),
250
+ )
251
+ return cur.fetchall()
252
+ ```
253
+ This function
254
+
255
+ - Converts the query text into an embedding (`query_vector`).
256
+ - Compares it with each product’s stored embedding (`embedding`) using vector distance.
257
+ - Returns the closest matches, including both metadata and the similarity score (`distance`).
258
+
259
+ ### Create an command-line interactive loop
260
+
261
+ ```python
262
+ def _main() -> None:
263
+ # Initialize the database connection pool.
264
+ pool = ConnectionPool(os.environ["COCOINDEX_DATABASE_URL"])
265
+ # Run queries in a loop to demonstrate the query capabilities.
266
+ while True:
267
+ query = input("Enter search query (or Enter to quit): ")
268
+ if query == "":
269
+ break
270
+ # Run the query function with the database connection pool and the query.
271
+ results = search(pool, query)
272
+ print("\nSearch results:")
273
+ for result in results:
274
+ score = 1.0 - result["distance"]
275
+ print(
276
+ f"[{score:.3f}] {result['product_category']} | {result['product_name']} | {result['amount']} | {result['total_value']}"
277
+ )
278
+ print(f" {result['description']}")
279
+ print("---")
280
+ print()
281
+
282
+ if __name__ == "__main__":
283
+ load_dotenv()
284
+ cocoindex.init()
285
+ _main()
286
+ ```
287
+
288
+ ### Run as a Service
289
+
290
+ This [example](https://cocoindex.io/docs/examples/image_search#fast-api-application) runs as a service using Fast API.
291
+
292
+
293
+ ## Why One Framework for Structured + Unstructured?
294
+
295
+ - One mental model: Treat files, APIs, and databases uniformly; AI steps are ordinary ops.
296
+ - Incremental by default: Use an ordinal column to sync only changes; no fragile glue jobs.
297
+ - Consistency: Embeddings are always derived from the exact transformed row state.
298
+ - Operational simplicity: One deployment, one lineage view, fewer moving parts.
299
+
@@ -71,34 +71,21 @@ with data_scope["documents"].row() as doc:
71
71
  ### Embed each chunk
72
72
 
73
73
  ```python
74
- @cocoindex.transform_flow()
75
- def text_to_embedding(text: cocoindex.DataSlice[str]) -> cocoindex.DataSlice[list[float]]:
76
- """
77
- Embed the text using a SentenceTransformer model.
78
- This is a shared logic between indexing and querying, so extract it as a function.
79
- """
80
- return text.transform(
74
+ with doc["chunks"].row() as chunk:
75
+ chunk["embedding"] = chunk["text"].transform(
81
76
  cocoindex.functions.SentenceTransformerEmbed(
82
- model="sentence-transformers/all-MiniLM-L6-v2"))
77
+ model="sentence-transformers/all-MiniLM-L6-v2"
78
+ )
79
+ )
80
+ doc_embeddings.collect(filename=doc["filename"], location=chunk["location"],
81
+ text=chunk["text"], embedding=chunk["embedding"])
83
82
  ```
84
- ![Embedding](/img/examples/simple_vector_index/embed.png)
85
-
86
- This code defines a transformation function that converts text into vector embeddings using the SentenceTransformer model.
87
- `@cocoindex.transform_flow()` is needed to share the transformation across indexing and query.
88
- This decorator marks this as a reusable transformation flow that can be called on specific input data from user code using `eval()`, as shown in the search function below.
89
83
 
90
84
  The `MiniLM-L6-v2` model is a good balance of speed and quality for text embeddings, though you can swap in other SentenceTransformer models as needed.
91
85
 
92
86
  <DocumentationButton url="https://cocoindex.io/docs/ops/functions#sentencetransformerembed" text="SentenceTransformerEmbed" margin="0 0 16px 0" />
93
-
94
- Plug in the `text_to_embedding` function and collect the embeddings.
95
87
 
96
- ```python
97
- with doc["chunks"].row() as chunk:
98
- chunk["embedding"] = text_to_embedding(chunk["text"])
99
- doc_embeddings.collect(filename=doc["filename"], location=chunk["location"],
100
- text=chunk["text"], embedding=chunk["embedding"])
101
- ```
88
+ ![Embedding](/img/examples/simple_vector_index/embed.png)
102
89
 
103
90
  ## Export the embeddings
104
91
 
@@ -119,10 +106,32 @@ CocoIndex supports other vector databases as well, with 1-line switch.
119
106
 
120
107
  ## Query the index
121
108
 
109
+ ### Define a shared flow for both indexing and querying
110
+
111
+ ```python
112
+ @cocoindex.transform_flow()
113
+ def text_to_embedding(text: cocoindex.DataSlice[str]) -> cocoindex.DataSlice[list[float]]:
114
+ """
115
+ Embed the text using a SentenceTransformer model.
116
+ This is a shared logic between indexing and querying, so extract it as a function.
117
+ """
118
+ return text.transform(
119
+ cocoindex.functions.SentenceTransformerEmbed(
120
+ model="sentence-transformers/all-MiniLM-L6-v2"))
121
+ ```
122
+
123
+ This code defines a transformation function that converts text into vector embeddings using the SentenceTransformer model.
124
+ `@cocoindex.transform_flow()` is needed to share the transformation across indexing and query.
125
+
126
+ This decorator marks this as a reusable transformation flow that can be called on specific input data from user code using `eval()`, as shown in the search function below.
127
+
128
+ ### Write query
129
+
122
130
  CocoIndex doesn't provide additional query interface at the moment. We can write SQL or rely on the query engine by the target storage, if any.
123
131
 
124
132
  <DocumentationButton url="https://cocoindex.io/docs/ops/targets#postgres" text="Postgres" margin="0 0 16px 0" />
125
133
 
134
+
126
135
  ```python
127
136
  def search(pool: ConnectionPool, query: str, top_k: int = 5):
128
137
  table_name = cocoindex.utils.get_target_storage_default_name(text_embedding_flow, "doc_embeddings")
@@ -166,6 +175,19 @@ if __name__ == "__main__":
166
175
  _main()
167
176
  ```
168
177
 
178
+ In the function above, most parts are standard query logic - you can use any libraries you like.
179
+ There're two CocoIndex-specific logic:
180
+
181
+ 1. Get the table name from the export target in the `text_embedding_flow` above.
182
+ Since the table name for the `Postgres` target is not explicitly specified in the `export()` call,
183
+ CocoIndex uses a default name.
184
+ `cocoindex.utils.get_target_default_name()` is a utility function to get the default table name for this case.
185
+
186
+ 2. Evaluate the transform flow defined above with the input query, to get the embedding.
187
+ It's done by the `eval()` method of the transform flow `text_to_embedding`.
188
+ The return type of this method is `NDArray[np.float32]` as declared in the `text_to_embedding()` function (`cocoindex.DataSlice[NDArray[np.float32]]`).
189
+
190
+
169
191
 
170
192
  ## Time to have fun!
171
193
  - Run the following command to setup and update the index.