cocoindex 0.1.29__tar.gz → 0.1.30__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.env.lib_debug +1 -1
- {cocoindex-0.1.29 → cocoindex-0.1.30}/Cargo.lock +1 -1
- {cocoindex-0.1.29 → cocoindex-0.1.30}/Cargo.toml +1 -1
- {cocoindex-0.1.29 → cocoindex-0.1.30}/PKG-INFO +1 -1
- cocoindex-0.1.30/examples/docs_to_knowledge_graph/README.md +80 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/auth_registry.py +5 -2
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/cli.py +40 -13
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/functions.py +7 -3
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/lib.py +6 -59
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/op.py +5 -4
- cocoindex-0.1.30/python/cocoindex/setting.py +77 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/storages.py +4 -5
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/typing.py +5 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/server.rs +12 -8
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/setup/driver.rs +2 -2
- cocoindex-0.1.29/examples/docs_to_knowledge_graph/README.md +0 -67
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.cargo/config.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.github/ISSUE_TEMPLATE//360/237/220/233-bug-report.md" +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.github/ISSUE_TEMPLATE//360/237/222/241-feature-request.md" +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.github/scripts/update_version.sh +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.github/workflows/CI.yml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.github/workflows/_test.yml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.github/workflows/docs.yml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.github/workflows/release.yml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.gitignore +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/.vscode/settings.json +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/CODE_OF_CONDUCT.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/CONTRIBUTING.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/LICENSE +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/dev/neo4j.yaml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/dev/postgres.yaml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/.gitignore +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/about/community.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/about/contributing.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/ai/llm.mdx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/basics.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/cli.mdx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/custom_function.mdx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/data_example.svg +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/data_types.mdx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/flow_def.mdx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/flow_example.svg +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/flow_methods.mdx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/core/initialization.mdx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/getting_started/installation.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/getting_started/markdown_files.zip +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/getting_started/overview.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/getting_started/quickstart.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/ops/functions.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/ops/sources.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docs/ops/storages.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/docusaurus.config.ts +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/package.json +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/sidebars.ts +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/src/components/HomepageFeatures/index.tsx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/src/components/HomepageFeatures/styles.module.css +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/src/css/custom.css +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/src/theme/Root.js +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/static/.nojekyll +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/static/img/docusaurus.png +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/static/img/favicon.ico +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/static/img/icon.svg +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/static/robots.txt +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/tsconfig.json +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/docs/yarn.lock +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/code_embedding/.env +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/code_embedding/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/code_embedding/main.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/code_embedding/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/docs_to_knowledge_graph/.env +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/docs_to_knowledge_graph/main.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/docs_to_knowledge_graph/markdown_files/1706.03762v7.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/docs_to_knowledge_graph/markdown_files/1810.04805v2.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/docs_to_knowledge_graph/markdown_files/rfc8259.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/docs_to_knowledge_graph/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/gdrive_text_embedding/.env.example +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/gdrive_text_embedding/.gitignore +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/gdrive_text_embedding/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/gdrive_text_embedding/data/1706.03762v7.docx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/gdrive_text_embedding/data/1810.04805v2.docx +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/gdrive_text_embedding/main.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/gdrive_text_embedding/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/.env +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/main.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/manuals/array.pdf +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/manuals/base64.pdf +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/manuals/copy.pdf +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/manuals/glob.pdf +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/manuals_llm_extraction/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/pdf_embedding/.env +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/pdf_embedding/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/pdf_embedding/main.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/pdf_embedding/pdf_files/1706.03762v7.pdf +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/pdf_embedding/pdf_files/1810.04805v2.pdf +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/pdf_embedding/pdf_files/rfc8259.pdf +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/pdf_embedding/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/.env +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/Text_Embedding.ipynb +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/main.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/markdown_files/1706.03762v7.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/markdown_files/1810.04805v2.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/markdown_files/rfc8259.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding_qdrant/.env +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding_qdrant/README.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding_qdrant/main.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding_qdrant/markdown_files/rfc8259.md +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/examples/text_embedding_qdrant/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/pyproject.toml +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/__init__.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/convert.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/flow.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/index.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/llm.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/py.typed +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/query.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/runtime.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/setup.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/sources.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/tests/__init__.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/python/cocoindex/tests/test_convert.py +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/base/field_attrs.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/base/json_schema.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/base/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/base/schema.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/base/spec.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/base/value.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/builder/analyzed_flow.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/builder/analyzer.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/builder/flow_builder.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/builder/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/builder/plan.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/db_tracking.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/db_tracking_setup.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/dumper.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/evaluator.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/live_updater.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/memoization.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/query.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/row_indexer.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/source_indexer.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/execution/stats.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/lib.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/lib_context.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/llm/anthropic.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/llm/gemini.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/llm/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/llm/ollama.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/llm/openai.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/factory_bases.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/functions/extract_by_llm.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/functions/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/functions/parse_json.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/functions/split_recursively.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/interface.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/py_factory.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/registration.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/registry.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/sdk.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/sources/google_drive.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/sources/local_file.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/sources/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/storages/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/storages/neo4j.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/storages/postgres.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/storages/qdrant.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/ops/storages/spec.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/prelude.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/py/convert.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/py/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/service/error.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/service/flows.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/service/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/service/search.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/settings.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/setup/auth_registry.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/setup/components.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/setup/db_metadata.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/setup/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/setup/states.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/utils/db.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/utils/fingerprint.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/utils/immutable.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/utils/mod.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/utils/retriable.rs +0 -0
- {cocoindex-0.1.29 → cocoindex-0.1.30}/src/utils/yaml_ser.rs +0 -0
@@ -2,7 +2,7 @@
|
|
2
2
|
name = "cocoindex"
|
3
3
|
# Version used for local development is always higher than others to take precedence.
|
4
4
|
# Will be overridden for specific release versions.
|
5
|
-
version = "0.1.
|
5
|
+
version = "0.1.30"
|
6
6
|
edition = "2021"
|
7
7
|
|
8
8
|
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
|
@@ -0,0 +1,80 @@
|
|
1
|
+
# Build Real-Time Knowledge Graph For Documents with LLM
|
2
|
+
|
3
|
+
We will process a list of documents and use LLM to extract relationships between the concepts in each document.
|
4
|
+
We will generate two kinds of relationships:
|
5
|
+
|
6
|
+
1. Relationships between subjects and objects. E.g., "CocoIndex supports Incremental Processing"
|
7
|
+
2. Mentions of entities in a document. E.g., "core/basics.mdx" mentions `CocoIndex` and `Incremental Processing`.
|
8
|
+
|
9
|
+
You can find a step by step blog for this project [here](https://cocoindex.io/blogs/knowledge-graph-for-docs)
|
10
|
+
|
11
|
+
Please drop [Cocoindex on Github](https://github.com/cocoindex-io/cocoindex) a star to support us if you like our work. Thank you so much with a warm coconut hug 🥥🤗. [](https://github.com/cocoindex-io/cocoindex)
|
12
|
+
|
13
|
+

|
14
|
+
|
15
|
+
|
16
|
+
## Prerequisite
|
17
|
+
* [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one.
|
18
|
+
* [Install Neo4j](https://cocoindex.io/docs/ops/storages#neo4j) if you don't have one.
|
19
|
+
* [Configure your OpenAI API key](https://cocoindex.io/docs/ai/llm#openai).
|
20
|
+
|
21
|
+
## Documentation
|
22
|
+
You can read the official CocoIndex Documentation for Property Graph Targets [here](https://cocoindex.io/docs/ops/storages#property-graph-targets).
|
23
|
+
|
24
|
+
## Run
|
25
|
+
|
26
|
+
### Build the index
|
27
|
+
|
28
|
+
Install dependencies:
|
29
|
+
|
30
|
+
```bash
|
31
|
+
pip install -e .
|
32
|
+
```
|
33
|
+
|
34
|
+
Setup:
|
35
|
+
|
36
|
+
```bash
|
37
|
+
python main.py cocoindex setup
|
38
|
+
```
|
39
|
+
|
40
|
+
Update index:
|
41
|
+
|
42
|
+
```bash
|
43
|
+
python main.py cocoindex update
|
44
|
+
```
|
45
|
+
|
46
|
+
### Browse the knowledge graph
|
47
|
+
|
48
|
+
After the knowledge graph is build, you can explore the knowledge graph you built in Neo4j Browser.
|
49
|
+
|
50
|
+
For the dev enviroment, you can connect neo4j browser using credentials:
|
51
|
+
- username: `neo4j`
|
52
|
+
- password: `cocoindex`
|
53
|
+
which is pre-configured in the our docker compose [config.yaml](https://raw.githubusercontent.com/cocoindex-io/cocoindex/refs/heads/main/dev/neo4j.yaml).
|
54
|
+
|
55
|
+
You can open it at [http://localhost:7474](http://localhost:7474), and run the following Cypher query to get all relationships:
|
56
|
+
|
57
|
+
```cypher
|
58
|
+
MATCH p=()-->() RETURN p
|
59
|
+
```
|
60
|
+
<img width="1366" alt="neo4j-for-coco-docs" src="https://github.com/user-attachments/assets/3c8b6329-6fee-4533-9480-571399b57e57" />
|
61
|
+
|
62
|
+
|
63
|
+
|
64
|
+
## CocoInsight
|
65
|
+
I used CocoInsight (Free beta now) to troubleshoot the index generation and understand the data lineage of the pipeline.
|
66
|
+
It just connects to your local CocoIndex server, with Zero pipeline data retention. Run following command to start CocoInsight:
|
67
|
+
|
68
|
+
```bash
|
69
|
+
python3 main.py cocoindex server -c https://cocoindex.io
|
70
|
+
```
|
71
|
+
|
72
|
+
And then open the url https://cocoindex.io/cocoinsight.
|
73
|
+
|
74
|
+
```
|
75
|
+
python main.py cocoindex server -c https://cocoindex.io
|
76
|
+
```
|
77
|
+
|
78
|
+
<img width="1430" alt="cocoinsight" src="https://github.com/user-attachments/assets/d5ada581-cceb-42bf-a949-132df674f3dd" />
|
79
|
+
|
80
|
+
|
@@ -3,16 +3,19 @@ Auth registry is used to register and reference auth entries.
|
|
3
3
|
"""
|
4
4
|
|
5
5
|
from dataclasses import dataclass
|
6
|
+
from typing import Generic, TypeVar
|
6
7
|
|
7
8
|
from . import _engine
|
8
9
|
from .convert import dump_engine_object
|
9
10
|
|
11
|
+
T = TypeVar("T")
|
12
|
+
|
10
13
|
@dataclass
|
11
|
-
class AuthEntryReference:
|
14
|
+
class AuthEntryReference(Generic[T]):
|
12
15
|
"""Reference an auth entry by its key."""
|
13
16
|
key: str
|
14
17
|
|
15
|
-
def add_auth_entry(key: str, value) -> AuthEntryReference:
|
18
|
+
def add_auth_entry(key: str, value: T) -> AuthEntryReference[T]:
|
16
19
|
"""Add an auth entry to the registry. Returns its reference."""
|
17
20
|
_engine.add_auth_entry(key, dump_engine_object(value))
|
18
21
|
return AuthEntryReference(key)
|
@@ -1,9 +1,9 @@
|
|
1
|
-
import asyncio
|
2
1
|
import click
|
3
2
|
import datetime
|
3
|
+
|
4
4
|
from rich.console import Console
|
5
5
|
|
6
|
-
from . import flow, lib
|
6
|
+
from . import flow, lib, setting
|
7
7
|
from .setup import sync_setup, drop_setup, flow_names_with_setup, apply_setup_changes
|
8
8
|
from .runtime import execution_context
|
9
9
|
|
@@ -21,7 +21,7 @@ def ls(show_all: bool):
|
|
21
21
|
"""
|
22
22
|
List all flows.
|
23
23
|
"""
|
24
|
-
current_flow_names =
|
24
|
+
current_flow_names = flow.flow_names()
|
25
25
|
persisted_flow_names = flow_names_with_setup()
|
26
26
|
remaining_persisted_flow_names = set(persisted_flow_names)
|
27
27
|
|
@@ -58,9 +58,9 @@ def show(flow_name: str | None, color: bool):
|
|
58
58
|
"""
|
59
59
|
Show the flow spec in a readable format with colored output.
|
60
60
|
"""
|
61
|
-
|
61
|
+
fl = _flow_by_name(flow_name)
|
62
62
|
console = Console(no_color=not color)
|
63
|
-
console.print(
|
63
|
+
console.print(fl._render_text())
|
64
64
|
|
65
65
|
@cli.command()
|
66
66
|
def setup():
|
@@ -149,32 +149,59 @@ def evaluate(flow_name: str | None, output_dir: str | None, cache: bool = True):
|
|
149
149
|
options = flow.EvaluateAndDumpOptions(output_dir=output_dir, use_cache=cache)
|
150
150
|
fl.evaluate_and_dump(options)
|
151
151
|
|
152
|
-
|
152
|
+
# Create ServerSettings lazily upon first call, as environment variables may be loaded from files, etc.
|
153
|
+
COCOINDEX_HOST = 'https://cocoindex.io'
|
153
154
|
|
154
155
|
@cli.command()
|
155
156
|
@click.option(
|
156
|
-
"-a", "--address", type=str,
|
157
|
-
help="The address to bind the server to, in the format of IP:PORT."
|
157
|
+
"-a", "--address", type=str,
|
158
|
+
help="The address to bind the server to, in the format of IP:PORT. "
|
159
|
+
"If unspecified, the address specified in COCOINDEX_SERVER_ADDRESS will be used.")
|
160
|
+
@click.option(
|
161
|
+
"-c", "--cors-origin", type=str,
|
162
|
+
help="The origins of the clients (e.g. CocoInsight UI) to allow CORS from. "
|
163
|
+
"Multiple origins can be specified as a comma-separated list. "
|
164
|
+
"e.g. `https://cocoindex.io,http://localhost:3000`. "
|
165
|
+
"Origins specified in COCOINDEX_SERVER_CORS_ORIGINS will also be included.")
|
166
|
+
@click.option(
|
167
|
+
"-ci", "--cors-cocoindex", is_flag=True, show_default=True, default=False,
|
168
|
+
help=f"Allow {COCOINDEX_HOST} to access the server.")
|
158
169
|
@click.option(
|
159
|
-
"-
|
160
|
-
help="
|
161
|
-
"e.g. `http://cocoindex.io` if you want to allow CocoInsight to access the server.")
|
170
|
+
"-cl", "--cors-local", type=int,
|
171
|
+
help="Allow http://localhost:<port> to access the server.")
|
162
172
|
@click.option(
|
163
173
|
"-L", "--live-update", is_flag=True, show_default=True, default=False,
|
164
174
|
help="Continuously watch changes from data sources and apply to the target index.")
|
165
175
|
@click.option(
|
166
176
|
"-q", "--quiet", is_flag=True, show_default=True, default=False,
|
167
177
|
help="Avoid printing anything to the standard output, e.g. statistics.")
|
168
|
-
def server(address: str, live_update: bool, quiet: bool, cors_origin: str | None
|
178
|
+
def server(address: str | None, live_update: bool, quiet: bool, cors_origin: str | None,
|
179
|
+
cors_cocoindex: bool, cors_local: int | None):
|
169
180
|
"""
|
170
181
|
Start a HTTP server providing REST APIs.
|
171
182
|
|
172
183
|
It will allow tools like CocoInsight to access the server.
|
173
184
|
"""
|
174
|
-
|
185
|
+
server_settings = setting.ServerSettings.from_env()
|
186
|
+
cors_origins: set[str] = set(server_settings.cors_origins or [])
|
187
|
+
if cors_origin is not None:
|
188
|
+
cors_origins.update(setting.ServerSettings.parse_cors_origins(cors_origin))
|
189
|
+
if cors_cocoindex:
|
190
|
+
cors_origins.add(COCOINDEX_HOST)
|
191
|
+
if cors_local is not None:
|
192
|
+
cors_origins.add(f"http://localhost:{cors_local}")
|
193
|
+
server_settings.cors_origins = list(cors_origins)
|
194
|
+
|
195
|
+
if address is not None:
|
196
|
+
server_settings.address = address
|
197
|
+
|
198
|
+
lib.start_server(server_settings)
|
199
|
+
|
175
200
|
if live_update:
|
176
201
|
options = flow.FlowLiveUpdaterOptions(live_mode=True, print_stats=not quiet)
|
177
202
|
execution_context.run(flow.update_all_flows(options))
|
203
|
+
if COCOINDEX_HOST in cors_origins:
|
204
|
+
click.echo(f"Open CocoInsight at: {COCOINDEX_HOST}/cocoinsight")
|
178
205
|
input("Press Enter to stop...")
|
179
206
|
|
180
207
|
|
@@ -1,10 +1,13 @@
|
|
1
1
|
"""All builtin functions."""
|
2
|
-
from typing import Annotated, Any
|
2
|
+
from typing import Annotated, Any, TYPE_CHECKING
|
3
3
|
|
4
|
-
import sentence_transformers
|
5
4
|
from .typing import Float32, Vector, TypeAttr
|
6
5
|
from . import op, llm
|
7
6
|
|
7
|
+
# Libraries that are heavy to import. Lazily import them later.
|
8
|
+
if TYPE_CHECKING:
|
9
|
+
import sentence_transformers
|
10
|
+
|
8
11
|
class ParseJson(op.FunctionSpec):
|
9
12
|
"""Parse a text into a JSON object."""
|
10
13
|
|
@@ -35,9 +38,10 @@ class SentenceTransformerEmbedExecutor:
|
|
35
38
|
"""Executor for SentenceTransformerEmbed."""
|
36
39
|
|
37
40
|
spec: SentenceTransformerEmbed
|
38
|
-
_model: sentence_transformers.SentenceTransformer
|
41
|
+
_model: "sentence_transformers.SentenceTransformer"
|
39
42
|
|
40
43
|
def analyze(self, text):
|
44
|
+
import sentence_transformers # pylint: disable=import-outside-toplevel
|
41
45
|
args = self.spec.args or {}
|
42
46
|
self._model = sentence_transformers.SentenceTransformer(self.spec.model, **args)
|
43
47
|
dim = self._model.get_sentence_embedding_dimension()
|
@@ -1,76 +1,23 @@
|
|
1
1
|
"""
|
2
2
|
Library level functions and states.
|
3
3
|
"""
|
4
|
-
import os
|
5
4
|
import sys
|
6
5
|
import functools
|
7
6
|
import inspect
|
8
7
|
|
9
|
-
from typing import Callable
|
10
|
-
from dataclasses import dataclass
|
8
|
+
from typing import Callable
|
11
9
|
|
12
10
|
from . import _engine
|
13
|
-
from . import flow, query, cli
|
11
|
+
from . import flow, query, cli, setting
|
14
12
|
from .convert import dump_engine_object
|
15
13
|
|
16
14
|
|
17
|
-
def
|
18
|
-
value = os.getenv(env_name)
|
19
|
-
if value is None:
|
20
|
-
if required:
|
21
|
-
raise ValueError(f"{env_name} is not set")
|
22
|
-
else:
|
23
|
-
target[name] = value
|
24
|
-
|
25
|
-
@dataclass
|
26
|
-
class DatabaseConnectionSpec:
|
27
|
-
url: str
|
28
|
-
user: str | None = None
|
29
|
-
password: str | None = None
|
30
|
-
|
31
|
-
@dataclass
|
32
|
-
class Settings:
|
33
|
-
"""Settings for the cocoindex library."""
|
34
|
-
database: DatabaseConnectionSpec
|
35
|
-
|
36
|
-
@classmethod
|
37
|
-
def from_env(cls) -> Self:
|
38
|
-
"""Load settings from environment variables."""
|
39
|
-
|
40
|
-
db_kwargs: dict[str, str] = dict()
|
41
|
-
_load_field(db_kwargs, "url", "COCOINDEX_DATABASE_URL", required=True)
|
42
|
-
_load_field(db_kwargs, "user", "COCOINDEX_DATABASE_USER")
|
43
|
-
_load_field(db_kwargs, "password", "COCOINDEX_DATABASE_PASSWORD")
|
44
|
-
database = DatabaseConnectionSpec(**db_kwargs)
|
45
|
-
return cls(database=database)
|
46
|
-
|
47
|
-
|
48
|
-
def init(settings: Settings):
|
15
|
+
def init(settings: setting.Settings):
|
49
16
|
"""Initialize the cocoindex library."""
|
50
17
|
_engine.init(dump_engine_object(settings))
|
51
18
|
|
52
|
-
@dataclass
|
53
|
-
class ServerSettings:
|
54
|
-
"""Settings for the cocoindex server."""
|
55
|
-
|
56
|
-
# The address to bind the server to.
|
57
|
-
address: str = "127.0.0.1:8080"
|
58
|
-
|
59
|
-
# The origin of the client (e.g. CocoInsight UI) to allow CORS from.
|
60
|
-
cors_origin: str | None = None
|
61
|
-
|
62
|
-
@classmethod
|
63
|
-
def from_env(cls) -> Self:
|
64
|
-
"""Load settings from environment variables."""
|
65
|
-
|
66
|
-
kwargs: dict[str, str] = dict()
|
67
|
-
_load_field(kwargs, "address", "COCOINDEX_SERVER_ADDRESS")
|
68
|
-
_load_field(kwargs, "cors_origin", "COCOINDEX_SERVER_CORS_ORIGIN")
|
69
|
-
|
70
|
-
return cls(**kwargs)
|
71
|
-
|
72
19
|
|
73
|
-
def start_server(settings: ServerSettings):
|
20
|
+
def start_server(settings: setting.ServerSettings):
|
74
21
|
"""Start the cocoindex server."""
|
75
22
|
flow.ensure_all_flows_built()
|
76
23
|
query.ensure_all_handlers_built()
|
@@ -81,7 +28,7 @@ def stop():
|
|
81
28
|
_engine.stop()
|
82
29
|
|
83
30
|
def main_fn(
|
84
|
-
settings: Settings | None = None,
|
31
|
+
settings: setting.Settings | None = None,
|
85
32
|
cocoindex_cmd: str = 'cocoindex',
|
86
33
|
) -> Callable[[Callable], Callable]:
|
87
34
|
"""
|
@@ -92,7 +39,7 @@ def main_fn(
|
|
92
39
|
"""
|
93
40
|
|
94
41
|
def _pre_init() -> None:
|
95
|
-
effective_settings = settings or Settings.from_env()
|
42
|
+
effective_settings = settings or setting.Settings.from_env()
|
96
43
|
init(effective_settings)
|
97
44
|
|
98
45
|
def _should_run_cli() -> bool:
|
@@ -5,10 +5,10 @@ import asyncio
|
|
5
5
|
import dataclasses
|
6
6
|
import inspect
|
7
7
|
|
8
|
-
from typing import
|
8
|
+
from typing import Protocol, Any, Callable, Awaitable, dataclass_transform
|
9
9
|
from enum import Enum
|
10
10
|
|
11
|
-
from .typing import encode_enriched_type
|
11
|
+
from .typing import encode_enriched_type, resolve_forward_ref
|
12
12
|
from .convert import encode_engine_value, make_engine_value_decoder
|
13
13
|
from . import _engine
|
14
14
|
|
@@ -214,10 +214,11 @@ def executor_class(**args) -> Callable[[type], type]:
|
|
214
214
|
"""
|
215
215
|
Decorate a class to provide an executor for an op.
|
216
216
|
"""
|
217
|
-
|
217
|
+
# Use `__annotations__` instead of `get_type_hints`, to avoid resolving forward references.
|
218
|
+
type_hints = cls.__annotations__
|
218
219
|
if 'spec' not in type_hints:
|
219
220
|
raise TypeError("Expect a `spec` field with type hint")
|
220
|
-
spec_cls = type_hints['spec']
|
221
|
+
spec_cls = resolve_forward_ref(type_hints['spec'])
|
221
222
|
sig = inspect.signature(cls.__call__)
|
222
223
|
return _register_op_factory(
|
223
224
|
category=spec_cls._op_category,
|
@@ -0,0 +1,77 @@
|
|
1
|
+
"""
|
2
|
+
Data types for settings of the cocoindex library.
|
3
|
+
"""
|
4
|
+
import os
|
5
|
+
|
6
|
+
from typing import Callable, Self, Any, overload
|
7
|
+
from dataclasses import dataclass
|
8
|
+
|
9
|
+
|
10
|
+
@dataclass
|
11
|
+
class DatabaseConnectionSpec:
|
12
|
+
"""
|
13
|
+
Connection spec for relational database.
|
14
|
+
Used by both internal and target storage.
|
15
|
+
"""
|
16
|
+
url: str
|
17
|
+
user: str | None = None
|
18
|
+
password: str | None = None
|
19
|
+
|
20
|
+
def _load_field(target: dict[str, Any], name: str, env_name: str, required: bool = False,
|
21
|
+
parse: Callable[[str], Any] | None = None):
|
22
|
+
value = os.getenv(env_name)
|
23
|
+
if value is None:
|
24
|
+
if required:
|
25
|
+
raise ValueError(f"{env_name} is not set")
|
26
|
+
else:
|
27
|
+
target[name] = value if parse is None else parse(value)
|
28
|
+
|
29
|
+
@dataclass
|
30
|
+
class Settings:
|
31
|
+
"""Settings for the cocoindex library."""
|
32
|
+
database: DatabaseConnectionSpec
|
33
|
+
|
34
|
+
@classmethod
|
35
|
+
def from_env(cls) -> Self:
|
36
|
+
"""Load settings from environment variables."""
|
37
|
+
|
38
|
+
db_kwargs: dict[str, str] = dict()
|
39
|
+
_load_field(db_kwargs, "url", "COCOINDEX_DATABASE_URL", required=True)
|
40
|
+
_load_field(db_kwargs, "user", "COCOINDEX_DATABASE_USER")
|
41
|
+
_load_field(db_kwargs, "password", "COCOINDEX_DATABASE_PASSWORD")
|
42
|
+
database = DatabaseConnectionSpec(**db_kwargs)
|
43
|
+
return cls(database=database)
|
44
|
+
|
45
|
+
@dataclass
|
46
|
+
class ServerSettings:
|
47
|
+
"""Settings for the cocoindex server."""
|
48
|
+
|
49
|
+
# The address to bind the server to.
|
50
|
+
address: str = "127.0.0.1:8080"
|
51
|
+
|
52
|
+
# The origins of the clients (e.g. CocoInsight UI) to allow CORS from.
|
53
|
+
cors_origins: list[str] | None = None
|
54
|
+
|
55
|
+
@classmethod
|
56
|
+
def from_env(cls) -> Self:
|
57
|
+
"""Load settings from environment variables."""
|
58
|
+
kwargs: dict[str, Any] = dict()
|
59
|
+
_load_field(kwargs, "address", "COCOINDEX_SERVER_ADDRESS")
|
60
|
+
_load_field(kwargs, "cors_origins", "COCOINDEX_SERVER_CORS_ORIGINS",
|
61
|
+
parse=ServerSettings.parse_cors_origins)
|
62
|
+
return cls(**kwargs)
|
63
|
+
|
64
|
+
@overload
|
65
|
+
@staticmethod
|
66
|
+
def parse_cors_origins(s: str) -> list[str]: ...
|
67
|
+
|
68
|
+
@overload
|
69
|
+
@staticmethod
|
70
|
+
def parse_cors_origins(s: str | None) -> list[str] | None: ...
|
71
|
+
|
72
|
+
@staticmethod
|
73
|
+
def parse_cors_origins(s):
|
74
|
+
"""
|
75
|
+
Parse the CORS origins from a string.
|
76
|
+
"""
|
77
|
+
return [o for e in s.split(",") if (o := e.strip()) != ""] if s is not None else None
|
@@ -5,11 +5,11 @@ from typing import Sequence
|
|
5
5
|
from . import op
|
6
6
|
from . import index
|
7
7
|
from .auth_registry import AuthEntryReference
|
8
|
+
from .setting import DatabaseConnectionSpec
|
8
9
|
|
9
10
|
class Postgres(op.StorageSpec):
|
10
11
|
"""Storage powered by Postgres and pgvector."""
|
11
|
-
|
12
|
-
database: AuthEntryReference | None = None
|
12
|
+
database: AuthEntryReference[DatabaseConnectionSpec] | None = None
|
13
13
|
table_name: str | None = None
|
14
14
|
|
15
15
|
@dataclass
|
@@ -72,15 +72,14 @@ NodeReferenceMapping = NodeFromFields
|
|
72
72
|
|
73
73
|
class Neo4j(op.StorageSpec):
|
74
74
|
"""Graph storage powered by Neo4j."""
|
75
|
-
|
76
|
-
connection: AuthEntryReference
|
75
|
+
connection: AuthEntryReference[Neo4jConnection]
|
77
76
|
mapping: Nodes | Relationships
|
78
77
|
|
79
78
|
class Neo4jDeclaration(op.DeclarationSpec):
|
80
79
|
"""Declarations for Neo4j."""
|
81
80
|
|
82
81
|
kind = "Neo4j"
|
83
|
-
connection: AuthEntryReference
|
82
|
+
connection: AuthEntryReference[Neo4jConnection]
|
84
83
|
nodes_label: str
|
85
84
|
primary_key_fields: Sequence[str]
|
86
85
|
vector_indexes: Sequence[index.VectorIndexDef] = ()
|
@@ -1,10 +1,7 @@
|
|
1
|
-
use crate::
|
1
|
+
use crate::prelude::*;
|
2
2
|
|
3
|
-
use
|
3
|
+
use crate::{lib_context::LibContext, service};
|
4
4
|
use axum::{routing, Router};
|
5
|
-
use futures::{future::BoxFuture, FutureExt};
|
6
|
-
use serde::Deserialize;
|
7
|
-
use std::sync::Arc;
|
8
5
|
use tower::ServiceBuilder;
|
9
6
|
use tower_http::{
|
10
7
|
cors::{AllowOrigin, CorsLayer},
|
@@ -14,7 +11,8 @@ use tower_http::{
|
|
14
11
|
#[derive(Deserialize, Debug)]
|
15
12
|
pub struct ServerSettings {
|
16
13
|
pub address: String,
|
17
|
-
|
14
|
+
#[serde(default)]
|
15
|
+
pub cors_origins: Vec<String>,
|
18
16
|
}
|
19
17
|
|
20
18
|
/// Initialize the server and return a future that will actually handle requests.
|
@@ -23,9 +21,15 @@ pub async fn init_server(
|
|
23
21
|
settings: ServerSettings,
|
24
22
|
) -> Result<BoxFuture<'static, ()>> {
|
25
23
|
let mut cors = CorsLayer::default();
|
26
|
-
|
24
|
+
debug!("cors_origins: {:?}", settings.cors_origins);
|
25
|
+
if !settings.cors_origins.is_empty() {
|
26
|
+
let origins: Vec<_> = settings
|
27
|
+
.cors_origins
|
28
|
+
.iter()
|
29
|
+
.map(|origin| origin.parse())
|
30
|
+
.collect::<Result<_, _>>()?;
|
27
31
|
cors = cors
|
28
|
-
.allow_origin(AllowOrigin::
|
32
|
+
.allow_origin(AllowOrigin::list(origins))
|
29
33
|
.allow_methods([
|
30
34
|
axum::http::Method::GET,
|
31
35
|
axum::http::Method::POST,
|
@@ -215,11 +215,11 @@ fn group_resource_states<'a>(
|
|
215
215
|
entry.existing.current = Some(current.clone());
|
216
216
|
}
|
217
217
|
if let Some(legacy_state_key) = &state.legacy_state_key {
|
218
|
-
if
|
218
|
+
if entry
|
219
219
|
.existing
|
220
220
|
.legacy_state_key
|
221
221
|
.as_ref()
|
222
|
-
.map_or(false, |v| v
|
222
|
+
.map_or(false, |v| v != legacy_state_key)
|
223
223
|
{
|
224
224
|
warn!(
|
225
225
|
"inconsistent legacy key: {:?}, {:?}",
|
@@ -1,67 +0,0 @@
|
|
1
|
-
# Build Knowledge Graph from Markdown Documents, with OpenAI, Neo4j and CocoIndex
|
2
|
-
|
3
|
-
In this example, we
|
4
|
-
|
5
|
-
* Extract relationships from Markdown documents.
|
6
|
-
* Build a knowledge graph from the relationships.
|
7
|
-
|
8
|
-
Please give [Cocoindex on Github](https://github.com/cocoindex-io/cocoindex) a star to support us if you like our work. Thank you so much with a warm coconut hug 🥥🤗. [](https://github.com/cocoindex-io/cocoindex)
|
9
|
-
|
10
|
-
## Prerequisite
|
11
|
-
|
12
|
-
Before running the example, you need to:
|
13
|
-
|
14
|
-
* [Install Postgres](https://cocoindex.io/docs/getting_started/installation#-install-postgres) if you don't have one.
|
15
|
-
* [Install Neo4j](https://cocoindex.io/docs/ops/storages#neo4j) if you don't have one.
|
16
|
-
* Install / configure LLM API. In this example we use OpenAI. You need to [configure OpenAI API key](https://cocoindex.io/docs/ai/llm#openai) before running the example. Alternatively, you can also follow the comments in source code to switch to Ollama, which runs LLM model locally, and get it ready following [this guide](https://cocoindex.io/docs/ai/llm#ollama).
|
17
|
-
|
18
|
-
## Run
|
19
|
-
|
20
|
-
### Build the index
|
21
|
-
|
22
|
-
Install dependencies:
|
23
|
-
|
24
|
-
```bash
|
25
|
-
pip install -e .
|
26
|
-
```
|
27
|
-
|
28
|
-
Setup:
|
29
|
-
|
30
|
-
```bash
|
31
|
-
python main.py cocoindex setup
|
32
|
-
```
|
33
|
-
|
34
|
-
Update index:
|
35
|
-
|
36
|
-
```bash
|
37
|
-
python main.py cocoindex update
|
38
|
-
```
|
39
|
-
|
40
|
-
### Browse the knowledge graph
|
41
|
-
|
42
|
-
After the knowledge graph is build, you can explore the knowledge graph you built in Neo4j Browser.
|
43
|
-
|
44
|
-
For the dev enviroment, you can connect neo4j browser using credentials:
|
45
|
-
- username: `neo4j`
|
46
|
-
- password: `cocoindex`
|
47
|
-
which is pre-configured in the our docker compose [config.yaml](https://raw.githubusercontent.com/cocoindex-io/cocoindex/refs/heads/main/dev/neo4j.yaml).
|
48
|
-
|
49
|
-
You can open it at [http://localhost:7474](http://localhost:7474), and run the following Cypher query to get all relationships:
|
50
|
-
|
51
|
-
```cypher
|
52
|
-
MATCH p=()-->() RETURN p
|
53
|
-
```
|
54
|
-
|
55
|
-
## CocoInsight
|
56
|
-
CocoInsight is a tool to help you understand your data pipeline and data index. CocoInsight is in Early Access now (Free) 😊 You found us! A quick 3 minute video tutorial about CocoInsight: [Watch on YouTube](https://youtu.be/ZnmyoHslBSc?si=pPLXWALztkA710r9).
|
57
|
-
|
58
|
-
Run CocoInsight to understand your RAG data pipeline:
|
59
|
-
|
60
|
-
```
|
61
|
-
python main.py cocoindex server -c https://cocoindex.io
|
62
|
-
```
|
63
|
-
|
64
|
-
Then open the CocoInsight UI at [https://cocoindex.io/cocoinsight](https://cocoindex.io/cocoinsight). It connects to your local CocoIndex server with zero data retention.
|
65
|
-
|
66
|
-
You can view the pipeline flow and the data preview in the CocoInsight UI:
|
67
|
-

|
File without changes
|
{cocoindex-0.1.29 → cocoindex-0.1.30}/.github/ISSUE_TEMPLATE//360/237/220/233-bug-report.md"
RENAMED
File without changes
|
{cocoindex-0.1.29 → cocoindex-0.1.30}/.github/ISSUE_TEMPLATE//360/237/222/241-feature-request.md"
RENAMED
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|
File without changes
|