benodb 0.17.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- benodb-0.17.0/.cargo/config.toml +3 -0
- benodb-0.17.0/.dockerignore +17 -0
- benodb-0.17.0/.env.example +51 -0
- benodb-0.17.0/.gitattributes +1 -0
- benodb-0.17.0/.gitignore +150 -0
- benodb-0.17.0/.instructions.md +67 -0
- benodb-0.17.0/.mailmap +6 -0
- benodb-0.17.0/AGENTS.md +117 -0
- benodb-0.17.0/BenoDB.png +0 -0
- benodb-0.17.0/CHANGELOG.md +3756 -0
- benodb-0.17.0/CONTRIBUTING.md +279 -0
- benodb-0.17.0/Cargo.lock +9804 -0
- benodb-0.17.0/Cargo.toml +259 -0
- benodb-0.17.0/Dockerfile +110 -0
- benodb-0.17.0/LICENSE-APACHE +201 -0
- benodb-0.17.0/LICENSE-MIT +21 -0
- benodb-0.17.0/NOTICE +22 -0
- benodb-0.17.0/PKG-INFO +1253 -0
- benodb-0.17.0/README.md +1204 -0
- benodb-0.17.0/SECURITY.md +124 -0
- benodb-0.17.0/THIRDPARTY_NOTICES.md +109 -0
- benodb-0.17.0/audit.toml +48 -0
- benodb-0.17.0/benodb-gpu-ann/Cargo.toml +44 -0
- benodb-0.17.0/benodb-gpu-ann/LICENSE-APACHE +201 -0
- benodb-0.17.0/benodb-gpu-ann/LICENSE-MIT +21 -0
- benodb-0.17.0/benodb-gpu-ann/NOTICE +13 -0
- benodb-0.17.0/benodb-gpu-ann/src/backend/cpu.rs +142 -0
- benodb-0.17.0/benodb-gpu-ann/src/backend/cuda.rs +334 -0
- benodb-0.17.0/benodb-gpu-ann/src/backend/metal.rs +172 -0
- benodb-0.17.0/benodb-gpu-ann/src/backend/mod.rs +216 -0
- benodb-0.17.0/benodb-gpu-ann/src/backend/nvrtc.rs +542 -0
- benodb-0.17.0/benodb-gpu-ann/src/backend/wgpu.rs +211 -0
- benodb-0.17.0/benodb-gpu-ann/src/cagra.rs +578 -0
- benodb-0.17.0/benodb-gpu-ann/src/e_means.rs +608 -0
- benodb-0.17.0/benodb-gpu-ann/src/fastscan.rs +392 -0
- benodb-0.17.0/benodb-gpu-ann/src/hnsw.rs +533 -0
- benodb-0.17.0/benodb-gpu-ann/src/ivf_fastscan.rs +279 -0
- benodb-0.17.0/benodb-gpu-ann/src/ivf_flat.rs +245 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/batch_distance.cu +165 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/cosine_distance.cu +61 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/hamming_distance.cu +43 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/hamming_packed.cu +28 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/inner_product.cu +40 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/jaccard_distance.cu +64 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/jaccard_packed.cu +41 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/kmeans_assignment.cu +33 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/l1_distance.cu +40 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/cuda/l2_distance.cu +40 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/cosine_distance.metal +41 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/hamming_distance.metal +31 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/hamming_packed.metal +26 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/inner_product.metal +29 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/jaccard_distance.metal +43 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/jaccard_packed.metal +30 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/kmeans_assignment.metal +41 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/l1_distance.metal +29 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/metal/l2_distance.metal +30 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/wgsl/wgpu_binary_kernel.wgsl +46 -0
- benodb-0.17.0/benodb-gpu-ann/src/kernels/wgsl/wgpu_kernel.wgsl +96 -0
- benodb-0.17.0/benodb-gpu-ann/src/kmeans.rs +194 -0
- benodb-0.17.0/benodb-gpu-ann/src/lib.rs +272 -0
- benodb-0.17.0/benodb-gpu-ann/src/metric.rs +33 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_cagra.rs +173 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_cross_backend.rs +249 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_edge_cases.rs +256 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_fuzz_no_panic.rs +120 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_hnsw.rs +163 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_ivf_fastscan.rs +81 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_ivf_flat.rs +101 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_parity_all_algorithms.rs +164 -0
- benodb-0.17.0/benodb-gpu-ann/tests/test_soak.rs +116 -0
- benodb-0.17.0/blog/articles/2026-10-22_graph_rag_howto.ipynb +2030 -0
- benodb-0.17.0/blog/articles/2026-10-27_graph_rag_howto.ipynb +2036 -0
- benodb-0.17.0/book.toml +10 -0
- benodb-0.17.0/build-connectors.sh +187 -0
- benodb-0.17.0/build.rs +58 -0
- benodb-0.17.0/clippy.toml +17 -0
- benodb-0.17.0/dbt-benodb/README.md +82 -0
- benodb-0.17.0/dbt-benodb/dbt/adapters/benodb/__init__.py +12 -0
- benodb-0.17.0/dbt-benodb/dbt/adapters/benodb/__version__.py +57 -0
- benodb-0.17.0/dbt-benodb/dbt/adapters/benodb/connections.py +206 -0
- benodb-0.17.0/dbt-benodb/dbt/adapters/benodb/impl.py +96 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/__init__.py +3 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/dbt_project.yml +10 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/adapters.sql +90 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/ddl.sql +89 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/graph.sql +503 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/json.sql +115 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/materializations/incremental.sql +108 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/materializations/table.sql +19 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/search.sql +55 -0
- benodb-0.17.0/dbt-benodb/dbt/include/benodb/macros/vector.sql +349 -0
- benodb-0.17.0/dbt-benodb/pyproject.toml +53 -0
- benodb-0.17.0/dbt-benodb/setup.py +10 -0
- benodb-0.17.0/deny.toml +116 -0
- benodb-0.17.0/deploy/observability/README.md +74 -0
- benodb-0.17.0/deploy/observability/docker-compose.observability.yml +54 -0
- benodb-0.17.0/deploy/observability/grafana/dashboards/benodb.json +162 -0
- benodb-0.17.0/deploy/observability/grafana/provisioning/dashboards/dashboards.yml +13 -0
- benodb-0.17.0/deploy/observability/grafana/provisioning/datasources/prometheus.yml +12 -0
- benodb-0.17.0/deploy/observability/otel-collector.yaml +32 -0
- benodb-0.17.0/deploy/observability/prometheus.yml +17 -0
- benodb-0.17.0/dml_copy.rs +693 -0
- benodb-0.17.0/docker-compose-hive.yml +104 -0
- benodb-0.17.0/docker-compose-rustfs-nessie.yml +135 -0
- benodb-0.17.0/docker-compose.production.yml +121 -0
- benodb-0.17.0/docker-compose.yml +153 -0
- benodb-0.17.0/docs/.nojekyll +0 -0
- benodb-0.17.0/docs/ADMIN_CLI.md +59 -0
- benodb-0.17.0/docs/ARCHITECTURE.md +98 -0
- benodb-0.17.0/docs/BENCHMARKING.md +652 -0
- benodb-0.17.0/docs/BENCHMARKING_PLAN.md +455 -0
- benodb-0.17.0/docs/CATALOG_USAGE.md +429 -0
- benodb-0.17.0/docs/CONCURRENCY.md +58 -0
- benodb-0.17.0/docs/CONFIGURATION.md +721 -0
- benodb-0.17.0/docs/EMEANS_SPECIFICATION.md +191 -0
- benodb-0.17.0/docs/GPU_SETUP_GUIDE.md +483 -0
- benodb-0.17.0/docs/GRAPH_RAG_EDGE_TABLES.md +374 -0
- benodb-0.17.0/docs/ICEBERG_COMPATIBILITY.md +75 -0
- benodb-0.17.0/docs/ICEBERG_STORAGE.md +211 -0
- benodb-0.17.0/docs/ICEBERG_V2_V3_API.md +249 -0
- benodb-0.17.0/docs/INDEX.md +76 -0
- benodb-0.17.0/docs/INSTALLATION.md +734 -0
- benodb-0.17.0/docs/MONITORING.md +150 -0
- benodb-0.17.0/docs/MUSIQUE_GRAPHRAG.md +128 -0
- benodb-0.17.0/docs/NO_PANIC_POLICY.md +146 -0
- benodb-0.17.0/docs/OPENSEARCH_COMPATIBILITY.md +310 -0
- benodb-0.17.0/docs/PGVECTOR_SQL_GUIDE.md +735 -0
- benodb-0.17.0/docs/PRODUCTION_READINESS_ASSESSMENT.md +136 -0
- benodb-0.17.0/docs/PYTHON_VECTOR_API.md +1071 -0
- benodb-0.17.0/docs/QDRANT_COMPATIBILITY.md +160 -0
- benodb-0.17.0/docs/QUICKSTART.md +262 -0
- benodb-0.17.0/docs/RECOVERY_RUNBOOK.md +101 -0
- benodb-0.17.0/docs/RESOURCE_LIMITS.md +197 -0
- benodb-0.17.0/docs/ROADMAP.md +474 -0
- benodb-0.17.0/docs/SUPPORT_MATRIX.md +127 -0
- benodb-0.17.0/docs/architecture/ADR_001_TIGHTENED_INDEX_OVERLAY_CONTRACT.md +546 -0
- benodb-0.17.0/docs/architecture/ADR_002_GLOBAL_VECTOR_CLUSTERING.md +170 -0
- benodb-0.17.0/docs/architecture/ADR_003_GRAPH_RAG_OUT_OF_CORE_MEMORY.md +85 -0
- benodb-0.17.0/docs/architecture/ADR_004_OBSERVABILITY_PIPELINE.md +145 -0
- benodb-0.17.0/docs/architecture/ADR_005_SQL_DDL_AND_MAINTENANCE_SURFACE.md +289 -0
- benodb-0.17.0/docs/architecture/ADR_006_TRINO_TABLE_FUNCTION_PUSHDOWN.md +409 -0
- benodb-0.17.0/docs/architecture/ADR_007_NATIVE_CONNECTOR_ARCHITECTURE_AND_UDF_EXPOSURE.md +169 -0
- benodb-0.17.0/docs/architecture/ADR_008_PENDING_COMMIT_STATE_MACHINE.md +135 -0
- benodb-0.17.0/docs/architecture/ADR_009_CATALOG_PARITY_AND_CAS_HARDENING.md +120 -0
- benodb-0.17.0/docs/architecture/ADR_010_ICEBERG_V3_DELETION_VECTORS_AND_CROSS_ENGINE_CONFORMANCE.md +57 -0
- benodb-0.17.0/docs/architecture/README.md +97 -0
- benodb-0.17.0/docs/governance/DEPENDENCY_RISK.md +183 -0
- benodb-0.17.0/docs/integrations/DBT.md +239 -0
- benodb-0.17.0/docs/integrations/FLIGHT_SQL.md +143 -0
- benodb-0.17.0/docs/integrations/ICEBERG.md +184 -0
- benodb-0.17.0/docs/integrations/JAVA_JNI.md +34 -0
- benodb-0.17.0/docs/integrations/MCP.md +141 -0
- benodb-0.17.0/docs/integrations/PYTHON.md +170 -0
- benodb-0.17.0/docs/integrations/README.md +27 -0
- benodb-0.17.0/docs/integrations/SEARCH_GATEWAY.md +116 -0
- benodb-0.17.0/docs/integrations/SNOWFLAKE.md +172 -0
- benodb-0.17.0/docs/integrations/SPARK.md +401 -0
- benodb-0.17.0/docs/integrations/TRINO.md +408 -0
- benodb-0.17.0/docs/integrations/TRINO_VERSION_BANDS.md +299 -0
- benodb-0.17.0/docs/requirements.txt +10 -0
- benodb-0.17.0/docs/source/_static/BenoDB.png +0 -0
- benodb-0.17.0/docs/source/_static/custom.css +11 -0
- benodb-0.17.0/docs/source/api/python.rst +40 -0
- benodb-0.17.0/docs/source/api/rust.rst +11 -0
- benodb-0.17.0/docs/source/conf.py +104 -0
- benodb-0.17.0/docs/source/index.rst +119 -0
- benodb-0.17.0/docs/tutorials/01_AI_RETRIEVAL_MCP.md +121 -0
- benodb-0.17.0/docs/tutorials/02_JSON_DOCUMENT_RETRIEVAL.md +76 -0
- benodb-0.17.0/docs/tutorials/03_GRAPH_TRAVERSAL.md +72 -0
- benodb-0.17.0/hive-config/hive-site.xml +75 -0
- benodb-0.17.0/plans/OUT_OF_CORE_SPEEDUP_ANALYSIS.md +353 -0
- benodb-0.17.0/plans/production_readiness_plan.md +829 -0
- benodb-0.17.0/proptest-regressions/core/index/gpu.txt +9 -0
- benodb-0.17.0/proptest-regressions/core/sql/vector_literal.txt +10 -0
- benodb-0.17.0/proptest-regressions/core/sql/vector_udf.txt +8 -0
- benodb-0.17.0/pyproject.toml +93 -0
- benodb-0.17.0/python/benodb/__init__.py +4387 -0
- benodb-0.17.0/python/benodb/cli.py +225 -0
- benodb-0.17.0/python/benodb/embeddings.py +100 -0
- benodb-0.17.0/run_test.py +6 -0
- benodb-0.17.0/rust-toolchain.toml +11 -0
- benodb-0.17.0/scripts/ai_pr_review.py +207 -0
- benodb-0.17.0/scripts/build_demo_dataset.py +370 -0
- benodb-0.17.0/scripts/build_three_tier.py +700 -0
- benodb-0.17.0/scripts/check_docs_links.py +77 -0
- benodb-0.17.0/scripts/clean_scratch.sh +83 -0
- benodb-0.17.0/scripts/download_wiki_dumps.py +131 -0
- benodb-0.17.0/scripts/git-hooks/pre-push +27 -0
- benodb-0.17.0/scripts/inject_canonical.py +46 -0
- benodb-0.17.0/scripts/install_hooks.sh +18 -0
- benodb-0.17.0/scripts/no_panic_baseline.txt +1 -0
- benodb-0.17.0/scripts/no_panic_check.sh +144 -0
- benodb-0.17.0/scripts/pre_release_check.sh +283 -0
- benodb-0.17.0/scripts/pre_release_soak.sh +103 -0
- benodb-0.17.0/scripts/prepare_demo.py +652 -0
- benodb-0.17.0/scripts/prepare_docs.sh +80 -0
- benodb-0.17.0/scripts/release_teardown.py +507 -0
- benodb-0.17.0/scripts/repro_hnsw_crash.py +92 -0
- benodb-0.17.0/scripts/services/README.md +76 -0
- benodb-0.17.0/scripts/services/benodb-search-runner.sh +13 -0
- benodb-0.17.0/scripts/services/benodb-search.conf +35 -0
- benodb-0.17.0/scripts/services/benodb-search.service +16 -0
- benodb-0.17.0/scripts/services/com.benodb.search.plist +21 -0
- benodb-0.17.0/scripts/services/install.sh +75 -0
- benodb-0.17.0/scripts/services/uninstall.sh +49 -0
- benodb-0.17.0/spark-benodb/README.md +144 -0
- benodb-0.17.0/spark-benodb/pom.xml +306 -0
- benodb-0.17.0/spark-benodb/src/main/resources/META-INF/services/org.apache.spark.sql.sources.DataSourceRegister +1 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBArrowIpc.scala +213 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBArrowUtils.scala +207 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBArrowWriter.scala +81 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBCatalogFunctions.scala +503 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBDataFrameOps.scala +56 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBJson.scala +133 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBQueryTable.scala +74 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBRowLevelOperation.scala +296 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBScanBuilder.scala +428 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBTable.scala +78 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBTableCatalog.scala +127 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/BenoDBWriteBuilder.scala +145 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/DefaultSource.scala +67 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/functions/BenoDBFunctions.scala +86 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/gpu/GpuContextResolver.scala +71 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/jni/BenoDBJNIBridge.scala +210 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/procedures/BenoDBTableResolver.scala +90 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/vector/SparseVector.scala +38 -0
- benodb-0.17.0/spark-benodb/src/main/scala/com/benodb/spark/vector/VectorDistance.scala +169 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/BenoDBCatalog.scala +58 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/AddIndexProcedure.scala +45 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/BuildIndexProcedure.scala +36 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/CompactTableProcedure.scala +36 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/DropIndexProcedure.scala +42 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/DropPrimaryKeyProcedure.scala +39 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/RebuildClusterIndexProcedure.scala +47 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/RebuildIndexProcedure.scala +40 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/RegionalDriftProcedure.scala +50 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/SetPrimaryKeyProcedure.scala +39 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/ShowIndexesProcedure.scala +78 -0
- benodb-0.17.0/spark-benodb/src/main/spark-3/com/benodb/spark/procedures/WarmupProcedure.scala +38 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/BenoDBCatalog.scala +80 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/AddIndexProcedure.scala +42 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/BenoDBProcedureScan.scala +25 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/BuildIndexProcedure.scala +31 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/CompactTableProcedure.scala +30 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/DropIndexProcedure.scala +36 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/DropPrimaryKeyProcedure.scala +31 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/RebuildClusterIndexProcedure.scala +42 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/RebuildIndexProcedure.scala +34 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/RegionalDriftProcedure.scala +68 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/SetPrimaryKeyProcedure.scala +31 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/ShowIndexesProcedure.scala +87 -0
- benodb-0.17.0/spark-benodb/src/main/spark-4/com/benodb/spark/procedures/WarmupProcedure.scala +33 -0
- benodb-0.17.0/spark-benodb/src/test/java/com/benodb/spark/BenoDBJavaSparkTest.java +112 -0
- benodb-0.17.0/spark-benodb/src/test/scala/com/benodb/spark/BenoDBSparkWarehouseTest.scala +325 -0
- benodb-0.17.0/spark-benodb/src/test/scala/com/benodb/spark/BenoDBSqlTopNTest.scala +61 -0
- benodb-0.17.0/spark-benodb/src/test/scala/com/benodb/spark/BenoDBTableResolverTest.scala +71 -0
- benodb-0.17.0/spark-benodb/src/test/scala/com/benodb/spark/GpuContextResolverTest.scala +28 -0
- benodb-0.17.0/spark-benodb/src/test/scala/com/benodb/spark/SparkProcedureCatalogTest.scala +88 -0
- benodb-0.17.0/spark-benodb/src/test/scala/com/benodb/spark/VectorSearchIntegrationTest.scala +105 -0
- benodb-0.17.0/src/bin/bdb-admin.rs +113 -0
- benodb-0.17.0/src/bin/bdb.rs +367 -0
- benodb-0.17.0/src/bin/bench_concurrency_correctness.rs +643 -0
- benodb-0.17.0/src/bin/bench_recovery.rs +179 -0
- benodb-0.17.0/src/bin/bench_throttling.rs +601 -0
- benodb-0.17.0/src/bin/check_table_paths.rs +41 -0
- benodb-0.17.0/src/bin/fix_table_paths.rs +117 -0
- benodb-0.17.0/src/bin/gateway.rs +259 -0
- benodb-0.17.0/src/bin/iceberg_rest.rs +1141 -0
- benodb-0.17.0/src/bin/ingest_wikipedia.rs +320 -0
- benodb-0.17.0/src/bin/reindex_inverted.rs +53 -0
- benodb-0.17.0/src/bin/relocate_table.rs +307 -0
- benodb-0.17.0/src/bin/setup_test_data.rs +39 -0
- benodb-0.17.0/src/bin/verify_bm25.rs +40 -0
- benodb-0.17.0/src/bin/verify_layered_indexing.rs +254 -0
- benodb-0.17.0/src/core/algorithms/communities.rs +308 -0
- benodb-0.17.0/src/core/algorithms/connected_components.rs +288 -0
- benodb-0.17.0/src/core/algorithms/frontier.rs +172 -0
- benodb-0.17.0/src/core/algorithms/label_propagation.rs +128 -0
- benodb-0.17.0/src/core/algorithms/mod.rs +5 -0
- benodb-0.17.0/src/core/algorithms/topological_sort.rs +97 -0
- benodb-0.17.0/src/core/auth.rs +359 -0
- benodb-0.17.0/src/core/cache.rs +1328 -0
- benodb-0.17.0/src/core/catalog/common.rs +584 -0
- benodb-0.17.0/src/core/catalog/config.rs +54 -0
- benodb-0.17.0/src/core/catalog/glue.rs +562 -0
- benodb-0.17.0/src/core/catalog/hive.rs +507 -0
- benodb-0.17.0/src/core/catalog/jdbc.rs +635 -0
- benodb-0.17.0/src/core/catalog/memory.rs +282 -0
- benodb-0.17.0/src/core/catalog/mod.rs +583 -0
- benodb-0.17.0/src/core/catalog/nessie.rs +637 -0
- benodb-0.17.0/src/core/catalog/policy.rs +1075 -0
- benodb-0.17.0/src/core/catalog/rest.rs +1076 -0
- benodb-0.17.0/src/core/catalog/unity.rs +637 -0
- benodb-0.17.0/src/core/clustering.rs +209 -0
- benodb-0.17.0/src/core/compaction.rs +948 -0
- benodb-0.17.0/src/core/config.rs +7 -0
- benodb-0.17.0/src/core/embeddings.rs +179 -0
- benodb-0.17.0/src/core/error.rs +400 -0
- benodb-0.17.0/src/core/fault_injection.rs +232 -0
- benodb-0.17.0/src/core/ffi.rs +3064 -0
- benodb-0.17.0/src/core/iceberg/delete.rs +876 -0
- benodb-0.17.0/src/core/iceberg/manifest.rs +530 -0
- benodb-0.17.0/src/core/iceberg/mod.rs +67 -0
- benodb-0.17.0/src/core/iceberg/schema.rs +244 -0
- benodb-0.17.0/src/core/iceberg/transform.rs +446 -0
- benodb-0.17.0/src/core/iceberg/types.rs +100 -0
- benodb-0.17.0/src/core/iceberg/value.rs +233 -0
- benodb-0.17.0/src/core/iceberg/writer.rs +1443 -0
- benodb-0.17.0/src/core/index/analyzer.rs +82 -0
- benodb-0.17.0/src/core/index/benoquant.rs +2154 -0
- benodb-0.17.0/src/core/index/bm25.rs +142 -0
- benodb-0.17.0/src/core/index/build_graph.rs +130 -0
- benodb-0.17.0/src/core/index/build_inverted.rs +812 -0
- benodb-0.17.0/src/core/index/build_vector.rs +157 -0
- benodb-0.17.0/src/core/index/cluster_index.rs +5246 -0
- benodb-0.17.0/src/core/index/csr_graph.rs +338 -0
- benodb-0.17.0/src/core/index/cuda/batch_distance.cu +240 -0
- benodb-0.17.0/src/core/index/cuda/cosine_distance.cu +61 -0
- benodb-0.17.0/src/core/index/cuda/hamming_distance.cu +43 -0
- benodb-0.17.0/src/core/index/cuda/hamming_packed.cu +28 -0
- benodb-0.17.0/src/core/index/cuda/inner_product.cu +43 -0
- benodb-0.17.0/src/core/index/cuda/jaccard_distance.cu +64 -0
- benodb-0.17.0/src/core/index/cuda/jaccard_packed.cu +41 -0
- benodb-0.17.0/src/core/index/cuda/kmeans_assignment.cu +33 -0
- benodb-0.17.0/src/core/index/cuda/l1_distance.cu +40 -0
- benodb-0.17.0/src/core/index/cuda/l2_distance.cu +40 -0
- benodb-0.17.0/src/core/index/distance.rs +982 -0
- benodb-0.17.0/src/core/index/e_means.rs +188 -0
- benodb-0.17.0/src/core/index/global_centroids.rs +182 -0
- benodb-0.17.0/src/core/index/gpu.rs +2296 -0
- benodb-0.17.0/src/core/index/hnsw_ivf.rs +3834 -0
- benodb-0.17.0/src/core/index/hnsw_rs/LICENSE-APACHE +201 -0
- benodb-0.17.0/src/core/index/hnsw_rs/LICENSE-MIT +21 -0
- benodb-0.17.0/src/core/index/hnsw_rs/annhdf5.rs +218 -0
- benodb-0.17.0/src/core/index/hnsw_rs/api.rs +86 -0
- benodb-0.17.0/src/core/index/hnsw_rs/arrow_hnsw.rs +462 -0
- benodb-0.17.0/src/core/index/hnsw_rs/arrow_ipc.rs +311 -0
- benodb-0.17.0/src/core/index/hnsw_rs/dist.rs +443 -0
- benodb-0.17.0/src/core/index/hnsw_rs/flatten.rs +129 -0
- benodb-0.17.0/src/core/index/hnsw_rs/hnsw.rs +1953 -0
- benodb-0.17.0/src/core/index/hnsw_rs/hnswio.rs +860 -0
- benodb-0.17.0/src/core/index/hnsw_rs/mod.rs +54 -0
- benodb-0.17.0/src/core/index/hnsw_rs/prelude.rs +5 -0
- benodb-0.17.0/src/core/index/hnsw_rs/test.rs +292 -0
- benodb-0.17.0/src/core/index/ivf.rs +671 -0
- benodb-0.17.0/src/core/index/json_path.rs +482 -0
- benodb-0.17.0/src/core/index/memory.rs +511 -0
- benodb-0.17.0/src/core/index/mod.rs +288 -0
- benodb-0.17.0/src/core/index/mps/cosine_distance.metal +41 -0
- benodb-0.17.0/src/core/index/mps/hamming_distance.metal +31 -0
- benodb-0.17.0/src/core/index/mps/hamming_packed.metal +26 -0
- benodb-0.17.0/src/core/index/mps/inner_product.metal +31 -0
- benodb-0.17.0/src/core/index/mps/jaccard_distance.metal +43 -0
- benodb-0.17.0/src/core/index/mps/jaccard_packed.metal +30 -0
- benodb-0.17.0/src/core/index/mps/kmeans_assignment.metal +41 -0
- benodb-0.17.0/src/core/index/mps/l1_distance.metal +29 -0
- benodb-0.17.0/src/core/index/mps/l2_distance.metal +30 -0
- benodb-0.17.0/src/core/index/nvrtc.rs +641 -0
- benodb-0.17.0/src/core/index/pq.rs +864 -0
- benodb-0.17.0/src/core/index/sq.rs +993 -0
- benodb-0.17.0/src/core/index/tokenizer.rs +96 -0
- benodb-0.17.0/src/core/index/turboquant.rs +6 -0
- benodb-0.17.0/src/core/index/vector_router.rs +898 -0
- benodb-0.17.0/src/core/index/wgpu_binary_kernel.wgsl +46 -0
- benodb-0.17.0/src/core/index/wgpu_kernel.wgsl +98 -0
- benodb-0.17.0/src/core/jni_util.rs +158 -0
- benodb-0.17.0/src/core/lock.rs +327 -0
- benodb-0.17.0/src/core/maintenance.rs +372 -0
- benodb-0.17.0/src/core/manifest/manager/commit.rs +1415 -0
- benodb-0.17.0/src/core/manifest/manager/load.rs +619 -0
- benodb-0.17.0/src/core/manifest/manager/partition.rs +251 -0
- benodb-0.17.0/src/core/manifest/manager/schema.rs +337 -0
- benodb-0.17.0/src/core/manifest/manager.rs +270 -0
- benodb-0.17.0/src/core/manifest/mod.rs +5 -0
- benodb-0.17.0/src/core/manifest/types.rs +1892 -0
- benodb-0.17.0/src/core/memory.rs +275 -0
- benodb-0.17.0/src/core/merge.rs +769 -0
- benodb-0.17.0/src/core/metadata.rs +617 -0
- benodb-0.17.0/src/core/mod.rs +52 -0
- benodb-0.17.0/src/core/nessie.rs +124 -0
- benodb-0.17.0/src/core/planner.rs +2445 -0
- benodb-0.17.0/src/core/puffin.rs +1212 -0
- benodb-0.17.0/src/core/query.rs +1507 -0
- benodb-0.17.0/src/core/reader/delete.rs +483 -0
- benodb-0.17.0/src/core/reader/filter.rs +1642 -0
- benodb-0.17.0/src/core/reader/mod.rs +553 -0
- benodb-0.17.0/src/core/reader/scan.rs +3405 -0
- benodb-0.17.0/src/core/reader/stats.rs +172 -0
- benodb-0.17.0/src/core/resources.rs +519 -0
- benodb-0.17.0/src/core/search/mod.rs +140 -0
- benodb-0.17.0/src/core/search/rrf.rs +73 -0
- benodb-0.17.0/src/core/segment.rs +2292 -0
- benodb-0.17.0/src/core/sql/catalog_ddl.rs +1445 -0
- benodb-0.17.0/src/core/sql/graph_udf/all_shortest_paths.rs +241 -0
- benodb-0.17.0/src/core/sql/graph_udf/betweenness_centrality.rs +229 -0
- benodb-0.17.0/src/core/sql/graph_udf/closeness_centrality.rs +170 -0
- benodb-0.17.0/src/core/sql/graph_udf/connected_components.rs +157 -0
- benodb-0.17.0/src/core/sql/graph_udf/degree_centrality.rs +164 -0
- benodb-0.17.0/src/core/sql/graph_udf/drift_search.rs +1058 -0
- benodb-0.17.0/src/core/sql/graph_udf/graph_neighbors.rs +193 -0
- benodb-0.17.0/src/core/sql/graph_udf/graph_table_functions.rs +1051 -0
- benodb-0.17.0/src/core/sql/graph_udf/graph_view.rs +1350 -0
- benodb-0.17.0/src/core/sql/graph_udf/jaccard_coefficient.rs +175 -0
- benodb-0.17.0/src/core/sql/graph_udf/label_propagation.rs +125 -0
- benodb-0.17.0/src/core/sql/graph_udf/leiden_communities.rs +380 -0
- benodb-0.17.0/src/core/sql/graph_udf/louvain_communities.rs +332 -0
- benodb-0.17.0/src/core/sql/graph_udf/mod.rs +80 -0
- benodb-0.17.0/src/core/sql/graph_udf/modularity.rs +332 -0
- benodb-0.17.0/src/core/sql/graph_udf/pagerank.rs +289 -0
- benodb-0.17.0/src/core/sql/graph_udf/personalized_pagerank.rs +461 -0
- benodb-0.17.0/src/core/sql/graph_udf/preferential_attachment.rs +201 -0
- benodb-0.17.0/src/core/sql/graph_udf/shortest_path.rs +216 -0
- benodb-0.17.0/src/core/sql/graph_udf/strongly_connected_components.rs +257 -0
- benodb-0.17.0/src/core/sql/graph_udf/subgraph.rs +562 -0
- benodb-0.17.0/src/core/sql/graph_udf/triangle_count.rs +146 -0
- benodb-0.17.0/src/core/sql/literal/binary.rs +192 -0
- benodb-0.17.0/src/core/sql/literal/dense.rs +92 -0
- benodb-0.17.0/src/core/sql/literal/mod.rs +1614 -0
- benodb-0.17.0/src/core/sql/literal/sparse.rs +110 -0
- benodb-0.17.0/src/core/sql/merge_into.rs +874 -0
- benodb-0.17.0/src/core/sql/mod.rs +1152 -0
- benodb-0.17.0/src/core/sql/optimizer/config.rs +340 -0
- benodb-0.17.0/src/core/sql/optimizer/index_join.rs +90 -0
- benodb-0.17.0/src/core/sql/optimizer/vector_search/mod.rs +84 -0
- benodb-0.17.0/src/core/sql/optimizer/vector_search/plan_detection.rs +109 -0
- benodb-0.17.0/src/core/sql/optimizer/vector_search/plan_rewriter.rs +118 -0
- benodb-0.17.0/src/core/sql/optimizer/vector_search/sort_expr_parser.rs +357 -0
- benodb-0.17.0/src/core/sql/optimizer.rs +14 -0
- benodb-0.17.0/src/core/sql/partition_rewriter.rs +38 -0
- benodb-0.17.0/src/core/sql/pgvector_rewriter.rs +258 -0
- benodb-0.17.0/src/core/sql/physical_plan/index_join.rs +385 -0
- benodb-0.17.0/src/core/sql/physical_plan/vector_merge.rs +230 -0
- benodb-0.17.0/src/core/sql/physical_plan/vector_scan.rs +342 -0
- benodb-0.17.0/src/core/sql/physical_plan.rs +346 -0
- benodb-0.17.0/src/core/sql/session.rs +496 -0
- benodb-0.17.0/src/core/sql/subscribe_table_function.rs +222 -0
- benodb-0.17.0/src/core/sql/udf/aggregate.rs +501 -0
- benodb-0.17.0/src/core/sql/udf/distance.rs +821 -0
- benodb-0.17.0/src/core/sql/udf/json.rs +669 -0
- benodb-0.17.0/src/core/sql/udf/mod.rs +184 -0
- benodb-0.17.0/src/core/sql/udf/sparse.rs +399 -0
- benodb-0.17.0/src/core/sql/udf/text.rs +216 -0
- benodb-0.17.0/src/core/sql/udf/transform.rs +765 -0
- benodb-0.17.0/src/core/sql/udf/vector_stats.rs +646 -0
- benodb-0.17.0/src/core/sql/vector_literal.rs +5 -0
- benodb-0.17.0/src/core/sql/vector_operators.rs +452 -0
- benodb-0.17.0/src/core/sql/vector_udf/aggregate.rs +356 -0
- benodb-0.17.0/src/core/sql/vector_udf/distance.rs +522 -0
- benodb-0.17.0/src/core/sql/vector_udf/mod.rs +6 -0
- benodb-0.17.0/src/core/sql/vector_udf/sparse.rs +242 -0
- benodb-0.17.0/src/core/sql/vector_udf/transform.rs +515 -0
- benodb-0.17.0/src/core/sql/vector_udf.rs +104 -0
- benodb-0.17.0/src/core/storage.rs +593 -0
- benodb-0.17.0/src/core/table/builder.rs +585 -0
- benodb-0.17.0/src/core/table/catalog.rs +605 -0
- benodb-0.17.0/src/core/table/coordinator.rs +336 -0
- benodb-0.17.0/src/core/table/detached.rs +745 -0
- benodb-0.17.0/src/core/table/fluent.rs +96 -0
- benodb-0.17.0/src/core/table/graph.rs +1549 -0
- benodb-0.17.0/src/core/table/graph_metadata.rs +337 -0
- benodb-0.17.0/src/core/table/index_config.rs +1379 -0
- benodb-0.17.0/src/core/table/ingest.rs +902 -0
- benodb-0.17.0/src/core/table/maintenance.rs +1752 -0
- benodb-0.17.0/src/core/table/merge.rs +169 -0
- benodb-0.17.0/src/core/table/mod.rs +945 -0
- benodb-0.17.0/src/core/table/preload.rs +1038 -0
- benodb-0.17.0/src/core/table/primary_key.rs +876 -0
- benodb-0.17.0/src/core/table/properties.rs +592 -0
- benodb-0.17.0/src/core/table/read.rs +2323 -0
- benodb-0.17.0/src/core/table/schema.rs +369 -0
- benodb-0.17.0/src/core/table/stage.rs +552 -0
- benodb-0.17.0/src/core/table/state.rs +289 -0
- benodb-0.17.0/src/core/table/stats.rs +1166 -0
- benodb-0.17.0/src/core/table/subscribe.rs +283 -0
- benodb-0.17.0/src/core/table/tests.rs +548 -0
- benodb-0.17.0/src/core/table/write.rs +2045 -0
- benodb-0.17.0/src/core/telemetry.rs +122 -0
- benodb-0.17.0/src/core/wal.rs +1815 -0
- benodb-0.17.0/src/lib.rs +450 -0
- benodb-0.17.0/src/python/catalog/glue.rs +155 -0
- benodb-0.17.0/src/python/catalog/hive.rs +100 -0
- benodb-0.17.0/src/python/catalog/jdbc.rs +110 -0
- benodb-0.17.0/src/python/catalog/memory.rs +104 -0
- benodb-0.17.0/src/python/catalog/mod.rs +241 -0
- benodb-0.17.0/src/python/catalog/nessie.rs +108 -0
- benodb-0.17.0/src/python/catalog/rest.rs +182 -0
- benodb-0.17.0/src/python/catalog/s3tables.rs +152 -0
- benodb-0.17.0/src/python/catalog/unity.rs +120 -0
- benodb-0.17.0/src/python/drift_bindings.rs +56 -0
- benodb-0.17.0/src/python/graph.rs +276 -0
- benodb-0.17.0/src/python/helpers.rs +875 -0
- benodb-0.17.0/src/python/manifest.rs +62 -0
- benodb-0.17.0/src/python/mod.rs +82 -0
- benodb-0.17.0/src/python/schema.rs +226 -0
- benodb-0.17.0/src/python/session.rs +130 -0
- benodb-0.17.0/src/python/stats.rs +103 -0
- benodb-0.17.0/src/python/subscribe.rs +119 -0
- benodb-0.17.0/src/python/table.rs +3735 -0
- benodb-0.17.0/src/python_distance.rs +1684 -0
- benodb-0.17.0/src/python_gpu_context.rs +215 -0
- benodb-0.17.0/src/telemetry/metrics.rs +345 -0
- benodb-0.17.0/src/telemetry/mod.rs +5 -0
- benodb-0.17.0/src/telemetry/otlp.rs +310 -0
- benodb-0.17.0/src/telemetry/tracing.rs +123 -0
- benodb-0.17.0/tests/all_types_index_test.rs +157 -0
- benodb-0.17.0/tests/ann_latency_probe.rs +128 -0
- benodb-0.17.0/tests/bin/generate_iceberg_manifests.rs +322 -0
- benodb-0.17.0/tests/bin/verify_iceberg_read_check.rs +81 -0
- benodb-0.17.0/tests/data/start_nessie.sh +7 -0
- benodb-0.17.0/tests/datafusion_rust_test.rs +112 -0
- benodb-0.17.0/tests/gpu_test_helpers.rs +16 -0
- benodb-0.17.0/tests/integration_test_csr_graph.rs +101 -0
- benodb-0.17.0/tests/integration_test_graph_mode.rs +272 -0
- benodb-0.17.0/tests/integration_test_hnsw_ivf_native.rs +389 -0
- benodb-0.17.0/tests/integration_test_hybrid_search_batches.rs +218 -0
- benodb-0.17.0/tests/integration_test_iceberg_zero_copy.rs +701 -0
- benodb-0.17.0/tests/integration_test_scan_splits.rs +97 -0
- benodb-0.17.0/tests/integration_test_sql_multi.rs +100 -0
- benodb-0.17.0/tests/oracle_fixture.rs +82 -0
- benodb-0.17.0/tests/python/verify_docstrings.py +109 -0
- benodb-0.17.0/tests/python/verify_fluent_api.py +167 -0
- benodb-0.17.0/tests/python/verify_unified_ingest.py +78 -0
- benodb-0.17.0/tests/read_manifest.rs +161 -0
- benodb-0.17.0/tests/schema_evolution_test.rs +78 -0
- benodb-0.17.0/tests/scratch.rs +22 -0
- benodb-0.17.0/tests/soak.rs +358 -0
- benodb-0.17.0/tests/stability.rs +70 -0
- benodb-0.17.0/tests/stress/README.md +44 -0
- benodb-0.17.0/tests/stress/conftest.py +92 -0
- benodb-0.17.0/tests/verify_admin_cli.sh +20 -0
- benodb-0.17.0/tests/verify_all_algos.py +84 -0
- benodb-0.17.0/tests/verify_catalog_commit.rs +84 -0
- benodb-0.17.0/tests/verify_compliance.rs +23 -0
- benodb-0.17.0/tests/verify_delete_correctness.rs +184 -0
- benodb-0.17.0/tests/verify_iceberg_python_delete.sh +90 -0
- benodb-0.17.0/tests/verify_iceberg_rest.sh +39 -0
- benodb-0.17.0/tests/verify_iceberg_rest_create.sh +52 -0
- benodb-0.17.0/tests/verify_iceberg_rest_delete.sh +108 -0
- benodb-0.17.0/tests/verify_iceberg_rest_remove_index.sh +135 -0
- benodb-0.17.0/tests/verify_iceberg_rest_update.sh +66 -0
- benodb-0.17.0/tests/verify_index_recovery.rs +101 -0
- benodb-0.17.0/tests/verify_metadata_creation.rs +41 -0
- benodb-0.17.0/tests/verify_mor_reads.rs +132 -0
- benodb-0.17.0/tests/verify_mor_writes.rs +91 -0
- benodb-0.17.0/tests/verify_partition_transforms.rs +81 -0
- benodb-0.17.0/tests/verify_partitioned_writes.rs +105 -0
- benodb-0.17.0/tests/verify_puffin_index.sh +85 -0
- benodb-0.17.0/tests/verify_rest_updates.sh +47 -0
- benodb-0.17.0/tests/verify_schema_compat.rs +415 -0
- benodb-0.17.0/trino-benodb/README.md +204 -0
- benodb-0.17.0/trino-benodb/etc/catalog/benodb.properties +9 -0
- benodb-0.17.0/trino-benodb/etc/catalog/glue_catalog.properties +13 -0
- benodb-0.17.0/trino-benodb/etc/catalog/iceberg.properties +15 -0
- benodb-0.17.0/trino-benodb/etc/catalog/iceberg_hive.properties +11 -0
- benodb-0.17.0/trino-benodb/etc/catalog/memory.properties +1 -0
- benodb-0.17.0/trino-benodb/etc/catalog/postgres.properties +4 -0
- benodb-0.17.0/trino-benodb/etc/config.properties +9 -0
- benodb-0.17.0/trino-benodb/etc/entrypoint.sh +45 -0
- benodb-0.17.0/trino-benodb/etc/jvm.config +11 -0
- benodb-0.17.0/trino-benodb/etc/node.properties +3 -0
- benodb-0.17.0/trino-benodb/pom.xml +264 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/AbstractBenoDBMetadata.java +672 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/AbstractBenoDBPageSinkProvider.java +62 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBArrowConverter.java +222 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBColumnHandle.java +85 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBConnectorFactory.java +211 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBInsertTableHandle.java +62 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBJNIBridge.java +149 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBMergeSink.java +186 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBMergeTableHandle.java +63 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBOutputTableHandle.java +62 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBPageSink.java +84 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBPageSource.java +228 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBPageSourceProvider.java +35 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBPlugin.java +24 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBProcedures.java +177 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBScalarFunctions.java +329 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBSplit.java +121 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBSplitManager.java +134 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBTableHandle.java +114 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/BenoDBTableUri.java +82 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/GraphRagSearchFunction.java +238 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/GraphRagSql.java +79 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/GraphSql.java +168 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/GraphTableFunction.java +159 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/GraphTableFunctionHandle.java +66 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/HybridSearchFunction.java +211 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/HybridSearchHandle.java +118 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/KeywordSearchFunction.java +185 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/KeywordSearchHandle.java +94 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/KeywordSearchSql.java +77 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/ListIndexesFunction.java +112 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/ListIndexesPageSource.java +113 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/ListIndexesSplit.java +45 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/ListIndexesTableHandle.java +62 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/RegionalDriftFunction.java +301 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/RegionalDriftPageSource.java +130 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/VectorSearchFunction.java +186 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/VectorSearchHandle.java +98 -0
- benodb-0.17.0/trino-benodb/src/main/java/com/benodb/trino/VectorSearchSql.java +142 -0
- benodb-0.17.0/trino-benodb/src/main/java-trino-d/com/benodb/trino/BenoDBMetadata.java +40 -0
- benodb-0.17.0/trino-benodb/src/main/java-trino-d/com/benodb/trino/BenoDBPageSinkProvider.java +34 -0
- benodb-0.17.0/trino-benodb/src/main/java-trino-e/com/benodb/trino/BenoDBMetadata.java +42 -0
- benodb-0.17.0/trino-benodb/src/main/java-trino-e/com/benodb/trino/BenoDBPageSinkProvider.java +36 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/KeywordSearchSqlTest.java +87 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/TestBenoDBQueries.java +552 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/TestBenoDBScalarFunctions.java +80 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/TestCrossCatalogJoin.java +124 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/TestFileGranularitySplits.java +91 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/TestGraphSqlAndTableUri.java +52 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/TrinoConnectorTest.java +374 -0
- benodb-0.17.0/trino-benodb/src/test/java/com/benodb/trino/VectorSearchSqlTest.java +126 -0
- benodb-0.17.0/wix/main.wxs +399 -0
|
@@ -0,0 +1,51 @@
|
|
|
1
|
+
# BenoStreamDB Environment Configuration
|
|
2
|
+
# Copy this file to .env and fill in your credentials.
|
|
3
|
+
# DO NOT commit .env to version control.
|
|
4
|
+
|
|
5
|
+
# -------------------------------------------------------
|
|
6
|
+
# RustFS (S3-compatible object storage)
|
|
7
|
+
# -------------------------------------------------------
|
|
8
|
+
RUSTFS_ROOT_USER=rustfsadmin
|
|
9
|
+
RUSTFS_ROOT_PASSWORD=rustfsadmin
|
|
10
|
+
|
|
11
|
+
# -------------------------------------------------------
|
|
12
|
+
# Nessie PostgreSQL backend
|
|
13
|
+
# -------------------------------------------------------
|
|
14
|
+
NESSIE_POSTGRES_DB=nessie
|
|
15
|
+
NESSIE_POSTGRES_USER=nessie
|
|
16
|
+
NESSIE_POSTGRES_PASSWORD=nessie
|
|
17
|
+
|
|
18
|
+
# -------------------------------------------------------
|
|
19
|
+
# S3 credentials (used by Nessie catalog and Trino)
|
|
20
|
+
# These should match the RustFS root user/password above
|
|
21
|
+
# unless you are using a separate S3-compatible service.
|
|
22
|
+
# -------------------------------------------------------
|
|
23
|
+
S3_ACCESS_KEY=rustfsadmin
|
|
24
|
+
S3_SECRET_KEY=rustfsadmin
|
|
25
|
+
S3_ENDPOINT=http://rustfs:9000
|
|
26
|
+
S3_REGION=us-east-1
|
|
27
|
+
|
|
28
|
+
# -------------------------------------------------------
|
|
29
|
+
# Trino S3 credentials (for iceberg/benostreamdb catalogs)
|
|
30
|
+
# -------------------------------------------------------
|
|
31
|
+
TRINO_S3_ACCESS_KEY=rustfsadmin
|
|
32
|
+
TRINO_S3_SECRET_KEY=rustfsadmin
|
|
33
|
+
|
|
34
|
+
# -------------------------------------------------------
|
|
35
|
+
# Trino PostgreSQL catalog credentials
|
|
36
|
+
# -------------------------------------------------------
|
|
37
|
+
TRINO_PG_CONNECTION_PASSWORD=indie
|
|
38
|
+
|
|
39
|
+
# -------------------------------------------------------
|
|
40
|
+
# External database connection (used by Trino entrypoint)
|
|
41
|
+
# -------------------------------------------------------
|
|
42
|
+
PG_CONN_RW=
|
|
43
|
+
|
|
44
|
+
# -------------------------------------------------------
|
|
45
|
+
# API keys (add as needed for your deployment)
|
|
46
|
+
# -------------------------------------------------------
|
|
47
|
+
# AWS_ACCESS_KEY_ID=
|
|
48
|
+
# AWS_SECRET_ACCESS_KEY=
|
|
49
|
+
# AZURE_STORAGE_ACCOUNT=
|
|
50
|
+
# AZURE_STORAGE_KEY=
|
|
51
|
+
# GCS_SERVICE_ACCOUNT_KEY=
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
*.so binary=true
|
benodb-0.17.0/.gitignore
ADDED
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# ── Rust ────────────────────────────────────────────────────────────────────
|
|
2
|
+
target/
|
|
3
|
+
**/*.rs.bk
|
|
4
|
+
|
|
5
|
+
# ── Python ──────────────────────────────────────────────────────────────────
|
|
6
|
+
__pycache__/
|
|
7
|
+
*.py[cod]
|
|
8
|
+
*$py.class
|
|
9
|
+
.venv/
|
|
10
|
+
.venv_iceberg/
|
|
11
|
+
.venv-cuda-shims/
|
|
12
|
+
venv/
|
|
13
|
+
venv_verify/
|
|
14
|
+
venv_demo/
|
|
15
|
+
test_venv/
|
|
16
|
+
!venv/requirements.txt
|
|
17
|
+
!venv/.gitkeep
|
|
18
|
+
*.egg-info/
|
|
19
|
+
|
|
20
|
+
# Python packaging build output (setuptools / maturin / `python -m build`)
|
|
21
|
+
dist/
|
|
22
|
+
build/
|
|
23
|
+
|
|
24
|
+
# Compiled Python extension (built by `maturin develop`)
|
|
25
|
+
python/**/*.so
|
|
26
|
+
python/**/*.pyd
|
|
27
|
+
python/**/*.dylib
|
|
28
|
+
|
|
29
|
+
# ── IDE / editor / tooling ──────────────────────────────────────────────────
|
|
30
|
+
.vscode/
|
|
31
|
+
.idea/
|
|
32
|
+
.metals/
|
|
33
|
+
.bloop/
|
|
34
|
+
.continue/
|
|
35
|
+
.qwen/
|
|
36
|
+
.agents/
|
|
37
|
+
AGENTS.md
|
|
38
|
+
.clinerules
|
|
39
|
+
|
|
40
|
+
# ── Compiled native binaries / debug symbols ────────────────────────────────
|
|
41
|
+
*.dylib
|
|
42
|
+
*.dSYM/
|
|
43
|
+
*.so
|
|
44
|
+
*.dll
|
|
45
|
+
*.exe
|
|
46
|
+
|
|
47
|
+
# ── BenoStream data artifacts ───────────────────────────────────────────────
|
|
48
|
+
*.parquet
|
|
49
|
+
*.idx
|
|
50
|
+
*.hnsw.graph
|
|
51
|
+
*.hnsw.data
|
|
52
|
+
*.del
|
|
53
|
+
_manifest/
|
|
54
|
+
_metadata/
|
|
55
|
+
test_admin_table/
|
|
56
|
+
test_explain_db/
|
|
57
|
+
|
|
58
|
+
# ── Local toolchain downloads (Maven/JDK fetched by build-connectors.sh) ────
|
|
59
|
+
.java/
|
|
60
|
+
.maven/
|
|
61
|
+
|
|
62
|
+
# ── Local vector DBs, scratch data & generated artifacts ────────────────────
|
|
63
|
+
data/
|
|
64
|
+
docker/data/
|
|
65
|
+
benchmarks/competitive/data/
|
|
66
|
+
hsdbg_bench_data/
|
|
67
|
+
benchmarks/scratch_*/
|
|
68
|
+
# Downloaded Zilliz/HippoRAG comparison corpora (large, reproducible from upstream)
|
|
69
|
+
benchmarks/musique_corpus.json
|
|
70
|
+
benchmarks/hippo_openie.json
|
|
71
|
+
benchmarks/zilliz_manifest.json
|
|
72
|
+
dist_archive/
|
|
73
|
+
dist_streaming/
|
|
74
|
+
|
|
75
|
+
# Pre-release soak gate outputs
|
|
76
|
+
soak-report.md
|
|
77
|
+
soak.log
|
|
78
|
+
|
|
79
|
+
# ── Documentation build output (Sphinx) ─────────────────────────────────────
|
|
80
|
+
docs/build/
|
|
81
|
+
|
|
82
|
+
# ── Build / test logs & scratch ─────────────────────────────────────────────
|
|
83
|
+
# Run logs are written here (see docs/BENCHMARKING.md), never committed.
|
|
84
|
+
logs/
|
|
85
|
+
*.log
|
|
86
|
+
*.pid
|
|
87
|
+
*.patch
|
|
88
|
+
build_errors*.txt
|
|
89
|
+
rust_test_output*.log
|
|
90
|
+
python_test_output*.log
|
|
91
|
+
python_feature_test_output*.log
|
|
92
|
+
test_output*.txt
|
|
93
|
+
*_output.txt
|
|
94
|
+
|
|
95
|
+
# ── Test caches ─────────────────────────────────────────────────────────────
|
|
96
|
+
.hypothesis/
|
|
97
|
+
.pytest_cache/
|
|
98
|
+
.ipynb_checkpoints/
|
|
99
|
+
|
|
100
|
+
# ── Local secrets — never commit ────────────────────────────────────────────
|
|
101
|
+
.streamlit/secrets.toml
|
|
102
|
+
.streamlit/config.toml
|
|
103
|
+
.env
|
|
104
|
+
.env.local
|
|
105
|
+
groq_api_key.txt
|
|
106
|
+
|
|
107
|
+
# ── Ad-hoc local scripts & scratch (not part of the repo) ───────────────────
|
|
108
|
+
scratch*.rs
|
|
109
|
+
debug_read*.py
|
|
110
|
+
patch_*.py
|
|
111
|
+
patch_*.sh
|
|
112
|
+
fix_*.py
|
|
113
|
+
run_*.sh
|
|
114
|
+
# Exceptions: these are benchmark fixtures, not ad-hoc scratch scripts. The bench
|
|
115
|
+
# image COPYs run_bench.sh, and the docs/quick-starts invoke the other three, so a
|
|
116
|
+
# clean clone could not build bsdb-bench-runner or follow the guide without them.
|
|
117
|
+
!benchmarks/competitors/run_bench.sh
|
|
118
|
+
!benchmarks/run_musique_docker.sh
|
|
119
|
+
!benchmarks/run_musique_entrypoint.sh
|
|
120
|
+
!benchmarks/run_neurips_docker.sh
|
|
121
|
+
bench_*.py
|
|
122
|
+
|
|
123
|
+
# ── Private / commercial planning (not part of the public repo) ─────────────
|
|
124
|
+
business_plan/
|
|
125
|
+
benolabs/
|
|
126
|
+
target-jni/
|
|
127
|
+
|
|
128
|
+
# ── Unintended large files / test artifacts ────────────────────────
|
|
129
|
+
test_db/
|
|
130
|
+
test_ingestion_db/
|
|
131
|
+
news_db/
|
|
132
|
+
scifact_news_db/
|
|
133
|
+
connector-artifacts/
|
|
134
|
+
test_c
|
|
135
|
+
test_prefetch
|
|
136
|
+
scratch_mem_test
|
|
137
|
+
|
|
138
|
+
# Blog article drafts (unpublished)
|
|
139
|
+
blog/articles/drafts/
|
|
140
|
+
# Quarantined pre-sweep benchmark artifacts (see benchmarks/run_sweep.py).
|
|
141
|
+
benchmarks/results/stale/
|
|
142
|
+
# Per-section quarantine archives of superseded competitor runs. They are the
|
|
143
|
+
# provenance trail for a report, not an input to it: the sweep reads
|
|
144
|
+
# `results/*.json` and the generator never looks inside `stale/`.
|
|
145
|
+
benchmarks/competitors/results/stale/
|
|
146
|
+
|
|
147
|
+
# Generated by scripts/prepare_docs.sh for the Sphinx build.
|
|
148
|
+
/docs/source/guides/
|
|
149
|
+
/docs/source/integrations/
|
|
150
|
+
/docs/source/tutorials/
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# BenoStreamDB Development Guidelines
|
|
2
|
+
|
|
3
|
+
**Foundation:** These guidelines are rooted in [The Pragmatic Programmer](https://pragprog.com/) principles.
|
|
4
|
+
|
|
5
|
+
## Core BenoStreamDB Principles
|
|
6
|
+
|
|
7
|
+
### Code Quality
|
|
8
|
+
- **Always attempt to fix warnings if possible** - Don't ignore compiler warnings; resolve them during implementation (Aligns with: Don't Live with Broken Windows #4)
|
|
9
|
+
- Use proper error handling instead of silencing warnings
|
|
10
|
+
- Maintain clean build output (zero clippy warnings)
|
|
11
|
+
- Type safety is paramount
|
|
12
|
+
- Refactor early and often as code is written
|
|
13
|
+
|
|
14
|
+
### Versioning & Releases
|
|
15
|
+
- Version numbers are injected from git tags during build
|
|
16
|
+
- `Cargo.toml` and `pyproject.toml` use dynamic versioning from `build.rs`
|
|
17
|
+
- Tag format: `v0.X.Y` (e.g., `v0.1.2`)
|
|
18
|
+
- All releases should be tagged and pushed with comprehensive release notes
|
|
19
|
+
|
|
20
|
+
### Licensing & Attribution
|
|
21
|
+
- Apache 2.0 / MIT dual license
|
|
22
|
+
- Properly attribute optimizations borrowed from iceberg-rust or other Apache-licensed projects
|
|
23
|
+
- Maintain THIRDPARTY_NOTICES.md with all third-party code usage
|
|
24
|
+
- Include inline code comments for borrowed patterns
|
|
25
|
+
|
|
26
|
+
### API Design
|
|
27
|
+
- Use native Rust names (PascalCase for types: `Cosine`, `L2`, `InnerProduct`)
|
|
28
|
+
- Python API uses lowercase for user-facing functions: `metric="cosine"`
|
|
29
|
+
- Support both primary parameter names and backward-compatible aliases
|
|
30
|
+
- Document all vector distance functions and index parameters
|
|
31
|
+
|
|
32
|
+
### Testing & Validation
|
|
33
|
+
- Unit tests required for new optimizer rules and query optimizations (Aligns with: Design to Test #20, Test Your Software #21)
|
|
34
|
+
- Explain() output should show correct index usage and query strategy
|
|
35
|
+
- Test suite must pass before any release tagging (Aligns with: Make Quality a Requirements Issue #7)
|
|
36
|
+
- Performance benchmarks are part of testing, not optional extras
|
|
37
|
+
- Integration tests for query planning changes
|
|
38
|
+
- Run full test suite before tagging releases
|
|
39
|
+
- Verify explain() output shows correct index detection
|
|
40
|
+
|
|
41
|
+
### Packaging
|
|
42
|
+
- Exclude build artifacts, virtual environments from Cargo.toml (Aligns with: Make It Easy to Reuse #11)
|
|
43
|
+
- Ensure PyPI builds succeed without external dependencies
|
|
44
|
+
- Test `maturin develop --release` before release tagging
|
|
45
|
+
- Dynamic versioning from git tags (build.rs injection)
|
|
46
|
+
|
|
47
|
+
## Current Focus Areas
|
|
48
|
+
|
|
49
|
+
### Hybrid Query Optimization
|
|
50
|
+
- Support multiple vector columns with ordering
|
|
51
|
+
- Compound primary key detection for automatic hybrid search
|
|
52
|
+
- Vector search always runs after scalar filters (push-down)
|
|
53
|
+
- Proper explain() output showing both scalar and vector execution
|
|
54
|
+
- Use tracer bullets: Implement end-to-end before perfecting details (Aligns with: Use Tracer Bullets #14)
|
|
55
|
+
|
|
56
|
+
### Iceberg-Rust Optimizations
|
|
57
|
+
- LIMIT pushdown to file scanning layer
|
|
58
|
+
- Row group skipping based on column statistics
|
|
59
|
+
- Manifest metadata caching for performance
|
|
60
|
+
- Predicate pushdown improvements
|
|
61
|
+
- Properly attribute all borrowed code patterns (Aligns with: Communicate! #9, Make It Easy to Reuse #11)
|
|
62
|
+
|
|
63
|
+
### Observability & Debugging
|
|
64
|
+
- Structured tracing via `tracing` crate (no println!) (Aligns with: Program Close to Problem Domain #16)
|
|
65
|
+
- Proper log levels (debug, info, warn, error)
|
|
66
|
+
- Explain query plans for user visibility (Aligns with: Use Tracer Bullets #14)
|
|
67
|
+
- Document query strategy through explain() output
|
benodb-0.17.0/.mailmap
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
1
|
+
# Normalize every historical author/committer to the project owner.
|
|
2
|
+
Rick Albright <rla3rd@gmail.com> <rick@hyperstreamdb.com>
|
|
3
|
+
Rick Albright <rla3rd@gmail.com> <ralbright@verityplatform.com>
|
|
4
|
+
Rick Albright <rla3rd@gmail.com> <ralbright@google.com>
|
|
5
|
+
Rick Albright <rla3rd@gmail.com> <rla3rd@users.noreply.github.com>
|
|
6
|
+
Rick Albright <rla3rd@gmail.com> <rla3rd@gmail.com>
|
benodb-0.17.0/AGENTS.md
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# BenoStreamDB Agent Steering & Release Guidelines
|
|
2
|
+
|
|
3
|
+
All AI agents, pairing assistants, and contributors operating in this repository must strictly adhere to the following release and verification engineering rules.
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 1. Mandatory Pre-Release Steering Gate
|
|
8
|
+
|
|
9
|
+
**NEVER tag, release, or push commits to GitHub without executing the local pre-release suite:**
|
|
10
|
+
|
|
11
|
+
```bash
|
|
12
|
+
./scripts/pre_release_check.sh
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
This script mirrors all required gates from `.github/workflows/ci.yml` and `.github/workflows/release.yml`:
|
|
16
|
+
1. `cargo fmt --check` (Rust code style & formatting)
|
|
17
|
+
2. `cargo clippy --all-targets` (Clippy zero-warning check under Rust 1.98.0)
|
|
18
|
+
3. `cargo check --all-targets` & `cargo check --bins` (Compilation of library, tests, and binaries)
|
|
19
|
+
4. `cargo doc --no-deps --features java` (Documentation link integrity)
|
|
20
|
+
5. `version-check` (Single-source version assertion across all connectors, POMs, and pyproject files)
|
|
21
|
+
6. Python regression tests (`tests/python/test_vector_string_conformance.py`)
|
|
22
|
+
|
|
23
|
+
If any check fails, **stop immediately and fix the underlying issue locally**. Never push code expecting CI to catch it.
|
|
24
|
+
|
|
25
|
+
---
|
|
26
|
+
|
|
27
|
+
## 2. Git Pre-Push Hook
|
|
28
|
+
|
|
29
|
+
The repository provides a tracked git pre-push hook in `scripts/git-hooks/pre-push`.
|
|
30
|
+
Install it in any local checkout:
|
|
31
|
+
|
|
32
|
+
```bash
|
|
33
|
+
./scripts/install_hooks.sh
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
The hook automatically intercepts `git push` (for both branches and tags) and blocks the push if `./scripts/pre_release_check.sh` fails. **Never bypass this hook using `--no-verify`.**
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## 3. Rust Formatting & Lints
|
|
41
|
+
|
|
42
|
+
- Always run `cargo fmt` after editing any Rust file in `src/`, `server/`, or `contrib/`.
|
|
43
|
+
- Ensure Rust code compiles cleanly with Clippy on toolchain `1.98.0` (`rustup run 1.98.0 cargo clippy --all-targets`).
|
|
44
|
+
|
|
45
|
+
---
|
|
46
|
+
|
|
47
|
+
## 4. Single Source of Truth for Versions
|
|
48
|
+
|
|
49
|
+
The single authoritative version is defined in:
|
|
50
|
+
```toml
|
|
51
|
+
# Cargo.toml
|
|
52
|
+
[workspace.package]
|
|
53
|
+
version = "X.Y.Z"
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
When bumping versions for any release:
|
|
57
|
+
1. Update `[workspace.package].version` in `Cargo.toml`.
|
|
58
|
+
2. Update inter-crate dependency pins in:
|
|
59
|
+
- `server/flight_sql/Cargo.toml` (`benodb = { version = "X.Y.Z", ... }`)
|
|
60
|
+
- `contrib/benodb-search/Cargo.toml` (`benodb = { version = "X.Y.Z", ... }`)
|
|
61
|
+
3. Update connector metadata:
|
|
62
|
+
- `spark-benodb/pom.xml` (`<revision>X.Y.Z</revision>`)
|
|
63
|
+
- `trino-benodb/pom.xml` (`<revision>X.Y.Z</revision>`)
|
|
64
|
+
- `dbt-benodb/pyproject.toml` (`benodb>=X.Y.Z`)
|
|
65
|
+
- `dbt-benodb/dbt/include/benodb/dbt_project.yml` (`version: X.Y.Z`)
|
|
66
|
+
4. Verify dynamic versioning in all Python packages (`pyproject.toml`, `server/flight_sql/pyproject.toml`, `contrib/benodb-search/pyproject.toml`, `contrib/benodb-mcp/pyproject.toml`).
|
|
67
|
+
5. Run `./scripts/pre_release_check.sh` to assert full synchronization.
|
|
68
|
+
|
|
69
|
+
---
|
|
70
|
+
|
|
71
|
+
## 5. In-Process SQL & Registration
|
|
72
|
+
|
|
73
|
+
- `Table.sql(query)` binds the table directly to the default alias `'t'` (`SELECT ... FROM t`).
|
|
74
|
+
- To query tables by named identifiers (`SELECT ... FROM events`), register the table in an in-process `Session`:
|
|
75
|
+
```python
|
|
76
|
+
session = bdb.Session()
|
|
77
|
+
session.register("events", table)
|
|
78
|
+
df = session.sql("SELECT * FROM events ...")
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
---
|
|
82
|
+
|
|
83
|
+
## 6. Authorial Voice for Documentation, Papers, and Release Collateral
|
|
84
|
+
|
|
85
|
+
Every word of prose committed to this repository follows the authorial voice of Richard Albright: the README, `docs/`, `CHANGELOG.md`, blog posts, benchmark reports, academic papers, and release notes. The authoritative guide is `../benolabs/.agents/skills/authorial-voice/SKILL.md` and the always-on steering rule is `../benolabs/.agents/rules/authorial_voice.md`. When those files and this summary disagree, read the guide, then ask.
|
|
86
|
+
|
|
87
|
+
### 6.1 Core Pillars
|
|
88
|
+
|
|
89
|
+
1. **Dual audience clarity.** Assume two readers at once: an engineer who will run the code, and a technical decision maker who signs the infrastructure bill. Open each major section with a plain statement of what happens and why it matters in operational terms (cost, memory ceilings, cluster count, vendor freedom), then follow immediately with the schema, the query, the benchmark controls, or the math.
|
|
90
|
+
2. **Empirical, never promotional.** Every claim carries a number, a mechanism, or a physical constraint. If a sentence could survive with the numbers deleted, it is an advertisement and does not belong.
|
|
91
|
+
3. **Systems grounded mechanics.** Name the actual components: Apache Iceberg, Apache Parquet, Puffin sidecars, Apache DataFusion, Arrow `RowSelection`, AVX2 kernels, cgroup limits. "The storage layer" is a failure to look.
|
|
92
|
+
4. **Honest about trade offs, negative results, and boundaries.** State what the system does not do, where performance inverts, and which scale is out of scope. A reader trusts a paper that concedes a loss more than one that reports only wins.
|
|
93
|
+
5. **Technical marketing woven throughout.** Never confine persuasion to the top of a document. Put it in section transitions, architecture comparisons, and benchmark callouts, and make it argue through architectural contrast rather than adjectives.
|
|
94
|
+
|
|
95
|
+
### 6.2 Punctuation and Mechanical Rules
|
|
96
|
+
|
|
97
|
+
- **No em dashes.** Do not join words with `---` in LaTeX or an em dash in prose. Split the sentence, use a colon, use parentheses, or use a contrastive connective. The construction reads as machine written.
|
|
98
|
+
- **No hyphenated compound modifiers between words.** Write `two stage`, `high dimensional`, `in process`, `single node`, `read only`, `build time`, `float point` compounds as separate words or restructure the sentence. Numeric and named compounds keep their hyphens (`32-KB`, `128-D`, `512-MiB`, `Walsh-Hadamard`, `Johnson-Lindenstrauss`, `deletion-vector-v1`, `--strict-hardware`). The tell the reader notices is not the hyphen itself, it is the pile of coined compounds standing in for a verb.
|
|
99
|
+
- **No tilde approximations.** Do not write `~GB`, `~1.3 MB`, or `$\sim 40$`. Print the figure the harness produced, state the bound, or describe the order of magnitude in words. The LaTeX tie in `512~MiB` is spacing and stays.
|
|
100
|
+
- **Define every acronym at first use.** Expand it in the abstract and again in the body before the abbreviation appears: Fast Walsh-Hadamard Transform (FWHT), maximum inner product search (MIPS), 8 bit scalar quantization (SQ8), Hierarchical Navigable Small World (HNSW), Product Quantization (PQ). A reader who meets `FWHT` cold stops reading.
|
|
101
|
+
- **One name per concept.** Do not alternate between `TQ8` and `BQ8` for the same quantizer, or between `OOM` and `out of memory` inside one section.
|
|
102
|
+
- **Active voice, finding first.** `We ingest the corpus and partition chunks by section boundaries`, not `The corpus is ingested and chunks are partitioned`. State the result, then the mechanism or measurement that supports it.
|
|
103
|
+
|
|
104
|
+
### 6.3 Forbidden Vocabulary and Replacements
|
|
105
|
+
|
|
106
|
+
| Avoid | Use |
|
|
107
|
+
|---|---|
|
|
108
|
+
| "seamless", "effortless", "magical" | Zero copy Arrow record batching; `pip install benodb` with no external service |
|
|
109
|
+
| "blazing fast", "revolutionary", "game changing" | Sub millisecond retrieval; AVX2 SIMD distance kernels at 1,700 QPS on eight cores |
|
|
110
|
+
| "massive I/O reduction" | 32 KB Parquet pages with Arrow `RowSelection` cutting rerank I/O from 512 MB to 1.3 MB |
|
|
111
|
+
| "next generation database" | Vector and graph engine over Apache Iceberg Puffin sidecars |
|
|
112
|
+
| "best in class vector search" | HNSW with BenoQuant TQ8 isotropic scalar quantization |
|
|
113
|
+
| "massive cost savings" | Eliminates the three system tax and the multi hop network round trips |
|
|
114
|
+
|
|
115
|
+
### 6.4 Scope Discipline for Release Text
|
|
116
|
+
|
|
117
|
+
State what a release does and does not contain before describing what it improves. Name the boundary explicitly, for example that the engine is single node with no distributed consensus, no cross-table multi statement transactions, and no automated failover, and that 100M vectors is the empirical single node boundary at a 4.8 times working set to RAM ratio. A preview note that hides its own scope is worse than no note.
|
benodb-0.17.0/BenoDB.png
ADDED
|
Binary file
|