dirsql 0.4.27__tar.gz → 0.4.28__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {dirsql-0.4.27 → dirsql-0.4.28}/Cargo.lock +1 -1
- {dirsql-0.4.27 → dirsql-0.4.28}/PKG-INFO +1 -1
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/define-tables.md +44 -0
- {dirsql-0.4.27/packages/rust → dirsql-0.4.28}/docs/howto/load-extension.md +5 -3
- {dirsql-0.4.27/packages/rust → dirsql-0.4.28}/docs/howto/search-by-meaning.md +6 -4
- {dirsql-0.4.27/packages/rust → dirsql-0.4.28}/docs/reference/config.md +85 -7
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/Cargo.toml +1 -1
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/define-tables.md +44 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/load-extension.md +5 -3
- {dirsql-0.4.27 → dirsql-0.4.28/packages/python}/docs/howto/search-by-meaning.md +6 -4
- {dirsql-0.4.27 → dirsql-0.4.28/packages/python}/docs/reference/config.md +85 -7
- dirsql-0.4.28/packages/python/e2e-attestations/claude-tackle-956-xzen74.json +7 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/define-tables.md +44 -0
- {dirsql-0.4.27 → dirsql-0.4.28/packages/rust}/docs/howto/load-extension.md +5 -3
- {dirsql-0.4.27/packages/python → dirsql-0.4.28/packages/rust}/docs/howto/search-by-meaning.md +6 -4
- {dirsql-0.4.27/packages/python → dirsql-0.4.28/packages/rust}/docs/reference/config.md +85 -7
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/db.rs +182 -88
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/lib.rs +178 -28
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/persist.rs +46 -8
- {dirsql-0.4.27 → dirsql-0.4.28}/Cargo.toml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/README.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/_async.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/_dirsql.pyi +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/discover_plugins/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/discover_plugins/discovered_fragments.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/discover_plugins/discovery_disabled.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/discover_plugins/fragment_path.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/discover_plugins/user_passed_config.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/discover_plugins/with_discovered_plugins.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/main.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/cli/resolve_config_extensions.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/py.typed +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/resolve_config_extensions.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/dirsql/resolve_extension.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/.claude/CLAUDE.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/.vitepress/config.ts +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/.vitepress/theme/index.ts +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/.vitepress/theme/lang.ts +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/AGENTS.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/explanation.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/getting-started.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/columns-from-paths.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/embed.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/extract-from-contents.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/parse-files-into-columns.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/persist.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/query-json.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/query-without-config.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/react-to-changes.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/skip-files.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/howto/write-a-plugin.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/index.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/package.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/plugins.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/pnpm-lock.yaml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/pnpm-workspace.yaml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/reference/cli.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/reference/columns.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/reference/hooks.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/reference/http-api.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/reference/path-tables.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/docs/reference/sdk.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/CHANGELOG.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/MIGRATIONS.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/README.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-13-drop-cwd-config-auto-detect.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-13-plugin-discovery.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-13-rename-extract-to-on-file.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-13-repeatable-config.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-13-sdk-default-baked-in-config.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-20-drop-implicit-files-table.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-21-remove-glob-captures.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-24-hookless-table-config-error.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-24-remove-stat-fact-injection.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-07-29-restore-python-310.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-02-release-profile.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-02-scan-failures.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-03-discovery-ext-resolution.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-03-manylinux-dynamic-binary.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-03-python-no-ignore.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-04-cast-lints.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-04-cli-in-process.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/2026-08-20-declared-table-name.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/changelog.d/README.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/conftest.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/.claude/CLAUDE.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/.vitepress/config.ts +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/.vitepress/theme/index.ts +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/.vitepress/theme/lang.ts +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/AGENTS.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/explanation.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/getting-started.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/columns-from-paths.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/embed.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/extract-from-contents.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/parse-files-into-columns.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/persist.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/query-json.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/query-without-config.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/react-to-changes.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/skip-files.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/howto/write-a-plugin.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/index.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/package.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/plugins.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/pnpm-lock.yaml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/pnpm-workspace.yaml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/reference/cli.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/reference/columns.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/reference/hooks.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/reference/http-api.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/reference/path-tables.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/docs/reference/sdk.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/e2e-attestations/claude-819-oneshot-timeout.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/e2e-attestations/claude-827-unquote-doubled-quotes.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/e2e-attestations/claude-github-issue-825-e4mhhp.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/e2e-attestations/claude-issue-951-0eease.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/e2e-attestations/claude-issue-962-0jjrfv.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/e2e-attestations/claude-issue-986-red-test-e7sjrb.json +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-13-drop-cwd-config-auto-detect.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-13-rename-extract-to-on-file.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-13-sdk-default-baked-in-config.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-20-drop-implicit-files-table.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-21-remove-glob-captures.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-24-hookless-table-config-error.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-24-remove-stat-fact-injection.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-07-29-restore-python-310.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-08-04-cli-in-process.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/2026-08-20-declared-table-name.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/migrations.d/README.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/src/lib.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/testing-conventions.toml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/conftest.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/e2e/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/e2e/fixtures/dirsql_plugin_fixture/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/e2e/fixtures/dirsql_plugin_fixture/dirsql.toml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/integration/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/integration/binding/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/python/tests/integration/hermetic/__init__.py +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/CHANGELOG.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/Cargo.toml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/MIGRATIONS.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/README.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/benches/db_bench.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/benches/differ_bench.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/benches/matcher_bench.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/benches/scanner_bench.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/explanation.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/getting-started.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/columns-from-paths.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/embed.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/extract-from-contents.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/parse-files-into-columns.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/persist.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/query-json.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/query-without-config.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/react-to-changes.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/skip-files.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/howto/write-a-plugin.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/index.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/plugins.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/reference/cli.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/reference/columns.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/reference/hooks.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/reference/http-api.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/reference/path-tables.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/docs/reference/sdk.md +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/bin/dirsql.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/cli/execute.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/cli/init.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/cli/mod.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/cli/router.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/cli/run.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/cli/serialize.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/cli/server.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/command.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/config.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/default_config.toml +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/differ.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/functions.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/infer.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/matcher.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/parsed_cache.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/parsed_vtab.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/path_table.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/scanner.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/sql_literal.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/vtab.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/packages/rust/src/watcher.rs +0 -0
- {dirsql-0.4.27 → dirsql-0.4.28}/pyproject.toml +0 -0
|
@@ -65,6 +65,50 @@ Add one `[[table]]` entry per table — each with its own `glob`, `ddl`, and
|
|
|
65
65
|
table — each table is an independent view. See
|
|
66
66
|
[`[[table]]`](../reference/config.md#table) for the remaining key, `strict`.
|
|
67
67
|
|
|
68
|
+
## Add indexes and full-text search
|
|
69
|
+
|
|
70
|
+
`ddl` is a SQL batch, not a single statement — SQLite runs the whole thing. So
|
|
71
|
+
a table can arrive with its own index, and with an FTS5 index kept current by
|
|
72
|
+
two triggers:
|
|
73
|
+
|
|
74
|
+
```toml
|
|
75
|
+
[[table]]
|
|
76
|
+
name = "posts"
|
|
77
|
+
glob = "posts/**/*.md"
|
|
78
|
+
on-file = "python3 extract.py {path}"
|
|
79
|
+
ddl = '''
|
|
80
|
+
CREATE TABLE posts (title TEXT, slug TEXT, body TEXT);
|
|
81
|
+
CREATE INDEX posts_slug ON posts(slug);
|
|
82
|
+
|
|
83
|
+
CREATE VIRTUAL TABLE posts_fts
|
|
84
|
+
USING fts5(body, content='posts', content_rowid='rowid');
|
|
85
|
+
CREATE TRIGGER posts_ai AFTER INSERT ON posts BEGIN
|
|
86
|
+
INSERT INTO posts_fts(rowid, body) VALUES (new.rowid, new.body);
|
|
87
|
+
END;
|
|
88
|
+
CREATE TRIGGER posts_ad AFTER DELETE ON posts BEGIN
|
|
89
|
+
INSERT INTO posts_fts(posts_fts, rowid, body)
|
|
90
|
+
VALUES ('delete', old.rowid, old.body);
|
|
91
|
+
END;
|
|
92
|
+
'''
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
dirsql query "SELECT slug FROM posts JOIN posts_fts ON posts.rowid = posts_fts.rowid
|
|
97
|
+
WHERE posts_fts MATCH 'recursion'" -c ./.dirsql.toml
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
```json
|
|
101
|
+
[{"slug":"again"}]
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
dirsql writes rows with plain `INSERT` and `DELETE`, so those triggers hold
|
|
105
|
+
through the initial load and every [watcher](./react-to-changes.md) event — you
|
|
106
|
+
maintain nothing. `name` still has to be the row table (`posts`); the virtual
|
|
107
|
+
table lives beside it under its own name. The batch runs once, when the table
|
|
108
|
+
is created, and editing any part of it rebuilds a
|
|
109
|
+
[persistent cache](./persist.md) from scratch. Full rules:
|
|
110
|
+
[Batch `ddl`](../reference/config.md#batch-ddl).
|
|
111
|
+
|
|
68
112
|
## Going further
|
|
69
113
|
|
|
70
114
|
- The parser mechanics — placeholders, stdout protocol, per-file failure
|
|
@@ -76,9 +76,11 @@ interpreter to resolve package names with
|
|
|
76
76
|
|
|
77
77
|
## Notes
|
|
78
78
|
|
|
79
|
-
- Extensions add **functions
|
|
80
|
-
|
|
81
|
-
|
|
79
|
+
- Extensions add **functions**, and virtual tables a table's `ddl` batch can
|
|
80
|
+
create. A `[[table]]`'s own `name` may not be a virtual table — that one is
|
|
81
|
+
the per-file row table `dirsql` inserts into — so declare the virtual table
|
|
82
|
+
alongside it in the same batch
|
|
83
|
+
([reference](../reference/config.md#batch-ddl)).
|
|
82
84
|
- Loading happens before any table DDL runs, and the SQL
|
|
83
85
|
`load_extension()` function is never exposed to queries
|
|
84
86
|
([reference](../reference/config.md#dirsql-extension)).
|
|
@@ -95,10 +95,12 @@ LIMIT 10
|
|
|
95
95
|
|
|
96
96
|
::: tip Top-k is `LIMIT k`
|
|
97
97
|
If you know `sqlite-vec` you may reach for its `MATCH … AND k = 10` idiom.
|
|
98
|
-
That syntax belongs to `sqlite-vec`'s `vec0` virtual table, which
|
|
99
|
-
does not use
|
|
100
|
-
expressions, `sqlite-vec`'s own documented pattern is exactly what
|
|
101
|
-
guide uses: `ORDER BY vec_distance_cosine(...) LIMIT k`.
|
|
98
|
+
That syntax belongs to `sqlite-vec`'s `vec0` virtual table, which the table
|
|
99
|
+
above does not use: a `[[table]]`'s own `name` is always a per-file row table.
|
|
100
|
+
For plain expressions, `sqlite-vec`'s own documented pattern is exactly what
|
|
101
|
+
this guide uses: `ORDER BY vec_distance_cosine(...) LIMIT k`. To get the `vec0`
|
|
102
|
+
idiom instead, declare the `vec0` table alongside the row table in the same
|
|
103
|
+
[`ddl` batch](../reference/config.md#batch-ddl) and fill it from a trigger.
|
|
102
104
|
:::
|
|
103
105
|
|
|
104
106
|
## Repeat runs are cheap
|
|
@@ -87,10 +87,11 @@ name you installed:
|
|
|
87
87
|
`path = "sqlite-vec-linux-x64"` (matching your platform), not
|
|
88
88
|
`path = "sqlite-vec"`, whose meta-package ships no loadable.
|
|
89
89
|
|
|
90
|
-
Extensions add **functions** callable in queries and in a table's DDL
|
|
91
|
-
|
|
92
|
-
`
|
|
93
|
-
|
|
90
|
+
Extensions add **functions** callable in queries and in a table's DDL, and
|
|
91
|
+
**virtual tables** a table's [`ddl` batch](#batch-ddl) can create. What a
|
|
92
|
+
`[[table]]`'s own `name` may not be is a virtual table: that one is the
|
|
93
|
+
per-file row table `dirsql` inserts into. Create the virtual table alongside
|
|
94
|
+
it, under its own name.
|
|
94
95
|
|
|
95
96
|
## `[[dirsql.function]]`
|
|
96
97
|
|
|
@@ -167,7 +168,7 @@ what its required `on-file` command emits — dirsql injects nothing (see
|
|
|
167
168
|
| Key | Required | Description |
|
|
168
169
|
|---|---|---|
|
|
169
170
|
| `name` | yes | The table's SQL name — the name you query it by. Declared, never derived from `ddl`: dirsql does not read the DDL text. The `ddl` must create a table by this name; if it doesn't, loading fails. |
|
|
170
|
-
| `ddl` | yes | A
|
|
171
|
+
| `ddl` | yes | A SQL batch, run verbatim — any number of statements. It must create a table called `name`; that table holds the file rows, and only the columns it declares are kept (keys the `on-file` command emits that are not declared are dropped). The rest of the batch is yours: indexes, virtual tables, triggers. See [Batch `ddl`](#batch-ddl). |
|
|
171
172
|
| `glob` | yes | Glob pattern matched against root-relative paths. Every table whose glob matches a file receives that file's rows — a file can populate multiple tables. A `{name}` segment is rewritten to `*` (it matches one path segment but captures nothing). |
|
|
172
173
|
| `on-file` | **yes** | A command run once per matched file; its stdout (a JSON array of row objects) becomes the file's rows. Must be non-empty. A `[[table]]` with no `on-file` is a load error (see [parse errors](#parse-errors)). See [Command hooks](./hooks.md#on-file). |
|
|
173
174
|
| `strict` | no (default `false`) | When `true`, rows whose keys do not exactly match the declared columns are rejected with an error: extra keys error, and every declared column must be supplied by the `on-file` output. When `false`, extra keys are dropped and missing columns become `NULL`. |
|
|
@@ -194,6 +195,72 @@ on-file = "uv run python extract_papers.py {path}"
|
|
|
194
195
|
strict = true
|
|
195
196
|
```
|
|
196
197
|
|
|
198
|
+
### Batch `ddl`
|
|
199
|
+
|
|
200
|
+
`ddl` is handed to SQLite whole, so a table declaration is not limited to one
|
|
201
|
+
statement:
|
|
202
|
+
|
|
203
|
+
```toml
|
|
204
|
+
[[table]]
|
|
205
|
+
name = "messages"
|
|
206
|
+
glob = "sessions/*/messages/*.json"
|
|
207
|
+
on-file = "jq -c '.' {path}"
|
|
208
|
+
ddl = '''
|
|
209
|
+
CREATE TABLE messages (session TEXT, idx INT, role TEXT, text TEXT);
|
|
210
|
+
CREATE INDEX messages_session ON messages(session);
|
|
211
|
+
|
|
212
|
+
CREATE VIRTUAL TABLE messages_fts
|
|
213
|
+
USING fts5(text, content='messages', content_rowid='rowid');
|
|
214
|
+
CREATE TRIGGER messages_ai AFTER INSERT ON messages BEGIN
|
|
215
|
+
INSERT INTO messages_fts(rowid, text) VALUES (new.rowid, new.text);
|
|
216
|
+
END;
|
|
217
|
+
CREATE TRIGGER messages_ad AFTER DELETE ON messages BEGIN
|
|
218
|
+
INSERT INTO messages_fts(messages_fts, rowid, text)
|
|
219
|
+
VALUES ('delete', old.rowid, old.text);
|
|
220
|
+
END;
|
|
221
|
+
'''
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
```bash
|
|
225
|
+
dirsql query "SELECT text FROM messages_fts WHERE messages_fts MATCH 'deploy'" \
|
|
226
|
+
-c ./.dirsql.toml
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Those two triggers are all a keyword index needs. dirsql writes file rows with
|
|
230
|
+
plain `INSERT` and `DELETE` — an update is a delete and an insert in one
|
|
231
|
+
transaction, and there is no `UPDATE` path on user rows — so triggers you
|
|
232
|
+
declare here stay current through the initial load and every
|
|
233
|
+
[watcher](../howto/react-to-changes.md) event. The same shape with a `vec0`
|
|
234
|
+
virtual table and an [`embed()`](#dirsql-function) call in the trigger gives
|
|
235
|
+
you stored vectors.
|
|
236
|
+
|
|
237
|
+
**dirsql never reads the batch.** After it runs, SQLite's own catalog
|
|
238
|
+
(`pragma_table_list`) settles what it produced:
|
|
239
|
+
|
|
240
|
+
- No table called `name` → a load error that lists what the batch *did*
|
|
241
|
+
create, so a typo is obvious.
|
|
242
|
+
- `name` is a **virtual** table → a load error. The declared table is the one
|
|
243
|
+
dirsql inserts file rows into, so it has to be a real row table. Create the
|
|
244
|
+
virtual table alongside it, under its own name.
|
|
245
|
+
- `name` is **`WITHOUT ROWID`** → a warning. Internal row bookkeeping is keyed
|
|
246
|
+
on rowid, so these will be rejected in a future release.
|
|
247
|
+
|
|
248
|
+
The whole batch runs in one transaction: if any statement fails, none of them
|
|
249
|
+
took effect, and the error is SQLite's own, prefixed with the config entry —
|
|
250
|
+
`table 'messages': SQLite error: near "(": syntax error`. Context, never
|
|
251
|
+
interpretation.
|
|
252
|
+
|
|
253
|
+
Two consequences of `ddl` running **once, when the table is created**:
|
|
254
|
+
|
|
255
|
+
- **Rows the batch inserts itself are not file-tracked.** No file owns them, so
|
|
256
|
+
they survive file deletions — and they vanish on any rebuild.
|
|
257
|
+
- **Editing `ddl` at all rebuilds a
|
|
258
|
+
[persistent cache](../howto/persist.md).** The config hash covers the entire
|
|
259
|
+
batch, so a new index, a different FTS5
|
|
260
|
+
tokenizer or a changed embedding model id forces a full sweep and re-ingest.
|
|
261
|
+
That is the only invalidation lane: dirsql tracks no ownership of what the
|
|
262
|
+
batch made.
|
|
263
|
+
|
|
197
264
|
### `on-file` row mapping
|
|
198
265
|
|
|
199
266
|
The command prints a JSON array of objects; each object becomes one row.
|
|
@@ -247,13 +314,24 @@ SDKs raise/reject) when:
|
|
|
247
314
|
- A `[[table]]` entry omits `name`, `ddl`, or `glob` (or `name` is
|
|
248
315
|
empty/whitespace).
|
|
249
316
|
- A `[[table]]` entry's `ddl` runs but creates no table by its `name`. The
|
|
250
|
-
error carries the entry's name
|
|
317
|
+
error carries the entry's name, lists what the batch did create, and points
|
|
318
|
+
at the fix:
|
|
251
319
|
|
|
252
|
-
> `table 'messages': its `ddl` ran but created no table called 'messages'. Set `name` to the table the `ddl` creates.`
|
|
320
|
+
> `table 'messages': its `ddl` ran but created no table called 'messages' (it created: mesages, mesages_fts). Set `name` to the table the `ddl` creates.`
|
|
253
321
|
|
|
254
322
|
dirsql asks SQLite's catalog rather than interpreting the DDL, so quoted
|
|
255
323
|
(`CREATE TABLE "messages"`), schema-qualified (`main.messages`) and
|
|
256
324
|
`IF NOT EXISTS` forms all match a plain `name = "messages"`.
|
|
325
|
+
- A `[[table]]` entry's `name` names a **virtual** table. The declared table
|
|
326
|
+
holds the file rows, so it must be a real row table:
|
|
327
|
+
|
|
328
|
+
> `table 'messages': its `ddl` created a virtual table called 'messages'. The declared table holds the file rows, so it must be a real row table; create the virtual table alongside it, under its own name.`
|
|
329
|
+
|
|
330
|
+
- A `[[table]]` entry's `ddl` is rejected by SQLite. Nothing the batch did
|
|
331
|
+
takes effect, and SQLite's own message is passed through under the entry's
|
|
332
|
+
name:
|
|
333
|
+
|
|
334
|
+
> `table 'messages': SQLite error: near "(": syntax error`
|
|
257
335
|
- A `[[table]]` entry omits `on-file` (or it is empty/whitespace). The error
|
|
258
336
|
names the offending glob and points at the fix:
|
|
259
337
|
|
|
@@ -4,7 +4,7 @@ name = "dirsql-py-ext"
|
|
|
4
4
|
# pypi/maturin handler can rewrite it via `write-version` before
|
|
5
5
|
# `maturin build`. `pyproject.toml` declares `dynamic = ["version"]`
|
|
6
6
|
# and maturin reads this field. Mirrors `packages/rust/Cargo.toml`.
|
|
7
|
-
version = "0.4.
|
|
7
|
+
version = "0.4.28"
|
|
8
8
|
# abi3 note: the `abi3-py310` pyo3 feature (below) builds ONE stable-ABI
|
|
9
9
|
# `cp310-abi3` wheel per platform that loads on every CPython >= 3.10
|
|
10
10
|
# (matching `pyproject.toml`'s `requires-python = ">=3.10"`), instead of
|
|
@@ -65,6 +65,50 @@ Add one `[[table]]` entry per table — each with its own `glob`, `ddl`, and
|
|
|
65
65
|
table — each table is an independent view. See
|
|
66
66
|
[`[[table]]`](../reference/config.md#table) for the remaining key, `strict`.
|
|
67
67
|
|
|
68
|
+
## Add indexes and full-text search
|
|
69
|
+
|
|
70
|
+
`ddl` is a SQL batch, not a single statement — SQLite runs the whole thing. So
|
|
71
|
+
a table can arrive with its own index, and with an FTS5 index kept current by
|
|
72
|
+
two triggers:
|
|
73
|
+
|
|
74
|
+
```toml
|
|
75
|
+
[[table]]
|
|
76
|
+
name = "posts"
|
|
77
|
+
glob = "posts/**/*.md"
|
|
78
|
+
on-file = "python3 extract.py {path}"
|
|
79
|
+
ddl = '''
|
|
80
|
+
CREATE TABLE posts (title TEXT, slug TEXT, body TEXT);
|
|
81
|
+
CREATE INDEX posts_slug ON posts(slug);
|
|
82
|
+
|
|
83
|
+
CREATE VIRTUAL TABLE posts_fts
|
|
84
|
+
USING fts5(body, content='posts', content_rowid='rowid');
|
|
85
|
+
CREATE TRIGGER posts_ai AFTER INSERT ON posts BEGIN
|
|
86
|
+
INSERT INTO posts_fts(rowid, body) VALUES (new.rowid, new.body);
|
|
87
|
+
END;
|
|
88
|
+
CREATE TRIGGER posts_ad AFTER DELETE ON posts BEGIN
|
|
89
|
+
INSERT INTO posts_fts(posts_fts, rowid, body)
|
|
90
|
+
VALUES ('delete', old.rowid, old.body);
|
|
91
|
+
END;
|
|
92
|
+
'''
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
dirsql query "SELECT slug FROM posts JOIN posts_fts ON posts.rowid = posts_fts.rowid
|
|
97
|
+
WHERE posts_fts MATCH 'recursion'" -c ./.dirsql.toml
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
```json
|
|
101
|
+
[{"slug":"again"}]
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
dirsql writes rows with plain `INSERT` and `DELETE`, so those triggers hold
|
|
105
|
+
through the initial load and every [watcher](./react-to-changes.md) event — you
|
|
106
|
+
maintain nothing. `name` still has to be the row table (`posts`); the virtual
|
|
107
|
+
table lives beside it under its own name. The batch runs once, when the table
|
|
108
|
+
is created, and editing any part of it rebuilds a
|
|
109
|
+
[persistent cache](./persist.md) from scratch. Full rules:
|
|
110
|
+
[Batch `ddl`](../reference/config.md#batch-ddl).
|
|
111
|
+
|
|
68
112
|
## Going further
|
|
69
113
|
|
|
70
114
|
- The parser mechanics — placeholders, stdout protocol, per-file failure
|
|
@@ -76,9 +76,11 @@ interpreter to resolve package names with
|
|
|
76
76
|
|
|
77
77
|
## Notes
|
|
78
78
|
|
|
79
|
-
- Extensions add **functions
|
|
80
|
-
|
|
81
|
-
|
|
79
|
+
- Extensions add **functions**, and virtual tables a table's `ddl` batch can
|
|
80
|
+
create. A `[[table]]`'s own `name` may not be a virtual table — that one is
|
|
81
|
+
the per-file row table `dirsql` inserts into — so declare the virtual table
|
|
82
|
+
alongside it in the same batch
|
|
83
|
+
([reference](../reference/config.md#batch-ddl)).
|
|
82
84
|
- Loading happens before any table DDL runs, and the SQL
|
|
83
85
|
`load_extension()` function is never exposed to queries
|
|
84
86
|
([reference](../reference/config.md#dirsql-extension)).
|
|
@@ -95,10 +95,12 @@ LIMIT 10
|
|
|
95
95
|
|
|
96
96
|
::: tip Top-k is `LIMIT k`
|
|
97
97
|
If you know `sqlite-vec` you may reach for its `MATCH … AND k = 10` idiom.
|
|
98
|
-
That syntax belongs to `sqlite-vec`'s `vec0` virtual table, which
|
|
99
|
-
does not use
|
|
100
|
-
expressions, `sqlite-vec`'s own documented pattern is exactly what
|
|
101
|
-
guide uses: `ORDER BY vec_distance_cosine(...) LIMIT k`.
|
|
98
|
+
That syntax belongs to `sqlite-vec`'s `vec0` virtual table, which the table
|
|
99
|
+
above does not use: a `[[table]]`'s own `name` is always a per-file row table.
|
|
100
|
+
For plain expressions, `sqlite-vec`'s own documented pattern is exactly what
|
|
101
|
+
this guide uses: `ORDER BY vec_distance_cosine(...) LIMIT k`. To get the `vec0`
|
|
102
|
+
idiom instead, declare the `vec0` table alongside the row table in the same
|
|
103
|
+
[`ddl` batch](../reference/config.md#batch-ddl) and fill it from a trigger.
|
|
102
104
|
:::
|
|
103
105
|
|
|
104
106
|
## Repeat runs are cheap
|
|
@@ -87,10 +87,11 @@ name you installed:
|
|
|
87
87
|
`path = "sqlite-vec-linux-x64"` (matching your platform), not
|
|
88
88
|
`path = "sqlite-vec"`, whose meta-package ships no loadable.
|
|
89
89
|
|
|
90
|
-
Extensions add **functions** callable in queries and in a table's DDL
|
|
91
|
-
|
|
92
|
-
`
|
|
93
|
-
|
|
90
|
+
Extensions add **functions** callable in queries and in a table's DDL, and
|
|
91
|
+
**virtual tables** a table's [`ddl` batch](#batch-ddl) can create. What a
|
|
92
|
+
`[[table]]`'s own `name` may not be is a virtual table: that one is the
|
|
93
|
+
per-file row table `dirsql` inserts into. Create the virtual table alongside
|
|
94
|
+
it, under its own name.
|
|
94
95
|
|
|
95
96
|
## `[[dirsql.function]]`
|
|
96
97
|
|
|
@@ -167,7 +168,7 @@ what its required `on-file` command emits — dirsql injects nothing (see
|
|
|
167
168
|
| Key | Required | Description |
|
|
168
169
|
|---|---|---|
|
|
169
170
|
| `name` | yes | The table's SQL name — the name you query it by. Declared, never derived from `ddl`: dirsql does not read the DDL text. The `ddl` must create a table by this name; if it doesn't, loading fails. |
|
|
170
|
-
| `ddl` | yes | A
|
|
171
|
+
| `ddl` | yes | A SQL batch, run verbatim — any number of statements. It must create a table called `name`; that table holds the file rows, and only the columns it declares are kept (keys the `on-file` command emits that are not declared are dropped). The rest of the batch is yours: indexes, virtual tables, triggers. See [Batch `ddl`](#batch-ddl). |
|
|
171
172
|
| `glob` | yes | Glob pattern matched against root-relative paths. Every table whose glob matches a file receives that file's rows — a file can populate multiple tables. A `{name}` segment is rewritten to `*` (it matches one path segment but captures nothing). |
|
|
172
173
|
| `on-file` | **yes** | A command run once per matched file; its stdout (a JSON array of row objects) becomes the file's rows. Must be non-empty. A `[[table]]` with no `on-file` is a load error (see [parse errors](#parse-errors)). See [Command hooks](./hooks.md#on-file). |
|
|
173
174
|
| `strict` | no (default `false`) | When `true`, rows whose keys do not exactly match the declared columns are rejected with an error: extra keys error, and every declared column must be supplied by the `on-file` output. When `false`, extra keys are dropped and missing columns become `NULL`. |
|
|
@@ -194,6 +195,72 @@ on-file = "uv run python extract_papers.py {path}"
|
|
|
194
195
|
strict = true
|
|
195
196
|
```
|
|
196
197
|
|
|
198
|
+
### Batch `ddl`
|
|
199
|
+
|
|
200
|
+
`ddl` is handed to SQLite whole, so a table declaration is not limited to one
|
|
201
|
+
statement:
|
|
202
|
+
|
|
203
|
+
```toml
|
|
204
|
+
[[table]]
|
|
205
|
+
name = "messages"
|
|
206
|
+
glob = "sessions/*/messages/*.json"
|
|
207
|
+
on-file = "jq -c '.' {path}"
|
|
208
|
+
ddl = '''
|
|
209
|
+
CREATE TABLE messages (session TEXT, idx INT, role TEXT, text TEXT);
|
|
210
|
+
CREATE INDEX messages_session ON messages(session);
|
|
211
|
+
|
|
212
|
+
CREATE VIRTUAL TABLE messages_fts
|
|
213
|
+
USING fts5(text, content='messages', content_rowid='rowid');
|
|
214
|
+
CREATE TRIGGER messages_ai AFTER INSERT ON messages BEGIN
|
|
215
|
+
INSERT INTO messages_fts(rowid, text) VALUES (new.rowid, new.text);
|
|
216
|
+
END;
|
|
217
|
+
CREATE TRIGGER messages_ad AFTER DELETE ON messages BEGIN
|
|
218
|
+
INSERT INTO messages_fts(messages_fts, rowid, text)
|
|
219
|
+
VALUES ('delete', old.rowid, old.text);
|
|
220
|
+
END;
|
|
221
|
+
'''
|
|
222
|
+
```
|
|
223
|
+
|
|
224
|
+
```bash
|
|
225
|
+
dirsql query "SELECT text FROM messages_fts WHERE messages_fts MATCH 'deploy'" \
|
|
226
|
+
-c ./.dirsql.toml
|
|
227
|
+
```
|
|
228
|
+
|
|
229
|
+
Those two triggers are all a keyword index needs. dirsql writes file rows with
|
|
230
|
+
plain `INSERT` and `DELETE` — an update is a delete and an insert in one
|
|
231
|
+
transaction, and there is no `UPDATE` path on user rows — so triggers you
|
|
232
|
+
declare here stay current through the initial load and every
|
|
233
|
+
[watcher](../howto/react-to-changes.md) event. The same shape with a `vec0`
|
|
234
|
+
virtual table and an [`embed()`](#dirsql-function) call in the trigger gives
|
|
235
|
+
you stored vectors.
|
|
236
|
+
|
|
237
|
+
**dirsql never reads the batch.** After it runs, SQLite's own catalog
|
|
238
|
+
(`pragma_table_list`) settles what it produced:
|
|
239
|
+
|
|
240
|
+
- No table called `name` → a load error that lists what the batch *did*
|
|
241
|
+
create, so a typo is obvious.
|
|
242
|
+
- `name` is a **virtual** table → a load error. The declared table is the one
|
|
243
|
+
dirsql inserts file rows into, so it has to be a real row table. Create the
|
|
244
|
+
virtual table alongside it, under its own name.
|
|
245
|
+
- `name` is **`WITHOUT ROWID`** → a warning. Internal row bookkeeping is keyed
|
|
246
|
+
on rowid, so these will be rejected in a future release.
|
|
247
|
+
|
|
248
|
+
The whole batch runs in one transaction: if any statement fails, none of them
|
|
249
|
+
took effect, and the error is SQLite's own, prefixed with the config entry —
|
|
250
|
+
`table 'messages': SQLite error: near "(": syntax error`. Context, never
|
|
251
|
+
interpretation.
|
|
252
|
+
|
|
253
|
+
Two consequences of `ddl` running **once, when the table is created**:
|
|
254
|
+
|
|
255
|
+
- **Rows the batch inserts itself are not file-tracked.** No file owns them, so
|
|
256
|
+
they survive file deletions — and they vanish on any rebuild.
|
|
257
|
+
- **Editing `ddl` at all rebuilds a
|
|
258
|
+
[persistent cache](../howto/persist.md).** The config hash covers the entire
|
|
259
|
+
batch, so a new index, a different FTS5
|
|
260
|
+
tokenizer or a changed embedding model id forces a full sweep and re-ingest.
|
|
261
|
+
That is the only invalidation lane: dirsql tracks no ownership of what the
|
|
262
|
+
batch made.
|
|
263
|
+
|
|
197
264
|
### `on-file` row mapping
|
|
198
265
|
|
|
199
266
|
The command prints a JSON array of objects; each object becomes one row.
|
|
@@ -247,13 +314,24 @@ SDKs raise/reject) when:
|
|
|
247
314
|
- A `[[table]]` entry omits `name`, `ddl`, or `glob` (or `name` is
|
|
248
315
|
empty/whitespace).
|
|
249
316
|
- A `[[table]]` entry's `ddl` runs but creates no table by its `name`. The
|
|
250
|
-
error carries the entry's name
|
|
317
|
+
error carries the entry's name, lists what the batch did create, and points
|
|
318
|
+
at the fix:
|
|
251
319
|
|
|
252
|
-
> `table 'messages': its `ddl` ran but created no table called 'messages'. Set `name` to the table the `ddl` creates.`
|
|
320
|
+
> `table 'messages': its `ddl` ran but created no table called 'messages' (it created: mesages, mesages_fts). Set `name` to the table the `ddl` creates.`
|
|
253
321
|
|
|
254
322
|
dirsql asks SQLite's catalog rather than interpreting the DDL, so quoted
|
|
255
323
|
(`CREATE TABLE "messages"`), schema-qualified (`main.messages`) and
|
|
256
324
|
`IF NOT EXISTS` forms all match a plain `name = "messages"`.
|
|
325
|
+
- A `[[table]]` entry's `name` names a **virtual** table. The declared table
|
|
326
|
+
holds the file rows, so it must be a real row table:
|
|
327
|
+
|
|
328
|
+
> `table 'messages': its `ddl` created a virtual table called 'messages'. The declared table holds the file rows, so it must be a real row table; create the virtual table alongside it, under its own name.`
|
|
329
|
+
|
|
330
|
+
- A `[[table]]` entry's `ddl` is rejected by SQLite. Nothing the batch did
|
|
331
|
+
takes effect, and SQLite's own message is passed through under the entry's
|
|
332
|
+
name:
|
|
333
|
+
|
|
334
|
+
> `table 'messages': SQLite error: near "(": syntax error`
|
|
257
335
|
- A `[[table]]` entry omits `on-file` (or it is empty/whitespace). The error
|
|
258
336
|
names the offending glob and points at the fix:
|
|
259
337
|
|
|
@@ -65,6 +65,50 @@ Add one `[[table]]` entry per table — each with its own `glob`, `ddl`, and
|
|
|
65
65
|
table — each table is an independent view. See
|
|
66
66
|
[`[[table]]`](../reference/config.md#table) for the remaining key, `strict`.
|
|
67
67
|
|
|
68
|
+
## Add indexes and full-text search
|
|
69
|
+
|
|
70
|
+
`ddl` is a SQL batch, not a single statement — SQLite runs the whole thing. So
|
|
71
|
+
a table can arrive with its own index, and with an FTS5 index kept current by
|
|
72
|
+
two triggers:
|
|
73
|
+
|
|
74
|
+
```toml
|
|
75
|
+
[[table]]
|
|
76
|
+
name = "posts"
|
|
77
|
+
glob = "posts/**/*.md"
|
|
78
|
+
on-file = "python3 extract.py {path}"
|
|
79
|
+
ddl = '''
|
|
80
|
+
CREATE TABLE posts (title TEXT, slug TEXT, body TEXT);
|
|
81
|
+
CREATE INDEX posts_slug ON posts(slug);
|
|
82
|
+
|
|
83
|
+
CREATE VIRTUAL TABLE posts_fts
|
|
84
|
+
USING fts5(body, content='posts', content_rowid='rowid');
|
|
85
|
+
CREATE TRIGGER posts_ai AFTER INSERT ON posts BEGIN
|
|
86
|
+
INSERT INTO posts_fts(rowid, body) VALUES (new.rowid, new.body);
|
|
87
|
+
END;
|
|
88
|
+
CREATE TRIGGER posts_ad AFTER DELETE ON posts BEGIN
|
|
89
|
+
INSERT INTO posts_fts(posts_fts, rowid, body)
|
|
90
|
+
VALUES ('delete', old.rowid, old.body);
|
|
91
|
+
END;
|
|
92
|
+
'''
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
```bash
|
|
96
|
+
dirsql query "SELECT slug FROM posts JOIN posts_fts ON posts.rowid = posts_fts.rowid
|
|
97
|
+
WHERE posts_fts MATCH 'recursion'" -c ./.dirsql.toml
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
```json
|
|
101
|
+
[{"slug":"again"}]
|
|
102
|
+
```
|
|
103
|
+
|
|
104
|
+
dirsql writes rows with plain `INSERT` and `DELETE`, so those triggers hold
|
|
105
|
+
through the initial load and every [watcher](./react-to-changes.md) event — you
|
|
106
|
+
maintain nothing. `name` still has to be the row table (`posts`); the virtual
|
|
107
|
+
table lives beside it under its own name. The batch runs once, when the table
|
|
108
|
+
is created, and editing any part of it rebuilds a
|
|
109
|
+
[persistent cache](./persist.md) from scratch. Full rules:
|
|
110
|
+
[Batch `ddl`](../reference/config.md#batch-ddl).
|
|
111
|
+
|
|
68
112
|
## Going further
|
|
69
113
|
|
|
70
114
|
- The parser mechanics — placeholders, stdout protocol, per-file failure
|
|
@@ -76,9 +76,11 @@ interpreter to resolve package names with
|
|
|
76
76
|
|
|
77
77
|
## Notes
|
|
78
78
|
|
|
79
|
-
- Extensions add **functions
|
|
80
|
-
|
|
81
|
-
|
|
79
|
+
- Extensions add **functions**, and virtual tables a table's `ddl` batch can
|
|
80
|
+
create. A `[[table]]`'s own `name` may not be a virtual table — that one is
|
|
81
|
+
the per-file row table `dirsql` inserts into — so declare the virtual table
|
|
82
|
+
alongside it in the same batch
|
|
83
|
+
([reference](../reference/config.md#batch-ddl)).
|
|
82
84
|
- Loading happens before any table DDL runs, and the SQL
|
|
83
85
|
`load_extension()` function is never exposed to queries
|
|
84
86
|
([reference](../reference/config.md#dirsql-extension)).
|
{dirsql-0.4.27/packages/python → dirsql-0.4.28/packages/rust}/docs/howto/search-by-meaning.md
RENAMED
|
@@ -95,10 +95,12 @@ LIMIT 10
|
|
|
95
95
|
|
|
96
96
|
::: tip Top-k is `LIMIT k`
|
|
97
97
|
If you know `sqlite-vec` you may reach for its `MATCH … AND k = 10` idiom.
|
|
98
|
-
That syntax belongs to `sqlite-vec`'s `vec0` virtual table, which
|
|
99
|
-
does not use
|
|
100
|
-
expressions, `sqlite-vec`'s own documented pattern is exactly what
|
|
101
|
-
guide uses: `ORDER BY vec_distance_cosine(...) LIMIT k`.
|
|
98
|
+
That syntax belongs to `sqlite-vec`'s `vec0` virtual table, which the table
|
|
99
|
+
above does not use: a `[[table]]`'s own `name` is always a per-file row table.
|
|
100
|
+
For plain expressions, `sqlite-vec`'s own documented pattern is exactly what
|
|
101
|
+
this guide uses: `ORDER BY vec_distance_cosine(...) LIMIT k`. To get the `vec0`
|
|
102
|
+
idiom instead, declare the `vec0` table alongside the row table in the same
|
|
103
|
+
[`ddl` batch](../reference/config.md#batch-ddl) and fill it from a trigger.
|
|
102
104
|
:::
|
|
103
105
|
|
|
104
106
|
## Repeat runs are cheap
|