dirsql 0.3.27__tar.gz → 0.3.29__tar.gz

{dirsql-0.3.27 → dirsql-0.3.29}/Cargo.lock

@@ -499,7 +499,7 @@ dependencies = [
 
 [[package]]
 name = "dirsql-py-ext"
-version = "0.3.27"
+version = "0.3.29"
 dependencies = [
  "dirsql",
  "pyo3",

dirsql-0.3.29/PKG-INFO

@@ -0,0 +1,121 @@
+Metadata-Version: 2.4
+Name: dirsql
+Version: 0.3.29
+Requires-Dist: pytest>=8 ; extra == 'dev'
+Requires-Dist: pytest-describe>=2 ; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23 ; extra == 'dev'
+Requires-Dist: pytest-cov>=5 ; extra == 'dev'
+Requires-Dist: ruff>=0.4 ; extra == 'dev'
+Requires-Dist: maturin>=1.0 ; extra == 'dev'
+Requires-Dist: ty==0.0.42 ; extra == 'dev'
+Provides-Extra: dev
+Summary: Ephemeral SQL index over a local directory
+Keywords: sql,filesystem,directory,sqlite,index
+Author: Kevin Scott
+License-Expression: MIT
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
+
+# `dirsql` (Python SDK)
+
+Ephemeral SQL index over a local directory. `dirsql` watches a filesystem, ingests structured files into an in-memory SQLite database, and exposes a SQL query interface -- the filesystem is always the source of truth.
+
+[Documentation](https://thekevinscott.github.io/dirsql/?lang=python)
+
+Also available as [`dirsql` on crates.io](https://crates.io/crates/dirsql) and [`dirsql` on npm](https://www.npmjs.com/package/dirsql).
+
+## Installation
+
+```bash
+pip install dirsql
+```
+
+Requires Python >= 3.12. Ships as a native extension (Rust via PyO3); prebuilt binary wheels are provided for common platforms.
+
+## Quick start
+
+`DirSQL` is async by default: the constructor returns immediately, scanning runs in a background thread, and you `await db.ready()` before querying. Each table is a `(ddl, glob, extract)` triple: the DDL defines the SQLite schema, the glob selects files (relative to the root), and `extract` turns a matched file into a list of row dicts. `dirsql` does not read file contents -- if `extract` needs the file body it reads `path` itself; return an empty list to skip a file.
+
+```python
+import asyncio
+import json
+from dirsql import DirSQL, Table
+
+async def main():
+    db = DirSQL(
+        "./my-blog",
+        tables=[
+            Table(
+                ddl="CREATE TABLE posts (title TEXT, author TEXT)",
+                glob="posts/*.json",
+                extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+            ),
+        ],
+    )
+    await db.ready()
+
+    posts = await db.query("SELECT * FROM posts WHERE author = 'alice'")
+    print(posts)
+
+asyncio.run(main())
+```
+
+## Multiple tables and joins
+
+```python
+db = DirSQL(
+    "./my-blog",
+    tables=[
+        Table(
+            ddl="CREATE TABLE posts (title TEXT, author_id TEXT)",
+            glob="posts/*.json",
+            extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+        ),
+        Table(
+            ddl="CREATE TABLE authors (id TEXT, name TEXT)",
+            glob="authors/*.json",
+            extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+        ),
+    ],
+)
+await db.ready()
+
+results = await db.query("""
+    SELECT posts.title, authors.name
+    FROM posts JOIN authors ON posts.author_id = authors.id
+""")
+```
+
+## Ignoring files
+
+Pass `ignore` patterns to skip files during scanning and watching:
+
+```python
+db = DirSQL(
+    "./my-blog",
+    ignore=["**/drafts/**", "**/.git/**"],
+    tables=[...],
+)
+```
+
+## Watching for changes
+
+`db.watch()` returns an async iterator of row-level change events as files change on disk:
+
+```python
+async for event in db.watch():
+    print(f"{event.action} on {event.table}: {event.row}")
+    if event.action == "error":
+        print(f"  error: {event.error}")
+```
+
+Each event has `.action` (`"insert"`, `"update"`, `"delete"`, or `"error"`), `.table`, `.row` (the new row, or the deleted row on `delete`), `.old_row` (the previous row, on `update`), `.file_path`, and `.error` (on `error`).
+
+## CLI
+
+`pip install dirsql` also installs a `dirsql` console script that runs an HTTP server exposing the SDK over HTTP: `POST /query` for SQL and `GET /events` for a Server-Sent Events change stream. Run `dirsql` (or `uvx dirsql`) to start it. See the [CLI guide](https://thekevinscott.github.io/dirsql/cli/).
+
+## License
+
+MIT
+

dirsql-0.3.29/README.md

@@ -0,0 +1,102 @@
+# `dirsql` (Python SDK)
+
+Ephemeral SQL index over a local directory. `dirsql` watches a filesystem, ingests structured files into an in-memory SQLite database, and exposes a SQL query interface -- the filesystem is always the source of truth.
+
+[Documentation](https://thekevinscott.github.io/dirsql/?lang=python)
+
+Also available as [`dirsql` on crates.io](https://crates.io/crates/dirsql) and [`dirsql` on npm](https://www.npmjs.com/package/dirsql).
+
+## Installation
+
+```bash
+pip install dirsql
+```
+
+Requires Python >= 3.12. Ships as a native extension (Rust via PyO3); prebuilt binary wheels are provided for common platforms.
+
+## Quick start
+
+`DirSQL` is async by default: the constructor returns immediately, scanning runs in a background thread, and you `await db.ready()` before querying. Each table is a `(ddl, glob, extract)` triple: the DDL defines the SQLite schema, the glob selects files (relative to the root), and `extract` turns a matched file into a list of row dicts. `dirsql` does not read file contents -- if `extract` needs the file body it reads `path` itself; return an empty list to skip a file.
+
+```python
+import asyncio
+import json
+from dirsql import DirSQL, Table
+
+async def main():
+    db = DirSQL(
+        "./my-blog",
+        tables=[
+            Table(
+                ddl="CREATE TABLE posts (title TEXT, author TEXT)",
+                glob="posts/*.json",
+                extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+            ),
+        ],
+    )
+    await db.ready()
+
+    posts = await db.query("SELECT * FROM posts WHERE author = 'alice'")
+    print(posts)
+
+asyncio.run(main())
+```
+
+## Multiple tables and joins
+
+```python
+db = DirSQL(
+    "./my-blog",
+    tables=[
+        Table(
+            ddl="CREATE TABLE posts (title TEXT, author_id TEXT)",
+            glob="posts/*.json",
+            extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+        ),
+        Table(
+            ddl="CREATE TABLE authors (id TEXT, name TEXT)",
+            glob="authors/*.json",
+            extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+        ),
+    ],
+)
+await db.ready()
+
+results = await db.query("""
+    SELECT posts.title, authors.name
+    FROM posts JOIN authors ON posts.author_id = authors.id
+""")
+```
+
+## Ignoring files
+
+Pass `ignore` patterns to skip files during scanning and watching:
+
+```python
+db = DirSQL(
+    "./my-blog",
+    ignore=["**/drafts/**", "**/.git/**"],
+    tables=[...],
+)
+```
+
+## Watching for changes
+
+`db.watch()` returns an async iterator of row-level change events as files change on disk:
+
+```python
+async for event in db.watch():
+    print(f"{event.action} on {event.table}: {event.row}")
+    if event.action == "error":
+        print(f"  error: {event.error}")
+```
+
+Each event has `.action` (`"insert"`, `"update"`, `"delete"`, or `"error"`), `.table`, `.row` (the new row, or the deleted row on `delete`), `.old_row` (the previous row, on `update`), `.file_path`, and `.error` (on `error`).
+
+## CLI
+
+`pip install dirsql` also installs a `dirsql` console script that runs an HTTP server exposing the SDK over HTTP: `POST /query` for SQL and `GET /events` for a Server-Sent Events change stream. Run `dirsql` (or `uvx dirsql`) to start it. See the [CLI guide](https://thekevinscott.github.io/dirsql/cli/).
+
+## License
+
+MIT

{dirsql-0.3.27 → dirsql-0.3.29}/docs/cli/index.md

@@ -44,11 +44,13 @@ dirsql
 
 :::
 
+::: warning Node version
+Requires **Node ≥ 20.11**.
+:::
+
 ::: tip For Rust library consumers
-The `cli` feature is **opt-in**. Adding `dirsql` as a library dependency
-(`cargo add dirsql`) pulls no CLI dependencies — only the core library. See the
-[Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust)
-for the library-vs-CLI feature split.
+The `cli` feature is **opt-in**. See the
+[Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust) for instructions on how to include.
 :::
 
 ## Quick start

{dirsql-0.3.27/packages/rust → dirsql-0.3.29}/docs/getting-started.md

@@ -19,7 +19,8 @@ cargo add dirsql
 ```
 
 ```bash [TypeScript]
-pnpm add dirsql
+# The npm CLI requires **Node ≥ 20.11**.
+npm add dirsql
 ```
 
 ```bash [CLI]
@@ -31,6 +32,8 @@ cargo install dirsql --features cli
 
 :::
 
+
+
 See the [CLI section](./cli/) for details on the command-line interface, and the [Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust) for the library-vs-CLI feature split.
 
 ## Quick start

{dirsql-0.3.27 → dirsql-0.3.29}/packages/python/Cargo.toml

@@ -4,7 +4,7 @@ name = "dirsql-py-ext"
 # pypi/maturin handler can rewrite it via `write-version` before
 # `maturin build`. `pyproject.toml` declares `dynamic = ["version"]`
 # and maturin reads this field. Mirrors `packages/rust/Cargo.toml`.
-version = "0.3.27"
+version = "0.3.29"
 edition.workspace = true
 publish = false
 readme = "README.md"

dirsql-0.3.29/packages/python/README.md

@@ -0,0 +1,102 @@
+# `dirsql` (Python SDK)
+
+Ephemeral SQL index over a local directory. `dirsql` watches a filesystem, ingests structured files into an in-memory SQLite database, and exposes a SQL query interface -- the filesystem is always the source of truth.
+
+[Documentation](https://thekevinscott.github.io/dirsql/?lang=python)
+
+Also available as [`dirsql` on crates.io](https://crates.io/crates/dirsql) and [`dirsql` on npm](https://www.npmjs.com/package/dirsql).
+
+## Installation
+
+```bash
+pip install dirsql
+```
+
+Requires Python >= 3.12. Ships as a native extension (Rust via PyO3); prebuilt binary wheels are provided for common platforms.
+
+## Quick start
+
+`DirSQL` is async by default: the constructor returns immediately, scanning runs in a background thread, and you `await db.ready()` before querying. Each table is a `(ddl, glob, extract)` triple: the DDL defines the SQLite schema, the glob selects files (relative to the root), and `extract` turns a matched file into a list of row dicts. `dirsql` does not read file contents -- if `extract` needs the file body it reads `path` itself; return an empty list to skip a file.
+
+```python
+import asyncio
+import json
+from dirsql import DirSQL, Table
+
+async def main():
+    db = DirSQL(
+        "./my-blog",
+        tables=[
+            Table(
+                ddl="CREATE TABLE posts (title TEXT, author TEXT)",
+                glob="posts/*.json",
+                extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+            ),
+        ],
+    )
+    await db.ready()
+
+    posts = await db.query("SELECT * FROM posts WHERE author = 'alice'")
+    print(posts)
+
+asyncio.run(main())
+```
+
+## Multiple tables and joins
+
+```python
+db = DirSQL(
+    "./my-blog",
+    tables=[
+        Table(
+            ddl="CREATE TABLE posts (title TEXT, author_id TEXT)",
+            glob="posts/*.json",
+            extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+        ),
+        Table(
+            ddl="CREATE TABLE authors (id TEXT, name TEXT)",
+            glob="authors/*.json",
+            extract=lambda path: [json.loads(open(path, encoding="utf-8").read())],
+        ),
+    ],
+)
+await db.ready()
+
+results = await db.query("""
+    SELECT posts.title, authors.name
+    FROM posts JOIN authors ON posts.author_id = authors.id
+""")
+```
+
+## Ignoring files
+
+Pass `ignore` patterns to skip files during scanning and watching:
+
+```python
+db = DirSQL(
+    "./my-blog",
+    ignore=["**/drafts/**", "**/.git/**"],
+    tables=[...],
+)
+```
+
+## Watching for changes
+
+`db.watch()` returns an async iterator of row-level change events as files change on disk:
+
+```python
+async for event in db.watch():
+    print(f"{event.action} on {event.table}: {event.row}")
+    if event.action == "error":
+        print(f"  error: {event.error}")
+```
+
+Each event has `.action` (`"insert"`, `"update"`, `"delete"`, or `"error"`), `.table`, `.row` (the new row, or the deleted row on `delete`), `.old_row` (the previous row, on `update`), `.file_path`, and `.error` (on `error`).
+
+## CLI
+
+`pip install dirsql` also installs a `dirsql` console script that runs an HTTP server exposing the SDK over HTTP: `POST /query` for SQL and `GET /events` for a Server-Sent Events change stream. Run `dirsql` (or `uvx dirsql`) to start it. See the [CLI guide](https://thekevinscott.github.io/dirsql/cli/).
+
+## License
+
+MIT

{dirsql-0.3.27 → dirsql-0.3.29}/packages/python/docs/cli/index.md

@@ -44,11 +44,13 @@ dirsql
 
 :::
 
+::: warning Node version
+Requires **Node ≥ 20.11**.
+:::
+
 ::: tip For Rust library consumers
-The `cli` feature is **opt-in**. Adding `dirsql` as a library dependency
-(`cargo add dirsql`) pulls no CLI dependencies — only the core library. See the
-[Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust)
-for the library-vs-CLI feature split.
+The `cli` feature is **opt-in**. See the
+[Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust) for instructions on how to include.
 :::
 
 ## Quick start

{dirsql-0.3.27 → dirsql-0.3.29/packages/python}/docs/getting-started.md

@@ -19,7 +19,8 @@ cargo add dirsql
 ```
 
 ```bash [TypeScript]
-pnpm add dirsql
+# The npm CLI requires **Node ≥ 20.11**.
+npm add dirsql
 ```
 
 ```bash [CLI]
@@ -31,6 +32,8 @@ cargo install dirsql --features cli
 
 :::
 
+
+
 See the [CLI section](./cli/) for details on the command-line interface, and the [Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust) for the library-vs-CLI feature split.
 
 ## Quick start

dirsql-0.3.29/packages/rust/README.md

@@ -0,0 +1,139 @@
+# `dirsql` (Rust SDK)
+
+Ephemeral SQL index over a local directory. `dirsql` watches a filesystem, ingests structured files into an in-memory SQLite database, and exposes a SQL query interface -- the filesystem is always the source of truth.
+
+[Documentation](https://thekevinscott.github.io/dirsql/?lang=rust)
+
+Also available as [`dirsql` on PyPI](https://pypi.org/project/dirsql/) and [`dirsql` on npm](https://www.npmjs.com/package/dirsql).
+
+## Installation
+
+```bash
+cargo add dirsql
+```
+
+## Quick start
+
+`DirSQL::new` scans the directory synchronously and returns a ready instance. Each table is a `(ddl, glob, extract)` triple: the DDL defines the SQLite schema, the glob selects files (relative to the root), and `extract` turns a matched file into rows (`Vec<HashMap<String, Value>>`). `dirsql` does not read file contents -- the callback reads `path` itself; return an empty `Vec` to skip a file.
+
+```rust
+use dirsql::{DirSQL, Table, Value};
+use std::collections::HashMap;
+
+// Convert a JSON object string into a dirsql row. Reused by the examples below.
+fn row_from_json(raw: &str) -> HashMap<String, Value> {
+    let v: serde_json::Value = serde_json::from_str(raw).unwrap();
+    let serde_json::Value::Object(obj) = v else { return HashMap::new() };
+    obj.into_iter()
+        .map(|(k, val)| {
+            let v = match val {
+                serde_json::Value::String(s) => Value::Text(s),
+                serde_json::Value::Number(n) => n
+                    .as_i64()
+                    .map(Value::Integer)
+                    .unwrap_or_else(|| Value::Real(n.as_f64().unwrap_or(0.0))),
+                serde_json::Value::Bool(b) => Value::Integer(b as i64),
+                serde_json::Value::Null => Value::Null,
+                other => Value::Text(other.to_string()),
+            };
+            (k, v)
+        })
+        .collect()
+}
+
+let db = DirSQL::new(
+    "./my-blog",
+    vec![Table::new(
+        "CREATE TABLE posts (title TEXT, author TEXT)",
+        "posts/*.json",
+        |path| vec![row_from_json(&std::fs::read_to_string(path).unwrap())],
+    )],
+)?;
+
+let posts = db.query("SELECT * FROM posts WHERE author = 'alice'")?;
+```
+
+## Multiple tables and joins
+
+```rust
+let db = DirSQL::new(
+    "./my-blog",
+    vec![
+        Table::new(
+            "CREATE TABLE posts (title TEXT, author_id TEXT)",
+            "posts/*.json",
+            |path| vec![row_from_json(&std::fs::read_to_string(path).unwrap())],
+        ),
+        Table::new(
+            "CREATE TABLE authors (id TEXT, name TEXT)",
+            "authors/*.json",
+            |path| vec![row_from_json(&std::fs::read_to_string(path).unwrap())],
+        ),
+    ],
+)?;
+
+let results = db.query(
+    "SELECT posts.title, authors.name \
+     FROM posts JOIN authors ON posts.author_id = authors.id",
+)?;
+```
+
+## Ignoring files
+
+Use `DirSQL::with_ignore` to skip files during scanning and watching:
+
+```rust
+let db = DirSQL::with_ignore(
+    "./my-blog",
+    vec![/* tables */],
+    vec!["**/drafts/**", "**/.git/**"],
+)?;
+```
+
+## Watching for changes
+
+`db.watch()` returns a stream of row-level change events as files change on disk. `.next()` comes from `StreamExt` in the `futures` crate (`cargo add futures`), driven inside an async runtime such as tokio:
+
+```rust
+use dirsql::RowEvent;
+use futures::StreamExt;
+
+let mut stream = db.watch()?;
+while let Some(event) = stream.next().await {
+    match event {
+        RowEvent::Insert { table, row, file_path } => {
+            println!("insert on {table} ({file_path}): {row:?}")
+        }
+        RowEvent::Update { table, old_row, new_row, file_path } => {
+            println!("update on {table} ({file_path}): {old_row:?} -> {new_row:?}")
+        }
+        RowEvent::Delete { table, row, file_path } => {
+            println!("delete on {table} ({file_path}): {row:?}")
+        }
+        RowEvent::Error { table, file_path, error } => {
+            println!("error on {table:?} {file_path:?}: {error}")
+        }
+    }
+}
+```
+
+## CLI
+
+```bash
+cargo install dirsql --features cli
+dirsql
+```
+
+Running `dirsql` starts an HTTP server bound to `localhost:7117` that exposes the SDK over HTTP: `POST /query` for SQL and `GET /events` for a Server-Sent Events change stream. Override with `--host`, `--port`, `--config`. See the [CLI guide](https://thekevinscott.github.io/dirsql/cli/).
+
+The `cli` feature is **opt-in** -- `cargo add dirsql` pulls no CLI dependencies. `cargo install dirsql` without `--features cli` silently installs nothing (`required-features` skips the bin target with no warning); always include the flag, or use `npx dirsql` / `uvx dirsql` for prebuilt binaries.
+
+### Feature flags
+
+| Feature | Default | Description |
+|---|---|---|
+| `cli` | no | Enables the `dirsql` binary and its dependencies. |
+
+## License
+
+MIT

{dirsql-0.3.27 → dirsql-0.3.29}/packages/rust/docs/cli/index.md

@@ -44,11 +44,13 @@ dirsql
 
 :::
 
+::: warning Node version
+Requires **Node ≥ 20.11**.
+:::
+
 ::: tip For Rust library consumers
-The `cli` feature is **opt-in**. Adding `dirsql` as a library dependency
-(`cargo add dirsql`) pulls no CLI dependencies — only the core library. See the
-[Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust)
-for the library-vs-CLI feature split.
+The `cli` feature is **opt-in**. See the
+[Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust) for instructions on how to include.
 :::
 
 ## Quick start

{dirsql-0.3.27/packages/python → dirsql-0.3.29/packages/rust}/docs/getting-started.md

@@ -19,7 +19,8 @@ cargo add dirsql
 ```
 
 ```bash [TypeScript]
-pnpm add dirsql
+# The npm CLI requires **Node ≥ 20.11**.
+npm add dirsql
 ```
 
 ```bash [CLI]
@@ -31,6 +32,8 @@ cargo install dirsql --features cli
 
 :::
 
+
+
 See the [CLI section](./cli/) for details on the command-line interface, and the [Rust library README](https://github.com/thekevinscott/dirsql/tree/main/packages/rust) for the library-vs-CLI feature split.
 
 ## Quick start