@nshiab/simple-data-analysis-core 2.0.4 → 2.0.6

package/README.md

@@ -60,8 +60,28 @@ npx @nshiab/setup-data-project
 bunx @nshiab/setup-data-project
 ```
 
+## Running a data pipeline
+
+Transformation methods are synchronous and chainable. SDA-core queues their work
+and combines compatible operations into a single DuckDB statement. Async methods
+such as `getData()`, `log()`, and `writeData()` execute pending work before
+returning results, so only the final method in a chain needs to be awaited.
+
+```ts
+await table
+  .filter("salary > 50000")
+  .selectColumns(["name", "salary"])
+  .log();
+```
+
+Use `await table.run()` or `await sdb.run()` to execute pending work without
+reading or exporting a result. Both execute queued operations across all tables
+in the database, in the order the methods were called.
+
 ## Performance benchmarks
 
+These are end-to-end workflow comparisons.
+
 These benchmarks compare SDA-core with raw DuckDB and popular Python and R
 libraries, measuring duration and peak memory.
 
@@ -69,6 +89,14 @@ They were run on a MacBook Pro with an Apple M4 Max and 64 GB of memory.
 
 <!-- benchmark-results:start -->
 
+**Tabular versions:** @duckdb/node-api 1.5.5-r.4; DuckDB v1.5.5 (Deno 2.9.6);
+SDA-core 2.0.5 (Deno 2.9.6); pandas 3.0.5 (Python 3.14.7); tidyverse 2.0.0 (R
+4.6.1).
+
+**Spatial versions:** @duckdb/node-api 1.5.5-r.4; DuckDB v1.5.5 (Deno 2.9.6);
+SDA-core 2.0.5 (Deno 2.9.6); GeoPandas 1.1.4 (Python 3.14.7); sf 1.1.2 (R
+4.6.1).
+
 ### Tabular workload
 
 Using 22,051,025 temperature records (`ahccd.csv`, 1.77 GB, in
@@ -76,12 +104,12 @@ Using 22,051,025 temperature records (`ahccd.csv`, 1.77 GB, in
 save the cleaned data, then calculate average temperatures by station and decade
 and export the sorted results.
 
-| Library version                           | Runtime       |  Mean duration | Duration difference | Mean peak memory | Memory difference |
-| ----------------------------------------- | ------------- | -------------: | ------------------: | ---------------: | ----------------: |
-| @duckdb/node-api 1.5.5-r.4; DuckDB v1.5.5 | Deno 2.9.6    |  1.14 ± 0.01 s |               -8.2% |         2,372 MB |             -7.1% |
-| SDA-core 2.0.0                            | Deno 2.9.6    |  1.24 ± 0.03 s |            baseline |         2,554 MB |          baseline |
-| pandas 3.0.5                              | Python 3.14.7 | 28.21 ± 0.01 s |            +2168.6% |         4,699 MB |            +84.0% |
-| tidyverse 2.0.0                           | R 4.6.1       | 78.81 ± 0.17 s |            +6236.7% |         8,178 MB |           +220.2% |
+| Library   |  Mean duration | Duration difference | Mean peak memory | Memory difference |
+| --------- | -------------: | ------------------: | ---------------: | ----------------: |
+| DuckDB    |  1.30 ± 0.03 s |               -9.7% |         2,457 MB |             -2.7% |
+| SDA-core  |  1.44 ± 0.03 s |            baseline |         2,524 MB |          baseline |
+| pandas    | 30.08 ± 0.07 s |            +1987.9% |         4,700 MB |            +86.2% |
+| tidyverse | 82.62 ± 0.22 s |            +5634.4% |         8,178 MB |           +224.0% |
 
 ### Spatial workload
 
@@ -91,15 +119,72 @@ in `benchmarks/data/`, we remove missing coordinates, create points, join trees
 to neighbourhoods, then count trees per neighbourhood and export the sorted
 results.
 
-| Library version                           | Runtime       | Mean duration | Duration difference | Mean peak memory | Memory difference |
-| ----------------------------------------- | ------------- | ------------: | ------------------: | ---------------: | ----------------: |
-| @duckdb/node-api 1.5.5-r.4; DuckDB v1.5.5 | Deno 2.9.6    | 0.72 ± 0.01 s |               -3.4% |           255 MB |             -7.2% |
-| SDA-core 2.0.0                            | Deno 2.9.6    | 0.75 ± 0.01 s |            baseline |           275 MB |          baseline |
-| GeoPandas 1.1.4                           | Python 3.14.7 | 1.11 ± 0.00 s |              +48.7% |           292 MB |             +6.0% |
-| sf 1.1.2                                  | R 4.6.1       | 1.58 ± 0.00 s |             +111.2% |           489 MB |            +78.0% |
+| Library   | Mean duration | Duration difference | Mean peak memory | Memory difference |
+| --------- | ------------: | ------------------: | ---------------: | ----------------: |
+| DuckDB    | 0.78 ± 0.00 s |               -5.1% |           255 MB |             -5.8% |
+| SDA-core  | 0.82 ± 0.01 s |            baseline |           271 MB |          baseline |
+| GeoPandas | 1.25 ± 0.01 s |              +51.4% |           294 MB |             +8.5% |
+| sf        | 1.80 ± 0.01 s |             +118.9% |           490 MB |            +81.1% |
 
 <!-- benchmark-results:end -->
 
+### Focused operation benchmarks
+
+These additional benchmarks compare Core and DuckDB on data transfer, joins,
+aggregations, and JavaScript geometry updates to help track and improve
+performance over time. Run `deno task benchmark-core` to regenerate these
+tables. See the [benchmark methodology](benchmarks/operations/README.md) for
+fixture sizes, validation, and measurement details. Peak process memory includes
+setup, warm-up, the measured operation, and validation.
+
+<!-- benchmark-operations:start -->
+
+#### Join followed by aggregation
+
+| Operation        |      Rows | Batch size | Implementation | Mean duration ± SD | Mean peak process memory |
+| ---------------- | --------: | ---------: | -------------- | -----------------: | -----------------------: |
+| Join → aggregate | 1,000,000 |          — | Core           |    12.11 ± 0.25 ms |                123.1 MiB |
+| Join → aggregate | 1,000,000 |          — | DuckDB         |    11.29 ± 0.33 ms |                134.2 MiB |
+
+#### JavaScript data transfer
+
+`loadArray()` measures JavaScript → DuckDB; `getData()` measures DuckDB →
+JavaScript. `updateWithJS()` measures the round trip with a simple numeric
+increment, including Core's staging and table replacement.
+
+| Operation      |    Rows | Batch size | Implementation | Mean duration ± SD | Mean peak process memory |
+| -------------- | ------: | ---------: | -------------- | -----------------: | -----------------------: |
+| loadArray()    | 100,000 |          — | Core           |     9.40 ± 0.68 ms |                202.3 MiB |
+| loadArray()    | 100,000 |          — | DuckDB         |    19.80 ± 0.94 ms |                230.8 MiB |
+| getData()      | 100,000 |          — | Core           |     9.29 ± 0.26 ms |                191.2 MiB |
+| getData()      | 100,000 |          — | DuckDB         |    12.54 ± 0.27 ms |                209.0 MiB |
+| updateWithJS() | 100,000 |       1000 | Core           |    92.45 ± 3.19 ms |                208.2 MiB |
+| updateWithJS() | 100,000 |       1000 | DuckDB         |    65.89 ± 1.13 ms |                239.6 MiB |
+| updateWithJS() | 100,000 |      10000 | Core           |    42.93 ± 1.05 ms |                218.6 MiB |
+| updateWithJS() | 100,000 |      10000 | DuckDB         |    41.76 ± 0.81 ms |                233.5 MiB |
+
+#### JavaScript geometry updates
+
+Points have one position per geometry; polygons have one ring with 1,001
+positions. Both implementations transfer GeoJSON through JavaScript and stage
+writes. These updates add a label and shift every longitude by 0.01 degrees. A
+batch size of — means all input rows at once. Attribute-only updates are also
+measured and retained in the raw results; their similar timings are omitted here
+to keep the table concise.
+
+| Operation                 |    Rows | Batch size | Implementation | Mean duration ± SD | Mean peak process memory |
+| ------------------------- | ------: | ---------: | -------------- | -----------------: | -----------------------: |
+| Points: geometry update   | 100,000 |          — | Core           |   182.59 ± 1.02 ms |                402.4 MiB |
+| Points: geometry update   | 100,000 |          — | DuckDB         |   180.20 ± 1.98 ms |                402.2 MiB |
+| Points: geometry update   | 100,000 |       1000 | Core           |   252.07 ± 6.59 ms |                267.9 MiB |
+| Points: geometry update   | 100,000 |       1000 | DuckDB         |   217.69 ± 2.43 ms |                245.4 MiB |
+| Polygons: geometry update |   1,000 |          — | Core           |   403.35 ± 1.92 ms |                938.3 MiB |
+| Polygons: geometry update |   1,000 |          — | DuckDB         |   408.62 ± 2.57 ms |               1104.8 MiB |
+| Polygons: geometry update |   1,000 |        100 | Core           |   418.45 ± 2.45 ms |                747.0 MiB |
+| Polygons: geometry update |   1,000 |        100 | DuckDB         |   416.90 ± 2.54 ms |                714.6 MiB |
+
+<!-- benchmark-operations:end -->
+
 ## Building extensions
 
 The full
@@ -115,6 +200,10 @@ chainable, define them on a `SimpleTable` subclass, give them a return type of
 create tables should always use `this.sdb.newTable()` so they also return your
 extended table type.
 
+Builders return `this` synchronously; their queued work runs when an observer
+such as `getData()`, `log()`, or `run()` is called. Observers execute pending
+work across the database in program order.
+
 ```ts
 import {
   SimpleDB as CoreDB,
@@ -172,3 +261,20 @@ The callback must await all asynchronous work that can queue builders. If it
 rejects, captured builders that have not already run are discarded. Builders
 already drained by an observer inside the callback remain applied;
 `queueAsyncBarrier()` does not provide database rollback.
+
+This differs from a failure while executing queued builders: during replay, the
+failing table's remaining chain is aborted, but unexecuted work for other tables
+is requeued for their next observation.
+
+Other useful helpers, imported from `@nshiab/simple-data-analysis-core/helpers`:
+
+- `updateColumnsWithJS()`: enrich selected columns asynchronously in bounded
+  batches inside `queueAsyncBarrier()`, preserving untouched columns, including
+  geometry. Return one result per input row, in the same order.
+- `queryDB()` with `mergeOptions()`: execute SQL with the library's logging and
+  diagnostic settings. Queries flush queued work before running.
+- `quoteIdentifier()`: quote table and column names safely when constructing
+  SQL.
+
+See the [exported helpers](src/helpers/index.ts) for the complete list and links
+to their documentation and examples. Helpers outside that list are internal.

package/esm/class/Simple.d.ts

@@ -18,9 +18,15 @@ export default class Simple {
      */
     typesToLog: boolean;
     /**
-     * The maximum number of characters to display for text-based cells. If undefined, the entire text is shown.
+     * The maximum number of characters to display for text and stringified nested cells, including the truncation marker. This does not control column width. Set to Infinity to show the entire text.
      *
-     * @defaultValue `undefined`
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB({ charsToLog: 150 });
+     * await sdb.newTable().loadArray([{ text: "x".repeat(200) }]).log();
+     * ```
+     *
+     * @defaultValue `75`
      */
     charsToLog: number | undefined;
     /**

package/esm/class/Simple.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"Simple.d.ts","sourceRoot":"","sources":["../../src/class/Simple.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,gBAAgB,EAChB,cAAc,EACd,WAAW,EACZ,MAAM,kBAAkB,CAAC;AAE1B;;;;GAIG;AACH,MAAM,CAAC,OAAO,OAAO,MAAM;IACzB;;;;OAIG;IACH,SAAS,EAAE,MAAM,CAAC;IAClB;;;;OAIG;IACH,UAAU,EAAE,OAAO,CAAC;IACpB;;;;OAIG;IACH,UAAU,EAAE,MAAM,GAAG,SAAS,CAAC;IAC/B;;OAEG;IACH,EAAE,EAAG,cAAc,CAAC;IACpB;;OAEG;IACH,UAAU,EAAG,gBAAgB,CAAC;IAC9B;;;;OAIG;IACH,gBAAgB,EAAE,OAAO,CAAC;IAC1B;;;;;;;;;;;;;;;;;;;;;;;;;OAyBG;IACH,QAAQ,EAAG,CACT,KAAK,EAAE,MAAM,EACb,UAAU,EAAE,gBAAgB,EAC5B,UAAU,EAAE,OAAO,EACnB,OAAO,EAAE;QACP,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;QACtB,UAAU,EAAE;YAAE,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAA;SAAE,GAAG,IAAI,CAAC;QAC9C,KAAK,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;QACtB,MAAM,CAAC,EAAE,WAAW,EAAE,CAAC;QACvB,MAAM,EAAE,OAAO,CAAC;QAChB,UAAU,EAAE,OAAO,CAAC;KACrB,KACE,OAAO,CACR;QACA,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;KACxB,EAAE,GACD,IAAI,CACP,CAAC;gBAGA,OAAO,GAAE;QACP,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,UAAU,CAAC,EAAE,OAAO,CAAC;KACjB;CAOT"}
+{"version":3,"file":"Simple.d.ts","sourceRoot":"","sources":["../../src/class/Simple.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACV,gBAAgB,EAChB,cAAc,EACd,WAAW,EACZ,MAAM,kBAAkB,CAAC;AAE1B;;;;GAIG;AACH,MAAM,CAAC,OAAO,OAAO,MAAM;IACzB;;;;OAIG;IACH,SAAS,EAAE,MAAM,CAAC;IAClB;;;;OAIG;IACH,UAAU,EAAE,OAAO,CAAC;IACpB;;;;;;;;;;OAUG;IACH,UAAU,EAAE,MAAM,GAAG,SAAS,CAAC;IAC/B;;OAEG;IACH,EAAE,EAAG,cAAc,CAAC;IACpB;;OAEG;IACH,UAAU,EAAG,gBAAgB,CAAC;IAC9B;;;;OAIG;IACH,gBAAgB,EAAE,OAAO,CAAC;IAC1B;;;;;;;;;;;;;;;;;;;;;;;;;OAyBG;IACH,QAAQ,EAAG,CACT,KAAK,EAAE,MAAM,EACb,UAAU,EAAE,gBAAgB,EAC5B,UAAU,EAAE,OAAO,EACnB,OAAO,EAAE;QACP,MAAM,EAAE,MAAM,GAAG,IAAI,CAAC;QACtB,UAAU,EAAE;YAAE,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAA;SAAE,GAAG,IAAI,CAAC;QAC9C,KAAK,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;QACtB,MAAM,CAAC,EAAE,WAAW,EAAE,CAAC;QACvB,MAAM,EAAE,OAAO,CAAC;QAChB,UAAU,EAAE,OAAO,CAAC;KACrB,KACE,OAAO,CACR;QACA,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;KACxB,EAAE,GACD,IAAI,CACP,CAAC;gBAGA,OAAO,GAAE;QACP,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,UAAU,CAAC,EAAE,OAAO,CAAC;KACjB;CAOT"}

package/esm/class/Simple.js

@@ -28,9 +28,15 @@ export default class Simple {
             value: void 0
         });
         /**
-         * The maximum number of characters to display for text-based cells. If undefined, the entire text is shown.
+         * The maximum number of characters to display for text and stringified nested cells, including the truncation marker. This does not control column width. Set to Infinity to show the entire text.
          *
-         * @defaultValue `undefined`
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB({ charsToLog: 150 });
+         * await sdb.newTable().loadArray([{ text: "x".repeat(200) }]).log();
+         * ```
+         *
+         * @defaultValue `75`
          */
         Object.defineProperty(this, "charsToLog", {
             enumerable: true,
@@ -100,7 +106,7 @@ export default class Simple {
             value: void 0
         });
         this.rowsToLog = options.rowsToLog ?? 10;
-        this.charsToLog = options.charsToLog;
+        this.charsToLog = options.charsToLog ?? 75;
         this.typesToLog = options.typesToLog ?? false;
         this.defaultTableName = false;
     }

package/esm/class/SimpleDB.d.ts

@@ -36,6 +36,73 @@ import Simple from "./Simple.js";
  */
 export default class SimpleDB<Table extends SimpleTable = SimpleTable> extends Simple {
     #private;
+    /**
+     * The operator syntax used by expressions and custom queries in this database.
+     * `"js"` translates JavaScript-style operators (`&&`, `||`, `==`, `===`,
+     * `!==`) and null comparisons within SQL expressions. It does not evaluate
+     * arbitrary JavaScript. `||` remains concatenation when string literals, list
+     * literals, explicit text casts, or `concat()` results make that intent clear.
+     * Otherwise `||` becomes OR; column types are not inferred. Use `concat()`
+     * when concatenation would otherwise be ambiguous, such as `first || last`.
+     * `"sql"` passes SQL through unchanged, including `||` and null comparisons.
+     * Choose the syntax when constructing the database; it applies to all tables.
+     *
+     * @defaultValue `"js"`
+     * @category Properties
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB({ expressionSyntax: "js" }); // The default mode
+     * // || becomes SQL OR: keep rows where at least one condition is true.
+     * await sdb.newTable().loadArray([{ active: true, admin: false }])
+     *   .filter("active || admin").log();
+     * ```
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB({ expressionSyntax: "js" });
+     * // && becomes SQL AND: both conditions must be true.
+     * // === becomes SQL =: compare the age with 18.
+     * await sdb.newTable().loadArray([{ active: true, age: 18 }])
+     *   .filter("active && age === 18").log();
+     * ```
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB({ expressionSyntax: "js" });
+     * // SQL AND and OR also work in JS mode; they pass through unchanged.
+     * // Keep active adults, or anyone who is an admin.
+     * await sdb.newTable().loadArray([{ active: true, age: 18, admin: false }])
+     *   .filter("(active AND age >= 18) OR admin").log();
+     * ```
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB({ expressionSyntax: "js" });
+     * // Bare column names do not establish concatenation. Use concat() explicitly.
+     * await sdb.newTable().loadArray([{ first: "Jane", last: "Doe" }])
+     *   .addColumn("name", "string", "concat(first, ' ', last)").log();
+     * ```
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB(); // JS mode is the default
+     * // The string literal ' ' makes concatenation clear, so both || stay unchanged.
+     * await sdb.newTable().loadArray([{ first: "Jane", last: "Doe" }])
+     *   .addColumn("name", "string", "first || ' ' || last").log();
+     * ```
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB({ expressionSyntax: "sql" });
+     * // SQL mode preserves || as concatenation: the name becomes "Jane Doe".
+     * await sdb.newTable().loadArray([{ first: "Jane", last: "Doe" }])
+     *   .addColumn("name", "string", "first || ' ' || last").log();
+     * ```
+     * @example
+     * ```ts
+     * const sdb = new SimpleDB({ expressionSyntax: "sql" });
+     * // Use SQL AND and OR for logical conditions; && and || are not translated.
+     * // Both active and adult must be true, unless admin is true.
+     * await sdb.newTable().loadArray([{ active: true, age: 18, admin: false }])
+     *   .filter("(active AND age >= 18) OR admin").log();
+     * ```
+     */
+    readonly expressionSyntax: "js" | "sql";
     /**
      * Whether to log each SQL statement immediately before execution.
      *
@@ -255,9 +322,10 @@ export default class SimpleDB<Table extends SimpleTable = SimpleTable> extends S
      * @param options.readOnly - Opens an existing DuckDB file read-only. Defaults to false. Requires a file and cannot be combined with overwrite.
      * @param options.logDuration - A flag indicating whether to log the total execution duration.
      * @param options.rowsToLog - The number of rows to display when logging a table.
-     * @param options.charsToLog - The maximum number of characters to display for text-based cells.
+     * @param options.charsToLog - The maximum characters to display for text and stringified nested cells, including truncation markers (default: 75). Independent of column width. Use Infinity for unlimited text.
      * @param options.typesToLog - A flag indicating whether to include data types when logging a table.
      * @param options.cacheVerbose - Whether to log cache hits and misses, code and input changes, TTL status, and cache read/write timing.
+     * @param options.expressionSyntax - Operator syntax for expressions and custom queries: `"js"` (default) translates JavaScript-style operators in SQL expressions; `"sql"` preserves SQL unchanged.
      * @param options.logSQL - A flag indicating whether to log SQL immediately before execution.
      * @param options.explainSQL - A flag indicating whether to log DuckDB query plans for supported statements.
      * @param options.duckDbCache - A flag indicating whether to use DuckDB's external file cache.
@@ -283,6 +351,7 @@ export default class SimpleDB<Table extends SimpleTable = SimpleTable> extends S
         charsToLog?: number;
         typesToLog?: boolean;
         cacheVerbose?: boolean;
+        expressionSyntax?: "js" | "sql";
         logSQL?: boolean;
         explainSQL?: boolean;
         duckDbCache?: boolean | null;
@@ -300,6 +369,7 @@ export default class SimpleDB<Table extends SimpleTable = SimpleTable> extends S
     start(): Promise<SimpleDB>;
     /**
      * Creates a new SimpleTable instance within the database.
+     * The name must be unique among registered tables, ignoring ASCII letter case.
      *
      * @param name - The name of the new table. If not provided, a default name is generated (e.g., "table1").
      * @returns A new table instance.
@@ -486,6 +556,7 @@ export default class SimpleDB<Table extends SimpleTable = SimpleTable> extends S
     }[]>;
     /**
      * Executes a custom SQL query directly against the DuckDB instance.
+     * With the default `SimpleDB.expressionSyntax: "js"`, queries support JavaScript-style operators (`&&`, `||`, `===`, `!==`). Set `expressionSyntax: "sql"` for unchanged SQL.
      * Queries run in UTC. When data is returned, temporal values use the same
      * JavaScript representations as `SimpleTable.getData()`.
      *

package/esm/class/SimpleDB.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"SimpleDB.d.ts","sourceRoot":"","sources":["../../src/class/SimpleDB.ts"],"names":[],"mappings":"AAGA,OAAO,WAAW,MAAM,kBAAkB,CAAC;AAG3C,OAAO,MAAM,MAAM,aAAa,CAAC;AA6BjC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAiCG;AAEH,MAAM,CAAC,OAAO,OAAO,QAAQ,CAAC,KAAK,SAAS,WAAW,GAAG,WAAW,CACnE,SAAQ,MAAM;;IACd;;;;;;;;;OASG;IACH,MAAM,EAAE,OAAO,CAAC;IAChB;;;;;;;;;OASG;IACH,UAAU,EAAE,OAAO,CAAC;IACpB;;;;;OAKG;IACH,gBAAgB,EAAE,MAAM,EAAE,CAAC;IAC3B;;;;;OAKG;IACH,aAAa,EAAE,MAAM,GAAG,SAAS,CAAC;IAClC;;;;;OAKG;IACH,cAAc,EAAE,MAAM,CAAC;IACvB;;;;;OAKG;IACH,WAAW,EAAE,OAAO,CAAC;IACrB;;;;;;OAMG;IACH,YAAY,EAAE,OAAO,CAAC;IACtB;;;;;OAKG;IACH,cAAc,EAAE,MAAM,CAAC;IACvB;;;;;OAKG;IACH,gBAAgB,EAAE,MAAM,CAAC;IACzB;;;;;OAKG;IACH,WAAW,EAAE,OAAO,CAAC;IACrB;;;;;;;OAOG;IACH,YAAY,EAAE,OAAO,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC;IAInC;;;;OAIG;IACH,cAAc,EAAE,MAAM,GAAG,SAAS,GAAG,QAAQ,CAAC;IAC9C;;;;;;;OAOG;IACH,YAAY,EAAE,MAAM,CAAC;IACrB;;;;;;;OAOG;IACH,UAAU,EAAE,MAAM,CAAC;IACnB;;;;;;OAMG;IACH,aAAa,EAAE,OAAO,CAAC;IACvB;;;;;OAKG;IACH,WAAW,EAAE,OAAO,GAAG,IAAI,CAAC;IAC5B;;;;;OAKG;IACH,WAAW,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC;;;;;OAKG;IACH,OAAO,EAAE,MAAM,GAAG,SAAS,CAAC;IAC5B;;;;;;OAMG;IACH,IAAI,EAAE,MAAM,CAAC;IACb;;;;;OAKG;IACH,SAAS,EAAE,OAAO,CAAC;IACnB;;;;;;;;;;;;;OAaG;IACH,QAAQ,EAAE,OAAO,CAAC;IAClB;;;;;;;;;;;;;;;;;;;;;;;;;;OA0BG;IACH,UAAU,EAAE,KACV,IAAI,EAAE,MAAM,EACZ,GAAG,EAAE,QAAQ,EACb,OAAO,CAAC,EAAE;QACR,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,UAAU,CAAC,EAAE,OAAO,CAAC;KACtB,KACE,KAAK,CAQC;IAEX;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAkCG;gBAED,OAAO,GAAE;QACP,IAAI,CAAC,EAAE,MAAM,CAAC;QACd,SAAS,CAAC,EAAE,OAAO,CAAC;QACpB,QAAQ,CAAC,EAAE,OAAO,CAAC;QACnB,WAAW,CAAC,EAAE,OAAO,CAAC;QACtB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,UAAU,CAAC,EAAE,OAAO,CAAC;QACrB,YAAY,CAAC,EAAE,OAAO,CAAC;QACvB,MAAM,CAAC,EAAE,OAAO,CAAC;QACjB,UAAU,CAAC,EAAE,OAAO,CAAC;QACrB,WAAW,CAAC,EAAE,OAAO,GAAG,IAAI,CAAC;QAC7B,WAAW,CAAC,EAAE,OAAO,CAAC;QACtB,WAAW,CAAC,EAAE,MAAM,CAAC;QACrB,OAAO,CAAC,EAAE,MAAM,CAAC;KACb;IAgCR;;;;;;OAMG;IACG,KAAK,IAAI,OAAO,CAAC,QAAQ,CAAC;IAqGhC;;;;;;;;;;;;;;;;;;;;;;;;OAwBG;IACH,QAAQ,CACN,IAAI,CAAC,EAAE,MAAM,GACZ,KAAK;IAuCR;;;;;;;;;;;;;OAaG;IACG,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;IAiB5C;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAgCG;IACG,YAAY,CAChB,MAAM,EAAE,KAAK,GAAG,MAAM,GAAG,CAAC,KAAK,GAAG,MAAM,CAAC,EAAE,GAC1C,OAAO,CAAC,IAAI,CAAC;IAKhB;;;;;;;;;;;;;;;;;;;;;;;;;;OA0BG;IACG,YAAY,CAChB,MAAM,EAAE,KAAK,GAAG,MAAM,GAAG,CAAC,KAAK,GAAG,MAAM,CAAC,EAAE,GAC1C,OAAO,CAAC,IAAI,CAAC;IAKhB;;;;;;;;;;;;OAYG;IACG,aAAa,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;IAIxC;;;;;;;;;;;;OAYG;IACG,aAAa,IAAI,OAAO,CAAC,IAAI,CAAC;IAYpC;;;;;;;;;;;;;;OAcG;IACH,SAAS,IAAI,SAAS,KAAK,EAAE;IAI7B;;;;;;;;;;;;;;;;;;;;;OAqBG;IACG,QAAQ,CAAC,KAAK,EAAE,KAAK,GAAG,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAMvD;;;;;;;;;;;;OAYG;IACG,aAAa,IAAI,OAAO,CAC5B;QACE,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;KACxB,EAAE,CACJ;IAeD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAiCG;IACG,WAAW,CACf,KAAK,EAAE,MAAM,EACb,OAAO,GAAE;QACP,UAAU,CAAC,EAAE,OAAO,CAAC;QACrB,KAAK,CAAC,EAAE,MAAM,CAAC;KACX,GACL,OAAO,CACN;QACA,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;KACxB,EAAE,GACD,IAAI,CACP;IAaD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAmCG;IACG,MAAM,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAKzC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAuCG;IACG,OAAO,CACX,IAAI,EAAE,MAAM,EACZ,OAAO,GAAE;QAAE,SAAS,CAAC,EAAE,OAAO,CAAC;QAAC,QAAQ,CAAC,EAAE,OAAO,CAAA;KAAO,GACxD,OAAO,CAAC,IAAI,CAAC;IAKhB;;;;;;;;;;;;;;;;;;;;;;OAsBG;IACG,GAAG,IAAI,OAAO,CAAC,QAAQ,CAAC;IAU9B;;;;;;;;;;;;;;;OAeG;IACG,KAAK,IAAI,OAAO,CAAC,QAAQ,CAAC;CAkGjC"}
+{"version":3,"file":"SimpleDB.d.ts","sourceRoot":"","sources":["../../src/class/SimpleDB.ts"],"names":[],"mappings":"AAGA,OAAO,WAAW,MAAM,kBAAkB,CAAC;AAG3C,OAAO,MAAM,MAAM,aAAa,CAAC;AA6BjC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAiCG;AAEH,MAAM,CAAC,OAAO,OAAO,QAAQ,CAAC,KAAK,SAAS,WAAW,GAAG,WAAW,CACnE,SAAQ,MAAM;;IACd;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAiEG;IACH,QAAQ,CAAC,gBAAgB,EAAE,IAAI,GAAG,KAAK,CAAC;IACxC;;;;;;;;;OASG;IACH,MAAM,EAAE,OAAO,CAAC;IAChB;;;;;;;;;OASG;IACH,UAAU,EAAE,OAAO,CAAC;IACpB;;;;;OAKG;IACH,gBAAgB,EAAE,MAAM,EAAE,CAAC;IAC3B;;;;;OAKG;IACH,aAAa,EAAE,MAAM,GAAG,SAAS,CAAC;IAClC;;;;;OAKG;IACH,cAAc,EAAE,MAAM,CAAC;IACvB;;;;;OAKG;IACH,WAAW,EAAE,OAAO,CAAC;IACrB;;;;;;OAMG;IACH,YAAY,EAAE,OAAO,CAAC;IACtB;;;;;OAKG;IACH,cAAc,EAAE,MAAM,CAAC;IACvB;;;;;OAKG;IACH,gBAAgB,EAAE,MAAM,CAAC;IACzB;;;;;OAKG;IACH,WAAW,EAAE,OAAO,CAAC;IACrB;;;;;;;OAOG;IACH,YAAY,EAAE,OAAO,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC;IAInC;;;;OAIG;IACH,cAAc,EAAE,MAAM,GAAG,SAAS,GAAG,QAAQ,CAAC;IAC9C;;;;;;;OAOG;IACH,YAAY,EAAE,MAAM,CAAC;IACrB;;;;;;;OAOG;IACH,UAAU,EAAE,MAAM,CAAC;IACnB;;;;;;OAMG;IACH,aAAa,EAAE,OAAO,CAAC;IACvB;;;;;OAKG;IACH,WAAW,EAAE,OAAO,GAAG,IAAI,CAAC;IAC5B;;;;;OAKG;IACH,WAAW,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC;;;;;OAKG;IACH,OAAO,EAAE,MAAM,GAAG,SAAS,CAAC;IAC5B;;;;;;OAMG;IACH,IAAI,EAAE,MAAM,CAAC;IACb;;;;;OAKG;IACH,SAAS,EAAE,OAAO,CAAC;IACnB;;;;;;;;;;;;;OAaG;IACH,QAAQ,EAAE,OAAO,CAAC;IAClB;;;;;;;;;;;;;;;;;;;;;;;;;;OA0BG;IACH,UAAU,EAAE,KACV,IAAI,EAAE,MAAM,EACZ,GAAG,EAAE,QAAQ,EACb,OAAO,CAAC,EAAE;QACR,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,UAAU,CAAC,EAAE,OAAO,CAAC;KACtB,KACE,KAAK,CAQC;IAEX;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAmCG;gBAED,OAAO,GAAE;QACP,IAAI,CAAC,EAAE,MAAM,CAAC;QACd,SAAS,CAAC,EAAE,OAAO,CAAC;QACpB,QAAQ,CAAC,EAAE,OAAO,CAAC;QACnB,WAAW,CAAC,EAAE,OAAO,CAAC;QACtB,SAAS,CAAC,EAAE,MAAM,CAAC;QACnB,UAAU,CAAC,EAAE,MAAM,CAAC;QACpB,UAAU,CAAC,EAAE,OAAO,CAAC;QACrB,YAAY,CAAC,EAAE,OAAO,CAAC;QACvB,gBAAgB,CAAC,EAAE,IAAI,GAAG,KAAK,CAAC;QAChC,MAAM,CAAC,EAAE,OAAO,CAAC;QACjB,UAAU,CAAC,EAAE,OAAO,CAAC;QACrB,WAAW,CAAC,EAAE,OAAO,GAAG,IAAI,CAAC;QAC7B,WAAW,CAAC,EAAE,OAAO,CAAC;QACtB,WAAW,CAAC,EAAE,MAAM,CAAC;QACrB,OAAO,CAAC,EAAE,MAAM,CAAC;KACb;IAuCR;;;;;;OAMG;IACG,KAAK,IAAI,OAAO,CAAC,QAAQ,CAAC;IAqGhC;;;;;;;;;;;;;;;;;;;;;;;;;OAyBG;IACH,QAAQ,CACN,IAAI,CAAC,EAAE,MAAM,GACZ,KAAK;IAuCR;;;;;;;;;;;;;OAaG;IACG,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;IAiB5C;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAgCG;IACG,YAAY,CAChB,MAAM,EAAE,KAAK,GAAG,MAAM,GAAG,CAAC,KAAK,GAAG,MAAM,CAAC,EAAE,GAC1C,OAAO,CAAC,IAAI,CAAC;IAKhB;;;;;;;;;;;;;;;;;;;;;;;;;;OA0BG;IACG,YAAY,CAChB,MAAM,EAAE,KAAK,GAAG,MAAM,GAAG,CAAC,KAAK,GAAG,MAAM,CAAC,EAAE,GAC1C,OAAO,CAAC,IAAI,CAAC;IAKhB;;;;;;;;;;;;OAYG;IACG,aAAa,IAAI,OAAO,CAAC,MAAM,EAAE,CAAC;IAIxC;;;;;;;;;;;;OAYG;IACG,aAAa,IAAI,OAAO,CAAC,IAAI,CAAC;IAYpC;;;;;;;;;;;;;;OAcG;IACH,SAAS,IAAI,SAAS,KAAK,EAAE;IAI7B;;;;;;;;;;;;;;;;;;;;;OAqBG;IACG,QAAQ,CAAC,KAAK,EAAE,KAAK,GAAG,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC;IAMvD;;;;;;;;;;;;OAYG;IACG,aAAa,IAAI,OAAO,CAC5B;QACE,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;KACxB,EAAE,CACJ;IAeD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAkCG;IACG,WAAW,CACf,KAAK,EAAE,MAAM,EACb,OAAO,GAAE;QACP,UAAU,CAAC,EAAE,OAAO,CAAC;QACrB,KAAK,CAAC,EAAE,MAAM,CAAC;KACX,GACL,OAAO,CACN;QACA,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC;KACxB,EAAE,GACD,IAAI,CACP;IAaD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAmCG;IACG,MAAM,CAAC,IAAI,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC;IAKzC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAuCG;IACG,OAAO,CACX,IAAI,EAAE,MAAM,EACZ,OAAO,GAAE;QAAE,SAAS,CAAC,EAAE,OAAO,CAAC;QAAC,QAAQ,CAAC,EAAE,OAAO,CAAA;KAAO,GACxD,OAAO,CAAC,IAAI,CAAC;IAKhB;;;;;;;;;;;;;;;;;;;;;;OAsBG;IACG,GAAG,IAAI,OAAO,CAAC,QAAQ,CAAC;IAU9B;;;;;;;;;;;;;;;OAeG;IACG,KAAK,IAAI,OAAO,CAAC,QAAQ,CAAC;CAkGjC"}

package/esm/class/SimpleDB.js

@@ -89,9 +89,10 @@ class SimpleDB extends Simple {
      * @param options.readOnly - Opens an existing DuckDB file read-only. Defaults to false. Requires a file and cannot be combined with overwrite.
      * @param options.logDuration - A flag indicating whether to log the total execution duration.
      * @param options.rowsToLog - The number of rows to display when logging a table.
-     * @param options.charsToLog - The maximum number of characters to display for text-based cells.
+     * @param options.charsToLog - The maximum characters to display for text and stringified nested cells, including truncation markers (default: 75). Independent of column width. Use Infinity for unlimited text.
      * @param options.typesToLog - A flag indicating whether to include data types when logging a table.
      * @param options.cacheVerbose - Whether to log cache hits and misses, code and input changes, TTL status, and cache read/write timing.
+     * @param options.expressionSyntax - Operator syntax for expressions and custom queries: `"js"` (default) translates JavaScript-style operators in SQL expressions; `"sql"` preserves SQL unchanged.
      * @param options.logSQL - A flag indicating whether to log SQL immediately before execution.
      * @param options.explainSQL - A flag indicating whether to log DuckDB query plans for supported statements.
      * @param options.duckDbCache - A flag indicating whether to use DuckDB's external file cache.
@@ -111,6 +112,78 @@ class SimpleDB extends Simple {
     constructor(options = {}) {
         super(options);
         _SimpleDB_instances.add(this);
+        /**
+         * The operator syntax used by expressions and custom queries in this database.
+         * `"js"` translates JavaScript-style operators (`&&`, `||`, `==`, `===`,
+         * `!==`) and null comparisons within SQL expressions. It does not evaluate
+         * arbitrary JavaScript. `||` remains concatenation when string literals, list
+         * literals, explicit text casts, or `concat()` results make that intent clear.
+         * Otherwise `||` becomes OR; column types are not inferred. Use `concat()`
+         * when concatenation would otherwise be ambiguous, such as `first || last`.
+         * `"sql"` passes SQL through unchanged, including `||` and null comparisons.
+         * Choose the syntax when constructing the database; it applies to all tables.
+         *
+         * @defaultValue `"js"`
+         * @category Properties
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB({ expressionSyntax: "js" }); // The default mode
+         * // || becomes SQL OR: keep rows where at least one condition is true.
+         * await sdb.newTable().loadArray([{ active: true, admin: false }])
+         *   .filter("active || admin").log();
+         * ```
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB({ expressionSyntax: "js" });
+         * // && becomes SQL AND: both conditions must be true.
+         * // === becomes SQL =: compare the age with 18.
+         * await sdb.newTable().loadArray([{ active: true, age: 18 }])
+         *   .filter("active && age === 18").log();
+         * ```
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB({ expressionSyntax: "js" });
+         * // SQL AND and OR also work in JS mode; they pass through unchanged.
+         * // Keep active adults, or anyone who is an admin.
+         * await sdb.newTable().loadArray([{ active: true, age: 18, admin: false }])
+         *   .filter("(active AND age >= 18) OR admin").log();
+         * ```
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB({ expressionSyntax: "js" });
+         * // Bare column names do not establish concatenation. Use concat() explicitly.
+         * await sdb.newTable().loadArray([{ first: "Jane", last: "Doe" }])
+         *   .addColumn("name", "string", "concat(first, ' ', last)").log();
+         * ```
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB(); // JS mode is the default
+         * // The string literal ' ' makes concatenation clear, so both || stay unchanged.
+         * await sdb.newTable().loadArray([{ first: "Jane", last: "Doe" }])
+         *   .addColumn("name", "string", "first || ' ' || last").log();
+         * ```
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB({ expressionSyntax: "sql" });
+         * // SQL mode preserves || as concatenation: the name becomes "Jane Doe".
+         * await sdb.newTable().loadArray([{ first: "Jane", last: "Doe" }])
+         *   .addColumn("name", "string", "first || ' ' || last").log();
+         * ```
+         * @example
+         * ```ts
+         * const sdb = new SimpleDB({ expressionSyntax: "sql" });
+         * // Use SQL AND and OR for logical conditions; && and || are not translated.
+         * // Both active and adult must be true, unless admin is true.
+         * await sdb.newTable().loadArray([{ active: true, age: 18, admin: false }])
+         *   .filter("(active AND age >= 18) OR admin").log();
+         * ```
+         */
+        Object.defineProperty(this, "expressionSyntax", {
+            enumerable: true,
+            configurable: true,
+            writable: true,
+            value: void 0
+        });
         /**
          * Whether to log each SQL statement immediately before execution.
          *
@@ -423,6 +496,11 @@ class SimpleDB extends Simple {
             writable: true,
             value: SimpleTable
         });
+        if (options.expressionSyntax !== undefined &&
+            !["js", "sql"].includes(options.expressionSyntax)) {
+            throw new Error('expressionSyntax must be "js" or "sql".');
+        }
+        this.expressionSyntax = options.expressionSyntax ?? "js";
         this.file = options.file ?? ":memory:";
         this.logSQL = options.logSQL ?? false;
         this.explainSQL = options.explainSQL ?? false;
@@ -475,6 +553,7 @@ class SimpleDB extends Simple {
     }
     /**
      * Creates a new SimpleTable instance within the database.
+     * The name must be unique among registered tables, ignoring ASCII letter case.
      *
      * @param name - The name of the new table. If not provided, a default name is generated (e.g., "table1").
      * @returns A new table instance.
@@ -728,6 +807,7 @@ class SimpleDB extends Simple {
     }
     /**
      * Executes a custom SQL query directly against the DuckDB instance.
+     * With the default `SimpleDB.expressionSyntax: "js"`, queries support JavaScript-style operators (`&&`, `||`, `===`, `!==`). Set `expressionSyntax: "sql"` for unchanged SQL.
      * Queries run in UTC. When data is returned, temporal values use the same
      * JavaScript representations as `SimpleTable.getData()`.
      *