@rip-lang/db 1.3.5 → 1.3.6

package/README.md

@@ -2,13 +2,12 @@
 
 # Rip DB - @rip-lang/db
 
-> **A lightweight DuckDB HTTP server with the official DuckDB UI built in**
+> **A lightweight DuckDB HTTP server with bulk inserts, an ActiveRecord-style client, and the official DuckDB UI built in**
 
 Rip DB turns any DuckDB database into a full-featured HTTP server — complete
-with the official DuckDB UI for interactive queries, notebooks, and data
-exploration. It connects to DuckDB via pure Bun FFI (no npm packages, no
-native build step) and implements DuckDB's binary serialization protocol
-to power the UI with native-speed data transfer.
+with the official DuckDB UI for interactive queries, bulk insert via DuckDB's
+Appender API (~200K rows/sec), and a clean Model interface that picks the
+optimal strategy automatically. Pure Bun FFI, zero npm dependencies for DuckDB.
 
 ## Quick Start
 
@@ -23,8 +22,20 @@ rip-db mydata.duckdb            # File-based database
 rip-db mydata.duckdb --port 8080
 ```
 
+```
+rip-db: DuckDB v1.4.4
+rip-db: rip-db v1.3.5
+rip-db: source mydata.duckdb
+rip-db: server http://localhost:4213
+```
+
 Open **http://localhost:4213** for the official DuckDB UI.
 
+The `source` line shows the active data source — today that's a local DuckDB
+file or `:memory:`, but the architecture supports any source DuckDB can attach:
+S3 buckets, PostgreSQL, MySQL, SQLite, Parquet files, CSV, and more via
+DuckDB's extension system.
+
 ## What It Does
 
 Rip DB sits between your clients and DuckDB, providing two interfaces:
@@ -47,77 +58,119 @@ Rip DB sits between your clients and DuckDB, providing two interfaces:
 **DuckDB UI** — The official DuckDB notebook interface loads instantly in your
 browser. Rip DB proxies the UI assets from ui.duckdb.org and implements the
 full binary serialization protocol that the UI uses to communicate with DuckDB.
-This includes query execution, SQL tokenization for syntax highlighting, and
-Server-Sent Events for real-time catalog updates.
 
 **JSON API** — Any HTTP client can execute SQL queries and receive JSON
-responses. Use it from curl, your application code, or any language that
-speaks HTTP.
+responses. Three execution strategies are selected automatically based on the
+request shape — the caller never needs to think about it.
 
 ## Features
 
 - **Official DuckDB UI** — Interactive notebooks, syntax highlighting, data exploration
+- **Bulk insert via Appender API** — ~200K rows/sec, bypasses SQL parsing entirely
+- **Batch prepared statements** — Prepare once, execute N times with different params
+- **ActiveRecord-style Model** — `User.find!`, `User.insert!`, `User.where(...).all!`
+- **Smart dispatch** — `Model.insert!` picks Appender for arrays, prepared statements for singles
 - **Full binary protocol** — Native DuckDB UI serialization implemented in Rip
 - **Pure Bun FFI** — Direct calls to DuckDB's C API using the modern chunk-based interface
 - **Zero npm dependencies for DuckDB** — Uses the system-installed DuckDB library
 - **Parameterized queries** — Prepared statements with type-safe parameter binding
 - **Complete type support** — All DuckDB types handled natively, including UUID, DECIMAL, TIMESTAMP, LIST, STRUCT, MAP
-- **DECIMAL precision preserved** — Exact string representation, never converted to floating point
-- **Timestamps as UTC** — All timestamps returned as JavaScript Date objects (UTC)
-- **Powered by @rip-lang/api** — Fast, lightweight HTTP server framework
 - **Single binary** — One `rip-db` command, one process, one database
 
-## JSON API
+## Database Client
 
-For programmatic access from any HTTP client.
+The real power of Rip DB is its client library. Import it from
+`@rip-lang/db/client` — it talks to a running `rip-db` server over HTTP.
 
-### POST /sql
+```coffee
+import { query, findOne, findAll, Model } from '@rip-lang/db/client'
+```
 
-Execute SQL with optional parameters:
+### The Balance: Model vs Raw SQL
 
-```bash
-curl -X POST http://localhost:4213/sql \
-  -H "Content-Type: application/json" \
-  -d '{"sql": "SELECT * FROM users WHERE id = $1", "params": [1]}'
-```
+Not every query needs an ORM, and not every query benefits from raw SQL.
+Rip DB gives you both and lets you choose the right tool:
 
-### POST /
+**Use the Model** for simple and medium queries — CRUD, where clauses,
+counts, upserts. The Model is shorter, safer, and handles parameterization
+automatically:
 
-Execute raw SQL (body is the query):
+```coffee
+User = Model 'users'
 
-```bash
-curl -X POST http://localhost:4213/ -d "SELECT 42 as answer"
+# These are cleaner than raw SQL
+user    = User.find! 42
+count   = User.count!
+active  = User.where(active: true).order('name').limit(10).all!
+created = User.insert! { name: 'Alice', email: 'alice@example.com' }
+User.upsert! { email: 'alice@example.com', name: 'Alice' }, on: 'email'
 ```
 
-Response format:
+**Use raw SQL** for complex queries — JOINs, GROUP BY, aggregates, subqueries.
+SQL is the most direct, readable expression for these. No ORM improves on it:
 
-```json
-{
-  "meta": [{"name": "answer", "type": "INTEGER"}],
-  "data": [[42]],
-  "rows": 1,
-  "time": 0.001
-}
+```coffee
+users = findAll! """
+  SELECT u.id, u.name, count(o.id) as order_count
+  FROM users u
+  LEFT JOIN orders o ON o.user_id = u.id
+  WHERE u.active = true
+  GROUP BY u.id, u.name
+  ORDER BY order_count DESC
+  """
 ```
 
-### Other Endpoints
+This isn't a compromise — it's the optimal approach. Simple queries get
+shorter with the Model. Complex queries stay clear with SQL. You never
+fight the abstraction.
 
-| Endpoint | Method | Description |
-|----------|--------|-------------|
-| `/health` | GET | Health check |
-| `/tables` | GET | List all tables |
-| `/schema/:table` | GET | Table schema |
+### Bulk Insert (Appender API)
 
-## Database Client
+Pass an array to `Model.insert!` and it automatically uses DuckDB's Appender
+API — the fastest possible insert path (~200K rows/sec). The Appender bypasses
+SQL parsing entirely, writing directly to DuckDB's columnar storage.
+
+```coffee
+# Single insert — uses prepared statement, returns the row
+user = User.insert! { name: 'Alice', email: 'alice@example.com' }
+
+# Bulk insert — uses Appender API, fastest path
+User.insert! [
+  { name: 'Alice', email: 'alice@example.com' }
+  { name: 'Bob', email: 'bob@example.com' }
+  { name: 'Charlie', email: 'charlie@example.com' }
+]
+```
+
+The caller writes the same `insert!` — the Model detects the array and
+picks the optimal strategy. Column subsets work too; missing columns get
+their default values.
 
-Rip DB includes an ActiveRecord-style database client for use in Rip
-applications. Import it from `@rip-lang/db/client` — it talks to a running
-`rip-db` server over HTTP with parameterized queries.
+### Bulk Upsert (Multi-Row VALUES)
+
+For upserts (INSERT ... ON CONFLICT), the Appender can't be used. The Model
+builds a multi-row VALUES statement with proper parameterization:
 
 ```coffee
-import { connect, query, findOne, findAll, Model } from '@rip-lang/db/client'
+# Single upsert
+Response.upsert! { email: 'alice@example.com', name: 'Alice' }, on: 'email'
+
+# Bulk upsert — one SQL statement with N value tuples
+Response.upsert! responses, on: 'email'
+```
+
+### Batch Queries (Prepared Statement Reuse)
+
+Pass an array of param arrays to `query!` and it reuses one prepared
+statement for all executions — one prepare, N bind-and-execute cycles:
 
-connect 'http://localhost:4213'   # optional — defaults to DB_URL env or localhost:4213
+```coffee
+# Execute the same UPDATE 3 times with different params
+query! "UPDATE reviews SET completed_at = $1 WHERE id = $2", [
+  [now, 1]
+  [now, 2]
+  [now, 3]
+]
 ```
 
 ### Low-Level Queries
@@ -177,11 +230,9 @@ User.where('age > $1', [21]).all!
 
 # OR conditions
 User.where(active: true).or(role: 'admin').all!
-User.where(active: true).or('role = $1', ['admin']).all!
 
 # NOT conditions
 User.where(active: true).not(role: 'banned').all!
-User.not(deleted_at: null).all!            # WHERE "deleted_at" IS NOT NULL
 ```
 
 #### Group & Having
@@ -196,13 +247,16 @@ User.group('role').having('count(*) > $1', [5]).select('role, count(*) as n').al
 All mutations return the affected row(s) via `RETURNING *`.
 
 ```coffee
-# Insert — returns the new record
+# Insert single — returns the new record
 user = User.insert! { first_name: 'Alice', email: 'alice@example.com' }
 
+# Insert bulk — uses Appender API (~200K rows/sec)
+User.insert! rows
+
 # Update by id — returns the updated record
 user = User.update! 42, { email: 'newemail@example.com' }
 
-# Upsert — insert or update on conflict
+# Upsert — insert or update on conflict (single or bulk)
 user = User.upsert! { email: 'alice@example.com', name: 'Alice' }, on: 'email'
 
 # Destroy by id — returns the deleted record
@@ -238,6 +292,20 @@ Archive = Model 'orders', 'archive_db'
 order = Archive.find! 99    # SELECT * FROM "archive_db"."orders" WHERE id = $1
 ```
 
+### Execution Strategy Summary
+
+The client picks the optimal execution path automatically:
+
+| Caller writes | Strategy | Speed |
+|---------------|----------|-------|
+| `Model.insert!(object)` | Prepared statement | Fast |
+| `Model.insert!(array)` | DuckDB Appender API | ~200K rows/sec |
+| `Model.upsert!(object)` | Prepared statement | Fast |
+| `Model.upsert!(array)` | Multi-row VALUES SQL | Fast (batch) |
+| `query!(sql, params)` | Prepared statement | Fast |
+| `query!(sql, [params...])` | Prepared stmt reuse | Fast (batch) |
+| `findOne!(sql)` / `findAll!(sql)` | Direct execution | Fast |
+
 ### Query Builder Reference
 
 | Method | Description |
@@ -274,12 +342,60 @@ order = Archive.find! 99    # SELECT * FROM "archive_db"."orders" WHERE id = $1
 | `Model.order(...)` | Start a chain with ORDER BY |
 | `Model.group(...)` | Start a chain with GROUP BY |
 | `Model.limit(n)` | Start a chain with LIMIT |
-| `Model.insert!(data)` | Insert and return new row |
+| `Model.insert!(data)` | Insert single object or bulk array |
 | `Model.update!(id, data)` | Update by id and return row |
-| `Model.upsert!(data, on:)` | Insert or update on conflict |
+| `Model.upsert!(data, on:)` | Insert or update on conflict (single or bulk) |
 | `Model.destroy!(id)` | Delete by id and return row |
 | `Model.query!(sql, params)` | Raw parameterized query |
 
+## JSON API
+
+For programmatic access from any HTTP client.
+
+### POST /sql
+
+The `/sql` endpoint accepts four shapes and dispatches automatically:
+
+```bash
+# Standard query
+curl -X POST http://localhost:4213/sql \
+  -H "Content-Type: application/json" \
+  -d '{"sql": "SELECT * FROM users WHERE id = $1", "params": [1]}'
+
+# Bulk insert (Appender API)
+curl -X POST http://localhost:4213/sql \
+  -H "Content-Type: application/json" \
+  -d '{"table": "users", "columns": ["name", "email"], "rows": [["Alice", "a@b.com"], ["Bob", "b@b.com"]]}'
+
+# Batch prepared statement
+curl -X POST http://localhost:4213/sql \
+  -H "Content-Type: application/json" \
+  -d '{"sql": "INSERT INTO t (a, b) VALUES ($1, $2)", "params": [[1, "x"], [2, "y"]]}'
+```
+
+| Shape | Dispatches to |
+|-------|---------------|
+| `{ sql }` | Raw execution |
+| `{ sql, params: [...] }` | Prepared statement |
+| `{ sql, params: [[...], ...] }` | Batch prepared (reuse stmt) |
+| `{ table, columns, rows }` | Appender API (fastest insert) |
+
+### POST /
+
+Execute raw SQL (body is the query):
+
+```bash
+curl -X POST http://localhost:4213/ -d "SELECT 42 as answer"
+```
+
+### Other Endpoints
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/health` | GET | Health check |
+| `/tables` | GET | List all tables |
+| `/schema/:table` | GET | Table schema |
+
 ## DuckDB UI
 
 The official DuckDB UI is available at the root URL. It provides:
@@ -299,19 +415,18 @@ Rip DB is built from three files:
 
 | File | Lines | Role |
 |------|-------|------|
-| `db.rip` | ~390 | HTTP server — routes, middleware, UI proxy |
-| `lib/duckdb.mjs` | ~800 | FFI driver — modern chunk-based DuckDB C API |
+| `db.rip` | ~430 | HTTP server — routes, middleware, UI proxy, bulk dispatch |
+| `lib/duckdb.mjs` | ~960 | FFI driver — chunk-based API, Appender, batch prepared |
 | `lib/duckdb-binary.rip` | ~550 | Binary serializer — DuckDB UI protocol |
+| `client.rip` | ~320 | HTTP client — Model factory, query builder, bulk insert |
 
 The FFI driver uses DuckDB's modern chunk-based API (`duckdb_fetch_chunk`,
 `duckdb_vector_get_data`) to read query results directly from columnar memory.
-No deprecated per-value functions, no intermediate copies. For complex types
-like DECIMAL, ENUM, LIST, and STRUCT, it uses DuckDB's logical type
-introspection to read values with full fidelity.
-
-The binary serializer implements the same wire protocol that DuckDB's official
-UI extension uses. It handles all DuckDB types including native 16-byte UUID
-serialization, uint64-aligned validity bitmaps, and proper timestamp encoding.
+For bulk inserts, it uses the Appender API (`duckdb_appender_create`,
+`duckdb_append_*`) which writes directly to DuckDB's storage engine, bypassing
+SQL parsing for maximum throughput. Prepared statement reuse
+(`duckdb_prepare` once, `duckdb_execute_prepared` N times) handles batch
+operations efficiently.
 
 ## Requirements
 

package/db.rip

@@ -102,8 +102,9 @@ port = parseInt(process.env.DB_PORT or portArg) or 4213
 # Open database and create persistent connection
 db = open(path)
 conn = db.connect()
-console.log "rip-db: database=#{path} (bun-ffi)"
-console.log "rip-db: DuckDB version=#{duckdbVersion()}"
+console.log "rip-db: DuckDB #{duckdbVersion()}"
+console.log "rip-db: rip-db v#{VERSION}"
+console.log "rip-db: source #{path}"
 
 # ==============================================================================
 # Helpers
@@ -415,7 +416,6 @@ get '/*' ->
 # Start Server
 # ==============================================================================
 
-start port: port
+console.log "rip-db: server http://localhost:#{port}"
 
-console.log "rip-db: listening on http://localhost:#{port}"
-console.log "rip-db: DuckDB UI available at http://localhost:#{port}/"
+start port: port, silent: true

package/lib/duckdb.mjs

@@ -92,25 +92,25 @@ const lib = dlopen(libPath, {
   duckdb_clear_bindings:   { args: ['ptr'], returns: 'i32' },
 
   // Appender API
-  duckdb_appender_create:  { args: ['ptr', 'ptr', 'ptr', 'ptr'], returns: 'i32' },
-  duckdb_appender_error:   { args: ['ptr'], returns: 'ptr' },
-  duckdb_appender_flush:   { args: ['ptr'], returns: 'i32' },
-  duckdb_appender_close:   { args: ['ptr'], returns: 'i32' },
-  duckdb_appender_destroy: { args: ['ptr'], returns: 'i32' },
-  duckdb_appender_end_row: { args: ['ptr'], returns: 'i32' },
-  duckdb_append_bool:      { args: ['ptr', 'bool'], returns: 'i32' },
-  duckdb_append_int32:     { args: ['ptr', 'i32'], returns: 'i32' },
-  duckdb_append_int64:     { args: ['ptr', 'i64'], returns: 'i32' },
-  duckdb_append_double:    { args: ['ptr', 'f64'], returns: 'i32' },
-  duckdb_append_varchar:   { args: ['ptr', 'ptr'], returns: 'i32' },
-  duckdb_append_null:          { args: ['ptr'], returns: 'i32' },
+  duckdb_appender_create:        { args: ['ptr', 'ptr', 'ptr', 'ptr'], returns: 'i32' },
+  duckdb_appender_error:         { args: ['ptr'], returns: 'ptr' },
+  duckdb_appender_flush:         { args: ['ptr'], returns: 'i32' },
+  duckdb_appender_close:         { args: ['ptr'], returns: 'i32' },
+  duckdb_appender_destroy:       { args: ['ptr'], returns: 'i32' },
+  duckdb_appender_end_row:       { args: ['ptr'], returns: 'i32' },
+  duckdb_append_bool:            { args: ['ptr', 'bool'], returns: 'i32' },
+  duckdb_append_int32:           { args: ['ptr', 'i32'], returns: 'i32' },
+  duckdb_append_int64:           { args: ['ptr', 'i64'], returns: 'i32' },
+  duckdb_append_double:          { args: ['ptr', 'f64'], returns: 'i32' },
+  duckdb_append_varchar:         { args: ['ptr', 'ptr'], returns: 'i32' },
+  duckdb_append_null:            { args: ['ptr'], returns: 'i32' },
   duckdb_appender_add_column:    { args: ['ptr', 'ptr'], returns: 'i32' },
   duckdb_appender_clear_columns: { args: ['ptr'], returns: 'i32' },
 
   // Result inspection
-  duckdb_column_count: { args: ['ptr'], returns: 'u64' },
-  duckdb_column_name:  { args: ['ptr', 'u64'], returns: 'ptr' },
-  duckdb_column_type:  { args: ['ptr', 'u64'], returns: 'i32' },
+  duckdb_column_count:  { args: ['ptr'], returns: 'u64' },
+  duckdb_column_name:   { args: ['ptr', 'u64'], returns: 'ptr' },
+  duckdb_column_type:   { args: ['ptr', 'u64'], returns: 'i32' },
   duckdb_result_error:  { args: ['ptr'], returns: 'ptr' },
 
   // Modern chunk-based API (non-deprecated)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@rip-lang/db",
-  "version": "1.3.5",
+  "version": "1.3.6",
   "description": "DuckDB server with official DuckDB UI — pure Bun FFI",
   "type": "module",
   "main": "db.rip",