@rip-lang/db 0.10.0 → 1.0.2

package/README.md

@@ -2,320 +2,159 @@
 
 # Rip DB - @rip-lang/db
 
-> **Simple RestFUL HTTP server for concurrent DuckDB read and write queries**
+> **A lightweight DuckDB HTTP server with the official DuckDB UI built in**
 
-Uses custom Zig-based DuckDB bindings (`lib/`) for fast, type-safe database access.
+Rip DB turns any DuckDB database into a full-featured HTTP server — complete
+with the official DuckDB UI for interactive queries, notebooks, and data
+exploration. It connects to DuckDB via pure Bun FFI (no npm packages, no
+native build step) and implements DuckDB's binary serialization protocol
+to power the UI with native-speed data transfer.
 
-## Overview
+## Quick Start
 
 ```bash
-# Install globally
-bun add -g @rip-lang/db
+# Install DuckDB and Rip DB
+brew install duckdb            # macOS (or see duckdb.org for Linux)
+bun add -g @rip-lang/db        # Installs rip-db command
 
-# Run with in-memory database
-rip-db
-
-# Run with file-based database
-rip-db mydb.duckdb
-
-# Specify port
-rip-db mydb.duckdb --port=8080
-```
-
-Or run directly with Rip:
-
-```bash
-rip db.rip :memory: --port=4000
+# Start the server
+rip-db                          # In-memory database
+rip-db mydata.duckdb            # File-based database
+rip-db mydata.duckdb --port 8080
 ```
 
-## API Endpoints
+Open **http://localhost:4213** for the official DuckDB UI.
 
-### POST /sql
+## What It Does
 
-Execute any SQL statement (queries or mutations) with JSON body.
+Rip DB sits between your clients and DuckDB, providing two interfaces:
 
-**Request:**
-```json
-{
-  "sql": "SELECT * FROM users WHERE id = ?",
-  "params": [1]
-}
 ```
-
-**Response (JSONCompact format):**
-```json
-{
-  "meta": [
-    {"name": "id", "type": "INTEGER"},
-    {"name": "name", "type": "VARCHAR"},
-    {"name": "email", "type": "VARCHAR"}
-  ],
-  "data": [
-    [1, "Alice", "alice@example.com"]
-  ],
-  "rows": 1,
-  "time": 0.001
-}
+┌─────────────┐          ┌─────────────┐          ┌─────────────┐
+│  DuckDB UI  │  binary  │   rip-db    │   FFI    │   DuckDB    │
+│  (Browser)  │◀────────▶│   (Bun)     │◀────────▶│  (native)   │
+└─────────────┘          └─────────────┘          └─────────────┘
+                               ▲
+                               │ HTTP/S
+                               │ (JSON)
+                               ▼
+                      ┌─────────────────┐
+                      │  HTTP Clients   │
+                      │  (curl, apps)   │
+                      └─────────────────┘
 ```
 
-### POST /
+**DuckDB UI** — The official DuckDB notebook interface loads instantly in your
+browser. Rip DB proxies the UI assets from ui.duckdb.org and implements the
+full binary serialization protocol that the UI uses to communicate with DuckDB.
+This includes query execution, SQL tokenization for syntax highlighting, and
+Server-Sent Events for real-time catalog updates.
 
-Execute SQL with raw body (duck-ui compatible). Used by [duck-ui](https://demo.duckui.com).
+**JSON API** — Any HTTP client can execute SQL queries and receive JSON
+responses. Use it from curl, your application code, or any language that
+speaks HTTP.
 
-**Request:**
-```
-POST / HTTP/1.1
-Content-Type: application/x-www-form-urlencoded
+## Features
 
-SELECT * FROM users WHERE id = 1
-```
-
-**Response:** Same JSONCompact format as `/sql`.
-
-### GET /health
+- **Official DuckDB UI** — Interactive notebooks, syntax highlighting, data exploration
+- **Full binary protocol** — Native DuckDB UI serialization implemented in Rip
+- **Pure Bun FFI** — Direct calls to DuckDB's C API using the modern chunk-based interface
+- **Zero npm dependencies for DuckDB** — Uses the system-installed DuckDB library
+- **Parameterized queries** — Prepared statements with type-safe parameter binding
+- **Complete type support** — All DuckDB types handled natively, including UUID, DECIMAL, TIMESTAMP, LIST, STRUCT, MAP
+- **DECIMAL precision preserved** — Exact string representation, never converted to floating point
+- **Timestamps as UTC** — All timestamps returned as JavaScript Date objects (UTC)
+- **Powered by @rip-lang/api** — Fast, lightweight HTTP server framework
+- **Single binary** — One `rip-db` command, one process, one database
 
-Simple health check (no database query).
-
-```json
-{ "ok": true }
-```
-
-### GET /status
-
-Database info including table list.
-
-```json
-{
-  "ok": true,
-  "database": "mydb.duckdb",
-  "tables": ["users", "orders", "products"],
-  "time": "2026-02-02T14:30:00.000Z"
-}
-```
-
-### GET /tables
-
-List all tables in the database.
-
-```json
-{
-  "ok": true,
-  "tables": ["users", "orders", "products"]
-}
-```
-
-### GET /schema/:table
-
-Get schema for a specific table.
-
-```json
-{
-  "ok": true,
-  "table": "users",
-  "columns": [
-    { "column_name": "id", "data_type": "INTEGER", "is_nullable": "NO" },
-    { "column_name": "name", "data_type": "VARCHAR", "is_nullable": "YES" },
-    { "column_name": "email", "data_type": "VARCHAR", "is_nullable": "YES" }
-  ]
-}
-```
+## JSON API
 
-### GET /ui
+For programmatic access from any HTTP client.
 
-Built-in SQL console. Open in browser:
-
-```
-http://localhost:4000/ui
-```
-
-## DuckDB UI Compatibility
-
-rip-db implements the official DuckDB UI binary protocol, making it compatible with
-DuckDB's built-in UI. This means you can use the beautiful DuckDB UI with rip-db!
-
-### Binary Protocol Endpoints
-
-| Endpoint          | Method | Purpose                    |
-|-------------------|--------|----------------------------|
-| `/ddb/run`        | POST   | Execute SQL (binary result)|
-| `/ddb/interrupt`  | POST   | Cancel running query       |
-| `/ddb/tokenize`   | POST   | Syntax highlighting        |
-| `/info`           | GET    | Version info               |
-
-### How It Works
-
-```
-┌─────────────┐     Binary Protocol    ┌─────────────┐
-│  DuckDB UI  │ ◀─────────────────────▶│   rip-db    │
-│  (Browser)  │                        │   Server    │
-└─────────────┘                        └─────────────┘
-```
-
-The UI sends SQL to `/ddb/run`, rip-db executes it and returns results in DuckDB's
-binary format. The UI has no idea it's talking to rip-db instead of native DuckDB!
-
-For protocol details, see [PROTOCOL.md](./PROTOCOL.md).
-
-## Examples
-
-### Create a table
-
-```bash
-curl -X POST http://localhost:4000/sql \
-  -H "Content-Type: application/json" \
-  -d '{"sql": "CREATE TABLE users (id INTEGER PRIMARY KEY, name VARCHAR, email VARCHAR)"}'
-```
+### POST /sql
 
-### Insert data
+Execute SQL with optional parameters:
 
 ```bash
-curl -X POST http://localhost:4000/sql \
+curl -X POST http://localhost:4213/sql \
   -H "Content-Type: application/json" \
-  -d '{"sql": "INSERT INTO users VALUES (?, ?, ?)", "params": [1, "Alice", "alice@example.com"]}'
+  -d '{"sql": "SELECT * FROM users WHERE id = $1", "params": [1]}'
 ```
 
-### Query with parameters
-
-```bash
-curl -X POST http://localhost:4000/sql \
-  -H "Content-Type: application/json" \
-  -d '{"sql": "SELECT * FROM users WHERE name LIKE ?", "params": ["%Ali%"]}'
-```
+### POST /
 
-### Aggregations
+Execute raw SQL (body is the query):
 
 ```bash
-curl -X POST http://localhost:4000/sql \
-  -H "Content-Type: application/json" \
-  -d '{"sql": "SELECT COUNT(*) as total, AVG(age) as avg_age FROM users"}'
+curl -X POST http://localhost:4213/ -d "SELECT 42 as answer"
 ```
 
-## Response Format
-
-All responses use JSONCompact format (compatible with DuckDB HTTP Server and duck-ui):
-
-| Field | Type | Description |
-|-------|------|-------------|
-| `meta` | object[] | Column metadata: `[{name, type}, ...]` |
-| `data` | any[][] | Row data as arrays |
-| `rows` | number | Number of rows returned |
-| `time` | number | Execution time in seconds |
-| `error` | string | Error message (on failure) |
+Response format:
 
-**Success response:**
 ```json
 {
-  "meta": [{"name": "id", "type": "INTEGER"}, {"name": "name", "type": "VARCHAR"}],
-  "data": [[1, "Alice"], [2, "Bob"]],
-  "rows": 2,
+  "meta": [{"name": "answer", "type": "INTEGER"}],
+  "data": [[42]],
+  "rows": 1,
   "time": 0.001
 }
 ```
 
-**Error response:**
-```json
-{"error": "Table 'foo' does not exist"}
-```
-
-This format is compatible with [duck-ui](https://demo.duckui.com) and other DuckDB HTTP clients.
-
-## Performance
+### Other Endpoints
 
-Our custom Zig-based DuckDB bindings deliver exceptional performance:
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/health` | GET | Health check |
+| `/tables` | GET | List all tables |
+| `/schema/:table` | GET | Table schema |
 
-| Operation | Latency | Throughput |
-|-----------|---------|------------|
-| Point lookup (WHERE id=?) | 0.09ms | 11,000 qps |
-| Range scan (LIMIT 100) | 0.20ms | 5,000 qps |
-| Aggregation (COUNT/AVG/MAX) | 0.12ms | 8,400 qps |
-| JOIN + GROUP BY | 0.25ms | 4,000 qps |
-| INSERT (single row) | 0.13ms | 7,700 qps |
+## DuckDB UI
 
-### Comparison to MySQL/PostgreSQL
+The official DuckDB UI is available at the root URL. It provides:
 
-| Operation | Our DuckDB | MySQL/PG (localhost) | Speedup |
-|-----------|------------|----------------------|---------|
-| Point lookup | **0.09ms** | 0.3-1ms | 3-10x faster |
-| Range scan | **0.20ms** | 1-5ms | 5-25x faster |
-| Aggregation | **0.12ms** | 2-20ms | 17-170x faster |
-| JOIN + GROUP BY | **0.25ms** | 5-50ms | 20-200x faster |
-| INSERT | **0.13ms** | 0.5-2ms | 4-15x faster |
+- **SQL Notebooks** — Write and execute queries in a notebook interface
+- **Syntax Highlighting** — Real-time SQL tokenization as you type
+- **Data Exploration** — Browse tables, schemas, and query results
+- **Multiple Databases** — Attach and query across databases
 
-**Why so fast?**
-- **Zero network latency** — DuckDB runs in-process
-- **No connection overhead** — No auth, handshake, or protocol parsing
-- **Columnar engine** — DuckDB is optimized for analytical queries
-- **Direct FFI** — Zig bindings call DuckDB's C API directly
+The UI communicates with Rip DB using DuckDB's binary serialization protocol,
+which Rip DB implements in full. This means the UI works exactly as it does
+with the official `duckdb -ui` command — same features, same performance.
 
-## Architecture & Concurrency
+## How It Works
 
-### Single Process, High Concurrency
+Rip DB is built from three files:
 
-DuckDB only allows **one process** to access a database file at a time (for write access). This means rip-db runs as a **single process** — it does NOT use rip-server's multi-worker model.
+| File | Lines | Role |
+|------|-------|------|
+| `db.rip` | ~390 | HTTP server — routes, middleware, UI proxy |
+| `lib/duckdb.mjs` | ~800 | FFI driver — modern chunk-based DuckDB C API |
+| `lib/duckdb-binary.rip` | ~550 | Binary serializer — DuckDB UI protocol |
 
-However, rip-db handles **high concurrency** through two mechanisms:
+The FFI driver uses DuckDB's modern chunk-based API (`duckdb_fetch_chunk`,
+`duckdb_vector_get_data`) to read query results directly from columnar memory.
+No deprecated per-value functions, no intermediate copies. For complex types
+like DECIMAL, ENUM, LIST, and STRUCT, it uses DuckDB's logical type
+introspection to read values with full fidelity.
 
-```
-┌─────────────────────────────────────────────────────────┐
-│                   rip-db (single process)               │
-├─────────────────────────────────────────────────────────┤
-│   Bun HTTP Server (async I/O)                           │
-│   ├─ Request 1 ──┐                                      │
-│   ├─ Request 2 ──┼── thousands of concurrent            │
-│   ├─ Request 3 ──┤   connections via event loop         │
-│   └─ Request N ──┘                                      │
-├─────────────────────────────────────────────────────────┤
-│   rip-api (routes, middleware)                          │
-├─────────────────────────────────────────────────────────┤
-│   DuckDB (multi-threaded query engine)                  │
-│   └─ Queries parallelized across CPU cores              │
-└─────────────────────────────────────────────────────────┘
-```
-
-**Layer 1: Bun's Async I/O**
-- Single JavaScript thread with event loop
-- Handles thousands of concurrent HTTP connections
-- Non-blocking I/O — while one request waits on DuckDB, others are processed
-- No thread-per-request overhead
-
-**Layer 2: DuckDB's Multi-Threading**
-- Queries are parallelized across all CPU cores internally
-- MVCC for concurrent reads
-- Writes serialized automatically by DuckDB
-
-### Why Not Multi-Process?
-
-| Approach | Works with DuckDB? | Why |
-|----------|-------------------|-----|
-| Multi-process (rip-server) | ❌ No | DuckDB only allows one process with write access |
-| Multi-threaded | ✅ Yes | DuckDB handles this internally |
-| Async I/O | ✅ Yes | Bun's event loop handles concurrent connections |
-
-### The Result
-
-Even as a single process, rip-db can handle:
-- **Thousands of concurrent connections** (Bun async I/O)
-- **Parallel query execution** (DuckDB multi-threading)
-- **High throughput** (11,000+ queries/sec on typical hardware)
-
-This is the correct architecture for DuckDB — single process, high concurrency.
+The binary serializer implements the same wire protocol that DuckDB's official
+UI extension uses. It handles all DuckDB types including native 16-byte UUID
+serialization, uint64-aligned validity bitmaps, and proper timestamp encoding.
 
 ## Requirements
 
-- Bun 1.0+
-- rip-lang 2.0+
-- @rip-lang/api 0.5+
+- **Bun** 1.0+
+- **DuckDB** library installed on the system
+  - macOS: `brew install duckdb`
+  - Linux: Install from [duckdb.org](https://duckdb.org/docs/installation)
+- **rip-lang** 2.8+ (installed automatically as a dependency)
 
-## Building from Source
-
-To rebuild the native DuckDB bindings (requires Zig 0.15+ and DuckDB):
+Set `DUCKDB_LIB_PATH` if DuckDB is not in a standard location:
 
 ```bash
-./src/build.sh
+DUCKDB_LIB_PATH=/path/to/libduckdb.dylib rip-db
 ```
 
-This creates `lib/{platform}-{arch}/duckdb.node` for your platform.
-
 ## License
 
 MIT

package/bin/rip-db

@@ -1,17 +1,12 @@
 #!/usr/bin/env bun
 
-import { execSync } from 'child_process';
-import { fileURLToPath } from 'url';
+import { execFileSync } from 'child_process';
 import { dirname, join } from 'path';
 
-const __filename = fileURLToPath(import.meta.url);
-const __dirname = dirname(__filename);
-
-const dbRip = join(__dirname, '..', 'db.rip');
-const args = process.argv.slice(2).join(' ');
+const dbRip = join(dirname(new URL(import.meta.url).pathname), '..', 'db.rip');
 
 try {
-  execSync(`rip ${dbRip} ${args}`, { stdio: 'inherit' });
-} catch (error) {
-  process.exit(error.status || 1);
+  execFileSync('rip', [dbRip, ...process.argv.slice(2)], { stdio: 'inherit' });
+} catch (e) {
+  process.exit(e.status || 1);
 }