@optimystic/quereus-plugin-optimystic 0.3.0

package/README.md

@@ -0,0 +1,358 @@
+# @optimystic/quereus-plugin-optimystic
+
+A Quereus virtual table plugin that provides SQL access to Optimystic distributed tree collections.
+
+## What's New
+
+- ✨ **TransactionId() function** - Get unique transaction identifiers for audit logging
+- 🔧 **Transaction-specific caching** - Better isolation and no stale data
+- 🧪 **Distributed testing** - Automated tests for multi-node operations
+
+See [CHANGELOG.md](./CHANGELOG.md) for details.
+
+## Overview
+
+This plugin allows you to query and manipulate Optimystic distributed tree collections using standard SQL syntax through Quereus. It bridges the gap between SQL databases and Optimystic's distributed data structures.
+
+For cryptographic functions (hashing, signing, verification), see the separate [@optimystic/quereus-plugin-crypto](../quereus-plugin-crypto) package.
+
+**📖 New to Optimystic SQL? See [QUICK-START.md](./QUICK-START.md) for a 5-minute tutorial!**
+
+## Installation
+
+```bash
+npm install @optimystic/quereus-plugin-optimystic
+```
+
+## Quick Start
+
+### Using Quoomb (Interactive SQL Console)
+
+The easiest way to get started is with [Quoomb](https://github.com/Digithought/quereus/tree/main/packages/quoomb-cli), the interactive SQL console for Quereus:
+
+```bash
+# Install Quoomb globally
+npm install -g @quereus/quoomb-cli
+
+# Start with Optimystic plugin using example config
+quoomb --config node_modules/@optimystic/quereus-plugin-optimystic/examples/quoomb.config.dev.json
+```
+
+Then in the Quoomb console:
+
+```sql
+CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)
+  USING optimystic('tree://app/users');
+
+INSERT INTO users VALUES (1, 'Alice'), (2, 'Bob');
+
+SELECT * FROM users;
+```
+
+**See [examples/README.md](./examples/README.md) for multi-node mesh setup and more configurations.**
+
+### Programmatic Usage
+
+```typescript
+import { Database } from '@quereus/quereus';
+import { loadPlugin } from '@quereus/quereus/util/plugin-loader.js';
+
+const db = new Database();
+
+// Load and register the plugin
+await loadPlugin('npm:@optimystic/quereus-plugin-optimystic', db, {
+  default_transactor: 'network',
+  default_key_network: 'libp2p',
+  enable_cache: true
+});
+
+// Create a virtual table backed by an Optimystic tree collection
+await db.exec(`
+  CREATE TABLE users USING optimystic(
+    'tree://myapp/users',
+    transactor='network',
+    keyNetwork='libp2p'
+  )
+`);
+
+// Query the distributed data
+const users = await db.all('SELECT * FROM users WHERE id = ?', ['user123']);
+```
+
+## Plugin Settings
+
+The plugin can be configured when loading with the following settings:
+
+- **default_transactor**: Default transactor type (`'network'`, `'test'`, or custom) - default: `'network'`
+- **default_key_network**: Default key network type (`'libp2p'`, `'test'`, or custom) - default: `'libp2p'`
+- **default_port**: Default port for libp2p nodes (0 = random) - default: `0`
+- **default_network_name**: Default network identifier - default: `'optimystic'`
+- **enable_cache**: Whether to cache collections between queries - default: `true`
+
+```typescript
+await loadPlugin('npm:@optimystic/quereus-plugin-optimystic', db, {
+  default_transactor: 'network',
+  default_key_network: 'libp2p',
+  default_port: 0,
+  default_network_name: 'myapp',
+  enable_cache: true
+});
+```
+
+## Virtual Table Options
+
+The plugin supports various configuration options passed as arguments to the `USING` clause:
+
+### Basic Options
+
+- **collectionUri** (required): URI identifying the tree collection (e.g., `'tree://myapp/users'`)
+- **transactor**: Type of transactor to use (`'network'`, `'test'`, or custom)
+- **keyNetwork**: Type of key network (`'libp2p'`, `'test'`, or custom)
+
+### Network Options
+
+- **port**: Port for libp2p node (default: from plugin settings)
+- **networkName**: Network identifier (default: from plugin settings)
+- **cache**: Enable local caching (default: from plugin settings)
+- **encoding**: Row encoding format (`'json'` or `'msgpack'`, default: `'json'`)
+
+### Example with Options
+
+```sql
+CREATE TABLE products USING optimystic(
+  'tree://store/products',
+  transactor='network',
+  keyNetwork='libp2p',
+  port=8080,
+  networkName='mystore',
+  cache=true,
+  encoding='json'
+);
+```
+
+## SQL Operations
+
+### SELECT Queries
+
+```sql
+-- Point lookups (optimized)
+SELECT * FROM users WHERE id = 'user123';
+
+-- Range scans
+SELECT * FROM users WHERE id BETWEEN 'user100' AND 'user200';
+
+-- Full table scans
+SELECT * FROM users ORDER BY id;
+```
+
+### INSERT Operations
+
+```sql
+INSERT INTO users (id, data) 
+VALUES ('user456', '{"name": "Alice", "email": "alice@example.com"}');
+```
+
+### UPDATE Operations
+
+```sql
+UPDATE users 
+SET data = '{"name": "Alice Smith", "email": "alice.smith@example.com"}'
+WHERE id = 'user456';
+```
+
+### DELETE Operations
+
+```sql
+DELETE FROM users WHERE id = 'user456';
+```
+
+## Data Model
+
+The plugin provides a simple key-value interface to Optimystic tree collections:
+
+- **Primary Key**: The `id` column serves as the tree key (TEXT type)
+- **Data**: The `data` column stores the value (TEXT type, can be JSON)
+- **Schema**: Fixed schema with `id` and `data` columns
+
+### Example Schema
+
+```sql
+-- The schema is automatically defined as:
+CREATE TABLE optimystic_tree (
+  id TEXT PRIMARY KEY,
+  data TEXT
+);
+```
+
+### Storing Complex Data
+
+```sql
+-- Store JSON data in the data column
+INSERT INTO users VALUES ('user123', '{"name": "Alice", "email": "alice@example.com"}');
+
+-- Query JSON data using Quereus JSON functions
+SELECT id, json_extract(data, '$.name') as name 
+FROM users 
+WHERE json_extract(data, '$.email') LIKE '%@example.com';
+```
+
+## Transactions
+
+The plugin supports Quereus transactions, which map to Optimystic's sync mechanism:
+
+```typescript
+await db.exec('BEGIN');
+await db.exec("INSERT INTO users VALUES ('u1', '{\"name\": \"John\", \"email\": \"john@example.com\"}')");
+await db.exec("INSERT INTO users VALUES ('u2', '{\"name\": \"Jane\", \"email\": \"jane@example.com\"}')");
+await db.exec('COMMIT'); // Syncs changes to the distributed network
+```
+
+- **BEGIN**: Creates a new Optimystic transactor
+- **COMMIT**: Syncs all collections used in the transaction
+- **ROLLBACK**: Discards local changes and clears collection cache
+
+### TransactionId() Function
+
+The plugin provides a `TransactionId()` SQL function that returns a unique identifier for the current transaction:
+
+```sql
+-- Get transaction ID
+BEGIN;
+SELECT TransactionId();  -- Returns base64url-encoded 32-byte ID
+COMMIT;
+
+-- Use in WITH CONTEXT clause
+CREATE TABLE audit_log (
+  id TEXT PRIMARY KEY,
+  action TEXT,
+  txn_id TEXT DEFAULT context_txn_id
+) USING optimystic('tree://app/audit')
+WITH CONTEXT (
+  context_txn_id TEXT
+);
+
+INSERT INTO audit_log (id, action)
+WITH CONTEXT context_txn_id = TransactionId()
+VALUES ('log1', 'user_created');
+```
+
+**Features:**
+- Returns NULL outside of transactions
+- Returns the same ID throughout a transaction
+- Different ID for each transaction
+- 32-byte format: 16 bytes SHA-256(peer ID) + 16 bytes random
+- Base64url encoded for safe use in SQL
+- Works immediately after BEGIN (no DML required)
+
+## Cryptographic Functions
+
+For cryptographic functions (hashing, signing, verification, random bytes), use the separate [@optimystic/quereus-plugin-crypto](../quereus-plugin-crypto) package:
+
+```typescript
+// Load both plugins
+await loadPlugin('npm:@optimystic/quereus-plugin-optimystic', db);
+await loadPlugin('npm:@optimystic/quereus-plugin-crypto', db);
+
+// Use crypto functions in SQL
+const hash = await db.get("SELECT digest('hello world', 'sha256', 'utf8', 'base64url') as hash");
+const nonce = await db.get("SELECT random_bytes(256) as nonce");
+```
+
+See the [crypto plugin documentation](../quereus-plugin-crypto/README.md) for complete details.
+
+## Custom Networks and Transactors
+
+You can register custom implementations for advanced use cases:
+
+```typescript
+import { registerKeyNetwork, registerTransactor } from '@optimystic/quereus-plugin-optimystic';
+
+// Register a custom key network
+registerKeyNetwork('mynetwork', MyCustomKeyNetwork);
+
+// Register a custom transactor
+registerTransactor('mytransactor', MyCustomTransactor);
+
+// Use in SQL
+await db.exec(`
+  CREATE TABLE data USING optimystic(
+    'tree://app/data',
+    transactor='mytransactor',
+    keyNetwork='mynetwork'
+  )
+`);
+```
+
+## Performance Considerations
+
+### Query Optimization
+
+- **Point Lookups**: Queries with `WHERE id = ?` are highly optimized
+- **Range Scans**: Queries with `WHERE id BETWEEN ? AND ?` use efficient tree iteration
+- **Full Scans**: Queries without key constraints iterate the entire collection
+
+### Caching
+
+- Collections are cached within transactions and between queries
+- Set `cache=false` to disable caching if memory is a concern
+- Cache is automatically cleared on rollback
+
+### Network Efficiency
+
+- Use transactions to batch multiple operations
+- Consider using `'test'` transactor for local development
+- Configure appropriate libp2p options for your network topology
+
+## Error Handling
+
+The plugin maps Optimystic errors to appropriate SQL error codes:
+
+- **Collection not found**: `SQLITE_CONSTRAINT_PRIMARYKEY`
+- **Network timeouts**: `SQLITE_BUSY`
+- **Decoding failures**: `SQLITE_CORRUPT`
+- **Configuration errors**: `SQLITE_ERROR`
+
+## Limitations
+
+- Primary key must be TEXT type (tree keys are strings)
+- No secondary indexes (use appropriate WHERE clauses for performance)
+- Cross-collection transactions not yet supported
+- Savepoints not implemented
+
+## Development
+
+### Building
+
+```bash
+npm run build
+```
+
+### Type Checking
+
+```bash
+npm run typecheck
+```
+
+### Testing
+
+```bash
+# Run all tests
+npm test
+
+# Run only distributed tests
+npm exec aegir test -t node -- --grep "Distributed"
+
+# Manual interactive mesh test
+npm run build
+node dist/test/manual-mesh-test.js
+```
+
+See [test/README.md](./test/README.md) for detailed testing documentation.
+
+## License
+
+MIT
+
+## Contributing
+
+Contributions are welcome! Please see the main Optimystic repository for contribution guidelines. 

package/dist/index.d.ts

@@ -0,0 +1,96 @@
+export { C as ColumnDefinition, L as LibP2PNodeOptions, O as OptimysticModule, b as OptimysticTreeOptions, a as OptimysticVirtualTable, P as ParsedOptimysticTreeOptions, R as RowData, T as TransactionState, r as register } from './plugin-B1lVAVv0.js';
+import { IKeyNetwork, ITransactionEngine, TransactionCoordinator, Transaction, ExecutionResult } from '@optimystic/db-core';
+import { Database, SqlParameters } from '@quereus/quereus';
+import '@libp2p/interface';
+
+/**
+ * Register a custom key network implementation
+ */
+declare function registerKeyNetwork(name: string, implementation: new (...args: any[]) => IKeyNetwork): void;
+/**
+ * Register a custom transactor implementation
+ */
+declare function registerTransactor(name: string, implementation: new (...args: any[]) => any): void;
+
+/**
+ * Engine ID for Quereus SQL transactions.
+ * Format: "quereus@{version}" where version matches the quereus package version.
+ */
+declare const QUEREUS_ENGINE_ID = "quereus@0.5.3";
+/**
+ * Statement format for Quereus transactions.
+ * Each statement is a SQL string with optional parameters.
+ */
+type QuereusStatement = {
+    /** The SQL statement to execute */
+    sql: string;
+    /** Optional parameters for the statement */
+    params?: SqlParameters;
+};
+/**
+ * Quereus-specific transaction engine for SQL execution.
+ *
+ * This engine:
+ * 1. Executes SQL statements through a Quereus database
+ * 2. Collects resulting actions from the virtual table module
+ * 3. Computes schema hash for validation
+ *
+ * Used for both initial execution (client creating transaction) and
+ * re-execution (validators verifying transaction).
+ */
+declare class QuereusEngine implements ITransactionEngine {
+    private readonly db;
+    private readonly coordinator;
+    private schemaHashCache;
+    private schemaVersion;
+    constructor(db: Database, coordinator: TransactionCoordinator);
+    /**
+     * Execute a transaction's statements and produce actions.
+     *
+     * For initial execution: Executes SQL through Quereus, which triggers
+     * the Optimystic virtual table module to apply actions.
+     *
+     * For validation: Re-executes the same SQL statements to verify
+     * they produce the same operations.
+     */
+    execute(transaction: Transaction): Promise<ExecutionResult>;
+    /**
+     * Get the schema hash for this engine.
+     *
+     * The schema hash is used for validation - all participants must have
+     * matching schema hashes for a transaction to be valid.
+     *
+     * Uses caching to avoid recomputing if schema hasn't changed.
+     */
+    getSchemaHash(): Promise<string>;
+    /**
+     * Invalidate the schema hash cache.
+     * Call this when the schema changes (e.g., after DDL statements).
+     */
+    invalidateSchemaCache(): void;
+    /**
+     * Get the current schema version number.
+     * Increments each time the schema cache is invalidated.
+     */
+    getSchemaVersion(): number;
+    /**
+     * Compute the schema hash from the database catalog.
+     *
+     * Uses the schema() table-valued function to get schema information,
+     * then hashes the canonical representation using SHA-256.
+     */
+    private computeSchemaHash;
+}
+/**
+ * Helper to create Quereus statement JSON for a transaction.
+ */
+declare function createQuereusStatement(sql: string, params?: SqlParameters): string;
+/**
+ * Helper to create an array of Quereus statements for a transaction.
+ */
+declare function createQuereusStatements(statements: Array<{
+    sql: string;
+    params?: SqlParameters;
+}>): string[];
+
+export { QUEREUS_ENGINE_ID, QuereusEngine, type QuereusStatement, createQuereusStatement, createQuereusStatements, registerKeyNetwork, registerTransactor };