@elizaos/plugin-sql 2.0.0-beta.1 → 2.0.11-beta.7

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Shaw Walters and elizaOS Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,359 +1,142 @@
-# DrizzleDatabaseAdapter
+# @elizaos/plugin-sql
 
-A PostgreSQL database adapter built with Drizzle ORM for the elizaOS ecosystem.
+SQL database adapter plugin for elizaOS — provides persistent storage via PostgreSQL or embedded PGlite (WASM), with Drizzle ORM, automatic schema migrations, and optional Row Level Security.
 
 ## Installation
 
 ```bash
-# Using bun
 bun add @elizaos/plugin-sql
 ```
 
-## Vector Dimensions
-
-The adapter supports the following vector dimensions:
-
-```typescript
-VECTOR_DIMS = {
-  SMALL: 384,
-  MEDIUM: 512,
-  LARGE: 768,
-  XL: 1024,
-  XXL: 1536,
-  XXXL: 3072,
-};
-```
-
-Important Note: Once an agent is initialized with a specific embedding dimension, it cannot be changed. Attempting to change the dimension will result in an error: "Cannot change embedding dimension for agent"
-
-## Features
+## Overview
 
-- Circuit breaker pattern for database failures
-- Automatic retries with exponential backoff
-- Connection pooling
-- Vector search capabilities
-- Memory management
-- Caching system
-- Room and participant management
-- Goal tracking system
+This plugin registers a `DatabaseAdapter` with the elizaOS agent runtime so that all core runtime persistence (memories, entities, rooms, tasks, cache, logs, relationships, etc.) works against a real SQL backend. On Node/Bun it selects PostgreSQL when `POSTGRES_URL` is set, otherwise falls back to embedded PGlite. In the browser build it always uses PGlite (WASM).
 
 ## Database Schema
 
-The plugin uses a structured schema with the following main tables:
+The plugin uses the following main tables:
 
-### Core Tables
+- **Agent**: Agent information and configurations
+- **Room / Channel**: Conversation rooms and messaging channels
+- **Participant / ChannelParticipant**: Participants in rooms and channels
+- **Memory**: Agent memories with vector embeddings for semantic search
+- **Embedding**: Vector embeddings for entities
+- **Entity**: Entities agents interact with
+- **Relationship**: Relationships between entities
+- **Component**: Agent components and configurations
+- **Tasks**: Tasks and goals
+- **Log**: System logs
+- **Cache**: Frequently accessed data cache
+- **World**: World settings and configurations
 
-- **Agent**: Stores agent information and configurations
-- **Room**: Manages conversation rooms and their settings
-- **Participant**: Tracks participants in rooms
-- **Memory**: Stores agent memories with vector embeddings for semantic search
-- **Embedding**: Manages vector embeddings for various entities
-- **Entity**: Represents entities that agents can interact with
-- **Relationship**: Tracks relationships between entities
-- **Component**: Stores agent components and their configurations
-- **Tasks**: Manages tasks and goals for agents
-- **Log**: Stores system logs
-- **Cache**: Provides a caching mechanism for frequently accessed data
-- **World**: Manages world settings and configurations
+Table definitions live in `src/schema/`.
 
-Each table is defined using Drizzle ORM schema definitions in the `src/schema` directory. The schema is designed to support the elizaOS ecosystem's requirements for agent-based systems.
+## Electric Sync (PGlite ↔ Electric Cloud)
 
-## Usage
+When `ELIZA_ELECTRIC_SYNC_URL` and `AGENT_ID` are set, PGlite connects to an Electric sync service and streams real-time updates for all core tables. Each agent syncs only its own rows (filtered by `agent_id` / `id`), preserving per-agent isolation in shared-Neon deployments.
 
-The adapter is typically used as part of the elizaOS runtime:
+### Local dev with Electric Cloud
 
-```typescript
-async function findDatabaseAdapter(runtime: IAgentRuntime) {
-  let adapter = runtime;
-
-  if (!adapter) {
-    const drizzleAdapterPlugin = await import("@elizaos/plugin-sql");
-    const drizzleAdapterPluginDefault = drizzleAdapterPlugin.default;
-    adapter = drizzleAdapterPluginDefault.adapter;
-    if (!adapter) {
-      throw new Error(
-        "Internal error: No database adapter found for default plugin-sql",
-      );
-    }
-  } else if (!adapter) {
-    throw new Error(
-      "Multiple database adapters found. You must have no more than one. Adjust your plugins configuration.",
-    );
-  }
-
-  const adapterInterface = await adapter?.init(runtime);
-  return adapterInterface;
-}
-```
+The e2e write-back test (`__tests__/integration/electric-write-back.test.ts`) uses a [Caddy](https://caddyserver.com) reverse proxy to forward shape requests to Electric Cloud with auth. Caddy is the proxy [recommended by Electric](https://electric.ax/docs/sync/guides/troubleshooting#missing-headers).
 
-## Error Handling Configuration
+```bash
+# 1. Set your Electric Cloud credentials (get these from dashboard.electric-sql.cloud)
+export ELECTRIC_CLOUD_SOURCE_ID=svc-xxxxxxxxxxxx
+export ELECTRIC_CLOUD_SECRET=eyJ...
 
-The adapter implements the following error handling configurations:
+# 2. Start the Caddy proxy
+caddy run --config plugins/plugin-sql/caddy/electric-proxy.Caddyfile
 
-```typescript
-{
-    failureThreshold: 5,
-    resetTimeout: 60000,
-    halfOpenMaxAttempts: 3,
-    maxRetries: 3,
-    baseDelay: 1000,  // 1 second
-    maxDelay: 10000,  // 10 seconds
-    jitterMax: 1000,  // 1 second
-    connectionTimeout: 5000  // 5 seconds
-}
+# 3. Run the e2e test
+bun run --cwd plugins/plugin-sql test -- \
+  __tests__/integration/electric-write-back.test.ts
 ```
 
-## Requirements
-
-- PostgreSQL with vector extension installed
-- Node.js or Bun (≥1.2.2)
+The Caddyfile at `plugins/plugin-sql/caddy/electric-proxy.Caddyfile` forwards every incoming request to `api.electric-sql.cloud` with auth query params appended.
 
 ## Environment Variables
 
-The plugin uses the following environment variables:
-
-- `POSTGRES_URL`: Connection string for PostgreSQL database (e.g., `postgresql://user:password@localhost:5432/dbname`)
-  - If not provided, the plugin will use PGlite as a fallback
-- `PGLITE_DATA_DIR`: (Optional) Directory for PGlite data storage (default: `./pglite`)
-
-These variables should be defined in a `.env` file at the root of your project.
+| Variable | Required | Default | Effect |
+|----------|----------|---------|--------|
+| `POSTGRES_URL` | No | — | PostgreSQL connection string. When absent, PGlite is used. |
+| `PGLITE_DATA_DIR` | No | `.eliza/.elizadb` | Directory (or `idb://` URL) for PGlite data storage. |
+| `ELIZA_ELECTRIC_SYNC_URL` | No | — | Base URL of Electric sync service (e.g. `http://localhost:3001` via Caddy). |
+| `AGENT_ID` | Conditional | — | UUID of the agent. Required when `ELIZA_ELECTRIC_SYNC_URL` is set (per-agent WHERE filter). |
+| `ELIZA_CLOUD_WRITE_BASE_URL` | No | — | Write-back cloud endpoint for forwarding local PGlite writes to Postgres. |
+| `ELIZA_CLOUD_SERVICE_KEY` | No | — | Service key for authenticating write-back requests. |
+| `ELECTRIC_CLOUD_SOURCE_ID` | Test-only | — | Electric Cloud source ID (consumed by Caddy, not the runtime). |
+| `ELECTRIC_CLOUD_SECRET` | Test-only | — | Electric Cloud JWT secret (consumed by Caddy, not the runtime). |
+| `ELIZA_PGLITE_DISABLE_EXTENSIONS` | No | `false` | Set to `1` to disable PGlite extensions (vector, live, fuzzystrmatch, Electric sync). |
+| `ENABLE_DATA_ISOLATION` | No | `false` | When `true`, enables PostgreSQL Row Level Security per-server isolation. |
+| `ELIZA_SERVER_ID` | Conditional | — | Required when `ENABLE_DATA_ISOLATION=true`; becomes the RLS server UUID. |
+| `ELIZA_ALLOW_DESTRUCTIVE_MIGRATIONS` | No | `false` | Allow column drops and other destructive schema changes at startup. |
+| `NODE_ENV` | No | `development` | `production` disables verbose migration logging and tightens safety checks. |
+
+Settings are read via `runtime.getSetting(key)` inside `plugin.init`.
 
-## Database Pool Configuration
-
-Default pool configuration:
+## Vector Dimensions
 
 ```typescript
-{
-    max: 20,
-    idleTimeoutMillis: 30000,
-    connectionTimeoutMillis: 5000
-}
+VECTOR_DIMS = {
+  SMALL: 384,
+  MEDIUM: 512,
+  LARGE: 768,
+  XL: 1024,
+  XXL: 1536,
+  XXXL: 3072,
+};
 ```
 
-## Migration Support
-
-elizaOS v1.0.0 introduces **dynamic runtime migrations** - automatic schema management that runs at startup without manual intervention. Plugins can define their schemas and the system handles all migrations automatically.
-
-### TLDR: What Changed?
-
-**Before (v0.x):** Manual migrations with `drizzle-kit generate` → `drizzle-kit push` → restart  
-**Now (v1.0.0):** Define schema in plugin → Start agent → Migrations run automatically ✨
+Once an agent is initialized with a specific embedding dimension, it cannot be changed without a new agent or manual DB surgery.
 
-### Key Features
+## Runtime Migrations
 
-- **Zero-Config Migrations**: No more manual migration commands
-- **Plugin Isolation**: Each plugin gets its own schema namespace
-- **Safety First**: Destructive changes blocked by default in production
-- **Concurrent Safety**: Built-in locks prevent race conditions
-- **Rollback Protection**: All migrations run in transactions
-
-### How It Works
-
-1. **Plugin defines schema** using Drizzle ORM:
+Plugins export a `schema` object; `DatabaseMigrationService` diffs the schema against the live DB at startup and runs migrations automatically. No manual `drizzle-kit generate` / `drizzle-kit push` step is needed in normal development.
 
 ```typescript
-// In your plugin's schema.ts
-import { pgTable, text, uuid } from "drizzle-orm/pg-core";
-
-export const myTable = pgTable("my_table", {
-  id: uuid("id").primaryKey(),
-  name: text("name").notNull(),
-});
-
-// Export schema in your plugin
+// In your plugin
 export const plugin = {
   name: "@your-org/plugin-name",
-  schema: schema, // Your Drizzle schema object
-  // ... rest of plugin
+  schema: schema, // Drizzle schema object
+  // ...
 };
 ```
 
-2. **Runtime detects changes** at startup:
-
-```bash
-[RuntimeMigrator] Starting migration for plugin: @your-org/plugin-name
-[RuntimeMigrator] Executing 2 SQL statements...
-[RuntimeMigrator] Migration completed successfully
-```
-
-3. **Automatic safety checks**:
-
-```bash
-# Destructive changes are blocked
-[RuntimeMigrator] Destructive migration blocked
-[RuntimeMigrator] Destructive operations detected:
-[RuntimeMigrator]   - Column "email" will be dropped from table "users"
-[RuntimeMigrator] To proceed:
-[RuntimeMigrator]   1. Set ELIZA_ALLOW_DESTRUCTIVE_MIGRATIONS=true
-[RuntimeMigrator]   2. Or use { force: true } option
-```
+Destructive changes (column drops, type changes) are blocked by default. Set `ELIZA_ALLOW_DESTRUCTIVE_MIGRATIONS=true` to allow them.
 
-### Migration Controls
+## Connection Management
 
-Control migration behavior via environment variables:
+Both `PostgresConnectionManager` and `PGliteClientManager` are stored under `Symbol.for("elizaos.plugin-sql.global-singletons")` on `globalThis`. This prevents multiple pools when the module is imported from multiple paths in the same process. Do not construct manager instances directly — always go through `createDatabaseAdapter()`.
 
-```bash
-# Allow destructive migrations (drops, type changes)
-ELIZA_ALLOW_DESTRUCTIVE_MIGRATIONS=true
-
-# Development vs Production
-NODE_ENV=production  # Stricter checks, verbose off by default
-NODE_ENV=development  # More permissive, verbose on
-```
-
-Or programmatically:
-
-```typescript
-await databaseAdapter.runPluginMigrations(plugins, {
-  verbose: true, // Show SQL statements
-  force: true, // Allow destructive changes
-  dryRun: true, // Preview without applying
-});
-```
-
-### Transitioning from Manual Migrations
-
-If you have existing manual Drizzle migrations:
-
-1. **Keep existing migrations** - They remain compatible
-2. **Add schema to plugin** - Export your Drizzle schema
-3. **First run** - Runtime migrator detects current state
-4. **Future changes** - Just update schema and restart
-
-Example transition:
-
-```typescript
-// Before: Manual migrations
-// 1. Edit schema
-// 2. Run: bunx drizzle-kit generate
-// 3. Run: bunx drizzle-kit push
-// 4. Restart agent
-
-// After: Runtime migrations
-// 1. Edit schema in plugin
-// 2. Restart agent (migrations run automatically)
-```
-
-### Schema Namespacing
-
-Plugins automatically get namespaced schemas for isolation:
-
-- `@elizaos/plugin-sql` → Uses `public` schema (core tables)
-- `@your-org/plugin-name` → Uses `your_org_plugin_name` schema
-- Prevents table name conflicts between plugins
-- Clean separation of concerns
-
-To use a custom schema:
-
-```typescript
-import { pgSchema } from "drizzle-orm/pg-core";
-
-const mySchema = pgSchema("my_custom_schema");
-export const myTable = mySchema.table("my_table", {
-  // ... columns
-});
-```
-
-### Debugging Migrations
-
-Check migration status:
-
-```typescript
-const migrator = migrationService.getMigrator();
-const status = await migrator.getStatus("@your-org/plugin-name");
-console.log(status);
-// {
-//   hasRun: true,
-//   lastMigration: { hash: "...", timestamp: ... },
-//   journal: [...],
-//   snapshots: 3
-// }
-```
+## Database Pool Configuration
 
-Preview changes without applying:
+Default Postgres pool configuration (`src/pg/manager.ts`):
 
 ```typescript
-const check = await migrator.checkMigration("@your-org/plugin-name", schema);
-if (check?.hasDataLoss) {
-  console.log("Warning: Destructive changes:", check.warnings);
+{
+    max: 20,
+    min: 2,
+    idleTimeoutMillis: 30000,
+    connectionTimeoutMillis: 5000,
+    keepAlive: true,
+    keepAliveInitialDelayMillis: 10000
 }
 ```
 
-### Database Support
-
-The plugin supports two database backends with automatic migration support:
+## Retry Configuration
 
-1. **PostgreSQL**: Production-ready with full feature support
-2. **PGlite**: Embedded database for development/testing
-
-Both use identical migration systems - develop locally with PGlite, deploy to PostgreSQL.
-
-### Troubleshooting
-
-**"Destructive migration blocked"**
-
-- Set `ELIZA_ALLOW_DESTRUCTIVE_MIGRATIONS=true` for development
-- For production, review changes carefully before enabling
-
-**"Migration already in progress"**
-
-- Another instance is running migrations
-- System will wait for lock automatically
-
-**"No changes detected"**
-
-- Schema matches database state
-- No migration needed
-
-**Manual migration needed?**
-
-- Use standard Drizzle Kit for complex scenarios:
-  ```bash
-  bunx drizzle-kit generate
-  bunx drizzle-kit migrate
-  ```
-
-## Clean Shutdown
-
-The adapter implements cleanup handlers for:
-
-- SIGINT
-- SIGTERM
-- beforeExit
-
-These ensure proper closing of database connections when the application shuts down.
-
-## Implementation Details
-
-### Connection Management
-
-The plugin uses a global singleton pattern to manage database connections. This approach ensures that:
-
-1. **Single Connection Per Process**: Only one connection manager instance exists per Node.js process, regardless of how many times the package is imported or initialized.
-
-2. **Resource Efficiency**: Prevents multiple connection pools to the same database, which could lead to resource exhaustion.
-
-3. **Consistent State**: Ensures all parts of the application share the same database connection state.
-
-4. **Proper Cleanup**: Facilitates proper cleanup of database connections during application shutdown, preventing connection leaks.
-
-This pattern is particularly important in monorepo setups or when the package is used by multiple modules within the same process. The implementation uses JavaScript Symbols to create a global registry that persists across module boundaries.
+`BaseDrizzleAdapter` retries failed operations with exponential backoff and jitter (`src/base.ts`):
 
 ```typescript
-// Example of the singleton pattern implementation
-const GLOBAL_SINGLETONS = Symbol.for("@elizaos/plugin-sql/global-singletons");
-
-// Store managers in a global symbol registry
-if (!globalSymbols[GLOBAL_SINGLETONS]) {
-  globalSymbols[GLOBAL_SINGLETONS] = {};
-}
-
-// Reuse existing managers or create new ones when needed
-if (!globalSingletons.postgresConnectionManager) {
-  globalSingletons.postgresConnectionManager = new PostgresConnectionManager(
-    config.postgresUrl,
-  );
+{
+    maxRetries: 3,
+    baseDelay: 1000,
+    maxDelay: 10000,
+    jitterMax: 1000
 }
 ```
 
-This approach is especially critical for PGlite connections, which require careful management to ensure proper shutdown and prevent resource leaks.
+## Requirements
+
+- Node.js or Bun
+- PostgreSQL with vector extension (for Postgres mode)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@elizaos/plugin-sql",
-  "version": "2.0.0-beta.1",
+  "version": "2.0.11-beta.7",
   "description": "",
   "type": "module",
   "main": "src/dist/index.js",
@@ -9,10 +9,15 @@
     "./package.json": "./package.json",
     ".": {
       "types": "./src/dist/index.node.d.ts",
+      "eliza-source": {
+        "types": "./src/index.node.ts",
+        "import": "./src/index.node.ts",
+        "default": "./src/index.node.ts"
+      },
       "bun": {
-        "types": "./src/index.ts",
-        "import": "./src/index.ts",
-        "default": "./src/index.ts"
+        "types": "./src/dist/index.node.d.ts",
+        "import": "./src/dist/node/index.node.js",
+        "default": "./src/dist/node/index.node.js"
       },
       "browser": {
         "types": "./src/dist/index.d.ts",
@@ -30,9 +35,9 @@
     "./drizzle": {
       "types": "./src/dist/drizzle/index.d.ts",
       "bun": {
-        "types": "./src/drizzle/index.ts",
-        "import": "./src/drizzle/index.ts",
-        "default": "./src/drizzle/index.ts"
+        "types": "./src/dist/drizzle/index.d.ts",
+        "import": "./src/dist/drizzle/index.js",
+        "default": "./src/dist/drizzle/index.js"
       },
       "import": "./src/dist/drizzle/index.js",
       "default": "./src/dist/drizzle/index.js"
@@ -40,12 +45,18 @@
     "./schema": {
       "types": "./src/dist/schema/index.d.ts",
       "bun": {
-        "types": "./src/schema/index.ts",
-        "import": "./src/schema/index.ts",
-        "default": "./src/schema/index.ts"
+        "types": "./src/dist/schema/index.d.ts",
+        "import": "./src/dist/schema/index.js",
+        "default": "./src/dist/schema/index.js"
       },
       "import": "./src/dist/schema/index.js",
       "default": "./src/dist/schema/index.js"
+    },
+    "./*.css": "./dist/*.css",
+    "./*": {
+      "types": "./dist/*.d.ts",
+      "import": "./dist/*.js",
+      "default": "./dist/*.js"
     }
   },
   "files": [
@@ -75,6 +86,7 @@
   },
   "dependencies": {
     "@electric-sql/pglite": "^0.4.0",
+    "@electric-sql/pglite-sync": "0.5.6",
     "@neondatabase/serverless": "^1.1.0",
     "drizzle-orm": "0.45.2",
     "pg": "^8.16.3",
@@ -93,9 +105,9 @@
     "vitest": "^4.0.0"
   },
   "peerDependencies": {
-    "@elizaos/core": "2.0.0-beta.1"
+    "@elizaos/core": "2.0.11-beta.7"
   },
-  "gitHead": "05d4ca11d769db8c7f54a722ee24b2ce2b951543",
+  "gitHead": "cdbc876f793d96073d7eb0d09715a031ce0cd32e",
   "publishConfig": {
     "access": "public"
   }