@earth-app/collegedb 1.0.8 → 1.1.1

package/README.md

@@ -1,6 +1,6 @@
 # CollegeDB
 
-_Cloudflare D1 Horizontal Sharding Router_
+Universal Database Horizontal Sharding Router
 
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
 [![GitHub Issues](https://img.shields.io/github/issues/earth-app/CollegeDB)](https://github.com/earth-app/CollegeDB/issues)
@@ -8,7 +8,32 @@ _Cloudflare D1 Horizontal Sharding Router_
 [![GitHub License](https://img.shields.io/github/license/earth-app/CollegeDB)](LICENSE)
 ![NPM Version](https://img.shields.io/npm/v/%40earth-app%2Fcollegedb)
 
-A TypeScript library for **true horizontal scaling** of SQLite-style databases on Cloudflare using D1 and KV. CollegeDB distributes your data across multiple D1 databases, with each table's records split by primary key across different database instances.
+A TypeScript library for **true horizontal scaling** of SQLite-style databases primarily for Cloudflare using D1 and KV, with additional provider adapters for Redis/Valkey KV and PostgreSQL/MySQL/SQLite SQL backends, plus Drizzle ORM interop across those SQL providers. CollegeDB distributes your data across multiple database shards, with each table's records split by primary key across different database instances.
+
+## Table of Contents
+
+- [Why CollegeDB](#why-collegedb)
+- [Features](#features)
+- [Getting Started](#getting-started)
+- [Benchmark Suite](#benchmark-suite)
+- [Provider Adapters](#provider-adapters)
+- [NuxtHub + Drizzle Recipes](#nuxthub--drizzle-recipes)
+- [Sandbox Benchmarks (Docker Compose)](#sandbox-benchmarks-docker-compose)
+- [Basic Usage](#basic-usage)
+- [Multi-Key Shard Mappings](#multi-key-shard-mappings)
+- [Drop-in Replacement for Existing Databases](#drop-in-replacement-for-existing-databases)
+- [Troubleshooting](#troubleshooting)
+- [API Reference](#api-reference)
+- [Architecture](#architecture)
+- [Cloudflare Setup](#cloudflare-setup)
+- [Monitoring and Maintenance](#monitoring-and-maintenance)
+- [Performance Analysis](#performance-analysis)
+- [Advanced Configuration](#advanced-configuration)
+- [Quick Reference](#quick-reference)
+- [Contributing](#contributing)
+- [License](#license)
+
+## Why CollegeDB
 
 CollegeDB implements **data distribution** where a single logical table is physically stored across multiple D1 databases:
 
@@ -39,30 +64,19 @@ This allows you to:
 - **Scale geographically** by placing shards in different regions
 - **Increase write throughput** by parallelizing across multiple database instances
 
-## 📈 Overview
+## Features
 
-CollegeDB provides a sharding layer on top of Cloudflare D1 databases, enabling you to:
+- Automatic query routing (primary key to shard mapping)
+- Provider adapters for Redis/Valkey/NuxtHub KV plus PostgreSQL/MySQL/SQLite SQL
+- Drizzle interop through existing SQL providers (`createPostgreSQLProvider`, `createMySQLProvider`, `createSQLiteProvider`)
+- Hyperdrive helpers for PostgreSQL and MySQL
+- Multiple allocation strategies: round-robin, random, hash, location-aware, and mixed read/write strategies
+- Durable Object shard coordination and shard statistics
+- Migration helpers for integrating existing datasets and rebalancing mappings
 
-- **Scale horizontally** by distributing table data across multiple D1 instances
-- **Route queries automatically** based on primary key mappings
-- **Maintain consistency** with KV-based shard mapping
-- **Optimize for geography** with location-aware shard allocation
-- **Monitor and rebalance** shard distribution
-- **Handle migrations** between shards seamlessly
+## Getting Started
 
-## 📦 Features
-
-- **🔀 Automatic Query Routing**: Primary key → shard mapping using Cloudflare KV
-- **🎯 Multiple Allocation Strategies**: Round-robin, random, hash-based, and location-aware distribution
-- **🔄 Mixed Strategy Support**: Different strategies for reads vs writes (e.g., location for writes, hash for reads)
-- **📊 Shard Coordination**: Durable Objects for allocation and statistics
-- **🛠 Migration Support**: Move data between shards with zero downtime
-- **🔄 Automatic Drop-in Replacement**: Zero-config integration with existing databases
-- **🤖 Smart Migration Detection**: Automatically discovers and maps existing data
-- **⚡ High Performance**: Optimized for Cloudflare Workers runtime
-- **🔧 TypeScript First**: Full type safety and excellent DX
-
-## Installation
+### Installation
 
 ```bash
 bun add @earth-app/collegedb
@@ -70,6 +84,373 @@ bun add @earth-app/collegedb
 npm install @earth-app/collegedb
 ```
 
+### NuxtHub + Drizzle with CollegeDB Routing
+
+Keep NuxtHub + Drizzle for schema/migrations and add CollegeDB as your routing layer.
+
+```typescript
+import { db as hubDb } from '@nuxthub/db';
+import { kv } from '@nuxthub/kv';
+import { sql } from 'drizzle-orm';
+import { drizzle } from 'drizzle-orm/d1';
+import { createNuxtHubKVProvider, createSQLiteProvider, first, initialize, run } from '@earth-app/collegedb';
+
+let initialized = false;
+
+function ensureCollegeDB(env: { DB_SECONDARY: D1Database }) {
+	if (initialized) return;
+
+	initialize({
+		kv: createNuxtHubKVProvider(kv),
+		shards: {
+			'db-primary': createSQLiteProvider(hubDb, sql),
+			'db-secondary': createSQLiteProvider(drizzle(env.DB_SECONDARY), sql)
+		},
+		strategy: 'hash'
+	});
+
+	initialized = true;
+}
+
+export default defineEventHandler(async (event) => {
+	const env = event.context.cloudflare.env;
+	ensureCollegeDB(env);
+
+	await run('post:123', 'INSERT OR REPLACE INTO blog_posts (id, title) VALUES (?, ?)', ['post:123', 'Hello from CollegeDB']);
+
+	const post = await first<{ id: string; title: string }>('post:123', 'SELECT id, title FROM blog_posts WHERE id = ?', ['post:123']);
+
+	return { post };
+});
+```
+
+### Drop-in Pattern for Existing `hub:db` + `hub:kv` Code
+
+```typescript
+// before
+import { eq } from 'drizzle-orm';
+import { db } from 'hub:db';
+import { kv } from 'hub:kv';
+import { blogPosts } from '~/server/db/schema';
+
+const cached = await kv.get('nuxtpress:post:slug');
+if (cached) return cached;
+
+const rows = await db.select().from(blogPosts).where(eq(blogPosts.slug, slug)).limit(1);
+await kv.set('nuxtpress:post:slug', rows[0], { ttl: 3600 });
+```
+
+```typescript
+// after (CollegeDB routing + same NuxtHub KV cache)
+import { kv } from '@nuxthub/kv';
+import { sql } from 'drizzle-orm';
+import { db } from 'hub:db';
+import { createNuxtHubKVProvider, createSQLiteProvider, first, initialize } from '@earth-app/collegedb';
+
+let initialized = false;
+
+function setup() {
+	if (initialized) return;
+	initialize({
+		kv: createNuxtHubKVProvider(kv),
+		shards: {
+			'db-primary': createSQLiteProvider(db, sql)
+		},
+		strategy: 'hash'
+	});
+	initialized = true;
+}
+
+setup();
+
+const cacheKey = `nuxtpress:post:${slug}`;
+const cached = await kv.get(cacheKey);
+if (cached) return cached;
+
+const row = await first<{ id: string; slug: string; title: string }>(
+	cacheKey,
+	'SELECT id, slug, title FROM blog_posts WHERE slug = ? LIMIT 1',
+	[slug]
+);
+
+await kv.set(cacheKey, row, { ttl: 3600 });
+```
+
+## Benchmark Suite
+
+CollegeDB includes a benchmark runner that executes each SQL+KV combination across adapter profiles, then generates a report with profile-specific matrices.
+
+### Adapter Profiles
+
+| Profile    | Purpose                                                                 |
+| ---------- | ----------------------------------------------------------------------- |
+| native     | Direct provider clients (Cloudflare bindings or driver-native adapters) |
+| drizzle    | Drizzle interop through SQL provider adapters                           |
+| hyperdrive | Hyperdrive connection-string wrappers for PostgreSQL/MySQL              |
+| nuxthub    | NuxtHub-style KV adapter with SQL provider interop                      |
+
+### Scenario Catalog
+
+| Scenario Key      | Scenario                         | What Happens                                                                                        | Workload Per Run                                                       |
+| ----------------- | -------------------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
+| basic_crud        | Basic CRUD round-trip            | Insert, read, update, and delete a user via routed queries.                                         | 20 iterations; 4 routed SQL ops per iteration                          |
+| advanced_usage    | Advanced lookup workflow         | Writes user+post, adds lookup aliases, then validates join and alias-based lookup.                  | 15 iterations; ~5 routed SQL ops + KV lookup-key updates per iteration |
+| migration_mapping | Migration-style mapping creation | Inserts legacy records on a fixed shard, then builds shard mappings in batch and validates routing. | 10 iterations; 20 legacy records mapped per iteration                  |
+| bulk_crud         | Bulk CRUD pressure               | Performs bulk inserts, half updates, and full delete sweep, then validates shard-wide totals.       | 7 iterations; 160 inserts + 80 updates + 160 deletes per iteration     |
+| indexing          | Indexed query scan               | Creates an index on posts(user_id) and repeatedly queries the indexed path.                         | 15 iterations after warmup dataset build                               |
+| metadata_fetch    | Metadata inspection              | Reads table metadata/introspection rows from one shard.                                             | 14 iterations; 1 metadata query per iteration                          |
+| pragma_or_info    | PRAGMA / server info             | Runs provider-specific PRAGMA/info query to sample low-level metadata latency.                      | 14 iterations; 1 pragma/info query per iteration                       |
+| counting          | Cross-shard counting             | Counts users across all shards to measure fanout aggregation overhead.                              | 14 iterations; all-shard count aggregation per iteration               |
+| shard_fanout      | Shard fanout query               | Runs query fanout to all shards and aggregates shard-level responses.                               | 14 iterations; 1 all-shards query per iteration                        |
+| reassignment      | Shard reassignment flow          | Creates a record, reassigns it to another shard, and verifies routed reads still succeed.           | 10 iterations; insert + reassignment + verification per iteration      |
+
+### Report Matrices
+
+Each generated report includes:
+
+- `Matrix: SQL x KV (Overall)`
+- `Matrix: Adapter Profiles (Overall Avg)`
+- `Matrix: Core Scenario Latency (avg/p95)`
+- `Matrix: Introspection and Routing Latency (avg/p95)`
+- `Cloudflare Worker (wrangler dev --local)`
+- `Matrix: Cloudflare Adapter Profiles (Overall Avg)`
+
+### Common Commands
+
+```bash
+bun run test:sandbox
+bun run test:sandbox:drizzle
+bun run test:sandbox:nuxthub
+bun run test:sandbox:hyperdrive
+```
+
+For Docker-based benchmark details and filtering options, see [Sandbox Benchmarks (Docker Compose)](#sandbox-benchmarks-docker-compose).
+
+## Provider Adapters
+
+CollegeDB can run with either native Cloudflare bindings or custom providers as long as they match the exported `KVStorage` and `SQLDatabase` interfaces.
+
+Drizzle interop is enabled by passing a Drizzle `sql` tag as the optional second argument to `createPostgreSQLProvider`, `createMySQLProvider`, or `createSQLiteProvider`.
+
+Supported adapters:
+
+- `createRedisKVProvider`
+- `createValkeyKVProvider`
+- `createNuxtHubKVProvider`
+- `createPostgreSQLProvider`
+- `createMySQLProvider`
+- `createSQLiteProvider`
+- `createDrizzleSQLProvider` (compatibility helper)
+- `createHyperdrivePostgresProvider`
+- `createHyperdriveMySQLProvider`
+
+```typescript
+import { createClient as createRedisClient } from 'redis';
+import { Pool } from 'pg';
+import { createPostgreSQLProvider, createRedisKVProvider, initialize, run, type CollegeDBConfig } from '@earth-app/collegedb';
+
+const redisClient = createRedisClient({ url: process.env.REDIS_URL });
+const pgPool = new Pool({ connectionString: process.env.POSTGRES_URL });
+
+const config: CollegeDBConfig = {
+	kv: createRedisKVProvider(redisClient),
+	shards: {
+		'pg-east': createPostgreSQLProvider(pgPool)
+	},
+	strategy: 'hash',
+	disableAutoMigration: true
+};
+
+async function bootstrap() {
+	await redisClient.connect();
+	initialize(config);
+	await run('user-1', 'INSERT INTO users (id, name) VALUES (?, ?)', ['user-1', 'Taylor']);
+}
+
+bootstrap().catch(console.error);
+```
+
+For Hyperdrive-backed SQL connections, use `createHyperdrivePostgresProvider` or `createHyperdriveMySQLProvider` with your database client factory.
+
+### NuxtHub Runtime Example (D1 + KV)
+
+```typescript
+import { db } from '@nuxthub/db';
+import { kv } from '@nuxthub/kv';
+import { sql } from 'drizzle-orm';
+import { createNuxtHubKVProvider, createSQLiteProvider, initialize, run, first } from '@earth-app/collegedb';
+
+initialize({
+	kv: createNuxtHubKVProvider(kv),
+	shards: {
+		'db-primary': createSQLiteProvider(db, sql)
+	},
+	strategy: 'hash'
+});
+
+await run('draft:home', 'INSERT OR REPLACE INTO drafts (id, content) VALUES (?, ?)', ['draft:home', '# Home']);
+
+const draft = await first<{ id: string; content: string }>('draft:home', 'SELECT id, content FROM drafts WHERE id = ?', ['draft:home']);
+```
+
+### Keep Drizzle Schema + Migrations, Route Queries with CollegeDB
+
+CollegeDB does not replace your Drizzle schema or NuxtHub migration workflow.
+
+```bash
+npx nuxt db generate
+npx nuxt db migrate
+```
+
+Use those migrations as-is, then route runtime reads/writes through CollegeDB adapters.
+
+For a complete non-Cloudflare setup, see `examples/provider-sandbox.ts`.
+
+## NuxtHub + Drizzle Recipes
+
+### Multi-Vendor Shards (Cloudflare + Postgres/MySQL)
+
+NuxtHub supports multiple deployment/database vendors. CollegeDB can shard across any SQL backends that Drizzle can connect to.
+
+```typescript
+import { sql } from 'drizzle-orm';
+import { drizzle as drizzlePg } from 'drizzle-orm/postgres-js';
+import { drizzle as drizzleMySQL } from 'drizzle-orm/mysql2';
+import { drizzle as drizzleD1 } from 'drizzle-orm/d1';
+import { kv } from '@nuxthub/kv';
+import postgres from 'postgres';
+import mysql from 'mysql2/promise';
+import {
+	createMySQLProvider,
+	createNuxtHubKVProvider,
+	createPostgreSQLProvider,
+	createSQLiteProvider,
+	initialize,
+	run
+} from '@earth-app/collegedb';
+
+const pgClient = postgres(process.env.POSTGRES_URL!);
+const mysqlPool = mysql.createPool(process.env.MYSQL_URL!);
+
+function setup(env: { DB_CF: D1Database }) {
+	initialize({
+		kv: createNuxtHubKVProvider(kv),
+		shards: {
+			'db-cf': createSQLiteProvider(drizzleD1(env.DB_CF), sql),
+			'db-pg': createPostgreSQLProvider(drizzlePg(pgClient), sql),
+			'db-mysql': createMySQLProvider(drizzleMySQL(mysqlPool), sql)
+		},
+		strategy: 'hash'
+	});
+}
+
+export default defineEventHandler(async (event) => {
+	setup(event.context.cloudflare.env);
+
+	await run('tenant:acme:user:1', 'INSERT INTO users (id, name) VALUES (?, ?)', ['tenant:acme:user:1', 'Ada']);
+});
+```
+
+### Keep NuxtHub Cache/KV Semantics Intact
+
+Use NuxtHub KV for app cache while CollegeDB uses its own key namespace for shard mappings:
+
+```typescript
+import { kv } from '@nuxthub/kv';
+import { first } from '@earth-app/collegedb';
+
+const cacheKey = `nuxtpress:post:${slug}`;
+const cached = await kv.get(cacheKey);
+if (cached) return cached;
+
+const post = await first(cacheKey, 'SELECT * FROM blog_posts WHERE slug = ? LIMIT 1', [slug]);
+await kv.set(cacheKey, post, { ttl: 3600 });
+```
+
+## Sandbox Benchmarks (Docker Compose)
+
+CollegeDB ships with an integration sandbox runner that benchmarks real latency across provider combinations.
+
+Requirements:
+
+- Docker + Docker Compose plugin
+- Bun
+- Wrangler (installed as a dev dependency and invoked by scripts)
+
+The Cloudflare benchmark path runs against the dedicated sandbox worker:
+
+- Worker entry: `sandbox/worker.ts`
+- Wrangler config: `sandbox/wrangler.jsonc`
+
+Main commands:
+
+```bash
+# Run full SQL x KV matrix plus Cloudflare local benchmark
+bun run test:sandbox
+
+# Run full SQL x KV matrix only
+bun run test:sandbox:all
+
+# Run Cloudflare local benchmark only (wrangler dev --local)
+bun run test:sandbox:cloudflare
+```
+
+Provider filters:
+
+```bash
+# One SQL provider against all KV providers (native profile by default)
+bun run test:sandbox:mysql
+bun run test:sandbox:postgres
+bun run test:sandbox:mariadb
+bun run test:sandbox:sqlite
+
+# One KV provider against all SQL providers (native profile by default)
+bun run test:sandbox:redis
+bun run test:sandbox:valkey
+
+# Run all SQL x KV combinations for one adapter profile
+bun run test:sandbox:drizzle
+bun run test:sandbox:nuxthub
+bun run test:sandbox:hyperdrive
+
+# Explicit pairwise combinations
+bun run test:sandbox:postgres+redis
+bun run test:sandbox:postgres+valkey
+bun run test:sandbox:mysql+redis
+bun run test:sandbox:mysql+valkey
+bun run test:sandbox:mariadb+redis
+bun run test:sandbox:mariadb+valkey
+bun run test:sandbox:sqlite+redis
+bun run test:sandbox:sqlite+valkey
+```
+
+Output behavior:
+
+- Every run writes a timestamped Markdown report to `sandbox/results/`
+- `sandbox/results/latest.md` is always updated to the newest report
+- The runner prints the report in-terminal using Bun's Markdown renderer with ANSI formatting
+- `test:sandbox` includes native, drizzle, hyperdrive, and nuxthub adapter profiles across supported SQL/KV combinations plus Cloudflare profile runs
+
+Benchmark coverage includes:
+
+- basic CRUD
+- advanced lookup/routing workflows
+- migration-style mapping creation
+- bulk CRUD
+- indexing queries
+- metadata fetch
+- pragma/info queries (provider-specific)
+- counting across shards
+- shard fanout aggregation
+- shard reassignment workflow
+
+How to read benchmark rows:
+
+- Latency cells are formatted as `average / p95` in milliseconds.
+- `FAILED` means the scenario returned an error.
+- `N/A` means the scenario was intentionally skipped in that environment.
+- Use the detailed section for full `avg`, `p50`, `p95`, `min`, `max`, and sample count (`n`).
+
 ## Basic Usage
 
 ```typescript
@@ -88,7 +469,7 @@ collegedb(
 	},
 	async () => {
 		// Create schema on new shards only (existing shards auto-detected)
-		await createSchema(env['db-new-shard']);
+		await createSchema(env['db-new-shard'], 'CREATE TABLE IF NOT EXISTS users (id TEXT PRIMARY KEY, name TEXT, email TEXT)');
 
 		// Insert data (automatically routed to appropriate shard)
 		await run('user-123', 'INSERT INTO users (id, name, email) VALUES (?, ?, ?)', ['user-123', 'Johnson', 'alice@example.com']);
@@ -222,8 +603,6 @@ collegedb(
 
 ### Adding Lookup Keys to Existing Mappings
 
-s
-
 ```typescript
 const mapper = new KVShardMapper(env.KV);
 
@@ -529,13 +908,13 @@ for (const [table, pkColumn] of Object.entries(customIntegration)) {
 }
 ```
 
-## 📚 API Reference
+## API Reference
 
 | Function                                   | Description                                                      | Parameters                 |
 | ------------------------------------------ | ---------------------------------------------------------------- | -------------------------- |
 | `collegedb(config, callback)`              | Initialize CollegeDB, then run a callback                        | `CollegeDBConfig, () => T` |
 | `initialize(config)`                       | Initialize CollegeDB with configuration                          | `CollegeDBConfig`          |
-| `createSchema(d1)`                         | Create database schema on a D1 instance                          | `D1Database`               |
+| `createSchema(db, schema)`                 | Create schema on a shard database                                | `SQLDatabase, string`      |
 | `prepare(key, sql)`                        | Prepare a SQL statement for execution                            | `string, string`           |
 | `run(key, sql, bindings)`                  | Execute a SQL query with primary key routing                     | `string, string, any[]`    |
 | `first(key, sql, bindings)`                | Execute a SQL query and return first result                      | `string, string, any[]`    |
@@ -552,17 +931,33 @@ for (const [table, pkColumn] of Object.entries(customIntegration)) {
 | `getDatabaseSizeForShard(shard)`           | Get size of a specific shard in bytes                            | `string`                   |
 | `flush()`                                  | Clear all shard mappings (development only)                      | `void`                     |
 
+### Provider Adapter Functions
+
+| Function                                                   | Description                                                          | Parameters                                               |
+| ---------------------------------------------------------- | -------------------------------------------------------------------- | -------------------------------------------------------- |
+| `createRedisKVProvider(client, options?)`                  | Adapt a Redis client to CollegeDB's `KVStorage` contract             | `RedisLikeClient, { scanCount?: number }`                |
+| `createValkeyKVProvider(client, options?)`                 | Adapt a Valkey client to CollegeDB's `KVStorage` contract            | `RedisLikeClient, { scanCount?: number }`                |
+| `createNuxtHubKVProvider(client)`                          | Adapt NuxtHub/Unstorage-style KV clients to `KVStorage`              | `NuxtHubKVLike`                                          |
+| `createPostgreSQLProvider(client, sqlTag?)`                | Adapt PostgreSQL or Drizzle PostgreSQL clients                       | `PostgresClientLike, sqlTag?`                            |
+| `createMySQLProvider(client, sqlTag?)`                     | Adapt MySQL/MariaDB or Drizzle MySQL/MariaDB clients                 | `MySQLClientLike, sqlTag?`                               |
+| `createSQLiteProvider(client, sqlTag?)`                    | Adapt SQLite/D1 or Drizzle SQLite/D1 clients                         | `SQLiteClientLike, sqlTag?`                              |
+| `createDrizzleSQLProvider(client, sqlTag)`                 | Generic Drizzle adapter (optional helper)                            | `DrizzleClientLike, DrizzleSqlTagLike`                   |
+| `createHyperdrivePostgresProvider(binding, clientFactory)` | Create a PostgreSQL `SQLDatabase` adapter using a Hyperdrive binding | `HyperdriveBindingLike, HyperdrivePostgresClientFactory` |
+| `createHyperdriveMySQLProvider(binding, clientFactory)`    | Create a MySQL `SQLDatabase` adapter using a Hyperdrive binding      | `HyperdriveBindingLike, HyperdriveMySQLClientFactory`    |
+| `isKVStorage(value)`                                       | Runtime guard for `KVStorage`                                        | `unknown`                                                |
+| `isSQLDatabase(value)`                                     | Runtime guard for `SQLDatabase`                                      | `unknown`                                                |
+
 ### Drop-in Replacement Functions
 
 | Function                                  | Description                                    | Parameters                     |
 | ----------------------------------------- | ---------------------------------------------- | ------------------------------ |
-| `autoDetectAndMigrate(d1, shard, config)` | Automatically detect and migrate existing data | `D1Database, string, config`   |
-| `checkMigrationNeeded(d1, shard, config)` | Check if database needs migration              | `D1Database, string, config`   |
-| `validateTableForSharding(d1, table)`     | Check if table is suitable for sharding        | `D1Database, string`           |
-| `discoverExistingPrimaryKeys(d1, table)`  | Find all primary keys in existing table        | `D1Database, string`           |
-| `integrateExistingDatabase(d1, shard)`    | Complete drop-in integration of existing DB    | `D1Database, string, mapper`   |
+| `autoDetectAndMigrate(d1, shard, config)` | Automatically detect and migrate existing data | `SQLDatabase, string, config`  |
+| `checkMigrationNeeded(d1, shard, config)` | Check if database needs migration              | `SQLDatabase, string, config`  |
+| `validateTableForSharding(d1, table)`     | Check if table is suitable for sharding        | `SQLDatabase, string`          |
+| `discoverExistingPrimaryKeys(d1, table)`  | Find all primary keys in existing table        | `SQLDatabase, string`          |
+| `integrateExistingDatabase(d1, shard)`    | Complete drop-in integration of existing DB    | `SQLDatabase, string, mapper`  |
 | `createMappingsForExistingKeys(keys)`     | Create shard mappings for existing keys        | `string[], string[], strategy` |
-| `listTables(d1)`                          | Get list of tables in database                 | `D1Database`                   |
+| `listTables(d1)`                          | Get list of tables in database                 | `SQLDatabase`                  |
 | `clearMigrationCache()`                   | Clear automatic migration cache                | `void`                         |
 
 ### Error Handling
@@ -645,15 +1040,19 @@ The main configuration interface supports both single strategies and mixed strat
 
 ```typescript
 interface CollegeDBConfig {
-	kv: KVNamespace;
+	kv: KVStorage;
 	coordinator?: DurableObjectNamespace;
-	shards: Record<string, D1Database>;
+	shards: Record<string, SQLDatabase>;
 	strategy?: ShardingStrategy | MixedShardingStrategy;
 	targetRegion?: D1Region;
 	shardLocations?: Record<string, ShardLocation>;
 	disableAutoMigration?: boolean; // Default: false
 	hashShardMappings?: boolean; // Default: true
 	maxDatabaseSize?: number; // Default: undefined (no limit)
+	mappingCacheTtlMs?: number; // Default: 30000
+	knownShardsCacheTtlMs?: number; // Default: 10000
+	sizeCacheTtlMs?: number; // Default: 30000
+	migrationConcurrency?: number; // Default: 25
 }
 ```
 
@@ -706,7 +1105,7 @@ const mixedStrategyConfig: CollegeDBConfig = {
 
 ### Database Size Management
 
-CollegeDB supports automatic size-based shard exclusion to prevent individual shards from becoming too large. This feature helps maintain optimal performance and prevents hitting D1 storage limits.
+CollegeDB supports automatic size-based shard exclusion to prevent individual shards from becoming too large. This feature helps maintain optimal performance and prevents hitting database storage limits.
 
 #### Configuration
 
@@ -782,12 +1181,12 @@ const config: CollegeDBConfig = {
 // "Excluded 2 shards due to size limits: db-east, db-central"
 ```
 
-#### Performance Impact
+#### Size-Limit Performance Impact
 
 - **Size Check Frequency**: Only performed during new allocations (not on reads)
 - **Query Efficiency**: Uses fast SQLite pragmas (microsecond execution time)
 - **Parallel Execution**: Size checks run concurrently across all shards
-- **Caching**: No caching implemented to ensure accurate real-time limits
+- **Caching**: Size checks are cached in-memory (controlled by `sizeCacheTtlMs`, default `30000`)
 
 ### Types
 
@@ -796,6 +1195,13 @@ CollegeDB exports TypeScript types for better development experience and type sa
 | Type                    | Description                    | Example                                             |
 | ----------------------- | ------------------------------ | --------------------------------------------------- |
 | `CollegeDBConfig`       | Main configuration object      | `{ kv, shards, strategy }`                          |
+| `KVStorage`             | Provider-agnostic KV contract  | `createRedisKVProvider(redisClient)`                |
+| `SQLDatabase`           | Provider-agnostic SQL contract | `createPostgreSQLProvider(pgPool)`                  |
+| `NuxtHubKVLike`         | NuxtHub/Unstorage KV contract  | `createNuxtHubKVProvider(kv)`                       |
+| `DrizzleClientLike`     | Minimal Drizzle DB contract    | `createPostgreSQLProvider(drizzleDb, sql)`          |
+| `DrizzleSqlTagLike`     | Drizzle SQL tag contract       | `createSQLiteProvider(drizzleDb, sql)`              |
+| `QueryResult`           | Standard query response shape  | `{ success, results, meta }`                        |
+| `QueryResultMeta`       | Query execution metadata       | `{ duration, changes?, last_row_id? }`              |
 | `ShardingStrategy`      | Single strategy options        | `'hash' \| 'location' \| 'round-robin' \| 'random'` |
 | `MixedShardingStrategy` | Mixed strategy configuration   | `{ read: 'hash', write: 'location' }`               |
 | `OperationType`         | Database operation types       | `'read' \| 'write'`                                 |
@@ -829,7 +1235,7 @@ const config: CollegeDBConfig = {
 };
 ```
 
-## 🏗 Architecture
+## Architecture
 
 ```txt
 ┌─────────────────────────────────────────────────────────────┐
@@ -900,7 +1306,7 @@ const config: CollegeDBConfig = {
 - **Random**: Random shard selection for load balancing
 - **Location**: Geographic proximity-based allocation for optimal latency
 
-## 🌐 Cloudflare Setup
+## Cloudflare Setup
 
 ### 1. Create D1 Databases
 
@@ -918,81 +1324,127 @@ wrangler d1 create collegedb-central
 wrangler kv namespace create "KV"
 ```
 
-### 3. Configure wrangler.toml
-
-```toml
-[[d1_databases]]
-binding = "db-east"
-database_name = "collegedb-east"
-database_id = "your-database-id"
+### 3. Configure wrangler.jsonc
 
-[[d1_databases]]
-binding = "db-west"
-database_name = "collegedb-west"
-database_id = "your-database-id"
-
-[[kv_namespaces]]
-binding = "KV"
-id = "your-kv-namespace-id"
-
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
+```jsonc
+{
+	"$schema": "./node_modules/wrangler/config-schema.json",
+	"name": "collegedb-app",
+	"main": "src/index.ts",
+	"compatibility_date": "2026-04-15",
+	"d1_databases": [
+		{
+			"binding": "db-east",
+			"database_name": "collegedb-east",
+			"database_id": "your-east-database-id"
+		},
+		{
+			"binding": "db-west",
+			"database_name": "collegedb-west",
+			"database_id": "your-west-database-id"
+		}
+	],
+	"kv_namespaces": [
+		{
+			"binding": "KV",
+			"id": "your-kv-namespace-id",
+			"preview_id": "your-kv-preview-id"
+		}
+	],
+	"durable_objects": {
+		"bindings": [
+			{
+				"name": "ShardCoordinator",
+				"class_name": "ShardCoordinator"
+			}
+		]
+	},
+	"migrations": [
+		{
+			"tag": "v1",
+			"new_sqlite_classes": ["ShardCoordinator"]
+		}
+	]
+}
 ```
 
-#### Complete wrangler.toml with ShardCoordinator
+#### Complete wrangler.jsonc with ShardCoordinator
 
-```toml
-name = "collegedb-app"
-main = "src/index.ts"
-compatibility_date = "2024-08-10"
-
-# D1 Database bindings
-[[d1_databases]]
-binding = "db-east"
-database_name = "collegedb-east"
-database_id = "your-east-database-id"
-
-[[d1_databases]]
-binding = "db-west"
-database_name = "collegedb-west"
-database_id = "your-west-database-id"
-
-[[d1_databases]]
-binding = "db-central"
-database_name = "collegedb-central"
-database_id = "your-central-database-id"
-
-# KV namespace for shard mappings
-[[kv_namespaces]]
-binding = "KV"
-id = "your-kv-namespace-id"
-preview_id = "your-kv-preview-id" # For local development
-
-# Durable Object for shard coordination
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
-
-# Environment-specific configurations
-[env.production]
-[[env.production.d1_databases]]
-binding = "db-east"
-database_name = "collegedb-prod-east"
-database_id = "your-prod-east-id"
-
-[[env.production.d1_databases]]
-binding = "db-west"
-database_name = "collegedb-prod-west"
-database_id = "your-prod-west-id"
-
-[[env.production.kv_namespaces]]
-binding = "KV"
-id = "your-prod-kv-namespace-id"
-
-[[env.production.durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
+```jsonc
+{
+	"$schema": "./node_modules/wrangler/config-schema.json",
+	"name": "collegedb-app",
+	"main": "src/index.ts",
+	"compatibility_date": "2026-04-15",
+	"d1_databases": [
+		{
+			"binding": "db-east",
+			"database_name": "collegedb-east",
+			"database_id": "your-east-database-id"
+		},
+		{
+			"binding": "db-west",
+			"database_name": "collegedb-west",
+			"database_id": "your-west-database-id"
+		},
+		{
+			"binding": "db-central",
+			"database_name": "collegedb-central",
+			"database_id": "your-central-database-id"
+		}
+	],
+	"kv_namespaces": [
+		{
+			"binding": "KV",
+			"id": "your-kv-namespace-id",
+			"preview_id": "your-kv-preview-id"
+		}
+	],
+	"durable_objects": {
+		"bindings": [
+			{
+				"name": "ShardCoordinator",
+				"class_name": "ShardCoordinator"
+			}
+		]
+	},
+	"migrations": [
+		{
+			"tag": "v1",
+			"new_sqlite_classes": ["ShardCoordinator"]
+		}
+	],
+	"env": {
+		"production": {
+			"d1_databases": [
+				{
+					"binding": "db-east",
+					"database_name": "collegedb-prod-east",
+					"database_id": "your-prod-east-id"
+				},
+				{
+					"binding": "db-west",
+					"database_name": "collegedb-prod-west",
+					"database_id": "your-prod-west-id"
+				}
+			],
+			"kv_namespaces": [
+				{
+					"binding": "KV",
+					"id": "your-prod-kv-namespace-id"
+				}
+			],
+			"durable_objects": {
+				"bindings": [
+					{
+						"name": "ShardCoordinator",
+						"class_name": "ShardCoordinator"
+					}
+				]
+			}
+		}
+	}
+}
 ```
 
 ### 3.1. Worker Script Setup (Required for ShardCoordinator)
@@ -1058,7 +1510,7 @@ wrangler deploy
 wrangler deploy --env production
 ```
 
-## 📊 Monitoring and Maintenance
+## Monitoring and Maintenance
 
 ### Shard Statistics
 
@@ -1342,7 +1794,7 @@ export default {
 };
 ```
 
-## ⚙️ Performance Analysis
+## Performance Analysis
 
 ### Scaling Performance Comparison
 
@@ -1732,7 +2184,7 @@ initialize({
 - Cost-sensitive deployments at small scale
 - **Single-strategy applications** where reads and writes have identical performance needs
 
-## �🔧 Advanced Configuration
+## Advanced Configuration
 
 ### Custom Allocation Strategy
 
@@ -1762,19 +2214,25 @@ CollegeDB includes an optional **ShardCoordinator** Durable Object that provides
 
 #### Durable Object Setup
 
-First, configure the Durable Object in your `wrangler.toml`:
+First, configure the Durable Object in your `wrangler.jsonc`:
 
-```toml
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
-
-# Export the ShardCoordinator class
-[durable_objects.bindings.script_name]
-# If using modules format
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
+```jsonc
+{
+	"durable_objects": {
+		"bindings": [
+			{
+				"name": "ShardCoordinator",
+				"class_name": "ShardCoordinator"
+			}
+		]
+	},
+	"migrations": [
+		{
+			"tag": "v1",
+			"new_sqlite_classes": ["ShardCoordinator"]
+		}
+	]
+}
 ```
 
 #### Basic Usage with ShardCoordinator
@@ -2043,7 +2501,7 @@ async function allocateWithFallback(coordinator: DurableObjectNamespace, primary
 }
 ```
 
-## 🚀 Quick Reference
+## Quick Reference
 
 ### Strategy Selection Guide
 
@@ -2161,7 +2619,7 @@ async function allocateWithFallback(coordinator: DurableObjectNamespace, primary
 | `me`   | Middle East           | Dubai            |
 | `af`   | Africa                | Johannesburg     |
 
-## 🤝 Contributing
+## Contributing
 
 1. Fork the repository
 2. Create a feature branch: `git checkout -b feature/amazing-feature`
@@ -2169,18 +2627,18 @@ async function allocateWithFallback(coordinator: DurableObjectNamespace, primary
 4. Push to branch: `git push origin feature/amazing-feature`
 5. Submit a pull request
 
-## 📝 License
+## License
 
 This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
 
-## 🔗 Links
+## Links
 
 - [Cloudflare D1 Documentation](https://developers.cloudflare.com/d1/)
 - [Cloudflare KV Documentation](https://developers.cloudflare.com/kv/)
 - [Cloudflare Workers Documentation](https://developers.cloudflare.com/workers/)
 - [Durable Objects Documentation](https://developers.cloudflare.com/durable-objects/)
 
-## 🆘 Support
+## Support
 
 - 📖 [Documentation](https://earth-app.github.io/CollegeDB)
 - 🐛 [Report Issues](https://github.com/earth-app/CollegeDB/issues)