@earth-app/collegedb 1.0.7 → 1.1.0

package/README.md

@@ -1,6 +1,6 @@
 # CollegeDB
 
-> Cloudflare D1 Horizontal Sharding Router
+Universal Database Horizontal Sharding Router
 
 [![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
 [![GitHub Issues](https://img.shields.io/github/issues/earth-app/CollegeDB)](https://github.com/earth-app/CollegeDB/issues)
@@ -8,7 +8,7 @@
 [![GitHub License](https://img.shields.io/github/license/earth-app/CollegeDB)](LICENSE)
 ![NPM Version](https://img.shields.io/npm/v/%40earth-app%2Fcollegedb)
 
-A TypeScript library for **true horizontal scaling** of SQLite-style databases on Cloudflare using D1 and KV. CollegeDB distributes your data across multiple D1 databases, with each table's records split by primary key across different database instances.
+A TypeScript library for **true horizontal scaling** of SQLite-style databases primarily for Cloudflare using D1 and KV, with additional provider adapters for Redis/Valkey KV and PostgreSQL/MySQL/SQLite SQL backends. CollegeDB distributes your data across multiple database shards, with each table's records split by primary key across different database instances.
 
 CollegeDB implements **data distribution** where a single logical table is physically stored across multiple D1 databases:
 
@@ -46,6 +46,7 @@ CollegeDB provides a sharding layer on top of Cloudflare D1 databases, enabling
 - **Scale horizontally** by distributing table data across multiple D1 instances
 - **Route queries automatically** based on primary key mappings
 - **Maintain consistency** with KV-based shard mapping
+- **Run on multiple providers** through `KVStorage` and `SQLDatabase` contracts
 - **Optimize for geography** with location-aware shard allocation
 - **Monitor and rebalance** shard distribution
 - **Handle migrations** between shards seamlessly
@@ -53,6 +54,7 @@ CollegeDB provides a sharding layer on top of Cloudflare D1 databases, enabling
 ## 📦 Features
 
 - **🔀 Automatic Query Routing**: Primary key → shard mapping using Cloudflare KV
+- **🧩 Provider Adapters (v1.1.0)**: Redis/Valkey KV + PostgreSQL/MySQL/SQLite SQL adapters while preserving Cloudflare compatibility
 - **🎯 Multiple Allocation Strategies**: Round-robin, random, hash-based, and location-aware distribution
 - **🔄 Mixed Strategy Support**: Different strategies for reads vs writes (e.g., location for writes, hash for reads)
 - **📊 Shard Coordination**: Durable Objects for allocation and statistics
@@ -62,18 +64,193 @@ CollegeDB provides a sharding layer on top of Cloudflare D1 databases, enabling
 - **⚡ High Performance**: Optimized for Cloudflare Workers runtime
 - **🔧 TypeScript First**: Full type safety and excellent DX
 
+## Benchmark Suite
+
+CollegeDB includes a comprehensive benchmark suite covering real-world latency across provider combinations and Cloudflare Worker routing paths.
+
+### Matrix
+
+| Scenario Key      | Scenario                         | What Happens                                                                                        | Workload Per Run                                                       |
+| ----------------- | -------------------------------- | --------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- |
+| basic_crud        | Basic CRUD round-trip            | Insert, read, update, and delete a user via routed queries.                                         | 20 iterations; 4 routed SQL ops per iteration                          |
+| advanced_usage    | Advanced lookup workflow         | Writes user+post, adds lookup aliases, then validates join and alias-based lookup.                  | 15 iterations; ~5 routed SQL ops + KV lookup-key updates per iteration |
+| migration_mapping | Migration-style mapping creation | Inserts legacy records on a fixed shard, then builds shard mappings in batch and validates routing. | 10 iterations; 20 legacy records mapped per iteration                  |
+| bulk_crud         | Bulk CRUD pressure               | Performs bulk inserts, half updates, and full delete sweep, then validates shard-wide totals.       | 7 iterations; 160 inserts + 80 updates + 160 deletes per iteration     |
+| indexing          | Indexed query scan               | Creates an index on posts(user_id) and repeatedly queries the indexed path.                         | 15 iterations after warmup dataset build                               |
+
+Real-world latency benchmarks across provider combinations (`average / p95`):
+
+| Combination     | Basic CRUD          | Advanced Operations | Migration           | Bulk CRUD             | Indexing             | Overall Avg |
+| --------------- | ------------------- | ------------------- | ------------------- | --------------------- | -------------------- | ----------- |
+| cloudflare      | 13.14 ms / 16.50 ms | 4.43 ms / 9.65 ms   | 27.68 ms / 30.69 ms | 156.30 ms / 163.76 ms | 67.17 ms / 106.63 ms | 28.40 ms    |
+| postgres+redis  | 2.77 ms / 3.90 ms   | 3.11 ms / 4.64 ms   | 6.55 ms / 8.07 ms   | 42.33 ms / 80.67 ms   | 0.34 ms / 0.61 ms    | 5.87 ms     |
+| postgres+valkey | 1.65 ms / 2.23 ms   | 2.10 ms / 2.82 ms   | 5.60 ms / 6.05 ms   | 33.13 ms / 43.69 ms   | 0.30 ms / 0.46 ms    | 4.64 ms     |
+| mysql+redis     | 5.11 ms / 8.38 ms   | 5.45 ms / 8.51 ms   | 27.41 ms / 61.56 ms | 92.70 ms / 139.70 ms  | 0.49 ms / 1.22 ms    | 13.91 ms    |
+| mysql+valkey    | 4.99 ms / 6.66 ms   | 4.21 ms / 6.42 ms   | 21.68 ms / 27.20 ms | 87.67 ms / 109.44 ms  | 0.55 ms / 1.92 ms    | 12.54 ms    |
+| mariadb+redis   | 2.64 ms / 5.90 ms   | 3.02 ms / 7.55 ms   | 6.48 ms / 7.66 ms   | 46.99 ms / 58.08 ms   | 0.37 ms / 1.08 ms    | 6.29 ms     |
+| mariadb+valkey  | 2.34 ms / 4.58 ms   | 2.91 ms / 5.69 ms   | 5.73 ms / 7.35 ms   | 45.04 ms / 61.42 ms   | 0.36 ms / 0.79 ms    | 5.96 ms     |
+| sqlite+redis    | 2.21 ms / 3.84 ms   | 2.43 ms / 3.14 ms   | 10.49 ms / 17.31 ms | 140.85 ms / 184.48 ms | 0.07 ms / 0.14 ms    | 15.87 ms    |
+| sqlite+valkey   | 1.36 ms / 1.80 ms   | 2.06 ms / 2.70 ms   | 6.36 ms / 8.77 ms   | 121.13 ms / 156.31 ms | 0.06 ms / 0.14 ms    | 13.42 ms    |
+
+### Overview
+
+CollegeDB includes an integration benchmark suite covering both local provider matrices and Cloudflare Worker routing paths.
+
+Top-level benchmark scenarios:
+
+- `basic_crud`: insert/read/update/delete round-trip routing
+- `advanced_usage`: join + multi-key lookup behavior
+- `migration_mapping`: batch mapping creation for existing keys
+- `bulk_crud`: high-volume insert/update/delete flow
+- `indexing`: indexed query latency under warm data
+- `metadata_fetch`: schema/metadata query latency
+- `pragma_or_info`: provider-specific pragma/info query latency
+- `counting`: shard-wide aggregate counting
+- `shard_fanout`: all-shards fanout query aggregation
+- `reassignment`: shard reassignment and routed-read validation
+
+Benchmark reports include:
+
+- an interpretation guide (`How To Read This Report`)
+- a benchmark catalog with per-run workload details
+- a compact overall matrix (passed/failed/skipped + overall average)
+- split scenario matrices for core workload latency and introspection/routing latency
+
 ## Installation
 
 ```bash
-bun add collegedb
+bun add @earth-app/collegedb
 # or
-npm install collegedb
+npm install @earth-app/collegedb
 ```
 
+## Provider Adapters
+
+CollegeDB can run with either native Cloudflare bindings or custom providers as long as they match the exported `KVStorage` and `SQLDatabase` interfaces.
+
+Supported adapters:
+
+- `createRedisKVProvider`
+- `createValkeyKVProvider`
+- `createPostgresSQLProvider`
+- `createMySQLSQLProvider`
+- `createSQLiteSQLProvider`
+- `createHyperdrivePostgresProvider`
+- `createHyperdriveMySQLProvider`
+
+```typescript
+import { createClient as createRedisClient } from 'redis';
+import { Pool } from 'pg';
+import { createPostgresSQLProvider, createRedisKVProvider, initialize, run, type CollegeDBConfig } from '@earth-app/collegedb';
+
+const redisClient = createRedisClient({ url: process.env.REDIS_URL });
+const pgPool = new Pool({ connectionString: process.env.POSTGRES_URL });
+
+const config: CollegeDBConfig = {
+	kv: createRedisKVProvider(redisClient),
+	shards: {
+		'pg-east': createPostgresSQLProvider(pgPool)
+	},
+	strategy: 'hash',
+	disableAutoMigration: true
+};
+
+async function bootstrap() {
+	await redisClient.connect();
+	initialize(config);
+	await run('user-1', 'INSERT INTO users (id, name) VALUES (?, ?)', ['user-1', 'Taylor']);
+}
+
+bootstrap().catch(console.error);
+```
+
+For Hyperdrive-backed SQL connections, use `createHyperdrivePostgresProvider` or `createHyperdriveMySQLProvider` with your database client factory.
+
+For a complete non-Cloudflare setup, see `examples/provider-sandbox.ts`.
+
+## Sandbox Benchmarks (Docker Compose)
+
+CollegeDB ships with an integration sandbox runner that benchmarks real latency across provider combinations.
+
+Requirements:
+
+- Docker + Docker Compose plugin
+- Bun
+- Wrangler (installed as a dev dependency and invoked by scripts)
+
+The Cloudflare benchmark path runs against the dedicated sandbox worker:
+
+- Worker entry: `sandbox/worker.ts`
+- Wrangler config: `sandbox/wrangler.jsonc`
+
+Main commands:
+
+```bash
+# Run full SQL x KV matrix plus Cloudflare local benchmark
+bun run test:sandbox
+
+# Run full SQL x KV matrix only
+bun run test:sandbox:all
+
+# Run Cloudflare local benchmark only (wrangler dev --local)
+bun run test:sandbox:cloudflare
+```
+
+Provider filters:
+
+```bash
+# One SQL provider against both KV providers
+bun run test:sandbox:mysql
+bun run test:sandbox:postgres
+bun run test:sandbox:mariadb
+bun run test:sandbox:sqlite
+
+# One KV provider against all SQL providers
+bun run test:sandbox:redis
+bun run test:sandbox:valkey
+
+# Explicit pairwise combinations
+bun run test:sandbox:postgres+redis
+bun run test:sandbox:postgres+valkey
+bun run test:sandbox:mysql+redis
+bun run test:sandbox:mysql+valkey
+bun run test:sandbox:mariadb+redis
+bun run test:sandbox:mariadb+valkey
+bun run test:sandbox:sqlite+redis
+bun run test:sandbox:sqlite+valkey
+```
+
+Output behavior:
+
+- Every run writes a timestamped Markdown report to `sandbox/results/`
+- `sandbox/results/latest.md` is always updated to the newest report
+- The runner prints the report in-terminal using Bun's Markdown renderer with ANSI formatting
+- `test:sandbox` produces a matrix for all SQL x KV combinations; filtered commands produce matrix subsets
+
+Benchmark coverage includes:
+
+- basic CRUD
+- advanced lookup/routing workflows
+- migration-style mapping creation
+- bulk CRUD
+- indexing queries
+- metadata fetch
+- pragma/info queries (provider-specific)
+- counting across shards
+- shard fanout aggregation
+- shard reassignment workflow
+
+How to read benchmark rows:
+
+- Latency cells are formatted as `average / p95` in milliseconds.
+- `FAILED` means the scenario returned an error.
+- `N/A` means the scenario was intentionally skipped in that environment.
+- Use the detailed section for full `avg`, `p50`, `p95`, `min`, `max`, and sample count (`n`).
+
 ## Basic Usage
 
 ```typescript
-import { collegedb, createSchema, run, first } from 'collegedb';
+import { collegedb, createSchema, run, first } from '@earth-app/collegedb';
 
 // Initialize with your Cloudflare bindings (existing databases work automatically!)
 collegedb(
@@ -88,7 +265,7 @@ collegedb(
 	},
 	async () => {
 		// Create schema on new shards only (existing shards auto-detected)
-		await createSchema(env['db-new-shard']);
+		await createSchema(env['db-new-shard'], 'CREATE TABLE IF NOT EXISTS users (id TEXT PRIMARY KEY, name TEXT, email TEXT)');
 
 		// Insert data (automatically routed to appropriate shard)
 		await run('user-123', 'INSERT INTO users (id, name, email) VALUES (?, ?, ?)', ['user-123', 'Johnson', 'alice@example.com']);
@@ -104,7 +281,7 @@ collegedb(
 ### Geographic Distribution Example
 
 ```typescript
-import { collegedb, first, run } from 'collegedb';
+import { collegedb, first, run } from '@earth-app/collegedb';
 
 // Optimize for North American users with geographic sharding
 collegedb(
@@ -141,7 +318,7 @@ collegedb(
 ### Mixed Strategy Example
 
 ```typescript
-import { collegedb, first, run, type MixedShardingStrategy } from 'collegedb';
+import { collegedb, first, run, type MixedShardingStrategy } from '@earth-app/collegedb';
 
 // Use location strategy for writes (optimal data placement) and hash for reads (optimal performance)
 const mixedStrategy: MixedShardingStrategy = {
@@ -194,7 +371,7 @@ This approach provides:
 CollegeDB supports **multiple lookup keys** for the same record, allowing you to query by username, email, ID, or any unique identifier. Keys are automatically hashed with SHA-256 for security and privacy.
 
 ```typescript
-import { collegedb, first, run, KVShardMapper } from 'collegedb';
+import { collegedb, first, run, KVShardMapper } from '@earth-app/collegedb';
 
 collegedb(
 	{
@@ -222,8 +399,6 @@ collegedb(
 
 ### Adding Lookup Keys to Existing Mappings
 
-s
-
 ```typescript
 const mapper = new KVShardMapper(env.KV);
 
@@ -291,7 +466,7 @@ CollegeDB supports **seamless, automatic integration** with existing D1 database
 4. **KV Namespace**: A Cloudflare KV namespace for storing shard mappings
 
 ```typescript
-import { collegedb, first, run } from 'collegedb';
+import { collegedb, first, run } from '@earth-app/collegedb';
 
 // Add your existing databases as shards - that's it!
 collegedb(
@@ -321,7 +496,7 @@ collegedb(
 You can manually validate databases before integration if needed:
 
 ```typescript
-import { validateTableForSharding, listTables } from 'collegedb';
+import { validateTableForSharding, listTables } from '@earth-app/collegedb';
 
 // Check database structure
 const tables = await listTables(env.ExistingDB);
@@ -343,7 +518,7 @@ for (const table of tables) {
 If you want to inspect existing data before automatic migration:
 
 ```typescript
-import { discoverExistingPrimaryKeys } from 'collegedb';
+import { discoverExistingPrimaryKeys } from '@earth-app/collegedb';
 
 // Discover all user IDs in existing users table
 const userIds = await discoverExistingPrimaryKeys(env.ExistingDB, 'users');
@@ -358,7 +533,7 @@ const orderIds = await discoverExistingPrimaryKeys(env.ExistingDB, 'orders', 'or
 For complete control over the integration process:
 
 ```typescript
-import { integrateExistingDatabase, KVShardMapper } from 'collegedb';
+import { integrateExistingDatabase, KVShardMapper } from '@earth-app/collegedb';
 
 const mapper = new KVShardMapper(env.KV);
 
@@ -386,7 +561,7 @@ if (result.success) {
 After integration, initialize CollegeDB with your existing databases as shards:
 
 ```typescript
-import { initialize, first } from 'collegedb';
+import { initialize, first } from '@earth-app/collegedb';
 
 // Include existing databases as shards
 initialize({
@@ -409,7 +584,7 @@ const user = await first('existing-user-123', 'SELECT * FROM users WHERE id = ?'
 The simplest possible integration - just add your existing databases:
 
 ```typescript
-import { initialize, first, run } from 'collegedb';
+import { initialize, first, run } from '@earth-app/collegedb';
 
 export default {
 	async fetch(request: Request, env: Env): Promise<Response> {
@@ -479,12 +654,12 @@ console.log(`Would process ${testResult.totalRecords} records from ${testResult.
 
 ```typescript
 // Simple rollback - clear all mappings
-import { KVShardMapper } from 'collegedb';
+import { KVShardMapper } from '@earth-app/collegedb';
 const mapper = new KVShardMapper(env.KV);
 await mapper.clearAllMappings(); // Returns to pre-migration state
 
 // Or clear cache to force re-detection
-import { clearMigrationCache } from 'collegedb';
+import { clearMigrationCache } from '@earth-app/collegedb';
 clearMigrationCache(); // Forces fresh migration check
 ```
 
@@ -535,7 +710,7 @@ for (const [table, pkColumn] of Object.entries(customIntegration)) {
 | ------------------------------------------ | ---------------------------------------------------------------- | -------------------------- |
 | `collegedb(config, callback)`              | Initialize CollegeDB, then run a callback                        | `CollegeDBConfig, () => T` |
 | `initialize(config)`                       | Initialize CollegeDB with configuration                          | `CollegeDBConfig`          |
-| `createSchema(d1)`                         | Create database schema on a D1 instance                          | `D1Database`               |
+| `createSchema(db, schema)`                 | Create schema on a shard database                                | `SQLDatabase, string`      |
 | `prepare(key, sql)`                        | Prepare a SQL statement for execution                            | `string, string`           |
 | `run(key, sql, bindings)`                  | Execute a SQL query with primary key routing                     | `string, string, any[]`    |
 | `first(key, sql, bindings)`                | Execute a SQL query and return first result                      | `string, string, any[]`    |
@@ -549,19 +724,34 @@ for (const [table, pkColumn] of Object.entries(customIntegration)) {
 | `reassignShard(key, newShard)`             | Move primary key to different shard                              | `string, string`           |
 | `listKnownShards()`                        | Get list of available shards                                     | `void`                     |
 | `getShardStats()`                          | Get statistics for all shards                                    | `void`                     |
+| `getDatabaseSizeForShard(shard)`           | Get size of a specific shard in bytes                            | `string`                   |
 | `flush()`                                  | Clear all shard mappings (development only)                      | `void`                     |
 
+### Provider Adapter Functions
+
+| Function                                                   | Description                                                          | Parameters                                               |
+| ---------------------------------------------------------- | -------------------------------------------------------------------- | -------------------------------------------------------- |
+| `createRedisKVProvider(client, options?)`                  | Adapt a Redis client to CollegeDB's `KVStorage` contract             | `RedisLikeClient, { scanCount?: number }`                |
+| `createValkeyKVProvider(client, options?)`                 | Adapt a Valkey client to CollegeDB's `KVStorage` contract            | `RedisLikeClient, { scanCount?: number }`                |
+| `createPostgresSQLProvider(client)`                        | Adapt a PostgreSQL client/pool to `SQLDatabase`                      | `PostgresClientLike`                                     |
+| `createMySQLSQLProvider(client)`                           | Adapt a MySQL/MariaDB client to `SQLDatabase`                        | `MySQLClientLike`                                        |
+| `createSQLiteSQLProvider(client)`                          | Adapt a SQLite client to `SQLDatabase`                               | `SQLiteClientLike`                                       |
+| `createHyperdrivePostgresProvider(binding, clientFactory)` | Create a PostgreSQL `SQLDatabase` adapter using a Hyperdrive binding | `HyperdriveBindingLike, HyperdrivePostgresClientFactory` |
+| `createHyperdriveMySQLProvider(binding, clientFactory)`    | Create a MySQL `SQLDatabase` adapter using a Hyperdrive binding      | `HyperdriveBindingLike, HyperdriveMySQLClientFactory`    |
+| `isKVStorage(value)`                                       | Runtime guard for `KVStorage`                                        | `unknown`                                                |
+| `isSQLDatabase(value)`                                     | Runtime guard for `SQLDatabase`                                      | `unknown`                                                |
+
 ### Drop-in Replacement Functions
 
 | Function                                  | Description                                    | Parameters                     |
 | ----------------------------------------- | ---------------------------------------------- | ------------------------------ |
-| `autoDetectAndMigrate(d1, shard, config)` | Automatically detect and migrate existing data | `D1Database, string, config`   |
-| `checkMigrationNeeded(d1, shard, config)` | Check if database needs migration              | `D1Database, string, config`   |
-| `validateTableForSharding(d1, table)`     | Check if table is suitable for sharding        | `D1Database, string`           |
-| `discoverExistingPrimaryKeys(d1, table)`  | Find all primary keys in existing table        | `D1Database, string`           |
-| `integrateExistingDatabase(d1, shard)`    | Complete drop-in integration of existing DB    | `D1Database, string, mapper`   |
+| `autoDetectAndMigrate(d1, shard, config)` | Automatically detect and migrate existing data | `SQLDatabase, string, config`  |
+| `checkMigrationNeeded(d1, shard, config)` | Check if database needs migration              | `SQLDatabase, string, config`  |
+| `validateTableForSharding(d1, table)`     | Check if table is suitable for sharding        | `SQLDatabase, string`          |
+| `discoverExistingPrimaryKeys(d1, table)`  | Find all primary keys in existing table        | `SQLDatabase, string`          |
+| `integrateExistingDatabase(d1, shard)`    | Complete drop-in integration of existing DB    | `SQLDatabase, string, mapper`  |
 | `createMappingsForExistingKeys(keys)`     | Create shard mappings for existing keys        | `string[], string[], strategy` |
-| `listTables(d1)`                          | Get list of tables in database                 | `D1Database`                   |
+| `listTables(d1)`                          | Get list of tables in database                 | `SQLDatabase`                  |
 | `clearMigrationCache()`                   | Clear automatic migration cache                | `void`                         |
 
 ### Error Handling
@@ -612,7 +802,7 @@ The `ShardCoordinator` is an optional Durable Object that provides centralized s
 #### Usage Example
 
 ```typescript
-import { ShardCoordinator } from 'collegedb';
+import { ShardCoordinator } from '@earth-app/collegedb';
 
 // Export for Cloudflare Workers runtime
 export { ShardCoordinator };
@@ -644,14 +834,19 @@ The main configuration interface supports both single strategies and mixed strat
 
 ```typescript
 interface CollegeDBConfig {
-	kv: KVNamespace;
+	kv: KVStorage;
 	coordinator?: DurableObjectNamespace;
-	shards: Record<string, D1Database>;
+	shards: Record<string, SQLDatabase>;
 	strategy?: ShardingStrategy | MixedShardingStrategy;
 	targetRegion?: D1Region;
 	shardLocations?: Record<string, ShardLocation>;
 	disableAutoMigration?: boolean; // Default: false
 	hashShardMappings?: boolean; // Default: true
+	maxDatabaseSize?: number; // Default: undefined (no limit)
+	mappingCacheTtlMs?: number; // Default: 30000
+	knownShardsCacheTtlMs?: number; // Default: 10000
+	sizeCacheTtlMs?: number; // Default: 30000
+	migrationConcurrency?: number; // Default: 25
 }
 ```
 
@@ -702,6 +897,91 @@ const mixedStrategyConfig: CollegeDBConfig = {
 };
 ```
 
+### Database Size Management
+
+CollegeDB supports automatic size-based shard exclusion to prevent individual shards from becoming too large. This feature helps maintain optimal performance and prevents hitting database storage limits.
+
+#### Configuration
+
+```typescript
+const config: CollegeDBConfig = {
+	kv: env.KV,
+	shards: {
+		'db-east': env.DB_EAST,
+		'db-west': env.DB_WEST,
+		'db-central': env.DB_CENTRAL
+	},
+	strategy: 'hash',
+	maxDatabaseSize: 500 * 1024 * 1024 // 500 MB limit per shard
+};
+```
+
+#### How It Works
+
+When `maxDatabaseSize` is configured:
+
+1. **Allocation Phase**: Before allocating new records, CollegeDB checks each shard's size using efficient SQLite pragmas
+2. **Size Filtering**: Shards exceeding the limit are excluded from new allocations
+3. **Fallback Protection**: If all shards exceed the limit, allocation continues to prevent complete failure
+4. **Existing Records**: Records already mapped to oversized shards remain accessible
+
+#### Size Check Implementation
+
+The size check uses SQLite's `PRAGMA page_count` and `PRAGMA page_size` for accurate, low-overhead size calculation:
+
+```sql
+-- Efficient size calculation (used internally)
+PRAGMA page_count;  -- Returns number of database pages
+PRAGMA page_size;   -- Returns size of each page in bytes
+-- Total size = page_count × page_size
+```
+
+#### Usage Examples
+
+```typescript
+// Conservative limit for high-performance scenarios
+const performanceConfig: CollegeDBConfig = {
+	// ... other config
+	maxDatabaseSize: 100 * 1024 * 1024, // 100 MB per shard
+	strategy: 'round-robin' // Ensures even distribution
+};
+
+// Standard production limit
+const productionConfig: CollegeDBConfig = {
+	// ... other config
+	maxDatabaseSize: 1024 * 1024 * 1024, // 1 GB per shard
+	strategy: 'hash' // Consistent allocation
+};
+
+// Check individual shard sizes
+import { getDatabaseSizeForShard } from '@earth-app/collegedb';
+
+const eastSize = await getDatabaseSizeForShard('db-east');
+console.log(`East shard: ${Math.round(eastSize / 1024 / 1024)} MB`);
+```
+
+#### Debug Logging
+
+Enable debug logging to monitor size-based exclusions:
+
+```typescript
+const config: CollegeDBConfig = {
+	// ... other config
+	maxDatabaseSize: 500 * 1024 * 1024,
+	debug: true // Logs when shards are excluded due to size
+};
+
+// Console output example:
+// "Excluded 2 shards due to size limits: db-east, db-central"
+```
+
+#### Size-Limit Performance Impact
+
+- **Size Check Frequency**: Only performed during new allocations (not on reads)
+- **Query Efficiency**: Uses fast SQLite pragmas (microsecond execution time)
+- **Parallel Execution**: Size checks run concurrently across all shards
+- **Caching**: Size checks are cached in-memory (controlled by `sizeCacheTtlMs`, default `30000`)
+
 ### Types
 
 CollegeDB exports TypeScript types for better development experience and type safety:
@@ -709,6 +989,10 @@ CollegeDB exports TypeScript types for better development experience and type sa
 | Type                    | Description                    | Example                                             |
 | ----------------------- | ------------------------------ | --------------------------------------------------- |
 | `CollegeDBConfig`       | Main configuration object      | `{ kv, shards, strategy }`                          |
+| `KVStorage`             | Provider-agnostic KV contract  | `createRedisKVProvider(redisClient)`                |
+| `SQLDatabase`           | Provider-agnostic SQL contract | `createPostgresSQLProvider(pgPool)`                 |
+| `QueryResult`           | Standard query response shape  | `{ success, results, meta }`                        |
+| `QueryResultMeta`       | Query execution metadata       | `{ duration, changes?, last_row_id? }`              |
 | `ShardingStrategy`      | Single strategy options        | `'hash' \| 'location' \| 'round-robin' \| 'random'` |
 | `MixedShardingStrategy` | Mixed strategy configuration   | `{ read: 'hash', write: 'location' }`               |
 | `OperationType`         | Database operation types       | `'read' \| 'write'`                                 |
@@ -719,7 +1003,7 @@ CollegeDB exports TypeScript types for better development experience and type sa
 #### Mixed Strategy Configuration
 
 ```typescript
-import type { MixedShardingStrategy, CollegeDBConfig } from 'collegedb';
+import type { MixedShardingStrategy, CollegeDBConfig } from '@earth-app/collegedb';
 
 // Type-safe mixed strategy configuration
 const mixedStrategy: MixedShardingStrategy = {
@@ -831,81 +1115,127 @@ wrangler d1 create collegedb-central
 wrangler kv namespace create "KV"
 ```
 
-### 3. Configure wrangler.toml
+### 3. Configure wrangler.jsonc
 
-```toml
-[[d1_databases]]
-binding = "db-east"
-database_name = "collegedb-east"
-database_id = "your-database-id"
-
-[[d1_databases]]
-binding = "db-west"
-database_name = "collegedb-west"
-database_id = "your-database-id"
-
-[[kv_namespaces]]
-binding = "KV"
-id = "your-kv-namespace-id"
-
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
+```jsonc
+{
+	"$schema": "./node_modules/wrangler/config-schema.json",
+	"name": "collegedb-app",
+	"main": "src/index.ts",
+	"compatibility_date": "2026-04-15",
+	"d1_databases": [
+		{
+			"binding": "db-east",
+			"database_name": "collegedb-east",
+			"database_id": "your-east-database-id"
+		},
+		{
+			"binding": "db-west",
+			"database_name": "collegedb-west",
+			"database_id": "your-west-database-id"
+		}
+	],
+	"kv_namespaces": [
+		{
+			"binding": "KV",
+			"id": "your-kv-namespace-id",
+			"preview_id": "your-kv-preview-id"
+		}
+	],
+	"durable_objects": {
+		"bindings": [
+			{
+				"name": "ShardCoordinator",
+				"class_name": "ShardCoordinator"
+			}
+		]
+	},
+	"migrations": [
+		{
+			"tag": "v1",
+			"new_sqlite_classes": ["ShardCoordinator"]
+		}
+	]
+}
 ```
 
-#### Complete wrangler.toml with ShardCoordinator
+#### Complete wrangler.jsonc with ShardCoordinator
 
-```toml
-name = "collegedb-app"
-main = "src/index.ts"
-compatibility_date = "2024-08-10"
-
-# D1 Database bindings
-[[d1_databases]]
-binding = "db-east"
-database_name = "collegedb-east"
-database_id = "your-east-database-id"
-
-[[d1_databases]]
-binding = "db-west"
-database_name = "collegedb-west"
-database_id = "your-west-database-id"
-
-[[d1_databases]]
-binding = "db-central"
-database_name = "collegedb-central"
-database_id = "your-central-database-id"
-
-# KV namespace for shard mappings
-[[kv_namespaces]]
-binding = "KV"
-id = "your-kv-namespace-id"
-preview_id = "your-kv-preview-id" # For local development
-
-# Durable Object for shard coordination
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
-
-# Environment-specific configurations
-[env.production]
-[[env.production.d1_databases]]
-binding = "db-east"
-database_name = "collegedb-prod-east"
-database_id = "your-prod-east-id"
-
-[[env.production.d1_databases]]
-binding = "db-west"
-database_name = "collegedb-prod-west"
-database_id = "your-prod-west-id"
-
-[[env.production.kv_namespaces]]
-binding = "KV"
-id = "your-prod-kv-namespace-id"
-
-[[env.production.durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
+```jsonc
+{
+	"$schema": "./node_modules/wrangler/config-schema.json",
+	"name": "collegedb-app",
+	"main": "src/index.ts",
+	"compatibility_date": "2026-04-15",
+	"d1_databases": [
+		{
+			"binding": "db-east",
+			"database_name": "collegedb-east",
+			"database_id": "your-east-database-id"
+		},
+		{
+			"binding": "db-west",
+			"database_name": "collegedb-west",
+			"database_id": "your-west-database-id"
+		},
+		{
+			"binding": "db-central",
+			"database_name": "collegedb-central",
+			"database_id": "your-central-database-id"
+		}
+	],
+	"kv_namespaces": [
+		{
+			"binding": "KV",
+			"id": "your-kv-namespace-id",
+			"preview_id": "your-kv-preview-id"
+		}
+	],
+	"durable_objects": {
+		"bindings": [
+			{
+				"name": "ShardCoordinator",
+				"class_name": "ShardCoordinator"
+			}
+		]
+	},
+	"migrations": [
+		{
+			"tag": "v1",
+			"new_sqlite_classes": ["ShardCoordinator"]
+		}
+	],
+	"env": {
+		"production": {
+			"d1_databases": [
+				{
+					"binding": "db-east",
+					"database_name": "collegedb-prod-east",
+					"database_id": "your-prod-east-id"
+				},
+				{
+					"binding": "db-west",
+					"database_name": "collegedb-prod-west",
+					"database_id": "your-prod-west-id"
+				}
+			],
+			"kv_namespaces": [
+				{
+					"binding": "KV",
+					"id": "your-prod-kv-namespace-id"
+				}
+			],
+			"durable_objects": {
+				"bindings": [
+					{
+						"name": "ShardCoordinator",
+						"class_name": "ShardCoordinator"
+					}
+				]
+			}
+		}
+	}
+}
 ```
 
 ### 3.1. Worker Script Setup (Required for ShardCoordinator)
@@ -914,7 +1244,7 @@ Create your main worker file with ShardCoordinator export:
 
 ```typescript
 // src/index.ts
-import { collegedb, ShardCoordinator, first, run } from 'collegedb';
+import { collegedb, ShardCoordinator, first, run } from '@earth-app/collegedb';
 
 // IMPORTANT: Export ShardCoordinator for Cloudflare Workers runtime
 export { ShardCoordinator };
@@ -978,7 +1308,7 @@ wrangler deploy --env production
 #### Using CollegeDB Functions
 
 ```typescript
-import { getShardStats, listKnownShards } from 'collegedb';
+import { getShardStats, listKnownShards } from '@earth-app/collegedb';
 
 // Get detailed statistics
 const stats = await getShardStats();
@@ -1086,7 +1416,7 @@ export default {
 ### Shard Rebalancing
 
 ```typescript
-import { reassignShard } from 'collegedb';
+import { reassignShard } from '@earth-app/collegedb';
 
 // Move a primary key to a different shard
 await reassignShard('user-123', 'db-west');
@@ -1675,25 +2005,31 @@ CollegeDB includes an optional **ShardCoordinator** Durable Object that provides
 
 #### Durable Object Setup
 
-First, configure the Durable Object in your `wrangler.toml`:
-
-```toml
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
+First, configure the Durable Object in your `wrangler.jsonc`:
 
-# Export the ShardCoordinator class
-[durable_objects.bindings.script_name]
-# If using modules format
-[[durable_objects.bindings]]
-name = "ShardCoordinator"
-class_name = "ShardCoordinator"
+```jsonc
+{
+	"durable_objects": {
+		"bindings": [
+			{
+				"name": "ShardCoordinator",
+				"class_name": "ShardCoordinator"
+			}
+		]
+	},
+	"migrations": [
+		{
+			"tag": "v1",
+			"new_sqlite_classes": ["ShardCoordinator"]
+		}
+	]
+}
 ```
 
 #### Basic Usage with ShardCoordinator
 
 ```typescript
-import { collegedb, ShardCoordinator } from 'collegedb';
+import { collegedb, ShardCoordinator } from '@earth-app/collegedb';
 
 // Export the Durable Object class for Cloudflare Workers
 export { ShardCoordinator };