prisma-sharding 0.0.5 → 0.0.7

package/README.md

@@ -2,6 +2,13 @@
 
 Lightweight database sharding library for Prisma with connection pooling, health monitoring, and CLI tools.
 
+## Backward Compatibility Policy
+
+This package treats its existing public API as stable. Production-hardening releases do not rename
+commands or public methods, change return shapes, move the package import path, remove exported
+errors, or add required fields to public outputs. Internal routing, execution, health, and CLI
+safety fixes preserve existing application code and the compact default CLI format.
+
 ## Installation
 
 ```bash
@@ -38,76 +45,77 @@ await sharding.connect();
 
 ## API
 
-| Method                       | Description                                   |
-| ---------------------------- | --------------------------------------------- |
-| `getShard(key)`              | Get Prisma client for a given key             |
-| `getShardById(shardId)`      | Get Prisma client by shard ID                 |
-| `getRandomShard()`           | Get random shard (for new records)            |
-| `findFirst(fn)`              | Search across all shards, return first result |
-| `runOnAll(fn)`               | Execute on all shards                         |
-| `getHealth()`                | Get health status of all shards               |
-| `connect()` / `disconnect()` | Lifecycle methods                             |
-
-### Step 2: Create a User (Assign to a Shard)
-
-New records should be created on a random shard for even distribution.
+| Method                       | Description                                    |
+| ---------------------------- | ---------------------------------------------- |
+| `getShard(key)`              | Deterministic client for a routing key         |
+| `getShardById(shardId)`      | Client for a persisted shard owner             |
+| `getRandomShard()`           | Random assignment; ownership must be recorded  |
+| `findFirst(fn)`              | Bounded exception-path search across shards    |
+| `runOnAll(fn)`               | Bounded admin/analytics execution              |
+| `getHealth()`                | Health status using the existing output shape  |
+| `connect()` / `disconnect()` | Lifecycle methods                              |
 
-```ts
-import { sharding } from '@/config/prisma';
-
-const client = sharding.getRandomShard();
+## Shard Ownership
 
-const user = await client.user.create({
-  data: {
-    email: 'user@example.com',
-    username: 'new_user',
-  },
-});
-```
+Every record needs one authoritative shard owner. Choose one of these patterns and use it
+consistently. Cross-shard search is a recovery path, not an ownership strategy.
 
-## Step 3: Access User by ID (Shard Routing)
+### Pattern A: Deterministic Ownership
 
-When you have a user ID, Prisma Sharding routes you to the correct shard automatically.
+Generate or obtain the routing key before inserting the record, then use the same key for every
+future operation:
 
-```ts
-const userId = 'abc123';
+```typescript
+import { sharding } from '@/config/prisma';
 
+const userId = crypto.randomUUID();
 const client = sharding.getShard(userId);
+const user = await client.user.create({
+  data: { id: userId, email: 'user@example.com', username: 'new_user' },
+});
 
-const user = await client.user.findUnique({
+const sameUser = await sharding.getShard(userId).user.findUnique({
   where: { id: userId },
 });
 ```
 
-`Important rule:`
-
-Once you get the shard client using a user ID, **all future operations for that user must use this same client**.
-
-That includes:
+Modulo routing uses the existing `hashString(key) % shardCount` placement. The hash function and
+configured shard order are data-placement contracts: changing either can move existing records and
+requires an explicit migration or dual-read plan. Consistent hashing also preserves configured
+shard IDs and supports custom IDs such as `tenant-east`.
 
-- Reading user data
-- Updating user data
-- Creating related records (profiles, posts, settings, etc)
+### Pattern B: Assigned Ownership
 
-Every user belongs to exactly one shard. Their entire data lives on that shard only.
+Random assignment can distribute new records, but the application must persist the assigned shard
+ID in a directory table, tenant registry, or equivalent ownership metadata:
 
-Do **not** switch shards or use a random shard for user related actions.
+```typescript
+const { client, shardId } = sharding.getRandomShardWithInfo();
+const user = await client.user.create({ data: { email, username } });
 
-Always do this:
+await shardDirectory.create({ data: { recordId: user.id, shardId } });
 
-```ts
-const client = sharding.getShard(userId);
+const ownership = await shardDirectory.findUniqueOrThrow({
+  where: { recordId: user.id },
+});
+const sameUser = await sharding
+  .getShardById(ownership.shardId)
+  .user.findUnique({ where: { id: user.id } });
 ```
 
-This guarantees all user data stays on the correct shard and avoids cross shard bugs.
+The existing `getRandomShard()` method still returns only a client. Calling it for a write and
+later calling `getShard(record.id)` is **not guaranteed to select the same shard**. If you use
+`getRandomShard()`, your application needs another reliable way to record which shard was selected.
+`weight` affects random assignment only; it never changes deterministic `getShard(key)` placement.
 
-### Step 4: Find User Without ID (Cross Shard Search)
+### Find Without Ownership Metadata
 
-If you do not have the user ID, search all shards in parallel.
-Use this only when necessary.
+`findFirst()` is bounded, timed, health-aware, and returns when the first non-null result arrives.
+Even so, one call can create work on multiple databases. Treat it as an exception, recovery, or
+administrative path. At high traffic it should not be the normal login, email lookup, user lookup,
+or tenant lookup path; maintain shard ownership metadata instead.
 
 ```typescript
-// Find user by email across ALL shards (parallel execution)
 const { result: user, client } = await sharding.findFirst(async (c) =>
   c.user.findFirst({ where: { email } })
 );
@@ -121,10 +129,10 @@ if (user && client) {
 }
 ```
 
-## Step 5: Run on All Shards (Admin or Analytics)
+### Run on All Shards
 
 ```typescript
-// Get counts from all shards
+// Appropriate for bounded admin or analytics work, not a normal request path.
 const counts = await sharding.runOnAll(async (client) => client.user.count());
 const totalUsers = counts.reduce((sum, count) => sum + count, 0);
 
@@ -136,6 +144,13 @@ const results = await sharding.runOnAllWithDetails(async (client, shardId) => {
 
 ### Health Monitoring
 
+`connect()` initializes all clients and starts background warmup for clients that implement
+`$connect()`, followed by an initial `SELECT 1` when `$queryRaw` is available. Warmup does not delay
+client availability, preserving existing startup behavior. Periodic checks have a deadline, cannot
+overlap, and update the existing `ShardHealth` shape. Deterministic routing still returns the
+record's owner when it is marked unhealthy; cross-shard work schedules healthy, lower-latency
+shards first.
+
 ```typescript
 // Get health of all shards
 const health = sharding.getHealth();
@@ -169,6 +184,7 @@ Add to your `package.json`:
 {
   "scripts": {
     "db:update": "prisma-sharding-update",
+    "migrate:shards": "prisma-sharding-migrate",
     "db:studio": "prisma-sharding-studio",
     "test:shards": "prisma-sharding-test"
   }
@@ -184,48 +200,155 @@ SHARD_2_URL=postgresql://user:pass@host:5432/db2
 SHARD_3_URL=postgresql://user:pass@host:5432/db3
 SHARD_ROUTING_STRATEGY=modulo  # or consistent-hash
 SHARD_STUDIO_BASE_PORT=51212   # optional, for studio
+SHARD_STUDIO_REUSE_EXISTING=true # optional, defaults to true
+SHARD_STUDIO_STRICT_PORT_CHECK=false # optional, defaults to false
+SHARD_STUDIO_START_TIMEOUT_MS=15000 # optional, defaults to 15000
+SHARD_STUDIO_VERBOSE=false # optional, defaults to false
+SHARD_CLI_VERBOSE=false # optional, verbose update/migrate output
+PRISMA_SHARDING_VERBOSE=false # optional, library lifecycle logs
 ```
 
 ### Commands
 
 #### `prisma-sharding-update` (Recommended)
 
-The "All-in-One" command. It generates Prisma Client types and migrates all shards.
-**Use this whenever you change `schema.prisma`.**
+The "All-in-One" development command generates Prisma Client types and synchronizes local/dev
+shards. Use this whenever you change `schema.prisma` during development. It is not a production
+migration workflow.
 
 1. Runs `prisma generate` (Updates TypeScript types)
-2. Runs `prisma db push` on all shards (Updates Databases)
+2. Runs `prisma db push` on all shards (Synchronizes development databases)
 
 ```bash
 yarn db:update
 
 ```
 
-**With Flags:**
-You can pass flags like `--force-reset` if you need to wipe data due to schema conflicts.
+Default output stays compact:
+
+```text
+🔄 Prisma Sharding Update
+
+✅ client  Generated
+✅ shard_1  Synced
+✅ shard_2  Synced
+✅ shard_3  Synced
+```
+
+Interactive terminals show a single inline loader while Prisma Client generation and each shard
+sync are running. The loader is replaced by the completed row and is disabled for piped or CI logs.
+
+Set `SHARD_CLI_VERBOSE=true` or `SHARD_UPDATE_VERBOSE=true` to include Prisma command
+output, masked database URLs, and detailed diagnostics.
+
+Existing flags remain unchanged and are forwarded as provided. For example:
 
 ```bash
 yarn db:update --force-reset
 
 ```
 
+The command does not inject `--accept-data-loss` automatically.
+
 #### `prisma-sharding-migrate`
 
-Only pushes schema to all shards (skips type generation). Useful for production deployment pipelines.
+This is the production/staging path. For each shard it checks migration status and runs
+`prisma migrate deploy`, applying committed migration artifacts without generating the client,
+resetting the database, or using `db push`.
 
 ```bash
 yarn migrate:shards
 
 ```
 
+Any failed shard makes the command exit non-zero and remains visible in the compact results.
+This command uses the same compact shard rows as `db:update`. Set
+`SHARD_CLI_VERBOSE=true` or `SHARD_MIGRATE_VERBOSE=true` for Prisma command output.
+
+Do not use `prisma db push` as a production migration strategy. Commit and review the Prisma
+migration directory, then run `prisma-sharding-migrate` during deployment. Verbose mode includes
+sanitized status/deploy commands, masked database URLs, exit codes, shard IDs, and next-step hints.
+
 #### `prisma-sharding-studio`
 
 Start Prisma Studio for all shards on sequential ports.
 
 ```bash
 yarn db:studio
-# Opens shard_1 on :51212, shard_2 on :51213, etc.
+```
 
+By default, ports are assigned from `SHARD_STUDIO_BASE_PORT`:
+
+```text
+shard_1 -> http://localhost:51212
+shard_2 -> http://localhost:51213
+shard_3 -> http://localhost:51214
+```
+
+Set `SHARD_STUDIO_BASE_PORT` to move the whole range:
+
+```bash
+SHARD_STUDIO_BASE_PORT=52000 yarn db:studio
+# shard_1 -> :52000, shard_2 -> :52001, etc.
+```
+
+Studio startup is safe to run from multiple local APIs. Before starting a shard Studio,
+the CLI checks whether the target port is already active. If it finds an existing Prisma
+Studio on that port, it reuses it instead of spawning another process:
+
+```text
+🗄️ Prisma Sharding Studio
+
+♻️ shard_1  http://localhost:51212
+♻️ shard_2  http://localhost:51213
+♻️ shard_3  http://localhost:51214
+```
+
+If a port is occupied by another process that does not look like Prisma Studio, the shard
+is marked with a warning and the CLI continues with the remaining shards. It will not kill,
+restart, or claim ownership of processes it did not start.
+
+Default output is intentionally compact:
+
+```text
+🗄️ Prisma Sharding Studio
+
+✅ shard_1  http://localhost:51212
+✅ shard_2  http://localhost:51213
+✅ shard_3  http://localhost:51214
+```
+
+Run with `SHARD_STUDIO_VERBOSE=true` to print port checks, masked database URLs, Prisma
+Studio child-process output, startup timings, and detailed failure diagnostics.
+
+Useful Studio environment variables:
+
+- `SHARD_STUDIO_BASE_PORT`: first port in the shard Studio range. Defaults to `51212`.
+- `SHARD_STUDIO_REUSE_EXISTING`: reuse already-running Prisma Studio ports. Defaults to `true`.
+- `SHARD_STUDIO_STRICT_PORT_CHECK`: when `true`, any failed shard makes the command exit
+  non-zero after stopping Studio processes started by that run. Defaults to `false`.
+- `SHARD_STUDIO_START_TIMEOUT_MS`: maximum time to wait for a newly spawned Studio to become
+  reachable. Defaults to `15000`.
+- `SHARD_STUDIO_STABILITY_MS`: short window a newly-ready Studio process must survive before
+  it is reported as started. Defaults to `500`.
+- `SHARD_STUDIO_SHUTDOWN_TIMEOUT_MS`: time to wait for owned Studio processes to close during
+  shutdown before sending a force-stop signal. Defaults to `5000`.
+- `SHARD_STUDIO_VERBOSE`: print detailed Studio startup diagnostics. Defaults to `false`.
+- `SHARD_STUDIO_DEBUG`: alias for `SHARD_STUDIO_VERBOSE`.
+
+When multiple APIs use the same shard configuration locally, the first API starts the Studio
+processes and later APIs reuse the existing Studio ports. Reused-only commands stay quietly
+attached, preventing process supervisors from printing a normal child-exit message. Pressing
+Ctrl+C only stops Studio processes started by the current CLI run; reused processes are left running.
+
+If you run Studio beside `nodemon`, prefer an explicit watch scope for the API process. Prisma
+Studio does not need to write to your app source, but broad nodemon defaults can restart on
+generated TypeScript or JSON files produced by other dev tooling:
+
+```bash
+nodemon --watch src --ext ts,json \
+  --ignore 'src/types/*.generated.ts' \
+  --exec tsx --env-file=.env --no-warnings src/server.ts
 ```
 
 #### `prisma-sharding-test`
@@ -289,7 +412,7 @@ Verified 5/5 users on correct shards
 | `shards`                  | `ShardConfig[]`                 | Required   | Array of shard configurations     |
 | `strategy`                | `'modulo' \| 'consistent-hash'` | `'modulo'` | Routing algorithm                 |
 | `createClient`            | `(url, shardId) => TClient`     | Required   | Factory to create Prisma clients  |
-| `healthCheckIntervalMs`   | `number`                        | `30000`    | Health check frequency            |
+| `healthCheckIntervalMs`   | `number`                        | `30000`    | Positive health check frequency   |
 | `circuitBreakerThreshold` | `number`                        | `3`        | Failures before marking unhealthy |
 
 ### Shard Config
@@ -298,7 +421,7 @@ Verified 5/5 users on correct shards
 interface ShardConfig {
   id: string; // Unique identifier (e.g., 'shard_1')
   url: string; // PostgreSQL connection string
-  weight?: number; // Optional weight for distribution
+  weight?: number; // Positive random-assignment weight only
   isReadReplica?: boolean;
 }
 ```
@@ -315,12 +438,64 @@ strategy: 'modulo';
 
 ### Consistent Hash
 
-Minimizes data movement when adding/removing shards.
+Uses a precomputed virtual-node ring and binary search. Custom and non-sequential shard IDs are
+supported. Adding or removing shards still changes ownership for part of the keyspace, so plan data
+movement before changing a production shard list.
 
 ```typescript
 strategy: 'consistent-hash';
 ```
 
+## Architecture and Scaling
+
+The public `PrismaSharding` layer validates and delegates without changing its established surface.
+Internally, the router owns key placement, the manager owns clients and health state, and one
+cross-shard executor owns concurrency, deadlines, health-aware scheduling, stable result ordering,
+and failure isolation. CLI commands share one shard parser and one sanitized child-process runner.
+
+| Layer | Responsibility |
+| --- | --- |
+| Public API | Validate, delegate, and preserve existing result shapes |
+| Router | Stable deterministic placement and weighted random assignment |
+| Shard manager | Client lifecycle, initial verification, health, and shutdown |
+| Cross-shard executor | Shared concurrency, deadlines, ordering, and failure isolation |
+| CLI | Safe migration/update/test/Studio orchestration with compact output |
+
+Low-level execution behavior is intentionally internal: fan-out concurrency and deadlines are
+central defaults, the hash function is unchanged, health checks use typed Prisma-like capability
+guards, successful `runOnAll()` results retain configured shard order, and errors stay isolated in
+the existing detailed result shape.
+
+Normal request flow should be:
+
+```text
+routing key or directory lookup -> one shard -> one Prisma operation
+```
+
+`findFirst()` and `runOnAll()` use bounded concurrency and per-shard deadlines, but they still
+multiply database work and tail-latency exposure. Reserve them for recovery, administration, and
+analytics. Pending Prisma queries may not be cancellable after an early `findFirst()` result, so
+the caller can resolve before all already-started database work has physically stopped.
+
+The executor deadline limits how long the package waits; it does **not** cancel the underlying
+Prisma or PostgreSQL query. Configure a database-level deadline as well, such as PostgreSQL
+`statement_timeout` or the equivalent adapter/provider query timeout, so timed-out work cannot
+continue consuming database resources indefinitely.
+
+### Connection Pool Budgeting
+
+Each shard client owns or uses a connection pool. Budget the fleet-wide maximum as:
+
+```text
+application instances × shards per instance × connections per shard pool
+```
+
+For example, 20 application instances × 8 shards × 10 connections can attempt 1,600 database
+connections. Set the adapter's pool limit and connection timeout deliberately per application
+instance and per shard. At larger fleet sizes, use PgBouncer or provider-managed pooling and verify
+that the database's total connection budget includes migrations, administration, and failover
+headroom. The sharding package does not create hidden extra Prisma clients.
+
 ## Error Handling
 
 ```typescript
@@ -337,6 +512,9 @@ try {
 
 ## Custom Logger
 
+The default logger prints warnings and errors only. Set `PRISMA_SHARDING_VERBOSE=true` to include
+initialization, shard connection, and shutdown lifecycle messages.
+
 ```typescript
 const sharding = new PrismaSharding({
   // ...config,