edinburgh 0.4.4 → 0.4.6

package/README.md

@@ -388,13 +388,9 @@ await Product.byCategory.batchProcess({is: "old"}, (product) => {
 // Commits every ~1 second or 4096 rows (configurable via limitSeconds, limitRows)
 ```
 
-### Schema Evolution
+### Lazy Schema Migrations
 
-Edinburgh handles schema changes automatically:
-
-- **Adding/removing fields**: Old rows are lazily migrated on read. New fields use their default value.
-- **Changing field types**: Requires a `static migrate()` function.
-- **Adding/removing indexes**: Requires running `npx migrate-edinburgh`.
+When you change a model's schema, Edinburgh will lazily try to migrate old records on access. This allows you to deploy code changes without downtime or a separate migration step. Optionally, you may provide a `static migrate(record: Record<string, any>)` function on the model to transform old records during lazy migration. If there is a migration error (like a new field without a default value, or an incompatible type change), a run-time error is thrown when loading the affected model instance.
 
 ```typescript
 @E.registerModel
@@ -402,15 +398,40 @@ class User extends E.Model<User> {
   static pk = E.primary(User, "id");
   id = E.field(E.identifier);
   name = E.field(E.string);
-  role = E.field(E.string, {default: "user"});  // new field
+  role = E.field(E.string);  // newly added field
 
   static migrate(record: Record<string, any>) {
-    record.role ??= "user";  // provide value for old rows
+    record.role ??= record.name.indexOf("admin") >= 0 ? "admin" : "user"; // set role based on name for old records
   }
 }
 ```
 
-Run `npx migrate-edinburgh` (or call `E.runMigration()`) after adding/removing indexes, changing index field types, or when a `migrate()` function affects indexed fields.
+Edinburgh will lazily (re)run the `migrate` function on an instance whenever its implementation (the literal function code) has changed. For robustness, make sure that your `migrate` function...
+- Is idempotent (meaning it can be safely run multiple times on the same row without changing the result after the first run), and
+- Should perform *all* transformation steps starting from the oldest version that could possibly still be in the database. (See the next section.)
+
+While lazy migration is convenient and often sufficient, in some cases you need migrations to happen immediately...
+
+### Forced Schema Migrations
+
+The `migrate-edinburgh` CLI tool will scan the entire database, pro-actively performing the following migrations:
+- **Populate secondary indexes**: If you added or changed secondary indexes, it will build them. Until you do, the indexes will be empty (or only contain instances that have been saved since the index was created).
+- **Migrate primary indexes**: In case you changed the primary key fields or field types (not recommended!) of a model, it will build the new primary index, as well as all secondary indexes (to point at the new primary keys). Until you do, all of your old data will appear to be missing! Note that this may fail on duplicates.
+- **Remove orphaned indexes**: If you removed or changed an index, the stale data will be deleted from the database.
+- **Rewrite primary data**: These are the types of migrations that would normally be done lazily on instance access. As there's usually not much benefit to doing this forcibly, and it can be very time-consuming (and generates a lot of I/O), this is *not* done by default. It may however be useful if you want to clean up the contents of your `migrate()` function, if you have control over all application deployments. Use the `--rewrite-data` flag to enable this.
+
+```bash
+npx migrate-edinburgh ./src/models.ts
+```
+
+Run `npx migrate-edinburgh` without arguments to see all options. You can also call `runMigration()` programmatically:
+
+```typescript
+import { runMigration } from "edinburgh";
+
+const result = await runMigration({ tables: ["User"] });
+console.log(result.secondaries);  // { User: 1500 }
+```
 
 ### preCommit Hook
 
@@ -540,7 +561,7 @@ await E.transact(() => {
 });
 ```
 
-### setMaxRetryCount · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L208)
+### setMaxRetryCount · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L209)
 
 Set the maximum number of retries for a transaction in case of conflicts.
 The default value is 6. Setting it to 0 will disable retries and cause transactions to fail immediately on conflict.
@@ -551,7 +572,7 @@ The default value is 6. Setting it to 0 will disable retries and cause transacti
 
 - `count: number` - The maximum number of retries for a transaction.
 
-### setOnSaveCallback · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L222)
+### setOnSaveCallback · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L223)
 
 Set a callback function to be called after a model is saved and committed.
 
@@ -564,11 +585,11 @@ Set a callback function to be called after a model is saved and committed.
 - A sequential number. Higher numbers have been committed after lower numbers.
 - A map of model instances to their changes. The change can be "created", "deleted", or an object containing the old values.
 
-### deleteEverything · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L227)
+### deleteEverything · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L228)
 
 **Signature:** `() => Promise<void>`
 
-### Model · [abstract class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L220)
+### Model · [abstract class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L221)
 
 [object Object],[object Object],[object Object],[object Object],[object Object]
 
@@ -591,7 +612,7 @@ class User extends E.Model<User> {
 }
 ```
 
-#### Model.tableName · [static property](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L227)
+#### Model.tableName · [static property](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L225)
 
 The database table name (defaults to class name).
 
@@ -603,13 +624,13 @@ When true, registerModel replaces an existing model with the same tableName.
 
 **Type:** `boolean`
 
-#### Model.fields · [static property](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L232)
+#### Model.fields · [static property](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L233)
 
 Field configuration metadata.
 
 **Type:** `Record<string | number | symbol, FieldConfig<unknown>>`
 
-#### Model.migrate · [static method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### Model.migrate · [static method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Optional migration function called when deserializing rows written with an older schema version.
 Receives a plain record with all fields (primary key fields + value fields) and should mutate it
@@ -644,7 +665,19 @@ class User extends E.Model<User> {
 }
 ```
 
-#### Model.findAll · [static method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### Model.initFields · [static method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
+
+Transform the model's `E.field` properties into the appropriate JavaScript properties. Normally this is done
+automatically when using `transact()`, but in case you need to access `Model.fields` directly before the first
+transaction, you can call this method manually.
+
+**Signature:** `(reset?: boolean) => void`
+
+**Parameters:**
+
+- `reset?: boolean`
+
+#### Model.findAll · [static method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Find all instances of this model in the database, ordered by primary key.
 
@@ -657,7 +690,7 @@ Find all instances of this model in the database, ordered by primary key.
 
 **Returns:** An iterator.
 
-#### Model.replaceInto · [static method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### Model.replaceInto · [static method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Load an existing instance by primary key and update it, or create a new one.
 
@@ -674,7 +707,7 @@ new instance is created with `obj` as its initial properties.
 
 **Returns:** The loaded-and-updated or newly created instance.
 
-#### model.preCommit · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.preCommit · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Optional hook called on each modified instance right before the transaction commits.
 Runs before data is written to disk, so changes made here are included in the commit.
@@ -687,9 +720,6 @@ Common use cases:
 
 **Signature:** `() => void`
 
-**Parameters:**
-
-
 **Examples:**
 
 ```typescript
@@ -706,25 +736,19 @@ class Post extends E.Model<Post> {
 }
 ```
 
-#### model.getPrimaryKey · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.getPrimaryKey · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `() => Uint8Array<ArrayBufferLike>`
 
-**Parameters:**
-
-
 **Returns:** The primary key for this instance.
 
-#### model.getPrimaryKeyHash · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.getPrimaryKeyHash · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `() => number`
 
-**Parameters:**
-
-
 **Returns:** A 53-bit positive integer non-cryptographic hash of the primary key, or undefined if not yet saved.
 
-#### model.isLazyField · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.isLazyField · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `(field: keyof this) => boolean`
 
@@ -732,15 +756,12 @@ class Post extends E.Model<Post> {
 
 - `field: keyof this`
 
-#### model.preventPersist · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.preventPersist · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Prevent this instance from being persisted to the database.
 
 **Signature:** `() => this`
 
-**Parameters:**
-
-
 **Returns:** This model instance for chaining.
 
 **Examples:**
@@ -751,7 +772,7 @@ user.name = "New Name";
 user.preventPersist(); // Changes won't be saved
 ```
 
-#### model.delete · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.delete · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Delete this model instance from the database.
 
@@ -759,9 +780,6 @@ Removes the instance and all its index entries from the database and prevents fu
 
 **Signature:** `() => void`
 
-**Parameters:**
-
-
 **Examples:**
 
 ```typescript
@@ -769,7 +787,7 @@ const user = User.load("user123");
 user.delete(); // Removes from database
 ```
 
-#### model.validate · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.validate · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Validate all fields in this model instance.
 
@@ -791,15 +809,12 @@ if (errors.length > 0) {
 }
 ```
 
-#### model.isValid · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.isValid · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Check if this model instance is valid.
 
 **Signature:** `() => boolean`
 
-**Parameters:**
-
-
 **Returns:** true if all validations pass.
 
 **Examples:**
@@ -809,27 +824,18 @@ const user = new User({name: "John"});
 if (!user.isValid()) shoutAtTheUser();
 ```
 
-#### model.getState · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.getState · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `() => "created" | "deleted" | "loaded" | "lazy"`
 
-**Parameters:**
-
-
-#### model.toString · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.toString · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `() => string`
 
-**Parameters:**
-
-
-#### model.[Symbol.for('nodejs.util.inspect.custom')] · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### model.[Symbol.for('nodejs.util.inspect.custom')] · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `() => string`
 
-**Parameters:**
-
-
 ### registerModel · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L113)
 
 Register a model class with the Edinburgh ORM system.
@@ -887,13 +893,13 @@ class User extends E.Model<User> {
 }
 ```
 
-### string · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### string · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Type wrapper instance for the string type.
 
 **Value:** `TypeWrapper<string>`
 
-### orderedString · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### orderedString · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Type wrapper instance for the ordered string type, which is just like a string
 except that it sorts lexicographically in the database (instead of by incrementing
@@ -903,37 +909,37 @@ may not contain null characters.
 
 **Value:** `TypeWrapper<string>`
 
-### number · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### number · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Type wrapper instance for the number type.
 
 **Value:** `TypeWrapper<number>`
 
-### dateTime · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### dateTime · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Type wrapper instance for the date/time type. Stored without timezone info, rounded to whole seconds.
 
 **Value:** `TypeWrapper<Date>`
 
-### boolean · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### boolean · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Type wrapper instance for the boolean type.
 
 **Value:** `TypeWrapper<boolean>`
 
-### identifier · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### identifier · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Type wrapper instance for the identifier type.
 
 **Value:** `TypeWrapper<string>`
 
-### undef · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### undef · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Type wrapper instance for the 'undefined' type.
 
 **Value:** `TypeWrapper<undefined>`
 
-### opt · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### opt · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create an optional type wrapper (allows undefined).
 
@@ -956,7 +962,7 @@ const optionalString = E.opt(E.string);
 const optionalNumber = E.opt(E.number);
 ```
 
-### or · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### or · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a union type wrapper from multiple type choices.
 
@@ -979,7 +985,7 @@ const stringOrNumber = E.or(E.string, E.number);
 const status = E.or("active", "inactive", "pending");
 ```
 
-### array · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### array · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create an array type wrapper with optional length constraints.
 
@@ -1003,7 +1009,7 @@ const stringArray = E.array(E.string);
 const boundedArray = E.array(E.number, {min: 1, max: 10});
 ```
 
-### set · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### set · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a Set type wrapper with optional length constraints.
 
@@ -1027,7 +1033,7 @@ const stringSet = E.set(E.string);
 const boundedSet = E.set(E.number, {min: 1, max: 10});
 ```
 
-### record · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### record · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a Record type wrapper for key-value objects with string or number keys.
 
@@ -1049,7 +1055,7 @@ Create a Record type wrapper for key-value objects with string or number keys.
 const scores = E.record(E.number);  // Record<string | number, number>
 ```
 
-### literal · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### literal · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a literal type wrapper for a constant value.
 
@@ -1072,7 +1078,7 @@ const statusType = E.literal("active");
 const countType = E.literal(42);
 ```
 
-### link · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### link · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a link type wrapper for model relationships.
 
@@ -1100,7 +1106,7 @@ class Post extends E.Model<Post> {
 }
 ```
 
-### index · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### index · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a secondary index on model fields, or a computed secondary index using a function.
 
@@ -1133,7 +1139,7 @@ class User extends E.Model<User> {
 }
 ```
 
-### primary · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### primary · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a primary index on model fields.
 
@@ -1160,7 +1166,7 @@ class User extends E.Model<User> {
 }
 ```
 
-### unique · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### unique · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Create a unique index on model fields, or a computed unique index using a function.
 
@@ -1193,7 +1199,7 @@ class User extends E.Model<User> {
 }
 ```
 
-### dump · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### dump · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Dump database contents for debugging.
 
@@ -1202,7 +1208,7 @@ This is primarily useful for development and debugging purposes.
 
 **Signature:** `() => void`
 
-### BaseIndex · [abstract class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L125)
+### BaseIndex · [abstract class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L123)
 
 Base class for database indexes for efficient lookups on model fields.
 
@@ -1219,7 +1225,7 @@ Indexes enable fast queries on specific field combinations and enforce uniquenes
 - `MyModel`: - The model class this index belongs to.
 - `_fieldNames`: - Array of field names that make up this index.
 
-#### baseIndex.find · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### baseIndex.find · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `(opts?: FindOptions<ARGS>) => IndexRangeIterator<M>`
 
@@ -1227,7 +1233,7 @@ Indexes enable fast queries on specific field combinations and enforce uniquenes
 
 - `opts: FindOptions<ARGS>` (optional)
 
-#### baseIndex.batchProcess · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### baseIndex.batchProcess · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 [object Object],[object Object],[object Object]
 
@@ -1238,14 +1244,11 @@ Indexes enable fast queries on specific field combinations and enforce uniquenes
 - `opts: FindOptions<ARGS> & { limitSeconds?: number; limitRows?: number }` (optional) - - Query options (same as `find()`), plus:
 - `callback: (row: InstanceType<M>) => void | Promise<void>` - - Called for each matching row within a transaction
 
-#### baseIndex.toString · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### baseIndex.toString · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 **Signature:** `() => string`
 
-**Parameters:**
-
-
-### UniqueIndex · [class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### UniqueIndex · [class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Unique index that stores references to the primary key.
 
@@ -1255,7 +1258,7 @@ Unique index that stores references to the primary key.
 - `F extends readonly (keyof InstanceType<M> & string)[]` - The field names that make up this index.
 - `ARGS extends readonly any[] = IndexArgTypes<M, F>`
 
-#### uniqueIndex.get · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### uniqueIndex.get · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Get a model instance by unique index key values.
 
@@ -1273,7 +1276,7 @@ Get a model instance by unique index key values.
 const userByEmail = User.byEmail.get("john@example.com");
 ```
 
-### PrimaryIndex · [class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### PrimaryIndex · [class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Primary index that stores the actual model data.
 
@@ -1282,7 +1285,7 @@ Primary index that stores the actual model data.
 - `M extends typeof Model` - The model class this index belongs to.
 - `F extends readonly (keyof InstanceType<M> & string)[]` - The field names that make up this index.
 
-#### primaryIndex.get · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### primaryIndex.get · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Get a model instance by primary key values.
 
@@ -1300,7 +1303,7 @@ Get a model instance by primary key values.
 const user = User.pk.get("john_doe");
 ```
 
-#### primaryIndex.getLazy · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+#### primaryIndex.getLazy · [method](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Does the same as as `get()`, but will delay loading the instance from disk until the first
 property access. In case it turns out the instance doesn't exist, an error will be thrown
@@ -1314,7 +1317,7 @@ at that time.
 
 **Returns:** The (lazily loaded) model instance.
 
-### SecondaryIndex · [class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L253)
+### SecondaryIndex · [class](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L255)
 
 Secondary index for non-unique lookups.
 
@@ -1342,7 +1345,7 @@ Secondary index for non-unique lookups.
 
 **Type:** `Map<number, Model<unknown>>`
 
-### DatabaseError · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L157)
+### DatabaseError · [constant](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L158)
 
 The DatabaseError class is used to represent errors that occur during database operations.
 It extends the built-in Error class and has a machine readable error code string property.
@@ -1352,10 +1355,10 @@ Invalid function arguments will throw TypeError.
 
 **Value:** `DatabaseErrorConstructor`
 
-### runMigration · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L118)
+### runMigration · [function](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L124)
 
-Run database migration: upgrade all rows to the latest schema version,
-convert old primary indices, and clean up orphaned secondary indices.
+Run database migration: populate secondary indexes for old-version rows,
+convert old primary indices, rewrite row data, and clean up orphaned indices.
 
 **Signature:** `(options?: MigrationOptions) => Promise<MigrationResult>`
 
@@ -1371,99 +1374,65 @@ Limit migration to specific table names.
 
 **Type:** `string[]`
 
-#### migrationOptions.convertOldPrimaries · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L23)
+#### migrationOptions.populateSecondaries · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L23)
+
+Populate secondary indexes for rows at old schema versions (default: true).
+
+**Type:** `boolean`
+
+#### migrationOptions.migratePrimaries · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L32)
 
-Whether to convert old primary indices for known tables (default: true).
+Convert old primary indices when primary key fields changed (default: true).
 
 **Type:** `boolean`
 
-#### migrationOptions.deleteOrphanedIndexes · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L31)
+#### migrationOptions.rewriteData · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L40)
 
-Whether to delete orphaned secondary/unique indices (default: true).
+Rewrite all row data to the latest schema version (default: false).
 
 **Type:** `boolean`
 
-#### migrationOptions.upgradeVersions · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L39)
+#### migrationOptions.removeOrphans · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L46)
 
-Whether to upgrade rows to the latest version (default: true).
+Delete orphaned secondary/unique index entries (default: true).
 
 **Type:** `boolean`
 
-#### migrationOptions.onProgress · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L44)
+#### migrationOptions.onProgress · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L48)
 
 Progress callback.
 
 **Type:** `(info: ProgressInfo) => void`
 
-### MigrationResult · [interface](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L49)
+### MigrationResult · [interface](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L51)
 
-#### migrationResult.secondaries · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L51)
+#### migrationResult.secondaries · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L54)
 
-Per-table stats for row upgrades.
+Per-table counts of secondary index entries populated.
 
 **Type:** `Record<string, number>`
 
-#### migrationResult.primaries · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L52)
+#### migrationResult.primaries · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L59)
 
-Per-table stats for old primary conversions.
+Per-table counts of old primary rows migrated.
 
 **Type:** `Record<string, number>`
 
-#### migrationResult.conversionFailures · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L58)
+#### migrationResult.conversionFailures · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L59)
 
 Per-table conversion failure counts by reason.
 
 **Type:** `Record<string, Record<string, number>>`
 
-#### migrationResult.orphaned · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L59)
-
-Number of orphaned index entries deleted.
-
-**Type:** `number`
-
-## Schema Migrations
-
-Edinburgh automatically tracks the schema version of each model. When you change fields, field types, indexes, or the `migrate()` function, Edinburgh detects a new schema version.
-
-### What happens automatically (lazy migration)
-
-Changes to regular (non-index) field values are migrated lazily. When a row with an old schema version is loaded from disk, it is deserialized using the old field types and transformed by the optional static `migrate()` function. This is transparent and requires no downtime.
-
-```typescript
-@E.registerModel
-class User extends E.Model<User> {
-  static pk = E.primary(User, "id");
-  id = E.field(E.identifier);
-  name = E.field(E.string);
-  role = E.field(E.string);  // newly added field
-
-  static migrate(record: Record<string, any>) {
-    record.role ??= "user";  // provide a default for old rows
-  }
-}
-```
-
-### What requires `migrate-edinburgh`
+#### migrationResult.rewritten · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L64)
 
-The `migrate-edinburgh` CLI tool (or the `runMigration()` API) must be run when:
+Per-table counts of rows rewritten to latest version.
 
-- **Adding or removing** secondary or unique indexes
-- **Changing the fields or types** of an existing index
-- A **`migrate()` function changes values** that are used in index fields
-
-The tool populates new indexes, removes orphaned ones, and updates index entries whose values were changed by `migrate()`. It does *not* rewrite primary data rows - lazy migration handles that on read.
-
-```bash
-npx migrate-edinburgh --import ./src/models.ts
-```
+**Type:** `Record<string, number>`
 
-Run `npx migrate-edinburgh` to see all of its options.
+#### migrationResult.orphans · [member](https://github.com/vanviegen/edinburgh/blob/main/src/edinburgh.ts#L68)
 
-You can also call `runMigration()` programmatically:
+Number of orphaned index entries deleted.
 
-```typescript
-import { runMigration } from "edinburgh";
+**Type:** `number`
 
-const result = await runMigration({ tables: ["User"] });
-console.log(result.upgraded);  // { User: 1500 }
-```