@lenne.tech/nest-server 11.30.0 → 11.31.1

package/migration-guides/11.29.x-to-11.30.x.md

@@ -7,7 +7,7 @@
 | **Breaking Changes** | None in nest-server's own API. Two transitive changes may affect projects: `@nestjs/websockets` removed from dependencies; `@nestjs/swagger` 11.4.6 blocks deep imports |
 | **New Features** | None (dependency maintenance release) |
 | **Bugfixes** | Dependency updates incl. security-relevant `ws` 8.21.1 override; 6 obsolete pnpm overrides removed |
-| **Migration Effort** | Very Low (~5 minutes) — most projects need no changes |
+| **Migration Effort** | Very Low — most projects need no changes |
 
 ---
 

package/migration-guides/11.30.x-to-11.31.x.md

@@ -0,0 +1,38 @@
+# Migration Guide: 11.30.x → 11.31.x
+
+## Overview
+
+| Category | Details |
+|----------|---------|
+| **Breaking Changes** | None |
+| **New Features** | `pnpm run check` steps report grouped by project (monorepo / api / app); orchestrator parity across all lt starters |
+| **Bugfixes** | Root-only chain steps now run correctly when workspace members exist |
+| **Migration Effort** | None — no code changes required |
+
+---
+
+## Quick Migration
+
+```bash
+# Update package
+npm install @lenne.tech/nest-server@11.31.x
+
+# Verify build
+npm run build
+
+# Run tests
+npm test
+```
+
+No code changes required in any project.
+
+---
+
+## What's New in 11.31.x
+
+### Grouped check report
+
+`pnpm run check` now groups its steps report by project (monorepo root, api,
+app), so in a fullstack workspace you immediately see which project a failing
+step belongs to. The check orchestrator also behaves identically across all
+lt starters (same chain semantics, same root-only step handling).

package/migration-guides/11.31.0-to-11.31.1.md

@@ -0,0 +1,89 @@
+# Migration Guide: 11.31.0 → 11.31.1
+
+> Note: written on the feature branch (`feature/migrate-safe-ts-js-transition`).
+> If the release lands under a different version number, rename this file accordingly.
+
+## Overview
+
+| Category | Details |
+|----------|---------|
+| **Breaking Changes** | None |
+| **New Features** | Extension-agnostic migration identity (safe ts-node → compiled-JS transition); strict integrity mode (`--strict` / `NSC__MIGRATE__STRICT` / `MigrationRunnerOptions.strict`); `status()` reports `missing` files; `migrate list` annotates missing files |
+| **Bugfixes** | Co-present `foo.ts` + `foo.js` no longer execute twice in one `up()` run (deduplicated by identity, `.js` wins) |
+| **Migration Effort** | None — no code changes required. Optional: enable strict mode in production images |
+
+---
+
+## Quick Migration
+
+```bash
+# Update package
+npm install @lenne.tech/nest-server@11.31.1
+
+# Verify build
+npm run build
+
+# Run tests
+npm test
+```
+
+No code changes required in any project.
+
+---
+
+## What's New in 11.31.1
+
+### Extension-agnostic migration identity (safe ts-node → compiled-JS transition)
+
+`migrate up`, `migrate down`, and `migrate list` now compare migrations by their
+timestamped file stem WITHOUT the `.ts`/`.js` extension. A migration recorded as
+`1699-foo.ts` (run via ts-node) matches the compiled `1699-foo.js` in a production
+image — already-applied migrations are no longer treated as "pending" and re-run
+after switching the image to compiled JavaScript. No state rewrite is needed;
+existing databases keep working as-is.
+
+If both `foo.ts` and `foo.js` are present in the migrations directory (e.g. an
+overlapping `outDir`), they now count as ONE migration: only the `.js` file is
+loaded and executed, with a warning. Previously both files would have executed.
+
+Note: the legacy `synchronizedUp()` helper drives the external `migrate` package's
+own state handling and keeps raw-filename identity — the new semantics apply to the
+`MigrationRunner`/built-in CLI path only.
+
+### Missing migration files tolerated by default (up/list)
+
+Old, already-applied migration files can be deleted (they live in git and can be
+restored). `migrate up` no longer needs their files to be present:
+
+- Default: `up` warns (`[migrate] N recorded migration file(s) missing — tolerated`) and continues, so the server still boots. Previously such entries were silently ignored — log-scraping setups will see a NEW warning.
+- `migrate list` marks affected entries with `(file missing)` and `status()` additionally returns them in a new `missing: string[]` field (additive, backward compatible).
+- `migrate down` ALWAYS fails hard when the rollback file is missing — unchanged from 11.31.0, with a clearer message (`… restore the file from git to roll back.`). Rollback is an explicit operator action, never a boot path.
+
+### Strict integrity mode (opt-in)
+
+Turn the tolerated drift into a hard error (`up` throws, `list` exits non-zero):
+
+| Activation path | Scope |
+|-----------------|-------|
+| `--strict` CLI flag | single invocation |
+| `NSC__MIGRATE__STRICT=1\|true\|yes` env var (case-insensitive) | CLI **and** programmatic `MigrationRunner` instances |
+| `new MigrationRunner({ strict: true, ... })` | programmatic |
+
+**Recommended for production images:** set `NSC__MIGRATE__STRICT=true`. In an
+immutable image, a recorded-but-missing migration file can only mean a broken build
+or a state-store mismatch — refusing to boot is the correct reaction there.
+
+---
+
+## Compatibility Notes
+
+- `status()` return type gained a `missing: string[]` field. Code that destructures
+  only `completed`/`pending` continues to work unchanged.
+- **Vendor mode:** `src/core/modules/migrate/migration-runner.ts` and
+  `src/core/modules/migrate/cli/migrate-cli.ts` are an atomic pair — sync BOTH
+  files together (the CLI passes `strict` into the runner's options interface and
+  imports `parseStrictEnv` from the runner).
+
+## Module Documentation
+
+- [Migrate module README](../src/core/modules/migrate/README.md) — see "Migration Identity & Strict Mode"

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lenne.tech/nest-server",
-  "version": "11.30.0",
+  "version": "11.31.1",
   "description": "Modern, fast, powerful Node.js web framework in TypeScript based on Nest with a GraphQL API and a connection to MongoDB (or other databases).",
   "keywords": [
     "node",

package/src/core/modules/migrate/README.md

@@ -432,6 +432,62 @@ The `migrate` command comes from `@lenne.tech/nest-server` - no external package
 
 See the [Migration Guide](./MIGRATION_FROM_NODEPIT.md) for detailed migration instructions from @nodepit.
 
+## Migration Identity & Strict Mode
+
+### Extension-agnostic identity (safe ts-node → compiled-JS transition)
+
+A migration's identity is its timestamped file stem **without** the `.ts`/`.js` extension
+(`migrationId('1699-foo.ts') === migrationId('1699-foo.js') === '1699-foo'`). `up`, `down`,
+and `list`/`status` all compare identities, not raw filenames.
+
+This makes the recommended production setup — running compiled `.js` migrations in the image
+while the state was recorded under `.ts` names via ts-node — safe: already-applied migrations
+are **not** re-run after the switch, and no state rewrite is needed. If both `foo.ts` and
+`foo.js` are present in the migrations directory (e.g. an overlapping `outDir`), they count
+as ONE migration: only the `.js` file is loaded and executed, and a warning is emitted.
+
+Note: identity normalization applies to the `MigrationRunner`/CLI path only. The legacy
+`synchronizedUp()` helper drives the external `migrate` package's own state handling and
+keeps raw-filename identity.
+
+### Missing migration files & strict mode
+
+Old, already-applied migration files can be deleted (they live in git and can be restored).
+By default the runner **tolerates** a recorded migration whose file is gone:
+
+- `migrate up` warns (`[migrate] N recorded migration file(s) missing — tolerated`) and continues, so the server still boots.
+- `migrate list` marks the entry with `(file missing)`.
+- `migrate down` **always fails hard** on a missing rollback file — independent of strict mode. Rollback is an explicit operator action, never a boot path; exiting 0 without rolling anything back would mislead scripts. Restore the file from git, then retry.
+
+Strict mode turns the tolerated drift into a hard error (`up` throws, `list` exits non-zero).
+Enable it via any of:
+
+| Activation path                              | Scope                                                                            |
+| -------------------------------------------- | -------------------------------------------------------------------------------- |
+| `--strict` CLI flag                          | single invocation                                                                |
+| `NSC__MIGRATE__STRICT=1\|true\|yes` env var  | CLI **and** programmatic runners (resolved in the `MigrationRunner` constructor) |
+| `new MigrationRunner({ strict: true, ... })` | programmatic                                                                     |
+
+**Recommended for production images:** set `NSC__MIGRATE__STRICT=true` in the container
+environment. In an immutable image, a recorded-but-missing migration file can only mean a
+broken build (empty/miscopied `migrations/` directory) or a state-store mismatch (wrong
+database) — both are conditions where refusing to boot is correct.
+
+### Programmatic usage (MigrationRunner)
+
+```typescript
+import { MigrationRunner, MongoStateStore } from '@lenne.tech/nest-server';
+
+const runner = new MigrationRunner({
+  migrationsDirectory: './migrations',
+  stateStore: new MongoStateStore(process.env.NSC__MONGOOSE__URI),
+  strict: true, // optional; defaults to NSC__MIGRATE__STRICT, else false
+});
+
+await runner.up();
+const { completed, missing, pending } = await runner.status();
+```
+
 ## Project Integration
 
 The migration utilities are designed to minimize boilerplate in your projects. Instead of copying multiple utility files, you can:

package/src/core/modules/migrate/cli/migrate-cli.ts

@@ -17,18 +17,21 @@
  *   --store, -s            Path to state store module
  *   --compiler, -c         Compiler to use (e.g., ts:./path/to/ts-compiler.js)
  *   --template-file, -t    Template file for creating migrations
+ *   --strict               Fail if a recorded migration's file is missing (default: off / tolerate).
+ *                          Applies to up/list; down always fails hard on a missing rollback file.
  */
 
 import * as fs from 'fs';
 import * as path from 'path';
 
-import { MigrationRunner } from '../migration-runner';
+import { MigrationRunner, parseStrictEnv } from '../migration-runner';
 import { MongoStateStore } from '../mongo-state-store';
 
 interface CliOptions {
   compiler?: string;
   migrationsDir: string;
   store?: string;
+  strict?: boolean;
   templateFile?: string;
 }
 
@@ -90,9 +93,11 @@ async function listMigrations(options: CliOptions) {
   const runner = new MigrationRunner({
     migrationsDirectory: path.resolve(process.cwd(), options.migrationsDir),
     stateStore,
+    strict: options.strict,
   });
 
   const status = await runner.status();
+  const missing = new Set(status.missing);
 
   console.log('\nMigration Status:');
   console.log('=================\n');
@@ -100,7 +105,7 @@ async function listMigrations(options: CliOptions) {
   if (status.completed.length > 0) {
     console.log('Completed:');
     status.completed.forEach((name) => {
-      console.log(`  ✓ ${name}`);
+      console.log(`  ✓ ${name}${missing.has(name) ? '  (file missing)' : ''}`);
     });
     console.log('');
   }
@@ -116,6 +121,14 @@ async function listMigrations(options: CliOptions) {
   if (status.completed.length === 0 && status.pending.length === 0) {
     console.log('No migrations found\n');
   }
+
+  // In strict mode, integrity drift is an error also on the inspection surface —
+  // exit non-zero (via main's error handler) instead of hiding it in the listing.
+  if (options.strict && status.missing.length > 0) {
+    throw new Error(
+      `Strict mode: ${status.missing.length} recorded migration file(s) missing: ${status.missing.join(', ')}`,
+    );
+  }
 }
 
 /**
@@ -198,6 +211,9 @@ function parseArgs(): { command: string; name?: string; options: CliOptions } {
   let name: string | undefined;
   const options: CliOptions = {
     migrationsDir: './migrations',
+    // Off by default: a recorded migration whose file was deleted is tolerated so the
+    // server still starts. Enable per-run via `--strict` or globally via NSC__MIGRATE__STRICT.
+    strict: parseStrictEnv(),
   };
 
   // Check if second arg is a name (not a flag)
@@ -217,6 +233,8 @@ function parseArgs(): { command: string; name?: string; options: CliOptions } {
       options.compiler = args[++i];
     } else if (arg === '--template-file' || arg === '-t') {
       options.templateFile = args[++i];
+    } else if (arg === '--strict') {
+      options.strict = true;
     }
   }
 
@@ -254,6 +272,7 @@ async function runDown(options: CliOptions) {
   const runner = new MigrationRunner({
     migrationsDirectory: path.resolve(process.cwd(), options.migrationsDir),
     stateStore,
+    strict: options.strict,
   });
 
   await runner.down();
@@ -269,6 +288,7 @@ async function runUp(options: CliOptions) {
   const runner = new MigrationRunner({
     migrationsDirectory: path.resolve(process.cwd(), options.migrationsDir),
     stateStore,
+    strict: options.strict,
   });
 
   await runner.up();
@@ -292,6 +312,8 @@ Options:
   --store, -s <path>                 Path to state store module
   --compiler, -c <compiler>          Compiler to use (e.g., ts:./path/to/ts-compiler.js)
   --template-file, -t <path>         Template file for creating migrations
+  --strict                           Fail if a recorded migration's file is missing (default: off / tolerate).
+                                     Applies to up/list; down always fails hard on a missing rollback file.
 
 Examples:
   migrate create add-user-email
@@ -302,6 +324,7 @@ Examples:
 
 Environment Variables:
   NODE_ENV                           Set environment (e.g., development, production)
+  NSC__MIGRATE__STRICT               1|true|yes → fail on a missing recorded migration file (default: tolerate)
 `);
 }
 
@@ -313,4 +336,5 @@ if (require.main === module) {
   });
 }
 
-export { main };
+// parseArgs is exported for unit testing only (same pattern as resolveCliPath in bin/migrate.js)
+export { main, parseArgs };

package/src/core/modules/migrate/migration-runner.ts

@@ -33,6 +33,59 @@ export interface MigrationFile {
  */
 export const DEFAULT_MIGRATION_FILE_PATTERN = /(?:(?<!\.d)\.ts|\.js)$/;
 
+/**
+ * Matches the trailing `.ts`/`.js` extension stripped by {@link migrationId}.
+ * Hoisted to module level so `migrationId` allocates no per-call RegExp wrapper.
+ */
+const MIGRATION_EXTENSION_PATTERN = /\.(ts|js)$/;
+
+/**
+ * Migration identity = the timestamped file stem WITHOUT its `.ts`/`.js` extension.
+ *
+ * A migration keeps its identity when a project switches its production image from
+ * ts-node (`1699-foo.ts`) to compiled JavaScript (`1699-foo.js`) — the recommended
+ * prod setup, since the image prunes ts-node. Comparing raw filenames would make
+ * every compiled `.js` migration look "pending" against a state recorded under `.ts`
+ * names and re-run already-applied migrations (data corruption on existing DBs).
+ * Normalising both sides makes the transition safe with no state rewrite.
+ *
+ * @param title Migration title, usually the filename (e.g. `1699000000000-foo.ts`)
+ * @returns The extension-agnostic identity (e.g. `1699000000000-foo`)
+ * @example
+ * migrationId('1699000000000-foo.ts'); // '1699000000000-foo'
+ * migrationId('1699000000000-foo.js'); // '1699000000000-foo'
+ * migrationId('1699000000000-foo');    // '1699000000000-foo' (already extensionless)
+ */
+export function migrationId(title: string): string {
+  return title.replace(MIGRATION_EXTENSION_PATTERN, '');
+}
+
+/**
+ * Parse the `NSC__MIGRATE__STRICT` environment variable into a boolean.
+ *
+ * Truthy values: `1`, `true`, `yes` (case-insensitive, surrounding whitespace ignored —
+ * a real risk with Docker/compose env files). Every other value means `false` (tolerate,
+ * the safe default); non-empty unrecognized values additionally emit a warning so a typo
+ * like `NSC__MIGRATE__STRICT=on` does not silently disable the control the operator
+ * intended to enable.
+ *
+ * Shared by the CLI (`migrate-cli.ts`) and the {@link MigrationRunner} constructor, so
+ * the env var behaves identically for CLI and programmatic runners.
+ */
+export function parseStrictEnv(value: string | undefined = process.env.NSC__MIGRATE__STRICT): boolean {
+  if (value === undefined) {
+    return false;
+  }
+  const normalized = value.trim().toLowerCase();
+  if (['1', 'true', 'yes'].includes(normalized)) {
+    return true;
+  }
+  if (normalized !== '' && !['0', 'false', 'no'].includes(normalized)) {
+    console.warn(`[migrate] Unrecognized NSC__MIGRATE__STRICT value "${value}" — treating as false (tolerate).`);
+  }
+  return false;
+}
+
 /**
  * Migration runner configuration
  */
@@ -43,6 +96,23 @@ export interface MigrationRunnerOptions {
   pattern?: RegExp;
   /** State store for tracking migrations */
   stateStore: MongoStateStore;
+  /**
+   * Fail hard when a migration recorded in the state has no file on disk.
+   *
+   * Default: resolved from the `NSC__MIGRATE__STRICT` environment variable (see
+   * {@link parseStrictEnv}), falling back to `false` (tolerate) — recorded migrations
+   * whose files were deleted are ignored: `up()` skips them (with a warning) and the
+   * server still starts. Migrations are tracked in git and can be restored, so old
+   * migration files can be pruned without blocking boot. Set `true` to enforce
+   * state/disk integrity (missing file → error in `up()` and `migrate list`).
+   *
+   * Applies to `up()` and `status()`/`migrate list` only — `down()` ALWAYS fails hard
+   * on a missing rollback file, because rollback is an explicit operator action and
+   * never a boot path (an exit 0 with no rollback performed would mislead scripts).
+   *
+   * @see `--strict` CLI flag and `NSC__MIGRATE__STRICT` env var in `cli/migrate-cli.ts`
+   */
+  strict?: boolean;
 }
 
 /**
@@ -72,12 +142,21 @@ export class MigrationRunner {
   private pattern: RegExp;
 
   constructor(options: MigrationRunnerOptions) {
-    this.options = options;
+    // Resolve the strict default from the environment so NSC__MIGRATE__STRICT works
+    // identically for programmatic runners and the CLI (which parses it itself).
+    this.options = { ...options, strict: options.strict ?? parseStrictEnv() };
     this.pattern = options.pattern || DEFAULT_MIGRATION_FILE_PATTERN;
   }
 
   /**
    * Load all migration files from the migrations directory
+   *
+   * Files are deduplicated by {@link migrationId}: when both `foo.ts` and `foo.js`
+   * are present (overlapping `outDir`, source + build output copied into one image),
+   * they are ONE migration — loading both would execute it twice in a single `up()`
+   * run (the exact double-execution the identity concept exists to prevent). The
+   * `.js` file wins deterministically (`.js` sorts before `.ts`), matching the
+   * compiled-production intent; the duplicate is skipped with a warning.
    */
   private async loadMigrationFiles(): Promise<MigrationFile[]> {
     const files = fs
@@ -86,8 +165,17 @@ export class MigrationRunner {
       .sort(); // Sort alphabetically (timestamp-based filenames will be in order)
 
     const migrations: MigrationFile[] = [];
+    const seenIds = new Set<string>();
 
     for (const file of files) {
+      const id = migrationId(file);
+      if (seenIds.has(id)) {
+        console.warn(
+          `[migrate] duplicate files for migration "${id}" — skipping ${file} (a same-named file was already loaded)`,
+        );
+        continue;
+      }
+
       const filePath = path.join(this.options.migrationsDirectory, file);
 
       const module = require(filePath);
@@ -101,6 +189,7 @@ export class MigrationRunner {
       const timestampMatch = file.match(/^(\d+)-/);
       const timestamp = timestampMatch ? parseInt(timestampMatch[1], 10) : Date.now();
 
+      seenIds.add(id);
       migrations.push({
         down: module.down,
         filePath,
@@ -121,9 +210,24 @@ export class MigrationRunner {
 
     const allMigrations = await this.loadMigrationFiles();
     const state = await this.options.stateStore.loadAsync();
-    const completedMigrations = (state.migrations || []).map((m) => m.title);
+    const recorded = state.migrations || [];
+    const presentIds = new Set(allMigrations.map((m) => migrationId(m.title)));
+
+    // Recorded migrations whose file is gone. Tolerated by default (git-tracked and
+    // restorable); in strict mode this is a hard error so integrity drift is caught.
+    const missing = recorded.filter((m) => !presentIds.has(migrationId(m.title)));
+    if (missing.length > 0) {
+      const list = missing.map((m) => m.title).join(', ');
+      if (this.options.strict) {
+        throw new Error(`Strict mode: ${missing.length} recorded migration file(s) missing: ${list}`);
+      }
+      console.warn(
+        `[migrate] ${missing.length} recorded migration file(s) missing — tolerated (strict=false): ${list}`,
+      );
+    }
 
-    const pendingMigrations = allMigrations.filter((m) => !completedMigrations.includes(m.title));
+    const completedIds = new Set(recorded.map((m) => migrationId(m.title)));
+    const pendingMigrations = allMigrations.filter((m) => !completedIds.has(migrationId(m.title)));
 
     if (pendingMigrations.length === 0) {
       console.log('No pending migrations');
@@ -181,10 +285,14 @@ export class MigrationRunner {
 
     const lastMigration = completedMigrations[completedMigrations.length - 1];
     const allMigrations = await this.loadMigrationFiles();
-    const migrationToRollback = allMigrations.find((m) => m.title === lastMigration.title);
+    const rollbackId = migrationId(lastMigration.title);
+    const migrationToRollback = allMigrations.find((m) => migrationId(m.title) === rollbackId);
 
     if (!migrationToRollback) {
-      throw new Error(`Migration file not found: ${lastMigration.title}`);
+      // Always a hard error, independent of `strict`: down() is an explicit operator
+      // action and never a boot path — the boot-tolerance rationale does not apply,
+      // and an exit 0 with no rollback performed would mislead scripted rollbacks.
+      throw new Error(`Migration file not found: ${lastMigration.title} — restore the file from git to roll back.`);
     }
 
     if (!migrationToRollback.down) {
@@ -216,18 +324,27 @@ export class MigrationRunner {
 
   /**
    * Get migration status
+   *
+   * `missing` lists recorded migrations whose file is gone from disk (identity-based,
+   * so a `.ts`-recorded migration with a compiled `.js` on disk is NOT missing) — the
+   * same integrity drift `up()` warns about, surfaced here so `migrate list` can show
+   * it before a deploy or file pruning.
    */
   async status(): Promise<{
     completed: string[];
+    missing: string[];
     pending: string[];
   }> {
     const allMigrations = await this.loadMigrationFiles();
     const state = await this.options.stateStore.loadAsync();
     const completedMigrations = (state.migrations || []).map((m) => m.title);
+    const completedIds = new Set(completedMigrations.map(migrationId));
+    const presentIds = new Set(allMigrations.map((m) => migrationId(m.title)));
 
     return {
       completed: completedMigrations,
-      pending: allMigrations.filter((m) => !completedMigrations.includes(m.title)).map((m) => m.title),
+      missing: completedMigrations.filter((title) => !presentIds.has(migrationId(title))),
+      pending: allMigrations.filter((m) => !completedIds.has(migrationId(m.title))).map((m) => m.title),
     };
   }