generatesaas 3.4.0 → 3.4.2

package/dist/skill/content/generatesaas-update/SKILL.md

@@ -122,8 +122,8 @@ Creates:
   `added` files; `unmodified`, `deleted`, and `removed` files have none.
 - `references/update-manifest.json` - lists of added, modified, and removed files,
   plus `renamed` (upstream renames detected by identical content: `[{from, to}]`)
-  and `sensitive` (database schema and `.env.example` changes, which carry follow-up work:
-  a schema push, new env vars)
+  and `sensitive` (database schema, `packages/database/drizzle/`, and
+  `.env.example` changes that carry follow-up work: migrations, new env vars)
 
 ### Step 2: Present Changelog
 
@@ -134,7 +134,7 @@ Each release's notes may be one of two shapes:
 - **Curated** - contains explicit `## Breaking`, `## Migration`, `## Features`, `## Fixes` (or similar) sections. Use those sections verbatim; they are authoritative.
 - **Raw** - an auto-generated flat list of change titles (one line per commit or PR, no sections). This is the common case. You can group the titles into features/fixes by reading them, but you **cannot** reliably infer breaking changes or migration steps from titles alone.
 
-Also read `sensitive` in `references/update-manifest.json`: database schema changes mean the live database has to be pushed to match, and may drop or rewrite existing data (see [Database](#database-schema)), and `.env.example` changes mean new or changed environment variables. Mention both in the summary and again in the final post-update steps.
+Also read `sensitive` in `references/update-manifest.json`: database schema changes mean a migration is likely required, anything under `packages/database/drizzle/` means upstream migrations that must be regenerated into the project's own history before they will run (see [Database](#database-schema-migrations)), and `.env.example` changes mean new or changed environment variables. Mention all three in the summary and again in the final post-update steps.
 
 Present a clear, organized summary:
 
@@ -143,9 +143,9 @@ Present a clear, organized summary:
 > **Breaking changes:** {from a Breaking section, or "⚠️ Not specified in the changelog - review the diffs in Step 7 carefully, especially database/schema and config files"}
 > **New features:** {list}
 > **Bug fixes:** {list}
-> **Follow-up steps required:** {from a Migration section, or "⚠️ None listed - if this update changes the database schema or required env vars, you may still have follow-up work. I'll flag schema/config changes as we review them."}
+> **Migration steps required:** {from a Migration section, or "⚠️ None listed - if this update changes the database schema or required env vars, you may still need to migrate. I'll flag schema/config changes as we review them."}
 
-**Be honest about uncertainty.** Never present an inferred "Breaking changes: None" or "Follow-up steps: None" as if it were authoritative when the changelog is raw PR titles - say it's not specified and that you'll watch for schema/config/env changes during the file review. A silently-missed schema change (e.g. a new DB column that never reaches the database) is the worst failure mode.
+**Be honest about uncertainty.** Never present an inferred "Breaking changes: None" or "Migration steps: None" as if it were authoritative when the changelog is raw PR titles - say it's not specified and that you'll watch for schema/config/env changes during the file review. A silently-missed migration (e.g. a new DB column) is the worst failure mode.
 
 Then ask: *"Ready to proceed with the file analysis?"*
 
@@ -341,13 +341,15 @@ Preview first (especially useful in Careful mode or when unsure), then apply:
 node __SKILL_ROOT__/generatesaas-update/scripts/apply-auto.js --dry-run   # lists every file it would update/create, changes nothing
 node __SKILL_ROOT__/generatesaas-update/scripts/apply-auto.js
 ```
-`apply-auto.js` copies only `unmodified` (safe replace) + `new` (any upstream-added file whose tree the buyer has not deleted). It will **never** touch `newInDeletedTree` files, and it aborts without writing anything if any source is missing from staging (no half-applied tree). Report what was applied, and mark all auto-update and new file items as `[x]` in the plan.
+`apply-auto.js` copies only `unmodified` (safe replace) + `new` (any upstream-added file whose tree the buyer has not deleted). It will **never** touch `newInDeletedTree` files, and it aborts without writing anything if any source is missing from staging (no half-applied tree). Report what was applied. Mark all auto-update and new file items as `[x]` in the plan - **except any file the script reports as held back**, which was never applied and moves to **Skipped** with the regenerate reason. Marking a held-back migration `[x]` records a table that does not exist as delivered, in the one artifact that outlives the session.
+
+It also holds back everything under `packages/database/drizzle/` and lists what it skipped. Those are NOT pending work you can copy in afterwards - migrations must be regenerated into the project's own history or they never run. See [Database](#database-schema-migrations).
 
 **Careful mode:**
 Present the list of safe auto-updates and new files to the user:
 > These {N} files haven't been customized and can be safely replaced with the new version. {N} new files will be created. Want me to apply all of these, or would you like to review any specific ones first?
 
-If the user wants to review specific files, show them the diff for each requested file. Apply only after approval.
+If the user wants to review specific files, show them the diff for each requested file. Apply only after approval. Do not count held-back migrations in either number - they are not being created, and presenting them as applied is the same error as marking them `[x]`.
 
 ### Step 7: Review Modified Files
 
@@ -537,7 +539,7 @@ This updates `.generatesaas/manifest.json` to the new version, regenerates file
 
 ### Post-update steps
 - Run `pnpm install` if dependencies changed
-- Push the schema if it was updated (`pnpm --filter @repo/database setup` in development)
+- Run database migrations if schema was updated
 - {any other steps from the changelog}
 
 ### Skipped changes
@@ -574,44 +576,32 @@ Config files are the most commonly customized.
 - Keep user-added business logic intact
 - Watch for new imports or dependency changes
 
-### Database (schema)
+### Database (schema, migrations)
 
 - NEVER auto-merge schema files - always present for user review
-- Warn about schema changes, and say what applying them will do to live data
-
-**This boilerplate keeps no migration history.** The schema files under `packages/database/src/db/`
-are the source of truth, and `db:deploy` applies them with `drizzle-kit push` and then verifies the
-live database really matches. There is no `drizzle/` directory, no `generate`, no `migrate`, and
-nothing to regenerate into a project's own history. An upstream schema change is an ordinary file
-change here: merge it after review, then apply it.
-
-1. Merge the upstream change into `packages/database/src/db/*.ts` (user review first, as above).
-2. Apply it: `pnpm --filter @repo/database setup` in development, and `db:deploy` on the next
-   deploy in production.
-
-**Say what push will do BEFORE the user applies it to a database with data in it.** Push makes the
-database match the schema files, and `db:deploy` passes `--force`, so it does not stop to ask:
-
-- a new table or column is added, which is safe;
-- a **removed** column or table is **dropped, with its data**;
-- a **narrowed** type is rewritten in place, and Postgres refuses the whole statement when a value
-  does not fit - it never truncates one silently.
-
-For anything in the last two categories, tell the user what will be dropped or rewritten and let
-them decide, rather than applying it as part of a routine update. If they need the old data, it has
-to be copied somewhere else first; push has no place to put a data transform and no down step.
-
-**Push can report success without applying anything.** `drizzle-kit push` prints its error and
-still exits 0 when a change cannot be applied. `db:deploy` is why that no longer matters in
-production: it re-reads the live schema afterwards and fails the boot naming the tables and columns
-that differ. If a user applies a change by hand with `push` instead, have them run `db:deploy`
-after it, or check the column themselves - a green push is not evidence on its own.
-
-**A project generated before push-only** may still have `packages/database/drizzle/` and `generate`
-/ `migrate` scripts. Those are vestigial: the update removes the scripts, and the directory can be
-deleted whenever the user is comfortable. Their database is already at whatever state the old
-migrations left it, so the first push simply diffs from there - usually a no-op if the schema
-matched. Review that first push on a production database rather than letting it ride a deploy.
+- Warn about schema changes that may require new migrations
+
+**Never copy an upstream migration into `packages/database/drizzle/` as-is, and never overwrite the project's `drizzle/meta/_journal.json` or `drizzle/meta/*_snapshot.json` with the upstream versions.** A project's migration history is its own: upstream changes get re-derived into it, never pasted over it. Regenerate instead:
+
+1. Merge the upstream schema change into `packages/database/src/db/*.ts` (user review first, as above).
+2. Run `pnpm --filter @repo/database generate`. Drizzle diffs the project's own latest snapshot against the merged schema and writes a fresh migration carrying a current timestamp, the next `idx`, a correct `prevId`, and a snapshot that describes the project's real schema.
+3. Open the upstream `.sql` under `.generatesaas/staging/packages/database/drizzle/` and copy any hand-written statements into the generated file: seeds, backfills, data migrations, `ON CONFLICT` guards. `generate` infers structure only, so those are lost otherwise.
+4. Run `pnpm --filter @repo/database migrate`.
+
+**Why the upstream file cannot just be added.** Drizzle applies a migration only when its journal `when` value is greater than the `created_at` of the last migration already applied to the database. A project that generated its own migration after the upstream one was authored sits above that mark, so a copied-in upstream file is **skipped silently and permanently**: no error, no warning, no log line, and `migrate` still exits 0. The table or column it was supposed to create never exists, and the miss surfaces later as runtime failures on whatever code path needs it. Taking the upstream `_journal.json` and snapshots instead of the project's own is the second half of the same trap: the final snapshot then knows nothing about the project's own tables, so the next `generate` re-emits `CREATE TABLE` for tables that already exist.
+
+**If step 2 reports no schema changes**, the upstream migration is data-only and there is nothing to regenerate. This is the ONLY case where the `.sql` is copied in by hand. Copying the file is not enough on its own: drizzle reads `_journal.json` and runs only the entries listed there, so a `.sql` sitting in the folder with no entry is never opened. Do all four, in this order:
+
+1. **Append an entry** to the project's own `_journal.json`, copied from the upstream entry. This is the load-bearing step - without it nothing else matters.
+2. Set that entry's `when` **greater than the largest `when` already applied** to the database, never the upstream value. Drizzle takes the highest `created_at` in `drizzle.__drizzle_migrations` as its floor and skips everything at or below it.
+3. Set that entry's `idx` to continue the project's own sequence (last `idx` + 1).
+4. Rename the `.sql` file so its name matches the entry's `tag`, or set the `tag` to the file's existing name. Drizzle loads the file **by tag** and throws if the two disagree.
+
+Still do not bring the upstream `meta/*_snapshot.json`, here least of all. Nothing needs it: `migrate` never reads snapshots, and `generate` needs only ONE previous snapshot, which must be the project's own. Adding an upstream one breaks `generate` whichever way it sorts - `meta/` is read in filename order, so an upstream snapshot that sorts LAST silently becomes the diff base (re-emitting `CREATE TABLE` for the project's own tables, and `DROP TABLE` for anything upstream carries that the project removed), while one that sorts earlier collides on `prevId` with the project's own, which prints a warning, writes no migration, and still exits 0.
+
+**A project generated on 3.3.0-3.4.1 has no `packages/database/drizzle/` directory** - those releases were push-only, so a push shaped its database and there is no history to regenerate into. Run `pnpm --filter @repo/database generate` once to write its baseline migration and commit it; until then `db:deploy` prints a warning and falls back to a push for that deploy. **Generate that baseline BEFORE merging any schema change from this update**, so it describes the schema the database actually has: `db:deploy` records a baseline as applied rather than executing it only when every table it creates already exists. A baseline generated after the merge creates tables the database lacks, and `db:deploy` then aborts with a message that says how to regenerate it - it never runs a baseline against a half-shaped database. Once the baseline exists, merge the upstream schema change and run `generate` again for the difference, as above.
+
+Whichever path applies, `db:deploy` re-reads the live schema afterwards and fails the boot naming any table or column that differs - so a migrate (or a push) that quietly changed nothing is caught at boot rather than at runtime.
 
 ### Package Dependencies (`package.json`)
 

package/dist/skill/content/generatesaas-update/scripts/apply-auto.js

@@ -10,6 +10,9 @@ const fs = require("node:fs");
 const path = require("node:path");
 const { findProjectRoot, ensureDir, STAGING_DIR } = require("./_helpers.js");
 
+/** Drizzle migration history. Held back from every auto-copy - see filesToCopy. */
+const MIGRATION_DIR = "packages/database/drizzle/";
+
 function main() {
 	const root = findProjectRoot();
 	if (!root) {
@@ -41,15 +44,19 @@ function main() {
 	// handles by prompt (never here).
 	const candidates = [...classification.unmodified, ...classification.new];
 
-	// NOTHING IS HELD BACK ANY MORE. This used to quarantine `packages/database/drizzle/`,
-	// because a pasted-in migration silently never runs. The boilerplate keeps no migration
-	// history now - the schema files are the source of truth and `db:deploy` pushes them - so
-	// there is no such directory upstream and nothing to quarantine. A schema change arrives as a
-	// change to `packages/database/src/db/`, which is an ordinary file like any other here.
-	const filesToCopy = candidates;
+	// Migrations are NEVER auto-applied, and both failure modes are silent.
+	// Drizzle runs a migration only when its journal `when` beats the newest one
+	// already applied, so an upstream .sql copied in below that mark never runs
+	// and `migrate` still exits 0; and an upstream meta snapshot becomes the diff
+	// base for the next `generate`, which then re-emits the buyer's own tables.
+	// The skill's Database section regenerates them into the project's own
+	// history instead, which is only possible if this step leaves them alone.
+	const heldMigrations = candidates.filter((f) => f.startsWith(MIGRATION_DIR));
+	const filesToCopy = candidates.filter((f) => !f.startsWith(MIGRATION_DIR));
 
 	if (filesToCopy.length === 0) {
 		console.log("No files to auto-update. All changed files need manual merge.");
+		reportHeldMigrations(heldMigrations);
 		return;
 	}
 
@@ -77,6 +84,7 @@ function main() {
 			console.log(`  ${unmodifiedSet.has(filePath) ? "update" : "create"}  ${filePath}`);
 		}
 		console.log("\n[dry run] No files were modified.");
+		reportHeldMigrations(heldMigrations);
 		return;
 	}
 
@@ -110,10 +118,36 @@ function main() {
 		}
 	}
 
+	reportHeldMigrations(heldMigrations);
 
 	console.log("\nAfter merging all files, run complete-update.js to finalize.");
 }
 
+/**
+ * Report migration files deliberately NOT copied, so the AI knows they are
+ * waiting rather than assuming the tree is settled. Silent when there are none.
+ *
+ * States the whole procedure rather than pointing at SKILL.md. The AI loads
+ * SKILL.md at the start of the update, BEFORE step 1a refreshes the bundle, so
+ * for one release after this ships it is following the previous release's prose
+ * - which says migrations can be added directly. This output has to stand alone
+ * against that reader, or the fix reads as an obstacle to work around.
+ */
+function reportHeldMigrations(heldMigrations) {
+	if (heldMigrations.length === 0) return;
+	console.log(`\n${heldMigrations.length} migration file(s) held back - NOT copied into the project:`);
+	for (const f of heldMigrations) console.log(`  - ${f}`);
+	console.log("\nDo NOT copy these in by hand, and do not take the upstream meta/ snapshots either.");
+	console.log("Drizzle runs a migration only when its journal entry is newer than the newest one");
+	console.log("already applied, so a pasted-in file never runs, `migrate` still exits 0, and the");
+	console.log("table it should have created silently does not exist. Regenerate instead:");
+	console.log("  1. Merge the upstream schema change into packages/database/src/db/");
+	console.log("  2. Run: pnpm --filter @repo/database generate");
+	console.log("  3. Copy any hand-written statements (seeds, backfills, data migrations) out of the");
+	console.log("     staged .sql at .generatesaas/staging/<path above> into the file step 2 wrote -");
+	console.log("     `generate` infers structure only, so those statements are lost otherwise");
+	console.log("  4. Run: pnpm --filter @repo/database migrate");
+}
 
 try {
 	main();

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "generatesaas",
-  "version": "3.4.0",
+  "version": "3.4.2",
   "type": "module",
   "description": "CLI for scaffolding and managing GenerateSaaS projects",
   "license": "UNLICENSED",