@thierrynakoa/fire-flow 12.2.2 → 13.0.1

package/skills-library/database-solutions/POSTGRES_SQL_TEMPLATE_BINDING_ERROR.md

@@ -0,0 +1,240 @@
+# PostgreSQL Tagged Template Literal Binding Errors - Solution & Prevention
+
+## The Problem
+
+When using the `postgres` (porsager/postgres) library with tagged template literals, nested template compositions cause binding errors. The error message is cryptic and doesn't point to the actual issue.
+
+### Error Message
+```
+PostgresError: Expected 1 bindings, saw 0
+```
+
+Or variations like:
+```
+PostgresError: bind message supplies 0 parameters, but prepared statement requires 1
+```
+
+### Why It Was Hard
+
+- The error doesn't indicate which query or which binding failed
+- Nested `sql` template literals appear to work in simple cases
+- The SQL-compatibility layer (proxy pattern) masks the actual postgres.js instance
+- Dynamic query building with conditionals creates complex nesting that breaks
+
+### Impact
+
+- API endpoints return 500 errors
+- Courses, users, or other data fail to load
+- The application appears broken with no obvious cause
+- Debugging requires understanding postgres.js internals
+
+---
+
+## The Solution
+
+### Root Cause
+
+The postgres.js tagged template literal syntax doesn't support composing nested template literals when building dynamic queries with conditionals.
+
+**Bad Code (Breaks):**
+```javascript
+export const getCourses = async (filters = {}) => {
+  let query = sql`SELECT * FROM courses`;
+  let hasWhere = false;
+
+  if (filters.is_published !== undefined) {
+    // This creates a nested template that breaks binding
+    query = sql`${query} WHERE is_published = ${filters.is_published}`;
+    hasWhere = true;
+  }
+
+  if (filters.category) {
+    const whereOrAnd = hasWhere ? sql`AND` : sql`WHERE`;
+    // Another nested composition - bindings get lost
+    query = sql`${query} ${whereOrAnd} category = ${filters.category}`;
+  }
+
+  return await query; // ERROR: Expected N bindings, saw 0
+};
+```
+
+### How to Fix
+
+Use `sql.unsafe()` with manual parameterized queries for dynamic query building:
+
+**Good Code (Works):**
+```javascript
+export const getCourses = async (filters = {}, options = {}) => {
+  const conditions = [];
+  const values = [];
+  let paramIndex = 1;
+
+  // Build conditions dynamically
+  if (filters.is_published !== undefined) {
+    conditions.push(`is_published = $${paramIndex++}`);
+    values.push(filters.is_published);
+  }
+
+  if (filters.category) {
+    conditions.push(`category = $${paramIndex++}`);
+    values.push(filters.category);
+  }
+
+  if (filters.instructor_id) {
+    conditions.push(`instructor_id = $${paramIndex++}`);
+    values.push(filters.instructor_id);
+  }
+
+  // Build the complete query
+  let queryText = 'SELECT * FROM courses';
+  if (conditions.length > 0) {
+    queryText += ' WHERE ' + conditions.join(' AND ');
+  }
+  queryText += ' ORDER BY created_at DESC';
+
+  // Add pagination
+  if (options.limit) {
+    queryText += ` LIMIT $${paramIndex++}`;
+    values.push(options.limit);
+  }
+  if (options.offset) {
+    queryText += ` OFFSET $${paramIndex++}`;
+    values.push(options.offset);
+  }
+
+  // Execute with sql.unsafe() - handles parameterized queries correctly
+  const result = await sql.unsafe(queryText, values);
+  return result;
+};
+```
+
+### When to Use Each Approach
+
+| Scenario | Use This |
+|----------|----------|
+| Static queries (no conditionals) | `sql`SELECT * FROM users WHERE id = ${id}`` |
+| Dynamic queries with conditionals | `sql.unsafe(queryText, values)` |
+| INSERT with dynamic columns | `sql.unsafe()` with built column/value lists |
+| UPDATE with dynamic fields | `sql.unsafe()` with built SET clause |
+
+### Complete UPDATE Example
+
+```javascript
+export const updateCourse = async (id, updateData) => {
+  const fieldMapping = {
+    isPublished: 'is_published',
+    coverImage: 'cover_image',
+    // ... map camelCase to snake_case
+  };
+
+  const mappedData = {};
+  Object.keys(updateData).forEach(key => {
+    if (key !== 'id') {
+      const dbField = fieldMapping[key] || key;
+      mappedData[dbField] = updateData[key];
+    }
+  });
+
+  const fields = Object.keys(mappedData).filter(
+    field => mappedData[field] !== undefined
+  );
+
+  if (fields.length === 0) {
+    return (await sql`SELECT * FROM courses WHERE id = ${id}`)[0];
+  }
+
+  // Build SET clause: "field1" = $1, "field2" = $2
+  const setClause = fields.map((f, i) => `"${f}" = $${i + 1}`).join(', ');
+  const values = fields.map(f => mappedData[f]);
+
+  // id is the last parameter
+  const result = await sql.unsafe(
+    `UPDATE courses SET ${setClause} WHERE id = $${values.length + 1} RETURNING *`,
+    [...values, id]
+  );
+
+  return result[0];
+};
+```
+
+---
+
+## Testing the Fix
+
+### Before (Broken)
+```bash
+GET /api/courses?is_published=true
+Response: 500 Internal Server Error
+Log: PostgresError: Expected 1 bindings, saw 0
+```
+
+### After (Working)
+```bash
+GET /api/courses?is_published=true
+Response: 200 OK
+Body: [{ id: "...", title: "Course 1", ... }]
+```
+
+### Test Cases
+```javascript
+// Test dynamic filtering
+const published = await getCourses({ is_published: true });
+const byCategory = await getCourses({ category: 'programming' });
+const combined = await getCourses({
+  is_published: true,
+  category: 'programming',
+  instructor_id: 'abc-123'
+});
+const paginated = await getCourses({}, { limit: 10, offset: 20 });
+```
+
+---
+
+## Prevention
+
+1. **Never nest sql template literals** for conditional query building
+2. **Use sql.unsafe()** for any dynamic query construction
+3. **Track parameter index** manually with `$${paramIndex++}`
+4. **Always use parameterized queries** - never interpolate values directly
+5. **Test with multiple filter combinations** before deploying
+
+---
+
+## Related Patterns
+
+- Dynamic INSERT queries with variable columns
+- Bulk UPDATE operations
+- Search queries with optional filters
+- Pagination with variable page sizes
+
+---
+
+## Common Mistakes to Avoid
+
+- Using `sql`${sql`...`}`` nesting for dynamic parts
+- Forgetting to increment parameter index
+- Mixing postgres.js template syntax with raw string interpolation
+- Not testing with multiple filter combinations
+- Using string interpolation for values (SQL injection risk!)
+
+---
+
+## Resources
+
+- [postgres.js Documentation](https://github.com/porsager/postgres)
+- [PostgreSQL Parameterized Queries](https://www.postgresql.org/docs/current/sql-prepare.html)
+
+---
+
+## Time to Implement
+
+**15-30 minutes** to refactor a broken query function
+
+## Difficulty Level
+
+This problem is easy to solve once understood, but hard to diagnose initially.
+
+---
+
+**Author Notes:**
+This issue cost 2+ hours of debugging. The error message gives no indication of which query failed or why. The key insight: postgres.js tagged templates are for static queries only. Any dynamic query building needs `sql.unsafe()` with manual parameter tracking.

package/skills-library/database-solutions/PRISMA_DB_PUSH_DATA_LOSS_PREVENTION.md

@@ -0,0 +1,141 @@
+# Prisma db push Data Loss Prevention
+
+## Problem
+
+Using `prisma db push` on a database with imported data (lexicons, Bible text, etc.) silently **drops and recreates tables**, wiping all imported data. This is catastrophic when tables contain thousands of rows from time-consuming import processes.
+
+## Root Cause
+
+`prisma db push` is designed for rapid prototyping. When schema changes are incompatible, it drops affected tables without warning. Unlike `prisma migrate dev`, it creates **no migration history** — so there's no `_prisma_migrations` table, and no way to know when data was lost.
+
+### How to Detect It Happened
+
+If your database has **no `_prisma_migrations` table**, `db push` was used instead of `migrate dev`:
+
+```sql
+SELECT EXISTS (
+  SELECT FROM information_schema.tables
+  WHERE table_name = '_prisma_migrations'
+);
+```
+
+## Solution
+
+### 1. Block `db push` in package.json
+
+Replace the default `db:push` script with a blocker that forces developers to use the explicit `--force` variant:
+
+```json
+{
+  "scripts": {
+    "db:push": "echo '⛔ BLOCKED: prisma db push drops tables and destroys imported data. Use prisma migrate dev instead. If you REALLY need db push, use: npm run db:push:force' && exit 1",
+    "db:push:force": "npx prisma db push",
+    "db:migrate": "npx prisma migrate dev"
+  }
+}
+```
+
+### 2. Add a Data Health Check Script
+
+Create a quick-check script that verifies critical tables have expected row counts:
+
+```typescript
+// server/data/check-data-counts.ts
+import { PrismaClient } from '@prisma/client';
+
+const prisma = new PrismaClient();
+
+interface TableCheck {
+  name: string;
+  count: () => Promise<number>;
+  minExpected: number;
+}
+
+const checks: TableCheck[] = [
+  { name: 'bible_text (KJV verses)', count: () => prisma.bibleText.count(), minExpected: 30000 },
+  { name: 'hebrew_lexicons', count: () => prisma.hebrewLexicon.count(), minExpected: 9000 },
+  { name: 'greek_lexicons', count: () => prisma.greekLexicon.count(), minExpected: 10000 },
+];
+
+(async () => {
+  console.log('\n  Data Health Check\n  ' + '='.repeat(50));
+  let allOk = true;
+
+  for (const check of checks) {
+    try {
+      const count = await check.count();
+      const status = count >= check.minExpected ? 'OK' : 'MISSING';
+      const icon = count >= check.minExpected ? '[OK]' : '[!!]';
+      if (status === 'MISSING') allOk = false;
+      console.log(`  ${icon} ${check.name}: ${count.toLocaleString()} rows`);
+    } catch {
+      console.log(`  [!!] ${check.name}: TABLE NOT FOUND`);
+      allOk = false;
+    }
+  }
+
+  if (!allOk) {
+    console.log('  Some data is missing! Run import scripts to restore.');
+  }
+  await prisma.$disconnect();
+})();
+```
+
+Add to package.json:
+```json
+{
+  "scripts": {
+    "db:check-data": "npx tsx server/data/check-data-counts.ts"
+  }
+}
+```
+
+### 3. Document Import Restoration Commands
+
+Always have clear import scripts ready:
+```json
+{
+  "scripts": {
+    "import-bible": "npx tsx server/data/import-bible.ts",
+    "import-lexicons": "npx tsx server/data/run-lexicon-import.ts"
+  }
+}
+```
+
+## Key Takeaways
+
+| Command | Safe? | Use When |
+|---------|-------|----------|
+| `prisma migrate dev` | Yes | Schema changes in development |
+| `prisma migrate deploy` | Yes | Schema changes in production |
+| `prisma db push` | **NO** | Never on databases with imported data |
+| `prisma db push` | Maybe | Fresh prototyping with no real data |
+
+## Additional Gotcha: Prisma CLI Only Reads `.env`
+
+Prisma CLI reads **only `.env`** — not `.env.local` or `.env.production`. If your project uses `.env.local` for development, you must ensure the correct `DATABASE_URL` is in `.env` before running any Prisma CLI commands (migrations, push, generate).
+
+## Additional Gotcha: dotenv Import Order
+
+When using ESM modules, `PrismaClient` instantiated at the top of a module executes **before** `dotenv.config()` in the importing file, because ESM imports are hoisted. The Prisma client reads `DATABASE_URL` at instantiation time.
+
+**Wrong:**
+```typescript
+// run-import.ts
+import { config } from 'dotenv';
+import { importData } from './import-data.js'; // PrismaClient instantiates HERE
+config(); // Too late — PrismaClient already read DATABASE_URL
+```
+
+**Fix:** Set `DATABASE_URL` as a shell environment variable before running:
+```bash
+DATABASE_URL="postgresql://user:pass@localhost:5433/db" npx tsx run-import.ts
+```
+
+## Tech Stack
+- Prisma ORM (any version)
+- PostgreSQL / MySQL
+- Node.js with ESM modules
+
+## Tags
+`prisma` `data-loss` `migration` `db-push` `postgresql` `dotenv` `esm`