thepopebot 1.2.56 → 1.2.57

package/README.md

@@ -130,55 +130,74 @@ docker compose up -d
 
 ---
 
-## CLI Commands
-
-| Command | Description |
-|---------|-------------|
-| `npx thepopebot init` | Scaffold a new project, or check for updated templates in an existing one |
-| `npx thepopebot diff [file]` | List files that differ from package templates, or show the diff for a specific file |
-| `npx thepopebot reset [file]` | List all template files, or restore a specific file (or directory) to the package default |
-| `npm run setup` | Run the full interactive setup wizard |
-| `npm run setup-telegram` | Reconfigure the Telegram webhook (useful when your ngrok URL changes) |
-
----
-
 ## Updating
 
-When thepopebot is updated via npm, your customizations are always preserved — template changes are never applied automatically. To update:
+**1. Update the package**
 
 ```bash
 npm update thepopebot
+```
+
+**2. Scaffold and update templates**
+
+```bash
 npx thepopebot init
 ```
 
-`init` will report which templates have drifted (new files are created automatically, existing files are never overwritten):
+This scaffolds any new template files, runs `npm install`, and updates `THEPOPEBOT_VERSION` in your local `.env`.
+
+**Managed files** — Infrastructure files tightly coupled to the package version (GitHub Actions workflows, docker-compose, event-handler Dockerfile) are **auto-updated** to match the new package version. These files must stay in your project because GitHub and Docker require them at specific paths, but they shouldn't drift from the package.
+
+**User-editable files** — Configuration and app files you're expected to customize (`config/`, `app/`, job Dockerfile) are **never overwritten**. If they differ from the package template, `init` reports the drift so you can review at your own pace:
 
 ```
 Updated templates available:
 These files differ from the current package templates.
 
   config/CRONS.json
-  .github/workflows/run-job.yml
 
 To view differences:  npx thepopebot diff <file>
 To reset to default:  npx thepopebot reset <file>
 ```
 
-Review and accept changes at your own pace:
-
 ```bash
 npx thepopebot diff config/CRONS.json    # see what changed
 npx thepopebot reset config/CRONS.json   # accept the new template
 ```
 
-Or manually merge the changes if you want to keep some of your edits.
+If you customize managed files (e.g., adding steps to a workflow), use `--no-managed` to skip auto-updates and handle them yourself:
+
+```bash
+npx thepopebot init --no-managed
+```
+
+**3. Rebuild for local dev**
+
+```bash
+npm run build
+```
 
-After updating, restart Docker to pick up new image versions:
+**4. Commit and push**
 
 ```bash
-docker compose up -d
+git add -A && git commit -m "upgrade thepopebot to vX.X.X"
+git push
 ```
 
+Pushing to `main` triggers the `rebuild-event-handler.yml` workflow on your server. It detects the version change, runs `thepopebot init`, updates `THEPOPEBOT_VERSION` in the server's `.env`, pulls the new Docker image, restarts the container, rebuilds `.next`, and reloads PM2 — no manual `docker compose` needed.
+
+---
+
+## CLI Commands
+
+| Command | Description |
+|---------|-------------|
+| `npx thepopebot init` | Scaffold a new project, or check for updated templates in an existing one |
+| `npx thepopebot diff [file]` | List files that differ from package templates, or show the diff for a specific file |
+| `npx thepopebot reset [file]` | List all template files, or restore a specific file (or directory) to the package default |
+| `npm run setup` | Run the full interactive setup wizard |
+| `npm run setup-telegram` | Reconfigure the Telegram webhook (useful when your ngrok URL changes) |
+
 ---
 
 ## Docs

package/bin/cli.js

@@ -11,6 +11,20 @@ const __dirname = path.dirname(__filename);
 const command = process.argv[2];
 const args = process.argv.slice(3);
 
+// Files tightly coupled to the package version that are auto-updated by init.
+// These live in the user's project because GitHub/Docker require them at specific paths,
+// but they shouldn't drift from the package version.
+const MANAGED_PATHS = [
+  '.github/workflows/',
+  'docker/event-handler/',
+  'docker-compose.yml',
+  '.dockerignore',
+];
+
+function isManaged(relPath) {
+  return MANAGED_PATHS.some(p => relPath === p || relPath.startsWith(p));
+}
+
 function printUsage() {
   console.log(`
 Usage: thepopebot <command>
@@ -48,6 +62,7 @@ function init() {
   const cwd = process.cwd();
   const packageDir = path.join(__dirname, '..');
   const templatesDir = path.join(packageDir, 'templates');
+  const noManaged = args.includes('--no-managed');
 
   console.log('\nScaffolding thepopebot project...\n');
 
@@ -55,6 +70,7 @@ function init() {
   const created = [];
   const skipped = [];
   const changed = [];
+  const updated = [];
 
   for (const relPath of templateFiles) {
     const src = path.join(templatesDir, relPath);
@@ -72,6 +88,12 @@ function init() {
       const destContent = fs.readFileSync(dest);
       if (srcContent.equals(destContent)) {
         skipped.push(relPath);
+      } else if (!noManaged && isManaged(relPath)) {
+        // Managed file differs — auto-update to match package
+        fs.mkdirSync(path.dirname(dest), { recursive: true });
+        fs.copyFileSync(src, dest);
+        updated.push(relPath);
+        console.log(`  Updated ${relPath}`);
       } else {
         changed.push(relPath);
         console.log(`  Skipped ${relPath} (already exists)`);
@@ -121,6 +143,14 @@ function init() {
     }
   }
 
+  // Report updated managed files
+  if (updated.length > 0) {
+    console.log('\n  Updated managed files:');
+    for (const file of updated) {
+      console.log(`    ${file}`);
+    }
+  }
+
   // Report changed templates
   if (changed.length > 0) {
     console.log('\n  Updated templates available:');

package/drizzle/0000_initial.sql

@@ -0,0 +1,52 @@
+CREATE TABLE `chats` (
+	`id` text PRIMARY KEY NOT NULL,
+	`user_id` text NOT NULL,
+	`title` text DEFAULT 'New Chat' NOT NULL,
+	`starred` integer DEFAULT 0 NOT NULL,
+	`created_at` integer NOT NULL,
+	`updated_at` integer NOT NULL
+);
+--> statement-breakpoint
+CREATE TABLE `messages` (
+	`id` text PRIMARY KEY NOT NULL,
+	`chat_id` text NOT NULL,
+	`role` text NOT NULL,
+	`content` text NOT NULL,
+	`created_at` integer NOT NULL
+);
+--> statement-breakpoint
+CREATE TABLE `notifications` (
+	`id` text PRIMARY KEY NOT NULL,
+	`notification` text NOT NULL,
+	`payload` text NOT NULL,
+	`read` integer DEFAULT 0 NOT NULL,
+	`created_at` integer NOT NULL
+);
+--> statement-breakpoint
+CREATE TABLE `settings` (
+	`id` text PRIMARY KEY NOT NULL,
+	`type` text NOT NULL,
+	`key` text NOT NULL,
+	`value` text NOT NULL,
+	`created_by` text,
+	`created_at` integer NOT NULL,
+	`updated_at` integer NOT NULL
+);
+--> statement-breakpoint
+CREATE TABLE `subscriptions` (
+	`id` text PRIMARY KEY NOT NULL,
+	`platform` text NOT NULL,
+	`channel_id` text NOT NULL,
+	`created_at` integer NOT NULL
+);
+--> statement-breakpoint
+CREATE TABLE `users` (
+	`id` text PRIMARY KEY NOT NULL,
+	`email` text NOT NULL,
+	`password_hash` text NOT NULL,
+	`role` text DEFAULT 'admin' NOT NULL,
+	`created_at` integer NOT NULL,
+	`updated_at` integer NOT NULL
+);
+--> statement-breakpoint
+CREATE UNIQUE INDEX `users_email_unique` ON `users` (`email`);

package/drizzle/meta/0000_snapshot.json

@@ -0,0 +1,321 @@
+{
+  "version": "6",
+  "dialect": "sqlite",
+  "id": "3312cdb9-4f3b-4004-921b-d6598d2eb2a1",
+  "prevId": "00000000-0000-0000-0000-000000000000",
+  "tables": {
+    "chats": {
+      "name": "chats",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "user_id": {
+          "name": "user_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "title": {
+          "name": "title",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false,
+          "default": "'New Chat'"
+        },
+        "starred": {
+          "name": "starred",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false,
+          "default": 0
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "updated_at": {
+          "name": "updated_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    },
+    "messages": {
+      "name": "messages",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "chat_id": {
+          "name": "chat_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "role": {
+          "name": "role",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "content": {
+          "name": "content",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    },
+    "notifications": {
+      "name": "notifications",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "notification": {
+          "name": "notification",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "payload": {
+          "name": "payload",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "read": {
+          "name": "read",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false,
+          "default": 0
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    },
+    "settings": {
+      "name": "settings",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "type": {
+          "name": "type",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "key": {
+          "name": "key",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "value": {
+          "name": "value",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "created_by": {
+          "name": "created_by",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": false,
+          "autoincrement": false
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "updated_at": {
+          "name": "updated_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    },
+    "subscriptions": {
+      "name": "subscriptions",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "platform": {
+          "name": "platform",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "channel_id": {
+          "name": "channel_id",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        }
+      },
+      "indexes": {},
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    },
+    "users": {
+      "name": "users",
+      "columns": {
+        "id": {
+          "name": "id",
+          "type": "text",
+          "primaryKey": true,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "email": {
+          "name": "email",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "password_hash": {
+          "name": "password_hash",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "role": {
+          "name": "role",
+          "type": "text",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false,
+          "default": "'admin'"
+        },
+        "created_at": {
+          "name": "created_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        },
+        "updated_at": {
+          "name": "updated_at",
+          "type": "integer",
+          "primaryKey": false,
+          "notNull": true,
+          "autoincrement": false
+        }
+      },
+      "indexes": {
+        "users_email_unique": {
+          "name": "users_email_unique",
+          "columns": [
+            "email"
+          ],
+          "isUnique": true
+        }
+      },
+      "foreignKeys": {},
+      "compositePrimaryKeys": {},
+      "uniqueConstraints": {},
+      "checkConstraints": {}
+    }
+  },
+  "views": {},
+  "enums": {},
+  "_meta": {
+    "schemas": {},
+    "tables": {},
+    "columns": {}
+  },
+  "internal": {
+    "indexes": {}
+  }
+}

package/drizzle/meta/_journal.json

@@ -0,0 +1,13 @@
+{
+  "version": "7",
+  "dialect": "sqlite",
+  "entries": [
+    {
+      "idx": 0,
+      "version": "6",
+      "when": 1771448716757,
+      "tag": "0000_initial",
+      "breakpoints": true
+    }
+  ]
+}

package/lib/db/index.js

@@ -1,7 +1,9 @@
 import fs from 'fs';
 import path from 'path';
+import { fileURLToPath } from 'url';
 import Database from 'better-sqlite3';
 import { drizzle } from 'drizzle-orm/better-sqlite3';
+import { migrate } from 'drizzle-orm/better-sqlite3/migrator';
 import { thepopebotDb, dataDir } from '../paths.js';
 import * as schema from './schema.js';
 
@@ -25,84 +27,24 @@ export function getDb() {
 }
 
 /**
- * Initialize the database — create tables if they don't exist.
+ * Initialize the database — apply pending migrations.
  * Called from instrumentation.js at server startup.
+ * Uses Drizzle Kit migrations from the package's drizzle/ folder.
  */
 export function initDatabase() {
-  // Ensure data directory exists
   if (!fs.existsSync(dataDir)) {
     fs.mkdirSync(dataDir, { recursive: true });
   }
 
   const sqlite = new Database(thepopebotDb);
   sqlite.pragma('journal_mode = WAL');
+  const db = drizzle(sqlite, { schema });
 
-  sqlite.exec(`
-    CREATE TABLE IF NOT EXISTS users (
-      id TEXT PRIMARY KEY,
-      email TEXT NOT NULL UNIQUE,
-      password_hash TEXT NOT NULL,
-      role TEXT NOT NULL DEFAULT 'admin',
-      created_at INTEGER NOT NULL,
-      updated_at INTEGER NOT NULL
-    )
-  `);
+  // Resolve migrations folder relative to this package (not process.cwd())
+  const __dirname = path.dirname(fileURLToPath(import.meta.url));
+  const migrationsFolder = path.join(__dirname, '..', '..', 'drizzle');
 
-  sqlite.exec(`
-    CREATE TABLE IF NOT EXISTS chats (
-      id TEXT PRIMARY KEY,
-      user_id TEXT NOT NULL,
-      title TEXT NOT NULL DEFAULT 'New Chat',
-      created_at INTEGER NOT NULL,
-      updated_at INTEGER NOT NULL
-    )
-  `);
-
-  sqlite.exec(`
-    CREATE TABLE IF NOT EXISTS messages (
-      id TEXT PRIMARY KEY,
-      chat_id TEXT NOT NULL,
-      role TEXT NOT NULL,
-      content TEXT NOT NULL,
-      created_at INTEGER NOT NULL
-    )
-  `);
-
-  sqlite.exec(`
-    CREATE TABLE IF NOT EXISTS notifications (
-      id TEXT PRIMARY KEY,
-      notification TEXT NOT NULL,
-      payload TEXT NOT NULL,
-      read INTEGER NOT NULL DEFAULT 0,
-      created_at INTEGER NOT NULL
-    )
-  `);
-
-  sqlite.exec(`
-    CREATE TABLE IF NOT EXISTS subscriptions (
-      id TEXT PRIMARY KEY,
-      platform TEXT NOT NULL,
-      channel_id TEXT NOT NULL,
-      created_at INTEGER NOT NULL
-    )
-  `);
-
-  sqlite.exec(`
-    CREATE TABLE IF NOT EXISTS settings (
-      id TEXT PRIMARY KEY,
-      type TEXT NOT NULL,
-      key TEXT NOT NULL,
-      value TEXT NOT NULL,
-      created_by TEXT,
-      created_at INTEGER NOT NULL,
-      updated_at INTEGER NOT NULL
-    )
-  `);
-
-  // Migration: add starred column to chats
-  try {
-    sqlite.exec(`ALTER TABLE chats ADD COLUMN starred INTEGER NOT NULL DEFAULT 0`);
-  } catch {}
+  migrate(db, { migrationsFolder });
 
   sqlite.close();
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "thepopebot",
-  "version": "1.2.56",
+  "version": "1.2.57",
   "type": "module",
   "description": "Create autonomous AI agents with a two-layer architecture: Next.js Event Handler + Docker Agent.",
   "bin": {
@@ -25,13 +25,16 @@
     "api/",
     "config/",
     "setup/",
-    "templates/"
+    "templates/",
+    "drizzle/"
   ],
   "scripts": {
     "build": "esbuild lib/chat/components/*.jsx lib/chat/components/**/*.jsx --outdir=lib/chat/components --format=esm --jsx=automatic --outbase=lib/chat/components",
     "prepublishOnly": "npm run build",
     "local": "bash bin/local.sh",
     "postinstall": "if [ -f bin/postinstall.js ]; then node bin/postinstall.js; fi",
+    "db:generate": "drizzle-kit generate",
+    "db:studio": "drizzle-kit studio",
     "test": "echo \"No tests yet\" && exit 0"
   },
   "keywords": [
@@ -85,6 +88,7 @@
     }
   },
   "devDependencies": {
+    "drizzle-kit": "^0.31.9",
     "esbuild": "^0.27.3"
   },
   "engines": {

package/templates/.github/workflows/rebuild-event-handler.yml

@@ -40,7 +40,6 @@ jobs:
             if [ -n "$OLD_TPB" ] && [ "$OLD_TPB" != "$NEW_TPB" ]; then
               # Version changed — run thepopebot init to scaffold new templates
               npx thepopebot init
-              npm install --omit=dev
 
               # Commit any template changes from init
               git add -A

package/templates/.github/workflows/upgrade-event-handler.yml

@@ -26,7 +26,7 @@ jobs:
             git config user.name "thepopebot"
             git config user.email "thepopebot@users.noreply.github.com"
 
-            npm install
+            npm install --omit=dev
             OLD_VERSION=$(node -p "require(\"./node_modules/thepopebot/package.json\").version")
             npm update thepopebot
             NEW_VERSION=$(node -p "require(\"./node_modules/thepopebot/package.json\").version")

package/templates/CLAUDE.md

@@ -55,12 +55,19 @@ This is an autonomous AI agent powered by [thepopebot](https://github.com/stephe
 
 ### Updating thepopebot
 
-When the package is updated via `npm update thepopebot`, template changes are **not** applied automatically — your customizations are preserved. To check for updates:
+When the package is updated via `npm update thepopebot`, run `npx thepopebot init` to apply changes:
 
-1. `npm update thepopebot` — updates the package
-2. `npx thepopebot init` — reports which templates have drifted (does not overwrite)
-3. `npx thepopebot diff <file>` — review what changed
-4. `npx thepopebot reset <file>` — accept the new template, or manually merge
+- **Managed files** (GitHub workflows, docker-compose, event-handler Dockerfile) are **auto-updated** to match the package version. These are infrastructure files that must stay in the project but shouldn't drift.
+- **User-editable files** (config, app pages, job Dockerfile) are **never overwritten**. Drifted files are reported so you can review them.
+
+```
+1. npm update thepopebot         — updates the package
+2. npx thepopebot init           — auto-updates managed files, reports drifted user files
+3. npx thepopebot diff <file>    — review what changed in a user-editable file
+4. npx thepopebot reset <file>   — accept the new template, or manually merge
+```
+
+Use `npx thepopebot init --no-managed` to skip auto-updates of managed files (for power users who customize workflows).
 
 ## API Endpoints