thepopebot 1.2.76-beta.2 → 1.2.76-beta.21

package/README.md

@@ -104,7 +104,7 @@ The wizard walks you through everything:
 **That's it.** Visit your APP_URL when the wizard finishes.
 
 - **Web Chat**: Visit your APP_URL to chat with your agent, create jobs, upload files
-- **Telegram** (optional): Run `npm run setup-telegram` to connect a Telegram bot
+- **Telegram** (optional): Connect a Telegram bot from `/admin/event-handler/telegram`
 - **Webhook**: Send a POST to `/api/create-agent-job` with your API key to create jobs programmatically
 - **Cron**: Edit `agent-job/CRONS.json` to schedule recurring jobs
 
@@ -143,9 +143,9 @@ See [Coding Agents](docs/CODING_AGENTS.md) for details on all five agent backend
 > ```bash
 > # Update .env and GitHub variable in one command:
 > npx thepopebot set-var APP_URL https://your-new-url.ngrok.io
-> # If Telegram is configured, re-register the webhook:
-> npm run setup-telegram
 > ```
+>
+> If Telegram is configured, click **Re-register webhook** at `/admin/event-handler/telegram` after the URL change.
 
 ---
 

package/api/CLAUDE.md

@@ -4,9 +4,14 @@ This directory contains the route handlers for all `/api/*` endpoints. These rou
 
 ## Auth
 
-All routes (except `/telegram/webhook` and `/github/webhook`, which use their own webhook secrets) require a valid API key passed via the `x-api-key` header. API keys are stored in the SQLite database and managed through the admin UI — they are NOT environment variables.
+Most routes require a valid API key passed via the `x-api-key` header. API keys are stored in the SQLite database and managed through the admin UI — they are NOT environment variables.
 
-Auth flow: `x-api-key` header -> `verifyApiKey()` -> database lookup (hashed, timing-safe comparison).
+**Public routes** (no API key needed): `/ping`, `/telegram/webhook` (Telegram webhook secret), `/github/webhook` (GitHub webhook secret), `/oauth/callback` (validated via short-lived `state` token).
+
+Auth flow: `x-api-key` header → `verifyApiKey()` → DB lookup (hashed, timing-safe comparison). Two key types exist:
+
+- **User-owned API keys** — long-lived, created via the admin UI, used by external callers (cURL, GitHub Actions, Telegram register).
+- **Per-job agent API keys** (`agent_job_api_key`) — short-lived, auto-created when an agent-job container launches (`createAgentJobApiKey()` in `lib/db/api-keys.js`), tied to the container name, and cleaned up by the maintenance cron after expiry. Only this key type is allowed to call `/api/get-agent-job-secret` (the route rejects other types).
 
 ## Do NOT use these routes for browser UI
 
@@ -25,10 +30,12 @@ Browser-facing data fetching uses **fetch route handlers** colocated with pages
 |--------|------|------|---------|
 | GET | `/api/ping` | None | Health check |
 | POST | `/api/create-agent-job` | `x-api-key` | Create agent job |
-| GET | `/api/get-agent-job-secret` | `x-api-key` | Get an agent job secret; oauth2 credentials return only the access_token (auto-refreshed) |
+| GET | `/api/get-agent-job-secret` | `agent_job_api_key` only | Get an agent job secret. `oauth2` credentials return only the access_token (auto-refreshed under a lock; rotated refresh tokens are persisted back). Other secret types return the raw value. |
+| POST | `/api/set-agent-job-secret` | `agent_job_api_key` only | Create or update an agent-job secret from inside the container (used by the `set-secret` skill). |
 | GET | `/api/agent-job-list-secrets` | `x-api-key` | List agent job secret keys (no values); returns `{secrets: [{key, isSet, updatedAt, secretType}]}` |
 | GET | `/api/agent-jobs/status` | `x-api-key` | Agent job status (query: `?agent_job_id=`) |
-| POST | `/api/telegram/webhook` | Telegram webhook secret | Telegram message handler |
+| POST | `/api/telegram/webhook` | Telegram webhook secret | Telegram message handler (per-user routing via `user_channels`; verifies via `/verify <code>`, dispatches `/session` commands) |
 | POST | `/api/telegram/register` | `x-api-key` | Register bot token + webhook URL |
 | POST | `/api/github/webhook` | GitHub webhook secret | GitHub event handler |
 | POST | `/api/cluster/:clusterId/role/:roleId/webhook` | `x-api-key` | Trigger cluster role execution |
+| GET/POST | `/api/oauth/callback` | `state` token | OAuth provider redirect target. Exchanges `code` for tokens, persists via `setAgentJobSecret(name, stored, 'oauth')`. |

package/api/index.js

@@ -1,9 +1,11 @@
-import { createHash, timingSafeEqual } from 'crypto';
+import { createHash, timingSafeEqual, randomUUID } from 'crypto';
 import { createAgentJob } from '../lib/tools/create-agent-job.js';
 import { setWebhook } from '../lib/tools/telegram.js';
 import { getAgentJobStatus, fetchAgentJobLog } from '../lib/tools/github.js';
 import { getTelegramAdapter } from '../lib/channels/index.js';
-import { chat, summarizeAgentJob } from '../lib/ai/index.js';
+import { dispatchCommand, dispatchPreAuthCommand } from '../lib/channels/commands/index.js';
+import { getByChannelChatId, setActiveThread } from '../lib/db/user-channels.js';
+import { chat, chatStream, summarizeAgentJob } from '../lib/ai/index.js';
 import { createNotification } from '../lib/db/notifications.js';
 import { loadTriggers } from '../lib/triggers.js';
 import { verifyApiKey } from '../lib/db/api-keys.js';
@@ -209,29 +211,65 @@ async function handleTelegramWebhook(request) {
 }
 
 /**
- * Process a normalized message through the AI layer with channel UX.
- * Message persistence is handled centrally by the AI layer.
+ * Resolve the incoming channel message to a user, dispatch any slash command,
+ * and otherwise stream the message through the AI layer using the user's
+ * active session.
  */
 async function processChannelMessage(adapter, normalized) {
-  await adapter.acknowledge(normalized.metadata);
-  const stopIndicator = adapter.startProcessingIndicator(normalized.metadata);
+  const { channel, channelChatId, metadata } = normalized;
+  const binding = getByChannelChatId(channel, channelChatId);
+
+  // Unbound chat → only /verify is accepted; everything else is silently ignored.
+  if (!binding || !binding.verifiedAt) {
+    const result = await dispatchPreAuthCommand(normalized, { channel, channelChatId });
+    if (result?.handled) {
+      await adapter.sendResponse(channelChatId, result.reply, metadata);
+    }
+    return;
+  }
+
+  const ctx = { channel, channelChatId, userId: binding.userId };
+
+  // Post-auth slash commands short-circuit the AI path.
+  const cmd = await dispatchCommand(normalized, ctx);
+  if (cmd?.handled) {
+    await adapter.sendResponse(channelChatId, cmd.reply, metadata);
+    return;
+  }
+
+  await adapter.acknowledge(metadata);
+  const stopIndicator = adapter.startProcessingIndicator(metadata);
 
   try {
-    const response = await chat(
-      normalized.threadId,
-      normalized.text,
-      normalized.attachments,
-      { userId: 'telegram', chatTitle: 'Telegram' }
-    );
-    await adapter.sendResponse(normalized.threadId, response, normalized.metadata);
+    let threadId = binding.activeThreadId;
+    if (!threadId) {
+      threadId = randomUUID();
+      setActiveThread(binding.userId, channel, threadId);
+    }
+
+    const envRepo = process.env.GH_OWNER && process.env.GH_REPO
+      ? `${process.env.GH_OWNER}/${process.env.GH_REPO}`
+      : '';
+    const streamOptions = {
+      userId: binding.userId,
+      chatTitle: 'Telegram',
+      repo: envRepo,
+      branch: 'main',
+      codeMode: false,
+      codeModeType: 'code',
+    };
+
+    if (adapter.streamChatResponse) {
+      const chunks = chatStream(threadId, normalized.text, normalized.attachments, streamOptions);
+      await adapter.streamChatResponse(channelChatId, chunks);
+    } else {
+      const response = await chat(threadId, normalized.text, normalized.attachments, streamOptions);
+      await adapter.sendResponse(channelChatId, response, metadata);
+    }
   } catch (err) {
     console.error('Failed to process message with AI:', err);
     await adapter
-      .sendResponse(
-        normalized.threadId,
-        'Sorry, I encountered an error processing your message.',
-        normalized.metadata
-      )
+      .sendResponse(channelChatId, 'Sorry, I encountered an error processing your message.', metadata)
       .catch(() => {});
   } finally {
     stopIndicator();

package/bin/CLAUDE.md

@@ -8,20 +8,23 @@ Entry point: `cli.js` (invoked via `npx thepopebot <command>`).
 |---------|---------|
 | `init [--no-managed] [--no-install]` | Scaffold project from templates, sync managed files, create `.env`, install deps |
 | `setup` | Run interactive setup wizard (see `setup/CLAUDE.md`) |
-| `setup-telegram` | Reconfigure Telegram webhook |
+| `setup-ssl` | Configure SSL with Let's Encrypt wildcard cert |
 | `upgrade [@beta\|version]` | Upgrade package, run init, rebuild, commit, push, restart Docker |
 | `reset [file]` | Restore a template file to defaults |
+| `reset-all` | Nuclear reset — restore entire project to fresh init state |
+| `audit` | Show project state vs. package templates (modified / missing / unknown) |
 | `diff [file]` | Show diff between user file and package template |
 | `reset-auth` | Regenerate `AUTH_SECRET` (invalidates all sessions) |
-| `set-var <KEY> [VALUE]` | Set GitHub repository variable |
+| `set-var <KEY> [VALUE]` | Set GitHub repository variable (also reads piped stdin) |
 | `user:password <email>` | Change user password |
-| `sync <path>` | Dev helper — sync local package to test install |
+| `sync <path>` | Dev helper — pack, build, upload local package to test install |
+| `sync --fast <path>` | Fast variant — copy source into the running container and rebuild `.next` only |
 
 ## Managed Paths System
 
 `managed-paths.js` defines files auto-synced by `init`. These are overwritten on every init/upgrade — users should not edit them.
 
-**Managed paths**: `.github/workflows/`, `docker-compose.yml`, `.dockerignore`, `.gitignore`, `agent-job/CLAUDE.md`, `event-handler/CLAUDE.md`, `skills/CLAUDE.md`.
+**Managed paths**: `.github/workflows/`, `docker-compose.yml`, `.dockerignore`, `.gitignore`.
 
 `isManaged(relPath)` — returns true if a path is managed (exact match or directory prefix).
 

package/bin/cli.js

@@ -53,7 +53,6 @@ Commands:
   upgrade|update [@beta|version]    Upgrade thepopebot (install, init, build, commit, push)
   setup                             Run interactive setup wizard
   setup-ssl                         Configure SSL with Let's Encrypt wildcard cert
-  setup-telegram                    Reconfigure Telegram webhook
   reset-auth                        Regenerate AUTH_SECRET (invalidates all sessions)
   reset [file]                      Restore a template file (or list available templates)
   reset-all                         Nuclear reset — restore entire project to fresh init state
@@ -86,12 +85,12 @@ function getTemplateFiles(templatesDir) {
   return files;
 }
 
-async function init() {
+async function init(options = {}) {
   let cwd = process.cwd();
   const packageDir = path.join(__dirname, '..');
   const templatesDir = path.join(packageDir, 'templates');
-  const noManaged = args.includes('--no-managed');
-  const noInstall = args.includes('--no-install');
+  const noManaged = options.noManaged ?? args.includes('--no-managed');
+  const noInstall = options.noInstall ?? args.includes('--no-install');
 
   // Guard: warn if the directory is not empty (unless it's an existing thepopebot project)
   const entries = fs.readdirSync(cwd);
@@ -256,7 +255,6 @@ async function init() {
       type: 'module',
       scripts: {
         setup: 'thepopebot setup',
-        'setup-telegram': 'thepopebot setup-telegram',
         'reset-auth': 'thepopebot reset-auth',
       },
       dependencies: {
@@ -277,33 +275,23 @@ async function init() {
     }
   }
 
-  // Create default skill activation symlinks
-  const defaultSkills = [];
-  const activeDir = path.join(cwd, 'skills', 'active');
-  fs.mkdirSync(activeDir, { recursive: true });
-  for (const skill of defaultSkills) {
-    const symlink = path.join(activeDir, skill);
-    if (!fs.existsSync(symlink)) {
-      createDirLink(`../library/${skill}`, symlink);
-      console.log(`  Created skills/active/${skill} → ../library/${skill}`);
+  // Create agent skill bridge symlinks (point to ../skills)
+  const skillBridges = [
+    { dir: '.pi', name: 'Pi' },
+    { dir: '.claude', name: 'Claude' },
+    { dir: '.codex', name: 'Codex' },
+    { dir: '.gemini', name: 'Gemini' },
+    { dir: '.kimi', name: 'Kimi' },
+  ];
+  for (const { dir, name } of skillBridges) {
+    const link = path.join(cwd, dir, 'skills');
+    if (!fs.existsSync(link)) {
+      fs.mkdirSync(path.dirname(link), { recursive: true });
+      createDirLink('../skills', link);
+      console.log(`  Created ${dir}/skills → ../skills`);
     }
   }
 
-  // Create .pi/skills → ../skills/active symlink
-  const piSkillsLink = path.join(cwd, '.pi', 'skills');
-  if (!fs.existsSync(piSkillsLink)) {
-    fs.mkdirSync(path.dirname(piSkillsLink), { recursive: true });
-    createDirLink('../skills/active', piSkillsLink);
-    console.log('  Created .pi/skills → ../skills/active');
-  }
-
-  // Create .claude/skills → ../skills/active symlink
-  const claudeSkillsLink = path.join(cwd, '.claude', 'skills');
-  if (!fs.existsSync(claudeSkillsLink)) {
-    fs.mkdirSync(path.dirname(claudeSkillsLink), { recursive: true });
-    createDirLink('../skills/active', claudeSkillsLink);
-    console.log('  Created .claude/skills → ../skills/active');
-  }
 
   // Report backed-up files
   if (backedUp.length > 0) {
@@ -420,7 +408,7 @@ function reset(filePath) {
       console.log(`  ${destPath(file)}`);
     }
     console.log('\nUsage: thepopebot reset <file>');
-    console.log('Example: thepopebot reset agent-job/SOUL.md\n');
+    console.log('Example: thepopebot reset agent-job/SYSTEM.md\n');
     return;
   }
 
@@ -502,7 +490,7 @@ function diff(filePath) {
       console.log('  All files match package templates.');
     }
     console.log('\nUsage: thepopebot diff <file>');
-    console.log('Example: thepopebot diff agent-job/SOUL.md\n');
+    console.log('Example: thepopebot diff agent-job/SYSTEM.md\n');
     return;
   }
 
@@ -656,8 +644,12 @@ const PROTECTED_PATHS = [
   'package.json',
   'docker-compose.custom.yml',
   '.claude/',
+  '.codex/',
+  '.gemini/',
+  '.kimi/',
   '.pi/',
   'skills/',
+  'agents/',
   'node_modules/',
 ];
 
@@ -749,10 +741,10 @@ async function resetAll() {
 
   console.log(`\n  Moved ${filesToMove.length} file(s) to .backups/${ts}/`);
 
-  // Run init to rebuild from templates
+  // Run init to rebuild from templates (call directly to use the same package version)
   console.log('\n  Running init to rebuild project...\n');
   try {
-    execSync('npx thepopebot init --no-install', { stdio: 'inherit', cwd });
+    await init({ noInstall: true });
   } catch {
     console.error('\n  Init failed. Your backup is at .backups/' + ts + '/\n');
     process.exit(1);
@@ -789,15 +781,6 @@ function setupSsl() {
   }
 }
 
-function setupTelegram() {
-  const setupScript = path.join(__dirname, '..', 'setup', 'setup-telegram.mjs');
-  try {
-    execFileSync(process.execPath, [setupScript], { stdio: 'inherit', cwd: process.cwd() });
-  } catch {
-    process.exit(1);
-  }
-}
-
 async function resetAuth() {
   const { randomBytes } = await import('crypto');
   const { updateEnvVariable } = await import(path.join(__dirname, '..', 'setup', 'lib', 'auth.mjs'));
@@ -1072,9 +1055,6 @@ switch (command) {
   case 'setup-ssl':
     setupSsl();
     break;
-  case 'setup-telegram':
-    setupTelegram();
-    break;
   case 'reset-auth':
     await resetAuth();
     break;

package/config/CLAUDE.md

@@ -1,9 +1,28 @@
-# config/ — Next.js Config Wrapper
+# config/ — Next.js Config + Server Bootstrap
 
 ## Next.js Config Wrapper (index.js)
 
-`withThepopebot()` wraps user's `next.config.mjs`. Adds `transpilePackages` and `serverExternalPackages` for the npm package's dependencies that need special bundling.
+`withThepopebot()` wraps the user's `next.config.mjs`. Adds `transpilePackages` and `serverExternalPackages` for npm-package dependencies that need special bundling (Drizzle, better-sqlite3, etc.).
 
-## Instrumentation (instrumentation.js)
+## Instrumentation Hook (instrumentation.js)
 
-Server startup hook loaded by Next.js. Sequence: loads `.env`, initializes database, starts cron scheduler, starts cluster runtime. Skipped during `next build` (checks `NEXT_PHASE`).
+Loaded by Next.js once on server start. The user's project re-exports it from their own `instrumentation.js`:
+
+```js
+export { register } from 'thepopebot/instrumentation';
+```
+
+### Boot sequence
+
+1. **Skip during `next build`** — checks `process.argv` for `'build'` to avoid keeping the event loop alive during build output.
+2. **Load `.env`** — `dotenv.config()` from project root.
+3. **Default `AUTH_URL` from `APP_URL`** — so NextAuth redirects to the correct host on sign-out.
+4. **Validate `AUTH_SECRET`** — throws if unset (required for session encryption).
+5. **`initDatabase()`** — `lib/db/index.js`. Opens SQLite, runs Drizzle migrations.
+6. **`migrateEnvToDb()`** — `lib/db/config.js`. Idempotent first-run migration of `.env` values into the settings table.
+7. **`loadCrons()`** — `lib/cron.js`. Reads `agent-job/CRONS.json`, schedules user-defined crons.
+8. **`startBuiltinCrons()`** — `lib/cron.js`. Starts internal crons (e.g., npm version check). Then warms the in-memory update flag from `lib/db/update-check.js`.
+9. **`startClusterRuntime()`** — `lib/cluster/runtime.js`. Registers cluster role triggers (cron + file watch + webhook).
+10. **`startMaintenanceCron()`** — `lib/maintenance.js`. Hourly cleanup of expired agent-job API keys and other housekeeping.
+
+`initialized` is module-scoped so the sequence runs exactly once even if `register()` is called more than once.

package/drizzle/0021_coding_agent_workspace.sql

@@ -0,0 +1 @@
+ALTER TABLE `code_workspaces` ADD `coding_agent` text;

package/drizzle/0022_organic_apocalypse.sql

@@ -0,0 +1,16 @@
+CREATE TABLE `user_channels` (
+	`id` text PRIMARY KEY NOT NULL,
+	`user_id` text NOT NULL,
+	`channel` text NOT NULL,
+	`channel_chat_id` text,
+	`code` text,
+	`code_expires_at` integer,
+	`verified_at` integer,
+	`active_thread_id` text,
+	`created_at` integer NOT NULL,
+	`updated_at` integer NOT NULL
+);
+--> statement-breakpoint
+CREATE UNIQUE INDEX `user_channels_user_channel_unique` ON `user_channels` (`user_id`,`channel`);--> statement-breakpoint
+CREATE UNIQUE INDEX `user_channels_channel_chat_id_unique` ON `user_channels` (`channel`,`channel_chat_id`);--> statement-breakpoint
+CREATE INDEX `user_channels_code_lookup` ON `user_channels` (`code`);

package/drizzle/0023_needy_ender_wiggin.sql

@@ -0,0 +1 @@
+ALTER TABLE `code_workspaces` ADD `scope` text;