social-autoposter 1.6.26 → 1.6.28

package/README.md

@@ -4,7 +4,7 @@ Open-source repo behind **[S4L (s4lai)](https://s4l.ai)**: an automated social p
 
 > The hosted managed version is **S4L** (written `s4lai`, domain `s4l.ai`): done-for-you Reddit and Twitter brand-awareness, $1/1K impressions, $50/1K site visits. See https://s4l.ai.
 
-Posts are written to a Postgres database via `DATABASE_URL` in `~/social-autoposter/.env`. Bring your own Postgres DB and apply `schema-postgres.sql` once. Each platform drives its own persistent Playwright MCP browser profile, so logins survive across runs.
+State (posts, replies, candidates, stats) is read and written through the hosted S4L HTTP API (`AUTOPOSTER_API_BASE` + an install key in `~/social-autoposter/.env`); no database to provision. Each platform drives its own persistent Playwright MCP browser profile, so logins survive across runs.
 
 ## Prerequisites
 
@@ -12,9 +12,8 @@ A new machine needs all of these before the pipeline can run end to end:
 
 - **macOS** (the launchd plists are mac-only; Linux users can crib the cron snippets from `setup/SKILL.md` Step 7)
 - **Node.js 16+** (for `npx`, the installer, and `@playwright/mcp` at runtime)
-- **Python 3.9+** with `pip3` (helper scripts; `psycopg2-binary` is auto-installed by the installer)
+- **Python 3.9+** with `pip3` (helper scripts; deps auto-installed by the installer)
 - **Claude Code CLI** on `PATH` (the cron scripts shell out to `claude -p` with a per-platform MCP config)
-- **`psql`** on `PATH` (a few scripts query Postgres directly)
 - One Chromium install per platform (created on first run by `@playwright/mcp` against the persistent profile dirs)
 
 Optional:
@@ -30,9 +29,9 @@ npx social-autoposter init
 
 `bin/cli.js` does all of the wiring in one shot:
 
-1. Copies `scripts/`, `skill/`, `setup/`, `SKILL.md`, `schema-postgres.sql`, and `browser-agent-configs/` into `~/social-autoposter/`
-2. Creates `config.json` from `config.example.json` and writes a blank `.env` template (fill in your own `DATABASE_URL` and optional `MOLTBOOK_API_KEY`)
-3. Installs `psycopg2-binary` via `pip3` if missing
+1. Copies `scripts/`, `skill/`, `setup/`, `SKILL.md`, and `browser-agent-configs/` into `~/social-autoposter/`
+2. Creates `config.json` from `config.example.json` and writes a blank `.env` template (fill in your S4L API key and optional `MOLTBOOK_API_KEY`)
+3. Installs the Python helper deps via `pip3` if missing
 4. Generates launchd plists in `~/social-autoposter/launchd/` with the user's actual `HOME` and `PATH`
 5. Installs the Playwright MCP configs to `~/.claude/browser-agent-configs/` (twitter, reddit, linkedin) with `__HOME__` and `__NODE_BIN__` placeholders substituted. Existing files are left alone, so any window-position tweaks survive `npx social-autoposter update`.
 6. Creates empty persistent browser profile dirs at `~/.claude/browser-profiles/{twitter,reddit,linkedin}`
@@ -48,7 +47,7 @@ npx social-autoposter update
 
 Tell your Claude Code agent: **"set up social autoposter"**. The interactive wizard in `setup/SKILL.md` walks through:
 
-1. Verifying the Postgres connection
+1. Verifying the S4L API connection
 2. Filling in `~/social-autoposter/config.json` with handles for Reddit, Twitter, LinkedIn, optional Moltbook
 3. A 5-question interview to draft your `content_angle`
 4. Capturing `projects` with `topics` (used by the tiered reply strategy)
@@ -65,10 +64,10 @@ launchd  ──▶  skill/run-{platform}.sh  ──▶  claude -p  --strict-mcp-
                        │                                                       │
                        │                                                       └──▶  ~/.claude/browser-profiles/{platform}/  (persistent userDataDir)
                        │
-                       ├──▶  scripts/find_threads.py, top_twitter_queries.py  (no browser, API + DB dedup)
+                       ├──▶  scripts/find_threads.py, top_twitter_queries.py  (no browser, API dedup)
                        ├──▶  scripts/pick_project.py            (weighted project rotation)
                        ├──▶  scripts/top_performers.py          (feedback report from past stats)
-                       └──▶  Postgres                           (DATABASE_URL in .env)
+                       └──▶  S4L HTTP API                       (AUTOPOSTER_API_BASE in .env)
 ```
 
 Each `skill/run-*.sh`:
@@ -104,7 +103,7 @@ launchctl load ~/Library/LaunchAgents/com.m13v.social-twitter-cycle.plist
 | `/social-autoposter engage` | Scan and reply to responses on our posts |
 | `/social-autoposter audit` | Full browser audit of all posts |
 
-View live stats at `https://s4l.ai/stats/<your-handle>` once posts start landing in Postgres.
+View live stats at `https://s4l.ai/stats/<your-handle>` once posts start landing.
 
 ## Repo layout
 
@@ -114,7 +113,6 @@ social-autoposter/
 ├── bin/cli.js                installer + dashboard launcher
 ├── browser-agent-configs/    Playwright MCP templates (twitter/reddit/linkedin)
 ├── config.example.json       config template
-├── schema-postgres.sql       Postgres schema
 ├── setup/SKILL.md            interactive setup wizard skill (locked)
 ├── scripts/                  Python and JS helpers (no browser, no LLM)
 ├── skill/                    shell wrappers invoked by launchd

package/bin/cli.js

@@ -16,7 +16,6 @@ const HOME = os.homedir();
 // Files/dirs to copy from npm package to ~/social-autoposter
 const COPY_TARGETS = [
   'scripts',
-  'schema-postgres.sql',
   'config.example.json',
   'requirements.txt',
   'SKILL.md',
@@ -39,11 +38,6 @@ MOLTBOOK_API_KEY=
 # AUTOPOSTER_API_KEY only if your install uses a bearer token.
 # AUTOPOSTER_API_BASE=https://s4l.ai
 # AUTOPOSTER_API_KEY=
-
-# Optional. Only the local dashboard (bin/server.js) still reads Postgres
-# directly; the posting pipelines do not. Leave blank unless you run the
-# dashboard. Format: postgresql://<user>:<password>@<host>/<db>?sslmode=require
-# DATABASE_URL=
 `;
 
 // Never overwrite these user files during update
@@ -128,12 +122,36 @@ function pipSupportsBreakSystemPackages(pythonBin) {
   return parseInt(m[1], 10) >= 23;
 }
 
+// True if the interpreter carries a PEP 668 EXTERNALLY-MANAGED marker in its
+// stdlib dir (Homebrew python, Debian/Ubuntu 23+). On these, a bare
+// `pip install` is GUARANTEED to fail with a loud "externally-managed-environment"
+// wall of text. Detecting it up front lets pipInstall skip that doomed first
+// attempt and go straight to --break-system-packages, so init output stays clean
+// and doesn't falsely look like a failed dependency install when it recovers.
+function pipIsExternallyManaged(pythonBin) {
+  const r = spawnSync(pythonBin, ['-c',
+    "import os,sys,sysconfig\n" +
+    "p=os.path.join(sysconfig.get_path('stdlib'),'EXTERNALLY-MANAGED')\n" +
+    "sys.exit(0 if os.path.exists(p) else 1)",
+  ]);
+  return r.status === 0;
+}
+
 // Install Python packages into a specific interpreter via `<py> -m pip install`.
-// Retries with --break-system-packages only when the resolved pip supports it
-// (PEP 668 environments: Homebrew python, Debian/Ubuntu 23+). Returns the
-// spawnSync result of the last attempt.
+// Behaviour by environment:
+//   - PEP 668 externally-managed interpreter (Homebrew python, Debian/Ubuntu 23+)
+//     with pip>=23: go STRAIGHT to --break-system-packages. The bare attempt
+//     would always fail loudly with externally-managed-environment, which made
+//     init look like "Python deps failed" even though the (silent) retry actually
+//     installed everything. No doomed first attempt, no false-alarm output.
+//   - Everything else: bare attempt, then retry with --break-system-packages only
+//     if it failed and pip supports the flag.
+// Returns the spawnSync result of the last attempt.
 function pipInstall(pythonBin, args) {
   const base = ['-m', 'pip', 'install', ...args];
+  if (pipIsExternallyManaged(pythonBin) && pipSupportsBreakSystemPackages(pythonBin)) {
+    return spawnSync(pythonBin, [...base, '--break-system-packages'], { stdio: 'inherit' });
+  }
   let r = spawnSync(pythonBin, base, { stdio: 'inherit' });
   if (r.status !== 0 && pipSupportsBreakSystemPackages(pythonBin)) {
     r = spawnSync(pythonBin, [...base, '--break-system-packages'], { stdio: 'inherit' });
@@ -429,7 +447,7 @@ function bootstrapVm() {
   // Install Python deps from requirements.txt. installBrowserHarness only
   // installs uv + mcp; it does NOT read requirements.txt, so without this the
   // VM is missing websocket-client (restore_twitter_session.py aborts on
-  // import) plus psycopg2-binary/playwright that the cycle scripts need.
+  // import) plus playwright that the cycle scripts need.
   installPythonDeps();
 
   // Restore the Twitter login if we have stored cookies and the Chrome is
@@ -530,6 +548,26 @@ function installBrowserHarness() {
     if (fs.existsSync(harnessBin)) {
       spawnSync(harnessBin, ['--reload'], { stdio: 'inherit' });
     }
+
+    // Contract check: server.py invokes `browser-harness -c <script>`. If an
+    // offline/rate-limited fetch left a stale checkout that predates the `-c`
+    // interface, the CLI still "installs" fine but every bh_run returns the
+    // usage banner and CDP looks "not connected". Verify the installed binary
+    // actually speaks `-c` (its no-arg usage string mentions it) and fail
+    // LOUDLY here instead of shipping a silently-broken twitter-harness.
+    if (fs.existsSync(harnessBin)) {
+      const probe = spawnSync(harnessBin, [], { stdio: 'pipe', encoding: 'utf8', timeout: 15000 });
+      const usage = `${probe.stdout || ''}${probe.stderr || ''}`;
+      if (!/\b-c\b/.test(usage)) {
+        console.error('    ERROR: installed browser-harness CLI does not accept `-c` (stale clone).');
+        console.error('    The twitter-harness MCP will return a usage banner / "CDP not connected".');
+        console.error(`    Fix: rm -rf ${harnessDir} && re-run \`social-autoposter init\` while online,`);
+        console.error('    or manually: git clone https://github.com/browser-use/browser-harness ' + harnessDir +
+          ' && ' + uvBin + ' tool install --force -e ' + harnessDir);
+      } else {
+        console.log('    browser-harness CLI verified (accepts -c).');
+      }
+    }
   }
 
   // Step 4: ensure mcp Python package available (server.py uses `from mcp.server.fastmcp ...`).
@@ -885,14 +923,14 @@ function installPythonDeps() {
   const reqPath = path.join(PKG_ROOT, 'requirements.txt');
   const args = fs.existsSync(reqPath)
     ? ['-r', reqPath, '-q']
-    : ['-q', 'psycopg2-binary', 'playwright'];
+    : ['-q', 'playwright'];
   // Install into the SAME interpreter the MCP server runs (SAPS_PYTHON =
   // Homebrew python), NOT bare pip3 which on macOS targets the Xcode CLT system
   // python — deps installed there are invisible to the scripts at runtime.
   // pipInstall() also gates --break-system-packages on pip>=23 so it doesn't
   // hard-fail against the ancient system pip.
   const pythonBin = findPythonBin();
-  console.log(`  installing Python deps (psycopg2-binary, playwright, ...) into ${pythonBin}`);
+  console.log(`  installing Python deps (playwright, ...) into ${pythonBin}`);
   const r = pipInstall(pythonBin, args);
   if (r.status !== 0) {
     console.warn('  WARNING: pip install failed — run manually:');
@@ -995,7 +1033,17 @@ if (cmd === 'init') {
   process.argv = [process.argv[0], process.argv[1], 'import', ...process.argv.slice(3)];
   require('./cookie-helper.js');
 } else if (!cmd) {
-  require('./server.js');
+  // The dashboard server (bin/server.js) is a local-only operator tool and is
+  // NOT shipped in the published package (it talks directly to Postgres). When
+  // it's absent, fall back to usage help instead of crashing on a missing require.
+  if (fs.existsSync(path.join(__dirname, 'server.js'))) {
+    require('./server.js');
+  } else {
+    console.log('social-autoposter — automated social posting for Claude agents');
+    console.log('');
+    console.log('The local dashboard is not part of the published package.');
+    console.log('Run `npx social-autoposter init` to set up, then drive it from your Claude agent.');
+  }
 } else {
   console.log('social-autoposter — automated social posting for Claude agents');
   console.log('');

package/mcp/dist/repo.js

@@ -10,7 +10,7 @@ const __dirname = path.dirname(__filename);
 // dist/repo.js -> repo root is two levels up (mcp/dist -> mcp -> repo root).
 // Override with SAPS_REPO_DIR for non-standard installs.
 export const REPO_DIR = process.env.SAPS_REPO_DIR || path.resolve(__dirname, "..", "..");
-// Python used by the pipeline (psycopg2 etc). Override per-install.
+// Python used by the pipeline (deps from requirements.txt). Override per-install.
 export const PYTHON = process.env.SAPS_PYTHON || "python3";
 // The locked pipeline script (run-twitter-cycle.sh) writes the draft plan to a
 // HARDCODED /tmp path (`PLAN_FILE="/tmp/twitter_cycle_plan_<batch>.json"`), and the
@@ -21,7 +21,7 @@ export const PYTHON = process.env.SAPS_PYTHON || "python3";
 // "No drafts in batch ...". Default to /tmp to match the script; allow an explicit
 // override for non-standard installs.
 export const TMP_DIR = process.env.SAPS_TMP_DIR || "/tmp";
-// Spawn a process inside the repo, inheriting the repo env (DATABASE_URL etc
+// Spawn a process inside the repo, inheriting the repo env (API base + keys
 // come from the install's environment / .env loaded by the scripts themselves).
 export function run(cmd, args, opts = {}) {
     return new Promise((resolve) => {

package/mcp/install.mjs

@@ -34,7 +34,7 @@ const nodeBin = whichNode();
 function whichPython() {
   // Prefer a real Homebrew python (has the pipeline's deps) over the macOS
   // /usr/bin/python3 stub, which can trigger an Xcode CLT install prompt and
-  // often lacks psycopg2 etc.
+  // often lacks the pipeline deps.
   const candidates = ["/opt/homebrew/bin/python3", "/usr/local/bin/python3"];
   try {
     const found = execSync("command -v python3 2>/dev/null", { shell: "/bin/bash" })

package/mcp/manifest.json

@@ -51,7 +51,7 @@
     "saps_python": {
       "type": "string",
       "title": "Python interpreter",
-      "description": "Python used by the pipeline (must have psycopg2 etc). Leave as default unless you know you need a different one.",
+      "description": "Python used by the pipeline (needs the deps in requirements.txt). Leave as default unless you know you need a different one.",
       "required": false,
       "default": "/opt/homebrew/bin/python3"
     }

package/mcp-servers/browser-harness/server.py

@@ -246,8 +246,13 @@ def ensure_chrome() -> dict:
     PID_FILE.write_text(str(proc.pid))
     _log(f"launched Chrome pid={proc.pid} port={PORT} profile={PROFILE_DIR} headless={HEADLESS}")
 
-    # Wait for CDP to be ready.
-    deadline = time.time() + 15
+    # Wait for CDP to be ready. First launch on a cold/fresh machine has to
+    # create the profile and run Chrome's first-run setup, which routinely
+    # exceeds 15s on a slow VM; an over-tight deadline returns launch_timeout
+    # and the caller runs against a port that was about to come up. 30s is the
+    # safe floor (override with BH_LAUNCH_TIMEOUT_SEC).
+    launch_timeout = int(os.environ.get("BH_LAUNCH_TIMEOUT_SEC", "30"))
+    deadline = time.time() + launch_timeout
     while time.time() < deadline:
         if _cdp_alive():
             return {"status": "started", "pid": proc.pid, "cdp": CDP_URL}
@@ -258,6 +263,8 @@ def ensure_chrome() -> dict:
         "pid": proc.pid,
         "cdp": CDP_URL,
         "log": str(LOG_FILE),
+        "waited_sec": launch_timeout,
+        "log_tail": _chrome_log_tail(),
     }
 
 
@@ -331,6 +338,56 @@ def stop_chrome() -> dict:
 
 # --- browser-harness exec wrapper ---
 
+def _chrome_log_tail(lines: int = 25) -> str:
+    """Last `lines` of the managed-Chrome log, for surfacing in CDP errors."""
+    try:
+        text = LOG_FILE.read_text(errors="replace")
+    except (FileNotFoundError, OSError):
+        return ""
+    return "\n".join(text.splitlines()[-lines:])
+
+
+def _ensure_cdp_ready() -> dict | None:
+    """Guarantee CDP is actually answering on PORT before we shell out to the
+    harness CLI. Returns None when CDP is live; otherwise returns a structured,
+    actionable error dict (and leaves the chrome log tail attached).
+
+    Without this gate, ensure_chrome() failures (no_chrome_binary,
+    launch_timeout) were swallowed and the CLI ran against a dead port, so the
+    agent saw a cryptic usage banner / connection error instead of the real
+    cause. This is the #1 fresh-install failure mode."""
+    res = ensure_chrome()
+    if _cdp_alive():
+        return None
+
+    # One self-heal attempt: a stale Chrome bound to the port but not speaking
+    # CDP (crashed renderer, half-dead profile) won't recover on its own.
+    _log(f"CDP not alive after ensure_chrome (status={res.get('status')}); attempting stop+relaunch")
+    stop_chrome()
+    time.sleep(1.0)
+    res = ensure_chrome()
+    if _cdp_alive():
+        return None
+
+    status = res.get("status", "unknown")
+    if status == "no_chrome_binary":
+        hint = res.get("hint", "Install Chrome/Chromium or set BH_CHROME_BIN.")
+    else:
+        hint = (
+            f"Chrome did not expose CDP on {CDP_URL} (status={status}). "
+            "On a headless Linux box ensure BH_HEADLESS=1 and a Chrome binary "
+            "are present; on macOS make sure no other Chrome owns the profile. "
+            f"See {LOG_FILE}."
+        )
+    return {
+        "ok": False,
+        "error": f"browser-harness CDP not connected: {hint}",
+        "cdp": CDP_URL,
+        "ensure_chrome": res,
+        "chrome_log_tail": _chrome_log_tail(),
+    }
+
+
 def _run_harness(script: str, timeout: int = EXEC_TIMEOUT_SEC) -> dict:
     if not shutil.which(BROWSER_HARNESS_BIN) and not Path(BROWSER_HARNESS_BIN).exists():
         return {
@@ -341,7 +398,9 @@ def _run_harness(script: str, timeout: int = EXEC_TIMEOUT_SEC) -> dict:
             ),
         }
 
-    ensure_chrome()
+    cdp_err = _ensure_cdp_ready()
+    if cdp_err is not None:
+        return cdp_err
 
     env = os.environ.copy()
     env["BU_CDP_URL"] = CDP_URL

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "social-autoposter",
-  "version": "1.6.26",
+  "version": "1.6.28",
   "description": "Automated social posting pipeline for Reddit, X/Twitter, LinkedIn, and Moltbook. Install as a Claude Code agent skill.",
   "bin": {
     "social-autoposter": "bin/cli.js"
@@ -12,7 +12,13 @@
   },
   "files": [
     "bin/",
+    "!bin/server.js",
+    "!bin/auth.js",
     "scripts/*.py",
+    "!scripts/db_direct.py",
+    "!scripts/_dm_record_sent.sh",
+    "!scripts/send_batch_dms.sh",
+    "!scripts/mint_podlog_subpage_*.py",
     "!scripts/tmp_*.py",
     "!scripts/insert_post*.py",
     "!scripts/_insert_post_*.py",
@@ -34,7 +40,6 @@
     "!scripts/seed_dashboard_users.py",
     "!scripts/send_dashboard_invite.py",
     "scripts/*.sh",
-    "schema-postgres.sql",
     "config.example.json",
     "requirements.txt",
     "SKILL.md",
@@ -69,8 +74,10 @@
     "node": ">=16"
   },
   "dependencies": {
-    "firebase-admin": "^13.8.0",
-    "pg": "^8.20.0",
     "ws": "^8.0.0"
+  },
+  "devDependencies": {
+    "firebase-admin": "^13.8.0",
+    "pg": "^8.20.0"
   }
 }

package/requirements.txt

@@ -4,7 +4,6 @@
 # Playwright browser binaries are downloaded separately (same interpreter:
 # `<saps_python> -m playwright install chromium`).
 
-psycopg2-binary
 playwright
 # CDP client used by scripts/restore_twitter_session.py and the cookie-grab/
 # CDP-driven helpers. Required on AppMaker VMs for the post-substitution

package/scripts/db.py

@@ -1,15 +1,20 @@
 #!/usr/bin/env python3
-"""Shared Postgres connection for social-autoposter.
+"""Shared env loader for social-autoposter (clean, shipped).
 
-Provides a thin psycopg2 wrapper with a sqlite3-compatible API so all
-scripts can use the same SQL without changes to query logic.
+The published package talks to the central store exclusively through the S4L
+HTTP API (scripts/http_api.py). It ships NO direct database dependency: no
+psycopg2, no DATABASE_URL requirement.
 
-DATABASE_URL is read from ~/social-autoposter/.env (pre-filled on install).
+This module provides `load_env()` (the only DB-agnostic helper every pipeline
+needs) and, for LOCAL operator installs only, re-exports the direct-Postgres
+connection layer from `db_direct.py` when that file is present. `db_direct.py`
+is excluded from the npm tarball, so on a clean install the direct-DB symbols
+resolve to a hard-error stub instead of importing psycopg2.
+
+.env is read from ~/social-autoposter/.env (pre-filled on install).
 """
 
 import os
-import re
-import sys
 
 ENV_PATH = os.path.expanduser("~/social-autoposter/.env")
 
@@ -24,181 +29,29 @@ def load_env():
                     os.environ.setdefault(k.strip(), v.strip())
 
 
-def _translate_sql(sql):
-    """Translate SQLite-specific SQL syntax to PostgreSQL."""
-    # ? placeholders -> %s
-    sql = sql.replace('?', '%s')
-    # datetime('now', '-N hours') -> NOW() - INTERVAL 'N hours'
-    sql = re.sub(r"datetime\('now',\s*'-(\d+) hours'\)", r"NOW() - INTERVAL '\1 hours'", sql)
-    # datetime('now', '-N days') -> NOW() - INTERVAL 'N days'
-    sql = re.sub(r"datetime\('now',\s*'-(\d+) days'\)", r"NOW() - INTERVAL '\1 days'", sql)
-    # datetime('now') -> NOW()
-    sql = re.sub(r"datetime\('now'\)", 'NOW()', sql)
-    # status_checked_at=datetime('now') already handled above
-    return sql
-
-
-_DNS_TRANSIENT_MARKERS = (
-    "could not translate host name",
-    "nodename nor servname provided",
-    "Temporary failure in name resolution",
-    "Name or service not known",
-)
-
-
-def _connect_with_retry(url):
-    """psycopg2.connect with 3-try backoff on DNS resolution failures.
-
-    Transient DNS blips from the local resolver (observed ~40 min on
-    2026-04-23 PDT) crashed every tick that happened to run during the
-    window. Retry only on name-resolution OperationalError messages so
-    auth or hard-down errors fail fast.
-    """
-    import time
-    import psycopg2
-    backoffs = (1.0, 2.0)
-    last_exc = None
-    for attempt in range(3):
-        try:
-            return psycopg2.connect(url, keepalives=1,
-                                    keepalives_idle=30,
-                                    keepalives_interval=10,
-                                    keepalives_count=5)
-        except psycopg2.OperationalError as exc:
-            msg = str(exc)
-            if not any(m in msg for m in _DNS_TRANSIENT_MARKERS):
-                raise
-            last_exc = exc
-            if attempt < len(backoffs):
-                print(f"[db] DNS transient, retrying in {backoffs[attempt]}s: {msg.strip()}",
-                      file=sys.stderr)
-                time.sleep(backoffs[attempt])
-    raise last_exc
-
-
-class PGConn:
-    """Thin psycopg2 wrapper with a sqlite3-compatible execute/commit/close API."""
-
-    def __init__(self, conn, url=None):
-        import psycopg2.extras
-        self._conn = conn
-        self._url = url
-        self._cursor_factory = psycopg2.extras.DictCursor
-        self._apply_session_defaults()
-
-    def _apply_session_defaults(self):
-        # 5 min per-statement ceiling. Prevents silent hangs when the
-        # pooler drops an idle socket and psycopg2's recv stalls forever
-        # (originally observed 2026-04-23/24 on Neon under stats.py
-        # --github-only; kept after Cloud SQL migration as defensive). The
-        # pooler rejects libpq `options=` at connect time, so we set this via
-        # SQL after connect. Reapplied after commit() because transaction-mode
-        # pooling resets session state on COMMIT.
-        try:
-            cur = self._conn.cursor()
-            cur.execute("SET statement_timeout = 300000")
-            cur.close()
-        except Exception:
-            pass
-
-    def _reconnect(self):
-        try:
-            self._conn.close()
-        except Exception:
-            pass
-        self._conn = _connect_with_retry(self._url)
-        self._apply_session_defaults()
-
-    def execute(self, sql, params=None):
-        import psycopg2
-        sql = _translate_sql(sql)
-        try:
-            cur = self._conn.cursor(cursor_factory=self._cursor_factory)
-            if params is not None:
-                cur.execute(sql, list(params))
-            else:
-                cur.execute(sql)
-            return cur
-        except psycopg2.OperationalError:
-            self._reconnect()
-            cur = self._conn.cursor(cursor_factory=self._cursor_factory)
-            if params is not None:
-                cur.execute(sql, list(params))
-            else:
-                cur.execute(sql)
-            return cur
-
-    def commit(self):
-        import psycopg2
-        try:
-            self._conn.commit()
-        except psycopg2.OperationalError:
-            self._reconnect()
-            self._conn.commit()
-        # The transaction-mode pooler resets session state on COMMIT, so
-        # re-apply statement_timeout for the next transaction.
-        self._apply_session_defaults()
+# Re-export the direct-Postgres layer when running on a local operator install
+# (db_direct.py present). In the published package db_direct.py is absent, so
+# these names resolve to a stub that fails loudly if anything tries to open a
+# direct DB connection — by design, the shipped pipelines use the HTTP API.
+try:
+    from db_direct import (  # noqa: F401
+        get_conn,
+        PGConn,
+        snapshot_post_views,
+    )
+except ImportError:
+    def _no_direct_db(*_args, **_kwargs):
+        raise RuntimeError(
+            "Direct database access is not available in this build. "
+            "The published social-autoposter package uses the S4L HTTP API "
+            "(scripts/http_api.py); set AUTOPOSTER_API_BASE in ~/social-autoposter/.env."
+        )
 
-    def close(self):
-        self._conn.close()
+    def get_conn(*_args, **_kwargs):  # noqa: F811
+        return _no_direct_db()
 
-    # No-op to absorb sqlite3.Row assignments
-    @property
-    def row_factory(self):
+    def snapshot_post_views(*_args, **_kwargs):  # noqa: F811
+        # No-op stub: stats snapshots are an operator-local concern.
         return None
 
-    @row_factory.setter
-    def row_factory(self, val):
-        pass
-
-
-def snapshot_post_views(db, post_id, views):
-    """UPSERT one row of post_views_daily with today's view/upvote/comment count.
-
-    Called from the Reddit + Twitter refresh jobs whenever a fresh view count
-    is observed for a post. Later observations on the same day overwrite the
-    earlier ones so end-of-day has the final number. The dashboard computes
-    daily deltas with LAG() over (post_id ORDER BY day) to render gains
-    earned on day D across all posts.
-
-    Upvotes and comments are read from posts.upvotes / posts.comments_count at
-    write time. Callers UPDATE those columns on the post row immediately
-    before calling this function, so the subselects pick up the fresh values
-    without requiring each caller to pass them explicitly.
-    """
-    if post_id is None or views is None:
-        return
-    try:
-        db.execute(
-            "INSERT INTO post_views_daily (post_id, day, views, upvotes, comments, captured_at) "
-            "VALUES (%s, CURRENT_DATE, %s, "
-            "  (SELECT upvotes FROM posts WHERE id = %s), "
-            "  (SELECT comments_count FROM posts WHERE id = %s), "
-            "  NOW()) "
-            "ON CONFLICT (post_id, day) DO UPDATE SET "
-            "  views = EXCLUDED.views, "
-            "  upvotes = EXCLUDED.upvotes, "
-            "  comments = EXCLUDED.comments, "
-            "  captured_at = EXCLUDED.captured_at",
-            [post_id, int(views), post_id, post_id],
-        )
-    except Exception:
-        pass
-
-
-def get_conn():
-    """Return a PGConn connected to the central Postgres database."""
-    load_env()
-    url = os.environ.get('DATABASE_URL')
-    if not url:
-        print("ERROR: DATABASE_URL not set in ~/social-autoposter/.env", file=sys.stderr)
-        print("  Re-run: npx social-autoposter init", file=sys.stderr)
-        sys.exit(1)
-    try:
-        import psycopg2  # noqa: F401 — verifies availability before retry helper imports it
-    except ImportError:
-        print("ERROR: psycopg2-binary not installed.", file=sys.stderr)
-        print("  Run: pip3 install psycopg2-binary", file=sys.stderr)
-        sys.exit(1)
-    conn = _connect_with_retry(url)
-    return PGConn(conn, url=url)
+    PGConn = None

package/skill/dm-outreach-twitter.sh

@@ -37,10 +37,11 @@ acquire_lock "dm-outreach-twitter" 2700
 REPO_DIR="$HOME/social-autoposter"
 SKILL_FILE="$REPO_DIR/SKILL.md"
 
-if [ -z "${DATABASE_URL:-}" ]; then
-    echo "ERROR: DATABASE_URL not set in ~/social-autoposter/.env"
-    exit 1
-fi
+# 2026-06-02: removed the vestigial DATABASE_URL gate. This rail talks to the
+# central store exclusively through the S4L HTTP API (scan_dm_candidates.py,
+# dm_outreach_twitter_helper.py, log_run.py all use scripts/http_api.py). No
+# direct Postgres connection is opened here, matching dm-outreach-reddit.sh and
+# dm-outreach-linkedin.sh (migrated 2026-05-12).
 # (LOG_DIR/LOG_FILE bootstrapped at top of script.)
 
 log() { echo "[$(date +%H:%M:%S)] $*" | tee -a "$LOG_FILE"; }

package/skill/run-twitter-cycle.sh

@@ -1260,6 +1260,13 @@ if [ "$EXTRACT_EXIT" -ne 0 ] || [ ! -f "$RAW_FILE" ]; then
     # context_overflow) or just "model found nothing relevant". Classify
     # the failure for the post-loop log_run summary; the loop control below
     # decides whether to retry or give up.
+    # SCAN_OUTPUT was a stale leftover from the pre-lean design (when the scan's
+    # stdout was captured into a shell var); the lean Phase 1 loop now tees its
+    # output to $LOG_FILE instead, so an empty-scan attempt hit `set -u` and
+    # aborted the whole cycle here. Feed the classifier the recent log tail (the
+    # actual scan output, where harness/Anthropic error signatures land) so we
+    # still distinguish a real error from "found nothing relevant".
+    SCAN_OUTPUT=$(tail -n 80 "$LOG_FILE" 2>/dev/null || true)
     PHASE1_REASON_LATEST=$(echo "$SCAN_OUTPUT" | python3 "$REPO_DIR/scripts/classify_run_error.py" 2>/dev/null)
     [ -z "$PHASE1_REASON_LATEST" ] && PHASE1_REASON_LATEST="phase1_no_tweets"
     LAST_PHASE1_REASON="$PHASE1_REASON_LATEST"