prompt-exporter 2.1.0 → 2.2.0

package/docs/DEVELOPER.md

@@ -18,6 +18,7 @@ src/
     registry.ts
     chatgpt/               # chatgpt source (api/, index.ts)
     lumo/                  # lumo.proton.me (CDP Redux export)
+    perplexity/            # perplexity.ai (REST via CDP cookies)
   services/                # storage, markdown, backup (source-agnostic)
   utils/                   # paths, credentials, cdp, browser
 aur/
@@ -26,74 +27,27 @@ aur/
   publish.sh
 ```
 
-## Release (2.0.x)
+## Build output
 
-`package.json` is track `package.json` (currently 2.0.1). Shipping steps:
+`npm run build` uses **tsup** (`tsup.config.ts`) to emit a **single minified CJS** file at `dist/index.cjs` (~500 KB):
 
-### 1. Push release prep (this commit) and tag
+- All runtime dependencies are bundled (`noExternal`); Node builtins stay external
+- Version is injected as `__PACKAGE_VERSION__`
+- Shebang comes from `src/index.ts`
+- CJS (not ESM) so bundled deps that use `require()` work at runtime
 
-```bash
-cd /path/to/prompt-exporter
-git push origin master
-
-git tag -a v2.0.0 -m "prompt-exporter 2.0.0"
-git push origin v2.0.0
-```
-
-### 2. GitHub Release
-
-```bash
-# If gh is installed:
-gh release create v2.0.0 --title "2.0.0" --notes-file - <<'EOF'
-See CHANGELOG.md for breaking changes (rebrand, Apache-2.0, multi-source layout).
-EOF
-
-# Or: GitHub → Releases → Draft from tag v2.0.0 → paste CHANGELOG 2.0.0 section
-```
-
-### 3. Fill AUR stable checksum
-
-```bash
-TARBALL_URL="https://github.com/azbarcea/prompt-exporter/archive/refs/tags/v2.0.0.tar.gz"
-SUM=$(curl -sL "$TARBALL_URL" | sha256sum | awk '{print $1}')
-echo "$SUM"
-
-# Update aur/prompt-exporter/PKGBUILD sha256sums and regenerate .SRCINFO:
-sed -i "s/sha256sums=.*/sha256sums=('${SUM}')/" aur/prompt-exporter/PKGBUILD
-( cd aur/prompt-exporter && makepkg --printsrcinfo > .SRCINFO )
+AUR `package()` installs only that file to `/usr/bin/prompt-exporter` (plus docs/licenses). Do **not** copy `node_modules` into the package.
 
-git add aur/prompt-exporter
-git commit -m "chore(aur): set sha256 for prompt-exporter 2.0.0"
-git push origin master
-```
-
-### 4. Publish AUR packages
-
-Requires SSH key registered on [AUR](https://aur.archlinux.org) (`ssh aur@aur.archlinux.org`).
-
-```bash
-# First-time package names (if empty clone fails, publish.sh inits and pushes):
-./aur/publish.sh prompt-exporter      # stable 2.0.0
-./aur/publish.sh prompt-exporter-git  # rolling
-```
-
-If `prompt-exporter` or `prompt-exporter-git` does not exist yet on AUR, create them once via the AUR web UI (“Submit”) or by pushing a new empty repo with `git push -u origin master` after the script’s `git init` path — you must be logged in as the maintainer.
-
-### 5. Optional: npm registry
-
-Not required (install via `github:azbarcea/prompt-exporter` works). To publish:
-
-```bash
-npm login
-npm publish --access public
-```
+## Release
 
-### 6. Smoke after install
+Ship with the Makefile (order: **npmjs → GitHub → AUR**):
 
 ```bash
-yay -S prompt-exporter
-prompt-exporter --version   # 2.0.0
-prompt-exporter sources
+make help
+make bump VERSION=x.y.z   # optional; commits version + CHANGELOG stub
+make npm-auth             # npm login or export NPM_TOKEN=…
+make release CONFIRM=1
+# resume past npm if needed: make release CONFIRM=1 SKIP_NPM=1
 ```
 
 ## Develop

package/docs/USER_GUIDE.md

@@ -8,7 +8,7 @@
 - Re-sync incrementally as chats change
 - Hand transcripts to another AI as context
 
-The first implemented sources are **chatgpt** (chatgpt.com) and **lumo** ([lumo.proton.me](https://lumo.proton.me/)). Planned next: `claude`, `gemini`, `deepseek`, `perplexity`, `copilot` (Microsoft 365), `grok`, and `kimi`.
+The first implemented sources are **chatgpt** (chatgpt.com), **lumo** ([lumo.proton.me](https://lumo.proton.me/)), and **perplexity** ([perplexity.ai](https://www.perplexity.ai/)). Planned next: `claude`, `gemini`, `deepseek`, `copilot` (Microsoft 365), `grok`, and `kimi`.
 
 **Platforms:** Linux is supported today; Windows and macOS are in focus for CDP/browser path parity.
 
@@ -24,6 +24,16 @@ yay -S prompt-exporter
 yay -S prompt-exporter-git
 ```
 
+The AUR package installs a single bundled binary at `/usr/bin/prompt-exporter` (no `node_modules` on disk).
+
+If you previously used `npm link` / a user install under `~/.local/bin` and then removed that path, bash may still **hash** the old location:
+
+```bash
+type prompt-exporter   # may show hashed ~/.local/bin/...
+hash -r                # clear the command hash table
+type prompt-exporter   # should resolve to /usr/bin/prompt-exporter
+```
+
 ### From GitHub (npm package)
 
 ```bash
@@ -104,6 +114,19 @@ prompt-exporter list --source lumo
 
 Auth prefers a logged-in Chromium CDP session. Node Bearer alone is often blocked by Cloudflare.
 
+### Perplexity
+
+There is no official public API for thread history. Sync calls Perplexity’s internal REST endpoints (`/rest/thread/list_ask_threads`, `/rest/thread/{uuid}`) from a logged-in Chromium tab (cookies / Cloudflare):
+
+```bash
+prompt-exporter chromium start --url https://www.perplexity.ai/
+# log in; Library threads should be visible
+prompt-exporter sync --source perplexity
+prompt-exporter list --source perplexity
+```
+
+`sync` is incremental by default (skips unchanged `update_time`).
+
 ## Commands
 
 ```text

package/package.json

@@ -1,13 +1,13 @@
 {
   "name": "prompt-exporter",
-  "version": "2.1.0",
-  "description": "CLI to sync AI conversation prompts from multiple sources (ChatGPT, Lumo, …) to local JSON + Markdown",
+  "version": "2.2.0",
+  "description": "CLI to sync AI conversation prompts from multiple sources (ChatGPT, Lumo, Perplexity, …) to local JSON + Markdown",
   "license": "Apache-2.0",
   "author": "Alexandru Zbarcea <alexz@apache.org>",
   "type": "module",
-  "main": "dist/index.js",
+  "main": "dist/index.cjs",
   "bin": {
-    "prompt-exporter": "dist/index.js"
+    "prompt-exporter": "dist/index.cjs"
   },
   "files": [
     "dist",
@@ -21,8 +21,8 @@
   },
   "scripts": {
     "dev": "node --disable-warning=DEP0205 --import tsx src/index.ts",
-    "build": "tsup src/index.ts --format esm --dts --clean",
-    "start": "node dist/index.js",
+    "build": "tsup",
+    "start": "node dist/index.cjs",
     "prepublishOnly": "npm run build",
     "typecheck": "tsc --noEmit",
     "test": "npm run build && node --disable-warning=DEP0205 --import tsx --test tests/*.test.ts",

package/dist/cdp-5CBMWFGT.js

@@ -1,28 +0,0 @@
-import {
-  activateCdpTarget,
-  cdpNavigate,
-  discoverCdpPort,
-  ensureChatGptSessionTab,
-  extractAccessTokenViaCdp,
-  getCdpEndpointInfo,
-  isCdpRunning,
-  isChatGptUrl,
-  isLumoUrl,
-  listCdpTargets,
-  openCdpTab,
-  withCdpSocket
-} from "./chunk-O5E5XFJR.js";
-export {
-  activateCdpTarget,
-  cdpNavigate,
-  discoverCdpPort,
-  ensureChatGptSessionTab,
-  extractAccessTokenViaCdp,
-  getCdpEndpointInfo,
-  isCdpRunning,
-  isChatGptUrl,
-  isLumoUrl,
-  listCdpTargets,
-  openCdpTab,
-  withCdpSocket
-};

package/dist/chunk-O5E5XFJR.js

@@ -1,446 +0,0 @@
-// src/utils/paths.ts
-import fs from "fs/promises";
-import os from "os";
-import path from "path";
-function getConfigDir() {
-  return process.env.PROMPT_EXPORTER_HOME?.trim() || path.join(os.homedir(), ".prompt-exporter");
-}
-function getDataParentDir() {
-  return process.env.PROMPT_EXPORTER_DATA?.trim() || process.env.CHATGPT_EXPORTER_DATA?.trim() || getConfigDir();
-}
-function getSourceDataDir(sourceId) {
-  return path.join(getDataParentDir(), sourceId);
-}
-function getDefaultDataDir() {
-  return getSourceDataDir("chatgpt");
-}
-function getSourceTokenPath(sourceId) {
-  return path.join(getConfigDir(), "sources", sourceId, "token");
-}
-function getIsolatedChromiumProfileDir() {
-  return path.join(getConfigDir(), "chromium");
-}
-function getSystemChromiumUserDataDir(browserPath) {
-  const home = os.homedir();
-  const binary = (browserPath || process.env.PROMPT_EXPORTER_BROWSER || process.env.CHATGPT_EXPORTER_BROWSER || "").toLowerCase();
-  if (process.platform === "darwin") {
-    if (binary.includes("chrome")) {
-      return path.join(home, "Library", "Application Support", "Google", "Chrome");
-    }
-    return path.join(home, "Library", "Application Support", "Chromium");
-  }
-  if (process.platform === "win32") {
-    const local = process.env.LOCALAPPDATA || path.join(home, "AppData", "Local");
-    if (binary.includes("chrome")) {
-      return path.join(local, "Google", "Chrome", "User Data");
-    }
-    return path.join(local, "Chromium", "User Data");
-  }
-  const config = process.env.XDG_CONFIG_HOME || path.join(home, ".config");
-  if (binary.includes("google-chrome") || /\/chrome$/.test(binary)) {
-    return path.join(config, "google-chrome");
-  }
-  return path.join(config, "chromium");
-}
-function resolveChromiumUserDataDir(options) {
-  const custom = options.userDataDir?.trim() || process.env.PROMPT_EXPORTER_USER_DATA_DIR?.trim() || process.env.CHATGPT_EXPORTER_USER_DATA_DIR?.trim();
-  if (custom) {
-    return { dir: custom, mode: "custom" };
-  }
-  if (options.isolated) {
-    return { dir: getIsolatedChromiumProfileDir(), mode: "isolated" };
-  }
-  return {
-    dir: getSystemChromiumUserDataDir(options.browserPath),
-    mode: "system"
-  };
-}
-var DEFAULT_CDP_PORT = 9222;
-var DEFAULT_SOURCE_ID = "chatgpt";
-async function ensureHomeLayout() {
-  const home = getConfigDir();
-  await fs.mkdir(home, { recursive: true });
-  await fs.mkdir(path.join(home, "sources"), { recursive: true });
-  const legacyHome = path.join(os.homedir(), ".chatgpt-exporter");
-  if (home !== legacyHome) {
-    try {
-      await fs.access(legacyHome);
-      const entries = await fs.readdir(legacyHome, { withFileTypes: true });
-      for (const entry of entries) {
-        const from = path.join(legacyHome, entry.name);
-        if (entry.name === "token") {
-          const destDir = path.join(home, "sources", "chatgpt");
-          await fs.mkdir(destDir, { recursive: true });
-          const dest = path.join(destDir, "token");
-          try {
-            await fs.access(dest);
-          } catch {
-            await fs.rename(from, dest).catch(async () => {
-              await fs.copyFile(from, dest);
-            });
-          }
-          continue;
-        }
-        if (entry.name === "data") {
-          const dest = path.join(home, "chatgpt");
-          try {
-            await fs.access(dest);
-          } catch {
-            await fs.rename(from, dest).catch(() => void 0);
-          }
-          continue;
-        }
-        if (entry.name === "chromium" || entry.name === "chatgpt") {
-          const dest = path.join(home, entry.name);
-          try {
-            await fs.access(dest);
-          } catch {
-            await fs.rename(from, dest).catch(() => void 0);
-          }
-        }
-      }
-    } catch {
-    }
-  }
-  const flatToken = path.join(home, "token");
-  const nestedToken = getSourceTokenPath("chatgpt");
-  try {
-    await fs.access(flatToken);
-    await fs.mkdir(path.dirname(nestedToken), { recursive: true });
-    try {
-      await fs.access(nestedToken);
-    } catch {
-      await fs.rename(flatToken, nestedToken).catch(async () => {
-        await fs.copyFile(flatToken, nestedToken);
-      });
-    }
-  } catch {
-  }
-}
-
-// src/utils/cdp.ts
-async function cdpHttp(port, path2, options = {}) {
-  const timeoutMs = options.timeoutMs ?? 2e3;
-  const url = `http://127.0.0.1:${port}${path2}`;
-  const controller = new AbortController();
-  const timer = setTimeout(() => controller.abort(), timeoutMs);
-  try {
-    const response = await fetch(url, {
-      method: options.method ?? "GET",
-      signal: controller.signal,
-      headers: { Accept: "application/json" }
-    });
-    if (!response.ok) {
-      throw new Error(`CDP HTTP ${response.status} for ${path2}`);
-    }
-    const text = await response.text();
-    if (!text) {
-      return void 0;
-    }
-    try {
-      return JSON.parse(text);
-    } catch {
-      if (options.allowPlainText) {
-        return text;
-      }
-      throw new Error(`CDP HTTP non-JSON response for ${path2}: ${text.slice(0, 80)}`);
-    }
-  } finally {
-    clearTimeout(timer);
-  }
-}
-async function isCdpRunning(port = DEFAULT_CDP_PORT, options = {}) {
-  const info = await getCdpEndpointInfo(port, options);
-  return info !== null;
-}
-async function discoverCdpPort(preferred = DEFAULT_CDP_PORT) {
-  const fromEnv = (process.env.PROMPT_EXPORTER_CDP_PORT || process.env.CHATGPT_EXPORTER_CDP_PORT)?.trim();
-  const candidates = [
-    fromEnv ? Number.parseInt(fromEnv, 10) : NaN,
-    preferred,
-    9222,
-    9223,
-    9229,
-    9333
-  ].filter((n) => Number.isFinite(n) && n > 0 && n <= 65535);
-  const seen = /* @__PURE__ */ new Set();
-  for (const port of candidates) {
-    if (seen.has(port)) continue;
-    seen.add(port);
-    if (await isCdpRunning(port)) return port;
-  }
-  return null;
-}
-async function getCdpEndpointInfo(port = DEFAULT_CDP_PORT, options = {}) {
-  try {
-    const data = await cdpHttp(port, "/json/version", options);
-    return {
-      port,
-      webSocketDebuggerUrl: typeof data.webSocketDebuggerUrl === "string" ? data.webSocketDebuggerUrl : void 0,
-      Browser: typeof data.Browser === "string" ? data.Browser : void 0,
-      "Protocol-Version": typeof data["Protocol-Version"] === "string" ? data["Protocol-Version"] : void 0
-    };
-  } catch {
-    return null;
-  }
-}
-function isChatGptUrl(url) {
-  try {
-    const parsed = new URL(url);
-    return parsed.hostname === "chatgpt.com" || parsed.hostname === "www.chatgpt.com" || parsed.hostname === "chat.openai.com";
-  } catch {
-    return false;
-  }
-}
-function isLumoUrl(url) {
-  try {
-    const parsed = new URL(url);
-    return parsed.hostname === "lumo.proton.me" || parsed.hostname.endsWith(".lumo.proton.me");
-  } catch {
-    return false;
-  }
-}
-function parseTarget(raw) {
-  if (typeof raw.id !== "string") return null;
-  return {
-    id: raw.id,
-    type: typeof raw.type === "string" ? raw.type : "unknown",
-    title: typeof raw.title === "string" ? raw.title : "",
-    url: typeof raw.url === "string" ? raw.url : "",
-    webSocketDebuggerUrl: typeof raw.webSocketDebuggerUrl === "string" ? raw.webSocketDebuggerUrl : void 0
-  };
-}
-async function listCdpTargets(port) {
-  const data = await cdpHttp(port, "/json/list");
-  if (!Array.isArray(data)) return [];
-  return data.map((item) => parseTarget(item)).filter((t) => t !== null);
-}
-async function openCdpTab(port, url) {
-  const path2 = `/json/new?${encodeURIComponent(url)}`;
-  const data = await cdpHttp(port, path2, { method: "PUT" }).catch(
-    async () => cdpHttp(port, path2)
-  );
-  const target = parseTarget(data);
-  if (!target) {
-    throw new Error("CDP /json/new did not return a target");
-  }
-  return target;
-}
-async function activateCdpTarget(port, id) {
-  await cdpHttp(port, `/json/activate/${encodeURIComponent(id)}`, {
-    allowPlainText: true
-  });
-}
-function getWebSocket() {
-  const WS = globalThis.WebSocket;
-  if (!WS) {
-    throw new Error(
-      "WebSocket is not available in this Node runtime (need Node 22+ or a WebSocket polyfill)"
-    );
-  }
-  return WS;
-}
-async function withCdpSocket(webSocketDebuggerUrl, fn) {
-  const WS = getWebSocket();
-  const ws = new WS(webSocketDebuggerUrl);
-  let nextId = 1;
-  const pending = /* @__PURE__ */ new Map();
-  const send = (method, params) => {
-    const id = nextId++;
-    return new Promise((resolve, reject) => {
-      pending.set(id, { resolve, reject });
-      ws.send(JSON.stringify({ id, method, params }));
-    });
-  };
-  try {
-    await new Promise((resolve, reject) => {
-      const timer = setTimeout(() => reject(new Error("CDP WebSocket connect timeout")), 5e3);
-      ws.addEventListener("open", () => {
-        clearTimeout(timer);
-        resolve();
-      });
-      ws.addEventListener("error", () => {
-        clearTimeout(timer);
-        reject(new Error("CDP WebSocket connection failed"));
-      });
-    });
-    ws.addEventListener("message", (event) => {
-      const raw = typeof event.data === "string" ? event.data : String(event.data);
-      let msg;
-      try {
-        msg = JSON.parse(raw);
-      } catch {
-        return;
-      }
-      const id = msg.id;
-      if (typeof id !== "number") return;
-      const waiter = pending.get(id);
-      if (!waiter) return;
-      pending.delete(id);
-      if (msg.error) {
-        const err = msg.error;
-        waiter.reject(
-          new Error(typeof err.message === "string" ? err.message : "CDP error")
-        );
-      } else {
-        waiter.resolve(msg.result ?? {});
-      }
-    });
-    return await fn(send);
-  } finally {
-    for (const [, waiter] of pending) {
-      waiter.reject(new Error("CDP WebSocket closed"));
-    }
-    pending.clear();
-    ws.close();
-  }
-}
-async function cdpNavigate(webSocketDebuggerUrl, url) {
-  const WS = getWebSocket();
-  const ws = new WS(webSocketDebuggerUrl);
-  let nextId = 1;
-  const pending = /* @__PURE__ */ new Map();
-  const send = (method, params) => {
-    const id = nextId++;
-    return new Promise((resolve, reject) => {
-      pending.set(id, { resolve, reject });
-      ws.send(JSON.stringify({ id, method, params }));
-    });
-  };
-  try {
-    await new Promise((resolve, reject) => {
-      const timer = setTimeout(() => reject(new Error("CDP WebSocket connect timeout")), 5e3);
-      ws.addEventListener("open", () => {
-        clearTimeout(timer);
-        resolve();
-      });
-      ws.addEventListener("error", () => {
-        clearTimeout(timer);
-        reject(new Error("CDP WebSocket connection failed"));
-      });
-    });
-    let loadResolve;
-    const loadPromise = new Promise((resolve) => {
-      loadResolve = resolve;
-    });
-    ws.addEventListener("message", (event) => {
-      const raw = typeof event.data === "string" ? event.data : String(event.data);
-      let msg;
-      try {
-        msg = JSON.parse(raw);
-      } catch {
-        return;
-      }
-      if (msg.method === "Page.loadEventFired") {
-        loadResolve?.();
-      }
-      const id = msg.id;
-      if (typeof id !== "number") return;
-      const waiter = pending.get(id);
-      if (!waiter) return;
-      pending.delete(id);
-      if (msg.error) {
-        const err = msg.error;
-        waiter.reject(
-          new Error(typeof err.message === "string" ? err.message : "CDP error")
-        );
-      } else {
-        waiter.resolve(msg.result ?? {});
-      }
-    });
-    await send("Page.enable");
-    await send("Page.navigate", { url });
-    await Promise.race([
-      loadPromise,
-      new Promise((r) => setTimeout(r, 5e3))
-    ]);
-    await new Promise((r) => setTimeout(r, 300));
-  } finally {
-    for (const [, waiter] of pending) {
-      waiter.reject(new Error("CDP WebSocket closed"));
-    }
-    pending.clear();
-    ws.close();
-  }
-}
-async function ensureChatGptSessionTab(port, sessionUrl) {
-  const targets = await listCdpTargets(port);
-  const pages = targets.filter((t) => t.type === "page" || t.type === "tab");
-  const existing = pages.find((t) => t.url.startsWith(sessionUrl)) ?? pages.find((t) => isChatGptUrl(t.url));
-  if (existing) {
-    await activateCdpTarget(port, existing.id);
-    if (existing.webSocketDebuggerUrl && existing.url !== sessionUrl) {
-      await cdpNavigate(existing.webSocketDebuggerUrl, sessionUrl);
-      const refreshed = (await listCdpTargets(port)).find((t) => t.id === existing.id);
-      return { target: refreshed ?? existing, reused: true };
-    }
-    if (existing.url !== sessionUrl) {
-      const created2 = await openCdpTab(port, sessionUrl);
-      return { target: created2, reused: true };
-    }
-    return { target: existing, reused: true };
-  }
-  const created = await openCdpTab(port, sessionUrl);
-  return { target: created, reused: false };
-}
-async function extractAccessTokenViaCdp(webSocketDebuggerUrl, options = {}) {
-  const attempts = options.attempts ?? 5;
-  const delayMs = options.delayMs ?? 400;
-  return withCdpSocket(webSocketDebuggerUrl, async (send) => {
-    await send("Runtime.enable");
-    for (let i = 0; i < attempts; i++) {
-      const result = await send("Runtime.evaluate", {
-        expression: `(() => {
-          const readBody = () => {
-            const text = document.body && document.body.innerText;
-            if (!text) return null;
-            try {
-              const parsed = JSON.parse(text.trim());
-              if (parsed && typeof parsed.accessToken === 'string') return parsed.accessToken;
-            } catch {}
-            return null;
-          };
-          const fromBody = readBody();
-          if (fromBody) return Promise.resolve(fromBody);
-          return fetch('https://chatgpt.com/api/auth/session', { credentials: 'include' })
-            .then((r) => r.json())
-            .then((j) => (j && typeof j.accessToken === 'string' ? j.accessToken : null))
-            .catch(() => null);
-        })()`,
-        awaitPromise: true,
-        returnByValue: true
-      });
-      const remote = result.result;
-      const value = remote?.value;
-      if (typeof value === "string" && value.startsWith("eyJ")) {
-        return value;
-      }
-      if (i < attempts - 1) {
-        await new Promise((r) => setTimeout(r, delayMs));
-      }
-    }
-    return null;
-  });
-}
-
-export {
-  getSourceDataDir,
-  getDefaultDataDir,
-  getSourceTokenPath,
-  resolveChromiumUserDataDir,
-  DEFAULT_CDP_PORT,
-  DEFAULT_SOURCE_ID,
-  ensureHomeLayout,
-  isCdpRunning,
-  discoverCdpPort,
-  getCdpEndpointInfo,
-  isChatGptUrl,
-  isLumoUrl,
-  listCdpTargets,
-  openCdpTab,
-  activateCdpTarget,
-  withCdpSocket,
-  cdpNavigate,
-  ensureChatGptSessionTab,
-  extractAccessTokenViaCdp
-};