@lincoln504/pi-research 1.0.3 → 1.0.5

package/dist/prompts/researcher.md

@@ -58,7 +58,7 @@ Write your report immediately after scraping is complete, or as soon as a tool r
 ## Guidelines
 
 - **Available Tools**:
-  - `scrape`: Fetch and read web pages (primary tool). Focus your energy here.
+  - `scrape`: Fetch and read web pages (primary tool). Focus your energy here. Do NOT scrape YouTube video links (watch / youtu.be / shorts) — a watch page returns YouTube's app shell, not the video's content; read videos with `youtube_transcript`. Scraping a YouTube channel or playlist page to DISCOVER video links is fine.
   - `youtube_transcript`: Read the captions/transcript of YouTube videos from your source list — it is the `scrape` for video. ONE call only, batching up to a few of the most relevant YouTube links. Cite these as `Source: YouTube Transcript`.
   - `stackexchange`: Use ONLY for Stack Overflow or Stack Exchange URLs.
   - `security_search`: Query NVD, CVE, OSV, CISA databases.

package/docs/CONFIGURATION.md

@@ -75,7 +75,7 @@ Research
 | `PI_RESEARCH_RETRY_DELAY_MS` | `2000` | 100–10000 | Base delay between retries. |
 | `PI_RESEARCH_WORKER_THREADS` | `4` | 1–10 | Browser worker processes. Higher = more throughput, more CPU/RAM. |
 | `PI_RESEARCH_WORKER_CONCURRENCY` | `2` | 1–10 | Tasks per worker process. |
-| `PI_RESEARCH_MODEL` | _(pi: session model; CLI/skill: required)_ | — | The model research runs on. **Required for the standalone CLI / agent skill** — they use only this configured model (never the model selected inside the pi extension) and refuse to start without one. On the SDK it selects the session model when no `model` option is given. In the pi extension it overrides researcher sub-agents and knowledge synthesis, while the coordinator and evaluator keep using the session model. Accepts `provider/id` or a bare model id. |
+| `PI_RESEARCH_MODEL` | _(pi: session model; CLI/skill: required)_ | — | The model research runs on. **Required for the standalone CLI / agent skill** — they use only this configured model (never the model selected inside the pi extension) and refuse to start without one (the CLI's per-run `--model` flag also satisfies this). On the SDK it selects the session model when no `model` option is given. In the pi extension it overrides researcher sub-agents and knowledge synthesis, while the coordinator and evaluator keep using the session model. Accepts `provider/id` or a bare model id. |
 | `PI_RESEARCH_DISABLED_TOOLS` | _(none)_ | — | Comma-separated research tools to disable for a run (`search`, `scrape`, `security_search`, `stackexchange`, `youtube_transcript`, `grep`). Removed from every researcher's toolset and named in the coordinator/evaluator prompt. |
 | `PI_RESEARCH_REPORT_EXPORT_ENABLED` (TUI) | `false` | — | Front-ends write a Markdown report to disk and surface its path. |
 | `PI_RESEARCH_REPORT_EXPORT_DIR` | _(smart cwd)_ | — | Pin exported reports to a fixed directory, bypassing the cwd-relative resolution. Useful for the agent skill, which runs from the host agent's arbitrary directory. |
@@ -132,7 +132,7 @@ API keys
 
 | Variable | Description |
 |----------|-------------|
-| `PI_RESEARCH_API_KEY` / `PI_RESEARCH_PROVIDER` | Explicit LLM credentials for SDK / CLI mode (not needed when pi's configuration — `~/.pi/agent/auth.json` — supplies the key). On the CLI / agent skill both can also live in `config.env` / `cli.env`. The provider is required alongside the key, or inferred from a `provider/model-id` `PI_RESEARCH_MODEL`. |
+| `PI_RESEARCH_API_KEY` / `PI_RESEARCH_PROVIDER` | Explicit LLM credentials for SDK / CLI mode (not needed when pi's configuration supplies the key). On the CLI / agent skill both can also live in `config.env` / `cli.env`. The provider is required alongside the key, or inferred from a `provider/model-id` `PI_RESEARCH_MODEL`. |
 | `STACKEXCHANGE_API_KEY` | Raises the Stack Exchange tool's limit from 300/day to 10,000/day. Obtain at <https://stackapps.com/apps/oauth>. |
 | `GITHUB_TOKEN` | Raises the security tool's GitHub Advisory limit from 60/hr to 5000/hr (any default-scope token). |
 | `NVD_API_KEY` | Raises the security tool's NVD limit ~10× and tightens request spacing. Request at <https://nvd.nist.gov/developers/request-an-api-key>. Recommended when using severity-filtered security searches: those issue a second (CVSS v2) NVD query to catch v2-only CVEs, which roughly doubles request time against the 6 s/request unauthenticated throttle. |

package/docs/KNOWLEDGE-STORE.md

@@ -174,9 +174,11 @@ built with, the store is migrated according to `PI_RESEARCH_MIGRATION_STRATEGY`:
 | `drop` | The old table is discarded and a fresh one created. Fast; no backup. |
 | `re-embed` | Every stored document is re-embedded with the new model into a new table, preserving history. Slowest. |
 
-If the chosen strategy fails, pi-research falls back to `backup`, then to `drop`,
-rather than leaving the store in a broken state. Changing the model from the
-`/research-config` menu always clears the current store and starts fresh.
+If `re-embed` fails, pi-research falls back to `backup`. A failed `backup` (or
+`drop`) aborts the migration instead: the store stays on the old model and the
+next open retries — data is never dropped unless `drop` was chosen explicitly.
+Changing the model from the `/research-config` menu always clears the current
+store and starts fresh.
 
 ### Managing the store
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lincoln504/pi-research",
-  "version": "1.0.3",
+  "version": "1.0.5",
   "description": "A set of web research tools (search, scrape, and others) and research agent system that uses them safely and efficiently.",
   "type": "module",
   "private": false,

package/skills/pi-research/SKILL.md

@@ -2,7 +2,7 @@
 name: pi-research
 description: "Multi-agent web research for any coding agent. Use for anything requiring current information from the internet — news, trends, product/service/company facts, statistics, people, places, events, APIs, library behavior, or 'what is X / how does X work' questions. Also use to search a local knowledge store of previously researched findings before doing live research. Powered by the pi-research engine (search, stealth scraping, security databases, Stack Exchange). Not for reading local files, running project commands, or analyzing this repository's own source."
 license: MIT
-metadata: { "author": "Lincoln Deen", "version": "1.0.3", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
+metadata: { "author": "Lincoln Deen", "version": "1.0.5", "package": "@lincoln504/pi-research", "engine": "pi-research-sdk", "openclaw": { "requires": { "bins": ["node"] } } }
 allowed-tools: Bash(node:*)
 ---
 

package/skills/pi-research/references/configuration.md

@@ -15,8 +15,8 @@ to configure (no research runs).
    export PI_RESEARCH_MODEL=openai/gpt-4o    # provider/model-id
    ```
 2. **Global config** `~/.pi/research/config.env` (same KEY=VALUE keys; read at startup).
-3. **pi configuration** (if you use `pi`): the API key comes from `~/.pi/agent/auth.json`
-   automatically — `PI_RESEARCH_MODEL` must still be set (env or `config.env`).
+3. **pi configuration** (if you use `pi`): the API key comes from your pi
+   configuration automatically — `PI_RESEARCH_MODEL` must still be set (env or `config.env`).
 
 Real env vars always win. Optional CLI-only overlay `~/.pi/research/cli.env` layers
 over `config.env` for the CLI only. Precedence:

package/src/cli.ts

@@ -39,6 +39,7 @@ import { validateInitialLink, MAX_INITIAL_LINKS } from './utils/url-utils.ts';
 import { exportResearchReport, appendExportMessage } from './utils/research-export.ts';
 import { getConfig, getGlobalConfigDir, getGlobalEnvFilePath, getInterfaceEnvFilePath, saveConfig, resetConfig, describeKnowledgeStoreMode, isProjectScopedKey, parseDotEnv } from './config.ts';
 import { getAgentDir } from '@earendil-works/pi-coding-agent';
+import { buildModelRegistry, safeGetAvailable } from './core/llm/model-registry-factory.ts';
 
 // ---------------------------------------------------------------------------
 // Constants
@@ -226,13 +227,39 @@ function fileExists(p: string): boolean {
   }
 }
 
+/**
+ * Are keys usable from pi's configuration? Answered by pi's OWN semantics, not
+ * by file presence: build pi's model registry (two small JSON reads — no
+ * services, network, or native deps) and ask for the authed subset. That check
+ * (`hasConfiguredAuth`) covers every source the registry consults at call
+ * time: an auth.json provider entry (api-key or OAuth), a per-provider apiKey
+ * embedded in models.json (env-template aware), and provider env vars like
+ * OPENAI_API_KEY. File presence alone is neither necessary (keys can live in
+ * models.json / env with no auth.json) nor sufficient (auth.json can be `{}`).
+ */
+function piKeysAvailable(piAuthPresent: boolean, provider?: string): boolean {
+  try {
+    const available = safeGetAvailable(buildModelRegistry(undefined, undefined));
+    if (!provider) return available.length > 0;
+    // Provider known (from the configured provider/model-id): keys must exist
+    // for THAT provider — a key for some other provider would pass pre-flight
+    // and then die on the first LLM call ("No API key found for <provider>").
+    return available.some((m) => m.provider === provider);
+  } catch {
+    // Cannot inspect (e.g. malformed models.json) — fall back to file
+    // presence: pre-flight must never block a possibly-working setup; the run
+    // itself reports the precise error.
+    return piAuthPresent;
+  }
+}
+
 export interface CredentialDetection {
   /** Where credentials will come from once initResearchSDK runs. */
   source: 'explicit-env' | 'pi-config' | 'none';
   apiKeyConfigured: boolean;
   provider?: string;
   model?: string;
-  modelFrom: 'PI_RESEARCH_MODEL' | 'config.env' | 'unset';
+  modelFrom: '--model' | 'PI_RESEARCH_MODEL' | 'config.env' | 'unset';
   piAuthPresent: boolean;
   piModelsPresent: boolean;
   /** A short, human-readable problem string, or undefined when ready. */
@@ -242,8 +269,17 @@ export interface CredentialDetection {
 /**
  * Detect how the SDK will authenticate WITHOUT initializing it. Used both for
  * the `status` command and for the fast-fail pre-flight before every run.
+ *
+ * `explicitModel` is a per-run `--model` flag. It must participate in the
+ * pre-flight: it outranks PI_RESEARCH_MODEL/config for the run, so (a) it
+ * satisfies the model-required check on its own, and (b) when no explicit
+ * PI_RESEARCH_PROVIDER is set, provider-aware key detection derives the
+ * provider from the FLAG's model — otherwise a configured model on provider X
+ * greenlights a `--model` run on keyless provider Y (or vice versa wrongly
+ * blocks a valid `--model` run). An explicit PI_RESEARCH_PROVIDER still wins:
+ * it names the provider the configured key belongs to.
  */
-function detectCredentials(): CredentialDetection {
+function detectCredentials(explicitModel?: string): CredentialDetection {
   const paths = resolvedConfigPaths();
   const piAuthPresent = fileExists(paths.piAuth);
   const piModelsPresent = fileExists(paths.piModels);
@@ -253,11 +289,14 @@ function detectCredentials(): CredentialDetection {
   const modelEnv = process.env['PI_RESEARCH_MODEL'];
   const modelConfig = getConfig(process.cwd(), 'cli').RESEARCH_MODEL;
 
-  const model = modelEnv ?? modelConfig;
-  const modelFrom = modelEnv ? 'PI_RESEARCH_MODEL' : modelConfig ? 'config.env' : 'unset';
+  const model = explicitModel ?? modelEnv ?? modelConfig;
+  const modelFrom = explicitModel ? '--model' : modelEnv ? 'PI_RESEARCH_MODEL' : modelConfig ? 'config.env' : 'unset';
   // Provider is either explicit, or inferred from a "provider/id" model string.
   const provider =
     providerEnv ?? (model && model.includes('/') ? model.slice(0, model.indexOf('/')) : undefined);
+  // Provider-aware when the configured model names one: a key for a DIFFERENT
+  // provider must not greenlight this run.
+  const piKeysUsable = piKeysAvailable(piAuthPresent, provider);
 
   // --- Explicit API-key path -------------------------------------------------
   if (apiKey) {
@@ -286,7 +325,7 @@ function detectCredentials(): CredentialDetection {
         piModelsPresent,
         problem:
           'PI_RESEARCH_API_KEY is set but no model is configured. Set PI_RESEARCH_MODEL to a ' +
-          '"provider/model-id" (e.g. openai/gpt-4o).',
+          '"provider/model-id" (e.g. openai/gpt-4o), or pass --model <provider/id> for a single run.',
       };
     }
     return {
@@ -300,13 +339,13 @@ function detectCredentials(): CredentialDetection {
     };
   }
 
-  // --- pi configuration path (key from auth.json) -----------------------------
-  if (piAuthPresent) {
-    // Keys come from auth.json, but the MODEL must be configured explicitly:
-    // the standalone CLI / agent skill run only on the configured model — they
-    // never follow the model selected inside the pi extension, and silently
-    // falling back to the registry's first authed entry picks a model nobody
-    // chose. Fail fast here (pre-flight, before any SDK init).
+  // --- pi configuration path (key from auth.json, models.json, or provider env) ---
+  if (piKeysUsable) {
+    // Keys come from pi's configuration, but the MODEL must be configured
+    // explicitly: the standalone CLI / agent skill run only on the configured
+    // model — they never follow the model selected inside the pi extension, and
+    // silently falling back to the registry's first authed entry picks a model
+    // nobody chose. Fail fast here (pre-flight, before any SDK init).
     if (!model) {
       return {
         source: 'pi-config',
@@ -317,15 +356,16 @@ function detectCredentials(): CredentialDetection {
         piAuthPresent,
         piModelsPresent,
         problem:
-          'pi credentials were found (auth.json), but no research model is configured. ' +
+          'pi credentials were found (in pi\'s configuration), but no research model is configured. ' +
           'The standalone CLI / agent skill run only on an explicitly configured model — they do not ' +
           'follow the model selected inside the pi extension. Set PI_RESEARCH_MODEL to a ' +
-          '"provider/model-id" (in the environment, or in config.env / cli.env below).',
+          '"provider/model-id" (in the environment, or in config.env / cli.env below), or pass ' +
+          '--model <provider/id> for a single run.',
       };
     }
     return {
       source: 'pi-config',
-      apiKeyConfigured: true, // key lives in auth.json; SDK reads it
+      apiKeyConfigured: true, // key lives in pi's configuration; the SDK reads it at call time
       provider,
       model,
       modelFrom,
@@ -346,7 +386,8 @@ function detectCredentials(): CredentialDetection {
     problem:
       'No model or API key is configured for pi-research. Provide credentials via environment ' +
       'variables (PI_RESEARCH_API_KEY + PI_RESEARCH_PROVIDER + PI_RESEARCH_MODEL) or, if you use ' +
-      'pi, via ~/.pi/agent/auth.json.',
+      "pi, via pi's configuration (a provider entry in ~/.pi/agent/auth.json, or a per-provider " +
+      'apiKey in ~/.pi/agent/models.json).',
   };
 }
 
@@ -370,8 +411,8 @@ function configBlock(det: CredentialDetection, extraNote?: string): string {
   lines.push(`  • api key configured: ${det.apiKeyConfigured}`);
   lines.push(`  • provider:           ${det.provider ?? '(unset — inferred from a provider/model-id model)'}`);
   lines.push(`  • model:              ${det.model ? `${det.model}  [from: ${det.modelFrom}]` : '(not set — set PI_RESEARCH_MODEL, required)'}`);
-  lines.push(`  • pi auth.json:       ${det.piAuthPresent ? 'present' : 'absent'}`);
-  lines.push(`  • pi models.json:     ${det.piModelsPresent ? 'present' : 'absent'}`);
+  lines.push(`  • pi config (auth.json):   ${det.piAuthPresent ? 'present' : 'absent'}`);
+  lines.push(`  • pi config (models.json): ${det.piModelsPresent ? 'present' : 'absent'}`);
   if (extraNote) lines.push(`  • note:               ${extraNote}`);
   return lines.join('\n');
 }
@@ -394,7 +435,9 @@ interface ResearchArgs {
  * markdown report to stdout, and always shuts down (even on failure).
  */
 async function cmdResearch(args: ResearchArgs): Promise<number> {
-  const det = detectCredentials();
+  // Pre-flight with the per-run --model flag folded in: it satisfies the
+  // model-required check and steers the provider-aware key detection.
+  const det = detectCredentials(args.model);
   if (det.problem) {
     if (args.json) toStdout(pretty({ ok: false, error: det.problem, exitCode: EXIT.CONFIG }));
     else toStderr(`\nError: ${det.problem}\n\n${configBlock(det)}\n`);
@@ -416,8 +459,8 @@ async function cmdResearch(args: ResearchArgs): Promise<number> {
 
   const depth: 0 | 1 | 2 | 3 =
     (args.depth ?? getConfig(process.cwd(), 'cli').DEFAULT_RESEARCH_DEPTH) as 0 | 1 | 2 | 3;
-  // An explicit --model wins over configured/PI_RESEARCH_MODEL for this run.
-  const runModel = args.model ?? det.model;
+  // det.model already reflects --model-first precedence (detectCredentials above).
+  const runModel = det.model;
   toStderr(`[pi-research] starting research (depth ${depth})${runModel ? ` with ${runModel}` : ''}…\n`);
 
   // Stream concise lifecycle progress to stderr so a long run is never silent.
@@ -763,8 +806,12 @@ function reportError(err: unknown, what: string, json?: boolean): number {
     lower.includes('unauthorized') ||
     lower.includes('api key');
 
+  // Rate limits win over config-shaped matches: a provider 429 often mentions
+  // "api key" in its body (e.g. "rate limit exceeded for this api key"), and
+  // classifying that as a setup problem sends the user to reconfigure a
+  // perfectly working key instead of waiting out the limit.
   const isRateLimit = lower.includes('429') || lower.includes('rate limit') || lower.includes('too many requests');
-  const exitCode = isConfigError ? EXIT.CONFIG : EXIT.SOFTWARE;
+  const exitCode = isConfigError && !isRateLimit ? EXIT.CONFIG : EXIT.SOFTWARE;
 
   // --json: emit a structured error on stdout so a machine consumer never has to
   // special-case plain-text stderr. Mirrors the { ok: true, ... } success shape.
@@ -777,15 +824,15 @@ function reportError(err: unknown, what: string, json?: boolean): number {
     return exitCode;
   }
 
-  if (isConfigError) {
-    toStderr(`\nError: pi-research ${what} failed: ${msg}\n\n${configBlock(detectCredentials())}\n`);
-    return exitCode;
-  }
-  // Rate limits are operational, not fatal setup problems.
+  // Rate limits are operational, not fatal setup problems — checked FIRST.
   if (isRateLimit) {
     toStderr(`\nError: pi-research ${what} halted: provider rate limit reached. Wait a moment and retry.\n  ${msg}\n`);
     return exitCode;
   }
+  if (isConfigError) {
+    toStderr(`\nError: pi-research ${what} failed: ${msg}\n\n${configBlock(detectCredentials())}\n`);
+    return exitCode;
+  }
   toStderr(`\nError: pi-research ${what} failed: ${msg}\n`);
   if (process.env['PI_RESEARCH_DEBUG'] === 'true' && err instanceof Error && err.stack) {
     toStderr('\n' + err.stack + '\n');
@@ -842,7 +889,7 @@ USAGE
 COMMANDS
   research  "<query>"            Run multi-agent web research (default depth: 1).
     --depth <0-3>                0 = quick, 1 = normal (default), 2 = deep, 3 = ultra.
-    --model <provider/id>        Override the model for this run.
+    --model <provider/id>        Model for this run (outranks PI_RESEARCH_MODEL; satisfies the required-model check).
     --exclude-tools <a,b>        Disable internal tools (e.g. security_search,stackexchange).
     --initial-links <url ...>    Seed URLs to investigate first; requires a query (-- ends options).
     --config <path>              Use this config file instead of the base config.env.
@@ -879,9 +926,9 @@ CONFIGURE
     pi config (keys):    ${p.piAuth}
     pi config (models):  ${p.piModels}
 
-  A model is REQUIRED: set PI_RESEARCH_MODEL to a "provider/model-id". This CLI and
-  the agent skill run only on that configured model — they do not follow the model
-  selected inside the pi extension.
+  A model is REQUIRED: set PI_RESEARCH_MODEL to a "provider/model-id" (or pass
+  --model per run). This CLI and the agent skill run only on that configured
+  model — they do not follow the model selected inside the pi extension.
 
   The pi extension has its own optional overlay file:
     pi extension: ${p.piIfaceEnv}

package/src/core/llm/model-registry-factory.ts

@@ -123,12 +123,13 @@ export function buildModelRegistry(apiKey?: string, provider?: string): ModelReg
     );
   }
 
-  // No explicit key: use the user's pi auth storage and model list if available.
+  // No explicit key: use the user's pi configuration when EITHER file exists.
   // ModelRegistry.create() reads embedded apiKeys from models.json providers
-  // directly (via providerRequestConfigs), so glm-coding and other custom
-  // providers with embedded keys are visible to hasConfiguredAuth() without
-  // needing to duplicate them into AuthStorage.
-  if (fs.existsSync(authPath)) {
+  // directly (via providerRequestConfigs), so custom providers with embedded
+  // keys work with no auth.json at all — auth.json presence must not gate
+  // models.json (AuthStorage.create tolerates a missing file). Env provider
+  // keys (e.g. OPENAI_API_KEY) are honored by AuthStorage in every branch.
+  if (fs.existsSync(authPath) || fs.existsSync(modelsJsonPath)) {
     const authStorage = AuthStorage.create(authPath);
     return ModelRegistry.create(
       authStorage,
@@ -152,7 +153,17 @@ export function constructMinimalModel(provider: string, modelId: string, _apiKey
     provider,
     id: modelId,
     name: modelId,
-    api: provider === 'openai' ? 'openai-completions' : (provider + '-conversations' as any),
+    // Map to a REAL pi-ai api id — "<provider>-conversations" is not one, and a
+    // keyed no-registry setup would pass pre-flight then die on the first call.
+    // Unknown providers default to the OpenAI-compatible protocol, which is what
+    // virtually every third-party endpoint (openrouter, groq, cerebras, …) speaks.
+    api: ({
+      openai: 'openai-completions',
+      anthropic: 'anthropic-messages',
+      google: 'google-generative-ai',
+      gemini: 'google-generative-ai',
+      mistral: 'mistral-conversations',
+    } as Record<string, string>)[provider] ?? 'openai-completions',
     baseUrl: '', // Provider-specific base URLs are handled by pi-ai internal registry
     reasoning: false,
     input: ['text'],

package/src/knowledge/store.ts

@@ -267,10 +267,12 @@ export class KnowledgeStore implements IKnowledgeStore {
               if (strategy !== 'drop' && strategy !== 'backup') {
                 logger.warn('[store] Falling back to backup strategy after migration failure');
                 await this.handleModelChange(storedModel || 'unknown', this.options.modelName, 'backup');
-              } else if (strategy === 'backup') {
-                logger.warn('[store] Falling back to drop strategy after backup failure');
-                await this.handleModelChange(storedModel || 'unknown', this.options.modelName, 'drop');
               } else {
+                // 'backup' failure must ABORT, never escalate to 'drop': backup is the
+                // data-preserving strategy, and silently destroying every row after a
+                // (possibly transient) failure is the opposite of what the user asked
+                // for. The store stays on the old model; the next open() retries.
+                // 'drop' failure has nowhere further to fall.
                 throw new Error(errorMsg, { cause: err });
               }
             }
@@ -332,8 +334,16 @@ export class KnowledgeStore implements IKnowledgeStore {
       throw new Error('Table not connected');
     }
 
-    const count = await this.table.countRows();
-    
+    // Best-effort: the count is only used for logging and the result summary, so a
+    // transient countRows() failure must not abort the backup (previously it threw
+    // while the fallback ladder could still escalate to a data-destroying drop).
+    let count = -1;
+    try {
+      count = await this.table.countRows();
+    } catch (err) {
+      logger.warn(`[store] countRows failed before backup (continuing): ${err instanceof Error ? err.message : String(err)}`);
+    }
+
     // Close current table handle
     this.table = null;
     
@@ -355,9 +365,9 @@ export class KnowledgeStore implements IKnowledgeStore {
     this.table = await this.createTable();
     // Table name unchanged (still canonical 'knowledge'), but persist to be safe.
     await this.saveManifest();
-    logger.info(`[store] Migration complete: ${count} documents backed up, fresh table created with model ${newModel}`);
+    logger.info(`[store] Migration complete: ${count >= 0 ? count : 'unknown count of'} documents backed up, fresh table created with model ${newModel}`);
 
-    return { strategy: 'backup', success: true, documentsProcessed: count };
+    return { strategy: 'backup', success: true, documentsProcessed: Math.max(0, count) };
   }
 
   private async migrationDrop(_oldModel: string, newModel: string): Promise<MigrationResult> {

package/src/orchestration/deep-research-orchestrator.ts

@@ -9,7 +9,7 @@ import { type Model } from '@earendil-works/pi-ai';
 import { logger } from '../logger.ts';
 import { safeUnref } from '../utils/safe-unref.ts';
 import { metrics } from '../utils/metrics.ts';
-import { getSteeringMessages, consumeQueuedMessages, getActiveSteeringMessages } from './session-state.ts';
+import { getSteeringMessages, consumeQueuedMessages, getActiveSteeringMessages, getFailedResearchers, getResearcherFailureReasons } from './session-state.ts';
 import { MAX_EXTRA_ROUNDS_WITH_STEERING } from '../constants.ts';
 import {
   getMaxRounds,
@@ -17,7 +17,7 @@ import {
 import type {
   ResearchObserver,
 } from '../core/interfaces/observer-interfaces.ts';
-import { HeadlessObserver, type HeadlessObserverOptions } from './headless-observer.ts';
+import { HeadlessObserver, makeSafeObserver, type HeadlessObserverOptions } from './headless-observer.ts';
 import {
   ServiceNames,
   type IResearchOrchestration,
@@ -61,11 +61,12 @@ export class DeepResearchOrchestrator {
   constructor(private options: DeepResearchOrchestratorOptions) {
     this.config = options.config || getConfig(options.ctx.cwd);
     
-    // Resolve observer: if options were provided instead of an instance, create the instance
+    // Resolve observer: if options were provided instead of an instance, create the instance.
+    // Either way, wrap it so a throwing observer callback can never fail the run.
     if (options.observer && typeof (options.observer as any).onProgress === 'function' && !(options.observer instanceof HeadlessObserver)) {
-       this.observer = new HeadlessObserver(options.observer as HeadlessObserverOptions);
+       this.observer = makeSafeObserver(new HeadlessObserver(options.observer as HeadlessObserverOptions));
     } else {
-       this.observer = options.observer as ResearchObserver | undefined;
+       this.observer = options.observer ? makeSafeObserver(options.observer as ResearchObserver) : undefined;
     }
 
     if (options.orchestrationService) {
@@ -478,15 +479,40 @@ export class DeepResearchOrchestrator {
       }
 
       let result = finalReport.content || '';
-      
-      // If content is empty but we have reports, build a fallback synthesis
+
+      // ZERO reports collected AND researchers actually failed — the run attempted
+      // research and produced nothing groundable. This gate is deliberately
+      // independent of the synthesis text: with the default single-researcher plan
+      // the fast-stop threshold (MAX_FAILED_RESEARCHERS = 2) never trips, so a lone
+      // researcher that exhausts its retries (e.g. a provider rate limit) reaches
+      // synthesis with no source material. Whether the LLM then returns an empty
+      // string OR chatty refusal prose ("no information was found"), that is a
+      // FAILED run: throw, and surface the recorded root causes so the user sees
+      // the 429/model error, not a shrug (and never a source-free report shipped
+      // as success). The failed-researcher condition keeps the designed
+      // immediate-synthesis path intact: a coordinator that chose to answer
+      // directly (no delegation, no failures) is not a failed run.
+      const synthesisCheckService = await getService<IResearchSynthesisService>(ServiceNames.RESEARCH_SYNTHESIS_SERVICE, ctx, container);
+      const hasReports = synthesisCheckService.hasReports(researchId);
+      const failedIds = getFailedResearchers(this.options.sessionId, researchId);
+      if (!hasReports && failedIds.length > 0) {
+          const reasons = getResearcherFailureReasons(this.options.sessionId, researchId);
+          const causes = failedIds
+            .map((id) => `researcher ${id}: ${reasons[id] ?? 'no usable report'}`)
+            .join('; ');
+          throw new Error(
+            `Research produced no report — no source material was collected. Causes: ${causes}`
+          );
+      }
+
       if (!result.trim()) {
-          const synthesisService = await getService<IResearchSynthesisService>(ServiceNames.RESEARCH_SYNTHESIS_SERVICE, ctx, container);
-          if (synthesisService.hasReports(researchId)) {
-              logger.warn(`[DeepOrchestrator] LLM returned empty synthesis for ${researchId}, building fallback from ${synthesisService.getReportCount(researchId)} reports`);
-              result = synthesisService.buildFallbackSynthesis(researchId, this.currentRound);
+          if (hasReports) {
+              // Content is empty but reports exist — build a fallback synthesis.
+              logger.warn(`[DeepOrchestrator] LLM returned empty synthesis for ${researchId}, building fallback from ${synthesisCheckService.getReportCount(researchId)} reports`);
+              result = synthesisCheckService.buildFallbackSynthesis(researchId, this.currentRound);
           } else {
-              result = 'Research completed but no summary was generated.';
+              // No reports, no recorded failures, and nothing to ship either.
+              throw new Error('Research produced no report — no source material was collected.');
           }
       }
 

package/src/orchestration/headless-observer.ts

@@ -12,6 +12,51 @@ import type { ResearchPlan } from '../core/interfaces/research-plan-types.ts';
 
 export type { HeadlessObserverOptions };
 
+/**
+ * Wrap an observer so a throwing (or rejecting) callback can never corrupt a
+ * research run. Observer methods are fire-and-forget notifications, but the
+ * orchestrators invoke them inline at load-bearing points (researcher
+ * completion, evaluation decisions, synthesis start) — an exception from
+ * user-supplied SDK/TUI observer code would otherwise surface as a *research*
+ * failure. HeadlessObserver already isolates its own onProgress callback; this
+ * extends the same isolation to raw ResearchObserver implementations. Sync
+ * throws are logged and swallowed; returned promises get a .catch so
+ * rejections never become unhandled.
+ */
+export function makeSafeObserver<T extends object>(observer: T): T {
+  return new Proxy(observer, {
+    get(target, prop) {
+      // Guard the ACCESS too, and resolve getters against the target (not the
+      // proxy): an observer exposing a method via an accessor that touches a
+      // private field would otherwise throw at `observer?.onX?.(...)` property
+      // access — outside the call-site try/catch below — which is exactly the
+      // failure this wrapper exists to absorb.
+      let value: unknown;
+      try {
+        value = Reflect.get(target, prop, target);
+      } catch (err) {
+        logger.warn(`[observer] Ignored error reading observer.${String(prop)}:`, err);
+        return undefined;
+      }
+      if (typeof value !== 'function') return value;
+      return (...args: unknown[]) => {
+        try {
+          const out = value.apply(target, args);
+          if (out && typeof (out as Promise<unknown>).then === 'function') {
+            return (out as Promise<unknown>).catch((err: unknown) => {
+              logger.warn(`[observer] Ignored rejection from observer.${String(prop)}:`, err);
+            });
+          }
+          return out;
+        } catch (err) {
+          logger.warn(`[observer] Ignored error from observer.${String(prop)}:`, err);
+          return undefined;
+        }
+      };
+    },
+  }) as T;
+}
+
 export class HeadlessObserver implements ResearchObserver {
   constructor(private options: HeadlessObserverOptions = {}) {}
 

package/src/orchestration/quick-research-orchestrator.ts

@@ -20,7 +20,7 @@ import { createResearcherSession } from './researcher.ts';
 import { ensureAssistantResponse, parseCitations } from '../utils/text-utils.ts';
 import { getMaxScrapeBatches } from '../constants.ts';
 import type { ResearchObserver } from '../core/interfaces/observer-interfaces.ts';
-import { HeadlessObserver, type HeadlessObserverOptions } from './headless-observer.ts';
+import { HeadlessObserver, makeSafeObserver, type HeadlessObserverOptions } from './headless-observer.ts';
 import { getService, tryGetServiceContainerFromCtx } from '../core/service-registry.ts';
 import { ServiceNames, type IWriterQueue, type IKnowledgeStoreService, type IResearchSynthesisService, type IResearchOrchestration } from '../core/service-interfaces.ts';
 import type { ResearchSessionService } from './research-session-service.ts';
@@ -51,11 +51,12 @@ export class QuickResearchOrchestrator {
   constructor(private options: QuickResearchOrchestratorOptions) {
     this.config = options.config || getConfig(options.ctx.cwd);
 
-    // Resolve observer: if options were provided instead of an instance, create the instance
+    // Resolve observer: if options were provided instead of an instance, create the instance.
+    // Either way, wrap it so a throwing observer callback can never fail the run.
     if (options.observer && typeof (options.observer as any).onProgress === 'function' && !(options.observer instanceof HeadlessObserver)) {
-       this.observer = new HeadlessObserver(options.observer as HeadlessObserverOptions);
+       this.observer = makeSafeObserver(new HeadlessObserver(options.observer as HeadlessObserverOptions));
     } else {
-       this.observer = options.observer as ResearchObserver | undefined;
+       this.observer = options.observer ? makeSafeObserver(options.observer as ResearchObserver) : undefined;
     }
   }
 

package/src/orchestration/research-orchestration-service.ts

@@ -358,8 +358,9 @@ export class ResearchOrchestrationService implements IResearchOrchestration {
             logger.debug(`[ResearchOrchestrationService] Researcher ${id} cancelled (aborted).`);
           } else {
             logger.error(`[ResearchOrchestrationService] Researcher ${id} failed: ${errMsg}`);
-            // Record failure for stopping logic
-            recordResearcherFailure(sessionId, researchId, id);
+            // Record failure (with its root cause) for stopping logic and for the
+            // zero-report failure surfaced at synthesis time.
+            recordResearcherFailure(sessionId, researchId, id, errMsg);
             // Notify observer
             observer?.onResearcherFailure?.(id, errMsg);
           }

package/src/orchestration/research-synthesis-service.ts

@@ -12,7 +12,7 @@
 import { parseCitations } from '../utils/text-utils.ts';
 import { normalizeCitations, formatCitedLinks, type GlobalCitation } from '../utils/citation-utils.ts';
 import { lastCitedLinksHeaderIndex } from '../utils/text-utils.ts';
-import { getScrapedLinks, normalizeUrl } from '../utils/shared-links.ts';
+import { getScrapedLinks, isTranscribedLink, normalizeUrl } from '../utils/shared-links.ts';
 import { logger } from '../logger.ts';
 import { ServiceLifecycle, type IService } from '../core/service-registry.ts';
 import { ServiceNames } from '../core/interfaces/service-names.ts';
@@ -188,9 +188,17 @@ export class ResearchSynthesisService implements IService {
     if (globalCitations.length === 0) {
       const scraped = getScrapedLinks(sessionId);
       if (scraped.length > 0) {
-        globalCitations = scraped.map((url, i) => ({ id: i + 1, url, description: '', source: 'Scrape' }));
+        // Label per actual provenance: the pool also contains transcript watch
+        // URLs the youtube_transcript tool registered — calling those "Scrape"
+        // misattributes the report's real source material.
+        globalCitations = scraped.map((url, i) => ({
+          id: i + 1,
+          url,
+          description: '',
+          source: isTranscribedLink(sessionId, url) ? 'YouTube Transcript' : 'Scrape',
+        }));
         logger.warn(
-          `[ResearchSynthesisService] No citations parsed from reports; rebuilt ${scraped.length} source(s) from scrape provenance.`,
+          `[ResearchSynthesisService] No citations parsed from reports; rebuilt ${scraped.length} source(s) from scrape/transcript provenance.`,
         );
       }
     }

package/src/orchestration/researcher-executor.ts

@@ -69,7 +69,7 @@ export async function runResearcher(options: RunResearcherOptions): Promise<void
     // use), recording under ctx.sessionId would file this no-links skip under a
     // different id than shouldStopResearch() checks, so the fast-stop guard would
     // under-count exactly the broad-search-failure case it exists to catch.
-    recordResearcherFailure(sessionId, researchId, id);
+    recordResearcherFailure(sessionId, researchId, id, 'no initial search results or historical links (search produced nothing usable)');
     metrics.increment('researcher_skipped_total', 1, { mode: 'deep', complexity: String(complexity), reason: 'no_initial_links' });
     observer?.onResearcherFailure?.(id, 'No initial search results or historical links available');
     return;