@companion-ai/feynman 0.5.17 → 0.5.19

package/.feynman/agents/researcher.md

@@ -31,7 +31,7 @@ You are Feynman's evidence-gathering subagent.
 | Biomedical papers | `source: "pubmed"`, then `source: "europepmc"` for open-access full-text sections | `semanticscholar` for citation counts |
 | Citation graph | `source: "openalex"` with `openalex_citations:` / `openalex_references:` | `semanticscholar` citation counts to spot seminal work |
 | Conceptual or recent work that keyword search misses | `source: "openalex"` with a `semantic:` query prefix | `semanticscholar` with `sort: "relevance"` |
-| Web, docs, repos, grey literature | `web_search` (if the default provider fails or is rate-limited, retry with `provider: "parallel-mcp"`, which needs no key) | `fetch_content` on the best results |
+| Web, docs, repos, grey literature | `web_search` (if the default provider fails or is rate-limited, retry with `provider` set to the plain string `parallel-mcp`, not a JSON-encoded list; it needs no key) | `fetch_content` on the best results |
 | Known paper ID | `source: "arxiv"` for arXiv IDs, `source: "crossref"` for DOIs | `fetch_content` on `arxiv.org/html/<id>` for full text |
 
 Run 2–4 reworded queries for each question (synonyms, the method's name, the problem's name) and merge the results. Do not trust one query's ranking; seminal papers often appear only under one phrasing or one sort order.

package/RELEASES.md

@@ -6,6 +6,19 @@ GitHub release notes are generated from the matching `## vX.Y.Z` section in this
 
 ## Unreleased
 
+## v0.5.19 - 2026-10-01
+
+- **Word drafts open without LibreOffice.** `document_parse` needs LibreOffice for `.docx` files, which most machines lack, so reviewing a thesis or draft in Word format failed. When LibreOffice is missing, Feynman now reads the document's paragraph text directly from the `.docx` and says so; install LibreOffice for page layout, tables, footnotes, and images.
+- `feynman_science_database_search` accepts the `"null"` some models send for an omitted `sort` or `limit`, which failed validation before the search ran.
+- **Pi 0.99.2** (faster prompt submission in long sessions; saved default models with extension providers load reliably), **pi-subagents 0.74.0**, and **pi-btw 0.7.0**. pi-subagents now ships the patched undici 8.10.2, so the undici security exceptions and Feynman's own undici pin are gone. It removed the `workflowScript` parameter; `/deepresearch` and `/summarize` now write the workflow script to `outputs/.plans/<slug>-workflow.js` and run it by path. Its other new form, an inline ```` ```js workflow ```` block, failed in testing when Claude put the block in its hidden reasoning, while a script file worked with every model, and the file stays next to the plan as a record of the run. Subagent children also follow the project's trust setting, and `/reload` no longer stops running background workflows.
+- Security: Pi 0.99.2 still locks brace-expansion 5.0.9; the exception for it stays until Pi ships 5.0.12.
+
+## v0.5.18 - 2026-09-30
+
+- **A rate-limited paper index no longer ends a research step.** When Semantic Scholar is rate-limited or down, a search is answered from OpenAlex, and an OpenAlex topic search that hits OpenAlex's anonymous rate limit or an outage is answered from Semantic Scholar. The result says first which index answered and why. Lookups by OpenAlex ID, DOI, author, venue, or citation still report the error, since the other index has no equivalent.
+- **alphaXiv tools answer when alphaXiv cannot.** If your alphaXiv login expired or alphaXiv does not respond, `alpha_search` returns OpenAlex semantic search results and `alpha_get_paper` returns the arXiv metadata and abstract for an arXiv ID, each marked as a fallback. Run `feynman alpha login` to get alphaXiv's reports back.
+- pi-web-access 0.35.0: OpenAI web search works when you signed in with ChatGPT (`feynman model login openai`), and Brave search keeps to its rate limits. The researcher is told to pass a search provider as a plain string, since some models sent `provider: "[\"parallel-mcp\"]"`, which `web_search` rejects; nicobailon/pi-web-access#491 makes `web_search` accept that form too.
+
 ## v0.5.17 - 2026-09-30
 
 - **`feynman model login openai` works again.** Since Pi 0.99, Sign in with ChatGPT needs a stable ID for the installation, which Feynman did not pass, so the login failed at once with "requires a device ID (UUID)". Feynman now shares the `deviceId` Pi's own `/login` keeps in `~/.feynman/agent/settings.json`.

package/extensions/research-tools/alpha.ts

@@ -13,6 +13,8 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 import { Type } from "typebox";
 
 import { extractPaperSections } from "./alpha-sections.js";
+import { searchArxiv } from "./science-database-arxiv.js";
+import { searchOpenAlex } from "./science-database-openalex.js";
 
 function formatText(value: unknown): string {
 	if (typeof value === "string") return value;
@@ -21,11 +23,23 @@ function formatText(value: unknown): string {
 
 // alphaXiv's API has changed without notice before; when a call fails, point
 // the model at the other paper sources instead of leaving it on a broken tool.
-async function withPaperFallback<T>(run: () => Promise<T>): Promise<T> {
+const ARXIV_ID = /(?:^|arxiv\.org\/(?:abs|pdf)\/|alphaxiv\.org\/(?:abs|overview)\/)(\d{4}\.\d{4,5}(?:v\d+)?|[a-z-]+(?:\.[A-Z]{2})?\/\d{7}(?:v\d+)?)/i;
+
+// When alphaXiv does not answer (expired login, outage), answer from another
+// paper source and say so first, instead of failing the research step.
+async function withPaperFallback<T>(
+	run: () => Promise<T>,
+	fallback?: { source: string; run: () => Promise<Record<string, unknown>> },
+): Promise<T | Record<string, unknown>> {
 	try {
 		return await run();
 	} catch (error) {
 		const message = (error instanceof Error ? error.message : String(error)).replace("`alpha login`", "`feynman alpha login`");
+		if (fallback) {
+			try {
+				return { fallbackNote: `alphaXiv did not answer (${message}); these results are from ${fallback.source}.`, ...(await fallback.run()) };
+			} catch {}
+		}
 		throw new Error(
 			`${message}\nalphaXiv did not answer this call. Use feynman_science_database_search (source "arxiv" or "semanticscholar") or fetch_content on https://arxiv.org/abs/<id> instead.`,
 		);
@@ -65,7 +79,10 @@ export function registerAlphaTools(pi: ExtensionAPI, signedIn = isLoggedIn()): v
 			),
 		}),
 		async execute(_toolCallId, params) {
-			const result = await withPaperFallback(() => searchPapers(params.query, params.mode?.trim() || "semantic"));
+			const result = await withPaperFallback(() => searchPapers(params.query, params.mode?.trim() || "semantic"), {
+				source: "OpenAlex semantic search",
+				run: () => searchOpenAlex({ source: "openalex", query: `semantic: ${params.query}`, limit: 10 }),
+			});
 			return { content: [{ type: "text", text: formatText(result) }], details: result };
 		},
 	});
@@ -87,7 +104,14 @@ export function registerAlphaTools(pi: ExtensionAPI, signedIn = isLoggedIn()): v
 			sections: Type.Optional(paperSectionsSchema),
 		}),
 		async execute(_toolCallId, params) {
-			const result = await withPaperFallback(() => getPaper(params.paper, { fullText: params.fullText }));
+			const arxivId = ARXIV_ID.exec(params.paper.trim())?.[1];
+			const result = await withPaperFallback(() => getPaper(params.paper, { fullText: params.fullText }), arxivId ? {
+				source: "arXiv metadata and abstract (no full text or sections)",
+				run: () => searchArxiv({ query: arxivId }),
+			} : undefined);
+			if ("fallbackNote" in result) {
+				return { content: [{ type: "text", text: formatText(result) }], details: result };
+			}
 			const extracted = extractPaperSections(result.content, params.section, params.sections);
 			const filteredResult = extracted.requested.length
 				? {

package/extensions/research-tools/docx-fallback.ts

@@ -0,0 +1,76 @@
+import { readFileSync } from "node:fs";
+import { resolve } from "node:path";
+import { inflateRawSync } from "node:zlib";
+
+import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
+
+// Reads one stored or deflated entry through the zip central directory.
+function readZipEntry(zip: Buffer, name: string): Buffer | undefined {
+	const end = zip.lastIndexOf(Buffer.from([0x50, 0x4b, 0x05, 0x06]));
+	if (end < 0) return undefined;
+	let offset = zip.readUInt32LE(end + 16);
+	for (let index = zip.readUInt16LE(end + 10); index > 0; index -= 1) {
+		if (zip.readUInt32LE(offset) !== 0x02014b50) return undefined;
+		const method = zip.readUInt16LE(offset + 10);
+		const size = zip.readUInt32LE(offset + 20);
+		const nameLength = zip.readUInt16LE(offset + 28);
+		const local = zip.readUInt32LE(offset + 42);
+		if (zip.toString("utf8", offset + 46, offset + 46 + nameLength) === name) {
+			const start = local + 30 + zip.readUInt16LE(local + 26) + zip.readUInt16LE(local + 28);
+			const data = zip.subarray(start, start + size);
+			return method === 0 ? data : method === 8 ? inflateRawSync(data) : undefined;
+		}
+		offset += 46 + nameLength + zip.readUInt16LE(offset + 30) + zip.readUInt16LE(offset + 32);
+	}
+	return undefined;
+}
+
+function decodeXml(text: string): string {
+	return text.replace(/&(?:#x([0-9a-f]+)|#(\d+)|(amp|lt|gt|quot|apos));/gi, (_match, hex: string, dec: string, named: string) => {
+		if (hex) return String.fromCodePoint(Number.parseInt(hex, 16));
+		if (dec) return String.fromCodePoint(Number(dec));
+		return { amp: "&", lt: "<", gt: ">", quot: '"', apos: "'" }[named.toLowerCase()] ?? "";
+	});
+}
+
+// Paragraph text of a .docx body: runs, tabs, and line breaks; no tables'
+// layout, headers, footnotes, or images.
+export function docxText(file: Buffer): string | undefined {
+	const xml = readZipEntry(file, "word/document.xml")?.toString("utf8");
+	if (!xml) return undefined;
+	return xml
+		.split("</w:p>")
+		.map((paragraph) =>
+			[...paragraph.matchAll(/<w:t(?:\s[^>]*)?>([^<]*)<\/w:t>|<w:tab\/>|<w:br\/>/g)]
+				.map((match) => (match[0] === "<w:tab/>" ? "\t" : match[0] === "<w:br/>" ? "\n" : decodeXml(match[1] ?? "")))
+				.join(""),
+		)
+		.join("\n")
+		.replace(/\n{3,}/g, "\n\n")
+		.trim();
+}
+
+// document_parse needs LibreOffice for Word files, which most machines lack.
+// Answer a .docx from its own XML instead of failing the read.
+export function registerDocxFallback(pi: ExtensionAPI): void {
+	pi.on("tool_result", (event, ctx) => {
+		if (event.toolName !== "document_parse" || !event.isError) return undefined;
+		const errorText = event.content.map((part) => (part.type === "text" ? part.text : "")).join("\n");
+		const path = typeof event.input.path === "string" ? event.input.path.replace(/^@/, "") : "";
+		if (!errorText.includes("LibreOffice is not installed") || !/\.doc[xm]$/i.test(path)) return undefined;
+		let text: string | undefined;
+		try {
+			text = docxText(readFileSync(resolve(ctx.cwd, path)));
+		} catch {
+			return undefined;
+		}
+		if (!text) return undefined;
+		return {
+			isError: false,
+			content: [{
+				type: "text",
+				text: `LibreOffice is not installed, so this is the document's paragraph text read directly from the .docx (no page layout, table structure, headers, footnotes, or images). Install LibreOffice for a full parse.\n\n${text}`,
+			}],
+		};
+	});
+}

package/extensions/research-tools/science-databases.ts

@@ -380,19 +380,54 @@ async function searchSemanticScholar(params: ScienceDatabaseSearchParams): Promi
 	};
 }
 
+// OpenAlex lookups by ID, DOI, author, venue, or citation have no Semantic
+// Scholar equivalent; only plain topic searches fall back.
+const OPENALEX_LOOKUP_QUERY =
+	/^(?:openalex_\w+|rate-limit|authors?|author-search|sources?|venues?|citations?|references?|work|detail)(?::|\s|$)|^(?:W\d+|doi:\s*10\.|10\.\S+\/)/i;
+
+function isIndexUnavailable(error: unknown): boolean {
+	if (error instanceof ScienceDatabaseRequestError) return error.status === 429 || error.status >= 500;
+	return error instanceof Error && /^OpenAlex request failed: (?:429|5\d\d)\b/.test(error.message);
+}
+
+// A rate-limited or overloaded index should not end a research step: answer a
+// topic search from the other general index, and say so first in the result.
+async function withIndexFallback(
+	run: () => Promise<Record<string, unknown>>,
+	fallback: (() => Promise<Record<string, unknown>>) | undefined,
+	describe: (reason: string) => string,
+): Promise<Record<string, unknown>> {
+	try {
+		return await run();
+	} catch (error) {
+		if (!fallback || !isIndexUnavailable(error)) throw error;
+		let result: Record<string, unknown>;
+		try {
+			result = await fallback();
+		} catch {
+			throw error;
+		}
+		return { fallbackNote: describe(error instanceof Error ? error.message : String(error)), ...result };
+	}
+}
+
 async function scienceDatabaseSearch(params: ScienceDatabaseSearchParams): Promise<Record<string, unknown>> {
 	if (params.source === "arxiv") return searchArxiv(params);
 	if (params.source === "biorxiv") return searchPreprints(params, "biorxiv");
 	if (params.source === "medrxiv") return searchPreprints(params, "medrxiv");
 	if (params.source === "pubmed") return searchPubMed(params);
 	if (params.source === "crossref") return searchCrossref(params);
-	if (params.source === "semanticscholar") return searchSemanticScholar(params);
+	const openAlex = () => searchOpenAlex({ limit: params.limit, query: params.query, source: "openalex" });
+	const semanticScholar = () => searchSemanticScholar({ ...params, source: "semanticscholar" });
+	if (params.source === "semanticscholar") {
+		return withIndexFallback(semanticScholar, openAlex, (reason) => `Semantic Scholar was unavailable (${reason}); these results are from OpenAlex.`);
+	}
 	if (params.source === "openalex") {
-		return searchOpenAlex({
-			limit: params.limit,
-			query: params.query,
-			source: params.source,
-		});
+		return withIndexFallback(
+			openAlex,
+			OPENALEX_LOOKUP_QUERY.test(params.query.trim()) ? undefined : semanticScholar,
+			(reason) => `OpenAlex was unavailable (${reason}); these results are from Semantic Scholar.`,
+		);
 	}
 	return searchEuropePmc(params);
 }
@@ -410,6 +445,17 @@ export function registerScienceDatabaseTools(pi: ExtensionAPI): void {
 			"Preserve returned PMIDs, PMCIDs, DOIs, arXiv IDs, Semantic Scholar paper IDs, preprint DOIs, OpenAlex W/A/S IDs, author ORCIDs, citation/reference counts, OA status, Europe PMC full-text statuses and section inventories, source URLs, and endpoint provenance in research artifacts and answers.",
 			"Treat database summaries as retrieval evidence, then verify decisive claims against the full paper when needed.",
 		],
+		// Some models fill omitted optional fields with "null", which fails validation.
+		prepareArguments: (args) => {
+			if (!args || typeof args !== "object" || Array.isArray(args)) return args as never;
+			const input = { ...(args as Record<string, unknown>) };
+			for (const key of ["sort", "limit"]) {
+				if (input[key] === null || input[key] === "" || input[key] === "null") delete input[key];
+			}
+			if (typeof input.sort === "string") input.sort = input.sort.trim().toLowerCase();
+			if (typeof input.limit === "string" && /^\d+$/.test(input.limit.trim())) input.limit = Number(input.limit);
+			return input as never;
+		},
 		parameters: Type.Object({
 			source: SCIENCE_DATABASE_SOURCE_SCHEMA,
 			query: Type.String({

package/extensions/research-tools.ts

@@ -3,6 +3,7 @@ import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
 import { registerAlphaTools } from "./research-tools/alpha.js";
 import { registerCurrentDateResearchContext } from "./research-tools/current-date.js";
 import { registerDiscoveryCommands } from "./research-tools/discovery.js";
+import { registerDocxFallback } from "./research-tools/docx-fallback.js";
 import { installFeynmanHeader } from "./research-tools/header.js";
 import { registerHelpCommand } from "./research-tools/help.js";
 import { registerHuggingFaceTools } from "./research-tools/huggingface.js";
@@ -23,6 +24,7 @@ export default function researchTools(pi: ExtensionAPI): void {
 	registerCurrentDateResearchContext(pi);
 	registerHuggingFaceTools(pi);
 	registerDiscoveryCommands(pi);
+	registerDocxFallback(pi);
 	registerHelpCommand(pi);
 	registerInitCommand(pi);
 	registerOutputsCommand(pi);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@companion-ai/feynman",
-  "version": "0.5.17",
+  "version": "0.5.19",
   "description": "Research-first CLI agent built on Pi and alphaXiv",
   "license": "MIT",
   "type": "module",
@@ -80,16 +80,15 @@
     "@clack/prompts": "^1.8.1",
     "@companion-ai/alpha-hub": "0.1.6",
     "fast-xml-parser": "5.11.1",
-    "pi-btw": "0.6.1",
+    "pi-btw": "0.7.0",
     "pi-docparser": "4.0.0",
-    "pi-subagents": "0.73.1",
-    "pi-web-access": "0.34.0",
-    "posthog-node": "^5.53.0",
-    "undici": "^8.11.2"
+    "pi-subagents": "0.74.0",
+    "pi-web-access": "0.35.0",
+    "posthog-node": "^5.53.0"
   },
   "peerDependencies": {
-    "@earendil-works/pi-ai": "0.99.1",
-    "@earendil-works/pi-coding-agent": "0.99.1",
+    "@earendil-works/pi-ai": "0.99.2",
+    "@earendil-works/pi-coding-agent": "0.99.2",
     "@earendil-works/pi-tui": "*",
     "typebox": "*"
   },
@@ -114,8 +113,5 @@
   "author": "Advait Paliwal",
   "publishConfig": {
     "access": "public"
-  },
-  "overrides": {
-    "undici": "$undici"
   }
 }

package/prompts/deepresearch.md

@@ -68,18 +68,23 @@ If subagents were chosen:
 - Keep `subagent` tool-call JSON small and valid.
 - Do not place multi-paragraph instructions inside the `subagent` JSON.
 - Use only supported `subagent` keys. Do not add extra keys such as `artifacts` unless the tool schema explicitly exposes them.
-- Use one async `workflowScript` with `await runs.all(...)` for parallel evidence gathering. Each item needs a unique stable `key`, plus its agent, short task, and output path. Set `globalConcurrencyLimit: 4` on the outer call.
+- For parallel evidence gathering, write a workflow script that returns `await runs.all(...)` to `outputs/.plans/<slug>-workflow.js` with the write tool. After the write succeeds, call `subagent` with `workflow: "./outputs/.plans/<slug>-workflow.js"`, `async: true`, and `globalConcurrencyLimit: 4`. Each item needs a unique stable `key`, plus its agent, short task, and output path. Do not pass the script text inside the tool call.
 - Read the ordered result array and record each child's `ok`, error, and returned output/artifact paths. Ordinary child failures are collected by `runs.all`; validation or infrastructure failure can still fail the workflow. Do not assume every output exists.
 - Prefer broad guidance such as "use paper search and web search"; if a PDF parser or paper fetch fails, the researcher must continue from metadata, abstracts, and web sources and mark PDF parsing as blocked.
 
-Example shape:
+Example shape: `outputs/.plans/<slug>-workflow.js` contains
+
+```js
+return await runs.all([
+  { key: "web", agent: "researcher", task: "Read outputs/.plans/<slug>-T1.md and write outputs/.drafts/<slug>-research-web.md.", output: "outputs/.drafts/<slug>-research-web.md" },
+  { key: "papers", agent: "researcher", task: "Read outputs/.plans/<slug>-T2.md and write outputs/.drafts/<slug>-research-papers.md.", output: "outputs/.drafts/<slug>-research-papers.md" },
+]);
+```
+
+and the `subagent` call is:
 
 ```json
-{
-  "workflowScript": "return await runs.all([{key:'web',agent:'researcher',task:'Read outputs/.plans/<slug>-T1.md and write outputs/.drafts/<slug>-research-web.md.',output:'outputs/.drafts/<slug>-research-web.md'},{key:'papers',agent:'researcher',task:'Read outputs/.plans/<slug>-T2.md and write outputs/.drafts/<slug>-research-papers.md.',output:'outputs/.drafts/<slug>-research-papers.md'}]);",
-  "async": true,
-  "globalConcurrencyLimit": 4
-}
+{ "workflow": "./outputs/.plans/<slug>-workflow.js", "async": true, "globalConcurrencyLimit": 4 }
 ```
 
 Continue independent work after launch, then consume completion results before synthesis. Relative output paths resolve against the workspace, so each child writes its declared path; verify each file on disk, and use the returned output reference for any that is missing. After evidence gathering, update the plan ledger and verification log. If research failed, record exactly what failed and proceed with a blocked or partial draft.

package/prompts/summarize.md

@@ -116,14 +116,12 @@ Briefly summarize: "Source is ~<chars> chars -> <N> chunks -> <N> researcher sub
 
 ### 3c. Dispatch researcher subagents
 
-Use one async workflow with a stable key for each chunk and a concurrency limit of four. Expand the example item below for the actual chunk inventory; retain the single-source restriction and explicit output declaration for every child.
-
-```json
-{
-  "workflowScript": "return await runs.all([{key:'chunk-NNN',agent:'researcher',task:'Read ONLY outputs/.notes/<slug>-chunk-NNN.txt. Extract: (1) key claims, (2) methodology or technical approach, (3) cited evidence. Do NOT use web_search or fetch external URLs — this is single-source summarization. If a claim appears to start or end mid-sentence at the file boundary, mark it BOUNDARY PARTIAL. Write to outputs/.notes/<slug>-summary-chunk-NNN.md.',output:'outputs/.notes/<slug>-summary-chunk-NNN.md'}]);",
-  "async": true,
-  "globalConcurrencyLimit": 4
-}
+Use one async workflow with a stable key for each chunk and a concurrency limit of four: write the script to `outputs/.plans/<slug>-workflow.js` with the write tool, then, after the write succeeds, call `subagent` with `{ "workflow": "./outputs/.plans/<slug>-workflow.js", "async": true, "globalConcurrencyLimit": 4 }`. Expand the example item below for the actual chunk inventory; retain the single-source restriction and explicit output declaration for every child.
+
+```js
+return await runs.all([
+  { key: "chunk-NNN", agent: "researcher", task: "Read ONLY outputs/.notes/<slug>-chunk-NNN.txt. Extract: (1) key claims, (2) methodology or technical approach, (3) cited evidence. Do NOT use web_search or fetch external URLs — this is single-source summarization. If a claim appears to start or end mid-sentence at the file boundary, mark it BOUNDARY PARTIAL. Write to outputs/.notes/<slug>-summary-chunk-NNN.md.", output: "outputs/.notes/<slug>-summary-chunk-NNN.md" },
+]);
 ```
 
 Consume completion results before aggregation. `runs.all` returns an ordered array including ordinary child failures; inspect each `ok` and error. A validation or infrastructure failure can still fail the workflow. Relative output paths resolve against the workspace; verify each chunk summary on disk, and use the returned output reference for any that is missing. Record missing or failed chunks rather than treating an async launch receipt as a finished summary.