pi-oracle 0.6.0 → 0.6.2

package/CHANGELOG.md

@@ -2,6 +2,21 @@
 
 ## Unreleased
 
+## 0.6.2 - 2026-04-13
+
+### Changed
+- `/oracle` and `/oracle-followup` now treat pre-dispatch `archive_too_large` submit failures as retryable archive-selection misses instead of immediate dead ends, so agents can automatically narrow the archive and retry before surfacing the problem to the user
+
+### Fixed
+- archive-too-large submit errors now explain the 250 MB limit in human-readable terms, state that dispatch stopped before launch, and describe an ordered retry plan for shrinking the archive
+- oracle tool guidance, docs, and sanity coverage now align on only ending the turn after successful/queued `oracle_submit` results instead of accidentally stopping on a retryable oversize failure
+
+## 0.6.1 - 2026-04-13
+
+### Fixed
+- whole-repo archive expansion now merges very large entry groups iteratively instead of using spread/flat patterns that can overflow the JavaScript call stack during `oracle_submit`
+- oracle sanity coverage now guards the large-entry merge path so broad archive submissions regress to a real archive/env error instead of `Maximum call stack size exceeded`
+
 ## 0.6.0 - 2026-04-13
 
 ### Added

package/README.md

@@ -54,6 +54,8 @@ The `/oracle` prompt now runs an early oracle preflight before it gathers repo c
 
 For explicitly narrow requests, `/oracle` should still prefer a context-rich relevant archive up to the 250 MB ceiling, including nearby tests, docs, config, and adjacent modules when that can improve answer quality. Reserve tightly minimal archives for an explicit user request for a tight archive, privacy-sensitive material, or size-constrained cases. It should also omit `preset` and use the configured default model unless the task clearly needs a different one.
 
+If a local archive still exceeds the 250 MB limit after default exclusions and automatic whole-repo pruning, the agent should treat that as a retryable archive-selection failure: shrink the archive automatically, retry with a smaller relevant slice, and explain what it cut only if it still cannot fit after the allowed retry budget.
+
 If you miss the wake-up, the result is still saved durably in the oracle job directory and can be read later.
 
 ## Example requests

package/docs/ORACLE_DESIGN.md

@@ -115,9 +115,11 @@ Instead it instructs the agent to:
 5. gather enough repo context to submit well and bias toward context-rich archives when they fit within the 250 MB ceiling
 6. if the request is narrow, start from the directly relevant area but still include nearby tests, docs, config, and adjacent modules when they may improve answer quality
 7. if the request is broad/repo-wide, gather broader context and usually archive `.`
-8. craft the oracle prompt
-9. call `oracle_submit`
-10. stop and wait for the completion wake-up (best-effort; durable oracle response/artifact state is already persisted outside session history)
+8. if `oracle_submit` fails before dispatch with an `archive_too_large` / upload-limit error, treat that as retryable: use the reported size summary plus any auto-pruned paths to cut scope and retry automatically with a smaller archive
+9. stop retrying after at most two total submit attempts for the same request; if it still does not fit, report what was cut and why
+10. craft the oracle prompt
+11. call `oracle_submit`
+12. stop and wait for the completion wake-up (best-effort; durable oracle response/artifact state is already persisted outside session history)
 
 ### `/oracle-auth`
 
@@ -148,7 +150,7 @@ The authenticated seed profile remains the source of truth for future oracle run
 
 ### `oracle_submit`
 
-Agent-facing submissions use **`preset`**; the canonical registry is `ORACLE_SUBMIT_PRESETS` in `extensions/oracle/lib/config.ts`. **`preset` is the only model-selection parameter** on `oracle_submit`. There are no `modelFamily`, `effort`, or `autoSwitchToThinking` fields. Submit-time inputs accept canonical preset ids plus matching human-readable labels/common hyphen-space variants, and the tool normalizes them back to the canonical id before persisting job state. Prompt-template guidance biases toward omitting `preset` and using the configured default unless the task clearly needs a different model or the user explicitly asked for one. It also biases toward context-rich archives up to the 250 MB ceiling, narrowing only when the user explicitly asks for a tight archive, privacy/sensitivity requires it, or size pressure forces it.
+Agent-facing submissions use **`preset`**; the canonical registry is `ORACLE_SUBMIT_PRESETS` in `extensions/oracle/lib/config.ts`. **`preset` is the only model-selection parameter** on `oracle_submit`. There are no `modelFamily`, `effort`, or `autoSwitchToThinking` fields. Submit-time inputs accept canonical preset ids plus matching human-readable labels/common hyphen-space variants, and the tool normalizes them back to the canonical id before persisting job state. Prompt-template guidance biases toward omitting `preset` and using the configured default unless the task clearly needs a different model or the user explicitly asked for one. It also biases toward context-rich archives up to the 250 MB ceiling, narrowing only when the user explicitly asks for a tight archive, privacy/sensitivity requires it, or size pressure forces it. When local archive creation still exceeds that ceiling after default exclusions and whole-repo auto-pruning, prompt guidance now treats the failure as a retryable archive-selection miss rather than a terminal dead end: agents should cut scope automatically, retry once or twice, and only surface the cut decisions if the archive still cannot fit.
 
 1. resolve the preset (submit-time or config default) into an execution snapshot
 2. resolve optional `followUpJobId` into a prior `chatUrl` and `conversationId`

package/extensions/oracle/lib/jobs.ts

@@ -43,6 +43,8 @@ export const ORACLE_NOTIFICATION_CLAIM_TTL_MS = 60_000;
 export const ORACLE_WAKEUP_MAX_ATTEMPTS = 3;
 export const ORACLE_WAKEUP_RETRY_DELAYS_MS = [0, 15_000, 60_000] as const;
 export const ORACLE_WAKEUP_POST_SEND_RETENTION_MS = 2 * 60 * 1000;
+const ORACLE_JOB_DIR_RM_MAX_RETRIES = 5;
+const ORACLE_JOB_DIR_RM_RETRY_DELAY_MS = 50;
 const ORACLE_COMPLETE_JOB_RETENTION_MS = 14 * 24 * 60 * 60 * 1000;
 const ORACLE_FAILED_JOB_RETENTION_MS = 30 * 24 * 60 * 60 * 1000;
 export const DEFAULT_ORACLE_JOBS_DIR = "/tmp";
@@ -507,7 +509,12 @@ export async function removeTerminalOracleJob(job: OracleJob): Promise<{ removed
       }));
       return { removed: false, cleanupReport };
     }
-    await rm(getJobDir(current.id), { recursive: true, force: true });
+    await rm(getJobDir(current.id), {
+      recursive: true,
+      force: true,
+      maxRetries: ORACLE_JOB_DIR_RM_MAX_RETRIES,
+      retryDelay: ORACLE_JOB_DIR_RM_RETRY_DELAY_MS,
+    });
     return { removed: true, cleanupReport };
   });
 }

package/extensions/oracle/lib/tools.ts

@@ -155,6 +155,20 @@ type ArchiveCreationResult = {
   includedEntries: string[];
 };
 
+function appendArchiveEntries(target: string[], source: Iterable<string>): void {
+  for (const entry of source) target.push(entry);
+}
+
+function mergeArchiveEntryGroups(groups: Iterable<Iterable<string>>): string[] {
+  const merged: string[] = [];
+  for (const group of groups) appendArchiveEntries(merged, group);
+  return merged;
+}
+
+export function mergeArchiveEntryGroupsForTesting(groups: Iterable<Iterable<string>>): string[] {
+  return mergeArchiveEntryGroups(groups);
+}
+
 function pathContainsSequence(relativePath: string, sequence: readonly string[]): boolean {
   const segments = relativePath.split("/").filter(Boolean);
   if (sequence.length === 0 || segments.length < sequence.length) return false;
@@ -268,7 +282,7 @@ async function expandArchiveEntries(cwd: string, relativePath: string, options?:
     for (const child of children.sort((a, b) => a.name.localeCompare(b.name))) {
       const childRelative = child.name;
       if (await shouldExcludeArchiveChild(join(cwd, childRelative), childRelative, child)) continue;
-      if (child.isDirectory()) results.push(...await expandArchiveEntries(cwd, childRelative));
+      if (child.isDirectory()) appendArchiveEntries(results, await expandArchiveEntries(cwd, childRelative));
       else results.push(childRelative);
     }
     return results;
@@ -284,7 +298,7 @@ async function expandArchiveEntries(cwd: string, relativePath: string, options?:
   for (const child of children.sort((a, b) => a.name.localeCompare(b.name))) {
     const childRelative = posix.join(normalized, child.name);
     if (await shouldExcludeArchiveChild(join(cwd, childRelative), childRelative, child, { forceInclude: options?.forceIncludeSubtree })) continue;
-    if (child.isDirectory()) results.push(...await expandArchiveEntries(cwd, childRelative, { forceIncludeSubtree: options?.forceIncludeSubtree }));
+    if (child.isDirectory()) appendArchiveEntries(results, await expandArchiveEntries(cwd, childRelative, { forceIncludeSubtree: options?.forceIncludeSubtree }));
     else results.push(childRelative);
   }
   return results;
@@ -294,11 +308,12 @@ async function resolveExpandedArchiveEntriesFromInputs(
   cwd: string,
   entries: Array<{ absolute: string; relative: string }>,
 ): Promise<string[]> {
-  return Array.from(new Set((await Promise.all(entries.map(async (entry) => {
+  const expandedGroups = await Promise.all(entries.map(async (entry) => {
     const statEntry = await lstat(entry.absolute);
     const forceIncludeSubtree = statEntry.isDirectory() && entry.relative !== "." && shouldExcludeArchivePath(entry.relative, true);
     return expandArchiveEntries(cwd, entry.relative, { forceIncludeSubtree });
-  }))).flat())).sort();
+  }));
+  return Array.from(new Set(mergeArchiveEntryGroups(expandedGroups))).sort();
 }
 
 export async function resolveExpandedArchiveEntries(cwd: string, files: string[]): Promise<string[]> {
@@ -323,14 +338,15 @@ function formatArchiveOversizeError(args: {
   const topLevel = summarizeTopLevelIncludedPaths(args.entrySizes);
   const adaptiveCandidates = summarizeAdaptivePruneCandidates(args.entrySizes, args.adaptivePruneMinBytes).slice(0, 7);
   return [
-    `Oracle archive exceeds ChatGPT upload limit after default exclusions${args.autoPrunedPrefixes.length > 0 ? " and automatic generic generated-output-dir pruning" : ""}: ${args.archiveBytes} bytes >= ${args.maxBytes} bytes`,
+    `Oracle archive exceeds ChatGPT upload limit (${formatBytes(args.maxBytes)}) after default exclusions${args.autoPrunedPrefixes.length > 0 ? " and automatic generic generated-output-dir pruning" : ""}.`,
+    `The local archive measured ${formatBytes(args.archiveBytes)} (${args.archiveBytes} bytes), so submission stopped before dispatch.`,
     args.autoPrunedPrefixes.length > 0 ? "Automatically pruned generic generated-output paths before failing:" : undefined,
     ...args.autoPrunedPrefixes.map((entry) => `- ${formatDirectoryLabel(entry.relativePath)} — ${formatBytes(entry.bytes)}`),
     topLevel.length > 0 ? "Approx top-level included sizes:" : undefined,
     ...topLevel.map((entry) => `- ${entry.relativePath} — ${formatBytes(entry.bytes)}`),
     adaptiveCandidates.length > 0 ? "Largest remaining generic generated-output-dir candidates:" : undefined,
     ...adaptiveCandidates.map((entry) => `- ${formatDirectoryLabel(entry.relativePath)} — ${formatBytes(entry.bytes)}`),
-    "Retry with narrower archive inputs, starting with modified files plus adjacent files plus directly relevant subtrees.",
+    "Recommended retry order: (1) remove the largest obviously irrelevant/generated/history/export content, (2) if it still does not fit, keep only the directly relevant subtrees plus adjacent docs/tests/config, (3) if it still does not fit, explain what was cut before asking the user.",
   ]
     .filter(Boolean)
     .join("\n");
@@ -863,7 +879,7 @@ function buildOracleToolErrorDetails(toolName: OracleToolErrorSource, error: unk
     return {
       code: "archive_too_large",
       message,
-      suggestedNextStep: "Retry with a narrower archive, starting with modified files plus adjacent files plus directly relevant subtrees.",
+      suggestedNextStep: "This failure is retryable. Retry automatically with fewer selected files: first remove the largest obviously irrelevant/generated/history/export content, then if needed narrow to the directly relevant subtrees plus adjacent docs/tests/config, and if it still does not fit explain what was cut before asking the user.",
     };
   }
 
@@ -1057,9 +1073,12 @@ export function registerOracleTools(pi: ExtensionAPI, workerPath: string, authWo
       "Do not default to a one-file archive for a single function, file, or stack trace if the relevant surrounding context still fits comfortably within the limit.",
       "When files='.' and the post-exclusion archive is still too large, submit automatically prunes the largest nested directories matching generic generated-output names like build/, dist/, out/, coverage/, and tmp/ outside obvious source roots like src/ and lib/ until the archive fits or no candidate remains; successful submissions report what was pruned.",
       "If a submitted oracle job later fails because upload is rejected, retry smaller: remove the largest obviously irrelevant/generated content first, then narrow to modified files plus adjacent files plus directly relevant subtrees, then explain the cut or ask the user if still needed.",
-      "If oracle_submit itself fails because the local archive still exceeds the upload limit after default exclusions and automatic generic generated-output-dir pruning, or for any other submit-time error, stop and report the error instead of retrying automatically.",
+      "If oracle_submit fails before dispatch with details.error.code === 'archive_too_large' or an upload-limit message, treat that as retryable and retry automatically with a smaller archive.",
+      "For archive_too_large retries, use the reported top-level size summary and any auto-pruned paths to decide what to cut: first remove the largest obviously irrelevant/generated/history/export content, then if needed narrow to the directly relevant subtrees plus adjacent docs/tests/config.",
+      "Do not loop forever: stop after at most two total oracle_submit attempts for the same request, and if it still does not fit explain what was cut and why.",
+      "For any other submit-time error, stop and report the error instead of retrying automatically.",
       "If oracle_submit returns a queued job instead of an immediately dispatched one, treat that as success and stop exactly the same way.",
-      "Stop after dispatching oracle_submit; do not continue the task while the oracle job is running.",
+      "After a successful or queued oracle_submit, stop; do not continue the task while the oracle job is running. If oracle_submit failed with retryable archive_too_large, narrow the archive and retry first.",
       "Use `preset` as the only model-selection parameter on oracle_submit. Canonical ids are preferred, and matching human-readable preset labels are normalized automatically. Omit preset to use the configured default.",
     ],
     parameters: ORACLE_SUBMIT_PARAMS,

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "pi-oracle",
-  "version": "0.6.0",
+  "version": "0.6.2",
   "description": "ChatGPT web-oracle extension for pi with isolated browser auth, async jobs, and project-context archives.",
   "private": false,
   "license": "MIT",

package/prompts/oracle-followup.md

@@ -39,10 +39,12 @@ Rules:
 - If the request depends on git state or pending changes (for example code review, ship readiness, or release approval), create a tracked diff bundle file inside the repo (for example under `.pi/`) containing `git status` plus `git diff` output, include that file in the archive, and tell the oracle to use it because the `.git` directory is not included in oracle exports.
 - When `files=["."]` and the post-exclusion archive is still too large, submit automatically prunes the largest nested directories matching generic generated-output names like `build/`, `dist/`, `out/`, `coverage/`, and `tmp/` outside obvious source roots like `src/` and `lib/` until the archive fits or no candidate remains. Successful submissions report what was pruned.
 - If a submitted oracle job later fails because upload is rejected, retry with a smaller archive in this order: (1) remove the largest obviously irrelevant/generated content, (2) if still too large, include modified files plus adjacent files plus directly relevant subtrees, (3) if still too large, explain the cut or ask the user.
+- If `oracle_submit` fails before dispatch with `details.error.code === "archive_too_large"` or an upload-limit message, that failure is retryable. Use the reported top-level size summary and any auto-pruned paths to choose a smaller archive and retry automatically.
+- For archive-too-large retries, cut scope in this order: (1) if you used `.`, remove the largest obviously irrelevant/generated/history/export/report content while preserving relevant source/docs/config/tests, (2) if it still does not fit, archive only the directly relevant subtrees plus adjacent docs/tests/config, (3) if it still does not fit after at most two total `oracle_submit` attempts, report what you cut and why.
 - Prefer the configured default (omit **`preset`**) unless the task clearly needs a different model or the user explicitly asked for one; then choose a canonical **`preset`** id.
-- If `oracle_submit` itself fails because the local archive still exceeds the upload limit after default exclusions and automatic generic generated-output-dir pruning, or for any other submit-time error, stop and report the error. Do not retry automatically.
+- For any other `oracle_submit` submit-time error, stop and report the error. Do not retry automatically.
 - If `oracle_submit` returns a queued job instead of an immediately dispatched one, treat that as success and end your turn exactly the same way.
-- After `oracle_submit` returns, end your turn. Do not keep working while the oracle runs.
+- After a successful or queued `oracle_submit`, end your turn. Do not keep working while the oracle runs. If `oracle_submit` failed with a retryable archive-too-large error, shrink the archive and retry first.
 
 User request:
 $@

package/prompts/oracle.md

@@ -35,10 +35,12 @@ Rules:
 - If the request depends on git state or pending changes (for example code review, ship readiness, or release approval), create a tracked diff bundle file inside the repo (for example under `.pi/`) containing `git status` plus `git diff` output, include that file in the archive, and tell the oracle to use it because the `.git` directory is not included in oracle exports.
 - When `files=["."]` and the post-exclusion archive is still too large, submit automatically prunes the largest nested directories matching generic generated-output names like `build/`, `dist/`, `out/`, `coverage/`, and `tmp/` outside obvious source roots like `src/` and `lib/` until the archive fits or no candidate remains. Successful submissions report what was pruned.
 - If a submitted oracle job later fails because upload is rejected, retry with a smaller archive in this order: (1) remove the largest obviously irrelevant/generated content, (2) if still too large, include modified files plus adjacent files plus directly relevant subtrees, (3) if still too large, explain the cut or ask the user.
+- If `oracle_submit` fails before dispatch with `details.error.code === "archive_too_large"` or an upload-limit message, that failure is retryable. Use the reported top-level size summary and any auto-pruned paths to choose a smaller archive and retry automatically.
+- For archive-too-large retries, cut scope in this order: (1) if you used `.`, remove the largest obviously irrelevant/generated/history/export/report content while preserving relevant source/docs/config/tests, (2) if it still does not fit, archive only the directly relevant subtrees plus adjacent docs/tests/config, (3) if it still does not fit after at most two total `oracle_submit` attempts, report what you cut and why.
 - Prefer the configured default (omit **`preset`**) unless the task clearly needs a different model or the user explicitly asked for one; then choose a canonical **`preset`** id.
-- If `oracle_submit` itself fails because the local archive still exceeds the upload limit after default exclusions and automatic generic generated-output-dir pruning, or for any other submit-time error, stop and report the error. Do not retry automatically.
+- For any other `oracle_submit` submit-time error, stop and report the error. Do not retry automatically.
 - If `oracle_submit` returns a queued job instead of an immediately dispatched one, treat that as success and end your turn exactly the same way.
-- After oracle_submit returns, end your turn. Do not keep working while the oracle runs.
+- After a successful or queued `oracle_submit`, end your turn. Do not keep working while the oracle runs. If `oracle_submit` failed with a retryable archive-too-large error, shrink the archive and retry first.
 
 User request:
 $@