pi-openai-codex-compat 0.0.9 → 0.0.10-alpha.1

package/APPLY_PATCH_INSTRUCTION_FEEDBACK.md

@@ -17,8 +17,8 @@ Feedback must:
 1. retain the aggregate changed-file summary;
 2. when an instruction ledger is needed, report every instruction without a
    model-facing limit;
-3. attribute filesystem effects, errors, and matching evidence to the
-   instruction that produced them;
+3. attribute filesystem effects and errors to the instruction that produced
+   them;
 4. report failed and not-run instructions;
 5. use concise, simplified technical English;
 6. report deterministic facts rather than speculative outcomes; and
@@ -134,8 +134,7 @@ Add a short clause only for:
 - a failure;
 - a non-obvious externally visible filesystem effect;
 - a partial effect;
-- a final state that is not verified; or
-- concise matcher evidence.
+- a final state that is not verified.
 
 Do not include:
 
@@ -143,7 +142,6 @@ Do not include:
 - a diff-availability warning;
 - verbose proof sections;
 - a duplicated raw error section;
-- a detached matcher section; or
 - prose that merely restates an ordinary successful operation.
 
 ## Terminology
@@ -227,7 +225,6 @@ Each instruction result stores:
 - completed externally visible filesystem effects;
 - relevant final path states;
 - an optional error;
-- optional matcher evidence; and
 - related instruction numbers.
 
 The operation label is sufficient for ordinary successful effects. Explicit
@@ -346,37 +343,17 @@ Do not tell the model that previous content was unreadable, a diff is
 unavailable, history is incomplete, or the result is inexact. Diff
 availability is a TUI/history concern, not an operation result.
 
-## Matcher feedback
-
-Matcher feedback belongs to the failed update instruction. Do not emit a
-detached `Matcher diagnostics:` section or repeat old/replacement patch text.
-
-Examples:
+## Matching failures
 
-```text
-3. [FAILED] Update file.ts - Old content was not found. Read the current file and use apply_patch again with updated instructions if needed.
-```
+A strict matching failure belongs to the failed update instruction. Do not
+emit a detached matcher-diagnostics section or repeat old/replacement patch
+text:
 
 ```text
-3. [FAILED] Update file.ts - The requested changes match in reverse source-file order at lines 10-12 and lines 30-32. Use apply_patch again with the requested changes in source-file order if needed.
+3. [FAILED] Update file.ts - Old content was not found.
 ```
 
-```text
-3. [FAILED] Update file.ts - Matching locations at lines 10-12 and lines 40-42 produce different results. Use apply_patch again with more specific surrounding context or smaller changes if needed.
-```
-
-```text
-3. [FAILED] Update file.ts - More than 256 possible ways to apply the requested changes were found. Use apply_patch again with more specific surrounding context or smaller changes if needed.
-```
-
-```text
-3. [FAILED] Update file.ts - Requested replacement found at lines 40-44, but old content was not found. Inspect the reported lines and use apply_patch again with updated instructions if needed.
-```
-
-The 64-location limit, different-result ambiguity, and 256-way limit use the
-same guidance because each requires more specific surrounding context or
-smaller requested changes. Reverse-order and overlap failures instead tell the
-model to reorder or separate the requested changes.
+Missing `@@` anchors analogously report `Context was not found.`.
 
 ## Not-run results
 
@@ -457,7 +434,6 @@ Expanded rendering nests these details beneath the relevant instruction:
 - applied textual diffs;
 - non-obvious filesystem effects;
 - final-state verification;
-- concise matcher locations; and
 - useful system-error details.
 
 Partial effects are not rendered as detached successful operations. Failed
@@ -483,7 +459,6 @@ type InstructionResult = {
   effects: InstructionEffect[];
   finalStates: FinalPathState[];
   error?: InstructionError;
-  matching?: MatchingEvidence;
   relatedInstructions?: number[];
 };
 ```
@@ -512,18 +487,6 @@ Only presentation differs:
 
 Semantic wording and facts remain shared.
 
-## Matcher candidate deduplication
-
-Before applying candidate and mapping limits:
-
-1. canonicalize candidates by complete byte-edit effect;
-2. remove byte-identical duplicate candidates;
-3. retain every semantically distinct candidate; and
-4. do not rank candidates or use heuristic preference.
-
-This prevents duplicate line-level and structural candidates from consuming
-the exhaustive mapping bound.
-
 ## Required tests
 
 ### Completeness
@@ -552,7 +515,6 @@ Generated model and TUI feedback does not contain:
 - `installed`
 - `dead`
 - `dominated`
-- `Matcher diagnostics`
 - the heading `Instructions:`
 
 When the conditional ledger is present, its heading is
@@ -583,8 +545,8 @@ Cover:
 - temporary entry remains without a false no-change statement; and
 - post-operation verification failure.
 
-Every formatter-matcher failure reason is covered with its direct retry
-guidance.
+Strict mismatch coverage verifies the source is unchanged and the original
+context or expected-lines error is reported.
 
 ### No patch-content repetition
 
@@ -602,16 +564,10 @@ With `applyPatchDebug` enabled:
 - changing the setting updates existing result components; and
 - partial previews remain unchanged.
 
-### Matcher deduplication
-
-A multi-group formatter-recovery fixture proves duplicate candidates do not
-consume the mapping limit.
-
 ## Documentation
 
 Implementation changes must also update:
 
 - `APPLY_PATCH_SEMANTIC_OPERATIONS.md`
-- `APPLY_PATCH_REMAINING_WORK.md`
 - `README.md`
 - `CHANGELOG.md`

package/CHANGELOG.md

@@ -2,6 +2,44 @@
 
 ## Unreleased
 
+### Fixed
+
+- Preserve regular-file permission modes across sequential `apply_patch`
+  replacements and state-changing moves, including files created by earlier
+  instructions in the same patch.
+- Resolve descendants through symlink parents established by earlier
+  instructions, keeping subsequent aliased operations in one virtual
+  filesystem state.
+- Resolve relative symlink targets from the canonical source or moved
+  destination directory when a parent path is itself a symlink.
+- Verify the complete final byte buffer after `apply_patch` adds and text
+  updates, so requested bytes appearing in a duplicate or unrelated region
+  cannot satisfy the result postcondition.
+- Classify repeated identical `apply_patch` adds from source-ordered virtual
+  content and exact spelling, avoiding redundant replacements after earlier
+  adds or moves while preserving case, Unicode, and symlink-parent identity.
+- Use actual or planned directory-entry spelling when proving dead
+  `apply_patch` operations and classifying same-entry moves on case-insensitive
+  or Unicode-normalizing filesystems.
+- Validate supplied identity and context chunks before `apply_patch` moves while preserving chunkless opaque moves and successful symlink and hard-link topology.
+- Remove formatter-tolerant `apply_patch` matching permanently and rely exclusively on the official Codex-compatible exact, trailing-trim, full-trim, and Unicode matcher.
+- Treat streamed Codex misalignment-policy violations as terminal without retrying and preserve the provider's error message.
+- Preserve unexpected parser, matcher, and asynchronous transport failures instead of replacing or detaching them.
+- Recover expired 60-minute Codex WebSockets in the provider response loop, retaining committed items, discarding provisional output, and waiting for Pi to record completed tool outputs before continuing.
+- Validate complete apply-patch result details against schema-derived contracts before rendering, including integer metadata and nested instruction relationships.
+- Distinguish an unavailable Pi compaction append capability from a missing percentage-compaction retention boundary.
+- Validate supported Responses input, completed-output, content, and tool-definition variants with closed schema-derived unions, failing closed on unknown or malformed item types while preserving additional fields on known variants.
+
+### Changed
+
+- Simplify `apply_patch` for its queued single-writer operating model: retain
+  Pi's mutation queue, remove the extension-local alias queue and
+  preflight-to-execution drift machinery, use direct writes for in-place text
+  updates, and preserve source-ordered no-change checkpoints without
+  filesystem revalidation.
+- Adopt the shared 2h2d Oxlint policy and exact supported Oxlint and Oxfmt versions.
+- Run isolated test files across four workers, bypass unasserted wall-clock response-retry waits in integration coverage, and assert the retry-delay calculation directly.
+
 ## 0.0.9 - 2026-08-16
 
 ### Fixed

package/README.md

@@ -31,7 +31,7 @@ Authenticate through Pi if needed:
 
 ## Compatibility baseline and differences
 
-The compatibility baseline is official Codex CLI `0.146.0`, released July 29, 2026. Upstream `main` at commit `0bdce9f424eb9b39d7b3a8811742d10b6fbf8d54` was also inspected on August 7, 2026, including post-release routing and default-tool namespace changes. Current upstream, rather than the older installed CLI, is authoritative where they differ: commit `f21dc46388` replaced direct Lite function/custom declarations with one canonical `functions` namespace. This section is the package's explicit compatibility contract: it distinguishes close protocol adaptations from deliberate Pi behavior, configurable defaults, known gaps, and unsupported Codex runtimes. See the [Responses Lite compatibility report](RESPONSES_LITE_COMPATIBILITY.md) and [Codex caching and transport comparison](CODEX_CACHE_RESEARCH.md) for source revisions, request-path findings, and live cache trajectories.
+The compatibility baseline is official Codex CLI `0.149.0`, released August 20, 2026, at commit `758ef40f50c1a458425c7cfbf1eb12cbc07af0b0`. It retains the `0.147.0` Responses Lite contract that groups direct function and custom declarations into one canonical `functions` namespace for namespace-capable providers. The `0.148.0`–`0.149.0` review adopted typed misalignment-policy failures while recording official unbounded connection recovery as an intentional bounded-transport deviation. The [official release compatibility log](OFFICIAL_CODEX_CLI_RELEASES.md) is the canonical release-by-release record of protocol and `apply_patch` alignment, intentional deviations, and excluded runtimes. This section is the package's user-facing compatibility contract. See the [Responses Lite compatibility report](RESPONSES_LITE_COMPATIBILITY.md) and [Codex caching and transport comparison](CODEX_CACHE_RESEARCH.md) for detailed request-path findings and live cache trajectories.
 
 ### Configurable defaults that differ from Codex
 
@@ -63,6 +63,7 @@ The compatibility baseline is official Codex CLI `0.146.0`, released July 29, 20
 | Cache preparation           | Before the first cache-enabled WebSocket turn, the package prewarms only the stable instruction/tool prefix: ordinary Responses uses empty `input`, while Responses Lite uses `additional_tools` plus the developer instructions. The first generated request then contributes only dynamic conversation input to the continuation. No explicit prompt-cache breakpoints are added.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              |
 | Mid-turn compaction         | Provider-boundary percentage compaction preserves a successful `end_turn:false` prefix as its own Pi assistant message, installs a checkpoint, and continues without synthetic model input. Pi threshold compaction normally runs after the agent response; after Codex output-token truncation, the extension queues a hidden continuation so threshold compaction completes before sampling resumes. Official Codex owns this sampling and compaction loop directly.                                                                                                                                                                                                                                                                                                                                                                                                                           |
 | Provider-owned follow-up    | Completed responses with `end_turn: false` continue immediately from completed native output without synthetic user input. Retryable `response.failed` and all `response.incomplete` events are resampled with the official five-retry stream budget, preserving completed output and cumulative usage while excluding unfinished attempt content. A `max_output_tokens` response that exhausts this budget still becomes Pi `stopReason: "length"` and uses the extension's unbounded host-level continuation recovery.                                                                                                                                                                                                                                                                                                                                                                         |
+| Transport recovery          | WebSocket failures before model-visible output receive five fresh-connection retries before sticky SSE fallback; SSE transport failures receive five retries. Official Codex `0.149.0` separately retries sampling connection-establishment failures indefinitely with delays capped at 60 seconds. The package remains bounded because Fetch does not expose Reqwest's narrower connection-error category, and a phase-only approximation could indefinitely repeat requests that reached the server.                                                                                                                                                                                                                                                                                                                                                                                           |
 | Compaction lifecycle events | Pre-turn percentage compaction writes through Pi's mutable session manager and cannot emit Pi's internal `session_compact` event through the public extension API. Mid-response percentage boundaries and manual, threshold, or overflow compactions initiated by Pi emit the normal lifecycle.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  |
 | Header hooks                | An internal percentage-compaction request reuses the already transformed provider headers. It cannot independently rerun Pi's `before_provider_headers` hook.                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
 | Native retained context     | Deliberately differs from current Codex. The package retains recent user/developer/system messages under the 64k budget before the opaque compaction item. Current Codex applies a second installed-history filter that drops developer/system wrappers and non-real-user messages, can retain eligible structured agent commentary, and trims oversized function outputs before compaction. Pi keeps its existing checkpoint shape by design.                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
@@ -297,10 +298,17 @@ Compatibility behavior:
 - `*** Add File` overwrites an existing file, matching Codex.
 - `*** Move to` overwrites an existing destination, matching Codex.
 - Hunk matching retries exact text, trailing-whitespace-insensitive text, fully trimmed text, and Codex's Unicode punctuation normalization.
-- After strict matching fails, uniquely determined formatter-only line reflow can recover through exact Tree-sitter tokens for JavaScript, JSX, TypeScript, TSX, Python, Go, Java, and Scala. Requested replacement lines remain opaque and exact.
-- Markdown recovery is limited to exact-cell tables and supported code inside typed fences. Plain prose reflow, optional punctuation differences, single-token structural recovery, and partial-line structural recovery reject.
+- Matching stops after the official Codex-compatible line matcher. There is no Tree-sitter, Markdown-table, code-fence, formatter-reflow, candidate-ranking, or output-equivalence fallback.
+- Move identity and context chunks must match before the entry is moved; chunkless moves remain byte-opaque.
 - The parser accepts Codex's lenient marker whitespace, blank update-context lines, and direct heredoc wrappers.
 - Empty and identity updates, identical adds, absent deletes, self-moves, and same-patch fulfilled moves succeed with concise `NO CHANGE` results. Inapplicable operations are `SKIPPED` only when later operations deterministically make every effect unobservable.
+- Repeated identical adds use source-ordered virtual content and exact
+  spelling, so an earlier add or move can satisfy a later add without a
+  redundant replacement.
+- State-dependent `NO CHANGE` results retain source-ordered execution
+  checkpoints without rereading the filesystem. A checkpoint after an earlier
+  failure is `NOT RUN`; empty updates, identity updates without moves, and
+  chunkless lexical self-moves remain unconditional.
 - Model-facing results retain the aggregate A/M/D summary. When any instruction is not applied or an applied instruction has feedback, they list every source-ordered instruction under `Patch instruction results:` as `N. [STATUS] operation`, without an instruction limit; ordinary all-applied results omit the ledger.
 - Combined text updates and moves are labeled `Update & Move`; move-only operations remain `Move`.
 - Replacement feedback always identifies the verified previous and resulting entry types. Symlink feedback also uses the raw target pathname stored in the symlink.
@@ -308,19 +316,43 @@ Compatibility behavior:
 - Opaque moves and symlink deletions use path-only history, so binary bytes and link-target bytes are not serialized as textual deletions.
 - The TUI retains Codex-style changed-file summaries and uses the same conditional instruction ledger; when present, `Ctrl+O` nests complete diffs beneath the instruction that produced them.
 - With `applyPatchDebug` enabled, the tool title becomes `apply_patch (debug)` and a completed collapsed result shows the exact text returned to the model without an extra renderer-only heading; expanding it with `Ctrl+O` still shows the normal visual summary and complete diffs.
-- Failed instruction feedback colocates its error, completed effects, final path states, and concise matcher evidence without repeating patch text or using speculative language. Matcher failures include direct guidance for retrying with updated, source-ordered, non-overlapping, or more specific instructions.
+- Failed instruction feedback colocates its error, completed effects, and final path states without repeating patch text or using speculative language. Matching failures report the original context or expected-lines mismatch.
 
 Filesystem behavior:
 
 - Relative paths resolve from Pi's current working directory; absolute paths and `..` traversal are honored.
 - `.git` paths are unrestricted.
 - Text updates follow live symlinks; adds replace live or dangling symlinks without writing through them; deletes remove only the symlink; pure moves move the source symlink; and state-changing moves create a regular file at the destination without writing updated text through a source or destination symlink.
+- Relative symlink targets, including targets containing `..`, resolve from
+  the canonical directory containing the link even when the source or moved
+  destination is reached through a symlink-parent alias.
 - Entry-only operations and no-op updates do not dereference cyclic or inaccessible symlink targets during mutation-queue acquisition.
 - Same-filesystem pure moves use native rename topology. Cross-filesystem moves copy through a temporary entry, create or replace the destination, and then unlink the source, producing an inode independent from remaining source hard links.
-- Strict and formatter-recovered edits preserve the matched region's local CRLF or mixed line endings.
+- Supplied identity or context chunks validate a pure move through the text matcher without rewriting the moved entry. A blank `@@` therefore requires valid UTF-8; omit chunks for arbitrary binary content.
+- Strict edits preserve the matched region's local CRLF or mixed line endings.
+- In-place text updates use a direct path write, following source symlinks and
+  preserving normal hard-link visibility.
+- Adds and text updates verify the complete expected final byte buffer
+  byte-for-byte before succeeding. This is whole-file equality, not a
+  substring search, so duplicate or unrelated regions cannot satisfy the
+  postcondition. Deletes verify absence; moves verify their source,
+  destination type, exact destination spelling, native identity where
+  available, known bytes, and raw symlink target.
 - The extension does not add path filtering, sandboxing, or approval prompts.
 - Every hunk is parsed and validated before filesystem writes begin.
-- Mutations participate in Pi's per-file mutation queue and an extension-local logical queue for case, Unicode, symlink-parent, and hard-link aliases. Both queues coordinate only concurrent `apply_patch` calls in the same Pi process and module instance; they do not coordinate separate Pi sessions, other processes, or unrelated edit/write tools.
+- No-change checkpoints are interleaved with mutations in patch order, so
+  later checkpoints are not reported as completed after an earlier failure.
+- Mutations participate in Pi's per-file mutation queue for the complete
+  preflight-and-execution window. Queue paths are acquired deterministically,
+  follow operation-specific symlink semantics, and include every move source
+  and destination. There is no additional extension-local alias queue.
+
+Operating model: relevant filesystem state is not modified outside the queued
+`apply_patch` execution window. This includes ancestor paths, symlink targets,
+hard-link aliases, callbacks, injected filesystem hooks, separate Pi sessions,
+and external processes. Preflight is authoritative under this model; the
+executor does not attempt cross-process drift detection or transactional
+isolation.
 
 A low-level I/O failure can still complete part of an instruction. The failed
 instruction reports every confirmed effect and final path state; when a path

package/THIRD_PARTY_NOTICES.md

@@ -2,7 +2,7 @@
 
 ## OpenAI Codex
 
-The `apply_patch` grammar, parser behavior, fuzzy matching, mutation semantics, result formatting, diff rendering, standalone image-generation behavior, standalone web-search behavior, tool schemas, and compatibility scenarios in this package are adapted for Pi from the public OpenAI Codex implementation.
+The `apply_patch` grammar, parser behavior, line matching, mutation semantics, result formatting, diff rendering, standalone image-generation behavior, standalone web-search behavior, tool schemas, and compatibility scenarios in this package are adapted for Pi from the public OpenAI Codex implementation.
 
 OpenAI Codex, Copyright 2025 OpenAI
 
@@ -19,29 +19,3 @@ Pi AI, Copyright (c) 2025 Mario Zechner
 Pi AI is licensed under the MIT License. A copy is included at [`LICENSES/pi-ai-MIT.txt`](LICENSES/pi-ai-MIT.txt).
 
 Source: <https://github.com/earendil-works/pi/tree/main/packages/ai>
-
-## Tree-sitter WASMs
-
-`@2h2d/tree-sitter-wasms` supplies lifecycle-free grammar WASM assets used for
-formatter-tolerant `apply_patch` matching.
-
-Tree-sitter WASMs, Copyright (c) 2026 Kaan Ozdokmeci
-
-Tree-sitter WASMs is licensed under the MIT License. A copy is included at
-[`LICENSES/tree-sitter-wasms-MIT.txt`](LICENSES/tree-sitter-wasms-MIT.txt).
-The dependency package includes the exact upstream license for each bundled
-grammar under its own `LICENSES` directory.
-
-Source: <https://github.com/2h2d-co/tree-sitter-wasms>
-
-## Web Tree-sitter
-
-`web-tree-sitter` supplies the official Tree-sitter WASM runtime used to load
-and execute the packaged grammars.
-
-Web Tree-sitter, Copyright (c) 2018 Max Brunsfeld
-
-Web Tree-sitter is licensed under the MIT License. A copy is included at
-[`LICENSES/web-tree-sitter-MIT.txt`](LICENSES/web-tree-sitter-MIT.txt).
-
-Source: <https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_web>