@codex-native/sdk 0.0.1 → 0.0.3

package/README.md

@@ -12,6 +12,24 @@ npm install @codex-native/sdk
 
 Requires Node.js 18+.
 
+## API Compatibility
+
+The Native SDK provides **full API compatibility** with the [TypeScript SDK](../typescript/). All core functionality—threads, streaming, structured output, and basic operations—works identically across both SDKs. The Native SDK adds Rust-powered performance and custom tool registration while maintaining the same interface.
+
+### Migration from TypeScript SDK
+
+Simply replace the import:
+
+```typescript
+// Before (TypeScript SDK)
+import { Codex } from "@openai/codex-sdk";
+
+// After (Native SDK - same API!)
+import { Codex } from "@codex-native/sdk";
+```
+
+All your existing code continues to work without changes.
+
 ## Quickstart
 
 ```typescript
@@ -50,9 +68,32 @@ for await (const event of events) {
 }
 ```
 
+### Mid-turn notifications
+
+You can publish lightweight updates during an active turn without adding another user
+message. Call `thread.sendBackgroundEvent()` inside a `runStreamed()` loop after the
+turn has started:
+
+```typescript
+const { events } = await thread.runStreamed("Generate the release notes");
+
+for await (const event of events) {
+  if (event.type === "turn.started") {
+    await thread.sendBackgroundEvent("Gathering changelog entries…");
+  } else if (event.type === "background_event") {
+    console.log(event.message);
+  }
+}
+```
+
+The streaming API surfaces these as `background_event` items so downstream consumers
+can display progress indicators or status notifications while the agent continues its
+turn.
+
 ### Structured output
 
-The Codex agent can produce a JSON response that conforms to a specified schema. The schema can be provided for each turn as a plain JSON object.
+The Codex agent can produce a JSON response that conforms to a specified schema. The schema
+can be provided for each turn as a plain JSON object.
 
 ```typescript
 const schema = {
@@ -69,7 +110,9 @@ const turn = await thread.run("Summarize repository status", { outputSchema: sch
 console.log(turn.finalResponse);
 ```
 
-You can also create a JSON schema from a [Zod schema](https://github.com/colinhacks/zod) using the [`zod-to-json-schema`](https://www.npmjs.com/package/zod-to-json-schema) package and setting the `target` to `"openAi"`.
+You can also create a JSON schema from a [Zod schema](https://github.com/colinhacks/zod)
+using the [`zod-to-json-schema`](https://www.npmjs.com/package/zod-to-json-schema)
+package and setting the `target` to "openAi".
 
 ```typescript
 const schema = z.object({
@@ -83,6 +126,75 @@ const turn = await thread.run("Summarize repository status", {
 console.log(turn.finalResponse);
 ```
 
+## Command-line interface
+
+In addition to the programmatic API, the package ships a `codex-native` CLI that mirrors the
+Rust `codex` binary. The CLI is available after `pnpm install` (or via `npx codex-native`).
+
+```bash
+# Run a one-off Codex turn using the native SDK
+codex-native run "Diagnose the failing integration test"
+
+# Launch the full-screen TUI backed by the native bindings
+codex-native tui
+```
+
+### Configuration discovery
+
+`codex-native` automatically merges configuration from the following locations (in priority order):
+
+1. The CLI flags supplied on the command line
+2. A `codex.config.*` file in the working directory (`.js`, `.cjs`, `.mjs`, `.ts`)
+3. A `codexNative` field in `package.json`
+
+Configuration files can export either an object or a function. The function form receives a
+context `{ cwd, configPath }` so you can derive settings dynamically.
+
+```typescript
+// codex.config.ts
+import type { CodexNativeConfig } from "@codex-native/sdk/cli";
+
+const config: CodexNativeConfig = {
+  defaults: {
+    run: { model: "gpt-5-codex" },
+    tui: { resumePicker: false },
+  },
+  tools: [
+    {
+      name: "read_me",
+      description: "Show the project README",
+      handler: async () => ({ output: await fs.promises.readFile("README.md", "utf8") }),
+    },
+  ],
+  interceptors: [
+    {
+      toolName: "shell",
+      handler: async ({ invocation, callBuiltin }) => {
+        console.log("shell call", invocation);
+        return callBuiltin();
+      },
+    },
+  ],
+};
+
+export default config;
+```
+
+Multiple `--plugin` flags can be passed to load additional modules. A plugin may export either a
+config object or a function that returns one. Plugin configs are merged on top of the base config.
+
+### Hooks and approvals
+
+Config files can specify hooks that run before every turn (`beforeStart`) and when streaming
+events (`onEvent`). You can also register an `approvals` callback to gate sensitive tool usage.
+These map directly to the native bindings so the same logic applies in both CLI and SDK usage.
+
+### TUI mode
+
+`codex-native tui` launches the same Ratatui-based interface used by the Rust `codex` tool. The
+CLI honors the same configuration sources listed above, so you can keep CLI, SDK, and TUI
+settings in a single `codex.config.ts` file.
+
 ### Attaching images
 
 Provide structured input entries when you need to include images alongside text. Text entries are concatenated into the final prompt while image entries are passed to Codex via the native bridge.
@@ -105,9 +217,80 @@ const thread = codex.resumeThread(savedThreadId);
 await thread.run("Implement the fix");
 ```
 
+### Forking a conversation
+
+Use `thread.fork()` to branch from an earlier user message and explore an alternate path without losing the original history. Provide the zero-based index of the user message you want to fork **before**.
+
+```typescript
+const codex = new Codex();
+const thread = codex.startThread({ skipGitRepoCheck: true });
+
+await thread.run("List flaky integration tests");
+await thread.run("Propose fixes for each flaky test");
+
+// Fork before the second user message (index 1)
+const branch = await thread.fork({
+  nthUserMessage: 1,
+  threadOptions: { model: "gpt-5-codex-mini" },
+});
+
+await branch.run("Focus on the payment suite instead");
+```
+
+The original `thread` continues unchanged while `branch` contains the forked history and a fresh thread id.
+
+### Running code reviews
+
+Invoke the native review workflow without crafting prompts manually. The SDK provides presets that mirror the `/review` slash command:
+
+```typescript
+const codex = new Codex();
+
+// Review everything that is staged, unstaged, or untracked
+const review = await codex.review({
+  target: { type: "current_changes" },
+});
+
+for (const finding of review.items) {
+  if (finding.type === "agent_message") {
+    console.log(finding.text);
+  }
+}
+```
+
+Additional presets let you review against another branch or a specific commit:
+
+```typescript
+await codex.review({
+  target: { type: "branch", baseBranch: "main" },
+});
+
+await codex.review({
+  target: {
+    type: "commit",
+    sha: "abc1234def5678",
+    subject: "Tighten input validation",
+  },
+});
+```
+
+For bespoke instructions, pass a custom prompt and optional hint:
+
+```typescript
+await codex.review({
+  target: {
+    type: "custom",
+    prompt: "Review only the data-access layer for regression risks.",
+    hint: "data-access layer",
+  },
+});
+```
+
 ### Working directory controls
 
-Codex runs in the current working directory by default. To avoid unrecoverable errors, Codex requires the working directory to be a Git repository. You can skip the Git repository check by passing the `skipGitRepoCheck` option when creating a thread.
+Codex runs in the current working directory by default. To avoid unrecoverable errors, Codex
+requires the working directory to be a Git repository. You can skip the Git repository check by
+passing the `skipGitRepoCheck` option when creating a thread.
 
 ```typescript
 const thread = codex.startThread({
@@ -124,6 +307,134 @@ The Native SDK provides additional capabilities beyond the TypeScript SDK:
 
 Register JavaScript functions as tools that Codex can discover and invoke during execution. Tools are registered globally on the `Codex` instance and become available to all threads and agents.
 
+> **Override built-ins:** If you register a tool whose `name` matches one of Codex's built-in tools (for example `read_file`, `local_shell`, or `apply_patch`), the native implementation is replaced for the lifetime of that `Codex` instance. This lets you customize or disable default behaviors while keeping the same tool interface.
+
+#### Built-in tool override cheat sheet
+
+All snippets assume you already created an instance with `const codex = new Codex();`. Registering any of these names swaps out Codex's default implementation.
+
+- `shell` – sandboxed shell command runner (models without unified exec)
+  ```typescript
+  codex.registerTool({
+    name: "shell",
+    handler: () => ({ error: "Shell disabled by policy", success: false }),
+  });
+  ```
+
+- `exec_command` – streaming command execution (available when unified exec is enabled)
+  ```typescript
+  codex.registerTool({
+    name: "exec_command",
+    handler: (_, invocation) => ({
+      output: `Pretend ran: ${invocation.arguments}`,
+      success: true,
+    }),
+  });
+  ```
+
+- `write_stdin` – feeds additional input into an in-flight `exec_command`
+  ```typescript
+  codex.registerTool({
+    name: "write_stdin",
+    handler: () => ({ output: "stdin blocked", success: false }),
+  });
+  ```
+
+- `local_shell` – simplified shell command helper (models that prefer local shell over unified exec)
+  ```typescript
+  codex.registerTool({
+    name: "local_shell",
+    handler: () => ({ output: "local shell override", success: true }),
+  });
+  ```
+
+- `list_mcp_resources`, `list_mcp_resource_templates`, `read_mcp_resource` – MCP discovery helpers
+  ```typescript
+  for (const name of [
+    "list_mcp_resources",
+    "list_mcp_resource_templates",
+    "read_mcp_resource",
+  ]) {
+    codex.registerTool({
+      name,
+      handler: () => ({ output: JSON.stringify({ notice: `${name} overridden` }) }),
+    });
+  }
+  ```
+
+- `update_plan` – emits high-level plan updates back to the host UI
+  ```typescript
+  codex.registerTool({
+    name: "update_plan",
+    handler: () => ({ output: "Plan updates disabled" }),
+  });
+  ```
+
+- `apply_patch` – applies patches authored by the agent
+  ```typescript
+  codex.registerTool({
+    name: "apply_patch",
+    handler: (_, { arguments }) => ({
+      output: `Custom patch handler received: ${arguments}`,
+      success: true,
+    }),
+  });
+  ```
+
+- `web_search` – performs outbound web searches (only on models with the feature enabled)
+  ```typescript
+  codex.registerTool({
+    name: "web_search",
+    handler: (_, { arguments }) => ({
+      output: `Search stub: ${arguments}`,
+      success: true,
+    }),
+  });
+  ```
+
+- `view_image` – attaches a local image for the model to inspect
+  ```typescript
+  codex.registerTool({
+    name: "view_image",
+    handler: (_, { arguments }) => ({
+      output: `Ignoring image path ${arguments}`,
+      success: true,
+    }),
+  });
+  ```
+
+- `grep_files`, `read_file`, `list_dir` – workspace inspection helpers (enabled via experimental flags)
+  ```typescript
+  for (const name of ["grep_files", "read_file", "list_dir"]) {
+    codex.registerTool({
+      name,
+      handler: (_, { arguments }) => ({
+        output: JSON.stringify({ name, arguments, overridden: true }),
+        success: true,
+      }),
+    });
+  }
+  ```
+
+- `test_sync_tool` – synchronization helper used in concurrency tests
+  ```typescript
+  codex.registerTool({
+    name: "test_sync_tool",
+    handler: () => ({ output: "Barrier skipped" }),
+  });
+  ```
+
+- MCP server tools – any name of the form `server::tool`
+  ```typescript
+  codex.registerTool({
+    name: "jira::create_issue",
+    handler: (_, { arguments }) => ({
+      output: `Custom Jira integration received ${arguments}`,
+      success: true,
+    }),
+  });
+  ```
+
 ```typescript
 const codex = new Codex();
 
@@ -182,6 +493,63 @@ Return an object with:
 - `success` (optional): Boolean indicating success/failure
 - `error` (optional): Error message if the tool execution failed
 
+### Tool Interceptors
+
+For more advanced use cases, register **tool interceptors** that can wrap built-in Codex tools with pre/post-processing logic while still executing the original implementation.
+
+```typescript
+const codex = new Codex();
+
+// Intercept exec_command calls to add custom timeout and logging
+codex.registerToolInterceptor("exec_command", async (invocation) => {
+  // Pre-processing: modify the arguments
+  const args = JSON.parse(invocation.arguments ?? "{}");
+  const enhancedArgs = {
+    ...args,
+    timeout_ms: args.timeout_ms ?? 10000, // Default 10s timeout
+    justification: args.justification ?? "intercepted",
+  };
+
+  // For now, interceptors return a placeholder response
+  // Future versions will support calling the builtin implementation
+  return {
+    output: `[INTERCEPTED] Would execute: ${JSON.stringify(enhancedArgs)}`,
+    success: true,
+  };
+});
+
+// Intercept apply_patch to add validation
+codex.registerToolInterceptor("apply_patch", async (invocation) => {
+  const args = JSON.parse(invocation.arguments ?? "{}");
+
+  // Add custom validation or preprocessing
+  if (!args.patch_content?.includes("diff")) {
+    return {
+      output: "Invalid patch format - must contain diff data",
+      success: false,
+    };
+  }
+
+  // Return modified result
+  return {
+    output: `[VALIDATED] ${args.patch_content}`,
+    success: true,
+  };
+});
+```
+
+**Key Differences from Tool Overrides:**
+
+- **Interceptors wrap** built-in tools instead of replacing them entirely
+- **Preserve sandboxing** - interceptors cannot bypass Codex's security policies
+- **Chainable** - multiple interceptors can be registered for the same tool
+- **Future enhancement** - interceptors will be able to call the underlying builtin implementation
+
+**Current Notes:**
+
+- Tool interceptors support decorating responses by calling `context.callBuiltin()`
+- Multiple interceptors per tool will be composed in registration order in a future release
+
 ### Agent Orchestration
 
 Create specialized agents with custom system prompts and tools for multi-agent workflows.
@@ -238,13 +606,63 @@ const securityResult = await securityAgent.run(
 
 Agents automatically have access to the conversation history, enabling seamless handoffs between specialized agents.
 
+### Reverie Archive APIs
+
+Query past Codex sessions directly from Node.js to surface relevant prior work without leaving the terminal.
+
+```typescript
+import {
+  reverieListConversations,
+  reverieSearchConversations,
+  reverieGetConversationInsights,
+} from "@codex-native/sdk";
+
+const codexHome = process.env.CODEX_HOME ?? `${process.env.HOME}/.codex`;
+
+// List the newest conversations (newest first)
+const conversations = await reverieListConversations(codexHome, 10);
+
+// Search for conversations mentioning "authentication"
+const matches = await reverieSearchConversations(codexHome, "authentication issue", 5);
+
+// Read the highlights/insights from a specific conversation rollout
+const insights = await reverieGetConversationInsights(matches[0].conversation.path, "JWT");
+```
+
+Results include `headRecords` and `tailRecords`, plus the TOON-encoded `headRecordsToon` and `tailRecordsToon` previews used by the Rust CLI/TUI, so you can plug them into custom dashboards or route them back into an agent as `<system notification>`s without wasting tokens.
+
+Need to compact your own JSON payloads before feeding them to an LLM? Call `encodeToToon(value)` from JavaScript to get the same Token-Oriented Object Notation that Codex now uses for reverie search/indexing.
+
+### Tokenizer Helpers (tiktoken)
+
+Access the same tiktoken-powered tokenizer used by Codex from JavaScript for budgeting prompts or implementing local ranking logic.
+
+```typescript
+import {
+  tokenizerCount,
+  tokenizerEncode,
+  tokenizerDecode,
+} from "@codex-native/sdk";
+
+const text = "hello world";
+
+// Count tokens using a specific encoding or model alias
+const tokens = tokenizerEncode(text, { encoding: "cl100k_base" });
+const count = tokenizerCount(text, { encoding: "cl100k_base" });
+
+// Round-trip
+const decoded = tokenizerDecode(tokens, { encoding: "cl100k_base" });
+```
+
+`encoding` accepts `"o200k_base"` or `"cl100k_base"`, and you can also pass `model: "gpt-5"` to mirror Codex’s model-to-encoding mapping. Set `withSpecialTokens: true` when you need precise accounting for schema-guided prompts.
+
 ## API Options
 
 ### Codex Constructor Options
 
 ```typescript
 interface CodexOptions {
-  apiKey?: string;              // Responses API key (defaults to ANTHROPIC_API_KEY env var)
+  apiKey?: string;              // Responses API key (defaults to OPENAI_API_KEY env var)
   baseUrl?: string;             // API base URL override
   skipGitRepoCheck?: boolean;   // Skip Git repository validation
 }
@@ -254,13 +672,62 @@ interface CodexOptions {
 
 ```typescript
 interface ThreadOptions {
-  model?: string;               // Model to use (e.g., "claude-sonnet-4")
+  model?: string;               // Model to use (e.g., "gpt-5-codex")
   sandboxMode?: "read-only" | "workspace-write" | "danger-full-access";
+  approvalMode?: "never" | "on-request" | "on-failure" | "untrusted";
+  workspaceWriteOptions?: {
+    networkAccess?: boolean;    // Enable network in workspace-write mode (default: false)
+    writableRoots?: string[];   // Additional writable directories
+    excludeTmpdirEnvVar?: boolean; // Exclude TMPDIR from writable roots
+    excludeSlashTmp?: boolean;  // Exclude /tmp from writable roots (Unix only)
+  };
   workingDirectory?: string;    // Directory to run Codex in
   skipGitRepoCheck?: boolean;   // Skip Git repository validation
+  fullAuto?: boolean;           // @deprecated Use sandboxMode and approvalMode
 }
 ```
 
+#### Sandbox Modes
+
+- **`read-only`**: AI can only read files, must approve all edits
+- **`workspace-write`**: AI can edit workspace files freely (with optional network)
+- **`danger-full-access`**: No sandbox (dangerous!)
+
+#### Approval Policies
+
+- **`never`**: Never ask for approval (commands execute automatically)
+- **`on-request`**: Model decides when to ask (default)
+- **`on-failure`**: Auto-approve but escalate on failure
+- **`untrusted`**: Only trusted commands auto-approved
+
+#### Network Access Configuration
+
+Enable network access in `workspace-write` mode:
+
+```typescript
+const thread = codex.startThread({
+  sandboxMode: "workspace-write",
+  workspaceWriteOptions: {
+    networkAccess: true
+  }
+});
+```
+
+#### Advanced Sandbox Configuration
+
+Configure additional writable directories:
+
+```typescript
+const thread = codex.startThread({
+  sandboxMode: "workspace-write",
+  workspaceWriteOptions: {
+    writableRoots: ["/path/to/additional/dir"],
+    excludeTmpdirEnvVar: false,
+    excludeSlashTmp: false
+  }
+});
+```
+
 ### Turn Options
 
 ```typescript
@@ -374,6 +841,16 @@ pnpm run release    # Publish all packages
 
 npm automatically installs the correct platform package as an optional dependency.
 
+## Releasing
+
+1. Update the version in `package.json` and keep the optional dependency versions in sync.
+2. Record the changes in `CHANGELOG.md` (add a new section for the release).
+3. Regenerate build artifacts: `pnpm run build`.
+4. Run the full test suite: `pnpm run test` (which invokes the native Jest runner).
+5. Publish platform binaries: `pnpm run publish:platforms` (or `npm run publish:platforms`).
+6. Publish the SDK itself: `node scripts/publish-sdk.mjs` (the `release` script chains everything).
+7. Tag the release in git.
+
 ## License
 
 See [LICENSE](../../LICENSE)