zidane 1.0.2 → 1.2.0

package/README.md

@@ -1 +1,724 @@
 ![Zidane](https://github.com/Tahul/zidane/blob/main/zidane.jpeg?raw=true)
+
+# Zidane
+
+An agent that goes straight to the goal.
+
+Minimal TypeScript agent loop built with [Bun](https://bun.sh).
+
+Hook into every step of the agent's execution using [hookable](https://github.com/unjs/hookable).
+
+Built to be embedded in other projects easily, extended through [providers](#providers), [harnesses](#harnesses), and [execution contexts](#execution-contexts).
+
+## Quickstart
+
+```bash
+# Install
+bun install
+
+# Authenticate with Anthropic OAuth (Claude Pro/Max)
+bun run auth
+
+# Run
+bun start --prompt "create a hello world express app"
+```
+
+## CLI
+
+```bash
+bun start \
+  --prompt "your task"    \   # required
+  --model claude-opus-4-6 \   # model id (default: claude-opus-4-6)
+  --provider anthropic    \   # anthropic | openrouter | cerebras
+  --harness basic         \   # tool set to use
+  --system "be concise"   \   # system prompt
+  --thinking off          \   # off | minimal | low | medium | high
+  --context process       \   # process | docker
+  --mcp '{"name":"fs","transport":"stdio","command":"npx","args":["-y","@modelcontextprotocol/server-filesystem","."]}'
+```
+
+The `--mcp` flag accepts a JSON object matching `McpServerConfig`. It can be passed multiple times.
+
+## Execution Contexts
+
+An execution context defines **where** the agent's tools run. All tool operations (shell, filesystem) go through it.
+
+### In-process (default)
+
+Runs in the same Node/Bun process. No isolation, fastest.
+
+```ts
+import { createAgent, createProcessContext } from 'zidane'
+
+const agent = createAgent({
+  harness,
+  provider,
+  // execution defaults to createProcessContext()
+})
+```
+
+### Docker
+
+Full container isolation via [dockerode](https://github.com/apocas/dockerode). Configurable resource limits.
+
+```bash
+# CLI
+bun start --prompt "run uname -a" --context docker
+bun start --prompt "build the app" --context docker --image node:22 --cwd /workspace
+```
+
+```ts
+import { createAgent, createDockerContext } from 'zidane'
+
+const agent = createAgent({
+  harness,
+  provider,
+  execution: createDockerContext({
+    image: 'node:22',
+    cwd: '/workspace',
+    limits: { memory: 512, cpu: '1.0' },
+  }),
+})
+```
+
+Requires `dockerode` as a peer dependency: `bun add dockerode`
+
+### Sandbox (remote)
+
+Offloads execution to a remote sandbox API. Implement the `SandboxProvider` interface for your provider (Rivet, E2B, etc.).
+
+```ts
+import { createAgent, createSandboxContext } from 'zidane'
+import type { SandboxProvider } from 'zidane'
+
+const myProvider: SandboxProvider = {
+  name: 'my-sandbox',
+  spawn: async (config) => { /* ... */ },
+  exec: async (id, command) => { /* ... */ },
+  readFile: async (id, path) => { /* ... */ },
+  writeFile: async (id, path, content) => { /* ... */ },
+  listFiles: async (id, path) => { /* ... */ },
+  destroy: async (id) => { /* ... */ },
+}
+
+const agent = createAgent({
+  harness,
+  provider,
+  execution: createSandboxContext(myProvider),
+})
+```
+
+### Execution Context Interface
+
+All contexts implement the same interface:
+
+```ts
+interface ExecutionContext {
+  type: 'process' | 'docker' | 'sandbox'
+  capabilities: { shell, filesystem, network, gpu }
+  spawn(config?): Promise<ExecutionHandle>
+  exec(handle, command, options?): Promise<ExecResult>
+  readFile(handle, path): Promise<string>
+  writeFile(handle, path, content): Promise<void>
+  listFiles(handle, path): Promise<string[]>
+  destroy(handle): Promise<void>
+}
+```
+
+Access the context from a running agent:
+
+```ts
+agent.execution          // ExecutionContext
+agent.execution.type     // 'process' | 'docker' | 'sandbox'
+agent.handle             // ExecutionHandle (after first run)
+await agent.destroy()    // clean up context resources
+```
+
+## Providers
+
+### Anthropic
+
+Direct Anthropic API with OAuth and API key support.
+
+```bash
+# OAuth (Claude Pro/Max subscription)
+bun run auth
+
+# Or API key
+ANTHROPIC_API_KEY=sk-ant-... bun start --prompt "hello"
+```
+
+### OpenRouter
+
+Access 200+ models through OpenRouter's unified API.
+
+```bash
+OPENROUTER_API_KEY=sk-or-... bun start \
+  --provider openrouter \
+  --model anthropic/claude-sonnet-4-6 \
+  --prompt "hello"
+```
+
+### Cerebras
+
+Ultra-fast inference on Cerebras wafer-scale hardware.
+
+```bash
+CEREBRAS_API_KEY=csk-... bun start \
+  --provider cerebras \
+  --model zai-glm-4.7 \
+  --prompt "hello"
+```
+
+## Thinking
+
+Extended reasoning for complex tasks. Maps to Anthropic's thinking API or OpenRouter's `:thinking` variant.
+
+```bash
+bun start --prompt "solve this proof" --thinking high
+```
+
+| Level | Budget |
+|---|---|
+| `off` | disabled |
+| `minimal` | 1k tokens |
+| `low` | 4k tokens |
+| `medium` | 10k tokens |
+| `high` | 32k tokens |
+
+## Tools (Harnesses)
+
+Tools are grouped into **harnesses**. The `basic` harness includes:
+
+| Tool | Description |
+|---|---|
+| `shell` | Execute shell commands |
+| `read_file` | Read file contents |
+| `write_file` | Write/create files |
+| `list_files` | List directory contents |
+| `spawn` | Spawn a sub-agent for a task |
+
+All paths are sandboxed to the working directory.
+
+Define a custom harness with `defineHarness`:
+
+```ts
+import { defineHarness } from 'zidane'
+
+const harness = defineHarness({
+  name: 'researcher',
+  system: 'You are a research assistant.',
+  tools: { ...basicTools },
+  mcpServers: [
+    { name: 'filesystem', transport: 'stdio', command: 'npx', args: ['-y', '@modelcontextprotocol/server-filesystem', '.'] },
+  ],
+})
+```
+
+## Sub-agent Spawning
+
+The `spawn` tool lets the agent delegate tasks to child agents. Children run independently and return their result as a tool response.
+
+### Static spawn tool
+
+```ts
+import { spawn, basicTools, defineHarness } from 'zidane'
+
+const harness = defineHarness({
+  name: 'orchestrator',
+  tools: { ...basicTools, spawn },
+})
+```
+
+Children inherit the parent's harness (and can spawn their own children).
+
+### Configurable factory
+
+Use `createSpawnTool` when you need custom concurrency limits, model overrides, or lifecycle callbacks.
+
+```ts
+import { createSpawnTool } from 'zidane'
+
+const spawnTool = createSpawnTool({
+  maxConcurrent: 5,
+  model: 'claude-haiku-4-5-20251001',
+  system: 'You are a focused sub-agent.',
+  thinking: 'low',
+  onSpawn: (child) => console.log(`started ${child.id}`),
+  onComplete: (child, stats) => console.log(`${child.id} done in ${stats.turns} turns`),
+})
+
+const harness = defineHarness({
+  name: 'orchestrator',
+  tools: { spawn: spawnTool },
+})
+```
+
+## MCP Servers
+
+Connect any MCP-compatible tool server. Tools are namespaced as `mcp_{serverName}_{toolName}`.
+
+### Agent-level
+
+```ts
+const agent = createAgent({
+  harness,
+  provider,
+  mcpServers: [
+    { name: 'filesystem', transport: 'stdio', command: 'npx', args: ['-y', '@modelcontextprotocol/server-filesystem', '.'] },
+    { name: 'search', transport: 'sse', url: 'http://localhost:3001/sse' },
+    { name: 'api', transport: 'streamable-http', url: 'http://localhost:3002/mcp' },
+  ],
+})
+```
+
+### Harness-level
+
+MCP servers can also be declared on the harness so they're shared across all agents using it.
+
+```ts
+const harness = defineHarness({
+  name: 'with-mcp',
+  tools: { ...basicTools },
+  mcpServers: [
+    { name: 'db', transport: 'stdio', command: 'node', args: ['db-server.js'] },
+  ],
+})
+```
+
+MCP connections are made lazily on the first `run()` call and reused across subsequent runs. They are closed when `agent.destroy()` is called.
+
+## Sessions
+
+Sessions give an agent persistent identity, message history, and run metadata across multiple calls or restarts.
+
+### Creating a session
+
+```ts
+import { createSession, createMemoryStore } from 'zidane/session'
+
+// In-memory (default, no persistence)
+const session = createSession({ id: 'my-session', agentId: 'my-agent' })
+
+// With a store for persistence
+const store = createMemoryStore()
+const session = createSession({ id: 'my-session', store })
+```
+
+### Storage backends
+
+Three built-in stores are available:
+
+```ts
+import { createMemoryStore, createSqliteStore, createRemoteStore } from 'zidane/session'
+
+// In-memory, fast, no disk I/O, lost on process restart
+const memStore = createMemoryStore()
+
+// SQLite, persistent, zero-dependency (uses Bun's built-in SQLite)
+const sqliteStore = createSqliteStore({ path: './sessions.db' })
+
+// Remote HTTP, delegates to a custom REST API
+const remoteStore = createRemoteStore({ url: 'https://api.example.com/sessions' })
+```
+
+### Agent integration
+
+```ts
+const agent = createAgent({
+  harness,
+  provider,
+  session,
+})
+
+await agent.run({ prompt: 'hello' })
+await session.save() // persist to store
+```
+
+### Session hooks
+
+```ts
+agent.hooks.hook('session:start', (ctx) => {
+  // ctx.sessionId, ctx.runId, ctx.prompt
+})
+
+agent.hooks.hook('session:end', (ctx) => {
+  // ctx.sessionId, ctx.runId
+  // ctx.status: 'completed' | 'aborted' | 'error'
+})
+
+agent.hooks.hook('session:messages', (ctx) => {
+  // ctx.sessionId, ctx.count
+  // fired after each turn (live message sync)
+})
+
+agent.hooks.hook('session:save', (ctx) => {
+  // ctx.sessionId
+  // fired after session.save() completes
+})
+
+agent.hooks.hook('session:meta', (ctx) => {
+  // ctx.sessionId, ctx.key, ctx.value
+  // fired when session.setMeta() is called
+})
+```
+
+Messages are synced to the session after every turn, not just at run start/end. If the agent crashes mid-run, you still have messages up to the last completed turn.
+
+### Restoring a session
+
+```ts
+import { loadSession } from 'zidane/session'
+
+const session = await loadSession(store, 'my-session')
+if (session) {
+  const agent = createAgent({ harness, provider, session })
+  await agent.run({ prompt: 'continue from before' })
+}
+```
+
+## Hooks
+
+The agent uses [hookable](https://github.com/unjs/hookable) for lifecycle events. Every hook receives a mutable context object.
+
+### Lifecycle
+
+```ts
+agent.hooks.hook('system:before', (ctx) => {
+  // ctx.system: system prompt text
+})
+
+agent.hooks.hook('turn:before', (ctx) => {
+  // ctx.turn: turn number
+  // ctx.options: StreamOptions being sent to provider
+})
+
+agent.hooks.hook('turn:after', (ctx) => {
+  // ctx.turn, ctx.usage { input, output }
+})
+
+agent.hooks.hook('agent:done', (ctx) => {
+  // ctx.totalIn, ctx.totalOut, ctx.turns, ctx.elapsed, ctx.children?
+})
+
+agent.hooks.hook('agent:abort', () => {
+  // fired when agent.abort() is called
+})
+```
+
+### Streaming
+
+```ts
+agent.hooks.hook('stream:text', (ctx) => {
+  // ctx.delta: new text chunk
+  // ctx.text: accumulated text so far
+})
+
+agent.hooks.hook('stream:end', (ctx) => {
+  // ctx.text: final complete text
+})
+```
+
+### Tool Execution
+
+```ts
+agent.hooks.hook('tool:before', (ctx) => {
+  // ctx.name, ctx.input
+})
+
+agent.hooks.hook('tool:after', (ctx) => {
+  // ctx.name, ctx.input, ctx.result
+})
+
+agent.hooks.hook('tool:error', (ctx) => {
+  // ctx.name, ctx.input, ctx.error
+})
+```
+
+### Tool Gate: block execution
+
+Mutate `ctx.block = true` to prevent a tool from running.
+
+```ts
+agent.hooks.hook('tool:gate', (ctx) => {
+  if (ctx.name === 'shell' && String(ctx.input.command).includes('rm -rf')) {
+    ctx.block = true
+    ctx.reason = 'dangerous command'
+  }
+})
+```
+
+### Tool Transform: modify output
+
+Mutate `ctx.result` or `ctx.isError` to transform tool results before they're sent back to the model.
+
+```ts
+agent.hooks.hook('tool:transform', (ctx) => {
+  if (ctx.result.length > 5000)
+    ctx.result = ctx.result.slice(0, 5000) + '\n... (truncated)'
+})
+```
+
+### Context Transform: prune messages
+
+Mutate `ctx.messages` before each LLM call for context window management.
+
+```ts
+agent.hooks.hook('context:transform', (ctx) => {
+  if (ctx.messages.length > 30)
+    ctx.messages.splice(2, ctx.messages.length - 30)
+})
+```
+
+### Spawn hooks
+
+Fired by the `spawn` tool when child agents are created.
+
+```ts
+agent.hooks.hook('spawn:before', (ctx) => {
+  // ctx.id: child agent id (e.g. 'child-1')
+  // ctx.task: the task prompt given to the child
+})
+
+agent.hooks.hook('spawn:complete', (ctx) => {
+  // ctx.id, ctx.task
+  // ctx.stats: AgentStats from the child run
+})
+
+agent.hooks.hook('spawn:error', (ctx) => {
+  // ctx.id, ctx.task, ctx.error
+})
+```
+
+### MCP hooks
+
+Fired during MCP server lifecycle.
+
+```ts
+agent.hooks.hook('mcp:connect', (ctx) => {
+  // ctx.name: server name
+  // ctx.transport: 'stdio' | 'sse' | 'streamable-http'
+  // ctx.tools: namespaced tool names discovered on this server
+})
+
+agent.hooks.hook('mcp:error', (ctx) => {
+  // ctx.name: server name
+  // ctx.error: connection error
+})
+
+agent.hooks.hook('mcp:close', (ctx) => {
+  // ctx.name: server name being closed
+})
+
+agent.hooks.hook('mcp:tool:before', (ctx) => {
+  // ctx.server: MCP server name
+  // ctx.tool: original tool name (not namespaced)
+  // ctx.input: tool arguments
+})
+
+agent.hooks.hook('mcp:tool:after', (ctx) => {
+  // ctx.server, ctx.tool, ctx.input
+  // ctx.result: tool output string
+})
+
+agent.hooks.hook('mcp:tool:error', (ctx) => {
+  // ctx.server, ctx.tool, ctx.input, ctx.error
+})
+```
+
+### Steering inject
+
+```ts
+agent.hooks.hook('steer:inject', (ctx) => {
+  // ctx.message: the steering message being injected
+})
+```
+
+## Steering and Follow-up
+
+### Steering: interrupt mid-run
+
+Inject a message while the agent is working. Delivered between tool calls, skipping remaining tools in the current turn.
+
+```ts
+agent.hooks.hook('tool:after', () => {
+  agent.steer('focus only on the tests directory')
+})
+```
+
+### Follow-up, continue after done
+
+Queue messages that extend the conversation after the agent finishes.
+
+```ts
+agent.followUp('now write tests for what you built')
+agent.followUp('then update the README')
+```
+
+## Parallel Tool Execution
+
+Execute multiple tool calls from a single turn concurrently.
+
+```ts
+const agent = createAgent({
+  harness,
+  provider,
+  toolExecution: 'parallel', // default: 'sequential'
+})
+```
+
+## Image Content
+
+Pass images alongside the prompt.
+
+```ts
+import { readFileSync } from 'fs'
+
+await agent.run({
+  prompt: 'describe this screenshot',
+  images: [{
+    type: 'image',
+    source: {
+      type: 'base64',
+      media_type: 'image/png',
+      data: readFileSync('screenshot.png').toString('base64'),
+    },
+  }],
+})
+```
+
+## Message Format
+
+All messages in zidane use the canonical `SessionMessage` format, with or without sessions:
+
+```ts
+type SessionContentBlock =
+  | { type: 'text', text: string }
+  | { type: 'image', mediaType: string, data: string }
+  | { type: 'tool_call', id: string, name: string, input: Record<string, unknown> }
+  | { type: 'tool_result', callId: string, output: string, isError?: boolean }
+  | { type: 'thinking', text: string }
+
+interface SessionMessage {
+  role: 'user' | 'assistant'
+  content: SessionContentBlock[]
+}
+```
+
+Providers convert to and from native wire formats internally. Converters are available for external interop:
+
+```ts
+import { fromAnthropic, toAnthropic, fromOpenAI, toOpenAI, autoDetectAndConvert } from 'zidane'
+```
+
+## Usage Tracking
+
+Every turn reports token usage. Provider-specific fields are optional:
+
+```ts
+interface TurnUsage {
+  input: number
+  output: number
+  cacheCreation?: number  // Anthropic: tokens written to cache
+  cacheRead?: number      // Anthropic: tokens read from cache
+  thinking?: number       // thinking tokens used
+  cost?: number           // USD cost reported by provider (e.g. OpenRouter)
+}
+```
+
+Per-turn data is available on `AgentStats` and `SessionRun`:
+
+```ts
+const stats = await agent.run({ prompt: 'hello' })
+stats.turnUsage   // TurnUsage[] per turn
+stats.cost        // total cost (sum of per-turn costs, if reported)
+
+// In session runs
+session.runs[0].turnUsage   // per-turn breakdown
+session.runs[0].totalUsage  // aggregated TurnUsage
+session.runs[0].cost        // total cost for this run
+```
+
+## State Management
+
+```ts
+agent.isRunning        // boolean: is a run in progress?
+agent.messages         // SessionMessage[]: conversation history
+agent.execution        // ExecutionContext: where tools run
+agent.handle           // ExecutionHandle: spawned context handle
+agent.abort()          // cancel the current run
+agent.reset()          // clear messages and queues
+await agent.destroy()  // clean up execution context and MCP connections
+await agent.waitForIdle() // wait for current run to complete
+```
+
+## Project Structure
+
+```
+src/
+  types.ts              shared types
+  agent.ts              createAgent, AgentHooks, state management
+  loop.ts               turn execution loop
+  start.ts              CLI entrypoint
+  auth.ts               Anthropic OAuth flow
+  index.ts              package exports
+  contexts/
+    types.ts            ExecutionContext interface, capabilities
+    process.ts          in-process context (default)
+    docker.ts           Docker container context
+    sandbox.ts          remote sandbox context
+    index.ts            barrel exports
+  tools/
+    index.ts            tool exports
+    validation.ts       tool argument validation
+    shell.ts            shell tool
+    read-file.ts        read_file tool
+    write-file.ts       write_file tool
+    list-files.ts       list_files tool
+    spawn.ts            spawn tool and createSpawnTool factory
+  providers/
+    index.ts            Provider interface
+    openai-compat.ts    shared OpenAI-compatible utilities
+    anthropic.ts        Anthropic provider
+    openrouter.ts       OpenRouter provider
+    cerebras.ts         Cerebras provider
+  harnesses/
+    index.ts            HarnessConfig, defineHarness, ToolContext
+    basic.ts            basic harness (shell, read, write, list, spawn)
+  mcp/
+    index.ts            MCP server connection and tool discovery
+  session/
+    index.ts            Session interface, createSession, loadSession
+    messages.ts         SessionMessage converters (Anthropic/OpenAI)
+    memory.ts           in-memory session store
+    sqlite.ts           SQLite-backed session store
+    remote.ts           HTTP remote session store
+  output/
+    terminal.ts         terminal rendering (md4x)
+test/
+  mock-provider.ts      mock provider for testing
+  mock-context.ts       mock execution context for testing
+  agent.test.ts         agent loop tests
+  contexts.test.ts      execution context tests
+  harness.test.ts       harness tests
+  mcp.test.ts           MCP connection and hook tests
+  spawn.test.ts         spawn tool and hook tests
+  validation.test.ts    validation tests
+  providers.test.ts     provider tests
+  openai-compat.test.ts OpenAI-compat utility tests
+  session.test.ts       session store and agent integration tests
+  session-messages.test.ts  SessionMessage converter tests
+```
+
+## Testing
+
+```bash
+bun test
+```
+
+300 tests with mock provider and mock execution context, no LLM calls or Docker needed.
+
+## License
+
+ISC
+