rubino-agent 0.3.0

data/docs/api/v1.md

@@ -0,0 +1,414 @@
+# HTTP API v1
+
+Base URL: `http://<host>:<port>/v1` (default port 4820)
+Auth: `Authorization: Bearer <RUBINO_API_KEY>` on every request, EXCEPT `GET /v1/health` and `GET /v1/metrics`.
+Content type: `application/json` unless noted.
+
+Every endpoint listed below is covered by an end-to-end contract spec in `spec/rubino/api/contract/`. If a behavior here disagrees with the contract suite, the contract suite is the source of truth.
+
+## Conventions
+
+| | |
+|---|---|
+| **IDs** | UUIDv4 strings (`SecureRandom.uuid`). |
+| **Timestamps** | ISO 8601 UTC with the `Z` suffix. |
+| **Error envelope** | `{"error": {"code": "string", "message": "string", "details": {...}?}}` — identical shape on every error code. `details` is present only when the error carries one (notably 422). |
+| **Status codes** | 200/201/202/204 success · 401 auth · 404 missing resource or route · 409 conflict · 422 validation · 500 internal · 502 upstream. |
+| **SSE** | streams use `id: <seq>\nevent: <type>\ndata: <json-payload>\n\n`. Supports `Last-Event-ID` for replay. |
+| **Bearer** | scheme is case-insensitive (`Bearer` and `bearer` both accepted). Wrong/missing token → 401 envelope. |
+| **Routing 404** | unknown route returns 404 in the standard envelope with `code: "not_found"`. |
+
+---
+
+## Health & metrics (unauthenticated)
+
+### `GET /v1/health` → 200 | 503
+```json
+{
+  "status": "ok",
+  "version": "0.3.0",
+  "deps": {
+    "db":        { "status": "ok" },
+    "scheduler": { "status": "ok", "scheduled_jobs": 0 }
+  }
+}
+```
+Returns 503 with `status: "degraded"` when any dep reports non-ok.
+
+### `GET /v1/metrics` → 200 `text/plain; version=0.0.4`
+Prometheus text exposition. Registered metrics include `http_requests_total`, `http_request_duration_seconds`, `cron_fires_total`, `webhook_deliveries_total`, `oauth_token_exchanges_total`, `runs_total`, `runs_completed_total`, `skills_loaded_total`, `skills_created_total`. The two skill counters measure adoption vs. creation — see **[docs/skills.md](../skills.md#observability)**.
+
+---
+
+## Sessions
+
+### `GET /v1/sessions` → 200
+Session index, most recent first. Query: `limit` (default 20, max 100), `q` (full-text search over message content; hits are deduped to sessions).
+```json
+{
+  "sessions": [
+    { "id": "uuid", "title": "string|null", "status": "string",
+      "created_at": "ts", "updated_at": "ts",
+      "message_count": 12, "token_count": 3456 }
+  ]
+}
+```
+
+### `POST /v1/sessions` → 201
+```json
+// req — all fields optional
+{ "title": "string?", "parent_id": "uuid?", "instructions": "string?" }
+// res
+{ "id": "uuid", "title": "string|null", "parent_id": "uuid|null", "created_at": "ts" }
+```
+
+### `GET /v1/sessions/:id` → 200 | 404
+```json
+{
+  "id": "uuid",
+  "title": "string|null",
+  "instructions": null,
+  "status": "string",
+  "created_at": "ts",
+  "messages": [
+    { "id": "uuid", "role": "user|assistant|tool", "content": "string", "created_at": "ts" }
+  ]
+}
+```
+
+### `DELETE /v1/sessions/:id` → 204 | 404
+Cascade-deletes messages, runs, events, summaries, tool calls.
+
+### `POST /v1/sessions/:id/retry` → 202 | 404 | 409
+Deletes the last user message (and everything after it), enqueues a fresh run with the same input.
+```json
+{ "run_id": "uuid", "session_id": "uuid", "status": "running" }
+```
+409 when the session has no user message to retry.
+
+### `POST /v1/sessions/:id/undo` → 200 | 404 | 409
+Removes the last user message and everything after it (no re-run).
+```json
+{ "removed_messages": 3 }
+```
+
+---
+
+## Runs
+
+### `POST /v1/sessions/:id/runs` → 201 | 404 | 422
+```json
+// req
+{
+  "input":       "string",                 // required
+  "attachments": ["string"]?,
+  "skills":      ["string"]?,
+  "model":       "string|null",
+  "provider":    "string|null"
+}
+// res
+{ "id": "uuid", "session_id": "uuid", "status": "running", "created_at": "ts" }
+```
+The run is dispatched to a background Executor immediately; tail `/v1/runs/:id/events` for state.
+
+### `POST /v1/runs/:id/stop` → 200 | 404
+Cooperative stop — flips the `stop_requested` flag; the executor exits between turns.
+```json
+{ "id": "uuid", "status": "stop_requested" }
+```
+
+### `GET /v1/runs/:id/events` → 200 (SSE) | 404
+Replays persisted events first (honoring `Last-Event-ID`), then polls for new ones until the run reaches a terminal status (`completed`, `failed`, `stopped`) or disappears. Headers:
+```
+content-type:       text/event-stream
+cache-control:      no-cache
+x-accel-buffering:  no
+```
+Event types that can appear on the stream: `message.completed`, `tool.started`, `tool.progress`, `tool.completed`, `artifact.created`, `input.injected`, `skill.loaded`, `subagent.spawned`, `subagent.completed`, `subagent.failed`, `approval.required` (`{approval_id, question, tool, command, choices, ...}`), `approval.decided` (`{approval_id, decision}`), `approval.expired` (`{approval_id}`), `clarify.required` (`{clarify_id, question}`), `run.completed`, `run.failed`, `run.stopped`.
+
+`approval.decided` follows the `approval.required` frame once a decision lands (posted via the approvals endpoint), carrying the chosen `decision`; `approval.expired` fires instead if the gate's await deadline elapses first, after which the server treats the call as a safe deny. `run.stopped` is emitted on the clean-stop branch (a stop request that the run honored) — it is both this distinct event AND the terminal `stopped` status.
+
+Unknown event types should be ignored rather than treated as errors, so a future addition never breaks a client.
+
+The `id:` sequence is a global monotonic counter: ids keep climbing across runs, so they are NOT per-run ordinals. The stream is already scoped to one run (`for_run`), so use the `id:` only for `Last-Event-ID` replay, not to infer how many events a run produced.
+
+API runs are **non-streaming**: the full answer arrives in the final `run.completed` frame. No `message.delta` is ever emitted — the agent loop disables token streaming for every API run (the always-registered `question` tool makes each run an interactive turn) — and there is no `reasoning.delta` event anywhere in the system. Do not wait for incremental text deltas.
+
+---
+
+## Approvals & clarifications
+
+Both endpoints unblock an in-process gate registered when the run started. Decisions sent to a run with no live gate return 409.
+
+### `POST /v1/runs/:run_id/approvals/:approval_id` → 200 | 404 | 409 | 422
+```json
+// req
+{ "decision": "once|session|always|always_prefix|always_command|deny|deny_always" }
+// res
+{ "approval_id": "string", "decision": "string" }
+```
+The enum is closed — anything else fails validation with 422. The approve values are kept in sync with `UI::API::APPROVE_DECISIONS`, plus the two explicit deny forms:
+
+| decision | effect |
+|---|---|
+| `once` | approve this call only |
+| `session` | approve this scope for the rest of the session (in-memory) |
+| `always_prefix` | approve + persist an allow rule for the command's prefix (offered only when a prefix rule is derivable) |
+| `always_command` | approve + persist an allow rule for this exact command |
+| `always` | back-compat alias for `always_command` (older clients post it) |
+| `deny` | deny this call once — nothing persisted, re-prompts next time |
+| `deny_always` | deny + persist a `permissions:deny` rule, auto-denied across sessions |
+
+The `approval.required` SSE frame carries a `choices` array listing the decisions offered for THAT request (e.g. `session` only when session caching applies, `always_prefix` only with a derivable prefix); clients should render `choices` rather than hardcoding the enum.
+
+### `POST /v1/runs/:run_id/clarifications/:clarify_id` → 200 | 404 | 409 | 422
+```json
+// req
+{ "response": "string" }
+// res
+{ "clarify_id": "string", "accepted": true }
+```
+
+---
+
+## Skills
+
+### `GET /v1/skills` → 200
+```json
+[{ "name": "string", "description": "string", "enabled": true }]
+```
+
+### `PUT /v1/skills/:name` → 200 | 404 | 422
+```json
+// req
+{ "enabled": true }
+// res
+{ "name": "string", "enabled": true }
+```
+
+---
+
+## Models
+
+### `GET /v1/models` → 200
+```json
+[{ "id": "provider/model", "provider": "string|null", "context_window": 128000 }]
+```
+Source defaults to `RubyLLM.models.all`; some fields may be `null` for models that don't expose them.
+
+---
+
+## Mode
+
+Process-level toggle that gates tool availability + approval policy. Lives in the Ruby heap (see `Rubino::Modes`); a restart resets to `default`. Valid values: `"default"`, `"plan"`, `"yolo"`.
+
+| mode | effect |
+|---|---|
+| `default` | all tools registered, approval rules from config |
+| `plan` | read-only tools only (no edits/shell/git) |
+| `yolo` | all tools, approval policy bypassed |
+
+### `GET /v1/mode` → 200
+```json
+{
+  "mode": "default",
+  "description": "all tools, approvals from config",
+  "available": [
+    { "mode": "default", "description": "..." },
+    { "mode": "plan",    "description": "..." },
+    { "mode": "yolo",    "description": "..." }
+  ]
+}
+```
+The minimum guaranteed shape is `{ "mode": "default"|"plan"|"yolo" }`; `description` and `available` are exposed so clients can render a picker without hardcoding the catalogue.
+
+### `PUT /v1/mode` → 200 | 422
+```json
+// req
+{ "mode": "default"|"plan"|"yolo" }
+// res
+{ "mode": "plan", "previous": "default", "description": "..." }
+```
+Emits the same `mode_changed` UI event the CLI fires on `/mode plan`, so any in-flight SSE stream notices. An invalid `mode` (missing, wrong type, or not in the enum) returns the canonical error envelope:
+```json
+{ "error": { "code": "validation", "message": "invalid request body", "details": { "errors": { "mode": ["..."] } } } }
+```
+
+---
+
+## Files
+
+Workspace is sandboxed under `config.paths.home`. Path traversal raises `Workspace::PathTraversal < ValidationError` → 422.
+
+### `GET /v1/files?path=relative/path` → 200 | 404 | 422
+Raw bytes as `application/octet-stream`. 422 when `path` query is missing/empty or escapes the sandbox. 404 when no file is at that path.
+
+### `POST /v1/files` → 201 | 422
+Multipart `multipart/form-data` with a `file` part. 422 when the content-type is not multipart, or the `file` field is missing.
+```json
+{ "id": "uuid", "filename": "string", "size": 12345 }
+```
+
+---
+
+## Cron jobs
+
+In-process `rufus-scheduler` singleton. The scheduler does not survive multi-process scale-out (every worker would run every tick).
+
+### `POST /v1/jobs` → 201 | 422
+```json
+// req
+{
+  "name":     "string",          // required
+  "schedule": "cron string",     // required
+  "prompt":   "string",          // required
+  "skills":   ["string"]?,
+  "model":    "string|null",
+  "provider": "string|null",
+  "deliver":  "local|webhook"    // optional, default "local"
+}
+// res — full serialized job
+{ "id": "uuid", "name": "string", "schedule": "string", "prompt": "string",
+  "skills": [], "model": null, "provider": null, "deliver": "local",
+  "enabled": true, "last_run_at": null, "last_run_id": null,
+  "created_at": "ts", "updated_at": "ts" }
+```
+
+### `GET /v1/jobs?include_disabled=true|false` → 200
+Array of serialized jobs. `include_disabled=false` filters out paused.
+
+### `GET /v1/jobs/:id` → 200 | 404
+### `PATCH /v1/jobs/:id` → 200 | 404 | 422
+Partial update. Same fields as create plus `enabled`. `skills` is an Array (returns as `skills`, never `skills_json` — the JSON column is internal).
+
+### `DELETE /v1/jobs/:id` → 204 | 404
+Removes the row AND unschedules the rufus handle.
+
+### `POST /v1/jobs/:id/pause` → 200
+Disables + unschedules. Returns the serialized job.
+
+### `POST /v1/jobs/:id/resume` → 200
+Enables + reschedules.
+
+### `POST /v1/jobs/:id/trigger` → 202 | 404
+Fires one run immediately, returns a run reference.
+```json
+{ "job_id": "uuid", "run_id": "uuid", "session_id": "uuid" }
+```
+
+Webhook delivery: when `deliver: "webhook"`, on run completion rubino POSTs to `RUBINO_WEBHOOK_URL` with `{ job_id, run_id, status, session_id }`.
+
+---
+
+## Memory
+
+The persistent memory store (same backend the CLI's `rubino memory` uses — see [`docs/memory.md`](../memory.md)).
+
+### `GET /v1/memory` → 200
+Query: `limit` (default 50, max 200), `offset`, `q` (substring filter on content).
+```json
+{
+  "memory": [
+    { "id": "uuid", "kind": "string", "content": "string",
+      "created_at": "ts", "updated_at": "ts" }
+  ]
+}
+```
+
+### `GET /v1/memory/stats` → 200
+```json
+{ "backend": "sqlite", "count": 42 }
+```
+
+### `DELETE /v1/memory/:id` → 204 | 404
+
+---
+
+## Tasks
+
+Background subagent runs spawned by the `task` tool (in-process registry; entries do not survive a restart).
+
+### `GET /v1/tasks` → 200
+```json
+{
+  "tasks": [
+    { "id": "string", "subagent": "string", "prompt": "string",
+      "status": "running|completed|failed|cancelled",
+      "started_at": "ts|null", "elapsed_seconds": 1.234,
+      "result_summary": "string|null" }
+  ]
+}
+```
+
+### `GET /v1/tasks/:id` → 200 | 404
+The summary shape above plus `finished_at`, the full `result`, and `error`.
+
+### `POST /v1/tasks/:id/stop` → 202 | 404 | 409
+Cooperative cancel of a running task (descendant ask-gates are cancelled too). Returns the task detail; 409 when the task already finished.
+
+---
+
+## OAuth
+
+See [`docs/oauth-providers.md`](../oauth-providers.md) for the full PKCE flow, encryption key requirements, and per-provider setup. The HTTP surface:
+
+### `GET /v1/oauth/providers` → 200
+Lists providers registered at boot via `OAuth::Registry.load_from_config!`. Empty when no `oauth.providers.*` section in config carries both `client_id` and `client_secret`.
+```json
+[{ "id": "github", "display_name": "Github", "scopes": ["repo", "user:email"] }]
+```
+
+### `POST /v1/oauth/providers/:id/connect` → 200 | 404 | 422
+Builds a PKCE authorize request. The client MUST persist `state` and `code_verifier` between connect and callback — rubino is stateless on the OAuth flow.
+```json
+// req
+{ "redirect_uri": "https://your-client/oauth/callback", "scopes": ["string"]? }
+// res
+{
+  "authorize_url": "https://provider.example/authorize?...",
+  "state":         "string",
+  "code_verifier": "string",
+  "provider":      "github"
+}
+```
+
+### `POST /v1/oauth/providers/:id/callback` → 201 | 404 | 422 | 502
+The client posts the code it received at `redirect_uri`, plus the `state`/`code_verifier` it kept from connect, plus the `expected_state` it stored (constant-time compared against `state`).
+```json
+// req — every field required
+{
+  "code":            "string",
+  "state":           "string",
+  "expected_state":  "string",
+  "code_verifier":   "string",
+  "redirect_uri":    "string"
+}
+// res 201 — serialized connection, tokens stripped
+{
+  "id":            "uuid",
+  "provider":      "github",
+  "account_id":    "string",
+  "account_email": "string|null",
+  "expires_at":    "ts|null",
+  "scopes":        ["string"],
+  "metadata":      { },
+  "created_at":    "ts",
+  "updated_at":    "ts"
+}
+```
+422 when `state != expected_state`. 502 when token exchange against the provider raises (counted under `oauth_token_exchanges_total{outcome="error"}`).
+
+### `GET /v1/oauth/connections` → 200
+Array of serialized connections. Tokens are NEVER returned over the wire.
+
+### `DELETE /v1/oauth/connections/:id` → 204 | 404
+Removes the row (encrypted tokens included). Does not attempt provider-side revocation.
+
+---
+
+## Operational notes
+
+- **Auth bypass** — only `/v1/health` and `/v1/metrics` (see `Auth::SKIP_PATHS`). Every other route requires a valid bearer.
+- **Token redaction** — the structured logger masks `access_token`, `refresh_token`, `id_token`, `client_secret`, `api_key`, `password`, `secret`, `bearer`, `authorization`, `http_authorization` (case-insensitive, recursive) before serialization.
+- **Single process** — the scheduler, gate registry, and event polling all live in the Ruby heap. Scaling to multiple workers would require pulling these out (Redis, Postgres LISTEN, etc.).
+- **Encryption key** — `RUBINO_ENCRYPTION_KEY` (32-byte base64) is required to use any OAuth route. Boot fails fast without it.

data/docs/architecture.md

@@ -0,0 +1,177 @@
+# Architecture
+
+## Overview
+
+rubino is a lightweight agent that runs on a PC or inside a VM. It follows a
+layered architecture with strict separation of concerns:
+
+```
+Presentation Layer     →  CLI, JSON API Server
+Orchestration Layer    →  Agent Router, Interaction Lifecycle
+Core Layer             →  Agent Loop, Context, Memory, Jobs, Tools
+Infrastructure Layer   →  LLM Adapter, Database, MCP, OAuth
+```
+
+## Key Design Principles
+
+1. **All output goes through UI** — No `puts`/`print` in core modules
+2. **LLM is isolated** — Only `LLM::RubyLLMAdapter` talks to ruby_llm
+3. **SQLite is the single database** — Sessions, memory, jobs, events
+4. **Event-driven** — Core emits events, UI/plugins subscribe
+5. **Plugin hooks** — 38 declared extension points for customization (design surface; few are wired today)
+6. **Config is not architecture** — Configuration describes what; architecture decides how
+
+## Module Map
+
+### `agent/`
+Multiple agent types and @mention routing exist as a design surface; the
+rubino runs a single agent by default and multi-agent routing is dormant.
+- `AgentRegistry` — Defines all agent types (build, plan, explore, general, utility)
+- `Router` — Routes input to appropriate agent via @mention
+- `Definition` — Agent type with model, tools, permissions, MCP scoping
+- `Runner` — Top-level orchestrator for a user interaction
+- `Loop` — Core LLM call + tool execution cycle
+- `IterationBudget` — Prevents runaway loops
+- `ToolExecutor` — Executes tools with approval and result formatting
+
+### `interaction/`
+- `Lifecycle` — Full turn lifecycle: input → memory → context → model → tools → persist → jobs
+- `State` — State machine (idle → calling_model → executing_tools → finished)
+- `EventBus` — Pub/sub for decoupling core from UI
+- `Events` — All typed event constants
+
+### `context/`
+- `PromptAssembler` — Builds the full prompt from all sources
+- `TokenBudget` — Calculates token usage and decides when to compact (`needs_compaction?`)
+- `Compressor` — Orchestrates compaction (flush memory → split → summarize → lineage)
+- `MessageBoundary` — Splits messages into head/middle/tail
+- `SummaryBuilder` — Generates structured summaries via LLM
+- `ToolPairSanitizer` — Keeps tool_call/result pairs intact
+- `FileDiscovery` — Finds project context files (.rubino.md, AGENTS.md, etc.)
+
+### `memory/`
+- `Store` — CRUD for memories (7 kinds: user_profile, preference, fact, etc.)
+- `Retriever` — Loads relevant memories for prompt inclusion
+- `Extractor` — Pattern-based extraction from conversations
+- `Deduplicator` — Jaccard similarity deduplication
+- `Flusher` — Pre-compaction memory flush
+
+### `session/`
+- `Repository` — Session CRUD with prefix-matching find
+- `Store` — Message persistence
+- `Message` — Value object with to_context / to_row
+
+Forking is not a dedicated class: a new session inherits history via the
+API's `parent_session_id` path.
+
+### `jobs/`
+- `Queue` — SQLite-backed job queue with priority and scheduling
+- `Runner` — Executes jobs, records runs
+- `Worker` — Polling loop for background processing
+- `Registry` — Maps job types to handler classes
+- Handlers: ExtractMemory, SummarizeSession, CompactSession, CleanupSessions, DistillSkill
+
+### `tools/`
+- `Base` — Abstract tool interface (name, description, input_schema, risk_level, call)
+- `Registry` — Singleton registry with enable/disable
+- `Result` — Structured result (success/error/denied)
+- The built-in tools (authoritative, drift-checked count and list in [tools.md](tools.md)) + custom tool loader + formatter integration
+- `CustomToolLoader` — DSL for user-defined tools
+
+### `llm/`
+- `RubyLLMAdapter` — Wraps ruby_llm (chat, stream, structured output)
+- `ProviderResolver` — Auto-detects provider from model name
+- `ModelRegistry` — Known models with context windows
+- `ContentBuilder` — Multipart content for vision (text + images)
+
+### `mcp/`
+Experimental — booted at chat startup when `mcp.servers` is configured
+(see [mcp.md](mcp.md)).
+- `Manager` — Manages multiple MCP client connections
+- `MCPToolWrapper` — Wraps MCP tools into Tools::Base interface
+
+### `security/`
+- `ApprovalPolicy` — Decides allow/ask/deny per tool call
+- `PatternMatcher` — Wildcard pattern matching for permissions
+- `DoomLoopDetector` — Detects repeated identical tool calls
+- `CommandAllowlist` — Pre-approved shell commands
+
+### `plugins/`
+- `Registry` — Central hook registry; the hook set (38 points) is declared in
+  `plugins.rb` as a design surface, with few hooks wired today
+- Loaded from `.rubino/plugins/`
+
+### `skills/`
+- `Skill` — Parsed SKILL.md with YAML frontmatter
+- `Registry` — Discovery from configured paths
+- `SkillTool` — Tool for on-demand skill loading
+
+### `commands/`
+- `Command` — Parsed command.md with template rendering
+- `Loader` — Discovery from configured paths
+- `Executor` — Handles slash commands and built-ins
+
+### `api/`
+- `Server` — Rack + Puma boot
+- `Router` — pattern-based dispatcher
+- `Middleware::{Auth,ErrorHandler,JsonParser}` — Bearer auth, typed-error mapping, JSON body parsing
+- `Operations::*` — request handlers (sessions, runs, approvals, clarifications, skills, models, files, cron jobs, oauth)
+
+### `oauth/`
+- `Provider` (+ `Github`, `Google`) — provider abstraction with PKCE auth flow
+- `Registry` — process-wide registry hydrated from config
+- `ConnectionRepository` — encrypted token persistence (AES-256-GCM via `TokenEncryptor`)
+
+### `config/`
+- `Loader` — Basic YAML loader
+- `EnhancedLoader` — Multi-layer precedence with substitutions
+- `RemoteConfig` — Enterprise remote config fetching
+- `Configuration` — Typed accessors for all config sections
+- `Writer` — Persists config changes
+- `Defaults` — All default values
+
+### `database/`
+- `Connection` — SQLite + WAL mode via Sequel
+- `Migrator` — Versioned migrations
+
+### `ui/`
+- `Base` — Abstract interface (info, error, stream, table, ask, confirm, etc.)
+- `CLI` — TTY-based terminal output
+- `Null` — Silent adapter for testing
+- `API` — Structured event collector
+
+## Data Flow
+
+```
+User Input
+  │
+  ├─→ Commands::Executor (if /command)
+  │     └─→ Render template → feed to agent
+  │
+  ├─→ Agent::Router (if @mention)
+  │     └─→ Select agent definition
+  │
+  └─→ Interaction::Lifecycle
+        │
+        ├─ Persist user message
+        ├─ Load memory (Retriever)
+        ├─ Extract images (ContentBuilder)
+        ├─ Build context (PromptAssembler)
+        ├─ Check token budget (TokenBudget)
+        ├─ Compact if needed (Compressor)
+        │
+        ├─ Agent::Loop
+        │    ├─ Call LLM (RubyLLMAdapter)
+        │    ├─ Stream to UI
+        │    ├─ If tool_calls:
+        │    │    ├─ Check permissions (ApprovalPolicy)
+        │    │    ├─ Check doom loop (DoomLoopDetector)
+        │    │    ├─ Execute tool (ToolExecutor)
+        │    │    ├─ Run plugin hooks
+        │    │    └─ Loop back to LLM
+        │    └─ Final text response
+        │
+        ├─ Persist session
+        ├─ Enqueue jobs (extract memory, summarize)
+        └─ Emit events → UI + SSE clients
+```