ai-runtime-engine 1.1.0

package/docs/GUIDE.md

@@ -0,0 +1,358 @@
+# AI Runtime — a hands-on guide
+
+A start-to-finish guide to **installing and using `ai-runtime-engine`**. It's written to be *studied*: read
+top to bottom, run the snippets as you go, and you'll understand both **how it works** and **how to use it**.
+
+You do **not** need an API key to follow along — a built-in mock provider lets you run everything offline.
+
+**Naming, once:** the **product** is *AI Runtime*, the **npm package** is `ai-runtime-engine`, and the
+**command-line tool** is `ai-runtime`. In code you use two classes: **`Runtime`** (the high-level API you'll
+use most) and **`AI`** (the low-level router underneath it).
+
+---
+
+## Contents
+
+1. [What you're installing](#1-what-youre-installing)
+2. [Install](#2-install)
+3. [Your first run (offline, no keys)](#3-your-first-run-offline-no-keys)
+4. [How it works — the mental model](#4-how-it-works--the-mental-model)
+5. [Using real providers](#5-using-real-providers)
+6. [How to use it — worked examples](#6-how-to-use-it--worked-examples)
+7. [The interactive terminal](#7-the-interactive-terminal)
+8. [Configuration](#8-configuration)
+9. [Safety model](#9-safety-model)
+10. [Where to go next](#10-where-to-go-next)
+
+---
+
+## 1. What you're installing
+
+`ai-runtime-engine` is two things in one package:
+
+- **A library** — `import { Runtime, AI } from 'ai-runtime-engine'` to embed the runtime in your own code.
+- **A CLI** — the `ai-runtime` command, for an interactive terminal and one-shot runs.
+
+The core idea: you configure whatever AI providers you have (cloud APIs, a local model, an internal
+endpoint), and the runtime **routes each request to the best available model** and runs it — so you write
+*what you want*, not *which provider to call*. Around that it adds modes, memory, skills, tools, planning,
+orchestration, comparison, and learning.
+
+Requirements: **Node.js 22+**.
+
+---
+
+## 2. Install
+
+As a dependency in your project (gives you the library and `npx ai-runtime`):
+
+```bash
+npm install ai-runtime-engine
+```
+
+Or globally, to get the `ai-runtime` command on your PATH:
+
+```bash
+npm install -g ai-runtime-engine
+ai-runtime --help
+```
+
+---
+
+## 3. Your first run (offline, no keys)
+
+You can run the whole pipeline with a **mock provider** — no credentials, no network. Create `first.mjs`:
+
+```js
+import { Runtime, MockProvider, makeModel } from 'ai-runtime-engine';
+
+// A fresh runtime with no persistence (nothing written to disk).
+const runtime = new Runtime();
+
+// Register one fake provider with one fake model that can "reason" and output "text".
+runtime.ai.registerProvider(new MockProvider({
+  id: 'demo',
+  models: [makeModel({
+    providerId: 'demo',
+    id: 'demo-1',
+    caps: { intelligence: ['reasoning'], output: ['text'] },
+  })],
+}));
+
+const result = await runtime.run({ input: 'Say hello and explain what you are.' });
+
+console.log('ok:', result.ok);
+console.log('text:', result.response?.text);
+console.log('chose:', result.routing?.selected);        // which provider/model was picked
+console.log('why:', result.routing?.ranked?.length, 'candidate(s) considered');
+```
+
+```bash
+node first.mjs
+```
+
+You'll see `ok: true`, a mock response, and a **routing trace** showing the decision. That trace is always
+there — every run tells you *what it did and why*.
+
+---
+
+## 4. How it works — the mental model
+
+A single `run()` flows through one path. You supply the capabilities; the runtime decides how to use them.
+
+```
+runtime.run({ input })
+   │
+   ▼  1. Pick a MODE — auto-detected, or you set it: chat · plan · execute · orchestrate · agent · compare · debug
+   │
+   ▼  2. Build the CONTEXT under a token budget (a one-line workspace summary, relevant memory, your request)
+   │
+   ▼  3. ROUTE to a model  ──►  the ONE router:
+   │        filter (capability · health · privacy · cost)  →  score  →  pick  →  execute  →  validate
+   │        →  fall back to the next-best model on failure
+   │
+   ▼  4. For multi-step work: PLAN → run tools/skills → succeed a step only on EVIDENCE → replan on failure
+   │
+   ▼  5. RESULT { ok, response, routing trace, + plan / comparison / memory as relevant }
+```
+
+Five ideas are worth internalizing:
+
+- **Providers are replaceable; capabilities are durable.** You route by *what a task needs* (reasoning,
+  JSON output, a big context window…), not by a provider name. Swap providers freely.
+- **One router.** *Every* model call — chat, planning, comparison, a skill's own call — goes through the
+  same router. There is never a second, hidden selection engine.
+- **`Runtime` vs `AI`.** `AI` is the low-level router (`ai.run({ task, input })`). `Runtime` wraps it and
+  adds modes, memory, skills, orchestration, etc. (`runtime.run({ input })`). Use `Runtime`; reach for `AI`
+  only when you want the bare router.
+- **Modes = what kind of work.** A quick answer is `chat`; a multi-step job is `orchestrate`; comparing
+  models is `compare`. `auto` (the default) guesses from your request.
+- **Evidence over claims.** A step "succeeds" only when there's proof — a tool returned ok, a validator
+  passed — never because a model said "done."
+
+---
+
+## 5. Using real providers
+
+To use a real model, name a provider and put its key in `.env`. The **fastest** way is the CLI:
+
+```bash
+ai-runtime setup     # detects your project, writes .ai-runtime/config.yaml + .env.example (asks first)
+ai-runtime doctor    # tells you which providers are configured and reachable
+```
+
+Then add a key to `.env`. Recommended starters:
+
+| Provider | Type | Key env var | Get a key |
+|---|---|---|---|
+| Google Gemini | Cloud | `GEMINI_API_KEY` | https://aistudio.google.com/apikey |
+| Groq | Cloud | `GROQ_API_KEY` | https://console.groq.com/keys |
+| OpenRouter | Cloud (aggregator) | `OPENROUTER_API_KEY` | https://openrouter.ai/keys |
+| Ollama | **Local, no key** | — | https://ollama.com |
+
+A minimal config (env-var **names** only — never the value):
+
+```yaml
+# .ai-runtime/config.yaml
+runtime:
+  defaultMode: auto
+providers:
+  - id: gemini
+    kind: gemini
+    apiKeyEnv: GEMINI_API_KEY
+  - id: ollama          # local; no key required
+    kind: ollama
+```
+
+In code, real providers load from that config:
+
+```js
+import { Runtime } from 'ai-runtime-engine';
+
+const runtime = await Runtime.load({ workspace: process.cwd() });   // reads config + .env
+const r = await runtime.run({ input: 'Summarize what this project does.' });
+console.log(r.response?.text);
+```
+
+---
+
+## 6. How to use it — worked examples
+
+The examples below use the offline mock so you can run them as-is; with real providers configured, drop the
+`registerProvider` line and use `await Runtime.load()` instead.
+
+```js
+import { Runtime, MockProvider, makeModel } from 'ai-runtime-engine';
+
+function demoRuntime() {
+  const rt = new Runtime();
+  rt.ai.registerProvider(new MockProvider({
+    id: 'demo',
+    models: [makeModel({ providerId: 'demo', id: 'demo-1', caps: { intelligence: ['reasoning', 'planning'], output: ['text', 'json'] } })],
+  }));
+  return rt;
+}
+```
+
+### a. Chat — a single answer
+
+```js
+const rt = demoRuntime();
+const r = await rt.run({ input: 'Explain what a race condition is.', mode: 'chat' });
+console.log(r.response?.text);
+```
+
+### b. Ask for a plan (no execution)
+
+```js
+const r = await rt.run({ input: 'Migrate this project off deprecated APIs', mode: 'plan' });
+console.log(r.plan);   // a structured plan; nothing is executed in `plan` mode
+```
+
+### c. Preview a multi-step task with dry-run (zero changes)
+
+```js
+const r = await rt.run({ input: 'Fix the failing tests', mode: 'orchestrate', dryRun: true });
+console.log(r.response?.text);   // shows the plan + what it WOULD touch, with no mutations
+```
+
+To let it actually act (write files, run shell, etc.), you grant permissions in config — everything is
+deny-by-default (see [Safety model](#9-safety-model)).
+
+### d. Compare models
+
+`compare` fans the same request across several models and reports where they agree, differ, and contradict
+each other:
+
+```js
+const rt = new Runtime();
+rt.ai.registerProvider(new MockProvider({ id: 'a', models: [makeModel({ providerId: 'a', id: 'a-1', caps: { intelligence: ['reasoning'], output: ['text'] } })], behavior: { kind: 'ok', text: 'use PostgreSQL' } }));
+rt.ai.registerProvider(new MockProvider({ id: 'b', models: [makeModel({ providerId: 'b', id: 'b-1', caps: { intelligence: ['reasoning'], output: ['text'] } })], behavior: { kind: 'ok', text: 'use MongoDB' } }));
+
+const cmp = await rt.compare({ goal: 'which database?', input: 'Recommend a database', pins: [{ provider: 'a' }, { provider: 'b' }] });
+console.log(cmp.analysis.unanimous, cmp.analysis.differences);
+```
+
+If you supply a `validate` function (e.g. run the tests), the answer that *passes* wins — evidence beats a
+judge model.
+
+### e. Remember facts across runs
+
+With persistence on (the default when you `Runtime.load()`), memory is durable and scoped:
+
+```js
+const rt = await Runtime.load();
+rt.memory.remember({ text: 'This project uses PostgreSQL', scope: 'project' });
+console.log(rt.memory.search('database'));   // finds the fact; it's injected into future chats
+```
+
+You can also just say it in chat: *"Remember that we deploy on Fridays."*
+
+### f. Add your own skill (no code)
+
+Drop a manifest into `.ai-runtime/skills/` and it's auto-loaded. A skill composes the built-in tools:
+
+```yaml
+# .ai-runtime/skills/read-readme.skill.yaml
+id: read-readme
+version: 1.0.0
+name: Read README
+description: read the project README
+steps:
+  - tool: filesystem
+    input: { op: read, path: README.md }
+```
+
+Run `ai-runtime skills` to see it loaded, or `ai-runtime skills --discover` to find skill files elsewhere in
+your repo (reported, not auto-loaded). Skills can also be shipped as npm packages — name them under
+`skills.packages` in config.
+
+### g. Steer which models get used
+
+- **Exclude** = a hard filter (never selected). **Prefer** = a soft nudge. **Neither** = normal routing.
+
+```js
+const r = await rt.run({ input: 'anything', routing: { excludeProviders: ['a'], preferProviders: ['b'] } });
+```
+
+Or via environment variables (`AI_EXCLUDE_PROVIDERS`, `AI_PREFER_MODELS`, …). Learning can nudge preferences
+from real outcomes, but **it can never override an exclusion**.
+
+---
+
+## 7. The interactive terminal
+
+The most fun way to explore. From a project that installed the package (or globally):
+
+```bash
+ai-runtime            # opens the interactive terminal
+```
+
+Type a request in plain language, or use a slash command:
+
+```
+/help                 list everything
+/info  /status        what's configured
+/providers            providers + access state
+/providers/all models  every provider's models
+/mode plan            set the mode
+/memory               list remembered facts
+/skills discover      find skill files in the repo
+/compare <request>    compare across providers
+/dry-run              toggle "plan only, no changes"
+/doctor  /cleanup     diagnostics · maintenance
+/exit
+```
+
+For scripts and CI, the one-shot form runs a single request:
+
+```bash
+ai-runtime run "Explain this error: TypeError: x is not a function" --json
+```
+
+---
+
+## 8. Configuration
+
+Three places, and only these:
+
+| Path | What | Commit? |
+|---|---|---|
+| `.ai-runtime/config.yaml` | Project config — settings + providers (env-var **names**). | ✅ yes |
+| `.env` | The secret **values** for those env vars. | ❌ no (gitignored) |
+| `~/.ai-runtime/` | Local state — conversations, memory, executions, cache, learning. | ❌ never |
+
+Handy optional environment variables: `AI_DEFAULT_MODE`, `AI_CONTEXT_MAX_TOKENS`, `AI_MAX_COST_USD`,
+`AI_MAX_CALLS`, `AI_MEMORY_ENABLED`, `AI_LEARNING_ENABLED`, `AI_EXCLUDE_*` / `AI_PREFER_*`,
+`AI_*_RETENTION_DAYS`, `AI_RUNTIME_HOME`. All optional; unset means the default.
+
+For the full router configuration (routing strategies, scoring weights, privacy posture, self-declared
+providers, the `run()`/`AI` reference), see [router.md](router.md).
+
+---
+
+## 9. Safety model
+
+The runtime is built to be safe to point at your real files and shell:
+
+- **Deny-by-default permissions.** Reads are allowed and **jailed to your workspace**; writes, shell, git
+  commit/push, and network are **off** until you grant them under `permissions:` in config. Destructive
+  shell commands always ask.
+- **Untrusted data.** Anything read from files, tools, the web, or MCP is treated as *data, never
+  instructions* — it can't change permissions, obtain secrets, or run commands.
+- **Secrets** are named by env var only; an inline key is rejected, and every value is redacted from all
+  output (including the terminal).
+
+Details in [security.md](security.md).
+
+---
+
+## 10. Where to go next
+
+- **[../README.md](../README.md)** — the concise reference (install, CLI, config at a glance).
+- **[router.md](router.md)** — the engine: routing strategies, scoring, providers, the `run()`/`AI` API.
+- **[architecture.md](architecture.md)** — how the router and Runtime layers fit together.
+- **[security.md](security.md)** — the full security model.
+
+Happy building. Remember the through-line: **describe what you want; the runtime picks the model and proves
+the result.**

package/docs/README.md

@@ -0,0 +1,21 @@
+# AI Runtime — documentation
+
+User-facing usage lives in the top-level [README](../README.md). These are the reference docs that ship
+with the `ai-runtime-engine` package:
+
+- **[GUIDE.md](GUIDE.md)** — a hands-on, study-friendly guide: install → first run → how it works → worked
+  examples (runnable offline, no API key needed).
+- **[architecture.md](architecture.md)** — the two layers, the one-router invariant, the shared
+  abstractions, the request lifecycle, and what is frozen.
+- **[router.md](router.md)** — the router engine reference: configuration, the `run()` API and `AI` class,
+  providers, tasks/capabilities, routing internals, reliability, and advanced features.
+- **[security.md](security.md)** — the untrusted-data boundary, filesystem jail, shell allowlist,
+  permissions, credentials, and the learning safety rail.
+
+Additional development records — the append-only decision log (`DECISIONS.md`), the phase-by-phase build
+log (`PROGRESS.md`), and the release checklist (`NPM_PUBLISHING.md`) — live in the (private) source
+repository and are not part of the published package.
+
+Per-subsystem detail is documented at the source: each module under `src/**` opens with a doc comment
+describing its contract and invariants, and the README's [How to use it](../README.md#how-to-use-it)
+section covers every capability from a user's perspective.

package/docs/architecture.md

@@ -0,0 +1,78 @@
+# Architecture
+
+AI Runtime is two layers: a provider-agnostic **router** and, on top of it, a **Runtime** that turns the
+router into an orchestration platform. This document is the map; the [README](../README.md) is the usage
+guide. (An append-only decision log, `DECISIONS.md`, records the rationale for each load-bearing choice in
+the source repository.)
+
+## The one invariant
+
+**There is exactly one selection engine.** Every model call — chat, planning, comparison, a skill's own
+`AI.run`, everything — goes through the router (`AI.run()`). The Runtime never scores or picks models
+itself; it decides *what kind of work* a request is and *how* to carry it out, then delegates each model
+call to the router. This keeps routing behavior consistent and testable, and keeps the router reusable on
+its own.
+
+## Layering
+
+| Layer | Question it answers | Where |
+|---|---|---|
+| Router | Which provider/model performs this AI operation? | `src/core/**`, `src/providers/**`, `src/plugin/ai.ts` |
+| Runtime | What is being asked; which mode handles it? | `src/runtime/**` |
+| Planner | What sequence of steps achieves the goal? | `src/orchestration/planner.ts` |
+| Skills | What reusable capability performs a step? | `src/skills/**` |
+| Tools | What concrete operation executes? | `src/tools/**` |
+| Orchestrator | What happens next; are we done; replan? | `src/orchestration/**` |
+| Memory | What durable information is worth remembering? | `src/memory/**` |
+| Learning | What providers/skills/plans work better? | `src/learning/**` |
+| Comparison | How do candidates compare, by evidence? | `src/comparison/**` |
+| Executions | How does a long run persist and resume? | `src/executions/**` |
+
+Conversation ≠ Memory ≠ Learning ≠ Repository ≠ Execution — separate stores, separate rules.
+
+## Three shared abstractions
+
+These keep the architecture stable as features accrete:
+
+- **RuntimeContext** (`src/runtime/context.ts`) — the single runtime-state container, assembled once per
+  run and passed by reference to mode resolution, the context compiler, planner, orchestrator, skills,
+  and tools. No other overlapping state object exists.
+- **ExecutionPolicy** (`src/runtime/policy.ts`) — the single authority for autonomy, approval, dry-run,
+  execution limits, and permissions. Every layer *reads* it; none re-derives policy.
+- **ArtifactRef** (`src/runtime/types.ts`) — large outputs are passed as references; the artifact store
+  owns the content and the context compiler decides if/when to inline it.
+
+## Request lifecycle
+
+```
+run(input) → RuntimeContext assembled → mode + policy resolved
+           → dispatch:
+               chat        → context compiled → AI.run() → router → provider
+               plan/exec/  → planner (AI.run) → validate plan → executor (tools/skills, evidence)
+               orchestrate   → observe → replan (bounded) ; persisted as an Execution, resumable
+               agent/debug → bounded loops over the same machinery
+               compare     → fan out AI.run() across pins → deterministic analysis → evidence ranking
+           → RuntimeResult (+ routing trace, memory, context, plan, execution, comparison, learning)
+```
+
+## Security model
+
+The runtime is authoritative; everything it reads through tools/files/web/MCP is **data, never
+instructions**. Filesystem access is jailed to the workspace (tested against `..`, absolute paths, and
+symlinks); shell is allowlist-first with a destructive denylist; git commit/push and network are
+individually gated; permissions are deny-by-default. Secrets are named by env var only and redacted at
+every egress including the terminal. See [security.md](security.md).
+
+## What is frozen
+
+The router core (`src/core/**` except the `filter.ts`/`scorer.ts` exclude/prefer extension points),
+`src/plugin/ai.ts`, `src/providers/**`, `src/config/{load,defaults,schema}.ts`, `src/security/**`, and
+`src/telemetry/**` are behavior-frozen: the Runtime is strictly additive over them, so the router stays
+backward compatible and independently usable.
+
+## Testing
+
+Offline and deterministic: a `MockProvider` stands in for real adapters, and injected clocks make timing
+reproducible. Every phase kept the full suite green (typecheck + tests + build), added regression coverage
+for new behavior, and ran an adversarial security/logic review before merge. CI runs all three on every
+push and PR.