@framers/agentos 0.1.155 → 0.1.157

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -6,7 +6,7 @@
6
6
 
7
7
  # AgentOS
8
8
 
9
- **Build autonomous AI agents with adaptive intelligence and emergent behaviors. Multimodal RAG, cognitive memory, streaming guardrails, voice pipeline, and 21 LLM providers. Open-source TypeScript runtime.**
9
+ **Build autonomous AI agents with adaptive intelligence, cognitive memory, and emergent behaviors. Open-source TypeScript runtime.**
10
10
 
11
11
  [![npm version](https://img.shields.io/npm/v/@framers/agentos?style=flat-square&logo=npm&color=cb3837)](https://www.npmjs.com/package/@framers/agentos)
12
12
  [![CI](https://img.shields.io/github/actions/workflow/status/framersai/agentos/ci.yml?branch=master&style=flat-square&logo=github&label=CI)](https://github.com/framersai/agentos/actions/workflows/ci.yml)
@@ -21,207 +21,127 @@
21
21
 
22
22
  ---
23
23
 
24
- ## Table of Contents
25
-
26
- - [Overview](#overview)
27
- - [Adaptive Intelligence](#adaptive-intelligence)
28
- - [Emergent Behaviors](#emergent-behaviors)
29
- - [Quick Start](#quick-start)
30
- - [1. Generate Text (Provider-First)](#1-generate-text-provider-first)
31
- - [2. Agent with Personality & Memory](#2-agent-with-personality--memory)
32
- - [3. Multimodal RAG](#3-multimodal-rag)
33
- - [4. Structured Output (Zod Validation)](#4-structured-output-zod-validation)
34
- - [5. Workflows & Graphs](#5-workflows--graphs)
35
- - [6. Multi-Agent Teams](#6-multi-agent-teams)
36
- - [7. Voice Pipeline](#7-voice-pipeline)
37
- - [8. Guardrails & Security](#8-guardrails--security)
38
- - [Default Models Per Provider](#default-models-per-provider)
39
- - [System Architecture](#system-architecture)
40
- - [Architecture Diagram](#architecture-diagram)
41
- - [Request Lifecycle](#request-lifecycle)
42
- - [Layer Breakdown](#layer-breakdown)
43
- - [Core Modules](#core-modules)
44
- - [API Layer](#api-layer)
45
- - [Cognitive Substrate (GMI)](#cognitive-substrate-gmi)
46
- - [LLM Provider Management](#llm-provider-management)
47
- - [Tool System](#tool-system)
48
- - [Extension System](#extension-system)
49
- - [Planning Engine](#planning-engine)
50
- - [Conversation Management](#conversation-management)
51
- - [RAG (Retrieval Augmented Generation)](#rag-retrieval-augmented-generation)
52
- - [Bundled Platform Knowledge](#bundled-platform-knowledge)
53
- - [Safety and Guardrails](#safety-and-guardrails)
54
- - [Human-in-the-Loop (HITL)](#human-in-the-loop-hitl)
55
- - [Channels System](#channels-system)
56
- - [Voice and Telephony](#voice-and-telephony)
57
- - [Unified Orchestration Layer](#unified-orchestration-layer)
58
- - [Multi-Agent Coordination](#multi-agent-coordination)
59
- - [Observability](#observability)
60
- - [Skills](#skills)
61
- - [Structured Output](#structured-output)
62
- - [Configuration](#configuration)
63
- - [Development (Quick Start)](#development-quick-start)
64
- - [Production](#production)
65
- - [Multiple Providers](#multiple-providers)
66
- - [Environment Variables](#environment-variables)
67
- - [API Reference](#api-reference)
68
- - [AgentOS Class](#agentos-class)
69
- - [IAgentOS Interface](#iagentos-interface)
70
- - [AgentOSInput](#agentosinput)
71
- - [AgentOSResponse Streaming](#agentosresponse-streaming)
72
- - [ITool Interface](#itool-interface)
73
- - [ExtensionDescriptor](#extensiondescriptor)
74
- - [IGuardrailService](#iguardrailservice)
75
- - [IHumanInteractionManager](#ihumaninteractionmanager)
76
- - [Usage Examples](#usage-examples)
77
- - [Orchestration Patterns](#orchestration-patterns)
78
- - [Streaming Chat](#streaming-chat)
79
- - [Adding Tools](#adding-tools)
80
- - [Multi-Agent Collaboration](#multi-agent-collaboration)
81
- - [Human-in-the-Loop Approvals](#human-in-the-loop-approvals)
82
- - [Structured Data Extraction](#structured-data-extraction)
83
- - [RAG Memory](#rag-memory)
84
- - [Custom Guardrails](#custom-guardrails)
85
- - [Channel Adapters](#channel-adapters)
86
- - [Voice Calls](#voice-calls)
87
- - [Package Exports](#package-exports)
88
- - [Internal Documentation](#internal-documentation)
89
- - [Contributing](#contributing)
90
- - [License](#license)
91
-
92
- ---
93
-
94
24
  ## Overview
95
25
 
96
- `@framers/agentos` is an open-source TypeScript AI agent runtime for building autonomous AI agents with **adaptive intelligence** and **emergent behaviors**. Agents continuously improve through meta-reflective prompt adaptation, self-evaluating response quality, personality-modulated cognition, and autonomous memory consolidation. At runtime, agents forge new tools, adapt their personality, compose workflows, and promote reliable capabilities for cross-agent reuse.
97
-
98
- The runtime provides multimodal RAG with cognitive memory (Ebbinghaus decay, 8 neuroscience-backed mechanisms including reconsolidation, retrieval-induced forgetting, involuntary recall, metacognitive FOK, temporal gist extraction, schema encoding, source confidence decay, and emotion regulation ��� all HEXACO personality-modulated), multi-agent orchestration, 37 channel adapters, 6 guardrail packs (PII redaction, ML classifiers, topicality, code safety, grounding guard, content policy rewriter) with prompt injection defense, 21 LLM providers, and 72 curated skills. Self-hostable and production-ready, it handles the full lifecycle from prompt construction through tool execution, safety evaluation, and streaming response delivery.
26
+ AgentOS is an open-source TypeScript runtime for building autonomous AI agents. Agents have HEXACO personality traits that shape how they think, cognitive memory with Ebbinghaus decay that determines what they remember, and emergent capabilities that let them forge new tools at runtime. The result is agents that adapt and improve without retraining.
99
27
 
100
- **Key facts:**
28
+ Unlike frameworks that focus purely on LLM orchestration, AgentOS treats each agent as a cognitive entity with its own personality, memory lifecycle, and behavioral adaptation loop. This makes it particularly suited for long-running agents, multi-agent teams, and applications where agent identity and consistency matter.
101
29
 
102
- | Property | Value |
103
- |----------|-------|
104
- | Package | `@framers/agentos` |
105
- | Language | TypeScript 5.4+ / Node.js 18+ |
106
- | License | Apache 2.0 |
30
+ **What makes it different:**
107
31
 
108
- **Runtime dependencies:**
32
+ | vs. | AgentOS differentiator |
33
+ |-----|------------------------|
34
+ | LangChain / LlamaIndex | Cognitive memory with 8 neuroscience-backed mechanisms, HEXACO personality modulation, runtime tool forging |
35
+ | Vercel AI SDK | Multi-agent teams with 6 strategies, full RAG pipeline with 7 vector backends, guardrail packs, voice/telephony |
36
+ | AutoGen / CrewAI | Unified orchestration layer (workflow DAGs, agent graphs, goal-driven missions), personality-driven edge routing |
109
37
 
110
- | Dependency | Purpose |
111
- |------------|---------|
112
- | `@opentelemetry/api` | Distributed tracing and metrics |
113
- | `ajv` + `ajv-formats` | JSON Schema validation for tool I/O |
114
- | `axios` | HTTP client for LLM provider APIs |
115
- | `lru-cache` | High-performance caching (dedup, embeddings, cost tracking) |
116
- | `natural` | NLP utilities (tokenization, stemming, sentiment) |
117
- | `pino` | Structured JSON logging |
118
- | `uuid` | Unique identifier generation |
119
- | `yaml` | YAML config parsing (agent configs, skill definitions) |
38
+ **Ecosystem packages:**
120
39
 
121
- **Required peer dependency:**
122
-
123
- | Dependency | Purpose |
124
- |------------|---------|
125
- | `@framers/sql-storage-adapter` | Cross-platform SQL persistence (SQLite, sql.js, IndexedDB, Postgres) required for memory, knowledge graph, and trace storage |
126
-
127
- **Optional peer dependencies:**
128
-
129
- | Peer Dependency | Purpose |
130
- |-----------------|---------|
131
- | `graphology` + `graphology-communities-louvain` | GraphRAG community detection and memory graph clustering |
132
- | `hnswlib-node` | HNSW-based approximate nearest neighbor search (falls back to brute-force without) |
133
- | `neo4j-driver` | Neo4j-backed knowledge graph and GraphRAG (falls back to SQLite without) |
40
+ | Package | Description |
41
+ |---------|-------------|
42
+ | [`@framers/agentos`](https://www.npmjs.com/package/@framers/agentos) | Core runtime -- agents, providers, memory, RAG, orchestration, guardrails |
43
+ | [`@framers/agentos-extensions`](https://www.npmjs.com/package/@framers/agentos-extensions) | Official extension registry (40+ extensions) |
44
+ | [`@framers/agentos-extensions-registry`](https://www.npmjs.com/package/@framers/agentos-extensions-registry) | Curated manifest builder for extension catalogs |
45
+ | [`@framers/agentos-skills`](https://www.npmjs.com/package/@framers/agentos-skills) | 80+ curated SKILL.md skill definitions |
46
+ | [`@framers/agentos-skills-registry`](https://www.npmjs.com/package/@framers/agentos-skills-registry) | Skills catalog SDK (query helpers + snapshot factories) |
47
+ | [`@framers/sql-storage-adapter`](https://www.npmjs.com/package/@framers/sql-storage-adapter) | Cross-platform SQL persistence (SQLite, sql.js, Postgres, IndexedDB) |
134
48
 
135
49
  ---
136
50
 
137
- ## Ecosystem
51
+ ## Install
138
52
 
139
- | Package | Description | Links |
140
- |---------|-------------|-------|
141
- | **@framers/agentos** | Core orchestration runtime | [![npm](https://img.shields.io/npm/v/@framers/agentos?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos) · [GitHub](https://github.com/framersai/agentos) |
142
- | **@framers/agentos-extensions** | Official extension registry (45+ extensions) | [![npm](https://img.shields.io/npm/v/@framers/agentos-extensions?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-extensions) · [GitHub](https://github.com/framersai/agentos-extensions) |
143
- | **@framers/agentos-extensions-registry** | Curated manifest builder | [![npm](https://img.shields.io/npm/v/@framers/agentos-extensions-registry?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-extensions-registry) · [GitHub](https://github.com/framersai/agentos-extensions-registry) |
144
- | **@framers/agentos-skills-registry** | Skills catalog SDK (query helpers + snapshot factories) | [![npm](https://img.shields.io/npm/v/@framers/agentos-skills-registry?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-skills-registry) · [GitHub](https://github.com/framersai/agentos-skills-registry) |
145
- | **@framers/agentos-skills** | Skills content (72 SKILL.md files + registry.json) | [![npm](https://img.shields.io/npm/v/@framers/agentos-skills?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-skills) · [GitHub](https://github.com/framersai/agentos-skills) |
146
- | **docs.agentos.sh** | Documentation site (Docusaurus) | [GitHub](https://github.com/framersai/agentos-live-docs) · [docs.agentos.sh](https://docs.agentos.sh) |
53
+ ```bash
54
+ npm install @framers/agentos
55
+ ```
147
56
 
148
- ### Guardrail Extensions
57
+ Set any provider's API key:
149
58
 
150
- | Package | What It Does | Links |
151
- |---------|-------------|-------|
152
- | **@framers/agentos-ext-pii-redaction** | Four-tier PII detection (regex + NLP + NER + LLM) | [![npm](https://img.shields.io/npm/v/@framers/agentos-ext-pii-redaction?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-ext-pii-redaction) · [Docs](https://docs.agentos.sh/extensions/built-in/pii-redaction) |
153
- | **@framers/agentos-ext-ml-classifiers** | Toxicity, injection, jailbreak via ONNX BERT | [![npm](https://img.shields.io/npm/v/@framers/agentos-ext-ml-classifiers?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-ext-ml-classifiers) · [Docs](https://docs.agentos.sh/extensions/built-in/ml-classifiers) |
154
- | **@framers/agentos-ext-topicality** | Embedding-based topic enforcement + drift detection | [![npm](https://img.shields.io/npm/v/@framers/agentos-ext-topicality?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-ext-topicality) · [Docs](https://docs.agentos.sh/extensions/built-in/topicality) |
155
- | **@framers/agentos-ext-code-safety** | OWASP Top 10 code scanning (25 regex rules) | [![npm](https://img.shields.io/npm/v/@framers/agentos-ext-code-safety?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-ext-code-safety) · [Docs](https://docs.agentos.sh/extensions/built-in/code-safety) |
156
- | **@framers/agentos-ext-grounding-guard** | RAG-grounded hallucination detection via NLI | [![npm](https://img.shields.io/npm/v/@framers/agentos-ext-grounding-guard?style=flat-square&logo=npm&label=)](https://www.npmjs.com/package/@framers/agentos-ext-grounding-guard) · [Docs](https://docs.agentos.sh/extensions/built-in/grounding-guard) |
157
- | **@framers/agentos-ext-content-policy-rewriter** | Opt-in content policy (8 categories, LLM rewrite/block, 4 presets) | [Docs](https://docs.agentos.sh/extensions/built-in/content-policy-rewriter) |
59
+ ```bash
60
+ export OPENAI_API_KEY=sk-... # or ANTHROPIC_API_KEY, GEMINI_API_KEY, GROQ_API_KEY, etc.
61
+ ```
158
62
 
159
63
  ---
160
64
 
161
- ## Adaptive Intelligence
65
+ ## Quick Start
162
66
 
163
- AgentOS agents continuously improve their behavior without retraining through five interconnected mechanisms:
67
+ ### 1. Generate Text
164
68
 
165
- | Mechanism | How It Works |
166
- |-----------|-------------|
167
- | **Meta-reflective prompt adaptation** | The PromptBuilder assembles a different system prompt every turn, dynamically incorporating personality traits, mood state, conversation history, retrieved memories, and available tools. No two turns use the same prompt. |
168
- | **Self-evaluating response quality** | The `self_evaluate` tool scores the agent's own output and adjusts parameters (temperature, verbosity, personality expression) in real time, creating a continuous feedback loop. |
169
- | **Personality-modulated cognition** | HEXACO traits modulate how the agent processes information. High openness increases creative associations during memory retrieval; high conscientiousness strengthens retrieval-induced forgetting of irrelevant data. |
170
- | **Autonomous memory consolidation** | The `ConsolidationLoop` prunes weak memories, strengthens frequently-accessed ones, and derives new insights from memory clusters. The agent's knowledge base improves over time without explicit training. |
171
- | **QueryRouter tiered classification** | The system adapts retrieval depth based on query complexity. Simple questions get fast keyword lookup; complex questions trigger full hybrid RAG with deep research. |
69
+ AgentOS auto-detects which provider to use from your environment variables and maps each provider to a sensible default model (see [Default Models Per Provider](#default-models-per-provider) below):
172
70
 
173
- ---
71
+ ```typescript
72
+ import { generateText } from '@framers/agentos';
174
73
 
175
- ## Emergent Behaviors
74
+ // Zero config -- auto-detects provider from env vars, uses its default model
75
+ // Priority: OPENAI_API_KEY → ANTHROPIC_API_KEY → OPENROUTER_API_KEY → GEMINI → ...
76
+ const result = await generateText({
77
+ prompt: 'Explain how TCP handshakes work in 3 bullets.',
78
+ });
79
+ console.log(result.text);
176
80
 
177
- AgentOS agents develop new capabilities at runtime rather than having them explicitly programmed:
81
+ // Pin a provider -- uses that provider's default model (e.g. anthropic → claude-sonnet-4)
82
+ const pinned = await generateText({
83
+ provider: 'anthropic',
84
+ prompt: 'Compare TCP and UDP.',
85
+ });
178
86
 
179
- | Capability | How It Works |
180
- |-----------|-------------|
181
- | **Runtime tool forging** | Agents create new tools on the fly via `forge_tool`. The `EmergentCapabilityEngine` uses sandboxed JavaScript execution and LLM-as-judge evaluation to safely create, test, and promote tools. |
182
- | **Self-improving personality** | Agents adapt their HEXACO personality traits within bounded limits via `adapt_personality`. Mutations persist with Ebbinghaus decay — strong repeated adaptations stick, while weak ones fade naturally. |
183
- | **Dynamic skill management** | Agents enable or disable skills at runtime via `manage_skills`, adapting their behavioral repertoire to the task at hand. |
184
- | **Composable workflow creation** | Agents compose registered tools into multi-step pipelines via `create_workflow`, building new capabilities from existing building blocks. |
185
- | **Tiered tool promotion** | Forged tools progress through session, agent, and shared tiers. Tools that prove reliable (5+ successful uses, >0.8 confidence score) auto-promote for cross-agent reuse. |
87
+ // Full control -- explicit provider + model override
88
+ const custom = await generateText({
89
+ provider: 'openai',
90
+ model: 'gpt-4o-mini', // override the default (gpt-4o)
91
+ prompt: 'What is the capital of France?',
92
+ });
93
+ ```
186
94
 
187
- ---
95
+ 16 providers supported. Auto-fallback on 402/429/5xx — if the primary provider fails, the next available provider is tried automatically.
188
96
 
189
- ## Quick Start
97
+ ### 2. Streaming
190
98
 
191
- ```bash
192
- npm install @framers/agentos
193
- ```
99
+ ```typescript
100
+ import { streamText } from '@framers/agentos';
194
101
 
195
- Set any provider's API key and you're ready:
102
+ const stream = streamText({
103
+ provider: 'openai',
104
+ prompt: 'Write a haiku about distributed systems.',
105
+ });
196
106
 
197
- ```bash
198
- export OPENAI_API_KEY=sk-... # or ANTHROPIC_API_KEY, GEMINI_API_KEY, GROQ_API_KEY, etc.
107
+ // Iterate token-by-token
108
+ for await (const chunk of stream.textStream) {
109
+ process.stdout.write(chunk);
110
+ }
111
+
112
+ // Or await the full result
113
+ const fullText = await stream.text;
114
+ const usage = await stream.usage;
115
+ console.log(`\n\nTokens used: ${usage.totalTokens}`);
199
116
  ```
200
117
 
201
- ### 1. Generate Text (Provider-First)
118
+ ### 3. Structured Output
202
119
 
203
- No model strings needed AgentOS picks the best default:
120
+ Extract typed data from unstructured text with Zod validation:
204
121
 
205
122
  ```typescript
206
- import { generateText, streamText } from '@framers/agentos';
123
+ import { generateObject } from '@framers/agentos';
124
+ import { z } from 'zod';
207
125
 
208
- const result = await generateText({
209
- provider: 'anthropic',
210
- prompt: 'Explain how TCP handshakes work in 3 bullets.',
126
+ const { object } = await generateObject({
127
+ provider: 'gemini',
128
+ schema: z.object({
129
+ name: z.string(),
130
+ sentiment: z.enum(['positive', 'negative', 'neutral']),
131
+ topics: z.array(z.string()),
132
+ }),
133
+ prompt: 'Analyze: "The new iPhone camera is incredible but the battery life is disappointing."',
211
134
  });
212
- console.log(result.text);
213
135
 
214
- // Streaming
215
- for await (const chunk of streamText({ provider: 'openai', prompt: 'Explain SYN-ACK.' }).textStream) {
216
- process.stdout.write(chunk);
217
- }
136
+ console.log(object);
137
+ // { name: "iPhone Review", sentiment: "neutral", topics: ["camera", "battery"] }
218
138
  ```
219
139
 
220
- 16 providers supported. Automatic fallback when one fails (402/429/5xx tries next available key).
140
+ Failed parses are automatically retried with error feedback so the model can self-correct.
221
141
 
222
- ### 2. Agent with Personality & Memory
142
+ ### 4. Agent with Personality & Memory
223
143
 
224
- Agents have HEXACO personality traits that shape their communication style, and cognitive memory with Ebbinghaus decay:
144
+ This is the key differentiator. Agents have HEXACO personality traits that shape their communication style, and cognitive memory that determines what they retain:
225
145
 
226
146
  ```typescript
227
147
  import { agent } from '@framers/agentos';
@@ -231,89 +151,77 @@ const tutor = agent({
231
151
  instructions: 'You are a patient computer science tutor.',
232
152
  personality: {
233
153
  openness: 0.9, // creative, exploratory answers
234
- conscientiousness: 0.95, // thorough, well-structured
235
- agreeableness: 0.85, // warm, encouraging tone
154
+ conscientiousness: 0.95, // thorough, well-structured
155
+ agreeableness: 0.85, // warm, encouraging tone
236
156
  },
237
157
  memory: {
238
158
  enabled: true,
239
- cognitive: true, // Ebbinghaus decay, reconsolidation, involuntary recall
159
+ cognitive: true, // Ebbinghaus decay, reconsolidation, involuntary recall
240
160
  },
161
+ tools: [{
162
+ name: 'run_code',
163
+ description: 'Execute a code snippet and return output.',
164
+ parameters: { type: 'object', properties: { code: { type: 'string' } }, required: ['code'] },
165
+ execute: async ({ code }) => ({ success: true, output: `Ran: ${code}` }),
166
+ }],
241
167
  });
242
168
 
169
+ // Sessions maintain conversation history
243
170
  const session = tutor.session('student-1');
244
171
  const reply = await session.send('Explain recursion with an analogy.');
245
172
  console.log(reply.text);
246
173
 
247
- // Memory persists across sessions — the agent remembers context
174
+ // Memory persists -- the agent remembers prior context
248
175
  const followUp = await session.send('Can you expand on that?');
249
176
  console.log(followUp.text);
250
- ```
251
177
 
252
- ### 3. Multimodal RAG
178
+ // Export agent config for portability
179
+ const config = tutor.exportJSON();
180
+ ```
253
181
 
254
- Ingest documents, images, and structured data — retrieve with HyDE (Hypothetical Document Embedding):
182
+ ### 5. Multimodal (Image, Video, Audio, OCR, Embeddings)
255
183
 
256
184
  ```typescript
257
- import { agent } from '@framers/agentos';
185
+ import {
186
+ generateImage,
187
+ performOCR,
188
+ embedText,
189
+ generateVideo,
190
+ generateMusic,
191
+ } from '@framers/agentos';
258
192
 
259
- const analyst = agent({
193
+ // Generate an image (supports OpenAI, Stability, Replicate, BFL, Fal)
194
+ const image = await generateImage({
260
195
  provider: 'openai',
261
- instructions: 'You are a financial analyst. Answer questions using provided documents.',
262
- rag: {
263
- enabled: true,
264
- mode: 'aggressive', // always retrieves before answering
265
- topK: 10,
266
- strategy: 'hyde', // generates hypothetical answers for better retrieval
267
- },
196
+ prompt: 'A cyberpunk cityscape at sunset, neon signs reflecting in rain puddles',
197
+ size: '1024x1024',
268
198
  });
199
+ console.log(image.images[0].url);
269
200
 
270
- const session = analyst.session('q4-review');
271
-
272
- // Ingest documents
273
- await session.ingest('Q4 revenue grew 23% YoY to $4.2B...');
274
- await session.ingest({ type: 'file', path: './earnings-report.pdf' });
275
-
276
- const answer = await session.send('What drove revenue growth in Q4?');
277
- console.log(answer.text); // Cites specific passages from ingested docs
278
- ```
279
-
280
- ### 4. Structured Output (Zod Validation)
281
-
282
- Extract typed data from unstructured text:
283
-
284
- ```typescript
285
- import { generateObject } from '@framers/agentos';
286
- import { z } from 'zod';
287
-
288
- const { object } = await generateObject({
289
- provider: 'gemini',
290
- schema: z.object({
291
- name: z.string(),
292
- sentiment: z.enum(['positive', 'negative', 'neutral']),
293
- topics: z.array(z.string()),
294
- }),
295
- prompt: 'Analyze: "The new iPhone camera is incredible but the battery life is disappointing."',
201
+ // OCR: extract text from images (progressive tiers: local -> cloud LLM)
202
+ const ocr = await performOCR({
203
+ image: '/tmp/receipt.png',
204
+ strategy: 'progressive', // tries fast local OCR first, falls back to cloud
296
205
  });
206
+ console.log(ocr.text);
297
207
 
298
- console.log(object);
299
- // { name: "iPhone Review", sentiment: "mixed", topics: ["camera", "battery"] }
300
- ```
301
-
302
- ### 5. Workflows & Graphs
303
-
304
- Deterministic multi-step pipelines with branching, parallel execution, and checkpointing:
305
-
306
- ```typescript
307
- import { workflow } from '@framers/agentos';
208
+ // Embeddings (OpenAI, Ollama, or any compatible provider)
209
+ const embedding = await embedText({
210
+ provider: 'openai',
211
+ input: ['Hello world', 'Goodbye world'],
212
+ dimensions: 256,
213
+ });
214
+ console.log(embedding.embeddings[0].length); // 256
308
215
 
309
- const pipeline = workflow('content-pipeline')
310
- .step('research', { provider: 'anthropic', instructions: 'Research the topic thoroughly.' })
311
- .step('draft', { provider: 'openai', instructions: 'Write a blog post from the research.' })
312
- .step('review', { provider: 'anthropic', instructions: 'Review for accuracy and tone.' })
313
- .step('publish', { tool: 'blog_publish' });
216
+ // Video generation
217
+ const video = await generateVideo({
218
+ prompt: 'A timelapse of a flower blooming',
219
+ });
314
220
 
315
- const result = await pipeline.run('Write a post about WebAssembly in 2026.');
316
- console.log(result.text);
221
+ // Music generation
222
+ const music = await generateMusic({
223
+ prompt: 'Lo-fi hip hop beat for studying',
224
+ });
317
225
  ```
318
226
 
319
227
  ### 6. Multi-Agent Teams
@@ -336,22 +244,87 @@ const team = agency({
336
244
  reviewer: {
337
245
  instructions: 'Check for accuracy and suggest improvements.',
338
246
  provider: 'gemini',
339
- dependsOn: ['writer'], // runs after writer
247
+ dependsOn: ['writer'], // runs after writer completes
340
248
  },
341
249
  },
342
- strategy: 'graph', // dependency-based DAG execution
343
- memory: { shared: true }, // agents share context
250
+ strategy: 'graph', // dependency-based DAG execution
251
+ memory: { shared: true }, // agents share context
344
252
  });
345
253
 
346
254
  const result = await team.generate('Compare TCP vs UDP for game networking.');
347
255
  console.log(result.text);
256
+
257
+ // Streaming works too
258
+ const stream = team.stream('Explain QUIC protocol benefits.');
259
+ for await (const chunk of stream.textStream) {
260
+ process.stdout.write(chunk);
261
+ }
348
262
  ```
349
263
 
350
264
  6 strategies: `sequential`, `parallel`, `debate`, `review-loop`, `hierarchical`, `graph`.
351
265
 
352
- ### 7. Voice Pipeline
266
+ Auto-detection: when any agent declares `dependsOn`, strategy defaults to `graph`.
267
+
268
+ ### 7. Orchestration (Workflows, Graphs, Missions)
269
+
270
+ Three authoring APIs that compile to one graph runtime. Choose based on how much control you need:
271
+
272
+ ```typescript
273
+ import {
274
+ workflow,
275
+ AgentGraph,
276
+ mission,
277
+ gmiNode,
278
+ toolNode,
279
+ START,
280
+ END,
281
+ } from '@framers/agentos/orchestration';
282
+ import { z } from 'zod';
283
+
284
+ // 1. workflow() -- deterministic DAG with typed I/O
285
+ const pipeline = workflow('content-pipeline')
286
+ .input(z.object({ topic: z.string() }))
287
+ .returns(z.object({ summary: z.string() }))
288
+ .step('research', { tool: 'web_search' })
289
+ .then('draft', { gmi: { instructions: 'Write a blog post from the research.' } })
290
+ .then('review', { gmi: { instructions: 'Review for accuracy and tone.' } })
291
+ .compile();
292
+
293
+ const result = await pipeline.invoke({ topic: 'WebAssembly in 2026' });
294
+
295
+ // 2. AgentGraph -- explicit nodes, edges, cycles, subgraphs
296
+ const graph = new AgentGraph({
297
+ input: z.object({ topic: z.string() }),
298
+ scratch: z.object({ draft: z.string().optional() }),
299
+ artifacts: z.object({ summary: z.string().optional() }),
300
+ })
301
+ .addNode('draft', gmiNode({ instructions: 'Draft a summary', executionMode: 'single_turn' }))
302
+ .addNode('publish', toolNode('publish_report'))
303
+ .addEdge(START, 'draft')
304
+ .addEdge('draft', 'publish')
305
+ .addEdge('publish', END)
306
+ .compile();
307
+
308
+ await graph.invoke({ topic: 'quantum computing' });
353
309
 
354
- Real-time speech-to-text and text-to-speech with streaming:
310
+ // 3. mission() -- goal-driven, planner decides the steps
311
+ const researcher = mission('deep-research')
312
+ .input(z.object({ topic: z.string() }))
313
+ .goal('Research {topic} and produce a cited summary')
314
+ .returns(z.object({ summary: z.string() }))
315
+ .planner({ strategy: 'plan_and_execute', maxSteps: 8 })
316
+ .compile();
317
+
318
+ // Preview the plan without executing
319
+ const preview = await researcher.explain({ topic: 'AI safety' });
320
+ console.log(preview.steps.map(s => s.id));
321
+ ```
322
+
323
+ All three support persistent checkpointing for fault recovery and time-travel debugging.
324
+
325
+ See [`docs/orchestration/WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md), [`docs/architecture/AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md), [`docs/orchestration/MISSION_API.md`](./docs/orchestration/MISSION_API.md).
326
+
327
+ ### 8. Voice Pipeline
355
328
 
356
329
  ```typescript
357
330
  import { agent } from '@framers/agentos';
@@ -364,15 +337,13 @@ const receptionist = agent({
364
337
  stt: { provider: 'deepgram' },
365
338
  },
366
339
  });
367
-
368
- // Voice sessions stream audio in real-time
369
- const call = receptionist.voiceSession('call-1');
370
- await call.start(); // begins listening
371
340
  ```
372
341
 
373
- ### 8. Guardrails & Security
342
+ Telephony providers: Twilio, Telnyx, Plivo. Speech: ElevenLabs, Deepgram, OpenAI Whisper, and more.
343
+
344
+ ### 9. Guardrails
374
345
 
375
- 5-tier security with PII redaction, prompt injection defense, and code scanning:
346
+ 5-tier security with 6 guardrail packs:
376
347
 
377
348
  ```typescript
378
349
  import { agent } from '@framers/agentos';
@@ -382,1802 +353,473 @@ const secureBot = agent({
382
353
  instructions: 'You are a customer support agent.',
383
354
  security: { tier: 'strict' },
384
355
  guardrails: {
385
- input: ['pii-redaction', 'ml-classifiers'], // block PII + detect injection
386
- output: ['grounding-guard', 'code-safety'], // prevent hallucination + unsafe code
356
+ input: ['pii-redaction', 'ml-classifiers'], // block PII + detect injection
357
+ output: ['grounding-guard', 'code-safety'], // prevent hallucination + unsafe code
387
358
  },
388
359
  });
389
360
  ```
390
361
 
391
- 5 tiers: `dangerous` `permissive` `balanced` `strict` `paranoid`.
362
+ 5 security tiers: `dangerous` > `permissive` > `balanced` > `strict` > `paranoid`.
363
+
364
+ 6 guardrail packs:
365
+ - **PII Redaction** -- four-tier detection (regex + NLP + NER + LLM)
366
+ - **ML Classifiers** -- toxicity, injection, jailbreak via ONNX BERT
367
+ - **Topicality** -- embedding-based topic enforcement + drift detection
368
+ - **Code Safety** -- OWASP Top 10 code scanning (25 regex rules)
369
+ - **Grounding Guard** -- RAG-source claim verification and hallucination detection
370
+ - **Content Policy Rewriter** -- 8 categories, LLM rewrite/block, 4 presets
392
371
 
393
372
  ### Default Models Per Provider
394
373
 
395
- | Provider | Default Model | Env Var |
396
- |---|---|---|
397
- | openai | gpt-4o | `OPENAI_API_KEY` |
398
- | anthropic | claude-sonnet-4 | `ANTHROPIC_API_KEY` |
399
- | gemini | gemini-2.5-flash | `GEMINI_API_KEY` |
400
- | ollama | llama3.2 | `OLLAMA_BASE_URL` |
401
- | groq | llama-3.3-70b | `GROQ_API_KEY` |
402
- | openrouter | openai/gpt-4o | `OPENROUTER_API_KEY` |
403
- | together | meta-llama/Meta-Llama-3.1-70B | `TOGETHER_API_KEY` |
404
- | mistral | mistral-large-latest | `MISTRAL_API_KEY` |
405
- | xai | grok-3-mini | `XAI_API_KEY` |
374
+ When you specify `provider` without `model`, these defaults are used:
375
+
376
+ | Provider | Default Text Model | Default Image Model | Env Var |
377
+ |---|---|---|---|
378
+ | `openai` | gpt-4o | gpt-image-1 | `OPENAI_API_KEY` |
379
+ | `anthropic` | claude-sonnet-4 | -- | `ANTHROPIC_API_KEY` |
380
+ | `gemini` | gemini-2.5-flash | -- | `GEMINI_API_KEY` |
381
+ | `ollama` | llama3.2 | stable-diffusion | `OLLAMA_BASE_URL` |
382
+ | `groq` | llama-3.3-70b-versatile | -- | `GROQ_API_KEY` |
383
+ | `openrouter` | openai/gpt-4o | -- | `OPENROUTER_API_KEY` |
384
+ | `together` | Meta-Llama-3.1-70B | -- | `TOGETHER_API_KEY` |
385
+ | `mistral` | mistral-large-latest | -- | `MISTRAL_API_KEY` |
386
+ | `xai` | grok-2 | -- | `XAI_API_KEY` |
387
+ | `stability` | -- | stable-diffusion-xl | `STABILITY_API_KEY` |
388
+ | `replicate` | -- | flux-1.1-pro | `REPLICATE_API_TOKEN` |
389
+ | `bfl` | -- | flux-pro-1.1 | `BFL_API_KEY` |
390
+ | `fal` | -- | fal-ai/flux/dev | `FAL_API_KEY` |
391
+ | `claude-code-cli` | claude-sonnet-4 | -- | `claude` binary on PATH |
392
+ | `gemini-cli` | gemini-2.5-flash | -- | `gemini` binary on PATH |
393
+ | `stable-diffusion-local` | -- | v1-5-pruned-emaonly | `STABLE_DIFFUSION_LOCAL_BASE_URL` |
406
394
 
407
- Automatic fallback: when a provider fails, AgentOS tries the next available API key.
395
+ Auto-detection priority: OpenAI > Anthropic > OpenRouter > Gemini > Groq > Together > Mistral > xAI > claude-code-cli > gemini-cli > Ollama > image-only providers.
408
396
 
409
397
  ---
410
398
 
411
- ## System Architecture
399
+ ## Core Concepts
412
400
 
413
- ### Architecture Diagram
401
+ ### GMI (Generalized Modular Intelligence)
414
402
 
415
- ```mermaid
416
- graph TB
417
- subgraph Runtime["AgentOS Runtime"]
418
- API["AgentOS API<br/><i>Service Facade</i>"] --> Orch["AgentOSOrchestrator<br/><i>Delegation Hub</i>"]
419
- API --> Stream["StreamingManager<br/><i>Async Gen Mgmt</i>"]
420
- Orch --> Prompt["PromptEngine<br/><i>Dynamic Prompts</i>"]
403
+ Each agent is backed by a GMI instance -- the "brain" that manages working memory, persona overlays, context assembly, and the cognitive loop. A single runtime can manage multiple GMI instances via `GMIManager`.
421
404
 
422
- subgraph GMI["GMI Manager Generalized Modular Intelligence"]
423
- WM["Working<br/>Memory"] ~~~ CM["Context<br/>Manager"] ~~~ PO["Persona<br/>Overlay"] ~~~ LM["Learning<br/>Module"]
424
- EM["Episodic<br/>Memory"] ~~~ SM["Semantic<br/>Memory"] ~~~ PM["Procedural<br/>Memory"]
425
- end
405
+ GMI components: working memory (7 +/- 2 slots, Baddeley's model), context manager, persona overlay switching, adaptation manager (learning rate, style drift), and multi-layer memory (episodic, semantic, procedural, prospective).
426
406
 
427
- Stream --> GMI
428
- Prompt --> GMI
407
+ See [`docs/architecture/ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md) for detailed diagrams and data flow.
429
408
 
430
- GMI --> Tools["Tool<br/>Orchestrator"]
431
- GMI --> RAG["RAG<br/>Memory"]
432
- GMI --> Plan["Planning<br/>Engine"]
433
- GMI --> WF["Workflow<br/>Engine"]
409
+ ### HEXACO Personality
434
410
 
435
- RAG --> Embed["Embedding<br/>Manager"]
436
- Plan --> ReAct["ReAct<br/>Reasoner"]
411
+ Six personality dimensions modulate agent behavior at every level -- from memory retrieval to response style:
437
412
 
438
- subgraph LLM["LLM Provider Manager"]
439
- OAI["OpenAI"] ~~~ Anth["Anthropic"] ~~~ Gem["Gemini"] ~~~ Oll["Ollama"] ~~~ OR["OpenRouter"]
440
- end
413
+ | Trait | High Value Effect | Low Value Effect |
414
+ |-------|-------------------|------------------|
415
+ | **Honesty-Humility** | Source skepticism, transparent reasoning | Confident assertions |
416
+ | **Emotionality** | Memory reconsolidation drift toward mood | Stable recall |
417
+ | **Extraversion** | Stronger feeling-of-knowing, verbose output | Reserved, concise output |
418
+ | **Agreeableness** | Emotion regulation during consolidation | Unfiltered expression |
419
+ | **Conscientiousness** | Retrieval-induced forgetting of irrelevant data | Broader recall |
420
+ | **Openness** | Involuntary recall, novelty-boosted encoding | Schema-conforming recall |
441
421
 
442
- Tools --> LLM
443
- Embed --> LLM
444
- ReAct --> LLM
422
+ Personality traits are set at agent creation and can be adapted within bounded limits at runtime via `adapt_personality`.
445
423
 
446
- Guard["Guardrail<br/>Service"] ~~~ CB["Circuit<br/>Breaker"] ~~~ CG["Cost<br/>Guard"] ~~~ SD["Stuck<br/>Detector"]
447
- Ext["Extension Manager<br/><i>12+ kinds</i>"] ~~~ Chan["Channel Router<br/><i>37 platforms</i>"] ~~~ Call["Call Manager<br/><i>Voice/Tel.</i>"]
448
- Obs["Observability<br/><i>OpenTelemetry</i>"] ~~~ HITL["HITL Manager<br/><i>Approval/Escal.</i>"] ~~~ Skill["Skill Registry<br/><i>SKILL.md</i>"]
449
- end
424
+ ### Cognitive Memory
450
425
 
451
- style Runtime fill:#0e0e18,stroke:#c9a227,stroke-width:2px,color:#f2f2fa
452
- style GMI fill:#151520,stroke:#00f5ff,stroke-width:1px,color:#f2f2fa
453
- style LLM fill:#151520,stroke:#8b5cf6,stroke-width:1px,color:#f2f2fa
454
- ```
426
+ 8 neuroscience-backed mechanisms, all HEXACO personality-modulated:
455
427
 
456
- ### Request Lifecycle
428
+ | Mechanism | Effect | Citation |
429
+ |-----------|--------|----------|
430
+ | Reconsolidation | Retrieved memories drift toward current mood | Nader, Schiller & LeDoux (2000). *Nature*, 406, 722-726 |
431
+ | Retrieval-Induced Forgetting | Retrieving one memory suppresses similar competitors | Anderson, Bjork & Bjork (1994). *JEP: Learning*, 20, 1063-1087 |
432
+ | Involuntary Recall | Random surfacing of old high-vividness memories | Berntsen (1996). *Applied Cognitive Psychology*, 10, 435-454 |
433
+ | Metacognitive FOK | Feeling-of-knowing scoring for tip-of-tongue states | Hart (1965). *JEPG*, 56, 208-216 |
434
+ | Temporal Gist Extraction | Old traces compressed to core assertions | Reyna & Brainerd (1995). *Developmental Review*, 15, 3-47 |
435
+ | Schema Encoding | Novel input boosted, schema-matching encoded efficiently | Bartlett (1932). *Remembering*. Cambridge University Press |
436
+ | Source Confidence Decay | Agent inferences decay faster than observations | Johnson, Hashtroudi & Lindsay (1993). *Psych. Bulletin*, 114, 3-28 |
437
+ | Emotion Regulation | Reappraisal + suppression during consolidation | Gross (1998). *Review of General Psychology*, 2, 271-299 |
457
438
 
458
- A single `processRequest()` call flows through these stages:
439
+ **HEXACO Personality Modulation** -- each mechanism's intensity is governed by one or more HEXACO traits:
459
440
 
460
- ```mermaid
461
- flowchart TD
462
- Input["User Input<br/><code>AgentOSInput</code>"] --> G1
441
+ | HEXACO Trait | Mechanisms Modulated | Effect |
442
+ |--------------|---------------------|--------|
443
+ | Emotionality | Reconsolidation | Higher emotionality increases mood-congruent drift rate |
444
+ | Conscientiousness | Retrieval-Induced Forgetting | Higher conscientiousness strengthens suppression of irrelevant competitors |
445
+ | Openness | Involuntary Recall, Schema Encoding | Higher openness increases involuntary recall probability and novelty boost |
446
+ | Extraversion | Metacognitive FOK | Higher extraversion strengthens feeling-of-knowing confidence signals |
447
+ | Honesty-Humility | Source Confidence Decay | Higher honesty increases skepticism of agent-inferred sources |
448
+ | Agreeableness | Emotion Regulation | Higher agreeableness strengthens reappraisal during consolidation |
463
449
 
464
- G1["1 · Input Guardrails<br/><i>evaluateInput()</i>"]
465
- G1 -->|ALLOW| Ctx
466
- G1 -.->|BLOCK / SANITIZE| Reject1["⛔ Rejected"]
450
+ **Using cognitive mechanisms with `agent()`:**
467
451
 
468
- Ctx["2 · Context Assembly<br/><i>ConversationManager + RAG</i>"] --> Prompt
469
- Prompt["3 · Prompt Construction<br/><i>PromptEngine</i>"] --> GMI
470
- GMI["4 · GMI Processing<br/><i>Working memory · Persona · Adaptation</i>"] --> LLM
452
+ ```typescript
453
+ import { agent } from '@framers/agentos';
471
454
 
472
- LLM["5 · LLM Call<br/><i>Provider Manager + Circuit Breaker</i>"] --> ToolCheck
455
+ const researcher = agent({
456
+ provider: 'anthropic',
457
+ instructions: 'You are a thorough research analyst.',
458
+ personality: {
459
+ openness: 0.9, // High openness -> more involuntary recall, stronger novelty bias
460
+ conscientiousness: 0.85, // High conscientiousness -> stronger RIF suppression
461
+ emotionality: 0.6, // Moderate -> moderate reconsolidation drift
462
+ },
463
+ memory: { enabled: true },
464
+ cognitiveMechanisms: {
465
+ // All 8 mechanisms enabled with defaults -- just pass {}
466
+ // Or tune individual mechanisms:
467
+ reconsolidation: { driftRate: 0.08 },
468
+ involuntaryRecall: { probability: 0.12 },
469
+ temporalGist: { ageThresholdDays: 30 },
470
+ },
471
+ });
472
+ ```
473
473
 
474
- ToolCheck{"Tool call?"}
475
- ToolCheck -->|Yes| ToolExec["6 · Tool Execution<br/><i>ToolOrchestrator + Permission Guard</i>"]
476
- ToolExec -->|Loop| LLM
477
- ToolCheck -->|No| G2
474
+ Pass `{}` for all defaults, or omit entirely to disable (zero overhead).
478
475
 
479
- G2["7 · Output Guardrails<br/><i>evaluateOutput()</i>"]
480
- G2 -->|ALLOW| Stream
481
- G2 -.->|BLOCK / SANITIZE| Reject2["⛔ Rejected"]
476
+ Memory is organized in a 4-tier hierarchy: `core/` (encoding, decay, working memory), `retrieval/` (composite scoring, graph, prospective), `pipeline/` (consolidation, observation, lifecycle), `io/` (ingestion, import/export).
482
477
 
483
- Stream["8 · Streaming Response<br/><i>AsyncGenerator yields chunks</i>"] --> Post
484
- Post["9 · Post-Processing<br/><i>Memory · Cost · Telemetry</i>"]
478
+ See [`docs/memory/COGNITIVE_MECHANISMS.md`](./docs/memory/COGNITIVE_MECHANISMS.md) for API reference and 30+ APA citations.
485
479
 
486
- style Input fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
487
- style G1 fill:#1c1c28,stroke:#ef4444,color:#f2f2fa
488
- style G2 fill:#1c1c28,stroke:#ef4444,color:#f2f2fa
489
- style LLM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa
490
- style ToolCheck fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
491
- style ToolExec fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
492
- style Post fill:#1c1c28,stroke:#10b981,color:#f2f2fa
493
- ```
480
+ ### Multimodal RAG
494
481
 
495
- ### Layer Breakdown
482
+ Complete retrieval-augmented generation pipeline:
496
483
 
497
- AgentOS is organized into six architectural layers:
484
+ - **7 vector backends:** InMemory, SQL (SQLite/Postgres), HNSW, Qdrant, Neo4j, Postgres+pgvector, Pinecone
485
+ - **4 retrieval strategies:** keyword, vector, hybrid (RRF), HyDE (Hypothetical Document Embedding)
486
+ - **GraphRAG:** entity/relationship extraction, Louvain community detection, local + global search
487
+ - **4-tier scaling path:** SQLite (dev) > HNSW sidecar (auto at 1K vectors) > Postgres+pgvector > Qdrant/Pinecone
488
+ - **Document ingestion:** PDF, DOCX, HTML, Markdown, URL
489
+ - **One-command migration** between any two backends via `MigrationEngine`
498
490
 
499
- | Layer | Directory | Purpose |
500
- |-------|-----------|---------|
501
- | **API** | `src/api/` | Public-facing facade, input/output types, orchestrator |
502
- | **Cognitive** | `src/cognitive_substrate/` | GMI instances, persona overlays, multi-layer memory |
503
- | **Core** | `src/core/` | 28 subdirectories: LLM, tools, safety, guardrails, planning, workflows, HITL, observability, etc. |
504
- | **Integration** | `src/channels/`, `src/voice/`, `src/extensions/` | External platform adapters, telephony, plugin system |
505
- | **Intelligence** | `src/rag/`, `src/skills/` | RAG pipeline, GraphRAG, skill loading |
506
- | **Infrastructure** | `src/config/`, `src/logging/`, `src/utils/`, `src/types/` | Configuration, structured logging, shared utilities |
491
+ See [`docs/memory/RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) and [`docs/memory/MULTIMODAL_RAG.md`](./docs/memory/MULTIMODAL_RAG.md).
507
492
 
508
- ---
493
+ ### Adaptive Intelligence & Metacognition
509
494
 
510
- ## Core Modules
495
+ Agents don't just respond — they monitor their own performance and adapt their behavior in real-time.
511
496
 
512
- ### API Layer
497
+ **MetapromptExecutor** rewrites the agent's own system prompt mid-conversation based on detected patterns:
513
498
 
514
- **Location:** `src/api/`
499
+ | Trigger | What happens | Example |
500
+ |---------|-------------|---------|
501
+ | **Frustration recovery** | Detects user frustration via SentimentTracker → simplifies language, offers alternatives | User asks same question 3x → agent acknowledges confusion and tries a different approach |
502
+ | **Confusion clarification** | Detects ambiguous query → asks targeted follow-up | Vague request → agent probes for specifics before acting |
503
+ | **Satisfaction reinforcement** | Detects positive feedback → reinforces successful patterns | User says "perfect" → agent remembers what worked |
504
+ | **Engagement boost** | Detects disengagement → adjusts tone, offers proactive suggestions | Short replies → agent becomes more concise and action-oriented |
505
+ | **Error recovery** | Detects tool failures → adjusts strategy | API call fails → agent switches to alternative approach |
506
+ | **Trait adjustment** | Bounded HEXACO mutation → personality evolves within limits | Agent becomes slightly more conscientious after repeated accuracy requests |
515
507
 
516
- The API layer is the primary entry point for all interactions with AgentOS.
508
+ Three trigger modes: `turn_interval` (periodic self-reflection), `event_based` (driven by sentiment events), and `manual` (flags in working memory).
517
509
 
518
- | File | Role |
519
- |------|------|
520
- | `AgentOS.ts` | Main service facade (1000+ lines). Implements `IAgentOS`. Initializes all subsystems and delegates to the orchestrator. |
521
- | `AgentOSOrchestrator.ts` | Delegation hub. Coordinates GMI processing, tool execution, guardrail evaluation, and streaming assembly. Streaming-first design using async generators. |
522
- | `interfaces/IAgentOS.ts` | Core contract defining `initialize()`, `processRequest()`, `handleToolResult()`, workflow management, persona listing, and conversation history retrieval. |
523
- | `types/AgentOSInput.ts` | Unified input structure: text, vision, audio, persona selection, user API keys, feedback, workflow/agency invocations, and processing options. |
524
- | `types/AgentOSResponse.ts` | Streaming output: 11 chunk types covering text deltas, tool calls, progress, errors, workflows, agency updates, and provenance events. |
510
+ **PromptProfileRouter** selects prompt strategies based on task classification — a code question gets a different prompt structure than a creative writing request.
525
511
 
526
- **Response chunk types:**
512
+ **Self-improvement** (opt-in, bounded):
527
513
 
528
514
  ```typescript
529
- enum AgentOSResponseChunkType {
530
- TEXT_DELTA // Incremental text tokens
531
- SYSTEM_PROGRESS // Internal processing updates
532
- TOOL_CALL_REQUEST // Agent requests tool execution
533
- TOOL_RESULT_EMISSION // Tool execution result
534
- UI_COMMAND // Client-side UI directives
535
- FINAL_RESPONSE // Aggregated final response
536
- ERROR // Error information
537
- METADATA_UPDATE // Session/context metadata changes
538
- WORKFLOW_UPDATE // Workflow progress notifications
539
- AGENCY_UPDATE // Multi-agent coordination events
540
- PROVENANCE_EVENT // Immutability/audit trail events
541
- }
515
+ const adaptive = agent({
516
+ provider: 'anthropic',
517
+ instructions: 'You are a research analyst.',
518
+ selfImprovement: {
519
+ enabled: true,
520
+ personality: {
521
+ maxDeltaPerSession: 0.15, // HEXACO traits can shift ±0.15 per session
522
+ decayToBaseline: true, // Drift back toward baseline during consolidation
523
+ },
524
+ skills: { enabled: true }, // Can enable/disable skills based on task
525
+ selfEvaluation: {
526
+ enabled: true, // LLM-based self-scoring after each turn
527
+ adjustParameters: true, // Auto-tune temperature/top-p based on scores
528
+ },
529
+ },
530
+ });
542
531
  ```
543
532
 
544
- ---
545
-
546
- ### Cognitive Substrate (GMI)
547
-
548
- **Location:** `src/cognitive_substrate/`
549
-
550
- The Generalized Modular Intelligence (GMI) is the core agent instance -- the "brain" of each agent. A single AgentOS runtime can manage multiple GMI instances via `GMIManager`.
551
-
552
- ```mermaid
553
- graph TB
554
- subgraph GMI["GMI Instance"]
555
- PO["PersonaOverlay Manager<br/><i>Personality switching</i>"] ~~~ CTX["Context Manager<br/><i>Dynamic prompts · User context · Session</i>"]
556
- WM["Working Memory<br/><i>Current session state · Tool context</i>"] ~~~ AM["Adaptation Manager<br/><i>Learning rate · Style drift · Tone</i>"]
557
- subgraph Mem["Multi-Layer Memory"]
558
- Ep["Episodic<br/><i>Events</i>"] ~~~ Sem["Semantic<br/><i>Facts</i>"] ~~~ Proc["Procedural<br/><i>Skills</i>"] ~~~ LT["Long-Term<br/><i>Archive</i>"]
559
- end
560
- PO --> Mem
561
- WM --> Mem
562
- end
563
- style GMI fill:#0e0e18,stroke:#c9a227,stroke-width:2px,color:#f2f2fa
564
- style Mem fill:#151520,stroke:#00f5ff,stroke-width:1px,color:#f2f2fa
565
- ```
533
+ See [`docs/architecture/ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md) for the full metacognition pipeline.
566
534
 
567
- **Key components:**
568
-
569
- - **GMI.ts** -- Core agent instance (2000+ lines). Manages the complete cognitive loop: context assembly, prompt construction, LLM invocation, tool handling, and response generation.
570
- - **GMIManager.ts** -- Lifecycle manager for multiple GMI instances. Handles creation, pooling, configuration, and teardown.
571
- - **IGMI.ts** -- Interface contract for GMI implementations.
572
- - **PersonaOverlayManager** -- Enables dynamic personality switching at runtime. Ships with 5+ built-in personas:
573
- - `Researcher` -- Analytical, citation-heavy responses
574
- - `Generalist` -- Balanced, conversational
575
- - `Atlas` -- Navigation and spatial reasoning
576
- - `Default Assistant` -- General-purpose helpful assistant
577
- - **Memory subsystem** (`memory/`) -- 5 memory types, 4-tier hierarchy, 8 cognitive mechanisms:
578
-
579
- **Memory types** (Tulving's long-term memory taxonomy):
580
- | Type | Cognitive Model | Usage |
581
- |------|----------------|-------|
582
- | `episodic` | Autobiographical events | Conversation events, interactions |
583
- | `semantic` | General knowledge/facts | Learned facts, preferences, schemas |
584
- | `procedural` | Skills and how-to | Workflows, tool usage patterns |
585
- | `prospective` | Future intentions | Goals, reminders, planned actions |
586
- | `working` | Baddeley's model | Current session state, active context (7 +/- 2 slots) |
587
-
588
- **4-tier directory hierarchy:**
589
- - `core/` -- Types, config, encoding (Yerkes-Dodson, flashbulb, HEXACO-weighted), decay (Ebbinghaus curves + spaced repetition), working memory, prompt assembly
590
- - `retrieval/` -- MemoryStore (6-signal composite scoring), memory graph (spreading activation), prospective memory, retrieval feedback
591
- - `pipeline/` -- Consolidation (merge, prune, promote, schema extraction), observation (buffer, compress, reflect), lifecycle management, infinite context window
592
- - `io/` -- Document ingestion (PDF, DOCX, HTML, Markdown, URL), import/export (JSON, Obsidian, SQLite, ChatGPT, CSV), facade API, memory tools, extensions
593
-
594
- **Cognitive mechanisms** (`memory/mechanisms/`) -- 8 optional, all HEXACO personality-modulated:
595
- | Mechanism | Citation | Effect |
596
- |-----------|----------|--------|
597
- | Reconsolidation | [Nader et al., 2000](https://doi.org/10.1038/35021052) | Retrieved memories drift toward current mood |
598
- | Retrieval-Induced Forgetting | [Anderson et al., 1994](https://doi.org/10.1037/0278-7393.20.5.1063) | Retrieving one memory suppresses similar competitors |
599
- | Involuntary Recall | [Berntsen, 2009](https://doi.org/10.1017/CBO9780511575921) | Random surfacing of old high-vividness memories |
600
- | Metacognitive FOK | [Nelson & Narens, 1990](https://doi.org/10.1016/S0079-7421(08)60053-5) | Feeling-of-knowing scoring for tip-of-tongue states |
601
- | Temporal Gist | [Reyna & Brainerd, 1995](https://doi.org/10.1016/1041-6080(95)90003-9) | Old traces compressed to core assertions |
602
- | Schema Encoding | [Bartlett, 1932](https://doi.org/10.1017/CBO9780511759185) | Novel input boosted, schema-matching encoded efficiently |
603
- | Source Confidence Decay | [Johnson et al., 1993](https://doi.org/10.1037/0033-2909.114.1.3) | Agent inferences decay faster than observations |
604
- | Emotion Regulation | [Gross, 1998](https://doi.org/10.1037/1089-2680.2.3.271) | Reappraisal + suppression during consolidation |
605
-
606
- See `docs/memory/COGNITIVE_MECHANISMS.md` for API reference and 30+ APA citations.
535
+ ### Emergent Capabilities
607
536
 
608
- ---
537
+ Agents with `emergent: true` create new tools at runtime:
609
538
 
610
- ### LLM Provider Management
539
+ - **Runtime tool forging** via `forge_tool` — sandboxed JavaScript execution + LLM-as-judge safety evaluation
540
+ - **Dynamic skill management** via `manage_skills` — enable/disable skills based on task
541
+ - **Tiered promotion:** session (in-memory) → agent (persisted after 5+ uses with >0.8 confidence) → shared (HITL-approved)
542
+ - **Self-improving personality** — bounded HEXACO trait adaptation with Ebbinghaus decay
611
543
 
612
- **Location:** `src/core/llm/`
544
+ ```typescript
545
+ const creative = agent({
546
+ provider: 'openai',
547
+ instructions: 'You solve problems creatively.',
548
+ emergent: {
549
+ enabled: true,
550
+ toolForging: true, // Can create new tools at runtime
551
+ maxForgedTools: 10, // Limit per session
552
+ promotionThreshold: 0.8, // Confidence required for permanent promotion
553
+ },
554
+ });
613
555
 
614
- The LLM layer abstracts multiple AI model providers behind a unified interface.
556
+ const session = creative.session('project');
557
+ await session.send('Parse this CSV and create a chart');
558
+ // Agent may forge a "csv_parser" tool if none exists,
559
+ // run it in a sandbox, and promote it if it works well.
615
560
 
616
- ```mermaid
617
- graph TD
618
- PM["AIModelProviderManager<br/><i>Provider registry · Routing · Fallback · Model switching</i>"]
619
- PM --> OAI["OpenAI Provider<br/><code>gpt-4o · gpt-4o-mini · o1</code>"]
620
- PM --> Anth["Anthropic Provider<br/><code>claude-sonnet-4 · claude-haiku</code>"]
621
- PM --> Gem["Gemini Provider<br/><code>gemini-2.5-flash · gemini-2.0</code>"]
622
- PM --> Oll["Ollama Provider<br/><code>llama3.2 · mistral · codellama</code>"]
623
- PM --> GQ["Groq Provider<br/><code>llama-3.3-70b · gemma2-9b</code>"]
624
- PM --> OR["OpenRouter Provider<br/><code>200+ models · auto-routing</code>"]
625
- PM --> Custom["Custom Provider<br/><i>IProvider interface</i>"]
626
- style PM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa
561
+ console.log(session.forgedTools());
562
+ // [{ name: "csv_parser", forgedAt: "...", uses: 3, confidence: 0.92 }]
627
563
  ```
628
564
 
629
- **Provider implementations:**
565
+ See [`docs/architecture/EMERGENT_CAPABILITIES.md`](./docs/architecture/EMERGENT_CAPABILITIES.md).
630
566
 
631
- | Provider | File | Models | Env Var |
632
- |----------|------|--------|---------|
633
- | OpenAI | `providers/implementations/OpenAIProvider.ts` | GPT-4o, GPT-4o-mini, o1, o3 | `OPENAI_API_KEY` |
634
- | Anthropic | `providers/implementations/AnthropicProvider.ts` | Claude Sonnet 4, Claude Haiku | `ANTHROPIC_API_KEY` |
635
- | Gemini | `providers/implementations/GeminiProvider.ts` | Gemini 2.5 Flash, Gemini 2.0 | `GEMINI_API_KEY` |
636
- | Groq | `providers/implementations/GroqProvider.ts` | Llama 3.3 70B, Gemma2 9B (fast inference) | `GROQ_API_KEY` |
637
- | Mistral | `providers/implementations/MistralProvider.ts` | Mistral Large, Mistral Small | `MISTRAL_API_KEY` |
638
- | Together | via OpenAI-compat | Meta-Llama 3.1 70B/8B | `TOGETHER_API_KEY` |
639
- | xAI | via OpenAI-compat | Grok-2, Grok-2 Mini | `XAI_API_KEY` |
640
- | Ollama | `providers/implementations/OllamaProvider.ts` | Any locally-hosted model | `OLLAMA_BASE_URL` |
641
- | OpenRouter | `providers/implementations/OpenRouterProvider.ts` | 200+ models from any provider | `OPENROUTER_API_KEY` |
567
+ ### Capability Discovery
642
568
 
643
- > **Auto-detection:** When no `provider` is specified, AgentOS scans env vars in priority order (OpenRouter > OpenAI > Anthropic > Gemini > Groq > Together > Mistral > xAI > Ollama) and uses the first configured provider. Set one env var and every helper call works automatically.
569
+ 3-tier semantic search that replaces static tool/skill dumps (~90% token reduction):
644
570
 
645
- **Additional components:**
571
+ - **Tier 0:** Category summaries (~150 tokens, always included)
572
+ - **Tier 1:** Top-5 semantic matches (~200 tokens)
573
+ - **Tier 2:** Full schemas on demand (~1500 tokens)
646
574
 
647
- - **PromptEngine** (`PromptEngine.ts`) -- Constructs prompts from system instructions, contextual elements, persona definitions, and dynamic user context. Supports template interpolation and conditional element inclusion.
648
- - **IProvider** (`providers/IProvider.ts`) -- Interface contract for adding custom LLM providers.
649
- - **Streaming adapters** -- All providers support token-level streaming via async generators with backpressure control.
575
+ The `discover_capabilities` meta-tool lets agents self-discover available tools, skills, and extensions at runtime.
650
576
 
651
- **Per-request model override (high-level helpers):**
577
+ ---
652
578
 
653
- ```typescript
654
- // Provider-first: AgentOS picks the default model
655
- await generateText({ provider: 'gemini', prompt: 'Summarize this document' });
579
+ ## Architecture
656
580
 
657
- // Override a specific model
658
- await generateText({ provider: 'anthropic', model: 'claude-haiku-4-5-20251001', prompt: '...' });
659
581
  ```
660
-
661
- **Per-request model override (full runtime):**
662
-
663
- ```typescript
664
- for await (const chunk of agent.processRequest({
665
- userId: 'user-1',
666
- sessionId: 'session-1',
667
- textInput: 'Summarize this document',
668
- options: {
669
- preferredProviderId: 'gemini',
670
- preferredModelId: 'gemini-2.5-flash',
671
- },
672
- })) { /* ... */ }
582
+ +------------------------------------------------------------------+
583
+ | AgentOS Runtime |
584
+ | |
585
+ | +-----------+ +--------------+ +-----------+ |
586
+ | | API Layer |-->| Orchestrator |-->| Streaming | |
587
+ | +-----------+ +--------------+ +-----------+ |
588
+ | | | |
589
+ | +-----v----------------v-----+ |
590
+ | | GMI (Generalized | +------------------+ |
591
+ | | Modular Intelligence) |---->| Tool Orchestrator | |
592
+ | | | +------------------+ |
593
+ | | Working Memory Persona | +------------------+ |
594
+ | | Context Mgr Adaptation |---->| RAG Pipeline | |
595
+ | | Episodic Semantic Proc. | +------------------+ |
596
+ | +-----------------------------+ +------------------+ |
597
+ | | ---->| Planning Engine | |
598
+ | +-----v-----+ +------------------+ |
599
+ | | LLM Providers (16) | |
600
+ | | OpenAI Anthropic Gemini | +----------+ +----------+ |
601
+ | | Ollama Groq OpenRouter | | Guardrails| | Channels | |
602
+ | | Together Mistral xAI ... | | (6 packs) | | (37) | |
603
+ | +----------------------------+ +----------+ +----------+ |
604
+ +------------------------------------------------------------------+
673
605
  ```
674
606
 
607
+ For the full architecture with data flow diagrams, request lifecycle, and layer breakdown, see [`docs/architecture/ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md).
608
+
675
609
  ---
676
610
 
677
- ### Tool System
611
+ ## Configuration
678
612
 
679
- **Location:** `src/core/tools/`
613
+ ### Environment Variables
680
614
 
681
- Tools are the primary mechanism for agents to interact with the outside world. Every tool implements the `ITool` interface.
615
+ ```bash
616
+ # LLM providers (set at least one)
617
+ OPENAI_API_KEY=sk-... # OpenAI (GPT-4o, GPT-4o-mini, o1, o3)
618
+ ANTHROPIC_API_KEY=sk-ant-... # Anthropic (Claude Sonnet 4, Claude Haiku)
619
+ GEMINI_API_KEY=AIza... # Google Gemini (2.5 Flash, 2.0)
620
+ OPENROUTER_API_KEY=sk-or-... # OpenRouter (200+ models, auto-routing)
621
+ GROQ_API_KEY=gsk_... # Groq (fast inference: Llama 3.3 70B)
622
+ TOGETHER_API_KEY=... # Together AI (Llama, Mixtral)
623
+ MISTRAL_API_KEY=... # Mistral (Mistral Large, Small)
624
+ XAI_API_KEY=xai-... # xAI (Grok-2)
625
+ OLLAMA_BASE_URL=http://localhost:11434 # Ollama (local models, no API key needed)
682
626
 
683
- ```mermaid
684
- graph TD
685
- TO["ToolOrchestrator<br/><i>Discovery · Selection · Validation · Execution</i>"]
686
- TO --> TE["ToolExecutor<br/><i>Validation · Invocation · Result wrap</i>"]
687
- TO --> TP["ToolPermission Manager<br/><i>RBAC · Per-user access</i>"]
688
- TO --> TG["ToolExecution Guard<br/><i>Timeout 30s · Circuit breaker</i>"]
689
- style TO fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
690
- ```
627
+ # Database (optional, defaults to in-memory)
628
+ DATABASE_URL=file:./data/agentos.db
691
629
 
692
- **ITool interface (abbreviated):**
630
+ # Observability (optional)
631
+ OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
632
+ OTEL_SERVICE_NAME=my-agent
693
633
 
694
- ```typescript
695
- interface ITool<TInput = any, TOutput = any> {
696
- readonly id: string; // Globally unique identifier
697
- readonly name: string; // LLM-facing function name
698
- readonly displayName: string; // Human-readable title
699
- readonly description: string; // Natural language description for LLM
700
- readonly inputSchema: JSONSchemaObject; // JSON Schema for input validation
701
- readonly outputSchema?: JSONSchemaObject; // JSON Schema for output validation
702
- readonly category?: string; // Grouping (e.g., "data_analysis")
703
- readonly hasSideEffects: boolean; // Whether the tool modifies external state
704
- readonly requiredCapabilities?: string[]; // Persona capabilities needed
705
-
706
- execute(
707
- args: TInput,
708
- context?: ToolExecutionContext
709
- ): Promise<ToolExecutionResult<TOutput>>;
710
- }
634
+ # Voice/Telephony (optional)
635
+ TWILIO_ACCOUNT_SID=AC...
636
+ TWILIO_AUTH_TOKEN=...
711
637
  ```
712
638
 
713
- **ToolExecutionResult:**
639
+ ### Multiple Providers
640
+
641
+ High-level helpers auto-detect from env vars -- no configuration object needed:
714
642
 
715
643
  ```typescript
716
- interface ToolExecutionResult<TOutput = any> {
717
- success: boolean;
718
- output?: TOutput;
719
- error?: string;
720
- contentType?: string; // MIME type (default: "application/json")
721
- details?: Record<string, any>;
722
- }
723
- ```
644
+ import { generateText } from '@framers/agentos';
724
645
 
725
- **ToolExecutionContext** provides the tool with calling agent identity (`gmiId`, `personaId`), user context, correlation ID for tracing, and optional session data.
646
+ // Each call picks the right credentials automatically
647
+ await generateText({ provider: 'anthropic', prompt: 'Hello from Claude' });
648
+ await generateText({ provider: 'openai', prompt: 'Hello from GPT' });
649
+ await generateText({ provider: 'gemini', prompt: 'Hello from Gemini' });
650
+ await generateText({ provider: 'ollama', prompt: 'Hello from Llama' });
726
651
 
727
- ---
652
+ // Omit provider -- auto-detects the first configured one
653
+ await generateText({ prompt: 'Hello from whichever provider is available' });
654
+ ```
728
655
 
729
- ### Extension System
656
+ For the full runtime with explicit provider configuration, see [`docs/getting-started/HIGH_LEVEL_API.md`](./docs/getting-started/HIGH_LEVEL_API.md).
730
657
 
731
- **Location:** `src/extensions/`
658
+ ### Fallback Providers
732
659
 
733
- The extension system is AgentOS's plugin architecture. Extensions are packaged as **packs** containing one or more **descriptors**, loaded via a **manifest**.
660
+ ```typescript
661
+ import { agent } from '@framers/agentos';
734
662
 
735
- ```
736
- ExtensionManifest
737
- |
738
- +-- ExtensionPackManifestEntry[]
739
- |
740
- +-- ExtensionPack (resolved via factory/module/package)
741
- |
742
- +-- ExtensionDescriptor[] (id + kind + payload)
743
- |
744
- +-- Registered in ExtensionRegistry
745
- |
746
- +-- Consumed by runtime (ToolOrchestrator,
747
- GuardrailService, WorkflowEngine, etc.)
663
+ const resilient = agent({
664
+ provider: 'anthropic',
665
+ instructions: 'You are a helpful assistant.',
666
+ fallbackProviders: [
667
+ { provider: 'openai' },
668
+ { provider: 'groq' },
669
+ ],
670
+ onFallback: (error, provider) => {
671
+ console.warn(`Falling back to ${provider}: ${error.message}`);
672
+ },
673
+ });
748
674
  ```
749
675
 
750
- **12 extension kinds:**
751
-
752
- | Kind Constant | Value | Payload Type |
753
- |---------------|-------|--------------|
754
- | `EXTENSION_KIND_TOOL` | `"tool"` | `ITool` |
755
- | `EXTENSION_KIND_GUARDRAIL` | `"guardrail"` | `IGuardrailService` |
756
- | `EXTENSION_KIND_RESPONSE_PROCESSOR` | `"response-processor"` | Response transform function |
757
- | `EXTENSION_KIND_WORKFLOW` | `"workflow"` | `WorkflowDescriptorPayload` |
758
- | `EXTENSION_KIND_WORKFLOW_EXECUTOR` | `"workflow-executor"` | Workflow step executor |
759
- | `EXTENSION_KIND_PERSONA` | `"persona"` | `IPersonaDefinition` |
760
- | `EXTENSION_KIND_PLANNING_STRATEGY` | `"planning-strategy"` | Planning algorithm |
761
- | `EXTENSION_KIND_HITL_HANDLER` | `"hitl-handler"` | Human interaction handler |
762
- | `EXTENSION_KIND_COMM_CHANNEL` | `"communication-channel"` | Agent-to-agent channel |
763
- | `EXTENSION_KIND_MEMORY_PROVIDER` | `"memory-provider"` | Memory backend |
764
- | `EXTENSION_KIND_MESSAGING_CHANNEL` | `"messaging-channel"` | External platform adapter |
765
- | `EXTENSION_KIND_PROVENANCE` | `"provenance"` | Audit/immutability handler |
766
-
767
- **ExtensionDescriptor:**
768
-
769
- ```typescript
770
- interface ExtensionDescriptor<TPayload = unknown> {
771
- id: string; // Unique within its kind
772
- kind: ExtensionKind; // One of the 12 kinds above
773
- priority?: number; // Higher loads later (overrides earlier)
774
- enableByDefault?: boolean; // Auto-enable on discovery
775
- metadata?: Record<string, unknown>; // Arbitrary metadata
776
- payload: TPayload; // The actual implementation
777
- source?: ExtensionSourceMetadata; // Provenance (package name, version)
778
- requiredSecrets?: ExtensionSecretRequirement[]; // API keys needed
779
- onActivate?: (ctx: ExtensionLifecycleContext) => Promise<void> | void;
780
- onDeactivate?: (ctx: ExtensionLifecycleContext) => Promise<void> | void;
781
- }
782
- ```
676
+ ---
783
677
 
784
- **ExtensionManifest:**
678
+ ## API Quick Reference
785
679
 
786
- ```typescript
787
- interface ExtensionManifest {
788
- packs: ExtensionPackManifestEntry[]; // Pack references
789
- overrides?: ExtensionOverrides; // Per-descriptor enable/disable/priority
790
- }
680
+ ### High-Level Functions
791
681
 
792
- // Packs can be resolved three ways:
793
- type ExtensionPackResolver =
794
- | { package: string; version?: string } // npm package
795
- | { module: string } // Local module path
796
- | { factory: () => Promise<ExtensionPack> | ExtensionPack }; // Inline factory
682
+ | Function | Description |
683
+ |----------|-------------|
684
+ | `generateText(opts)` | Single-call text generation with multi-step tool calling |
685
+ | `streamText(opts)` | Streaming text generation with async iterables |
686
+ | `generateObject(opts)` | Zod-validated structured output extraction |
687
+ | `streamObject(opts)` | Streaming structured output |
688
+ | `embedText(opts)` | Text embedding generation (single or batch) |
689
+ | `generateImage(opts)` | Image generation (OpenAI, Stability, Replicate, BFL, Fal) |
690
+ | `editImage(opts)` | Image editing/inpainting |
691
+ | `upscaleImage(opts)` | Image upscaling |
692
+ | `variateImage(opts)` | Image variations |
693
+ | `generateVideo(opts)` | Video generation |
694
+ | `analyzeVideo(opts)` | Video analysis and understanding |
695
+ | `detectScenes(opts)` | Scene detection in video |
696
+ | `generateMusic(opts)` | Music generation |
697
+ | `generateSFX(opts)` | Sound effect generation |
698
+ | `performOCR(opts)` | Text extraction from images (progressive tiers) |
699
+ | `agent(opts)` | Stateful agent with personality, memory, and sessions |
700
+ | `agency(opts)` | Multi-agent team with strategy-based coordination |
701
+ | `hitl(opts)` | Human-in-the-loop approval handler |
702
+
703
+ ### Orchestration Builders
704
+
705
+ | Builder | Import Path | Description |
706
+ |---------|-------------|-------------|
707
+ | `workflow(name)` | `@framers/agentos/orchestration` | Deterministic DAG with typed steps |
708
+ | `AgentGraph` | `@framers/agentos/orchestration` | Explicit graph with cycles, subgraphs |
709
+ | `mission(name)` | `@framers/agentos/orchestration` | Goal-driven, planner decides steps |
710
+
711
+ ### Core Types
712
+
713
+ ```typescript
714
+ import type {
715
+ AgentOSInput, // Full runtime input structure
716
+ AgentOSResponse, // Streaming response chunk
717
+ ITool, // Tool interface (id, name, inputSchema, execute)
718
+ ToolExecutionResult, // Tool result (success, output, error)
719
+ AgentOptions, // agent() configuration
720
+ AgencyOptions, // agency() configuration
721
+ GenerateTextOptions, // generateText() / streamText() options
722
+ GenerateImageOptions, // generateImage() options
723
+ GenerateObjectOptions, // generateObject() options
724
+ EmbedTextOptions, // embedText() options
725
+ ExtensionDescriptor, // Extension pack descriptor
726
+ IGuardrailService, // Guardrail interface
727
+ IChannelAdapter, // Channel adapter interface
728
+ } from '@framers/agentos';
797
729
  ```
798
730
 
799
- **Loading pipeline:**
800
-
801
- 1. `ExtensionLoader` resolves pack entries from the manifest
802
- 2. `ExtensionRegistry` registers descriptors by kind, applying priority stacking
803
- 3. `ExtensionManager` provides runtime access: `getTools()`, `getGuardrails()`, `getWorkflows()`, etc.
804
- 4. `MultiRegistryLoader` supports loading from multiple remote registries
731
+ ### Full Runtime
805
732
 
806
- **Lifecycle context and shared services:**
733
+ The `AgentOS` class provides the full-featured runtime with GMI management, extension loading, and streaming:
807
734
 
808
- - `ExtensionLifecycleContext.getSecret(secretId)` gives packs host-resolved secrets at activation time
809
- - `ExtensionLifecycleContext.services` provides a shared `ISharedServiceRegistry` for lazy singleton reuse across packs
810
- - heavyweight dependencies such as NLP pipelines, ONNX models, embedding functions, and NLI models should be loaded through the shared registry rather than per-descriptor globals
735
+ ```typescript
736
+ import { AgentOS, AgentOSResponseChunkType } from '@framers/agentos';
737
+ import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
811
738
 
812
- **Built-in guardrail packs exported by `@framers/agentos`:**
739
+ const agent = new AgentOS();
740
+ await agent.initialize(await createTestAgentOSConfig());
813
741
 
814
- | Pack | Import Path | Guardrail ID | Tool IDs | Purpose |
815
- |------|-------------|--------------|----------|---------|
816
- | PII Redaction | `@framers/agentos-ext-pii-redaction` | `pii-redaction-guardrail` | `pii_scan`, `pii_redact` | Four-tier PII detection and redaction |
817
- | ML Classifiers | `@framers/agentos-ext-ml-classifiers` | `ml-classifier-guardrail` | `classify_content` | Toxicity, prompt-injection, and jailbreak detection |
818
- | Topicality | `@framers/agentos-ext-topicality` | `topicality-guardrail` | `check_topic` | On-topic enforcement and session drift detection |
819
- | Code Safety | `@framers/agentos-ext-code-safety` | `code-safety-guardrail` | `scan_code` | Regex-based code risk scanning across fenced code and tool args |
820
- | Grounding Guard | `@framers/agentos-ext-grounding-guard` | `grounding-guardrail` | `check_grounding` | RAG-source claim verification and hallucination detection |
821
- | Content Policy Rewriter | `@framers/agentos-ext-content-policy-rewriter` | `content-policy-rewriter-guardrail` | — | Opt-in content policy: 8 categories (illegal_harmful, adult, profanity, violence, self_harm, hate_speech, illegal_activity, custom), keyword pre-filter + LLM judge/rewriter, 4 presets |
742
+ for await (const chunk of agent.processRequest({
743
+ userId: 'user-1',
744
+ sessionId: 'session-1',
745
+ textInput: 'Explain how TCP handshakes work',
746
+ })) {
747
+ if (chunk.type === AgentOSResponseChunkType.TEXT_DELTA) {
748
+ process.stdout.write(chunk.textDelta);
749
+ }
750
+ }
751
+ ```
822
752
 
823
753
  ---
824
754
 
825
- ### Planning Engine
826
-
827
- **Location:** `src/orchestration/planner/`
755
+ ## Package Exports
828
756
 
829
- The planning engine enables multi-step task decomposition and execution using ReAct (Reasoning + Acting) patterns.
757
+ AgentOS provides 112 export paths for fine-grained imports:
830
758
 
831
- ```mermaid
832
- graph TD
833
- PE["PlanningEngine<br/><i>Task decomposition · Step sequencing · Execution loop</i>"]
834
- PE --> IF["IPlanningEngine Interface"]
835
- IF --- M1["createPlan(goal, context) → Plan"]
836
- IF --- M2["executePlan(plan) → AsyncGenerator‹PlanStepResult›"]
837
- IF --- M3["revisePlan(plan, feedback) → Plan"]
838
- style PE fill:#1c1c28,stroke:#10b981,color:#f2f2fa
839
- style IF fill:#151520,stroke:#c9a227,color:#f2f2fa
840
- ```
759
+ ```typescript
760
+ // Main entry -- all public types and classes
761
+ import { AgentOS, generateText, streamText, agent, agency } from '@framers/agentos';
841
762
 
842
- Plans are composed of typed steps that the agent executes sequentially, with the ability to revise the plan based on intermediate results. The planning engine integrates with:
763
+ // Configuration
764
+ import { createAgentOSConfig, createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
843
765
 
844
- - **Tool system** for action execution
845
- - **Guardrails** for step-level safety checks
846
- - **HITL** for human approval of high-risk steps
847
- - **Memory** for persisting plan state across sessions
848
-
849
- See [`docs/orchestration/PLANNING_ENGINE.md`](./docs/orchestration/PLANNING_ENGINE.md) for the full planning system specification.
850
-
851
- ---
852
-
853
- ### Conversation Management
854
-
855
- **Location:** `src/core/conversation/`
856
-
857
- Manages session state, message history, and long-term memory persistence.
858
-
859
- - **ConversationManager** -- Creates and retrieves conversation contexts, manages rolling message windows, and coordinates memory persistence.
860
- - **ConversationContext** -- Immutable snapshot of a conversation: messages, metadata, active persona, user context.
861
- - **IRollingSummaryMemorySink** -- Interface for persisting conversation summaries that compress long conversations into retrievable memory.
862
- - **ILongTermMemoryRetriever** -- Interface for retrieving relevant past conversations during context assembly.
863
-
864
- ---
865
-
866
- ### RAG (Retrieval Augmented Generation)
867
-
868
- **Location:** `src/rag/`
869
-
870
- A complete RAG pipeline with pluggable vector stores, embedding management, and document ingestion.
871
-
872
- ```mermaid
873
- graph TD
874
- RA["RetrievalAugmentor<br/><i>Ingestion · Retrieval · Document management</i>"]
875
- RA --> EM["EmbeddingManager<br/><i>Model selection · Caching · Batch</i>"]
876
- RA --> VSM["VectorStoreManager<br/><i>Multi-provider vector storage</i>"]
877
- EM --> LLM["LLM Provider<br/><i>Embedding models</i>"]
878
- VSM --> Mem["InMemory<br/><i>dev</i>"]
879
- VSM --> SQL["SQL<br/><i>prod</i>"]
880
- VSM --> HNSW["HNSW<br/><i>local ANN</i>"]
881
- VSM --> Qd["Qdrant<br/><i>cloud</i>"]
882
- style RA fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
883
- style VSM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa
884
- ```
885
-
886
- **Vector store implementations:**
887
-
888
- | Store | Import | Use Case |
889
- |-------|--------|----------|
890
- | `InMemoryVectorStore` | `@framers/agentos/rag` | Development and testing |
891
- | `SqlVectorStore` | `@framers/agentos/rag` | Production (SQLite/Postgres via `@framers/sql-storage-adapter`) |
892
- | `HnswlibVectorStore` | `@framers/agentos/rag` | High-performance local ANN search |
893
- | `QdrantVectorStore` | `@framers/agentos/rag` | Cloud-hosted vector database |
894
- | `Neo4jVectorStore` | `@framers/agentos/rag` | Neo4j 5.x native vector indexes with shared connection pooling |
895
- | `PostgresVectorStore` | `@framers/agentos/rag` | Postgres + pgvector with HNSW indexes and RRF hybrid search |
896
- | `PineconeVectorStore` | `@framers/agentos/rag` | Pinecone cloud via fetch API with metadata filtering |
897
-
898
- > **Database Persistence: [`@framers/sql-storage-adapter`](https://github.com/framersai/sql-storage-adapter)**
899
- >
900
- > All SQL-backed storage in AgentOS — including `SqlVectorStore`, conversation persistence, and memory archival — is powered by the `@framers/sql-storage-adapter` package. It provides a unified interface across 7 database backends with automatic runtime detection:
901
- >
902
- > | Adapter | Runtime | Use Case |
903
- > |---------|---------|----------|
904
- > | `better-sqlite3` | Node.js | Production (default) |
905
- > | `pg` | Node.js | PostgreSQL for cloud deployments |
906
- > | `sql.js` | Browser/WASM | Client-side storage |
907
- > | `capacitor` | Mobile | iOS/Android via Capacitor |
908
- > | `electron` | Desktop | Electron apps with IPC bridge |
909
- > | `indexeddb` | Browser | Fallback browser storage |
910
- > | `memory` | Any | Testing and development |
911
- >
912
- > ```typescript
913
- > import { createDatabase } from '@framers/sql-storage-adapter';
914
- > const db = await createDatabase(); // auto-detects best adapter
915
- > ```
916
-
917
- **GraphRAG:**
918
-
919
- AgentOS supports both in-memory `GraphRAGEngine` and persistent `Neo4jGraphRAGEngine` (at `src/rag/graphrag/`) for knowledge-graph-enhanced retrieval:
920
-
921
- - Entity and relationship extraction from documents
922
- - Community detection via Louvain algorithm (requires `graphology` peer dependency)
923
- - Local search (entity-centric) and global search (community-summarized)
924
- - Hybrid retrieval combining vector similarity with graph traversal
925
-
926
- ```typescript
927
- import { GraphRAGEngine } from '@framers/agentos/rag/graphrag';
928
- import { Neo4jGraphRAGEngine } from '@framers/agentos/rag/graphrag';
929
- import type { GraphRAGConfig, GraphEntity, GraphRelationship } from '@framers/agentos/rag/graphrag';
930
- ```
931
-
932
- See [`docs/memory/RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) and [`docs/memory/MULTIMODAL_RAG.md`](./docs/memory/MULTIMODAL_RAG.md) for detailed configuration guides.
933
-
934
- **Memory Scaling:**
935
-
936
- AgentOS provides a 4-tier vector storage scaling path that grows with your deployment:
937
-
938
- | Tier | Backend | When to Use |
939
- |------|---------|-------------|
940
- | 0 | SQLite (default) | Development, small datasets (< 1K vectors) |
941
- | 1 | HNSW sidecar | Auto-activates at 1K vectors for local ANN search |
942
- | 2 | Postgres + pgvector | Production cloud deployments with RRF hybrid search |
943
- | 3 | Qdrant / Pinecone | High-scale managed vector databases |
944
-
945
- - **One-command migration** via `MigrationEngine` — move data between any two backends with `migrate({ from, to })`
946
- - **Docker auto-setup** — `MigrationEngine.ensureBackend('qdrant')` or `ensureBackend('postgres')` pulls and starts containers automatically
947
- - **Binary blob embedding storage** — 3-4x faster than JSON serialization for high-throughput ingest
948
- - **HNSW sidecar auto-activation** — transparently upgrades from brute-force to approximate nearest neighbors at 1K vectors
949
- - **`embed()` config option** — opt-in query-time embedding generation for dynamic content
950
-
951
- See [`docs/memory/MEMORY_SCALING.md`](./docs/memory/MEMORY_SCALING.md), [`docs/memory/POSTGRES_BACKEND.md`](./docs/memory/POSTGRES_BACKEND.md), [`docs/memory/QDRANT_BACKEND.md`](./docs/memory/QDRANT_BACKEND.md), and [`docs/memory/PINECONE_BACKEND.md`](./docs/memory/PINECONE_BACKEND.md) for backend-specific guides.
952
-
953
- ---
954
-
955
- ### Bundled Platform Knowledge
956
-
957
- **Location:** `knowledge/platform-corpus.json`
958
-
959
- AgentOS ships with **244 pre-built knowledge entries** covering the entire platform surface area. When the QueryRouter initializes, these entries are automatically loaded alongside your project-specific documentation corpus, giving every agent instant knowledge about AgentOS capabilities with zero configuration.
960
-
961
- **Coverage breakdown:**
962
-
963
- | Category | Count | What it covers |
964
- |----------|-------|---------------|
965
- | Tools | 105 | Every tool and channel adapter (Discord, Telegram, LinkedIn, Bluesky, etc.) |
966
- | Skills | 80 | All curated skills from the skills registry |
967
- | FAQ | 30 | Common questions (voice setup, supported models, streaming, OCR, etc.) |
968
- | API | 14 | Core API functions (generateText, streamText, agent, agency, etc.) |
969
- | Troubleshooting | 15 | Common errors and their fixes (missing API keys, model not found, etc.) |
970
-
971
- **How it works:**
972
-
973
- - During `router.init()`, the QueryRouter loads `knowledge/platform-corpus.json` from the package directory
974
- - Platform entries are merged into the same corpus as user docs, making them searchable via both vector and keyword retrieval
975
- - The keyword fallback index covers platform entries, so they work even without an embedding API key
976
-
977
- **Disabling platform knowledge:**
978
-
979
- ```typescript
980
- const router = new QueryRouter({
981
- knowledgeCorpus: ['./docs'],
982
- includePlatformKnowledge: false, // Skip bundled platform entries
983
- });
984
- ```
985
-
986
- When disabled, the router only loads your `knowledgeCorpus` directories. This is useful when you want a purely project-specific knowledge base or are building a non-AgentOS product.
987
-
988
- ---
989
-
990
- ### Safety and Guardrails
991
-
992
- **Location:** `src/safety/runtime/` and `src/safety/guardrails/`
993
-
994
- AgentOS provides defense-in-depth safety through two complementary systems.
995
-
996
- #### Safety Primitives (`safety/runtime/`)
997
-
998
- Five runtime safety components:
999
-
1000
- | Component | Export | Purpose |
1001
- |-----------|--------|---------|
1002
- | **CircuitBreaker** | `CircuitBreaker` | Three-state (closed/open/half-open) wrapper for LLM calls. Configurable failure threshold, reset timeout, and half-open probe count. Throws `CircuitOpenError` when tripped. |
1003
- | **ActionDeduplicator** | `ActionDeduplicator` | Hash-based recent action tracking with LRU eviction. Prevents redundant tool calls and repeated operations within a configurable time window. |
1004
- | **StuckDetector** | `StuckDetector` | Detects three patterns: repeated outputs, repeated errors, and oscillation (A-B-A-B cycles). Returns `StuckDetection` with reason and confidence. |
1005
- | **CostGuard** | `CostGuard` | Per-agent session and daily spending caps. Defaults: $1/session, $5/day. Throws `CostCapExceededError` when limits are hit. Tracks token usage across all LLM calls. |
1006
- | **ToolExecutionGuard** | `ToolExecutionGuard` | Per-tool timeout (30s default) with independent circuit breakers per tool ID. Reports `ToolHealthReport` for monitoring. Throws `ToolTimeoutError`. |
1007
-
1008
- ```typescript
1009
- import {
1010
- CircuitBreaker,
1011
- CostGuard,
1012
- StuckDetector,
1013
- ActionDeduplicator,
1014
- ToolExecutionGuard,
1015
- } from '@framers/agentos/safety/runtime';
1016
- ```
1017
-
1018
- See [`docs/safety/SAFETY_PRIMITIVES.md`](./docs/safety/SAFETY_PRIMITIVES.md) for the full safety API reference.
1019
-
1020
- #### Guardrails (`safety/guardrails/`)
1021
-
1022
- Content-level input/output filtering:
1023
-
1024
- ```typescript
1025
- interface IGuardrailService {
1026
- config?: {
1027
- evaluateStreamingChunks?: boolean;
1028
- maxStreamingEvaluations?: number;
1029
- canSanitize?: boolean;
1030
- timeoutMs?: number;
1031
- };
1032
-
1033
- evaluateInput?(payload: GuardrailInputPayload): Promise<GuardrailEvaluationResult | null>;
1034
- evaluateOutput?(payload: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null>;
1035
- }
1036
- ```
1037
-
1038
- `GuardrailOutputPayload` also carries `ragSources?: RagRetrievedChunk[]`, which enables grounding-aware output checks against retrieved context.
1039
-
1040
- Guardrails run at two points in the request lifecycle:
1041
- 1. **Pre-processing** -- `evaluateInput()` inspects user input before orchestration
1042
- 2. **Post-processing** -- `evaluateOutput()` inspects streaming chunks and/or the final response before emission
1043
-
1044
- When multiple guardrails are registered, AgentOS uses a **two-phase dispatcher**:
1045
- 1. **Phase 1 (sequential sanitizers)** -- guardrails with `config.canSanitize === true` run in registration order so each sanitizer sees the cumulative sanitized text
1046
- 2. **Phase 2 (parallel classifiers)** -- all remaining guardrails run concurrently with worst-action aggregation (`BLOCK > FLAG > ALLOW`)
1047
-
1048
- `ParallelGuardrailDispatcher` powers both input evaluation and output stream wrapping, with per-guardrail `timeoutMs` fail-open behavior for slow or degraded classifiers.
1049
-
1050
- Multiple guardrails can be composed via the extension system, and each receives full context (user ID, session ID, persona ID, conversation ID, metadata) for context-aware policy decisions.
1051
-
1052
- See [`docs/safety/GUARDRAILS_USAGE.md`](./docs/safety/GUARDRAILS_USAGE.md) for implementation patterns.
1053
-
1054
- ---
1055
-
1056
- ### Human-in-the-Loop (HITL)
1057
-
1058
- **Location:** `src/orchestration/hitl/`
1059
-
1060
- The HITL system enables agents to request human approval, clarification, and collaboration at key decision points.
1061
-
1062
- **Core interface: `IHumanInteractionManager`**
1063
-
1064
- Three interaction modes:
1065
-
1066
- | Mode | Method | Use Case |
1067
- |------|--------|----------|
1068
- | **Approval** | `requestApproval()` | Gate high-risk actions (database mutations, financial operations, external communications) |
1069
- | **Clarification** | `requestClarification()` | Ask the user for missing information before proceeding |
1070
- | **Escalation** | `escalateToHuman()` | Hand off to a human operator when the agent cannot proceed |
1071
-
1072
- **PendingAction structure:**
1073
-
1074
- ```typescript
1075
- interface PendingAction {
1076
- actionId: string;
1077
- description: string;
1078
- severity: 'low' | 'medium' | 'high' | 'critical';
1079
- category?: 'data_modification' | 'external_api' | 'financial' |
1080
- 'communication' | 'system' | 'other';
1081
- agentId: string;
1082
- context: Record<string, unknown>;
1083
- potentialConsequences?: string[];
1084
- reversible: boolean;
1085
- estimatedCost?: { amount: number; currency: string };
1086
- timeoutMs?: number;
1087
- alternatives?: AlternativeAction[];
1088
- }
1089
- ```
1090
-
1091
- HITL integrates with the planning engine so individual plan steps can require approval, and with the extension system via `EXTENSION_KIND_HITL_HANDLER` for custom approval UIs.
1092
-
1093
- See [`docs/safety/HUMAN_IN_THE_LOOP.md`](./docs/safety/HUMAN_IN_THE_LOOP.md) for the full HITL specification.
1094
-
1095
- ---
1096
-
1097
- ### Channels System
1098
-
1099
- **Location:** `src/channels/`
1100
-
1101
- Unified adapters for 40 external messaging and social platforms.
1102
-
1103
- **40 supported platforms:**
1104
-
1105
- | Priority | Platforms |
1106
- |----------|-----------|
1107
- | **P0** (Core + Social) | Telegram, WhatsApp, Discord, Slack, Webchat, Twitter/X, Instagram, Reddit, YouTube, LinkedIn, Facebook, Threads, Bluesky |
1108
- | **P1** | Signal, iMessage, Google Chat, Microsoft Teams, Pinterest, TikTok, Mastodon, Dev.to, Hashnode, Medium, WordPress |
1109
- | **P2** | Matrix, Zalo, Email, SMS, Farcaster, Lemmy, Google Business |
1110
- | **P3** | Nostr, Twitch, Line, Feishu, Mattermost, Nextcloud Talk, Tlon, IRC, Zalo Personal |
1111
-
1112
- **29 capability flags:**
1113
-
1114
- Each adapter declares its capabilities, allowing consumers to check before attempting unsupported actions:
1115
-
1116
- ```
1117
- text, rich_text, images, video, audio, voice_notes, documents,
1118
- stickers, reactions, threads, typing_indicator, read_receipts,
1119
- group_chat, channels, buttons, inline_keyboard, embeds,
1120
- mentions, editing, deletion
1121
- stories, reels, hashtags, polls, carousel,
1122
- engagement_metrics, scheduling, dm_automation, content_discovery
1123
- ```
1124
-
1125
- **IChannelAdapter** -- Unified interface for bidirectional messaging:
1126
-
1127
- - `connect()` / `disconnect()` -- Lifecycle management
1128
- - `sendMessage()` -- Outbound messages with platform-specific formatting
1129
- - `onMessage()` -- Inbound message handler registration
1130
- - `getConnectionInfo()` -- Connection health monitoring
1131
- - `capabilities` -- Declared capability set
1132
-
1133
- **ChannelRouter** -- Routes inbound messages to the appropriate agent and outbound responses to the correct platform adapter. Supports multi-platform agents (one agent, many channels).
1134
-
1135
- **Connection status:** `disconnected` -> `connecting` -> `connected` -> `reconnecting` -> `error`
1136
-
1137
- Channel adapters are registered as extensions via `EXTENSION_KIND_MESSAGING_CHANNEL`.
1138
-
1139
- See [`docs/architecture/PLATFORM_SUPPORT.md`](./docs/architecture/PLATFORM_SUPPORT.md) for platform-specific configuration.
1140
-
1141
- ---
1142
-
1143
- ### Voice and Telephony
1144
-
1145
- **Location:** `src/voice/`
1146
-
1147
- Enable agents to make and receive phone calls via telephony providers.
1148
-
1149
- **Call state machine:**
1150
-
1151
- ```
1152
- initiated --> ringing --> answered --> active --> speaking <--> listening
1153
- | | | | | |
1154
- +------------+-----------+----------+------------+------------+
1155
- | (any non-terminal state) |
1156
- v
1157
- [Terminal States]
1158
- completed | hangup-user | hangup-bot | timeout | error |
1159
- failed | no-answer | busy | voicemail
1160
- ```
1161
-
1162
- **Providers:**
1163
-
1164
- | Provider | Support |
1165
- |----------|---------|
1166
- | Twilio | Full voice + SMS |
1167
- | Telnyx | Full voice |
1168
- | Plivo | Full voice |
1169
- | Mock | Testing/development |
1170
-
1171
- **Key components:**
1172
-
1173
- - **CallManager** -- Manages call lifecycle, state transitions, and event dispatch
1174
- - **IVoiceCallProvider** -- Interface for telephony provider adapters
1175
- - **telephony-audio.ts** -- Audio stream handling and format conversion
1176
-
1177
- Voice providers are registered via `EXTENSION_KIND_TOOL` with the `voice-call-provider` category.
1178
-
1179
- ---
1180
-
1181
- ### Unified Orchestration Layer
1182
-
1183
- **Location:** `src/orchestration/`
1184
-
1185
- Three authoring APIs compile to one `CompiledExecutionGraph` IR executed by a single `GraphRuntime`. Persistent checkpointing enables time-travel debugging and fault recovery.
1186
-
1187
- Current status: the builders, IR, checkpoints, and base runtime are real. Some advanced routes are still partial in the shared runtime: discovery edges need discovery wiring, personality edges need a trait source, and `extension` / `subgraph` execution still requires a higher-level bridge runtime.
1188
-
1189
- | API | Level | Use case |
1190
- |-----|-------|----------|
1191
- | **`AgentGraph`** | Low-level | Explicit nodes, edges, cycles, subgraphs — full graph control |
1192
- | **`workflow()`** | Mid-level | Deterministic DAG chains with step/branch/parallel — Zod-typed I/O |
1193
- | **`mission()`** | High-level | Intent-driven — declare goal + constraints, PlanningEngine decides steps |
1194
-
1195
- **Differentiators vs LangGraph / Mastra:** memory-aware state, capability discovery routing, personality-driven edges, inter-step guardrails, streaming at every node transition.
1196
-
1197
- ```typescript
1198
- import { AgentGraph, toolNode, gmiNode, START, END } from '@framers/agentos/orchestration';
1199
- import { z } from 'zod';
1200
-
1201
- // Low-level: explicit graph with checkpoints
1202
- const graph = new AgentGraph({
1203
- input: z.object({ topic: z.string() }),
1204
- scratch: z.object({ draft: z.string().optional() }),
1205
- artifacts: z.object({ status: z.string().optional(), summary: z.string().optional() }),
1206
- })
1207
- .addNode('draft', gmiNode({ instructions: 'Draft a concise summary', executionMode: 'single_turn' }))
1208
- .addNode('publish', toolNode('publish_report'))
1209
- .addEdge(START, 'draft')
1210
- .addEdge('draft', 'publish')
1211
- .addEdge('publish', END)
1212
- .compile();
1213
-
1214
- const result = await graph.invoke({ topic: 'quantum computing' });
1215
-
1216
- // Mid-level: deterministic workflow
1217
- import { workflow } from '@framers/agentos/orchestration';
1218
-
1219
- const flow = workflow('onboarding')
1220
- .input(z.object({ email: z.string() }))
1221
- .returns(z.object({ userId: z.string() }))
1222
- .step('validate', { tool: 'email_validator' })
1223
- .then('create', { tool: 'user_service' })
1224
- .compile();
1225
-
1226
- // High-level: intent-driven
1227
- import { mission } from '@framers/agentos/orchestration';
1228
-
1229
- const researcher = mission('research')
1230
- .input(z.object({ topic: z.string() }))
1231
- .goal('Research {topic} and produce a cited summary')
1232
- .returns(z.object({ summary: z.string() }))
1233
- .planner({ strategy: 'plan_and_execute', maxSteps: 8 })
1234
- .compile();
1235
-
1236
- const plan = await researcher.explain({ topic: 'AI safety' }); // preview plan without executing
1237
- ```
1238
-
1239
- See [`docs/orchestration/UNIFIED_ORCHESTRATION.md`](./docs/orchestration/UNIFIED_ORCHESTRATION.md), [`docs/architecture/AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md), [`docs/orchestration/WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md), [`docs/orchestration/MISSION_API.md`](./docs/orchestration/MISSION_API.md), [`docs/orchestration/CHECKPOINTING.md`](./docs/orchestration/CHECKPOINTING.md).
1240
-
1241
- Runnable examples: [`examples/agent-graph.mjs`](./examples/agent-graph.mjs), [`examples/workflow-dsl.mjs`](./examples/workflow-dsl.mjs), [`examples/mission-api.mjs`](./examples/mission-api.mjs)
1242
-
1243
- #### Legacy WorkflowEngine
1244
-
1245
- The original `WorkflowEngine` (`src/orchestration/workflows/`) continues to work for existing consumers. The new orchestration layer is opt-in and runs alongside it.
1246
-
1247
- ```typescript
1248
- const definitions = agent.listWorkflowDefinitions();
1249
- const instance = await agent.startWorkflow('data-pipeline-v1', input);
1250
- const status = await agent.getWorkflow(instance.workflowId);
1251
- ```
1252
-
1253
- ---
1254
-
1255
- ### Multi-Agent Coordination
1256
-
1257
- **Location:** `src/agents/agency/`
1258
-
1259
- Enables teams of agents to collaborate on shared goals.
1260
-
1261
- - **AgencyRegistry** -- Register and manage agent teams with role assignments
1262
- - **AgentCommunicationBus** -- Inter-agent message passing with typed events and handoffs
1263
- - **AgencyMemoryManager** -- Shared memory space with vector search for agency-wide knowledge
1264
-
1265
- ```mermaid
1266
- graph TD
1267
- AR["AgencyRegistry"]
1268
- AR --> A1["Agent (GMI)<br/><b>Researcher</b>"]
1269
- AR --> A2["Agent (GMI)<br/><b>Writer</b>"]
1270
- AR --> A3["Agent (GMI)<br/><b>Reviewer</b>"]
1271
- AR --> A4["Agent (GMI)<br/><b>Deployer</b>"]
1272
- A1 & A2 & A3 & A4 --> Bus["Communication Bus<br/><i>Events · Handoffs · Sync</i>"]
1273
- A1 & A2 & A3 & A4 --> Mem["Agency Memory Manager<br/><i>Shared vector memory</i>"]
1274
- style AR fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
1275
- style Bus fill:#151520,stroke:#00f5ff,color:#f2f2fa
1276
- style Mem fill:#151520,stroke:#8b5cf6,color:#f2f2fa
1277
- ```
1278
-
1279
- See [`docs/architecture/AGENT_COMMUNICATION.md`](./docs/architecture/AGENT_COMMUNICATION.md) for the full multi-agent specification.
1280
-
1281
- ---
1282
-
1283
- ### Observability
1284
-
1285
- **Location:** `src/evaluation/observability/`
1286
-
1287
- OpenTelemetry-native observability for tracing, metrics, and cost tracking.
1288
-
1289
- - **ITracer** / **Tracer** -- Span creation and propagation for distributed tracing
1290
- - **otel.ts** -- `configureAgentOSObservability()` sets up the OpenTelemetry SDK with custom exporters
1291
- - **Metrics** -- Token usage, latency percentiles, tool execution counts, error rates
1292
- - **Cost tracking** -- Per-request and aggregate cost computation across providers
1293
-
1294
- ```typescript
1295
- import { configureAgentOSObservability } from '@framers/agentos';
1296
-
1297
- configureAgentOSObservability({
1298
- serviceName: 'my-agent',
1299
- traceExporter: myOTLPExporter,
1300
- metricExporter: myMetricsExporter,
1301
- });
1302
- ```
1303
-
1304
- See [`docs/observability/OBSERVABILITY.md`](./docs/observability/OBSERVABILITY.md) and [`docs/safety/COST_OPTIMIZATION.md`](./docs/safety/COST_OPTIMIZATION.md) for setup guides.
1305
-
1306
- ---
1307
-
1308
- ### Skills
1309
-
1310
- **Location:** `src/skills/`
1311
-
1312
- Skills are portable, self-describing agent capabilities defined in `SKILL.md` files.
1313
-
1314
- - **SkillRegistry** -- Discovers and registers available skills
1315
- - **SkillLoader** -- Parses SKILL.md format (YAML frontmatter + markdown body)
1316
- - **SKILL.md format** -- Declarative skill definition with name, description, required tools, and behavioral instructions
1317
-
1318
- See [`docs/extensions/SKILLS.md`](./docs/extensions/SKILLS.md) for the skill authoring guide.
1319
-
1320
- ---
1321
-
1322
- ### Structured Output
1323
-
1324
- **Location:** `src/structured/output/`
1325
-
1326
- Extract typed, validated data from unstructured text using JSON Schema.
1327
-
1328
- - **StructuredOutputManager** -- Coordinates schema-constrained generation with validation
1329
- - **JSON Schema validation** -- Input/output validation via ajv with format support
1330
- - **Parallel function calls** -- Multiple tool invocations in a single LLM turn
1331
- - **Entity extraction** -- Named entity recognition with schema constraints
1332
-
1333
- See [`docs/orchestration/STRUCTURED_OUTPUT.md`](./docs/orchestration/STRUCTURED_OUTPUT.md) for usage patterns.
1334
-
1335
- ---
1336
-
1337
- ### Emergent Capabilities
1338
-
1339
- **Location:** `src/emergent/`
1340
-
1341
- Agents with `emergent: true` create new tools at runtime — compose existing tools via a step DSL or write sandboxed JavaScript. An LLM-as-judge evaluates safety and correctness. Tools earn trust through tiered promotion: session (in-memory) → agent (persisted, auto-promoted after 5+ uses with >0.8 confidence) → shared (human-approved HITL gate).
1342
-
1343
- See [`docs/architecture/EMERGENT_CAPABILITIES.md`](./docs/architecture/EMERGENT_CAPABILITIES.md) for details.
1344
-
1345
- ---
1346
-
1347
- ## Configuration
1348
-
1349
- ### Development (Quick Start)
1350
-
1351
- `createTestAgentOSConfig()` provides sensible defaults for local development:
1352
-
1353
- ```typescript
1354
- import { AgentOS } from '@framers/agentos';
1355
- import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
1356
-
1357
- const agent = new AgentOS();
1358
- await agent.initialize(await createTestAgentOSConfig());
1359
- ```
1360
-
1361
- ### Production
1362
-
1363
- `createAgentOSConfig()` reads from environment variables:
1364
-
1365
- ```typescript
1366
- import { AgentOS } from '@framers/agentos';
1367
- import { createAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
1368
-
1369
- const agent = new AgentOS();
1370
- await agent.initialize(await createAgentOSConfig());
1371
- ```
1372
-
1373
- ### Multiple Providers
1374
-
1375
- **High-level helpers** (recommended): Just set env vars. No config object needed.
1376
-
1377
- ```bash
1378
- # Set as many provider keys as you like -- AgentOS uses them on demand
1379
- export ANTHROPIC_API_KEY=sk-ant-...
1380
- export OPENAI_API_KEY=sk-...
1381
- export GEMINI_API_KEY=AIza...
1382
- export OLLAMA_BASE_URL=http://localhost:11434
1383
- ```
1384
-
1385
- ```typescript
1386
- import { generateText } from '@framers/agentos';
1387
-
1388
- // Each call picks the right credentials from env vars automatically
1389
- await generateText({ provider: 'anthropic', prompt: 'Hello from Claude' });
1390
- await generateText({ provider: 'openai', prompt: 'Hello from GPT' });
1391
- await generateText({ provider: 'gemini', prompt: 'Hello from Gemini' });
1392
- await generateText({ provider: 'ollama', prompt: 'Hello from Llama' });
1393
-
1394
- // Or omit provider entirely -- auto-detects the first configured one
1395
- await generateText({ prompt: 'Hello from whichever provider is available' });
1396
- ```
1397
-
1398
- **Full runtime** — configure multiple providers with explicit fallback:
1399
-
1400
- ```typescript
1401
- const agent = new AgentOS();
1402
- const config = await createTestAgentOSConfig();
1403
-
1404
- await agent.initialize({
1405
- ...config,
1406
- modelProviderManagerConfig: {
1407
- providers: [
1408
- { providerId: 'anthropic', enabled: true, isDefault: true,
1409
- config: { apiKey: process.env.ANTHROPIC_API_KEY } },
1410
- { providerId: 'openai', enabled: true,
1411
- config: { apiKey: process.env.OPENAI_API_KEY } },
1412
- { providerId: 'gemini', enabled: true,
1413
- config: { apiKey: process.env.GEMINI_API_KEY } },
1414
- { providerId: 'ollama', enabled: true,
1415
- config: { baseUrl: process.env.OLLAMA_BASE_URL || 'http://localhost:11434' } },
1416
- ],
1417
- },
1418
- gmiManagerConfig: {
1419
- ...config.gmiManagerConfig,
1420
- defaultGMIBaseConfigDefaults: {
1421
- ...(config.gmiManagerConfig.defaultGMIBaseConfigDefaults ?? {}),
1422
- defaultLlmProviderId: 'anthropic',
1423
- defaultLlmModelId: 'claude-sonnet-4-20250514',
1424
- },
1425
- },
1426
- });
1427
-
1428
- // Override per request:
1429
- for await (const chunk of agent.processRequest({
1430
- userId: 'user-1',
1431
- sessionId: 'session-1',
1432
- textInput: 'Hello',
1433
- options: {
1434
- preferredProviderId: 'gemini',
1435
- preferredModelId: 'gemini-2.5-flash',
1436
- },
1437
- })) { /* ... */ }
1438
- ```
1439
-
1440
- ### Environment Variables
1441
-
1442
- ```bash
1443
- # LLM providers (set at least one)
1444
- OPENAI_API_KEY=sk-... # OpenAI (GPT-4o, GPT-4o-mini, o1, o3)
1445
- ANTHROPIC_API_KEY=sk-ant-... # Anthropic (Claude Sonnet 4, Claude Haiku)
1446
- GEMINI_API_KEY=AIza... # Google Gemini (2.5 Flash, 2.0)
1447
- OPENROUTER_API_KEY=sk-or-... # OpenRouter (200+ models, auto-routing)
1448
- GROQ_API_KEY=gsk_... # Groq (fast inference: Llama 3.3 70B)
1449
- TOGETHER_API_KEY=... # Together AI (Llama, Mixtral)
1450
- MISTRAL_API_KEY=... # Mistral (Mistral Large, Small)
1451
- XAI_API_KEY=xai-... # xAI (Grok-2)
1452
- OLLAMA_BASE_URL=http://localhost:11434 # Ollama (local models, no API key needed)
1453
-
1454
- # Database
1455
- DATABASE_URL=file:./data/agentos.db
1456
-
1457
- # Observability (optional)
1458
- OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
1459
- OTEL_SERVICE_NAME=my-agent
1460
-
1461
- # Voice/Telephony (optional)
1462
- TWILIO_ACCOUNT_SID=AC...
1463
- TWILIO_AUTH_TOKEN=...
1464
- TELNYX_API_KEY=KEY...
1465
- ```
1466
-
1467
- ---
1468
-
1469
- ## API Reference
1470
-
1471
- ### AgentOS Class
1472
-
1473
- The main service facade. Implements `IAgentOS`.
1474
-
1475
- ```typescript
1476
- class AgentOS implements IAgentOS {
1477
- // Lifecycle
1478
- initialize(config: AgentOSConfig): Promise<void>;
1479
- shutdown(): Promise<void>;
1480
-
1481
- // Core interaction (streaming-first)
1482
- processRequest(input: AgentOSInput): AsyncGenerator<AgentOSResponse>;
1483
- handleToolResult(streamId, toolCallId, toolName, toolOutput, isSuccess, errorMessage?):
1484
- AsyncGenerator<AgentOSResponse>;
1485
-
1486
- // Personas
1487
- listPersonas(): IPersonaDefinition[];
1488
- setActivePersona(personaId: string): Promise<void>;
1489
-
1490
- // Conversation
1491
- getConversationHistory(sessionId: string): Promise<ConversationContext>;
1492
-
1493
- // Workflows
1494
- listWorkflowDefinitions(): WorkflowDefinition[];
1495
- startWorkflow(definitionId, input, options?): Promise<WorkflowInstance>;
1496
- getWorkflow(workflowId): Promise<WorkflowInstance | null>;
1497
- updateWorkflowTask(workflowId, taskId, update): Promise<void>;
1498
- queryWorkflows(options?): Promise<WorkflowInstance[]>;
1499
-
1500
- // Feedback
1501
- submitFeedback(feedback: UserFeedbackPayload): Promise<void>;
1502
-
1503
- // Exposed managers (for advanced usage)
1504
- readonly llmProviderManager: AIModelProviderManager;
1505
- readonly extensionManager: ExtensionManager;
1506
- readonly conversationManager: ConversationManager;
1507
- }
1508
- ```
1509
-
1510
- ### IAgentOS Interface
1511
-
1512
- The core contract. See `src/api/interfaces/IAgentOS.ts` for the full interface definition.
1513
-
1514
- ### AgentOSInput
1515
-
1516
- ```typescript
1517
- interface AgentOSInput {
1518
- userId: string;
1519
- organizationId?: string; // Multi-tenant routing
1520
- sessionId: string;
1521
- textInput: string | null;
1522
- visionInput?: VisionInputData; // Image/video input
1523
- audioInput?: AudioInputData; // Audio input
1524
- preferredPersonaId?: string; // Request specific persona
1525
- userApiKeys?: Record<string, string>; // User-provided API keys
1526
- feedback?: UserFeedbackPayload; // Inline feedback
1527
- workflowInvocation?: WorkflowInvocationRequest;
1528
- agencyInvocation?: AgencyInvocationRequest;
1529
- memoryControl?: AgentOSMemoryControl;
1530
- options?: ProcessingOptions; // Model override, temperature, etc.
1531
- }
1532
- ```
1533
-
1534
- ### AgentOSResponse Streaming
1535
-
1536
- All core methods return `AsyncGenerator<AgentOSResponse>`. Each yielded chunk has a `type` discriminant:
1537
-
1538
- ```typescript
1539
- // Handle all chunk types:
1540
- for await (const chunk of agent.processRequest(input)) {
1541
- switch (chunk.type) {
1542
- case AgentOSResponseChunkType.TEXT_DELTA:
1543
- process.stdout.write(chunk.textDelta);
1544
- break;
1545
- case AgentOSResponseChunkType.TOOL_CALL_REQUEST:
1546
- console.log('Tools requested:', chunk.toolCalls);
1547
- break;
1548
- case AgentOSResponseChunkType.TOOL_RESULT_EMISSION:
1549
- console.log('Tool result:', chunk.toolName, chunk.toolResult);
1550
- break;
1551
- case AgentOSResponseChunkType.SYSTEM_PROGRESS:
1552
- console.log('Progress:', chunk.message, chunk.progressPercentage);
1553
- break;
1554
- case AgentOSResponseChunkType.WORKFLOW_UPDATE:
1555
- console.log('Workflow:', chunk.workflowProgress);
1556
- break;
1557
- case AgentOSResponseChunkType.AGENCY_UPDATE:
1558
- console.log('Agency event:', chunk.agencyEvent);
1559
- break;
1560
- case AgentOSResponseChunkType.ERROR:
1561
- console.error('Error:', chunk.error);
1562
- break;
1563
- case AgentOSResponseChunkType.FINAL_RESPONSE:
1564
- console.log('Complete:', chunk.finalText);
1565
- break;
1566
- }
1567
- }
1568
- ```
1569
-
1570
- ### ITool Interface
1571
-
1572
- See the [Tool System](#tool-system) section above for the full interface. Tools are registered via extension packs:
1573
-
1574
- ```typescript
1575
- const descriptor: ExtensionDescriptor<ITool> = {
1576
- id: 'my-tool',
1577
- kind: EXTENSION_KIND_TOOL,
1578
- payload: myToolImplementation,
1579
- };
1580
- ```
1581
-
1582
- ### ExtensionDescriptor
1583
-
1584
- See the [Extension System](#extension-system) section above for the full type definition and all 12 extension kinds.
1585
-
1586
- ### IGuardrailService
1587
-
1588
- ```typescript
1589
- interface IGuardrailService {
1590
- evaluateInput?(
1591
- input: AgentOSInput,
1592
- context: GuardrailContext,
1593
- ): Promise<GuardrailEvaluationResult>;
1594
-
1595
- evaluateOutput?(
1596
- output: string,
1597
- context: GuardrailContext,
1598
- ): Promise<GuardrailEvaluationResult>;
1599
- }
1600
-
1601
- interface GuardrailEvaluationResult {
1602
- action: GuardrailAction; // ALLOW | FLAG | SANITIZE | BLOCK
1603
- reason?: string; // Human-readable explanation
1604
- reasonCode?: string; // Machine-readable code
1605
- modifiedText?: string; // Required when action is SANITIZE
1606
- metadata?: Record<string, any>; // Additional context for logging
1607
- }
1608
- ```
1609
-
1610
- ### IHumanInteractionManager
1611
-
1612
- ```typescript
1613
- interface IHumanInteractionManager {
1614
- requestApproval(action: PendingAction): Promise<ApprovalDecision>;
1615
- requestClarification(request: ClarificationRequest): Promise<ClarificationResponse>;
1616
- escalateToHuman(context: EscalationContext): Promise<EscalationResult>;
1617
- getPendingActions(): Promise<PendingAction[]>;
1618
- }
1619
- ```
1620
-
1621
- ---
1622
-
1623
- ## Usage Examples
1624
-
1625
- ### Orchestration Patterns
1626
-
1627
- Use the new orchestration layer based on how much control you need:
1628
-
1629
- - `workflow()` for deterministic sequential/parallel DAGs
1630
- - `AgentGraph` for explicit cycles, retries, and custom routing
1631
- - `mission()` when you know the goal but want the planner to decide the steps
1632
-
1633
- ```typescript
1634
- import {
1635
- AgentGraph,
1636
- END,
1637
- START,
1638
- gmiNode,
1639
- mission,
1640
- toolNode,
1641
- workflow,
1642
- } from '@framers/agentos/orchestration';
1643
- import { z } from 'zod';
1644
-
1645
- // 1. Deterministic DAG: sequential steps with a parallel fan-out/join
1646
- const onboarding = workflow('user-onboarding')
1647
- .input(z.object({ email: z.string().email() }))
1648
- .returns(z.object({ userId: z.string() }))
1649
- .step('validate', { tool: 'email_validator' })
1650
- .then('create-account', { tool: 'user_service' })
1651
- .parallel(
1652
- [
1653
- { tool: 'send_welcome_email' },
1654
- { tool: 'provision_default_workspace' },
1655
- ],
1656
- {
1657
- strategy: 'all',
1658
- merge: { 'scratch.completedTasks': 'concat' },
1659
- },
1660
- )
1661
- .compile();
1662
-
1663
- // 2. Explicit graph: fixed publish pipeline with explicit nodes and edges
1664
- const reviewGraph = new AgentGraph({
1665
- input: z.object({ topic: z.string() }),
1666
- scratch: z.object({ draft: z.string().optional() }),
1667
- artifacts: z.object({ status: z.string().optional(), summary: z.string().optional() }),
1668
- })
1669
- .addNode('draft', gmiNode({ instructions: 'Draft the release note.', executionMode: 'single_turn' }))
1670
- .addNode('publish', toolNode('publish_report'))
1671
- .addEdge(START, 'draft')
1672
- .addEdge('draft', 'publish')
1673
- .addEdge('publish', END)
1674
- .compile();
1675
-
1676
- // 3. Goal-first mission: preview the generated plan before running it
1677
- const researcher = mission('deep-research')
1678
- .input(z.object({ topic: z.string() }))
1679
- .goal('Research {{topic}} thoroughly and produce a cited summary')
1680
- .returns(z.object({ summary: z.string() }))
1681
- .planner({ strategy: 'plan_and_execute', maxSteps: 8 })
1682
- .compile();
1683
-
1684
- const preview = await researcher.explain({ topic: 'AI safety' });
1685
- console.log(preview.steps.map((step) => step.id));
1686
- ```
1687
-
1688
- For deeper examples, see [`docs/architecture/AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md), [`docs/orchestration/WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md), [`docs/orchestration/MISSION_API.md`](./docs/orchestration/MISSION_API.md), the runnable examples [`examples/agent-graph.mjs`](./examples/agent-graph.mjs), [`examples/workflow-dsl.mjs`](./examples/workflow-dsl.mjs), [`examples/mission-api.mjs`](./examples/mission-api.mjs), and the legacy dependency-ordered example [`examples/multi-agent-workflow.mjs`](./examples/multi-agent-workflow.mjs).
1689
-
1690
- ### Streaming Chat
1691
-
1692
- ```typescript
1693
- import { AgentOS, AgentOSResponseChunkType } from '@framers/agentos';
1694
- import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
1695
-
1696
- const agent = new AgentOS();
1697
- await agent.initialize(await createTestAgentOSConfig());
1698
-
1699
- for await (const chunk of agent.processRequest({
1700
- userId: 'user-1',
1701
- sessionId: 'session-1',
1702
- textInput: 'Explain how TCP handshakes work',
1703
- })) {
1704
- if (chunk.type === AgentOSResponseChunkType.TEXT_DELTA) {
1705
- process.stdout.write(chunk.textDelta);
1706
- }
1707
- }
1708
- ```
1709
-
1710
- ### Adding Tools
1711
-
1712
- Tools are registered via extension packs and called automatically by the model:
1713
-
1714
- ```typescript
1715
- import {
1716
- AgentOS,
1717
- AgentOSResponseChunkType,
1718
- EXTENSION_KIND_TOOL,
1719
- type ExtensionManifest,
1720
- type ExtensionPack,
1721
- type ITool,
1722
- } from '@framers/agentos';
1723
- import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
1724
-
1725
- const weatherTool: ITool = {
1726
- id: 'get-weather',
1727
- name: 'get_weather',
1728
- displayName: 'Get Weather',
1729
- description: 'Returns current weather for a city.',
1730
- category: 'utility',
1731
- hasSideEffects: false,
1732
- inputSchema: {
1733
- type: 'object',
1734
- properties: { city: { type: 'string', description: 'City name' } },
1735
- required: ['city'],
1736
- },
1737
- execute: async (args) => ({
1738
- success: true,
1739
- output: { text: `Weather in ${args.city}: 22 C, partly cloudy` },
1740
- }),
1741
- };
1742
-
1743
- const manifest: ExtensionManifest = {
1744
- packs: [{
1745
- factory: async () => ({
1746
- name: 'my-tools',
1747
- descriptors: [{ id: weatherTool.id, kind: EXTENSION_KIND_TOOL, payload: weatherTool }],
1748
- } satisfies ExtensionPack),
1749
- }],
1750
- };
1751
-
1752
- const agent = new AgentOS();
1753
- const config = await createTestAgentOSConfig();
1754
- await agent.initialize({ ...config, extensionManifest: manifest });
1755
-
1756
- for await (const chunk of agent.processRequest({
1757
- userId: 'user-1',
1758
- sessionId: 'session-1',
1759
- textInput: 'What is the weather in Tokyo?',
1760
- })) {
1761
- switch (chunk.type) {
1762
- case AgentOSResponseChunkType.TEXT_DELTA:
1763
- process.stdout.write(chunk.textDelta);
1764
- break;
1765
- case AgentOSResponseChunkType.TOOL_CALL_REQUEST:
1766
- console.log('Tool calls:', chunk.toolCalls);
1767
- break;
1768
- case AgentOSResponseChunkType.TOOL_RESULT_EMISSION:
1769
- console.log('Tool result:', chunk.toolResult);
1770
- break;
1771
- }
1772
- }
1773
- ```
1774
-
1775
- ### Multi-Agent Collaboration
1776
-
1777
- ```typescript
1778
- import { AgentCommunicationBus } from '@framers/agentos';
1779
-
1780
- const bus = new AgentCommunicationBus({
1781
- routingConfig: { enableRoleRouting: true, enableLoadBalancing: true },
1782
- });
1783
-
1784
- bus.registerAgent('coordinator-gmi', 'agency-docs', 'coordinator');
1785
- bus.registerAgent('researcher-gmi', 'agency-docs', 'researcher');
1786
- bus.registerAgent('writer-gmi', 'agency-docs', 'writer');
1787
-
1788
- bus.subscribe(
1789
- 'researcher-gmi',
1790
- async (message) => {
1791
- if (message.type !== 'question') return;
1792
- await bus.sendToAgent(message.fromAgentId, {
1793
- type: 'answer',
1794
- fromAgentId: 'researcher-gmi',
1795
- content: { findings: ['auth edge cases', 'missing audit trail', 'weak retry policy'] },
1796
- inReplyTo: message.messageId,
1797
- priority: 'normal',
1798
- });
1799
- },
1800
- { messageTypes: ['question'] },
1801
- );
1802
-
1803
- bus.subscribe(
1804
- 'writer-gmi',
1805
- async (message) => {
1806
- if (message.type !== 'task_delegation') return;
1807
- await bus.sendToAgent(message.fromAgentId, {
1808
- type: 'answer',
1809
- fromAgentId: 'writer-gmi',
1810
- content: { accepted: true },
1811
- inReplyTo: message.messageId,
1812
- priority: 'normal',
1813
- });
1814
- },
1815
- { messageTypes: ['task_delegation'] },
1816
- );
1817
-
1818
- await bus.sendToRole('agency-docs', 'researcher', {
1819
- type: 'task_delegation',
1820
- fromAgentId: 'coordinator-gmi',
1821
- content: { topic: 'auth module', instructions: 'Find the risky edge cases.' },
1822
- priority: 'high',
1823
- });
1824
-
1825
- const review = await bus.requestResponse('researcher-gmi', {
1826
- type: 'question',
1827
- fromAgentId: 'coordinator-gmi',
1828
- content: 'What are the top three findings?',
1829
- priority: 'high',
1830
- timeoutMs: 30_000,
1831
- });
1832
-
1833
- const handoff = await bus.handoff('researcher-gmi', 'writer-gmi', {
1834
- taskId: 'auth-audit',
1835
- taskDescription: 'Turn the findings into a release note draft',
1836
- progress: 0.8,
1837
- completedWork: [review.content],
1838
- remainingWork: ['Write polished summary'],
1839
- context: { audience: 'engineering' },
1840
- reason: 'completion',
1841
- instructions: 'Summarize the findings in concise release-note style.',
1842
- });
1843
-
1844
- console.log(handoff.accepted);
1845
- ```
1846
-
1847
- Runnable example: [`examples/agent-communication-bus.mjs`](./examples/agent-communication-bus.mjs)
1848
-
1849
- ### Human-in-the-Loop Approvals
1850
-
1851
- ```typescript
1852
- import { HumanInteractionManager } from '@framers/agentos';
1853
-
1854
- const hitl = new HumanInteractionManager({ defaultTimeoutMs: 300_000 });
1855
-
1856
- const decision = await hitl.requestApproval({
1857
- actionId: 'archive-inactive',
1858
- description: 'Archive 50K inactive accounts older than 2 years',
1859
- severity: 'high',
1860
- category: 'data_modification',
1861
- agentId: 'data-cleanup-agent',
1862
- context: { affectedRows: 50_000, table: 'users' },
1863
- reversible: true,
1864
- potentialConsequences: ['Users will lose access to archived data'],
1865
- alternatives: [
1866
- { alternativeId: 'soft_delete', description: 'Mark as inactive instead' },
1867
- { alternativeId: 'export_first', description: 'Export to CSV before archiving' },
1868
- ],
1869
- });
1870
-
1871
- if (decision.approved) {
1872
- await executeArchive();
1873
- } else if (decision.selectedAlternative) {
1874
- await executeAlternative(decision.selectedAlternative);
1875
- }
1876
- ```
1877
-
1878
- ### Structured Data Extraction
1879
-
1880
- ```typescript
1881
- import { AgentOS, StructuredOutputManager } from '@framers/agentos';
1882
- import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
1883
-
1884
- const agent = new AgentOS();
1885
- await agent.initialize(await createTestAgentOSConfig());
1886
-
1887
- const structured = new StructuredOutputManager({
1888
- llmProviderManager: agent.llmProviderManager,
1889
- });
1890
-
1891
- const contact = await structured.generate({
1892
- prompt: 'Extract: "Meeting with Sarah Chen (sarah@startup.io) on Jan 15 re: Series A"',
1893
- schema: {
1894
- type: 'object',
1895
- properties: {
1896
- name: { type: 'string' },
1897
- email: { type: 'string', format: 'email' },
1898
- date: { type: 'string' },
1899
- topic: { type: 'string' },
1900
- },
1901
- required: ['name', 'email'],
1902
- },
1903
- schemaName: 'ContactInfo',
1904
- });
1905
- // Result: { name: 'Sarah Chen', email: 'sarah@startup.io', date: 'Jan 15', topic: 'Series A' }
1906
- ```
1907
-
1908
- ### RAG Memory
1909
-
1910
- ```typescript
1911
- import { AgentOS } from '@framers/agentos';
1912
- import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
1913
-
1914
- const agent = new AgentOS();
1915
- const config = await createTestAgentOSConfig();
1916
-
1917
- await agent.initialize({
1918
- ...config,
1919
- ragConfig: {
1920
- embeddingManagerConfig: {
1921
- embeddingModels: [
1922
- { modelId: 'text-embedding-3-small', providerId: 'openai',
1923
- dimension: 1536, isDefault: true },
1924
- ],
1925
- },
1926
- vectorStoreManagerConfig: {
1927
- managerId: 'rag-vsm',
1928
- providers: [
1929
- { id: 'sql-store', type: 'sql', storage: { filePath: './data/vectors.db' } },
1930
- ],
1931
- defaultProviderId: 'sql-store',
1932
- defaultEmbeddingDimension: 1536,
1933
- },
1934
- dataSourceConfigs: [{
1935
- dataSourceId: 'conversations',
1936
- displayName: 'Conversation Memory',
1937
- vectorStoreProviderId: 'sql-store',
1938
- actualNameInProvider: 'conversations',
1939
- embeddingDimension: 1536,
1940
- }],
1941
- retrievalAugmentorConfig: {
1942
- defaultDataSourceId: 'conversations',
1943
- categoryBehaviors: [],
1944
- },
1945
- },
1946
- });
1947
-
1948
- // Agent now retrieves relevant context from vector memory before responding
1949
- ```
1950
-
1951
- ### Custom Guardrails
1952
-
1953
- ```typescript
1954
- import {
1955
- AgentOS,
1956
- type IGuardrailService,
1957
- GuardrailAction,
1958
- type GuardrailInputPayload,
1959
- type GuardrailOutputPayload,
1960
- } from '@framers/agentos';
1961
- import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
1962
-
1963
- const piiGuardrail: IGuardrailService = {
1964
- config: {
1965
- evaluateStreamingChunks: true,
1966
- canSanitize: true,
1967
- },
1968
-
1969
- async evaluateInput({ input }: GuardrailInputPayload) {
1970
- // Check for SSN patterns in user input
1971
- const ssnPattern = /\b\d{3}-\d{2}-\d{4}\b/g;
1972
- if (input.textInput && ssnPattern.test(input.textInput)) {
1973
- return {
1974
- action: GuardrailAction.SANITIZE,
1975
- modifiedText: input.textInput.replace(ssnPattern, '[SSN REDACTED]'),
1976
- reason: 'PII detected in user input',
1977
- reasonCode: 'PII_SSN',
1978
- };
1979
- }
1980
- return { action: GuardrailAction.ALLOW };
1981
- },
1982
-
1983
- async evaluateOutput({ chunk }: GuardrailOutputPayload) {
1984
- const text =
1985
- chunk.type === 'TEXT_DELTA'
1986
- ? chunk.textDelta ?? ''
1987
- : chunk.type === 'FINAL_RESPONSE'
1988
- ? chunk.finalResponseText ?? ''
1989
- : '';
1990
-
1991
- if (text.toLowerCase().includes('password')) {
1992
- return {
1993
- action: GuardrailAction.BLOCK,
1994
- reason: 'Output contains potentially sensitive credential information',
1995
- reasonCode: 'CREDENTIAL_LEAK',
1996
- };
1997
- }
1998
- return { action: GuardrailAction.ALLOW };
1999
- },
2000
- };
2001
-
2002
- const agent = new AgentOS();
2003
- const config = await createTestAgentOSConfig();
2004
- await agent.initialize({ ...config, guardrailService: piiGuardrail });
2005
- ```
2006
-
2007
- > **For production PII redaction**, use the built-in `createPiiRedactionGuardrail()` extension
2008
- > instead of hand-rolled regex. It provides four-tier detection (regex + NLP + BERT NER +
2009
- > LLM-as-judge), streaming support, and configurable redaction styles. See
2010
- > [GUARDRAILS_USAGE.md](./docs/safety/GUARDRAILS_USAGE.md) for full examples.
2011
-
2012
- ### Channel Adapters
2013
-
2014
- ```typescript
2015
- import {
2016
- EXTENSION_KIND_MESSAGING_CHANNEL,
2017
- type ExtensionManifest,
2018
- type IChannelAdapter,
2019
- type ChannelPlatform,
2020
- } from '@framers/agentos';
2021
-
2022
- // Implement a custom channel adapter
2023
- const myAdapter: IChannelAdapter = {
2024
- platform: 'webchat' as ChannelPlatform,
2025
- capabilities: new Set(['text', 'rich_text', 'images', 'typing_indicator']),
2026
-
2027
- async connect() { /* establish connection */ },
2028
- async disconnect() { /* clean up */ },
2029
- async sendMessage(channelId, message) { /* send outbound */ },
2030
- onMessage(handler) { /* register inbound handler */ },
2031
- getConnectionInfo() {
2032
- return { status: 'connected', connectedSince: new Date().toISOString() };
2033
- },
2034
- };
2035
-
2036
- // Register via extension manifest
2037
- const manifest: ExtensionManifest = {
2038
- packs: [{
2039
- factory: async () => ({
2040
- name: 'my-channels',
2041
- descriptors: [{
2042
- id: 'webchat-adapter',
2043
- kind: EXTENSION_KIND_MESSAGING_CHANNEL,
2044
- payload: myAdapter,
2045
- }],
2046
- }),
2047
- }],
2048
- };
2049
- ```
2050
-
2051
- ### Voice Calls
2052
-
2053
- ```typescript
2054
- import { CallManager, type IVoiceCallProvider } from '@framers/agentos';
2055
-
2056
- const callManager = new CallManager();
2057
-
2058
- // Initiate an outbound call
2059
- const call = await callManager.initiateCall({
2060
- provider: 'twilio',
2061
- to: '+1234567890',
2062
- from: '+0987654321',
2063
- agentId: 'support-agent',
2064
- });
2065
-
2066
- // Monitor call state transitions
2067
- call.on('stateChange', (newState) => {
2068
- console.log(`Call ${call.id}: ${newState}`);
2069
- // initiated -> ringing -> answered -> active -> speaking <-> listening -> completed
2070
- });
2071
-
2072
- // Handle call completion
2073
- call.on('completed', (summary) => {
2074
- console.log('Duration:', summary.durationMs);
2075
- console.log('Transcript:', summary.transcript);
2076
- });
2077
- ```
2078
-
2079
- ---
2080
-
2081
- ## Package Exports
2082
-
2083
- AgentOS provides 112 export paths for fine-grained imports. Key entry points:
2084
-
2085
- ```typescript
2086
- // Main entry -- all public types and classes
2087
- import { AgentOS, AgentOSResponseChunkType, /* ... */ } from '@framers/agentos';
2088
-
2089
- // Configuration
2090
- import { createAgentOSConfig, createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
766
+ // Orchestration (workflow, graph, mission builders)
767
+ import { workflow, AgentGraph, mission, START, END } from '@framers/agentos/orchestration';
2091
768
 
2092
769
  // Safety primitives
2093
770
  import { CircuitBreaker, CostGuard, StuckDetector } from '@framers/agentos/safety/runtime';
2094
771
 
2095
772
  // Guardrails
2096
- import { GuardrailAction } from '@framers/agentos/safety/guardrails';
2097
- import { ParallelGuardrailDispatcher } from '@framers/agentos/safety/guardrails';
773
+ import { GuardrailAction, ParallelGuardrailDispatcher } from '@framers/agentos/safety/guardrails';
2098
774
 
2099
- // Tools
2100
- import type { ITool, ToolExecutionResult } from '@framers/agentos/core/tools';
2101
-
2102
- // HITL
2103
- import type { IHumanInteractionManager } from '@framers/agentos/orchestration/hitl';
2104
-
2105
- // RAG
775
+ // RAG and GraphRAG
2106
776
  import { VectorStoreManager, EmbeddingManager, RetrievalAugmentor } from '@framers/agentos/rag';
2107
777
  import { GraphRAGEngine } from '@framers/agentos/rag/graphrag';
2108
778
 
2109
779
  // Skills
2110
780
  import { SkillRegistry, SkillLoader } from '@framers/agentos/skills';
2111
781
 
2112
- // Extension runtime helpers and built-in guardrail packs
2113
- import { SharedServiceRegistry } from '@framers/agentos';
2114
- import { createPiiRedactionGuardrail } from '@framers/agentos-ext-pii-redaction';
2115
- import { createMLClassifierGuardrail } from '@framers/agentos-ext-ml-classifiers';
2116
- import { createTopicalityGuardrail } from '@framers/agentos-ext-topicality';
2117
- import { createCodeSafetyGuardrail } from '@framers/agentos-ext-code-safety';
2118
- import { createGroundingGuardrail } from '@framers/agentos-ext-grounding-guard';
782
+ // Tools
783
+ import type { ITool, ToolExecutionResult } from '@framers/agentos/core/tools';
784
+
785
+ // HITL
786
+ import type { IHumanInteractionManager } from '@framers/agentos/orchestration/hitl';
2119
787
 
2120
- // Deep imports (wildcard exports)
788
+ // Deep imports via wildcard (up to 4 levels)
2121
789
  import { SomeType } from '@framers/agentos/safety/runtime/CircuitBreaker';
2122
- import { SomeConfig } from '@framers/agentos/config/ToolOrchestratorConfig';
2123
790
  ```
2124
791
 
2125
- Wildcard exports support paths up to 4 levels deep:
2126
- - `./*` -- `dist/*.js`
2127
- - `./*/*` -- `dist/*/*.js`
2128
- - `./*/*/*` -- `dist/*/*/*.js`
2129
- - `./*/*/*/*` -- `dist/*/*/*/*.js`
2130
-
2131
792
  ---
2132
793
 
2133
- ## Internal Documentation
794
+ ## Documentation
2134
795
 
2135
- The `docs/` directory contains specification and reference documents:
2136
-
2137
- | Document | Description |
2138
- |----------|-------------|
2139
- | [`AGENCY_API.md`](./docs/orchestration/AGENCY_API.md) | `agency()` reference: all 5 strategies, HITL, guardrails, RAG, voice, nested agencies, full-featured example |
2140
- | [`HIGH_LEVEL_API.md`](./docs/getting-started/HIGH_LEVEL_API.md) | `generateText()`, `streamText()`, `generateObject()`, `streamObject()`, `embedText()`, `generateImage()`, single `agent()` |
2141
- | [`ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md) | Complete system architecture with data flow diagrams |
2142
- | [`SAFETY_PRIMITIVES.md`](./docs/safety/SAFETY_PRIMITIVES.md) | Circuit breaker, cost guard, stuck detection, dedup API reference |
2143
- | [`PLANNING_ENGINE.md`](./docs/orchestration/PLANNING_ENGINE.md) | ReAct reasoning, multi-step task planning specification |
2144
- | [`HUMAN_IN_THE_LOOP.md`](./docs/safety/HUMAN_IN_THE_LOOP.md) | Approval workflows, clarification, escalation patterns |
2145
- | [`GUARDRAILS_USAGE.md`](./docs/safety/GUARDRAILS_USAGE.md) | Input/output guardrail implementation patterns |
2146
- | [`RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) | Vector store setup, embedding models, data source config |
796
+ | Guide | What it covers |
797
+ |-------|---------------|
798
+ | [`ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md) | Full system architecture, data flow diagrams, layer breakdown |
799
+ | [`HIGH_LEVEL_API.md`](./docs/getting-started/HIGH_LEVEL_API.md) | `generateText`, `streamText`, `generateObject`, `agent` reference |
800
+ | [`AGENCY_API.md`](./docs/orchestration/AGENCY_API.md) | `agency()` -- all strategies, HITL, guardrails, RAG, nested agencies |
801
+ | [`UNIFIED_ORCHESTRATION.md`](./docs/orchestration/UNIFIED_ORCHESTRATION.md) | Orchestration layer overview (workflow, graph, mission) |
802
+ | [`WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md) | `workflow()` DSL reference |
803
+ | [`AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md) | `AgentGraph` builder reference |
804
+ | [`MISSION_API.md`](./docs/orchestration/MISSION_API.md) | `mission()` goal-driven orchestration |
805
+ | [`CHECKPOINTING.md`](./docs/orchestration/CHECKPOINTING.md) | Persistent checkpointing and fault recovery |
806
+ | [`COGNITIVE_MECHANISMS.md`](./docs/memory/COGNITIVE_MECHANISMS.md) | 8 cognitive memory mechanisms, 30+ APA citations |
807
+ | [`RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) | Vector store setup, embedding models, data sources |
2147
808
  | [`MULTIMODAL_RAG.md`](./docs/memory/MULTIMODAL_RAG.md) | Image, audio, and document RAG pipelines |
2148
- | [`STRUCTURED_OUTPUT.md`](./docs/orchestration/STRUCTURED_OUTPUT.md) | JSON schema validation, entity extraction, function calling |
2149
- | [`AGENT_COMMUNICATION.md`](./docs/architecture/AGENT_COMMUNICATION.md) | Inter-agent messaging, handoffs, shared memory |
2150
- | [`TOOL_CALLING_AND_LOADING.md`](./docs/extensions/TOOL_CALLING_AND_LOADING.md) | Tool registration, discovery, execution pipeline |
2151
- | [`OBSERVABILITY.md`](./docs/observability/OBSERVABILITY.md) | OpenTelemetry setup, custom spans, metrics export |
2152
- | [`COST_OPTIMIZATION.md`](./docs/safety/COST_OPTIMIZATION.md) | Token usage monitoring, caching strategies, model routing |
2153
- | [`SKILLS.md`](./docs/extensions/SKILLS.md) | SKILL.md format specification, skill authoring guide |
2154
- | [`PLATFORM_SUPPORT.md`](./docs/architecture/PLATFORM_SUPPORT.md) | Channel platform capabilities and adapter configuration |
2155
- | [`ECOSYSTEM.md`](./docs/architecture/ECOSYSTEM.md) | Extension ecosystem, official packs, community extensions |
2156
- | [`PROVENANCE_IMMUTABILITY.md`](./docs/safety/PROVENANCE_IMMUTABILITY.md) | Sealed agents, signed event ledger, external anchoring |
2157
- | [`IMMUTABLE_AGENTS.md`](./docs/safety/IMMUTABLE_AGENTS.md) | Agent sealing, toolset manifests, revision tracking |
2158
- | [`RFC_EXTENSION_STANDARDS.md`](./docs/extensions/RFC_EXTENSION_STANDARDS.md) | Extension pack authoring standards and conventions |
2159
- | [`EVALUATION_FRAMEWORK.md`](./docs/observability/EVALUATION_FRAMEWORK.md) | Agent evaluation, benchmarking, quality metrics |
2160
- | [`RECURSIVE_SELF_BUILDING_AGENTS.md`](./docs/architecture/RECURSIVE_SELF_BUILDING_AGENTS.md) | Self-modifying agent patterns |
2161
- | [`LOGGING.md`](./docs/observability/LOGGING.md) | Structured logging configuration with pino |
2162
- | [`CLIENT_SIDE_STORAGE.md`](./docs/memory/CLIENT_SIDE_STORAGE.md) | Browser-compatible storage adapters |
2163
- | [`SQL_STORAGE_QUICKSTART.md`](./docs/getting-started/SQL_STORAGE_QUICKSTART.md) | SQLite/Postgres setup with `@framers/sql-storage-adapter` |
2164
- | [`RELEASING.md`](./docs/getting-started/RELEASING.md) | Release process and semantic versioning |
2165
-
2166
- ---
2167
-
2168
- ## Key Design Patterns
2169
-
2170
- 1. **Interface-driven design** -- All major components define interface contracts (`IAgentOS`, `ITool`, `IGuardrailService`, `IChannelAdapter`, `IVoiceCallProvider`, `IVectorStore`, etc.). Implementations are swappable.
2171
-
2172
- 2. **Streaming-first** -- Core interaction methods return `AsyncGenerator<AgentOSResponse>` for token-level streaming with natural backpressure. Consumers process chunks as they arrive.
2173
-
2174
- 3. **Extension system** -- Pluggable components via `ExtensionDescriptor` with 12 kinds, priority stacking, lifecycle hooks, and secret management. Extensions can be loaded from npm packages, local modules, or inline factories.
2175
-
2176
- 4. **Multi-provider** -- LLM providers, vector stores, voice providers, and channel adapters all support multiple backend implementations with runtime switching.
2177
-
2178
- 5. **Safety layering** -- Defense-in-depth: input guardrails, output guardrails, circuit breakers, cost guards, tool execution guards, stuck detection, action deduplication, and HITL approval gates.
2179
-
2180
- 6. **Observability** -- OpenTelemetry integration throughout the stack with distributed tracing, custom metrics, cost tracking, and structured logging via pino.
809
+ | [`MEMORY_SCALING.md`](./docs/memory/MEMORY_SCALING.md) | 4-tier vector storage scaling path |
810
+ | [`GUARDRAILS_USAGE.md`](./docs/safety/GUARDRAILS_USAGE.md) | Guardrail implementation patterns |
811
+ | [`SAFETY_PRIMITIVES.md`](./docs/safety/SAFETY_PRIMITIVES.md) | Circuit breaker, cost guard, stuck detection |
812
+ | [`HUMAN_IN_THE_LOOP.md`](./docs/safety/HUMAN_IN_THE_LOOP.md) | Approval workflows, clarification, escalation |
813
+ | [`PLANNING_ENGINE.md`](./docs/orchestration/PLANNING_ENGINE.md) | ReAct reasoning, task planning |
814
+ | [`STRUCTURED_OUTPUT.md`](./docs/orchestration/STRUCTURED_OUTPUT.md) | JSON schema validation, entity extraction |
815
+ | [`AGENT_COMMUNICATION.md`](./docs/architecture/AGENT_COMMUNICATION.md) | Inter-agent messaging and handoffs |
816
+ | [`PLATFORM_SUPPORT.md`](./docs/architecture/PLATFORM_SUPPORT.md) | 37 channel platform capabilities |
817
+ | [`OBSERVABILITY.md`](./docs/observability/OBSERVABILITY.md) | OpenTelemetry setup, tracing, metrics |
818
+ | [`COST_OPTIMIZATION.md`](./docs/safety/COST_OPTIMIZATION.md) | Token usage, caching, model routing |
819
+ | [`SKILLS.md`](./docs/extensions/SKILLS.md) | SKILL.md format, skill authoring guide |
820
+ | [`EMERGENT_CAPABILITIES.md`](./docs/architecture/EMERGENT_CAPABILITIES.md) | Runtime tool forging, tiered promotion |
821
+ | [`SQL_STORAGE_QUICKSTART.md`](./docs/getting-started/SQL_STORAGE_QUICKSTART.md) | SQLite/Postgres setup |
822
+ | [`ECOSYSTEM.md`](./docs/architecture/ECOSYSTEM.md) | Extension ecosystem and official packs |
2181
823
 
2182
824
  ---
2183
825
 
@@ -2191,8 +833,6 @@ pnpm run build
2191
833
  pnpm run test
2192
834
  ```
2193
835
 
2194
- **Available scripts:**
2195
-
2196
836
  | Script | Purpose |
2197
837
  |--------|---------|
2198
838
  | `pnpm run build` | Clean, compile TypeScript, resolve aliases, fix ESM imports |
@@ -2222,6 +862,6 @@ See the [Contributing Guide](https://github.com/framersai/agentos/blob/master/CO
2222
862
  <img src="https://raw.githubusercontent.com/framersai/agentos/master/assets/frame-logo-green-no-tagline.svg" alt="Frame.dev" height="40" />
2223
863
  </a>
2224
864
 
2225
- **Built by [Frame.dev](https://frame.dev)** · [@framersai](https://github.com/framersai)
865
+ **Built by [Frame.dev](https://frame.dev)**
2226
866
 
2227
867
  </div>