@framers/agentos 0.1.156 → 0.1.158
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +554 -1914
- package/dist/api/agent.d.ts.map +1 -1
- package/dist/api/agent.js +11 -0
- package/dist/api/agent.js.map +1 -1
- package/dist/api/types.d.ts +15 -0
- package/dist/api/types.d.ts.map +1 -1
- package/dist/api/types.js.map +1 -1
- package/dist/query-router/QueryRouter.d.ts.map +1 -1
- package/dist/query-router/QueryRouter.js.map +1 -1
- package/dist/query-router/types.d.ts +12 -0
- package/dist/query-router/types.d.ts.map +1 -1
- package/dist/query-router/types.js +1 -0
- package/dist/query-router/types.js.map +1 -1
- package/dist/rag/citation/CitationVerifier.d.ts +29 -0
- package/dist/rag/citation/CitationVerifier.d.ts.map +1 -0
- package/dist/rag/citation/CitationVerifier.js +116 -0
- package/dist/rag/citation/CitationVerifier.js.map +1 -0
- package/dist/rag/citation/cosine.d.ts +10 -0
- package/dist/rag/citation/cosine.d.ts.map +1 -0
- package/dist/rag/citation/cosine.js +21 -0
- package/dist/rag/citation/cosine.js.map +1 -0
- package/dist/rag/citation/index.d.ts +8 -0
- package/dist/rag/citation/index.d.ts.map +1 -0
- package/dist/rag/citation/index.js +7 -0
- package/dist/rag/citation/index.js.map +1 -0
- package/dist/rag/citation/types.d.ts +68 -0
- package/dist/rag/citation/types.d.ts.map +1 -0
- package/dist/rag/citation/types.js +9 -0
- package/dist/rag/citation/types.js.map +1 -0
- package/dist/rag/index.d.ts +2 -0
- package/dist/rag/index.d.ts.map +1 -1
- package/dist/rag/index.js +2 -0
- package/dist/rag/index.js.map +1 -1
- package/dist/rag/reranking/IRerankerService.d.ts +9 -0
- package/dist/rag/reranking/IRerankerService.d.ts.map +1 -1
- package/dist/rag/reranking/RerankerService.d.ts +12 -1
- package/dist/rag/reranking/RerankerService.d.ts.map +1 -1
- package/dist/rag/reranking/RerankerService.js +46 -0
- package/dist/rag/reranking/RerankerService.js.map +1 -1
- package/dist/rag/reranking/index.d.ts +2 -0
- package/dist/rag/reranking/index.d.ts.map +1 -1
- package/dist/rag/reranking/index.js +1 -0
- package/dist/rag/reranking/index.js.map +1 -1
- package/dist/rag/reranking/providers/LlmJudgeReranker.d.ts +55 -0
- package/dist/rag/reranking/providers/LlmJudgeReranker.d.ts.map +1 -0
- package/dist/rag/reranking/providers/LlmJudgeReranker.js +121 -0
- package/dist/rag/reranking/providers/LlmJudgeReranker.js.map +1 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@
|
|
|
6
6
|
|
|
7
7
|
# AgentOS
|
|
8
8
|
|
|
9
|
-
**Build autonomous AI agents with adaptive intelligence
|
|
9
|
+
**Build autonomous AI agents with adaptive intelligence, cognitive memory, and emergent behaviors. Open-source TypeScript runtime.**
|
|
10
10
|
|
|
11
11
|
[](https://www.npmjs.com/package/@framers/agentos)
|
|
12
12
|
[](https://github.com/framersai/agentos/actions/workflows/ci.yml)
|
|
@@ -21,207 +21,127 @@
|
|
|
21
21
|
|
|
22
22
|
---
|
|
23
23
|
|
|
24
|
-
## Table of Contents
|
|
25
|
-
|
|
26
|
-
- [Overview](#overview)
|
|
27
|
-
- [Adaptive Intelligence](#adaptive-intelligence)
|
|
28
|
-
- [Emergent Behaviors](#emergent-behaviors)
|
|
29
|
-
- [Quick Start](#quick-start)
|
|
30
|
-
- [1. Generate Text (Provider-First)](#1-generate-text-provider-first)
|
|
31
|
-
- [2. Agent with Personality & Memory](#2-agent-with-personality--memory)
|
|
32
|
-
- [3. Multimodal RAG](#3-multimodal-rag)
|
|
33
|
-
- [4. Structured Output (Zod Validation)](#4-structured-output-zod-validation)
|
|
34
|
-
- [5. Workflows & Graphs](#5-workflows--graphs)
|
|
35
|
-
- [6. Multi-Agent Teams](#6-multi-agent-teams)
|
|
36
|
-
- [7. Voice Pipeline](#7-voice-pipeline)
|
|
37
|
-
- [8. Guardrails & Security](#8-guardrails--security)
|
|
38
|
-
- [Default Models Per Provider](#default-models-per-provider)
|
|
39
|
-
- [System Architecture](#system-architecture)
|
|
40
|
-
- [Architecture Diagram](#architecture-diagram)
|
|
41
|
-
- [Request Lifecycle](#request-lifecycle)
|
|
42
|
-
- [Layer Breakdown](#layer-breakdown)
|
|
43
|
-
- [Core Modules](#core-modules)
|
|
44
|
-
- [API Layer](#api-layer)
|
|
45
|
-
- [Cognitive Substrate (GMI)](#cognitive-substrate-gmi)
|
|
46
|
-
- [LLM Provider Management](#llm-provider-management)
|
|
47
|
-
- [Tool System](#tool-system)
|
|
48
|
-
- [Extension System](#extension-system)
|
|
49
|
-
- [Planning Engine](#planning-engine)
|
|
50
|
-
- [Conversation Management](#conversation-management)
|
|
51
|
-
- [RAG (Retrieval Augmented Generation)](#rag-retrieval-augmented-generation)
|
|
52
|
-
- [Bundled Platform Knowledge](#bundled-platform-knowledge)
|
|
53
|
-
- [Safety and Guardrails](#safety-and-guardrails)
|
|
54
|
-
- [Human-in-the-Loop (HITL)](#human-in-the-loop-hitl)
|
|
55
|
-
- [Channels System](#channels-system)
|
|
56
|
-
- [Voice and Telephony](#voice-and-telephony)
|
|
57
|
-
- [Unified Orchestration Layer](#unified-orchestration-layer)
|
|
58
|
-
- [Multi-Agent Coordination](#multi-agent-coordination)
|
|
59
|
-
- [Observability](#observability)
|
|
60
|
-
- [Skills](#skills)
|
|
61
|
-
- [Structured Output](#structured-output)
|
|
62
|
-
- [Configuration](#configuration)
|
|
63
|
-
- [Development (Quick Start)](#development-quick-start)
|
|
64
|
-
- [Production](#production)
|
|
65
|
-
- [Multiple Providers](#multiple-providers)
|
|
66
|
-
- [Environment Variables](#environment-variables)
|
|
67
|
-
- [API Reference](#api-reference)
|
|
68
|
-
- [AgentOS Class](#agentos-class)
|
|
69
|
-
- [IAgentOS Interface](#iagentos-interface)
|
|
70
|
-
- [AgentOSInput](#agentosinput)
|
|
71
|
-
- [AgentOSResponse Streaming](#agentosresponse-streaming)
|
|
72
|
-
- [ITool Interface](#itool-interface)
|
|
73
|
-
- [ExtensionDescriptor](#extensiondescriptor)
|
|
74
|
-
- [IGuardrailService](#iguardrailservice)
|
|
75
|
-
- [IHumanInteractionManager](#ihumaninteractionmanager)
|
|
76
|
-
- [Usage Examples](#usage-examples)
|
|
77
|
-
- [Orchestration Patterns](#orchestration-patterns)
|
|
78
|
-
- [Streaming Chat](#streaming-chat)
|
|
79
|
-
- [Adding Tools](#adding-tools)
|
|
80
|
-
- [Multi-Agent Collaboration](#multi-agent-collaboration)
|
|
81
|
-
- [Human-in-the-Loop Approvals](#human-in-the-loop-approvals)
|
|
82
|
-
- [Structured Data Extraction](#structured-data-extraction)
|
|
83
|
-
- [RAG Memory](#rag-memory)
|
|
84
|
-
- [Custom Guardrails](#custom-guardrails)
|
|
85
|
-
- [Channel Adapters](#channel-adapters)
|
|
86
|
-
- [Voice Calls](#voice-calls)
|
|
87
|
-
- [Package Exports](#package-exports)
|
|
88
|
-
- [Internal Documentation](#internal-documentation)
|
|
89
|
-
- [Contributing](#contributing)
|
|
90
|
-
- [License](#license)
|
|
91
|
-
|
|
92
|
-
---
|
|
93
|
-
|
|
94
24
|
## Overview
|
|
95
25
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
The runtime provides multimodal RAG with cognitive memory (Ebbinghaus decay, 8 neuroscience-backed mechanisms including reconsolidation, retrieval-induced forgetting, involuntary recall, metacognitive FOK, temporal gist extraction, schema encoding, source confidence decay, and emotion regulation ��� all HEXACO personality-modulated), multi-agent orchestration, 37 channel adapters, 6 guardrail packs (PII redaction, ML classifiers, topicality, code safety, grounding guard, content policy rewriter) with prompt injection defense, 21 LLM providers, and 72 curated skills. Self-hostable and production-ready, it handles the full lifecycle from prompt construction through tool execution, safety evaluation, and streaming response delivery.
|
|
26
|
+
AgentOS is an open-source TypeScript runtime for building autonomous AI agents. Agents have personality traits that shape how they think, cognitive memory that determines what they remember, and emergent capabilities that let them forge new tools at runtime. The result is agents that adapt and improve without retraining.
|
|
99
27
|
|
|
100
|
-
**
|
|
28
|
+
Unlike frameworks that focus purely on LLM orchestration, AgentOS treats each agent as a **Generalized Mind Instance** (GMI) — a cognitive entity with its own identity, personality, memory lifecycle, and behavioral adaptation loop. This makes it particularly suited for long-running agents, multi-agent teams, and applications where agent consistency matters.
|
|
101
29
|
|
|
102
|
-
|
|
103
|
-
|----------|-------|
|
|
104
|
-
| Package | `@framers/agentos` |
|
|
105
|
-
| Language | TypeScript 5.4+ / Node.js 18+ |
|
|
106
|
-
| License | Apache 2.0 |
|
|
30
|
+
**What makes it different:**
|
|
107
31
|
|
|
108
|
-
|
|
32
|
+
| vs. | AgentOS differentiator |
|
|
33
|
+
|-----|------------------------|
|
|
34
|
+
| LangChain / LlamaIndex | Cognitive memory with 8 neuroscience-backed mechanisms, HEXACO personality modulation, runtime tool forging |
|
|
35
|
+
| Vercel AI SDK | Multi-agent teams with 6 strategies, full RAG pipeline with 7 vector backends, guardrail packs, voice/telephony |
|
|
36
|
+
| AutoGen / CrewAI | Unified orchestration layer (workflow DAGs, agent graphs, goal-driven missions), personality-driven edge routing |
|
|
109
37
|
|
|
110
|
-
|
|
111
|
-
|------------|---------|
|
|
112
|
-
| `@opentelemetry/api` | Distributed tracing and metrics |
|
|
113
|
-
| `ajv` + `ajv-formats` | JSON Schema validation for tool I/O |
|
|
114
|
-
| `axios` | HTTP client for LLM provider APIs |
|
|
115
|
-
| `lru-cache` | High-performance caching (dedup, embeddings, cost tracking) |
|
|
116
|
-
| `natural` | NLP utilities (tokenization, stemming, sentiment) |
|
|
117
|
-
| `pino` | Structured JSON logging |
|
|
118
|
-
| `uuid` | Unique identifier generation |
|
|
119
|
-
| `yaml` | YAML config parsing (agent configs, skill definitions) |
|
|
38
|
+
**Ecosystem packages:**
|
|
120
39
|
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
|
124
|
-
|
|
125
|
-
| `@framers/
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
| Peer Dependency | Purpose |
|
|
130
|
-
|-----------------|---------|
|
|
131
|
-
| `graphology` + `graphology-communities-louvain` | GraphRAG community detection and memory graph clustering |
|
|
132
|
-
| `hnswlib-node` | HNSW-based approximate nearest neighbor search (falls back to brute-force without) |
|
|
133
|
-
| `neo4j-driver` | Neo4j-backed knowledge graph and GraphRAG (falls back to SQLite without) |
|
|
40
|
+
| Package | Description |
|
|
41
|
+
|---------|-------------|
|
|
42
|
+
| [`@framers/agentos`](https://www.npmjs.com/package/@framers/agentos) | Core runtime -- agents, providers, memory, RAG, orchestration, guardrails |
|
|
43
|
+
| [`@framers/agentos-extensions`](https://www.npmjs.com/package/@framers/agentos-extensions) | Official extension registry (40+ extensions) |
|
|
44
|
+
| [`@framers/agentos-extensions-registry`](https://www.npmjs.com/package/@framers/agentos-extensions-registry) | Curated manifest builder for extension catalogs |
|
|
45
|
+
| [`@framers/agentos-skills`](https://www.npmjs.com/package/@framers/agentos-skills) | 80+ curated SKILL.md skill definitions |
|
|
46
|
+
| [`@framers/agentos-skills-registry`](https://www.npmjs.com/package/@framers/agentos-skills-registry) | Skills catalog SDK (query helpers + snapshot factories) |
|
|
47
|
+
| [`@framers/sql-storage-adapter`](https://www.npmjs.com/package/@framers/sql-storage-adapter) | Cross-platform SQL persistence (SQLite, sql.js, Postgres, IndexedDB) |
|
|
134
48
|
|
|
135
49
|
---
|
|
136
50
|
|
|
137
|
-
##
|
|
51
|
+
## Install
|
|
138
52
|
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
| **@framers/agentos-extensions** | Official extension registry (45+ extensions) | [](https://www.npmjs.com/package/@framers/agentos-extensions) · [GitHub](https://github.com/framersai/agentos-extensions) |
|
|
143
|
-
| **@framers/agentos-extensions-registry** | Curated manifest builder | [](https://www.npmjs.com/package/@framers/agentos-extensions-registry) · [GitHub](https://github.com/framersai/agentos-extensions-registry) |
|
|
144
|
-
| **@framers/agentos-skills-registry** | Skills catalog SDK (query helpers + snapshot factories) | [](https://www.npmjs.com/package/@framers/agentos-skills-registry) · [GitHub](https://github.com/framersai/agentos-skills-registry) |
|
|
145
|
-
| **@framers/agentos-skills** | Skills content (72 SKILL.md files + registry.json) | [](https://www.npmjs.com/package/@framers/agentos-skills) · [GitHub](https://github.com/framersai/agentos-skills) |
|
|
146
|
-
| **docs.agentos.sh** | Documentation site (Docusaurus) | [GitHub](https://github.com/framersai/agentos-live-docs) · [docs.agentos.sh](https://docs.agentos.sh) |
|
|
53
|
+
```bash
|
|
54
|
+
npm install @framers/agentos
|
|
55
|
+
```
|
|
147
56
|
|
|
148
|
-
|
|
57
|
+
Set any provider's API key:
|
|
149
58
|
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
| **@framers/agentos-ext-ml-classifiers** | Toxicity, injection, jailbreak via ONNX BERT | [](https://www.npmjs.com/package/@framers/agentos-ext-ml-classifiers) · [Docs](https://docs.agentos.sh/extensions/built-in/ml-classifiers) |
|
|
154
|
-
| **@framers/agentos-ext-topicality** | Embedding-based topic enforcement + drift detection | [](https://www.npmjs.com/package/@framers/agentos-ext-topicality) · [Docs](https://docs.agentos.sh/extensions/built-in/topicality) |
|
|
155
|
-
| **@framers/agentos-ext-code-safety** | OWASP Top 10 code scanning (25 regex rules) | [](https://www.npmjs.com/package/@framers/agentos-ext-code-safety) · [Docs](https://docs.agentos.sh/extensions/built-in/code-safety) |
|
|
156
|
-
| **@framers/agentos-ext-grounding-guard** | RAG-grounded hallucination detection via NLI | [](https://www.npmjs.com/package/@framers/agentos-ext-grounding-guard) · [Docs](https://docs.agentos.sh/extensions/built-in/grounding-guard) |
|
|
157
|
-
| **@framers/agentos-ext-content-policy-rewriter** | Opt-in content policy (8 categories, LLM rewrite/block, 4 presets) | [Docs](https://docs.agentos.sh/extensions/built-in/content-policy-rewriter) |
|
|
59
|
+
```bash
|
|
60
|
+
export OPENAI_API_KEY=sk-... # or ANTHROPIC_API_KEY, GEMINI_API_KEY, GROQ_API_KEY, etc.
|
|
61
|
+
```
|
|
158
62
|
|
|
159
63
|
---
|
|
160
64
|
|
|
161
|
-
##
|
|
65
|
+
## Quick Start
|
|
162
66
|
|
|
163
|
-
|
|
67
|
+
### 1. Generate Text
|
|
164
68
|
|
|
165
|
-
|
|
166
|
-
|-----------|-------------|
|
|
167
|
-
| **Meta-reflective prompt adaptation** | The PromptBuilder assembles a different system prompt every turn, dynamically incorporating personality traits, mood state, conversation history, retrieved memories, and available tools. No two turns use the same prompt. |
|
|
168
|
-
| **Self-evaluating response quality** | The `self_evaluate` tool scores the agent's own output and adjusts parameters (temperature, verbosity, personality expression) in real time, creating a continuous feedback loop. |
|
|
169
|
-
| **Personality-modulated cognition** | HEXACO traits modulate how the agent processes information. High openness increases creative associations during memory retrieval; high conscientiousness strengthens retrieval-induced forgetting of irrelevant data. |
|
|
170
|
-
| **Autonomous memory consolidation** | The `ConsolidationLoop` prunes weak memories, strengthens frequently-accessed ones, and derives new insights from memory clusters. The agent's knowledge base improves over time without explicit training. |
|
|
171
|
-
| **QueryRouter tiered classification** | The system adapts retrieval depth based on query complexity. Simple questions get fast keyword lookup; complex questions trigger full hybrid RAG with deep research. |
|
|
69
|
+
AgentOS auto-detects which provider to use from your environment variables and maps each provider to a sensible default model (see [Default Models Per Provider](#default-models-per-provider) below):
|
|
172
70
|
|
|
173
|
-
|
|
71
|
+
```typescript
|
|
72
|
+
import { generateText } from '@framers/agentos';
|
|
174
73
|
|
|
175
|
-
|
|
74
|
+
// Zero config -- auto-detects provider from env vars, uses its default model
|
|
75
|
+
// Priority: OPENAI_API_KEY → ANTHROPIC_API_KEY → OPENROUTER_API_KEY → GEMINI → ...
|
|
76
|
+
const result = await generateText({
|
|
77
|
+
prompt: 'Explain how TCP handshakes work in 3 bullets.',
|
|
78
|
+
});
|
|
79
|
+
console.log(result.text);
|
|
176
80
|
|
|
177
|
-
|
|
81
|
+
// Pin a provider -- uses that provider's default model (e.g. anthropic → claude-sonnet-4)
|
|
82
|
+
const pinned = await generateText({
|
|
83
|
+
provider: 'anthropic',
|
|
84
|
+
prompt: 'Compare TCP and UDP.',
|
|
85
|
+
});
|
|
178
86
|
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
87
|
+
// Full control -- explicit provider + model override
|
|
88
|
+
const custom = await generateText({
|
|
89
|
+
provider: 'openai',
|
|
90
|
+
model: 'gpt-4o-mini', // override the default (gpt-4o)
|
|
91
|
+
prompt: 'What is the capital of France?',
|
|
92
|
+
});
|
|
93
|
+
```
|
|
186
94
|
|
|
187
|
-
|
|
95
|
+
16 providers supported. Auto-fallback on 402/429/5xx — if the primary provider fails, the next available provider is tried automatically.
|
|
188
96
|
|
|
189
|
-
|
|
97
|
+
### 2. Streaming
|
|
190
98
|
|
|
191
|
-
```
|
|
192
|
-
|
|
193
|
-
```
|
|
99
|
+
```typescript
|
|
100
|
+
import { streamText } from '@framers/agentos';
|
|
194
101
|
|
|
195
|
-
|
|
102
|
+
const stream = streamText({
|
|
103
|
+
provider: 'openai',
|
|
104
|
+
prompt: 'Write a haiku about distributed systems.',
|
|
105
|
+
});
|
|
196
106
|
|
|
197
|
-
|
|
198
|
-
|
|
107
|
+
// Iterate token-by-token
|
|
108
|
+
for await (const chunk of stream.textStream) {
|
|
109
|
+
process.stdout.write(chunk);
|
|
110
|
+
}
|
|
111
|
+
|
|
112
|
+
// Or await the full result
|
|
113
|
+
const fullText = await stream.text;
|
|
114
|
+
const usage = await stream.usage;
|
|
115
|
+
console.log(`\n\nTokens used: ${usage.totalTokens}`);
|
|
199
116
|
```
|
|
200
117
|
|
|
201
|
-
###
|
|
118
|
+
### 3. Structured Output
|
|
202
119
|
|
|
203
|
-
|
|
120
|
+
Extract typed data from unstructured text with Zod validation:
|
|
204
121
|
|
|
205
122
|
```typescript
|
|
206
|
-
import {
|
|
123
|
+
import { generateObject } from '@framers/agentos';
|
|
124
|
+
import { z } from 'zod';
|
|
207
125
|
|
|
208
|
-
const
|
|
209
|
-
provider: '
|
|
210
|
-
|
|
126
|
+
const { object } = await generateObject({
|
|
127
|
+
provider: 'gemini',
|
|
128
|
+
schema: z.object({
|
|
129
|
+
name: z.string(),
|
|
130
|
+
sentiment: z.enum(['positive', 'negative', 'neutral']),
|
|
131
|
+
topics: z.array(z.string()),
|
|
132
|
+
}),
|
|
133
|
+
prompt: 'Analyze: "The new iPhone camera is incredible but the battery life is disappointing."',
|
|
211
134
|
});
|
|
212
|
-
console.log(result.text);
|
|
213
135
|
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
process.stdout.write(chunk);
|
|
217
|
-
}
|
|
136
|
+
console.log(object);
|
|
137
|
+
// { name: "iPhone Review", sentiment: "neutral", topics: ["camera", "battery"] }
|
|
218
138
|
```
|
|
219
139
|
|
|
220
|
-
|
|
140
|
+
Failed parses are automatically retried with error feedback so the model can self-correct.
|
|
221
141
|
|
|
222
|
-
###
|
|
142
|
+
### 4. Agent with Personality & Memory
|
|
223
143
|
|
|
224
|
-
Agents have HEXACO personality traits that shape their communication style, and cognitive memory
|
|
144
|
+
This is the key differentiator. Agents have HEXACO personality traits that shape their communication style, and cognitive memory that determines what they retain:
|
|
225
145
|
|
|
226
146
|
```typescript
|
|
227
147
|
import { agent } from '@framers/agentos';
|
|
@@ -231,89 +151,77 @@ const tutor = agent({
|
|
|
231
151
|
instructions: 'You are a patient computer science tutor.',
|
|
232
152
|
personality: {
|
|
233
153
|
openness: 0.9, // creative, exploratory answers
|
|
234
|
-
conscientiousness: 0.95,
|
|
235
|
-
agreeableness: 0.85,
|
|
154
|
+
conscientiousness: 0.95, // thorough, well-structured
|
|
155
|
+
agreeableness: 0.85, // warm, encouraging tone
|
|
236
156
|
},
|
|
237
157
|
memory: {
|
|
238
158
|
enabled: true,
|
|
239
|
-
cognitive: true,
|
|
159
|
+
cognitive: true, // Ebbinghaus decay, reconsolidation, involuntary recall
|
|
240
160
|
},
|
|
161
|
+
tools: [{
|
|
162
|
+
name: 'run_code',
|
|
163
|
+
description: 'Execute a code snippet and return output.',
|
|
164
|
+
parameters: { type: 'object', properties: { code: { type: 'string' } }, required: ['code'] },
|
|
165
|
+
execute: async ({ code }) => ({ success: true, output: `Ran: ${code}` }),
|
|
166
|
+
}],
|
|
241
167
|
});
|
|
242
168
|
|
|
169
|
+
// Sessions maintain conversation history
|
|
243
170
|
const session = tutor.session('student-1');
|
|
244
171
|
const reply = await session.send('Explain recursion with an analogy.');
|
|
245
172
|
console.log(reply.text);
|
|
246
173
|
|
|
247
|
-
// Memory persists
|
|
174
|
+
// Memory persists -- the agent remembers prior context
|
|
248
175
|
const followUp = await session.send('Can you expand on that?');
|
|
249
176
|
console.log(followUp.text);
|
|
250
|
-
```
|
|
251
177
|
|
|
252
|
-
|
|
178
|
+
// Export agent config for portability
|
|
179
|
+
const config = tutor.exportJSON();
|
|
180
|
+
```
|
|
253
181
|
|
|
254
|
-
|
|
182
|
+
### 5. Multimodal (Image, Video, Audio, OCR, Embeddings)
|
|
255
183
|
|
|
256
184
|
```typescript
|
|
257
|
-
import {
|
|
185
|
+
import {
|
|
186
|
+
generateImage,
|
|
187
|
+
performOCR,
|
|
188
|
+
embedText,
|
|
189
|
+
generateVideo,
|
|
190
|
+
generateMusic,
|
|
191
|
+
} from '@framers/agentos';
|
|
258
192
|
|
|
259
|
-
|
|
193
|
+
// Generate an image (supports OpenAI, Stability, Replicate, BFL, Fal)
|
|
194
|
+
const image = await generateImage({
|
|
260
195
|
provider: 'openai',
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
enabled: true,
|
|
264
|
-
mode: 'aggressive', // always retrieves before answering
|
|
265
|
-
topK: 10,
|
|
266
|
-
strategy: 'hyde', // generates hypothetical answers for better retrieval
|
|
267
|
-
},
|
|
196
|
+
prompt: 'A cyberpunk cityscape at sunset, neon signs reflecting in rain puddles',
|
|
197
|
+
size: '1024x1024',
|
|
268
198
|
});
|
|
199
|
+
console.log(image.images[0].url);
|
|
269
200
|
|
|
270
|
-
|
|
271
|
-
|
|
272
|
-
|
|
273
|
-
|
|
274
|
-
await session.ingest({ type: 'file', path: './earnings-report.pdf' });
|
|
275
|
-
|
|
276
|
-
const answer = await session.send('What drove revenue growth in Q4?');
|
|
277
|
-
console.log(answer.text); // Cites specific passages from ingested docs
|
|
278
|
-
```
|
|
279
|
-
|
|
280
|
-
### 4. Structured Output (Zod Validation)
|
|
281
|
-
|
|
282
|
-
Extract typed data from unstructured text:
|
|
283
|
-
|
|
284
|
-
```typescript
|
|
285
|
-
import { generateObject } from '@framers/agentos';
|
|
286
|
-
import { z } from 'zod';
|
|
287
|
-
|
|
288
|
-
const { object } = await generateObject({
|
|
289
|
-
provider: 'gemini',
|
|
290
|
-
schema: z.object({
|
|
291
|
-
name: z.string(),
|
|
292
|
-
sentiment: z.enum(['positive', 'negative', 'neutral']),
|
|
293
|
-
topics: z.array(z.string()),
|
|
294
|
-
}),
|
|
295
|
-
prompt: 'Analyze: "The new iPhone camera is incredible but the battery life is disappointing."',
|
|
201
|
+
// OCR: extract text from images (progressive tiers: local -> cloud LLM)
|
|
202
|
+
const ocr = await performOCR({
|
|
203
|
+
image: '/tmp/receipt.png',
|
|
204
|
+
strategy: 'progressive', // tries fast local OCR first, falls back to cloud
|
|
296
205
|
});
|
|
206
|
+
console.log(ocr.text);
|
|
297
207
|
|
|
298
|
-
|
|
299
|
-
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
|
|
303
|
-
|
|
304
|
-
|
|
305
|
-
|
|
306
|
-
```typescript
|
|
307
|
-
import { workflow } from '@framers/agentos';
|
|
208
|
+
// Embeddings (OpenAI, Ollama, or any compatible provider)
|
|
209
|
+
const embedding = await embedText({
|
|
210
|
+
provider: 'openai',
|
|
211
|
+
input: ['Hello world', 'Goodbye world'],
|
|
212
|
+
dimensions: 256,
|
|
213
|
+
});
|
|
214
|
+
console.log(embedding.embeddings[0].length); // 256
|
|
308
215
|
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
.step('publish', { tool: 'blog_publish' });
|
|
216
|
+
// Video generation
|
|
217
|
+
const video = await generateVideo({
|
|
218
|
+
prompt: 'A timelapse of a flower blooming',
|
|
219
|
+
});
|
|
314
220
|
|
|
315
|
-
|
|
316
|
-
|
|
221
|
+
// Music generation
|
|
222
|
+
const music = await generateMusic({
|
|
223
|
+
prompt: 'Lo-fi hip hop beat for studying',
|
|
224
|
+
});
|
|
317
225
|
```
|
|
318
226
|
|
|
319
227
|
### 6. Multi-Agent Teams
|
|
@@ -336,22 +244,87 @@ const team = agency({
|
|
|
336
244
|
reviewer: {
|
|
337
245
|
instructions: 'Check for accuracy and suggest improvements.',
|
|
338
246
|
provider: 'gemini',
|
|
339
|
-
dependsOn: ['writer'],
|
|
247
|
+
dependsOn: ['writer'], // runs after writer completes
|
|
340
248
|
},
|
|
341
249
|
},
|
|
342
|
-
strategy: 'graph',
|
|
343
|
-
memory: { shared: true },
|
|
250
|
+
strategy: 'graph', // dependency-based DAG execution
|
|
251
|
+
memory: { shared: true }, // agents share context
|
|
344
252
|
});
|
|
345
253
|
|
|
346
254
|
const result = await team.generate('Compare TCP vs UDP for game networking.');
|
|
347
255
|
console.log(result.text);
|
|
256
|
+
|
|
257
|
+
// Streaming works too
|
|
258
|
+
const stream = team.stream('Explain QUIC protocol benefits.');
|
|
259
|
+
for await (const chunk of stream.textStream) {
|
|
260
|
+
process.stdout.write(chunk);
|
|
261
|
+
}
|
|
348
262
|
```
|
|
349
263
|
|
|
350
264
|
6 strategies: `sequential`, `parallel`, `debate`, `review-loop`, `hierarchical`, `graph`.
|
|
351
265
|
|
|
352
|
-
|
|
266
|
+
Auto-detection: when any agent declares `dependsOn`, strategy defaults to `graph`.
|
|
267
|
+
|
|
268
|
+
### 7. Orchestration (Workflows, Graphs, Missions)
|
|
269
|
+
|
|
270
|
+
Three authoring APIs that compile to one graph runtime. Choose based on how much control you need:
|
|
271
|
+
|
|
272
|
+
```typescript
|
|
273
|
+
import {
|
|
274
|
+
workflow,
|
|
275
|
+
AgentGraph,
|
|
276
|
+
mission,
|
|
277
|
+
gmiNode,
|
|
278
|
+
toolNode,
|
|
279
|
+
START,
|
|
280
|
+
END,
|
|
281
|
+
} from '@framers/agentos/orchestration';
|
|
282
|
+
import { z } from 'zod';
|
|
283
|
+
|
|
284
|
+
// 1. workflow() -- deterministic DAG with typed I/O
|
|
285
|
+
const pipeline = workflow('content-pipeline')
|
|
286
|
+
.input(z.object({ topic: z.string() }))
|
|
287
|
+
.returns(z.object({ summary: z.string() }))
|
|
288
|
+
.step('research', { tool: 'web_search' })
|
|
289
|
+
.then('draft', { gmi: { instructions: 'Write a blog post from the research.' } })
|
|
290
|
+
.then('review', { gmi: { instructions: 'Review for accuracy and tone.' } })
|
|
291
|
+
.compile();
|
|
292
|
+
|
|
293
|
+
const result = await pipeline.invoke({ topic: 'WebAssembly in 2026' });
|
|
294
|
+
|
|
295
|
+
// 2. AgentGraph -- explicit nodes, edges, cycles, subgraphs
|
|
296
|
+
const graph = new AgentGraph({
|
|
297
|
+
input: z.object({ topic: z.string() }),
|
|
298
|
+
scratch: z.object({ draft: z.string().optional() }),
|
|
299
|
+
artifacts: z.object({ summary: z.string().optional() }),
|
|
300
|
+
})
|
|
301
|
+
.addNode('draft', gmiNode({ instructions: 'Draft a summary', executionMode: 'single_turn' }))
|
|
302
|
+
.addNode('publish', toolNode('publish_report'))
|
|
303
|
+
.addEdge(START, 'draft')
|
|
304
|
+
.addEdge('draft', 'publish')
|
|
305
|
+
.addEdge('publish', END)
|
|
306
|
+
.compile();
|
|
307
|
+
|
|
308
|
+
await graph.invoke({ topic: 'quantum computing' });
|
|
353
309
|
|
|
354
|
-
|
|
310
|
+
// 3. mission() -- goal-driven, planner decides the steps
|
|
311
|
+
const researcher = mission('deep-research')
|
|
312
|
+
.input(z.object({ topic: z.string() }))
|
|
313
|
+
.goal('Research {topic} and produce a cited summary')
|
|
314
|
+
.returns(z.object({ summary: z.string() }))
|
|
315
|
+
.planner({ strategy: 'plan_and_execute', maxSteps: 8 })
|
|
316
|
+
.compile();
|
|
317
|
+
|
|
318
|
+
// Preview the plan without executing
|
|
319
|
+
const preview = await researcher.explain({ topic: 'AI safety' });
|
|
320
|
+
console.log(preview.steps.map(s => s.id));
|
|
321
|
+
```
|
|
322
|
+
|
|
323
|
+
All three support persistent checkpointing for fault recovery and time-travel debugging.
|
|
324
|
+
|
|
325
|
+
See [`docs/orchestration/WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md), [`docs/architecture/AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md), [`docs/orchestration/MISSION_API.md`](./docs/orchestration/MISSION_API.md).
|
|
326
|
+
|
|
327
|
+
### 8. Voice Pipeline
|
|
355
328
|
|
|
356
329
|
```typescript
|
|
357
330
|
import { agent } from '@framers/agentos';
|
|
@@ -364,15 +337,13 @@ const receptionist = agent({
|
|
|
364
337
|
stt: { provider: 'deepgram' },
|
|
365
338
|
},
|
|
366
339
|
});
|
|
367
|
-
|
|
368
|
-
// Voice sessions stream audio in real-time
|
|
369
|
-
const call = receptionist.voiceSession('call-1');
|
|
370
|
-
await call.start(); // begins listening
|
|
371
340
|
```
|
|
372
341
|
|
|
373
|
-
|
|
342
|
+
Telephony providers: Twilio, Telnyx, Plivo. Speech: ElevenLabs, Deepgram, OpenAI Whisper, and more.
|
|
343
|
+
|
|
344
|
+
### 9. Guardrails
|
|
374
345
|
|
|
375
|
-
5-tier security with
|
|
346
|
+
5-tier security with 6 guardrail packs:
|
|
376
347
|
|
|
377
348
|
```typescript
|
|
378
349
|
import { agent } from '@framers/agentos';
|
|
@@ -382,1802 +353,473 @@ const secureBot = agent({
|
|
|
382
353
|
instructions: 'You are a customer support agent.',
|
|
383
354
|
security: { tier: 'strict' },
|
|
384
355
|
guardrails: {
|
|
385
|
-
input: ['pii-redaction', 'ml-classifiers'],
|
|
386
|
-
output: ['grounding-guard', 'code-safety'],
|
|
356
|
+
input: ['pii-redaction', 'ml-classifiers'], // block PII + detect injection
|
|
357
|
+
output: ['grounding-guard', 'code-safety'], // prevent hallucination + unsafe code
|
|
387
358
|
},
|
|
388
359
|
});
|
|
389
360
|
```
|
|
390
361
|
|
|
391
|
-
5 tiers: `dangerous`
|
|
362
|
+
5 security tiers: `dangerous` > `permissive` > `balanced` > `strict` > `paranoid`.
|
|
363
|
+
|
|
364
|
+
6 guardrail packs:
|
|
365
|
+
- **PII Redaction** -- four-tier detection (regex + NLP + NER + LLM)
|
|
366
|
+
- **ML Classifiers** -- toxicity, injection, jailbreak via ONNX BERT
|
|
367
|
+
- **Topicality** -- embedding-based topic enforcement + drift detection
|
|
368
|
+
- **Code Safety** -- OWASP Top 10 code scanning (25 regex rules)
|
|
369
|
+
- **Grounding Guard** -- RAG-source claim verification and hallucination detection
|
|
370
|
+
- **Content Policy Rewriter** -- 8 categories, LLM rewrite/block, 4 presets
|
|
392
371
|
|
|
393
372
|
### Default Models Per Provider
|
|
394
373
|
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
|
398
|
-
|
|
399
|
-
|
|
|
400
|
-
|
|
|
401
|
-
|
|
|
402
|
-
|
|
|
403
|
-
|
|
|
404
|
-
|
|
|
405
|
-
|
|
|
374
|
+
When you specify `provider` without `model`, these defaults are used:
|
|
375
|
+
|
|
376
|
+
| Provider | Default Text Model | Default Image Model | Env Var |
|
|
377
|
+
|---|---|---|---|
|
|
378
|
+
| `openai` | gpt-4o | gpt-image-1 | `OPENAI_API_KEY` |
|
|
379
|
+
| `anthropic` | claude-sonnet-4 | -- | `ANTHROPIC_API_KEY` |
|
|
380
|
+
| `gemini` | gemini-2.5-flash | -- | `GEMINI_API_KEY` |
|
|
381
|
+
| `ollama` | llama3.2 | stable-diffusion | `OLLAMA_BASE_URL` |
|
|
382
|
+
| `groq` | llama-3.3-70b-versatile | -- | `GROQ_API_KEY` |
|
|
383
|
+
| `openrouter` | openai/gpt-4o | -- | `OPENROUTER_API_KEY` |
|
|
384
|
+
| `together` | Meta-Llama-3.1-70B | -- | `TOGETHER_API_KEY` |
|
|
385
|
+
| `mistral` | mistral-large-latest | -- | `MISTRAL_API_KEY` |
|
|
386
|
+
| `xai` | grok-2 | -- | `XAI_API_KEY` |
|
|
387
|
+
| `stability` | -- | stable-diffusion-xl | `STABILITY_API_KEY` |
|
|
388
|
+
| `replicate` | -- | flux-1.1-pro | `REPLICATE_API_TOKEN` |
|
|
389
|
+
| `bfl` | -- | flux-pro-1.1 | `BFL_API_KEY` |
|
|
390
|
+
| `fal` | -- | fal-ai/flux/dev | `FAL_API_KEY` |
|
|
391
|
+
| `claude-code-cli` | claude-sonnet-4 | -- | `claude` binary on PATH |
|
|
392
|
+
| `gemini-cli` | gemini-2.5-flash | -- | `gemini` binary on PATH |
|
|
393
|
+
| `stable-diffusion-local` | -- | v1-5-pruned-emaonly | `STABLE_DIFFUSION_LOCAL_BASE_URL` |
|
|
406
394
|
|
|
407
|
-
|
|
395
|
+
Auto-detection priority: OpenAI > Anthropic > OpenRouter > Gemini > Groq > Together > Mistral > xAI > claude-code-cli > gemini-cli > Ollama > image-only providers.
|
|
408
396
|
|
|
409
397
|
---
|
|
410
398
|
|
|
411
|
-
##
|
|
399
|
+
## Core Concepts
|
|
412
400
|
|
|
413
|
-
###
|
|
401
|
+
### GMI (Generalized Modular Intelligence)
|
|
414
402
|
|
|
415
|
-
|
|
416
|
-
graph TB
|
|
417
|
-
subgraph Runtime["AgentOS Runtime"]
|
|
418
|
-
API["AgentOS API<br/><i>Service Facade</i>"] --> Orch["AgentOSOrchestrator<br/><i>Delegation Hub</i>"]
|
|
419
|
-
API --> Stream["StreamingManager<br/><i>Async Gen Mgmt</i>"]
|
|
420
|
-
Orch --> Prompt["PromptEngine<br/><i>Dynamic Prompts</i>"]
|
|
403
|
+
Each agent is backed by a GMI instance -- the "brain" that manages working memory, persona overlays, context assembly, and the cognitive loop. A single runtime can manage multiple GMI instances via `GMIManager`.
|
|
421
404
|
|
|
422
|
-
|
|
423
|
-
WM["Working<br/>Memory"] ~~~ CM["Context<br/>Manager"] ~~~ PO["Persona<br/>Overlay"] ~~~ LM["Learning<br/>Module"]
|
|
424
|
-
EM["Episodic<br/>Memory"] ~~~ SM["Semantic<br/>Memory"] ~~~ PM["Procedural<br/>Memory"]
|
|
425
|
-
end
|
|
405
|
+
GMI components: working memory (7 +/- 2 slots, Baddeley's model), context manager, persona overlay switching, adaptation manager (learning rate, style drift), and multi-layer memory (episodic, semantic, procedural, prospective).
|
|
426
406
|
|
|
427
|
-
|
|
428
|
-
Prompt --> GMI
|
|
407
|
+
See [`docs/architecture/ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md) for detailed diagrams and data flow.
|
|
429
408
|
|
|
430
|
-
|
|
431
|
-
GMI --> RAG["RAG<br/>Memory"]
|
|
432
|
-
GMI --> Plan["Planning<br/>Engine"]
|
|
433
|
-
GMI --> WF["Workflow<br/>Engine"]
|
|
409
|
+
### HEXACO Personality
|
|
434
410
|
|
|
435
|
-
|
|
436
|
-
Plan --> ReAct["ReAct<br/>Reasoner"]
|
|
411
|
+
Six personality dimensions modulate agent behavior at every level -- from memory retrieval to response style:
|
|
437
412
|
|
|
438
|
-
|
|
439
|
-
|
|
440
|
-
|
|
413
|
+
| Trait | High Value Effect | Low Value Effect |
|
|
414
|
+
|-------|-------------------|------------------|
|
|
415
|
+
| **Honesty-Humility** | Source skepticism, transparent reasoning | Confident assertions |
|
|
416
|
+
| **Emotionality** | Memory reconsolidation drift toward mood | Stable recall |
|
|
417
|
+
| **Extraversion** | Stronger feeling-of-knowing, verbose output | Reserved, concise output |
|
|
418
|
+
| **Agreeableness** | Emotion regulation during consolidation | Unfiltered expression |
|
|
419
|
+
| **Conscientiousness** | Retrieval-induced forgetting of irrelevant data | Broader recall |
|
|
420
|
+
| **Openness** | Involuntary recall, novelty-boosted encoding | Schema-conforming recall |
|
|
441
421
|
|
|
442
|
-
|
|
443
|
-
Embed --> LLM
|
|
444
|
-
ReAct --> LLM
|
|
422
|
+
Personality traits are set at agent creation and can be adapted within bounded limits at runtime via `adapt_personality`.
|
|
445
423
|
|
|
446
|
-
|
|
447
|
-
Ext["Extension Manager<br/><i>12+ kinds</i>"] ~~~ Chan["Channel Router<br/><i>37 platforms</i>"] ~~~ Call["Call Manager<br/><i>Voice/Tel.</i>"]
|
|
448
|
-
Obs["Observability<br/><i>OpenTelemetry</i>"] ~~~ HITL["HITL Manager<br/><i>Approval/Escal.</i>"] ~~~ Skill["Skill Registry<br/><i>SKILL.md</i>"]
|
|
449
|
-
end
|
|
424
|
+
### Cognitive Memory
|
|
450
425
|
|
|
451
|
-
|
|
452
|
-
style GMI fill:#151520,stroke:#00f5ff,stroke-width:1px,color:#f2f2fa
|
|
453
|
-
style LLM fill:#151520,stroke:#8b5cf6,stroke-width:1px,color:#f2f2fa
|
|
454
|
-
```
|
|
426
|
+
8 neuroscience-backed mechanisms, all HEXACO personality-modulated:
|
|
455
427
|
|
|
456
|
-
|
|
428
|
+
| Mechanism | Effect | Citation |
|
|
429
|
+
|-----------|--------|----------|
|
|
430
|
+
| Reconsolidation | Retrieved memories drift toward current mood | [Nader, Schiller & LeDoux (2000)](https://doi.org/10.1038/35021052). *Nature*, 406, 722-726 |
|
|
431
|
+
| Retrieval-Induced Forgetting | Retrieving one memory suppresses similar competitors | [Anderson, Bjork & Bjork (1994)](https://doi.org/10.1037/0278-7393.20.5.1063). *JEP: Learning*, 20, 1063-1087 |
|
|
432
|
+
| Involuntary Recall | Random surfacing of old high-vividness memories | [Berntsen (1996)](https://doi.org/10.1002/(SICI)1099-0720(199610)10:5%3C435::AID-ACP395%3E3.0.CO;2-8). *Applied Cognitive Psychology*, 10, 435-454 |
|
|
433
|
+
| Metacognitive FOK | Feeling-of-knowing scoring for tip-of-tongue states | [Hart (1965)](https://doi.org/10.1037/h0022263). *JEPG*, 56, 208-216 |
|
|
434
|
+
| Temporal Gist Extraction | Old traces compressed to core assertions | [Reyna & Brainerd (1995)](https://doi.org/10.1006/drev.1995.1002). *Developmental Review*, 15, 3-47 |
|
|
435
|
+
| Schema Encoding | Novel input boosted, schema-matching encoded efficiently | [Bartlett (1932)](https://doi.org/10.1017/CBO9780511759185). *Remembering*. Cambridge University Press |
|
|
436
|
+
| Source Confidence Decay | Agent inferences decay faster than observations | [Johnson, Hashtroudi & Lindsay (1993)](https://doi.org/10.1037/0033-2909.114.1.3). *Psych. Bulletin*, 114, 3-28 |
|
|
437
|
+
| Emotion Regulation | Reappraisal + suppression during consolidation | [Gross (1998)](https://doi.org/10.1037/1089-2680.2.3.271). *Review of General Psychology*, 2, 271-299 |
|
|
457
438
|
|
|
458
|
-
|
|
439
|
+
**HEXACO Personality Modulation** -- each mechanism's intensity is governed by one or more HEXACO traits:
|
|
459
440
|
|
|
460
|
-
|
|
461
|
-
|
|
462
|
-
|
|
441
|
+
| HEXACO Trait | Mechanisms Modulated | Effect |
|
|
442
|
+
|--------------|---------------------|--------|
|
|
443
|
+
| Emotionality | Reconsolidation | Higher emotionality increases mood-congruent drift rate |
|
|
444
|
+
| Conscientiousness | Retrieval-Induced Forgetting | Higher conscientiousness strengthens suppression of irrelevant competitors |
|
|
445
|
+
| Openness | Involuntary Recall, Schema Encoding | Higher openness increases involuntary recall probability and novelty boost |
|
|
446
|
+
| Extraversion | Metacognitive FOK | Higher extraversion strengthens feeling-of-knowing confidence signals |
|
|
447
|
+
| Honesty-Humility | Source Confidence Decay | Higher honesty increases skepticism of agent-inferred sources |
|
|
448
|
+
| Agreeableness | Emotion Regulation | Higher agreeableness strengthens reappraisal during consolidation |
|
|
463
449
|
|
|
464
|
-
|
|
465
|
-
G1 -->|ALLOW| Ctx
|
|
466
|
-
G1 -.->|BLOCK / SANITIZE| Reject1["⛔ Rejected"]
|
|
450
|
+
**Using cognitive mechanisms with `agent()`:**
|
|
467
451
|
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
GMI["4 · GMI Processing<br/><i>Working memory · Persona · Adaptation</i>"] --> LLM
|
|
452
|
+
```typescript
|
|
453
|
+
import { agent } from '@framers/agentos';
|
|
471
454
|
|
|
472
|
-
|
|
455
|
+
const researcher = agent({
|
|
456
|
+
provider: 'anthropic',
|
|
457
|
+
instructions: 'You are a thorough research analyst.',
|
|
458
|
+
personality: {
|
|
459
|
+
openness: 0.9, // High openness -> more involuntary recall, stronger novelty bias
|
|
460
|
+
conscientiousness: 0.85, // High conscientiousness -> stronger RIF suppression
|
|
461
|
+
emotionality: 0.6, // Moderate -> moderate reconsolidation drift
|
|
462
|
+
},
|
|
463
|
+
memory: { enabled: true },
|
|
464
|
+
cognitiveMechanisms: {
|
|
465
|
+
// All 8 mechanisms enabled with defaults -- just pass {}
|
|
466
|
+
// Or tune individual mechanisms:
|
|
467
|
+
reconsolidation: { driftRate: 0.08 },
|
|
468
|
+
involuntaryRecall: { probability: 0.12 },
|
|
469
|
+
temporalGist: { ageThresholdDays: 30 },
|
|
470
|
+
},
|
|
471
|
+
});
|
|
472
|
+
```
|
|
473
473
|
|
|
474
|
-
|
|
475
|
-
ToolCheck -->|Yes| ToolExec["6 · Tool Execution<br/><i>ToolOrchestrator + Permission Guard</i>"]
|
|
476
|
-
ToolExec -->|Loop| LLM
|
|
477
|
-
ToolCheck -->|No| G2
|
|
474
|
+
Pass `{}` for all defaults, or omit entirely to disable (zero overhead).
|
|
478
475
|
|
|
479
|
-
|
|
480
|
-
G2 -->|ALLOW| Stream
|
|
481
|
-
G2 -.->|BLOCK / SANITIZE| Reject2["⛔ Rejected"]
|
|
476
|
+
Memory is organized in a 4-tier hierarchy: `core/` (encoding, decay, working memory), `retrieval/` (composite scoring, graph, prospective), `pipeline/` (consolidation, observation, lifecycle), `io/` (ingestion, import/export).
|
|
482
477
|
|
|
483
|
-
|
|
484
|
-
Post["9 · Post-Processing<br/><i>Memory · Cost · Telemetry</i>"]
|
|
478
|
+
See [`docs/memory/COGNITIVE_MECHANISMS.md`](./docs/memory/COGNITIVE_MECHANISMS.md) for API reference and 30+ APA citations.
|
|
485
479
|
|
|
486
|
-
|
|
487
|
-
style G1 fill:#1c1c28,stroke:#ef4444,color:#f2f2fa
|
|
488
|
-
style G2 fill:#1c1c28,stroke:#ef4444,color:#f2f2fa
|
|
489
|
-
style LLM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa
|
|
490
|
-
style ToolCheck fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
|
|
491
|
-
style ToolExec fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
|
|
492
|
-
style Post fill:#1c1c28,stroke:#10b981,color:#f2f2fa
|
|
493
|
-
```
|
|
480
|
+
### Multimodal RAG
|
|
494
481
|
|
|
495
|
-
|
|
482
|
+
Complete retrieval-augmented generation pipeline:
|
|
496
483
|
|
|
497
|
-
|
|
484
|
+
- **7 vector backends:** InMemory, SQL (SQLite/Postgres), HNSW, Qdrant, Neo4j, Postgres+pgvector, Pinecone
|
|
485
|
+
- **4 retrieval strategies:** keyword, vector, hybrid (RRF), HyDE (Hypothetical Document Embedding)
|
|
486
|
+
- **GraphRAG:** entity/relationship extraction, Louvain community detection, local + global search
|
|
487
|
+
- **4-tier scaling path:** SQLite (dev) > HNSW sidecar (auto at 1K vectors) > Postgres+pgvector > Qdrant/Pinecone
|
|
488
|
+
- **Document ingestion:** PDF, DOCX, HTML, Markdown, URL
|
|
489
|
+
- **One-command migration** between any two backends via `MigrationEngine`
|
|
498
490
|
|
|
499
|
-
|
|
500
|
-
|-------|-----------|---------|
|
|
501
|
-
| **API** | `src/api/` | Public-facing facade, input/output types, orchestrator |
|
|
502
|
-
| **Cognitive** | `src/cognitive_substrate/` | GMI instances, persona overlays, multi-layer memory |
|
|
503
|
-
| **Core** | `src/core/` | 28 subdirectories: LLM, tools, safety, guardrails, planning, workflows, HITL, observability, etc. |
|
|
504
|
-
| **Integration** | `src/channels/`, `src/voice/`, `src/extensions/` | External platform adapters, telephony, plugin system |
|
|
505
|
-
| **Intelligence** | `src/rag/`, `src/skills/` | RAG pipeline, GraphRAG, skill loading |
|
|
506
|
-
| **Infrastructure** | `src/config/`, `src/logging/`, `src/utils/`, `src/types/` | Configuration, structured logging, shared utilities |
|
|
491
|
+
See [`docs/memory/RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) and [`docs/memory/MULTIMODAL_RAG.md`](./docs/memory/MULTIMODAL_RAG.md).
|
|
507
492
|
|
|
508
|
-
|
|
493
|
+
### Adaptive Intelligence & Metacognition
|
|
509
494
|
|
|
510
|
-
|
|
495
|
+
Agents don't just respond — they monitor their own performance and adapt their behavior in real-time.
|
|
511
496
|
|
|
512
|
-
|
|
497
|
+
**MetapromptExecutor** rewrites the agent's own system prompt mid-conversation based on detected patterns:
|
|
513
498
|
|
|
514
|
-
|
|
499
|
+
| Trigger | What happens | Example |
|
|
500
|
+
|---------|-------------|---------|
|
|
501
|
+
| **Frustration recovery** | Detects user frustration via SentimentTracker → simplifies language, offers alternatives | User asks same question 3x → agent acknowledges confusion and tries a different approach |
|
|
502
|
+
| **Confusion clarification** | Detects ambiguous query → asks targeted follow-up | Vague request → agent probes for specifics before acting |
|
|
503
|
+
| **Satisfaction reinforcement** | Detects positive feedback → reinforces successful patterns | User says "perfect" → agent remembers what worked |
|
|
504
|
+
| **Engagement boost** | Detects disengagement → adjusts tone, offers proactive suggestions | Short replies → agent becomes more concise and action-oriented |
|
|
505
|
+
| **Error recovery** | Detects tool failures → adjusts strategy | API call fails → agent switches to alternative approach |
|
|
506
|
+
| **Trait adjustment** | Bounded HEXACO mutation → personality evolves within limits | Agent becomes slightly more conscientious after repeated accuracy requests |
|
|
515
507
|
|
|
516
|
-
|
|
508
|
+
Three trigger modes: `turn_interval` (periodic self-reflection), `event_based` (driven by sentiment events), and `manual` (flags in working memory).
|
|
517
509
|
|
|
518
|
-
|
|
519
|
-
|------|------|
|
|
520
|
-
| `AgentOS.ts` | Main service facade (1000+ lines). Implements `IAgentOS`. Initializes all subsystems and delegates to the orchestrator. |
|
|
521
|
-
| `AgentOSOrchestrator.ts` | Delegation hub. Coordinates GMI processing, tool execution, guardrail evaluation, and streaming assembly. Streaming-first design using async generators. |
|
|
522
|
-
| `interfaces/IAgentOS.ts` | Core contract defining `initialize()`, `processRequest()`, `handleToolResult()`, workflow management, persona listing, and conversation history retrieval. |
|
|
523
|
-
| `types/AgentOSInput.ts` | Unified input structure: text, vision, audio, persona selection, user API keys, feedback, workflow/agency invocations, and processing options. |
|
|
524
|
-
| `types/AgentOSResponse.ts` | Streaming output: 11 chunk types covering text deltas, tool calls, progress, errors, workflows, agency updates, and provenance events. |
|
|
510
|
+
**PromptProfileRouter** selects prompt strategies based on task classification — a code question gets a different prompt structure than a creative writing request.
|
|
525
511
|
|
|
526
|
-
**
|
|
512
|
+
**Self-improvement** (opt-in, bounded):
|
|
527
513
|
|
|
528
514
|
```typescript
|
|
529
|
-
|
|
530
|
-
|
|
531
|
-
|
|
532
|
-
|
|
533
|
-
|
|
534
|
-
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
538
|
-
|
|
539
|
-
|
|
540
|
-
|
|
541
|
-
|
|
515
|
+
const adaptive = agent({
|
|
516
|
+
provider: 'anthropic',
|
|
517
|
+
instructions: 'You are a research analyst.',
|
|
518
|
+
selfImprovement: {
|
|
519
|
+
enabled: true,
|
|
520
|
+
personality: {
|
|
521
|
+
maxDeltaPerSession: 0.15, // HEXACO traits can shift ±0.15 per session
|
|
522
|
+
decayToBaseline: true, // Drift back toward baseline during consolidation
|
|
523
|
+
},
|
|
524
|
+
skills: { enabled: true }, // Can enable/disable skills based on task
|
|
525
|
+
selfEvaluation: {
|
|
526
|
+
enabled: true, // LLM-based self-scoring after each turn
|
|
527
|
+
adjustParameters: true, // Auto-tune temperature/top-p based on scores
|
|
528
|
+
},
|
|
529
|
+
},
|
|
530
|
+
});
|
|
542
531
|
```
|
|
543
532
|
|
|
544
|
-
|
|
545
|
-
|
|
546
|
-
### Cognitive Substrate (GMI)
|
|
547
|
-
|
|
548
|
-
**Location:** `src/cognitive_substrate/`
|
|
549
|
-
|
|
550
|
-
The Generalized Modular Intelligence (GMI) is the core agent instance -- the "brain" of each agent. A single AgentOS runtime can manage multiple GMI instances via `GMIManager`.
|
|
551
|
-
|
|
552
|
-
```mermaid
|
|
553
|
-
graph TB
|
|
554
|
-
subgraph GMI["GMI Instance"]
|
|
555
|
-
PO["PersonaOverlay Manager<br/><i>Personality switching</i>"] ~~~ CTX["Context Manager<br/><i>Dynamic prompts · User context · Session</i>"]
|
|
556
|
-
WM["Working Memory<br/><i>Current session state · Tool context</i>"] ~~~ AM["Adaptation Manager<br/><i>Learning rate · Style drift · Tone</i>"]
|
|
557
|
-
subgraph Mem["Multi-Layer Memory"]
|
|
558
|
-
Ep["Episodic<br/><i>Events</i>"] ~~~ Sem["Semantic<br/><i>Facts</i>"] ~~~ Proc["Procedural<br/><i>Skills</i>"] ~~~ LT["Long-Term<br/><i>Archive</i>"]
|
|
559
|
-
end
|
|
560
|
-
PO --> Mem
|
|
561
|
-
WM --> Mem
|
|
562
|
-
end
|
|
563
|
-
style GMI fill:#0e0e18,stroke:#c9a227,stroke-width:2px,color:#f2f2fa
|
|
564
|
-
style Mem fill:#151520,stroke:#00f5ff,stroke-width:1px,color:#f2f2fa
|
|
565
|
-
```
|
|
533
|
+
See [`docs/architecture/ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md) for the full metacognition pipeline.
|
|
566
534
|
|
|
567
|
-
|
|
568
|
-
|
|
569
|
-
- **GMI.ts** -- Core agent instance (2000+ lines). Manages the complete cognitive loop: context assembly, prompt construction, LLM invocation, tool handling, and response generation.
|
|
570
|
-
- **GMIManager.ts** -- Lifecycle manager for multiple GMI instances. Handles creation, pooling, configuration, and teardown.
|
|
571
|
-
- **IGMI.ts** -- Interface contract for GMI implementations.
|
|
572
|
-
- **PersonaOverlayManager** -- Enables dynamic personality switching at runtime. Ships with 5+ built-in personas:
|
|
573
|
-
- `Researcher` -- Analytical, citation-heavy responses
|
|
574
|
-
- `Generalist` -- Balanced, conversational
|
|
575
|
-
- `Atlas` -- Navigation and spatial reasoning
|
|
576
|
-
- `Default Assistant` -- General-purpose helpful assistant
|
|
577
|
-
- **Memory subsystem** (`memory/`) -- 5 memory types, 4-tier hierarchy, 8 cognitive mechanisms:
|
|
578
|
-
|
|
579
|
-
**Memory types** (Tulving's long-term memory taxonomy):
|
|
580
|
-
| Type | Cognitive Model | Usage |
|
|
581
|
-
|------|----------------|-------|
|
|
582
|
-
| `episodic` | Autobiographical events | Conversation events, interactions |
|
|
583
|
-
| `semantic` | General knowledge/facts | Learned facts, preferences, schemas |
|
|
584
|
-
| `procedural` | Skills and how-to | Workflows, tool usage patterns |
|
|
585
|
-
| `prospective` | Future intentions | Goals, reminders, planned actions |
|
|
586
|
-
| `working` | Baddeley's model | Current session state, active context (7 +/- 2 slots) |
|
|
587
|
-
|
|
588
|
-
**4-tier directory hierarchy:**
|
|
589
|
-
- `core/` -- Types, config, encoding (Yerkes-Dodson, flashbulb, HEXACO-weighted), decay (Ebbinghaus curves + spaced repetition), working memory, prompt assembly
|
|
590
|
-
- `retrieval/` -- MemoryStore (6-signal composite scoring), memory graph (spreading activation), prospective memory, retrieval feedback
|
|
591
|
-
- `pipeline/` -- Consolidation (merge, prune, promote, schema extraction), observation (buffer, compress, reflect), lifecycle management, infinite context window
|
|
592
|
-
- `io/` -- Document ingestion (PDF, DOCX, HTML, Markdown, URL), import/export (JSON, Obsidian, SQLite, ChatGPT, CSV), facade API, memory tools, extensions
|
|
593
|
-
|
|
594
|
-
**Cognitive mechanisms** (`memory/mechanisms/`) -- 8 optional, all HEXACO personality-modulated:
|
|
595
|
-
| Mechanism | Citation | Effect |
|
|
596
|
-
|-----------|----------|--------|
|
|
597
|
-
| Reconsolidation | [Nader et al., 2000](https://doi.org/10.1038/35021052) | Retrieved memories drift toward current mood |
|
|
598
|
-
| Retrieval-Induced Forgetting | [Anderson et al., 1994](https://doi.org/10.1037/0278-7393.20.5.1063) | Retrieving one memory suppresses similar competitors |
|
|
599
|
-
| Involuntary Recall | [Berntsen, 2009](https://doi.org/10.1017/CBO9780511575921) | Random surfacing of old high-vividness memories |
|
|
600
|
-
| Metacognitive FOK | [Nelson & Narens, 1990](https://doi.org/10.1016/S0079-7421(08)60053-5) | Feeling-of-knowing scoring for tip-of-tongue states |
|
|
601
|
-
| Temporal Gist | [Reyna & Brainerd, 1995](https://doi.org/10.1016/1041-6080(95)90003-9) | Old traces compressed to core assertions |
|
|
602
|
-
| Schema Encoding | [Bartlett, 1932](https://doi.org/10.1017/CBO9780511759185) | Novel input boosted, schema-matching encoded efficiently |
|
|
603
|
-
| Source Confidence Decay | [Johnson et al., 1993](https://doi.org/10.1037/0033-2909.114.1.3) | Agent inferences decay faster than observations |
|
|
604
|
-
| Emotion Regulation | [Gross, 1998](https://doi.org/10.1037/1089-2680.2.3.271) | Reappraisal + suppression during consolidation |
|
|
605
|
-
|
|
606
|
-
See `docs/memory/COGNITIVE_MECHANISMS.md` for API reference and 30+ APA citations.
|
|
535
|
+
### Emergent Capabilities
|
|
607
536
|
|
|
608
|
-
|
|
537
|
+
Agents with `emergent: true` create new tools at runtime:
|
|
609
538
|
|
|
610
|
-
|
|
539
|
+
- **Runtime tool forging** via `forge_tool` — sandboxed JavaScript execution + LLM-as-judge safety evaluation
|
|
540
|
+
- **Dynamic skill management** via `manage_skills` — enable/disable skills based on task
|
|
541
|
+
- **Tiered promotion:** session (in-memory) → agent (persisted after 5+ uses with >0.8 confidence) → shared (HITL-approved)
|
|
542
|
+
- **Self-improving personality** — bounded HEXACO trait adaptation with Ebbinghaus decay
|
|
611
543
|
|
|
612
|
-
|
|
544
|
+
```typescript
|
|
545
|
+
const creative = agent({
|
|
546
|
+
provider: 'openai',
|
|
547
|
+
instructions: 'You solve problems creatively.',
|
|
548
|
+
emergent: {
|
|
549
|
+
enabled: true,
|
|
550
|
+
toolForging: true, // Can create new tools at runtime
|
|
551
|
+
maxForgedTools: 10, // Limit per session
|
|
552
|
+
promotionThreshold: 0.8, // Confidence required for permanent promotion
|
|
553
|
+
},
|
|
554
|
+
});
|
|
613
555
|
|
|
614
|
-
|
|
556
|
+
const session = creative.session('project');
|
|
557
|
+
await session.send('Parse this CSV and create a chart');
|
|
558
|
+
// Agent may forge a "csv_parser" tool if none exists,
|
|
559
|
+
// run it in a sandbox, and promote it if it works well.
|
|
615
560
|
|
|
616
|
-
|
|
617
|
-
|
|
618
|
-
PM["AIModelProviderManager<br/><i>Provider registry · Routing · Fallback · Model switching</i>"]
|
|
619
|
-
PM --> OAI["OpenAI Provider<br/><code>gpt-4o · gpt-4o-mini · o1</code>"]
|
|
620
|
-
PM --> Anth["Anthropic Provider<br/><code>claude-sonnet-4 · claude-haiku</code>"]
|
|
621
|
-
PM --> Gem["Gemini Provider<br/><code>gemini-2.5-flash · gemini-2.0</code>"]
|
|
622
|
-
PM --> Oll["Ollama Provider<br/><code>llama3.2 · mistral · codellama</code>"]
|
|
623
|
-
PM --> GQ["Groq Provider<br/><code>llama-3.3-70b · gemma2-9b</code>"]
|
|
624
|
-
PM --> OR["OpenRouter Provider<br/><code>200+ models · auto-routing</code>"]
|
|
625
|
-
PM --> Custom["Custom Provider<br/><i>IProvider interface</i>"]
|
|
626
|
-
style PM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa
|
|
561
|
+
console.log(session.forgedTools());
|
|
562
|
+
// [{ name: "csv_parser", forgedAt: "...", uses: 3, confidence: 0.92 }]
|
|
627
563
|
```
|
|
628
564
|
|
|
629
|
-
|
|
565
|
+
See [`docs/architecture/EMERGENT_CAPABILITIES.md`](./docs/architecture/EMERGENT_CAPABILITIES.md).
|
|
630
566
|
|
|
631
|
-
|
|
632
|
-
|----------|------|--------|---------|
|
|
633
|
-
| OpenAI | `providers/implementations/OpenAIProvider.ts` | GPT-4o, GPT-4o-mini, o1, o3 | `OPENAI_API_KEY` |
|
|
634
|
-
| Anthropic | `providers/implementations/AnthropicProvider.ts` | Claude Sonnet 4, Claude Haiku | `ANTHROPIC_API_KEY` |
|
|
635
|
-
| Gemini | `providers/implementations/GeminiProvider.ts` | Gemini 2.5 Flash, Gemini 2.0 | `GEMINI_API_KEY` |
|
|
636
|
-
| Groq | `providers/implementations/GroqProvider.ts` | Llama 3.3 70B, Gemma2 9B (fast inference) | `GROQ_API_KEY` |
|
|
637
|
-
| Mistral | `providers/implementations/MistralProvider.ts` | Mistral Large, Mistral Small | `MISTRAL_API_KEY` |
|
|
638
|
-
| Together | via OpenAI-compat | Meta-Llama 3.1 70B/8B | `TOGETHER_API_KEY` |
|
|
639
|
-
| xAI | via OpenAI-compat | Grok-2, Grok-2 Mini | `XAI_API_KEY` |
|
|
640
|
-
| Ollama | `providers/implementations/OllamaProvider.ts` | Any locally-hosted model | `OLLAMA_BASE_URL` |
|
|
641
|
-
| OpenRouter | `providers/implementations/OpenRouterProvider.ts` | 200+ models from any provider | `OPENROUTER_API_KEY` |
|
|
567
|
+
### Capability Discovery
|
|
642
568
|
|
|
643
|
-
|
|
569
|
+
3-tier semantic search that replaces static tool/skill dumps (~90% token reduction):
|
|
644
570
|
|
|
645
|
-
**
|
|
571
|
+
- **Tier 0:** Category summaries (~150 tokens, always included)
|
|
572
|
+
- **Tier 1:** Top-5 semantic matches (~200 tokens)
|
|
573
|
+
- **Tier 2:** Full schemas on demand (~1500 tokens)
|
|
646
574
|
|
|
647
|
-
|
|
648
|
-
- **IProvider** (`providers/IProvider.ts`) -- Interface contract for adding custom LLM providers.
|
|
649
|
-
- **Streaming adapters** -- All providers support token-level streaming via async generators with backpressure control.
|
|
575
|
+
The `discover_capabilities` meta-tool lets agents self-discover available tools, skills, and extensions at runtime.
|
|
650
576
|
|
|
651
|
-
|
|
577
|
+
---
|
|
652
578
|
|
|
653
|
-
|
|
654
|
-
// Provider-first: AgentOS picks the default model
|
|
655
|
-
await generateText({ provider: 'gemini', prompt: 'Summarize this document' });
|
|
579
|
+
## Architecture
|
|
656
580
|
|
|
657
|
-
// Override a specific model
|
|
658
|
-
await generateText({ provider: 'anthropic', model: 'claude-haiku-4-5-20251001', prompt: '...' });
|
|
659
581
|
```
|
|
660
|
-
|
|
661
|
-
|
|
662
|
-
|
|
663
|
-
|
|
664
|
-
|
|
665
|
-
|
|
666
|
-
|
|
667
|
-
|
|
668
|
-
|
|
669
|
-
|
|
670
|
-
|
|
671
|
-
|
|
672
|
-
|
|
582
|
+
+------------------------------------------------------------------+
|
|
583
|
+
| AgentOS Runtime |
|
|
584
|
+
| |
|
|
585
|
+
| +-----------+ +--------------+ +-----------+ |
|
|
586
|
+
| | API Layer |-->| Orchestrator |-->| Streaming | |
|
|
587
|
+
| +-----------+ +--------------+ +-----------+ |
|
|
588
|
+
| | | |
|
|
589
|
+
| +-----v----------------v-----+ |
|
|
590
|
+
| | GMI (Generalized | +------------------+ |
|
|
591
|
+
| | Modular Intelligence) |---->| Tool Orchestrator | |
|
|
592
|
+
| | | +------------------+ |
|
|
593
|
+
| | Working Memory Persona | +------------------+ |
|
|
594
|
+
| | Context Mgr Adaptation |---->| RAG Pipeline | |
|
|
595
|
+
| | Episodic Semantic Proc. | +------------------+ |
|
|
596
|
+
| +-----------------------------+ +------------------+ |
|
|
597
|
+
| | ---->| Planning Engine | |
|
|
598
|
+
| +-----v-----+ +------------------+ |
|
|
599
|
+
| | LLM Providers (16) | |
|
|
600
|
+
| | OpenAI Anthropic Gemini | +----------+ +----------+ |
|
|
601
|
+
| | Ollama Groq OpenRouter | | Guardrails| | Channels | |
|
|
602
|
+
| | Together Mistral xAI ... | | (6 packs) | | (37) | |
|
|
603
|
+
| +----------------------------+ +----------+ +----------+ |
|
|
604
|
+
+------------------------------------------------------------------+
|
|
673
605
|
```
|
|
674
606
|
|
|
607
|
+
For the full architecture with data flow diagrams, request lifecycle, and layer breakdown, see [`docs/architecture/ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md).
|
|
608
|
+
|
|
675
609
|
---
|
|
676
610
|
|
|
677
|
-
|
|
611
|
+
## Configuration
|
|
678
612
|
|
|
679
|
-
|
|
613
|
+
### Environment Variables
|
|
680
614
|
|
|
681
|
-
|
|
615
|
+
```bash
|
|
616
|
+
# LLM providers (set at least one)
|
|
617
|
+
OPENAI_API_KEY=sk-... # OpenAI (GPT-4o, GPT-4o-mini, o1, o3)
|
|
618
|
+
ANTHROPIC_API_KEY=sk-ant-... # Anthropic (Claude Sonnet 4, Claude Haiku)
|
|
619
|
+
GEMINI_API_KEY=AIza... # Google Gemini (2.5 Flash, 2.0)
|
|
620
|
+
OPENROUTER_API_KEY=sk-or-... # OpenRouter (200+ models, auto-routing)
|
|
621
|
+
GROQ_API_KEY=gsk_... # Groq (fast inference: Llama 3.3 70B)
|
|
622
|
+
TOGETHER_API_KEY=... # Together AI (Llama, Mixtral)
|
|
623
|
+
MISTRAL_API_KEY=... # Mistral (Mistral Large, Small)
|
|
624
|
+
XAI_API_KEY=xai-... # xAI (Grok-2)
|
|
625
|
+
OLLAMA_BASE_URL=http://localhost:11434 # Ollama (local models, no API key needed)
|
|
682
626
|
|
|
683
|
-
|
|
684
|
-
|
|
685
|
-
TO["ToolOrchestrator<br/><i>Discovery · Selection · Validation · Execution</i>"]
|
|
686
|
-
TO --> TE["ToolExecutor<br/><i>Validation · Invocation · Result wrap</i>"]
|
|
687
|
-
TO --> TP["ToolPermission Manager<br/><i>RBAC · Per-user access</i>"]
|
|
688
|
-
TO --> TG["ToolExecution Guard<br/><i>Timeout 30s · Circuit breaker</i>"]
|
|
689
|
-
style TO fill:#1c1c28,stroke:#00f5ff,color:#f2f2fa
|
|
690
|
-
```
|
|
627
|
+
# Database (optional, defaults to in-memory)
|
|
628
|
+
DATABASE_URL=file:./data/agentos.db
|
|
691
629
|
|
|
692
|
-
|
|
630
|
+
# Observability (optional)
|
|
631
|
+
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
|
|
632
|
+
OTEL_SERVICE_NAME=my-agent
|
|
693
633
|
|
|
694
|
-
|
|
695
|
-
|
|
696
|
-
|
|
697
|
-
readonly name: string; // LLM-facing function name
|
|
698
|
-
readonly displayName: string; // Human-readable title
|
|
699
|
-
readonly description: string; // Natural language description for LLM
|
|
700
|
-
readonly inputSchema: JSONSchemaObject; // JSON Schema for input validation
|
|
701
|
-
readonly outputSchema?: JSONSchemaObject; // JSON Schema for output validation
|
|
702
|
-
readonly category?: string; // Grouping (e.g., "data_analysis")
|
|
703
|
-
readonly hasSideEffects: boolean; // Whether the tool modifies external state
|
|
704
|
-
readonly requiredCapabilities?: string[]; // Persona capabilities needed
|
|
705
|
-
|
|
706
|
-
execute(
|
|
707
|
-
args: TInput,
|
|
708
|
-
context?: ToolExecutionContext
|
|
709
|
-
): Promise<ToolExecutionResult<TOutput>>;
|
|
710
|
-
}
|
|
634
|
+
# Voice/Telephony (optional)
|
|
635
|
+
TWILIO_ACCOUNT_SID=AC...
|
|
636
|
+
TWILIO_AUTH_TOKEN=...
|
|
711
637
|
```
|
|
712
638
|
|
|
713
|
-
|
|
639
|
+
### Multiple Providers
|
|
640
|
+
|
|
641
|
+
High-level helpers auto-detect from env vars -- no configuration object needed:
|
|
714
642
|
|
|
715
643
|
```typescript
|
|
716
|
-
|
|
717
|
-
success: boolean;
|
|
718
|
-
output?: TOutput;
|
|
719
|
-
error?: string;
|
|
720
|
-
contentType?: string; // MIME type (default: "application/json")
|
|
721
|
-
details?: Record<string, any>;
|
|
722
|
-
}
|
|
723
|
-
```
|
|
644
|
+
import { generateText } from '@framers/agentos';
|
|
724
645
|
|
|
725
|
-
|
|
646
|
+
// Each call picks the right credentials automatically
|
|
647
|
+
await generateText({ provider: 'anthropic', prompt: 'Hello from Claude' });
|
|
648
|
+
await generateText({ provider: 'openai', prompt: 'Hello from GPT' });
|
|
649
|
+
await generateText({ provider: 'gemini', prompt: 'Hello from Gemini' });
|
|
650
|
+
await generateText({ provider: 'ollama', prompt: 'Hello from Llama' });
|
|
726
651
|
|
|
727
|
-
|
|
652
|
+
// Omit provider -- auto-detects the first configured one
|
|
653
|
+
await generateText({ prompt: 'Hello from whichever provider is available' });
|
|
654
|
+
```
|
|
728
655
|
|
|
729
|
-
|
|
656
|
+
For the full runtime with explicit provider configuration, see [`docs/getting-started/HIGH_LEVEL_API.md`](./docs/getting-started/HIGH_LEVEL_API.md).
|
|
730
657
|
|
|
731
|
-
|
|
658
|
+
### Fallback Providers
|
|
732
659
|
|
|
733
|
-
|
|
660
|
+
```typescript
|
|
661
|
+
import { agent } from '@framers/agentos';
|
|
734
662
|
|
|
735
|
-
|
|
736
|
-
|
|
737
|
-
|
|
738
|
-
|
|
739
|
-
|
|
740
|
-
|
|
741
|
-
|
|
742
|
-
|
|
743
|
-
|
|
744
|
-
|
|
745
|
-
|
|
746
|
-
+-- Consumed by runtime (ToolOrchestrator,
|
|
747
|
-
GuardrailService, WorkflowEngine, etc.)
|
|
663
|
+
const resilient = agent({
|
|
664
|
+
provider: 'anthropic',
|
|
665
|
+
instructions: 'You are a helpful assistant.',
|
|
666
|
+
fallbackProviders: [
|
|
667
|
+
{ provider: 'openai' },
|
|
668
|
+
{ provider: 'groq' },
|
|
669
|
+
],
|
|
670
|
+
onFallback: (error, provider) => {
|
|
671
|
+
console.warn(`Falling back to ${provider}: ${error.message}`);
|
|
672
|
+
},
|
|
673
|
+
});
|
|
748
674
|
```
|
|
749
675
|
|
|
750
|
-
|
|
751
|
-
|
|
752
|
-
| Kind Constant | Value | Payload Type |
|
|
753
|
-
|---------------|-------|--------------|
|
|
754
|
-
| `EXTENSION_KIND_TOOL` | `"tool"` | `ITool` |
|
|
755
|
-
| `EXTENSION_KIND_GUARDRAIL` | `"guardrail"` | `IGuardrailService` |
|
|
756
|
-
| `EXTENSION_KIND_RESPONSE_PROCESSOR` | `"response-processor"` | Response transform function |
|
|
757
|
-
| `EXTENSION_KIND_WORKFLOW` | `"workflow"` | `WorkflowDescriptorPayload` |
|
|
758
|
-
| `EXTENSION_KIND_WORKFLOW_EXECUTOR` | `"workflow-executor"` | Workflow step executor |
|
|
759
|
-
| `EXTENSION_KIND_PERSONA` | `"persona"` | `IPersonaDefinition` |
|
|
760
|
-
| `EXTENSION_KIND_PLANNING_STRATEGY` | `"planning-strategy"` | Planning algorithm |
|
|
761
|
-
| `EXTENSION_KIND_HITL_HANDLER` | `"hitl-handler"` | Human interaction handler |
|
|
762
|
-
| `EXTENSION_KIND_COMM_CHANNEL` | `"communication-channel"` | Agent-to-agent channel |
|
|
763
|
-
| `EXTENSION_KIND_MEMORY_PROVIDER` | `"memory-provider"` | Memory backend |
|
|
764
|
-
| `EXTENSION_KIND_MESSAGING_CHANNEL` | `"messaging-channel"` | External platform adapter |
|
|
765
|
-
| `EXTENSION_KIND_PROVENANCE` | `"provenance"` | Audit/immutability handler |
|
|
766
|
-
|
|
767
|
-
**ExtensionDescriptor:**
|
|
768
|
-
|
|
769
|
-
```typescript
|
|
770
|
-
interface ExtensionDescriptor<TPayload = unknown> {
|
|
771
|
-
id: string; // Unique within its kind
|
|
772
|
-
kind: ExtensionKind; // One of the 12 kinds above
|
|
773
|
-
priority?: number; // Higher loads later (overrides earlier)
|
|
774
|
-
enableByDefault?: boolean; // Auto-enable on discovery
|
|
775
|
-
metadata?: Record<string, unknown>; // Arbitrary metadata
|
|
776
|
-
payload: TPayload; // The actual implementation
|
|
777
|
-
source?: ExtensionSourceMetadata; // Provenance (package name, version)
|
|
778
|
-
requiredSecrets?: ExtensionSecretRequirement[]; // API keys needed
|
|
779
|
-
onActivate?: (ctx: ExtensionLifecycleContext) => Promise<void> | void;
|
|
780
|
-
onDeactivate?: (ctx: ExtensionLifecycleContext) => Promise<void> | void;
|
|
781
|
-
}
|
|
782
|
-
```
|
|
676
|
+
---
|
|
783
677
|
|
|
784
|
-
|
|
678
|
+
## API Quick Reference
|
|
785
679
|
|
|
786
|
-
|
|
787
|
-
interface ExtensionManifest {
|
|
788
|
-
packs: ExtensionPackManifestEntry[]; // Pack references
|
|
789
|
-
overrides?: ExtensionOverrides; // Per-descriptor enable/disable/priority
|
|
790
|
-
}
|
|
680
|
+
### High-Level Functions
|
|
791
681
|
|
|
792
|
-
|
|
793
|
-
|
|
794
|
-
|
|
795
|
-
|
|
796
|
-
|
|
682
|
+
| Function | Description |
|
|
683
|
+
|----------|-------------|
|
|
684
|
+
| `generateText(opts)` | Single-call text generation with multi-step tool calling |
|
|
685
|
+
| `streamText(opts)` | Streaming text generation with async iterables |
|
|
686
|
+
| `generateObject(opts)` | Zod-validated structured output extraction |
|
|
687
|
+
| `streamObject(opts)` | Streaming structured output |
|
|
688
|
+
| `embedText(opts)` | Text embedding generation (single or batch) |
|
|
689
|
+
| `generateImage(opts)` | Image generation (OpenAI, Stability, Replicate, BFL, Fal) |
|
|
690
|
+
| `editImage(opts)` | Image editing/inpainting |
|
|
691
|
+
| `upscaleImage(opts)` | Image upscaling |
|
|
692
|
+
| `variateImage(opts)` | Image variations |
|
|
693
|
+
| `generateVideo(opts)` | Video generation |
|
|
694
|
+
| `analyzeVideo(opts)` | Video analysis and understanding |
|
|
695
|
+
| `detectScenes(opts)` | Scene detection in video |
|
|
696
|
+
| `generateMusic(opts)` | Music generation |
|
|
697
|
+
| `generateSFX(opts)` | Sound effect generation |
|
|
698
|
+
| `performOCR(opts)` | Text extraction from images (progressive tiers) |
|
|
699
|
+
| `agent(opts)` | Stateful agent with personality, memory, and sessions |
|
|
700
|
+
| `agency(opts)` | Multi-agent team with strategy-based coordination |
|
|
701
|
+
| `hitl(opts)` | Human-in-the-loop approval handler |
|
|
702
|
+
|
|
703
|
+
### Orchestration Builders
|
|
704
|
+
|
|
705
|
+
| Builder | Import Path | Description |
|
|
706
|
+
|---------|-------------|-------------|
|
|
707
|
+
| `workflow(name)` | `@framers/agentos/orchestration` | Deterministic DAG with typed steps |
|
|
708
|
+
| `AgentGraph` | `@framers/agentos/orchestration` | Explicit graph with cycles, subgraphs |
|
|
709
|
+
| `mission(name)` | `@framers/agentos/orchestration` | Goal-driven, planner decides steps |
|
|
710
|
+
|
|
711
|
+
### Core Types
|
|
712
|
+
|
|
713
|
+
```typescript
|
|
714
|
+
import type {
|
|
715
|
+
AgentOSInput, // Full runtime input structure
|
|
716
|
+
AgentOSResponse, // Streaming response chunk
|
|
717
|
+
ITool, // Tool interface (id, name, inputSchema, execute)
|
|
718
|
+
ToolExecutionResult, // Tool result (success, output, error)
|
|
719
|
+
AgentOptions, // agent() configuration
|
|
720
|
+
AgencyOptions, // agency() configuration
|
|
721
|
+
GenerateTextOptions, // generateText() / streamText() options
|
|
722
|
+
GenerateImageOptions, // generateImage() options
|
|
723
|
+
GenerateObjectOptions, // generateObject() options
|
|
724
|
+
EmbedTextOptions, // embedText() options
|
|
725
|
+
ExtensionDescriptor, // Extension pack descriptor
|
|
726
|
+
IGuardrailService, // Guardrail interface
|
|
727
|
+
IChannelAdapter, // Channel adapter interface
|
|
728
|
+
} from '@framers/agentos';
|
|
797
729
|
```
|
|
798
730
|
|
|
799
|
-
|
|
800
|
-
|
|
801
|
-
1. `ExtensionLoader` resolves pack entries from the manifest
|
|
802
|
-
2. `ExtensionRegistry` registers descriptors by kind, applying priority stacking
|
|
803
|
-
3. `ExtensionManager` provides runtime access: `getTools()`, `getGuardrails()`, `getWorkflows()`, etc.
|
|
804
|
-
4. `MultiRegistryLoader` supports loading from multiple remote registries
|
|
731
|
+
### Full Runtime
|
|
805
732
|
|
|
806
|
-
|
|
733
|
+
The `AgentOS` class provides the full-featured runtime with GMI management, extension loading, and streaming:
|
|
807
734
|
|
|
808
|
-
|
|
809
|
-
|
|
810
|
-
|
|
735
|
+
```typescript
|
|
736
|
+
import { AgentOS, AgentOSResponseChunkType } from '@framers/agentos';
|
|
737
|
+
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
811
738
|
|
|
812
|
-
|
|
739
|
+
const agent = new AgentOS();
|
|
740
|
+
await agent.initialize(await createTestAgentOSConfig());
|
|
813
741
|
|
|
814
|
-
|
|
815
|
-
|
|
816
|
-
|
|
817
|
-
|
|
818
|
-
|
|
819
|
-
|
|
820
|
-
|
|
821
|
-
|
|
742
|
+
for await (const chunk of agent.processRequest({
|
|
743
|
+
userId: 'user-1',
|
|
744
|
+
sessionId: 'session-1',
|
|
745
|
+
textInput: 'Explain how TCP handshakes work',
|
|
746
|
+
})) {
|
|
747
|
+
if (chunk.type === AgentOSResponseChunkType.TEXT_DELTA) {
|
|
748
|
+
process.stdout.write(chunk.textDelta);
|
|
749
|
+
}
|
|
750
|
+
}
|
|
751
|
+
```
|
|
822
752
|
|
|
823
753
|
---
|
|
824
754
|
|
|
825
|
-
|
|
826
|
-
|
|
827
|
-
**Location:** `src/orchestration/planner/`
|
|
755
|
+
## Package Exports
|
|
828
756
|
|
|
829
|
-
|
|
757
|
+
AgentOS provides 112 export paths for fine-grained imports:
|
|
830
758
|
|
|
831
|
-
```
|
|
832
|
-
|
|
833
|
-
|
|
834
|
-
PE --> IF["IPlanningEngine Interface"]
|
|
835
|
-
IF --- M1["createPlan(goal, context) → Plan"]
|
|
836
|
-
IF --- M2["executePlan(plan) → AsyncGenerator‹PlanStepResult›"]
|
|
837
|
-
IF --- M3["revisePlan(plan, feedback) → Plan"]
|
|
838
|
-
style PE fill:#1c1c28,stroke:#10b981,color:#f2f2fa
|
|
839
|
-
style IF fill:#151520,stroke:#c9a227,color:#f2f2fa
|
|
840
|
-
```
|
|
759
|
+
```typescript
|
|
760
|
+
// Main entry -- all public types and classes
|
|
761
|
+
import { AgentOS, generateText, streamText, agent, agency } from '@framers/agentos';
|
|
841
762
|
|
|
842
|
-
|
|
763
|
+
// Configuration
|
|
764
|
+
import { createAgentOSConfig, createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
843
765
|
|
|
844
|
-
|
|
845
|
-
|
|
846
|
-
- **HITL** for human approval of high-risk steps
|
|
847
|
-
- **Memory** for persisting plan state across sessions
|
|
848
|
-
|
|
849
|
-
See [`docs/orchestration/PLANNING_ENGINE.md`](./docs/orchestration/PLANNING_ENGINE.md) for the full planning system specification.
|
|
850
|
-
|
|
851
|
-
---
|
|
852
|
-
|
|
853
|
-
### Conversation Management
|
|
854
|
-
|
|
855
|
-
**Location:** `src/core/conversation/`
|
|
856
|
-
|
|
857
|
-
Manages session state, message history, and long-term memory persistence.
|
|
858
|
-
|
|
859
|
-
- **ConversationManager** -- Creates and retrieves conversation contexts, manages rolling message windows, and coordinates memory persistence.
|
|
860
|
-
- **ConversationContext** -- Immutable snapshot of a conversation: messages, metadata, active persona, user context.
|
|
861
|
-
- **IRollingSummaryMemorySink** -- Interface for persisting conversation summaries that compress long conversations into retrievable memory.
|
|
862
|
-
- **ILongTermMemoryRetriever** -- Interface for retrieving relevant past conversations during context assembly.
|
|
863
|
-
|
|
864
|
-
---
|
|
865
|
-
|
|
866
|
-
### RAG (Retrieval Augmented Generation)
|
|
867
|
-
|
|
868
|
-
**Location:** `src/rag/`
|
|
869
|
-
|
|
870
|
-
A complete RAG pipeline with pluggable vector stores, embedding management, and document ingestion.
|
|
871
|
-
|
|
872
|
-
```mermaid
|
|
873
|
-
graph TD
|
|
874
|
-
RA["RetrievalAugmentor<br/><i>Ingestion · Retrieval · Document management</i>"]
|
|
875
|
-
RA --> EM["EmbeddingManager<br/><i>Model selection · Caching · Batch</i>"]
|
|
876
|
-
RA --> VSM["VectorStoreManager<br/><i>Multi-provider vector storage</i>"]
|
|
877
|
-
EM --> LLM["LLM Provider<br/><i>Embedding models</i>"]
|
|
878
|
-
VSM --> Mem["InMemory<br/><i>dev</i>"]
|
|
879
|
-
VSM --> SQL["SQL<br/><i>prod</i>"]
|
|
880
|
-
VSM --> HNSW["HNSW<br/><i>local ANN</i>"]
|
|
881
|
-
VSM --> Qd["Qdrant<br/><i>cloud</i>"]
|
|
882
|
-
style RA fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
|
|
883
|
-
style VSM fill:#1c1c28,stroke:#8b5cf6,color:#f2f2fa
|
|
884
|
-
```
|
|
885
|
-
|
|
886
|
-
**Vector store implementations:**
|
|
887
|
-
|
|
888
|
-
| Store | Import | Use Case |
|
|
889
|
-
|-------|--------|----------|
|
|
890
|
-
| `InMemoryVectorStore` | `@framers/agentos/rag` | Development and testing |
|
|
891
|
-
| `SqlVectorStore` | `@framers/agentos/rag` | Production (SQLite/Postgres via `@framers/sql-storage-adapter`) |
|
|
892
|
-
| `HnswlibVectorStore` | `@framers/agentos/rag` | High-performance local ANN search |
|
|
893
|
-
| `QdrantVectorStore` | `@framers/agentos/rag` | Cloud-hosted vector database |
|
|
894
|
-
| `Neo4jVectorStore` | `@framers/agentos/rag` | Neo4j 5.x native vector indexes with shared connection pooling |
|
|
895
|
-
| `PostgresVectorStore` | `@framers/agentos/rag` | Postgres + pgvector with HNSW indexes and RRF hybrid search |
|
|
896
|
-
| `PineconeVectorStore` | `@framers/agentos/rag` | Pinecone cloud via fetch API with metadata filtering |
|
|
897
|
-
|
|
898
|
-
> **Database Persistence: [`@framers/sql-storage-adapter`](https://github.com/framersai/sql-storage-adapter)**
|
|
899
|
-
>
|
|
900
|
-
> All SQL-backed storage in AgentOS — including `SqlVectorStore`, conversation persistence, and memory archival — is powered by the `@framers/sql-storage-adapter` package. It provides a unified interface across 7 database backends with automatic runtime detection:
|
|
901
|
-
>
|
|
902
|
-
> | Adapter | Runtime | Use Case |
|
|
903
|
-
> |---------|---------|----------|
|
|
904
|
-
> | `better-sqlite3` | Node.js | Production (default) |
|
|
905
|
-
> | `pg` | Node.js | PostgreSQL for cloud deployments |
|
|
906
|
-
> | `sql.js` | Browser/WASM | Client-side storage |
|
|
907
|
-
> | `capacitor` | Mobile | iOS/Android via Capacitor |
|
|
908
|
-
> | `electron` | Desktop | Electron apps with IPC bridge |
|
|
909
|
-
> | `indexeddb` | Browser | Fallback browser storage |
|
|
910
|
-
> | `memory` | Any | Testing and development |
|
|
911
|
-
>
|
|
912
|
-
> ```typescript
|
|
913
|
-
> import { createDatabase } from '@framers/sql-storage-adapter';
|
|
914
|
-
> const db = await createDatabase(); // auto-detects best adapter
|
|
915
|
-
> ```
|
|
916
|
-
|
|
917
|
-
**GraphRAG:**
|
|
918
|
-
|
|
919
|
-
AgentOS supports both in-memory `GraphRAGEngine` and persistent `Neo4jGraphRAGEngine` (at `src/rag/graphrag/`) for knowledge-graph-enhanced retrieval:
|
|
920
|
-
|
|
921
|
-
- Entity and relationship extraction from documents
|
|
922
|
-
- Community detection via Louvain algorithm (requires `graphology` peer dependency)
|
|
923
|
-
- Local search (entity-centric) and global search (community-summarized)
|
|
924
|
-
- Hybrid retrieval combining vector similarity with graph traversal
|
|
925
|
-
|
|
926
|
-
```typescript
|
|
927
|
-
import { GraphRAGEngine } from '@framers/agentos/rag/graphrag';
|
|
928
|
-
import { Neo4jGraphRAGEngine } from '@framers/agentos/rag/graphrag';
|
|
929
|
-
import type { GraphRAGConfig, GraphEntity, GraphRelationship } from '@framers/agentos/rag/graphrag';
|
|
930
|
-
```
|
|
931
|
-
|
|
932
|
-
See [`docs/memory/RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) and [`docs/memory/MULTIMODAL_RAG.md`](./docs/memory/MULTIMODAL_RAG.md) for detailed configuration guides.
|
|
933
|
-
|
|
934
|
-
**Memory Scaling:**
|
|
935
|
-
|
|
936
|
-
AgentOS provides a 4-tier vector storage scaling path that grows with your deployment:
|
|
937
|
-
|
|
938
|
-
| Tier | Backend | When to Use |
|
|
939
|
-
|------|---------|-------------|
|
|
940
|
-
| 0 | SQLite (default) | Development, small datasets (< 1K vectors) |
|
|
941
|
-
| 1 | HNSW sidecar | Auto-activates at 1K vectors for local ANN search |
|
|
942
|
-
| 2 | Postgres + pgvector | Production cloud deployments with RRF hybrid search |
|
|
943
|
-
| 3 | Qdrant / Pinecone | High-scale managed vector databases |
|
|
944
|
-
|
|
945
|
-
- **One-command migration** via `MigrationEngine` — move data between any two backends with `migrate({ from, to })`
|
|
946
|
-
- **Docker auto-setup** — `MigrationEngine.ensureBackend('qdrant')` or `ensureBackend('postgres')` pulls and starts containers automatically
|
|
947
|
-
- **Binary blob embedding storage** — 3-4x faster than JSON serialization for high-throughput ingest
|
|
948
|
-
- **HNSW sidecar auto-activation** — transparently upgrades from brute-force to approximate nearest neighbors at 1K vectors
|
|
949
|
-
- **`embed()` config option** — opt-in query-time embedding generation for dynamic content
|
|
950
|
-
|
|
951
|
-
See [`docs/memory/MEMORY_SCALING.md`](./docs/memory/MEMORY_SCALING.md), [`docs/memory/POSTGRES_BACKEND.md`](./docs/memory/POSTGRES_BACKEND.md), [`docs/memory/QDRANT_BACKEND.md`](./docs/memory/QDRANT_BACKEND.md), and [`docs/memory/PINECONE_BACKEND.md`](./docs/memory/PINECONE_BACKEND.md) for backend-specific guides.
|
|
952
|
-
|
|
953
|
-
---
|
|
954
|
-
|
|
955
|
-
### Bundled Platform Knowledge
|
|
956
|
-
|
|
957
|
-
**Location:** `knowledge/platform-corpus.json`
|
|
958
|
-
|
|
959
|
-
AgentOS ships with **244 pre-built knowledge entries** covering the entire platform surface area. When the QueryRouter initializes, these entries are automatically loaded alongside your project-specific documentation corpus, giving every agent instant knowledge about AgentOS capabilities with zero configuration.
|
|
960
|
-
|
|
961
|
-
**Coverage breakdown:**
|
|
962
|
-
|
|
963
|
-
| Category | Count | What it covers |
|
|
964
|
-
|----------|-------|---------------|
|
|
965
|
-
| Tools | 105 | Every tool and channel adapter (Discord, Telegram, LinkedIn, Bluesky, etc.) |
|
|
966
|
-
| Skills | 80 | All curated skills from the skills registry |
|
|
967
|
-
| FAQ | 30 | Common questions (voice setup, supported models, streaming, OCR, etc.) |
|
|
968
|
-
| API | 14 | Core API functions (generateText, streamText, agent, agency, etc.) |
|
|
969
|
-
| Troubleshooting | 15 | Common errors and their fixes (missing API keys, model not found, etc.) |
|
|
970
|
-
|
|
971
|
-
**How it works:**
|
|
972
|
-
|
|
973
|
-
- During `router.init()`, the QueryRouter loads `knowledge/platform-corpus.json` from the package directory
|
|
974
|
-
- Platform entries are merged into the same corpus as user docs, making them searchable via both vector and keyword retrieval
|
|
975
|
-
- The keyword fallback index covers platform entries, so they work even without an embedding API key
|
|
976
|
-
|
|
977
|
-
**Disabling platform knowledge:**
|
|
978
|
-
|
|
979
|
-
```typescript
|
|
980
|
-
const router = new QueryRouter({
|
|
981
|
-
knowledgeCorpus: ['./docs'],
|
|
982
|
-
includePlatformKnowledge: false, // Skip bundled platform entries
|
|
983
|
-
});
|
|
984
|
-
```
|
|
985
|
-
|
|
986
|
-
When disabled, the router only loads your `knowledgeCorpus` directories. This is useful when you want a purely project-specific knowledge base or are building a non-AgentOS product.
|
|
987
|
-
|
|
988
|
-
---
|
|
989
|
-
|
|
990
|
-
### Safety and Guardrails
|
|
991
|
-
|
|
992
|
-
**Location:** `src/safety/runtime/` and `src/safety/guardrails/`
|
|
993
|
-
|
|
994
|
-
AgentOS provides defense-in-depth safety through two complementary systems.
|
|
995
|
-
|
|
996
|
-
#### Safety Primitives (`safety/runtime/`)
|
|
997
|
-
|
|
998
|
-
Five runtime safety components:
|
|
999
|
-
|
|
1000
|
-
| Component | Export | Purpose |
|
|
1001
|
-
|-----------|--------|---------|
|
|
1002
|
-
| **CircuitBreaker** | `CircuitBreaker` | Three-state (closed/open/half-open) wrapper for LLM calls. Configurable failure threshold, reset timeout, and half-open probe count. Throws `CircuitOpenError` when tripped. |
|
|
1003
|
-
| **ActionDeduplicator** | `ActionDeduplicator` | Hash-based recent action tracking with LRU eviction. Prevents redundant tool calls and repeated operations within a configurable time window. |
|
|
1004
|
-
| **StuckDetector** | `StuckDetector` | Detects three patterns: repeated outputs, repeated errors, and oscillation (A-B-A-B cycles). Returns `StuckDetection` with reason and confidence. |
|
|
1005
|
-
| **CostGuard** | `CostGuard` | Per-agent session and daily spending caps. Defaults: $1/session, $5/day. Throws `CostCapExceededError` when limits are hit. Tracks token usage across all LLM calls. |
|
|
1006
|
-
| **ToolExecutionGuard** | `ToolExecutionGuard` | Per-tool timeout (30s default) with independent circuit breakers per tool ID. Reports `ToolHealthReport` for monitoring. Throws `ToolTimeoutError`. |
|
|
1007
|
-
|
|
1008
|
-
```typescript
|
|
1009
|
-
import {
|
|
1010
|
-
CircuitBreaker,
|
|
1011
|
-
CostGuard,
|
|
1012
|
-
StuckDetector,
|
|
1013
|
-
ActionDeduplicator,
|
|
1014
|
-
ToolExecutionGuard,
|
|
1015
|
-
} from '@framers/agentos/safety/runtime';
|
|
1016
|
-
```
|
|
1017
|
-
|
|
1018
|
-
See [`docs/safety/SAFETY_PRIMITIVES.md`](./docs/safety/SAFETY_PRIMITIVES.md) for the full safety API reference.
|
|
1019
|
-
|
|
1020
|
-
#### Guardrails (`safety/guardrails/`)
|
|
1021
|
-
|
|
1022
|
-
Content-level input/output filtering:
|
|
1023
|
-
|
|
1024
|
-
```typescript
|
|
1025
|
-
interface IGuardrailService {
|
|
1026
|
-
config?: {
|
|
1027
|
-
evaluateStreamingChunks?: boolean;
|
|
1028
|
-
maxStreamingEvaluations?: number;
|
|
1029
|
-
canSanitize?: boolean;
|
|
1030
|
-
timeoutMs?: number;
|
|
1031
|
-
};
|
|
1032
|
-
|
|
1033
|
-
evaluateInput?(payload: GuardrailInputPayload): Promise<GuardrailEvaluationResult | null>;
|
|
1034
|
-
evaluateOutput?(payload: GuardrailOutputPayload): Promise<GuardrailEvaluationResult | null>;
|
|
1035
|
-
}
|
|
1036
|
-
```
|
|
1037
|
-
|
|
1038
|
-
`GuardrailOutputPayload` also carries `ragSources?: RagRetrievedChunk[]`, which enables grounding-aware output checks against retrieved context.
|
|
1039
|
-
|
|
1040
|
-
Guardrails run at two points in the request lifecycle:
|
|
1041
|
-
1. **Pre-processing** -- `evaluateInput()` inspects user input before orchestration
|
|
1042
|
-
2. **Post-processing** -- `evaluateOutput()` inspects streaming chunks and/or the final response before emission
|
|
1043
|
-
|
|
1044
|
-
When multiple guardrails are registered, AgentOS uses a **two-phase dispatcher**:
|
|
1045
|
-
1. **Phase 1 (sequential sanitizers)** -- guardrails with `config.canSanitize === true` run in registration order so each sanitizer sees the cumulative sanitized text
|
|
1046
|
-
2. **Phase 2 (parallel classifiers)** -- all remaining guardrails run concurrently with worst-action aggregation (`BLOCK > FLAG > ALLOW`)
|
|
1047
|
-
|
|
1048
|
-
`ParallelGuardrailDispatcher` powers both input evaluation and output stream wrapping, with per-guardrail `timeoutMs` fail-open behavior for slow or degraded classifiers.
|
|
1049
|
-
|
|
1050
|
-
Multiple guardrails can be composed via the extension system, and each receives full context (user ID, session ID, persona ID, conversation ID, metadata) for context-aware policy decisions.
|
|
1051
|
-
|
|
1052
|
-
See [`docs/safety/GUARDRAILS_USAGE.md`](./docs/safety/GUARDRAILS_USAGE.md) for implementation patterns.
|
|
1053
|
-
|
|
1054
|
-
---
|
|
1055
|
-
|
|
1056
|
-
### Human-in-the-Loop (HITL)
|
|
1057
|
-
|
|
1058
|
-
**Location:** `src/orchestration/hitl/`
|
|
1059
|
-
|
|
1060
|
-
The HITL system enables agents to request human approval, clarification, and collaboration at key decision points.
|
|
1061
|
-
|
|
1062
|
-
**Core interface: `IHumanInteractionManager`**
|
|
1063
|
-
|
|
1064
|
-
Three interaction modes:
|
|
1065
|
-
|
|
1066
|
-
| Mode | Method | Use Case |
|
|
1067
|
-
|------|--------|----------|
|
|
1068
|
-
| **Approval** | `requestApproval()` | Gate high-risk actions (database mutations, financial operations, external communications) |
|
|
1069
|
-
| **Clarification** | `requestClarification()` | Ask the user for missing information before proceeding |
|
|
1070
|
-
| **Escalation** | `escalateToHuman()` | Hand off to a human operator when the agent cannot proceed |
|
|
1071
|
-
|
|
1072
|
-
**PendingAction structure:**
|
|
1073
|
-
|
|
1074
|
-
```typescript
|
|
1075
|
-
interface PendingAction {
|
|
1076
|
-
actionId: string;
|
|
1077
|
-
description: string;
|
|
1078
|
-
severity: 'low' | 'medium' | 'high' | 'critical';
|
|
1079
|
-
category?: 'data_modification' | 'external_api' | 'financial' |
|
|
1080
|
-
'communication' | 'system' | 'other';
|
|
1081
|
-
agentId: string;
|
|
1082
|
-
context: Record<string, unknown>;
|
|
1083
|
-
potentialConsequences?: string[];
|
|
1084
|
-
reversible: boolean;
|
|
1085
|
-
estimatedCost?: { amount: number; currency: string };
|
|
1086
|
-
timeoutMs?: number;
|
|
1087
|
-
alternatives?: AlternativeAction[];
|
|
1088
|
-
}
|
|
1089
|
-
```
|
|
1090
|
-
|
|
1091
|
-
HITL integrates with the planning engine so individual plan steps can require approval, and with the extension system via `EXTENSION_KIND_HITL_HANDLER` for custom approval UIs.
|
|
1092
|
-
|
|
1093
|
-
See [`docs/safety/HUMAN_IN_THE_LOOP.md`](./docs/safety/HUMAN_IN_THE_LOOP.md) for the full HITL specification.
|
|
1094
|
-
|
|
1095
|
-
---
|
|
1096
|
-
|
|
1097
|
-
### Channels System
|
|
1098
|
-
|
|
1099
|
-
**Location:** `src/channels/`
|
|
1100
|
-
|
|
1101
|
-
Unified adapters for 40 external messaging and social platforms.
|
|
1102
|
-
|
|
1103
|
-
**40 supported platforms:**
|
|
1104
|
-
|
|
1105
|
-
| Priority | Platforms |
|
|
1106
|
-
|----------|-----------|
|
|
1107
|
-
| **P0** (Core + Social) | Telegram, WhatsApp, Discord, Slack, Webchat, Twitter/X, Instagram, Reddit, YouTube, LinkedIn, Facebook, Threads, Bluesky |
|
|
1108
|
-
| **P1** | Signal, iMessage, Google Chat, Microsoft Teams, Pinterest, TikTok, Mastodon, Dev.to, Hashnode, Medium, WordPress |
|
|
1109
|
-
| **P2** | Matrix, Zalo, Email, SMS, Farcaster, Lemmy, Google Business |
|
|
1110
|
-
| **P3** | Nostr, Twitch, Line, Feishu, Mattermost, Nextcloud Talk, Tlon, IRC, Zalo Personal |
|
|
1111
|
-
|
|
1112
|
-
**29 capability flags:**
|
|
1113
|
-
|
|
1114
|
-
Each adapter declares its capabilities, allowing consumers to check before attempting unsupported actions:
|
|
1115
|
-
|
|
1116
|
-
```
|
|
1117
|
-
text, rich_text, images, video, audio, voice_notes, documents,
|
|
1118
|
-
stickers, reactions, threads, typing_indicator, read_receipts,
|
|
1119
|
-
group_chat, channels, buttons, inline_keyboard, embeds,
|
|
1120
|
-
mentions, editing, deletion
|
|
1121
|
-
stories, reels, hashtags, polls, carousel,
|
|
1122
|
-
engagement_metrics, scheduling, dm_automation, content_discovery
|
|
1123
|
-
```
|
|
1124
|
-
|
|
1125
|
-
**IChannelAdapter** -- Unified interface for bidirectional messaging:
|
|
1126
|
-
|
|
1127
|
-
- `connect()` / `disconnect()` -- Lifecycle management
|
|
1128
|
-
- `sendMessage()` -- Outbound messages with platform-specific formatting
|
|
1129
|
-
- `onMessage()` -- Inbound message handler registration
|
|
1130
|
-
- `getConnectionInfo()` -- Connection health monitoring
|
|
1131
|
-
- `capabilities` -- Declared capability set
|
|
1132
|
-
|
|
1133
|
-
**ChannelRouter** -- Routes inbound messages to the appropriate agent and outbound responses to the correct platform adapter. Supports multi-platform agents (one agent, many channels).
|
|
1134
|
-
|
|
1135
|
-
**Connection status:** `disconnected` -> `connecting` -> `connected` -> `reconnecting` -> `error`
|
|
1136
|
-
|
|
1137
|
-
Channel adapters are registered as extensions via `EXTENSION_KIND_MESSAGING_CHANNEL`.
|
|
1138
|
-
|
|
1139
|
-
See [`docs/architecture/PLATFORM_SUPPORT.md`](./docs/architecture/PLATFORM_SUPPORT.md) for platform-specific configuration.
|
|
1140
|
-
|
|
1141
|
-
---
|
|
1142
|
-
|
|
1143
|
-
### Voice and Telephony
|
|
1144
|
-
|
|
1145
|
-
**Location:** `src/voice/`
|
|
1146
|
-
|
|
1147
|
-
Enable agents to make and receive phone calls via telephony providers.
|
|
1148
|
-
|
|
1149
|
-
**Call state machine:**
|
|
1150
|
-
|
|
1151
|
-
```
|
|
1152
|
-
initiated --> ringing --> answered --> active --> speaking <--> listening
|
|
1153
|
-
| | | | | |
|
|
1154
|
-
+------------+-----------+----------+------------+------------+
|
|
1155
|
-
| (any non-terminal state) |
|
|
1156
|
-
v
|
|
1157
|
-
[Terminal States]
|
|
1158
|
-
completed | hangup-user | hangup-bot | timeout | error |
|
|
1159
|
-
failed | no-answer | busy | voicemail
|
|
1160
|
-
```
|
|
1161
|
-
|
|
1162
|
-
**Providers:**
|
|
1163
|
-
|
|
1164
|
-
| Provider | Support |
|
|
1165
|
-
|----------|---------|
|
|
1166
|
-
| Twilio | Full voice + SMS |
|
|
1167
|
-
| Telnyx | Full voice |
|
|
1168
|
-
| Plivo | Full voice |
|
|
1169
|
-
| Mock | Testing/development |
|
|
1170
|
-
|
|
1171
|
-
**Key components:**
|
|
1172
|
-
|
|
1173
|
-
- **CallManager** -- Manages call lifecycle, state transitions, and event dispatch
|
|
1174
|
-
- **IVoiceCallProvider** -- Interface for telephony provider adapters
|
|
1175
|
-
- **telephony-audio.ts** -- Audio stream handling and format conversion
|
|
1176
|
-
|
|
1177
|
-
Voice providers are registered via `EXTENSION_KIND_TOOL` with the `voice-call-provider` category.
|
|
1178
|
-
|
|
1179
|
-
---
|
|
1180
|
-
|
|
1181
|
-
### Unified Orchestration Layer
|
|
1182
|
-
|
|
1183
|
-
**Location:** `src/orchestration/`
|
|
1184
|
-
|
|
1185
|
-
Three authoring APIs compile to one `CompiledExecutionGraph` IR executed by a single `GraphRuntime`. Persistent checkpointing enables time-travel debugging and fault recovery.
|
|
1186
|
-
|
|
1187
|
-
Current status: the builders, IR, checkpoints, and base runtime are real. Some advanced routes are still partial in the shared runtime: discovery edges need discovery wiring, personality edges need a trait source, and `extension` / `subgraph` execution still requires a higher-level bridge runtime.
|
|
1188
|
-
|
|
1189
|
-
| API | Level | Use case |
|
|
1190
|
-
|-----|-------|----------|
|
|
1191
|
-
| **`AgentGraph`** | Low-level | Explicit nodes, edges, cycles, subgraphs — full graph control |
|
|
1192
|
-
| **`workflow()`** | Mid-level | Deterministic DAG chains with step/branch/parallel — Zod-typed I/O |
|
|
1193
|
-
| **`mission()`** | High-level | Intent-driven — declare goal + constraints, PlanningEngine decides steps |
|
|
1194
|
-
|
|
1195
|
-
**Differentiators vs LangGraph / Mastra:** memory-aware state, capability discovery routing, personality-driven edges, inter-step guardrails, streaming at every node transition.
|
|
1196
|
-
|
|
1197
|
-
```typescript
|
|
1198
|
-
import { AgentGraph, toolNode, gmiNode, START, END } from '@framers/agentos/orchestration';
|
|
1199
|
-
import { z } from 'zod';
|
|
1200
|
-
|
|
1201
|
-
// Low-level: explicit graph with checkpoints
|
|
1202
|
-
const graph = new AgentGraph({
|
|
1203
|
-
input: z.object({ topic: z.string() }),
|
|
1204
|
-
scratch: z.object({ draft: z.string().optional() }),
|
|
1205
|
-
artifacts: z.object({ status: z.string().optional(), summary: z.string().optional() }),
|
|
1206
|
-
})
|
|
1207
|
-
.addNode('draft', gmiNode({ instructions: 'Draft a concise summary', executionMode: 'single_turn' }))
|
|
1208
|
-
.addNode('publish', toolNode('publish_report'))
|
|
1209
|
-
.addEdge(START, 'draft')
|
|
1210
|
-
.addEdge('draft', 'publish')
|
|
1211
|
-
.addEdge('publish', END)
|
|
1212
|
-
.compile();
|
|
1213
|
-
|
|
1214
|
-
const result = await graph.invoke({ topic: 'quantum computing' });
|
|
1215
|
-
|
|
1216
|
-
// Mid-level: deterministic workflow
|
|
1217
|
-
import { workflow } from '@framers/agentos/orchestration';
|
|
1218
|
-
|
|
1219
|
-
const flow = workflow('onboarding')
|
|
1220
|
-
.input(z.object({ email: z.string() }))
|
|
1221
|
-
.returns(z.object({ userId: z.string() }))
|
|
1222
|
-
.step('validate', { tool: 'email_validator' })
|
|
1223
|
-
.then('create', { tool: 'user_service' })
|
|
1224
|
-
.compile();
|
|
1225
|
-
|
|
1226
|
-
// High-level: intent-driven
|
|
1227
|
-
import { mission } from '@framers/agentos/orchestration';
|
|
1228
|
-
|
|
1229
|
-
const researcher = mission('research')
|
|
1230
|
-
.input(z.object({ topic: z.string() }))
|
|
1231
|
-
.goal('Research {topic} and produce a cited summary')
|
|
1232
|
-
.returns(z.object({ summary: z.string() }))
|
|
1233
|
-
.planner({ strategy: 'plan_and_execute', maxSteps: 8 })
|
|
1234
|
-
.compile();
|
|
1235
|
-
|
|
1236
|
-
const plan = await researcher.explain({ topic: 'AI safety' }); // preview plan without executing
|
|
1237
|
-
```
|
|
1238
|
-
|
|
1239
|
-
See [`docs/orchestration/UNIFIED_ORCHESTRATION.md`](./docs/orchestration/UNIFIED_ORCHESTRATION.md), [`docs/architecture/AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md), [`docs/orchestration/WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md), [`docs/orchestration/MISSION_API.md`](./docs/orchestration/MISSION_API.md), [`docs/orchestration/CHECKPOINTING.md`](./docs/orchestration/CHECKPOINTING.md).
|
|
1240
|
-
|
|
1241
|
-
Runnable examples: [`examples/agent-graph.mjs`](./examples/agent-graph.mjs), [`examples/workflow-dsl.mjs`](./examples/workflow-dsl.mjs), [`examples/mission-api.mjs`](./examples/mission-api.mjs)
|
|
1242
|
-
|
|
1243
|
-
#### Legacy WorkflowEngine
|
|
1244
|
-
|
|
1245
|
-
The original `WorkflowEngine` (`src/orchestration/workflows/`) continues to work for existing consumers. The new orchestration layer is opt-in and runs alongside it.
|
|
1246
|
-
|
|
1247
|
-
```typescript
|
|
1248
|
-
const definitions = agent.listWorkflowDefinitions();
|
|
1249
|
-
const instance = await agent.startWorkflow('data-pipeline-v1', input);
|
|
1250
|
-
const status = await agent.getWorkflow(instance.workflowId);
|
|
1251
|
-
```
|
|
1252
|
-
|
|
1253
|
-
---
|
|
1254
|
-
|
|
1255
|
-
### Multi-Agent Coordination
|
|
1256
|
-
|
|
1257
|
-
**Location:** `src/agents/agency/`
|
|
1258
|
-
|
|
1259
|
-
Enables teams of agents to collaborate on shared goals.
|
|
1260
|
-
|
|
1261
|
-
- **AgencyRegistry** -- Register and manage agent teams with role assignments
|
|
1262
|
-
- **AgentCommunicationBus** -- Inter-agent message passing with typed events and handoffs
|
|
1263
|
-
- **AgencyMemoryManager** -- Shared memory space with vector search for agency-wide knowledge
|
|
1264
|
-
|
|
1265
|
-
```mermaid
|
|
1266
|
-
graph TD
|
|
1267
|
-
AR["AgencyRegistry"]
|
|
1268
|
-
AR --> A1["Agent (GMI)<br/><b>Researcher</b>"]
|
|
1269
|
-
AR --> A2["Agent (GMI)<br/><b>Writer</b>"]
|
|
1270
|
-
AR --> A3["Agent (GMI)<br/><b>Reviewer</b>"]
|
|
1271
|
-
AR --> A4["Agent (GMI)<br/><b>Deployer</b>"]
|
|
1272
|
-
A1 & A2 & A3 & A4 --> Bus["Communication Bus<br/><i>Events · Handoffs · Sync</i>"]
|
|
1273
|
-
A1 & A2 & A3 & A4 --> Mem["Agency Memory Manager<br/><i>Shared vector memory</i>"]
|
|
1274
|
-
style AR fill:#1c1c28,stroke:#c9a227,color:#f2f2fa
|
|
1275
|
-
style Bus fill:#151520,stroke:#00f5ff,color:#f2f2fa
|
|
1276
|
-
style Mem fill:#151520,stroke:#8b5cf6,color:#f2f2fa
|
|
1277
|
-
```
|
|
1278
|
-
|
|
1279
|
-
See [`docs/architecture/AGENT_COMMUNICATION.md`](./docs/architecture/AGENT_COMMUNICATION.md) for the full multi-agent specification.
|
|
1280
|
-
|
|
1281
|
-
---
|
|
1282
|
-
|
|
1283
|
-
### Observability
|
|
1284
|
-
|
|
1285
|
-
**Location:** `src/evaluation/observability/`
|
|
1286
|
-
|
|
1287
|
-
OpenTelemetry-native observability for tracing, metrics, and cost tracking.
|
|
1288
|
-
|
|
1289
|
-
- **ITracer** / **Tracer** -- Span creation and propagation for distributed tracing
|
|
1290
|
-
- **otel.ts** -- `configureAgentOSObservability()` sets up the OpenTelemetry SDK with custom exporters
|
|
1291
|
-
- **Metrics** -- Token usage, latency percentiles, tool execution counts, error rates
|
|
1292
|
-
- **Cost tracking** -- Per-request and aggregate cost computation across providers
|
|
1293
|
-
|
|
1294
|
-
```typescript
|
|
1295
|
-
import { configureAgentOSObservability } from '@framers/agentos';
|
|
1296
|
-
|
|
1297
|
-
configureAgentOSObservability({
|
|
1298
|
-
serviceName: 'my-agent',
|
|
1299
|
-
traceExporter: myOTLPExporter,
|
|
1300
|
-
metricExporter: myMetricsExporter,
|
|
1301
|
-
});
|
|
1302
|
-
```
|
|
1303
|
-
|
|
1304
|
-
See [`docs/observability/OBSERVABILITY.md`](./docs/observability/OBSERVABILITY.md) and [`docs/safety/COST_OPTIMIZATION.md`](./docs/safety/COST_OPTIMIZATION.md) for setup guides.
|
|
1305
|
-
|
|
1306
|
-
---
|
|
1307
|
-
|
|
1308
|
-
### Skills
|
|
1309
|
-
|
|
1310
|
-
**Location:** `src/skills/`
|
|
1311
|
-
|
|
1312
|
-
Skills are portable, self-describing agent capabilities defined in `SKILL.md` files.
|
|
1313
|
-
|
|
1314
|
-
- **SkillRegistry** -- Discovers and registers available skills
|
|
1315
|
-
- **SkillLoader** -- Parses SKILL.md format (YAML frontmatter + markdown body)
|
|
1316
|
-
- **SKILL.md format** -- Declarative skill definition with name, description, required tools, and behavioral instructions
|
|
1317
|
-
|
|
1318
|
-
See [`docs/extensions/SKILLS.md`](./docs/extensions/SKILLS.md) for the skill authoring guide.
|
|
1319
|
-
|
|
1320
|
-
---
|
|
1321
|
-
|
|
1322
|
-
### Structured Output
|
|
1323
|
-
|
|
1324
|
-
**Location:** `src/structured/output/`
|
|
1325
|
-
|
|
1326
|
-
Extract typed, validated data from unstructured text using JSON Schema.
|
|
1327
|
-
|
|
1328
|
-
- **StructuredOutputManager** -- Coordinates schema-constrained generation with validation
|
|
1329
|
-
- **JSON Schema validation** -- Input/output validation via ajv with format support
|
|
1330
|
-
- **Parallel function calls** -- Multiple tool invocations in a single LLM turn
|
|
1331
|
-
- **Entity extraction** -- Named entity recognition with schema constraints
|
|
1332
|
-
|
|
1333
|
-
See [`docs/orchestration/STRUCTURED_OUTPUT.md`](./docs/orchestration/STRUCTURED_OUTPUT.md) for usage patterns.
|
|
1334
|
-
|
|
1335
|
-
---
|
|
1336
|
-
|
|
1337
|
-
### Emergent Capabilities
|
|
1338
|
-
|
|
1339
|
-
**Location:** `src/emergent/`
|
|
1340
|
-
|
|
1341
|
-
Agents with `emergent: true` create new tools at runtime — compose existing tools via a step DSL or write sandboxed JavaScript. An LLM-as-judge evaluates safety and correctness. Tools earn trust through tiered promotion: session (in-memory) → agent (persisted, auto-promoted after 5+ uses with >0.8 confidence) → shared (human-approved HITL gate).
|
|
1342
|
-
|
|
1343
|
-
See [`docs/architecture/EMERGENT_CAPABILITIES.md`](./docs/architecture/EMERGENT_CAPABILITIES.md) for details.
|
|
1344
|
-
|
|
1345
|
-
---
|
|
1346
|
-
|
|
1347
|
-
## Configuration
|
|
1348
|
-
|
|
1349
|
-
### Development (Quick Start)
|
|
1350
|
-
|
|
1351
|
-
`createTestAgentOSConfig()` provides sensible defaults for local development:
|
|
1352
|
-
|
|
1353
|
-
```typescript
|
|
1354
|
-
import { AgentOS } from '@framers/agentos';
|
|
1355
|
-
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
1356
|
-
|
|
1357
|
-
const agent = new AgentOS();
|
|
1358
|
-
await agent.initialize(await createTestAgentOSConfig());
|
|
1359
|
-
```
|
|
1360
|
-
|
|
1361
|
-
### Production
|
|
1362
|
-
|
|
1363
|
-
`createAgentOSConfig()` reads from environment variables:
|
|
1364
|
-
|
|
1365
|
-
```typescript
|
|
1366
|
-
import { AgentOS } from '@framers/agentos';
|
|
1367
|
-
import { createAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
1368
|
-
|
|
1369
|
-
const agent = new AgentOS();
|
|
1370
|
-
await agent.initialize(await createAgentOSConfig());
|
|
1371
|
-
```
|
|
1372
|
-
|
|
1373
|
-
### Multiple Providers
|
|
1374
|
-
|
|
1375
|
-
**High-level helpers** (recommended): Just set env vars. No config object needed.
|
|
1376
|
-
|
|
1377
|
-
```bash
|
|
1378
|
-
# Set as many provider keys as you like -- AgentOS uses them on demand
|
|
1379
|
-
export ANTHROPIC_API_KEY=sk-ant-...
|
|
1380
|
-
export OPENAI_API_KEY=sk-...
|
|
1381
|
-
export GEMINI_API_KEY=AIza...
|
|
1382
|
-
export OLLAMA_BASE_URL=http://localhost:11434
|
|
1383
|
-
```
|
|
1384
|
-
|
|
1385
|
-
```typescript
|
|
1386
|
-
import { generateText } from '@framers/agentos';
|
|
1387
|
-
|
|
1388
|
-
// Each call picks the right credentials from env vars automatically
|
|
1389
|
-
await generateText({ provider: 'anthropic', prompt: 'Hello from Claude' });
|
|
1390
|
-
await generateText({ provider: 'openai', prompt: 'Hello from GPT' });
|
|
1391
|
-
await generateText({ provider: 'gemini', prompt: 'Hello from Gemini' });
|
|
1392
|
-
await generateText({ provider: 'ollama', prompt: 'Hello from Llama' });
|
|
1393
|
-
|
|
1394
|
-
// Or omit provider entirely -- auto-detects the first configured one
|
|
1395
|
-
await generateText({ prompt: 'Hello from whichever provider is available' });
|
|
1396
|
-
```
|
|
1397
|
-
|
|
1398
|
-
**Full runtime** — configure multiple providers with explicit fallback:
|
|
1399
|
-
|
|
1400
|
-
```typescript
|
|
1401
|
-
const agent = new AgentOS();
|
|
1402
|
-
const config = await createTestAgentOSConfig();
|
|
1403
|
-
|
|
1404
|
-
await agent.initialize({
|
|
1405
|
-
...config,
|
|
1406
|
-
modelProviderManagerConfig: {
|
|
1407
|
-
providers: [
|
|
1408
|
-
{ providerId: 'anthropic', enabled: true, isDefault: true,
|
|
1409
|
-
config: { apiKey: process.env.ANTHROPIC_API_KEY } },
|
|
1410
|
-
{ providerId: 'openai', enabled: true,
|
|
1411
|
-
config: { apiKey: process.env.OPENAI_API_KEY } },
|
|
1412
|
-
{ providerId: 'gemini', enabled: true,
|
|
1413
|
-
config: { apiKey: process.env.GEMINI_API_KEY } },
|
|
1414
|
-
{ providerId: 'ollama', enabled: true,
|
|
1415
|
-
config: { baseUrl: process.env.OLLAMA_BASE_URL || 'http://localhost:11434' } },
|
|
1416
|
-
],
|
|
1417
|
-
},
|
|
1418
|
-
gmiManagerConfig: {
|
|
1419
|
-
...config.gmiManagerConfig,
|
|
1420
|
-
defaultGMIBaseConfigDefaults: {
|
|
1421
|
-
...(config.gmiManagerConfig.defaultGMIBaseConfigDefaults ?? {}),
|
|
1422
|
-
defaultLlmProviderId: 'anthropic',
|
|
1423
|
-
defaultLlmModelId: 'claude-sonnet-4-20250514',
|
|
1424
|
-
},
|
|
1425
|
-
},
|
|
1426
|
-
});
|
|
1427
|
-
|
|
1428
|
-
// Override per request:
|
|
1429
|
-
for await (const chunk of agent.processRequest({
|
|
1430
|
-
userId: 'user-1',
|
|
1431
|
-
sessionId: 'session-1',
|
|
1432
|
-
textInput: 'Hello',
|
|
1433
|
-
options: {
|
|
1434
|
-
preferredProviderId: 'gemini',
|
|
1435
|
-
preferredModelId: 'gemini-2.5-flash',
|
|
1436
|
-
},
|
|
1437
|
-
})) { /* ... */ }
|
|
1438
|
-
```
|
|
1439
|
-
|
|
1440
|
-
### Environment Variables
|
|
1441
|
-
|
|
1442
|
-
```bash
|
|
1443
|
-
# LLM providers (set at least one)
|
|
1444
|
-
OPENAI_API_KEY=sk-... # OpenAI (GPT-4o, GPT-4o-mini, o1, o3)
|
|
1445
|
-
ANTHROPIC_API_KEY=sk-ant-... # Anthropic (Claude Sonnet 4, Claude Haiku)
|
|
1446
|
-
GEMINI_API_KEY=AIza... # Google Gemini (2.5 Flash, 2.0)
|
|
1447
|
-
OPENROUTER_API_KEY=sk-or-... # OpenRouter (200+ models, auto-routing)
|
|
1448
|
-
GROQ_API_KEY=gsk_... # Groq (fast inference: Llama 3.3 70B)
|
|
1449
|
-
TOGETHER_API_KEY=... # Together AI (Llama, Mixtral)
|
|
1450
|
-
MISTRAL_API_KEY=... # Mistral (Mistral Large, Small)
|
|
1451
|
-
XAI_API_KEY=xai-... # xAI (Grok-2)
|
|
1452
|
-
OLLAMA_BASE_URL=http://localhost:11434 # Ollama (local models, no API key needed)
|
|
1453
|
-
|
|
1454
|
-
# Database
|
|
1455
|
-
DATABASE_URL=file:./data/agentos.db
|
|
1456
|
-
|
|
1457
|
-
# Observability (optional)
|
|
1458
|
-
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
|
|
1459
|
-
OTEL_SERVICE_NAME=my-agent
|
|
1460
|
-
|
|
1461
|
-
# Voice/Telephony (optional)
|
|
1462
|
-
TWILIO_ACCOUNT_SID=AC...
|
|
1463
|
-
TWILIO_AUTH_TOKEN=...
|
|
1464
|
-
TELNYX_API_KEY=KEY...
|
|
1465
|
-
```
|
|
1466
|
-
|
|
1467
|
-
---
|
|
1468
|
-
|
|
1469
|
-
## API Reference
|
|
1470
|
-
|
|
1471
|
-
### AgentOS Class
|
|
1472
|
-
|
|
1473
|
-
The main service facade. Implements `IAgentOS`.
|
|
1474
|
-
|
|
1475
|
-
```typescript
|
|
1476
|
-
class AgentOS implements IAgentOS {
|
|
1477
|
-
// Lifecycle
|
|
1478
|
-
initialize(config: AgentOSConfig): Promise<void>;
|
|
1479
|
-
shutdown(): Promise<void>;
|
|
1480
|
-
|
|
1481
|
-
// Core interaction (streaming-first)
|
|
1482
|
-
processRequest(input: AgentOSInput): AsyncGenerator<AgentOSResponse>;
|
|
1483
|
-
handleToolResult(streamId, toolCallId, toolName, toolOutput, isSuccess, errorMessage?):
|
|
1484
|
-
AsyncGenerator<AgentOSResponse>;
|
|
1485
|
-
|
|
1486
|
-
// Personas
|
|
1487
|
-
listPersonas(): IPersonaDefinition[];
|
|
1488
|
-
setActivePersona(personaId: string): Promise<void>;
|
|
1489
|
-
|
|
1490
|
-
// Conversation
|
|
1491
|
-
getConversationHistory(sessionId: string): Promise<ConversationContext>;
|
|
1492
|
-
|
|
1493
|
-
// Workflows
|
|
1494
|
-
listWorkflowDefinitions(): WorkflowDefinition[];
|
|
1495
|
-
startWorkflow(definitionId, input, options?): Promise<WorkflowInstance>;
|
|
1496
|
-
getWorkflow(workflowId): Promise<WorkflowInstance | null>;
|
|
1497
|
-
updateWorkflowTask(workflowId, taskId, update): Promise<void>;
|
|
1498
|
-
queryWorkflows(options?): Promise<WorkflowInstance[]>;
|
|
1499
|
-
|
|
1500
|
-
// Feedback
|
|
1501
|
-
submitFeedback(feedback: UserFeedbackPayload): Promise<void>;
|
|
1502
|
-
|
|
1503
|
-
// Exposed managers (for advanced usage)
|
|
1504
|
-
readonly llmProviderManager: AIModelProviderManager;
|
|
1505
|
-
readonly extensionManager: ExtensionManager;
|
|
1506
|
-
readonly conversationManager: ConversationManager;
|
|
1507
|
-
}
|
|
1508
|
-
```
|
|
1509
|
-
|
|
1510
|
-
### IAgentOS Interface
|
|
1511
|
-
|
|
1512
|
-
The core contract. See `src/api/interfaces/IAgentOS.ts` for the full interface definition.
|
|
1513
|
-
|
|
1514
|
-
### AgentOSInput
|
|
1515
|
-
|
|
1516
|
-
```typescript
|
|
1517
|
-
interface AgentOSInput {
|
|
1518
|
-
userId: string;
|
|
1519
|
-
organizationId?: string; // Multi-tenant routing
|
|
1520
|
-
sessionId: string;
|
|
1521
|
-
textInput: string | null;
|
|
1522
|
-
visionInput?: VisionInputData; // Image/video input
|
|
1523
|
-
audioInput?: AudioInputData; // Audio input
|
|
1524
|
-
preferredPersonaId?: string; // Request specific persona
|
|
1525
|
-
userApiKeys?: Record<string, string>; // User-provided API keys
|
|
1526
|
-
feedback?: UserFeedbackPayload; // Inline feedback
|
|
1527
|
-
workflowInvocation?: WorkflowInvocationRequest;
|
|
1528
|
-
agencyInvocation?: AgencyInvocationRequest;
|
|
1529
|
-
memoryControl?: AgentOSMemoryControl;
|
|
1530
|
-
options?: ProcessingOptions; // Model override, temperature, etc.
|
|
1531
|
-
}
|
|
1532
|
-
```
|
|
1533
|
-
|
|
1534
|
-
### AgentOSResponse Streaming
|
|
1535
|
-
|
|
1536
|
-
All core methods return `AsyncGenerator<AgentOSResponse>`. Each yielded chunk has a `type` discriminant:
|
|
1537
|
-
|
|
1538
|
-
```typescript
|
|
1539
|
-
// Handle all chunk types:
|
|
1540
|
-
for await (const chunk of agent.processRequest(input)) {
|
|
1541
|
-
switch (chunk.type) {
|
|
1542
|
-
case AgentOSResponseChunkType.TEXT_DELTA:
|
|
1543
|
-
process.stdout.write(chunk.textDelta);
|
|
1544
|
-
break;
|
|
1545
|
-
case AgentOSResponseChunkType.TOOL_CALL_REQUEST:
|
|
1546
|
-
console.log('Tools requested:', chunk.toolCalls);
|
|
1547
|
-
break;
|
|
1548
|
-
case AgentOSResponseChunkType.TOOL_RESULT_EMISSION:
|
|
1549
|
-
console.log('Tool result:', chunk.toolName, chunk.toolResult);
|
|
1550
|
-
break;
|
|
1551
|
-
case AgentOSResponseChunkType.SYSTEM_PROGRESS:
|
|
1552
|
-
console.log('Progress:', chunk.message, chunk.progressPercentage);
|
|
1553
|
-
break;
|
|
1554
|
-
case AgentOSResponseChunkType.WORKFLOW_UPDATE:
|
|
1555
|
-
console.log('Workflow:', chunk.workflowProgress);
|
|
1556
|
-
break;
|
|
1557
|
-
case AgentOSResponseChunkType.AGENCY_UPDATE:
|
|
1558
|
-
console.log('Agency event:', chunk.agencyEvent);
|
|
1559
|
-
break;
|
|
1560
|
-
case AgentOSResponseChunkType.ERROR:
|
|
1561
|
-
console.error('Error:', chunk.error);
|
|
1562
|
-
break;
|
|
1563
|
-
case AgentOSResponseChunkType.FINAL_RESPONSE:
|
|
1564
|
-
console.log('Complete:', chunk.finalText);
|
|
1565
|
-
break;
|
|
1566
|
-
}
|
|
1567
|
-
}
|
|
1568
|
-
```
|
|
1569
|
-
|
|
1570
|
-
### ITool Interface
|
|
1571
|
-
|
|
1572
|
-
See the [Tool System](#tool-system) section above for the full interface. Tools are registered via extension packs:
|
|
1573
|
-
|
|
1574
|
-
```typescript
|
|
1575
|
-
const descriptor: ExtensionDescriptor<ITool> = {
|
|
1576
|
-
id: 'my-tool',
|
|
1577
|
-
kind: EXTENSION_KIND_TOOL,
|
|
1578
|
-
payload: myToolImplementation,
|
|
1579
|
-
};
|
|
1580
|
-
```
|
|
1581
|
-
|
|
1582
|
-
### ExtensionDescriptor
|
|
1583
|
-
|
|
1584
|
-
See the [Extension System](#extension-system) section above for the full type definition and all 12 extension kinds.
|
|
1585
|
-
|
|
1586
|
-
### IGuardrailService
|
|
1587
|
-
|
|
1588
|
-
```typescript
|
|
1589
|
-
interface IGuardrailService {
|
|
1590
|
-
evaluateInput?(
|
|
1591
|
-
input: AgentOSInput,
|
|
1592
|
-
context: GuardrailContext,
|
|
1593
|
-
): Promise<GuardrailEvaluationResult>;
|
|
1594
|
-
|
|
1595
|
-
evaluateOutput?(
|
|
1596
|
-
output: string,
|
|
1597
|
-
context: GuardrailContext,
|
|
1598
|
-
): Promise<GuardrailEvaluationResult>;
|
|
1599
|
-
}
|
|
1600
|
-
|
|
1601
|
-
interface GuardrailEvaluationResult {
|
|
1602
|
-
action: GuardrailAction; // ALLOW | FLAG | SANITIZE | BLOCK
|
|
1603
|
-
reason?: string; // Human-readable explanation
|
|
1604
|
-
reasonCode?: string; // Machine-readable code
|
|
1605
|
-
modifiedText?: string; // Required when action is SANITIZE
|
|
1606
|
-
metadata?: Record<string, any>; // Additional context for logging
|
|
1607
|
-
}
|
|
1608
|
-
```
|
|
1609
|
-
|
|
1610
|
-
### IHumanInteractionManager
|
|
1611
|
-
|
|
1612
|
-
```typescript
|
|
1613
|
-
interface IHumanInteractionManager {
|
|
1614
|
-
requestApproval(action: PendingAction): Promise<ApprovalDecision>;
|
|
1615
|
-
requestClarification(request: ClarificationRequest): Promise<ClarificationResponse>;
|
|
1616
|
-
escalateToHuman(context: EscalationContext): Promise<EscalationResult>;
|
|
1617
|
-
getPendingActions(): Promise<PendingAction[]>;
|
|
1618
|
-
}
|
|
1619
|
-
```
|
|
1620
|
-
|
|
1621
|
-
---
|
|
1622
|
-
|
|
1623
|
-
## Usage Examples
|
|
1624
|
-
|
|
1625
|
-
### Orchestration Patterns
|
|
1626
|
-
|
|
1627
|
-
Use the new orchestration layer based on how much control you need:
|
|
1628
|
-
|
|
1629
|
-
- `workflow()` for deterministic sequential/parallel DAGs
|
|
1630
|
-
- `AgentGraph` for explicit cycles, retries, and custom routing
|
|
1631
|
-
- `mission()` when you know the goal but want the planner to decide the steps
|
|
1632
|
-
|
|
1633
|
-
```typescript
|
|
1634
|
-
import {
|
|
1635
|
-
AgentGraph,
|
|
1636
|
-
END,
|
|
1637
|
-
START,
|
|
1638
|
-
gmiNode,
|
|
1639
|
-
mission,
|
|
1640
|
-
toolNode,
|
|
1641
|
-
workflow,
|
|
1642
|
-
} from '@framers/agentos/orchestration';
|
|
1643
|
-
import { z } from 'zod';
|
|
1644
|
-
|
|
1645
|
-
// 1. Deterministic DAG: sequential steps with a parallel fan-out/join
|
|
1646
|
-
const onboarding = workflow('user-onboarding')
|
|
1647
|
-
.input(z.object({ email: z.string().email() }))
|
|
1648
|
-
.returns(z.object({ userId: z.string() }))
|
|
1649
|
-
.step('validate', { tool: 'email_validator' })
|
|
1650
|
-
.then('create-account', { tool: 'user_service' })
|
|
1651
|
-
.parallel(
|
|
1652
|
-
[
|
|
1653
|
-
{ tool: 'send_welcome_email' },
|
|
1654
|
-
{ tool: 'provision_default_workspace' },
|
|
1655
|
-
],
|
|
1656
|
-
{
|
|
1657
|
-
strategy: 'all',
|
|
1658
|
-
merge: { 'scratch.completedTasks': 'concat' },
|
|
1659
|
-
},
|
|
1660
|
-
)
|
|
1661
|
-
.compile();
|
|
1662
|
-
|
|
1663
|
-
// 2. Explicit graph: fixed publish pipeline with explicit nodes and edges
|
|
1664
|
-
const reviewGraph = new AgentGraph({
|
|
1665
|
-
input: z.object({ topic: z.string() }),
|
|
1666
|
-
scratch: z.object({ draft: z.string().optional() }),
|
|
1667
|
-
artifacts: z.object({ status: z.string().optional(), summary: z.string().optional() }),
|
|
1668
|
-
})
|
|
1669
|
-
.addNode('draft', gmiNode({ instructions: 'Draft the release note.', executionMode: 'single_turn' }))
|
|
1670
|
-
.addNode('publish', toolNode('publish_report'))
|
|
1671
|
-
.addEdge(START, 'draft')
|
|
1672
|
-
.addEdge('draft', 'publish')
|
|
1673
|
-
.addEdge('publish', END)
|
|
1674
|
-
.compile();
|
|
1675
|
-
|
|
1676
|
-
// 3. Goal-first mission: preview the generated plan before running it
|
|
1677
|
-
const researcher = mission('deep-research')
|
|
1678
|
-
.input(z.object({ topic: z.string() }))
|
|
1679
|
-
.goal('Research {{topic}} thoroughly and produce a cited summary')
|
|
1680
|
-
.returns(z.object({ summary: z.string() }))
|
|
1681
|
-
.planner({ strategy: 'plan_and_execute', maxSteps: 8 })
|
|
1682
|
-
.compile();
|
|
1683
|
-
|
|
1684
|
-
const preview = await researcher.explain({ topic: 'AI safety' });
|
|
1685
|
-
console.log(preview.steps.map((step) => step.id));
|
|
1686
|
-
```
|
|
1687
|
-
|
|
1688
|
-
For deeper examples, see [`docs/architecture/AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md), [`docs/orchestration/WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md), [`docs/orchestration/MISSION_API.md`](./docs/orchestration/MISSION_API.md), the runnable examples [`examples/agent-graph.mjs`](./examples/agent-graph.mjs), [`examples/workflow-dsl.mjs`](./examples/workflow-dsl.mjs), [`examples/mission-api.mjs`](./examples/mission-api.mjs), and the legacy dependency-ordered example [`examples/multi-agent-workflow.mjs`](./examples/multi-agent-workflow.mjs).
|
|
1689
|
-
|
|
1690
|
-
### Streaming Chat
|
|
1691
|
-
|
|
1692
|
-
```typescript
|
|
1693
|
-
import { AgentOS, AgentOSResponseChunkType } from '@framers/agentos';
|
|
1694
|
-
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
1695
|
-
|
|
1696
|
-
const agent = new AgentOS();
|
|
1697
|
-
await agent.initialize(await createTestAgentOSConfig());
|
|
1698
|
-
|
|
1699
|
-
for await (const chunk of agent.processRequest({
|
|
1700
|
-
userId: 'user-1',
|
|
1701
|
-
sessionId: 'session-1',
|
|
1702
|
-
textInput: 'Explain how TCP handshakes work',
|
|
1703
|
-
})) {
|
|
1704
|
-
if (chunk.type === AgentOSResponseChunkType.TEXT_DELTA) {
|
|
1705
|
-
process.stdout.write(chunk.textDelta);
|
|
1706
|
-
}
|
|
1707
|
-
}
|
|
1708
|
-
```
|
|
1709
|
-
|
|
1710
|
-
### Adding Tools
|
|
1711
|
-
|
|
1712
|
-
Tools are registered via extension packs and called automatically by the model:
|
|
1713
|
-
|
|
1714
|
-
```typescript
|
|
1715
|
-
import {
|
|
1716
|
-
AgentOS,
|
|
1717
|
-
AgentOSResponseChunkType,
|
|
1718
|
-
EXTENSION_KIND_TOOL,
|
|
1719
|
-
type ExtensionManifest,
|
|
1720
|
-
type ExtensionPack,
|
|
1721
|
-
type ITool,
|
|
1722
|
-
} from '@framers/agentos';
|
|
1723
|
-
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
1724
|
-
|
|
1725
|
-
const weatherTool: ITool = {
|
|
1726
|
-
id: 'get-weather',
|
|
1727
|
-
name: 'get_weather',
|
|
1728
|
-
displayName: 'Get Weather',
|
|
1729
|
-
description: 'Returns current weather for a city.',
|
|
1730
|
-
category: 'utility',
|
|
1731
|
-
hasSideEffects: false,
|
|
1732
|
-
inputSchema: {
|
|
1733
|
-
type: 'object',
|
|
1734
|
-
properties: { city: { type: 'string', description: 'City name' } },
|
|
1735
|
-
required: ['city'],
|
|
1736
|
-
},
|
|
1737
|
-
execute: async (args) => ({
|
|
1738
|
-
success: true,
|
|
1739
|
-
output: { text: `Weather in ${args.city}: 22 C, partly cloudy` },
|
|
1740
|
-
}),
|
|
1741
|
-
};
|
|
1742
|
-
|
|
1743
|
-
const manifest: ExtensionManifest = {
|
|
1744
|
-
packs: [{
|
|
1745
|
-
factory: async () => ({
|
|
1746
|
-
name: 'my-tools',
|
|
1747
|
-
descriptors: [{ id: weatherTool.id, kind: EXTENSION_KIND_TOOL, payload: weatherTool }],
|
|
1748
|
-
} satisfies ExtensionPack),
|
|
1749
|
-
}],
|
|
1750
|
-
};
|
|
1751
|
-
|
|
1752
|
-
const agent = new AgentOS();
|
|
1753
|
-
const config = await createTestAgentOSConfig();
|
|
1754
|
-
await agent.initialize({ ...config, extensionManifest: manifest });
|
|
1755
|
-
|
|
1756
|
-
for await (const chunk of agent.processRequest({
|
|
1757
|
-
userId: 'user-1',
|
|
1758
|
-
sessionId: 'session-1',
|
|
1759
|
-
textInput: 'What is the weather in Tokyo?',
|
|
1760
|
-
})) {
|
|
1761
|
-
switch (chunk.type) {
|
|
1762
|
-
case AgentOSResponseChunkType.TEXT_DELTA:
|
|
1763
|
-
process.stdout.write(chunk.textDelta);
|
|
1764
|
-
break;
|
|
1765
|
-
case AgentOSResponseChunkType.TOOL_CALL_REQUEST:
|
|
1766
|
-
console.log('Tool calls:', chunk.toolCalls);
|
|
1767
|
-
break;
|
|
1768
|
-
case AgentOSResponseChunkType.TOOL_RESULT_EMISSION:
|
|
1769
|
-
console.log('Tool result:', chunk.toolResult);
|
|
1770
|
-
break;
|
|
1771
|
-
}
|
|
1772
|
-
}
|
|
1773
|
-
```
|
|
1774
|
-
|
|
1775
|
-
### Multi-Agent Collaboration
|
|
1776
|
-
|
|
1777
|
-
```typescript
|
|
1778
|
-
import { AgentCommunicationBus } from '@framers/agentos';
|
|
1779
|
-
|
|
1780
|
-
const bus = new AgentCommunicationBus({
|
|
1781
|
-
routingConfig: { enableRoleRouting: true, enableLoadBalancing: true },
|
|
1782
|
-
});
|
|
1783
|
-
|
|
1784
|
-
bus.registerAgent('coordinator-gmi', 'agency-docs', 'coordinator');
|
|
1785
|
-
bus.registerAgent('researcher-gmi', 'agency-docs', 'researcher');
|
|
1786
|
-
bus.registerAgent('writer-gmi', 'agency-docs', 'writer');
|
|
1787
|
-
|
|
1788
|
-
bus.subscribe(
|
|
1789
|
-
'researcher-gmi',
|
|
1790
|
-
async (message) => {
|
|
1791
|
-
if (message.type !== 'question') return;
|
|
1792
|
-
await bus.sendToAgent(message.fromAgentId, {
|
|
1793
|
-
type: 'answer',
|
|
1794
|
-
fromAgentId: 'researcher-gmi',
|
|
1795
|
-
content: { findings: ['auth edge cases', 'missing audit trail', 'weak retry policy'] },
|
|
1796
|
-
inReplyTo: message.messageId,
|
|
1797
|
-
priority: 'normal',
|
|
1798
|
-
});
|
|
1799
|
-
},
|
|
1800
|
-
{ messageTypes: ['question'] },
|
|
1801
|
-
);
|
|
1802
|
-
|
|
1803
|
-
bus.subscribe(
|
|
1804
|
-
'writer-gmi',
|
|
1805
|
-
async (message) => {
|
|
1806
|
-
if (message.type !== 'task_delegation') return;
|
|
1807
|
-
await bus.sendToAgent(message.fromAgentId, {
|
|
1808
|
-
type: 'answer',
|
|
1809
|
-
fromAgentId: 'writer-gmi',
|
|
1810
|
-
content: { accepted: true },
|
|
1811
|
-
inReplyTo: message.messageId,
|
|
1812
|
-
priority: 'normal',
|
|
1813
|
-
});
|
|
1814
|
-
},
|
|
1815
|
-
{ messageTypes: ['task_delegation'] },
|
|
1816
|
-
);
|
|
1817
|
-
|
|
1818
|
-
await bus.sendToRole('agency-docs', 'researcher', {
|
|
1819
|
-
type: 'task_delegation',
|
|
1820
|
-
fromAgentId: 'coordinator-gmi',
|
|
1821
|
-
content: { topic: 'auth module', instructions: 'Find the risky edge cases.' },
|
|
1822
|
-
priority: 'high',
|
|
1823
|
-
});
|
|
1824
|
-
|
|
1825
|
-
const review = await bus.requestResponse('researcher-gmi', {
|
|
1826
|
-
type: 'question',
|
|
1827
|
-
fromAgentId: 'coordinator-gmi',
|
|
1828
|
-
content: 'What are the top three findings?',
|
|
1829
|
-
priority: 'high',
|
|
1830
|
-
timeoutMs: 30_000,
|
|
1831
|
-
});
|
|
1832
|
-
|
|
1833
|
-
const handoff = await bus.handoff('researcher-gmi', 'writer-gmi', {
|
|
1834
|
-
taskId: 'auth-audit',
|
|
1835
|
-
taskDescription: 'Turn the findings into a release note draft',
|
|
1836
|
-
progress: 0.8,
|
|
1837
|
-
completedWork: [review.content],
|
|
1838
|
-
remainingWork: ['Write polished summary'],
|
|
1839
|
-
context: { audience: 'engineering' },
|
|
1840
|
-
reason: 'completion',
|
|
1841
|
-
instructions: 'Summarize the findings in concise release-note style.',
|
|
1842
|
-
});
|
|
1843
|
-
|
|
1844
|
-
console.log(handoff.accepted);
|
|
1845
|
-
```
|
|
1846
|
-
|
|
1847
|
-
Runnable example: [`examples/agent-communication-bus.mjs`](./examples/agent-communication-bus.mjs)
|
|
1848
|
-
|
|
1849
|
-
### Human-in-the-Loop Approvals
|
|
1850
|
-
|
|
1851
|
-
```typescript
|
|
1852
|
-
import { HumanInteractionManager } from '@framers/agentos';
|
|
1853
|
-
|
|
1854
|
-
const hitl = new HumanInteractionManager({ defaultTimeoutMs: 300_000 });
|
|
1855
|
-
|
|
1856
|
-
const decision = await hitl.requestApproval({
|
|
1857
|
-
actionId: 'archive-inactive',
|
|
1858
|
-
description: 'Archive 50K inactive accounts older than 2 years',
|
|
1859
|
-
severity: 'high',
|
|
1860
|
-
category: 'data_modification',
|
|
1861
|
-
agentId: 'data-cleanup-agent',
|
|
1862
|
-
context: { affectedRows: 50_000, table: 'users' },
|
|
1863
|
-
reversible: true,
|
|
1864
|
-
potentialConsequences: ['Users will lose access to archived data'],
|
|
1865
|
-
alternatives: [
|
|
1866
|
-
{ alternativeId: 'soft_delete', description: 'Mark as inactive instead' },
|
|
1867
|
-
{ alternativeId: 'export_first', description: 'Export to CSV before archiving' },
|
|
1868
|
-
],
|
|
1869
|
-
});
|
|
1870
|
-
|
|
1871
|
-
if (decision.approved) {
|
|
1872
|
-
await executeArchive();
|
|
1873
|
-
} else if (decision.selectedAlternative) {
|
|
1874
|
-
await executeAlternative(decision.selectedAlternative);
|
|
1875
|
-
}
|
|
1876
|
-
```
|
|
1877
|
-
|
|
1878
|
-
### Structured Data Extraction
|
|
1879
|
-
|
|
1880
|
-
```typescript
|
|
1881
|
-
import { AgentOS, StructuredOutputManager } from '@framers/agentos';
|
|
1882
|
-
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
1883
|
-
|
|
1884
|
-
const agent = new AgentOS();
|
|
1885
|
-
await agent.initialize(await createTestAgentOSConfig());
|
|
1886
|
-
|
|
1887
|
-
const structured = new StructuredOutputManager({
|
|
1888
|
-
llmProviderManager: agent.llmProviderManager,
|
|
1889
|
-
});
|
|
1890
|
-
|
|
1891
|
-
const contact = await structured.generate({
|
|
1892
|
-
prompt: 'Extract: "Meeting with Sarah Chen (sarah@startup.io) on Jan 15 re: Series A"',
|
|
1893
|
-
schema: {
|
|
1894
|
-
type: 'object',
|
|
1895
|
-
properties: {
|
|
1896
|
-
name: { type: 'string' },
|
|
1897
|
-
email: { type: 'string', format: 'email' },
|
|
1898
|
-
date: { type: 'string' },
|
|
1899
|
-
topic: { type: 'string' },
|
|
1900
|
-
},
|
|
1901
|
-
required: ['name', 'email'],
|
|
1902
|
-
},
|
|
1903
|
-
schemaName: 'ContactInfo',
|
|
1904
|
-
});
|
|
1905
|
-
// Result: { name: 'Sarah Chen', email: 'sarah@startup.io', date: 'Jan 15', topic: 'Series A' }
|
|
1906
|
-
```
|
|
1907
|
-
|
|
1908
|
-
### RAG Memory
|
|
1909
|
-
|
|
1910
|
-
```typescript
|
|
1911
|
-
import { AgentOS } from '@framers/agentos';
|
|
1912
|
-
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
1913
|
-
|
|
1914
|
-
const agent = new AgentOS();
|
|
1915
|
-
const config = await createTestAgentOSConfig();
|
|
1916
|
-
|
|
1917
|
-
await agent.initialize({
|
|
1918
|
-
...config,
|
|
1919
|
-
ragConfig: {
|
|
1920
|
-
embeddingManagerConfig: {
|
|
1921
|
-
embeddingModels: [
|
|
1922
|
-
{ modelId: 'text-embedding-3-small', providerId: 'openai',
|
|
1923
|
-
dimension: 1536, isDefault: true },
|
|
1924
|
-
],
|
|
1925
|
-
},
|
|
1926
|
-
vectorStoreManagerConfig: {
|
|
1927
|
-
managerId: 'rag-vsm',
|
|
1928
|
-
providers: [
|
|
1929
|
-
{ id: 'sql-store', type: 'sql', storage: { filePath: './data/vectors.db' } },
|
|
1930
|
-
],
|
|
1931
|
-
defaultProviderId: 'sql-store',
|
|
1932
|
-
defaultEmbeddingDimension: 1536,
|
|
1933
|
-
},
|
|
1934
|
-
dataSourceConfigs: [{
|
|
1935
|
-
dataSourceId: 'conversations',
|
|
1936
|
-
displayName: 'Conversation Memory',
|
|
1937
|
-
vectorStoreProviderId: 'sql-store',
|
|
1938
|
-
actualNameInProvider: 'conversations',
|
|
1939
|
-
embeddingDimension: 1536,
|
|
1940
|
-
}],
|
|
1941
|
-
retrievalAugmentorConfig: {
|
|
1942
|
-
defaultDataSourceId: 'conversations',
|
|
1943
|
-
categoryBehaviors: [],
|
|
1944
|
-
},
|
|
1945
|
-
},
|
|
1946
|
-
});
|
|
1947
|
-
|
|
1948
|
-
// Agent now retrieves relevant context from vector memory before responding
|
|
1949
|
-
```
|
|
1950
|
-
|
|
1951
|
-
### Custom Guardrails
|
|
1952
|
-
|
|
1953
|
-
```typescript
|
|
1954
|
-
import {
|
|
1955
|
-
AgentOS,
|
|
1956
|
-
type IGuardrailService,
|
|
1957
|
-
GuardrailAction,
|
|
1958
|
-
type GuardrailInputPayload,
|
|
1959
|
-
type GuardrailOutputPayload,
|
|
1960
|
-
} from '@framers/agentos';
|
|
1961
|
-
import { createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
1962
|
-
|
|
1963
|
-
const piiGuardrail: IGuardrailService = {
|
|
1964
|
-
config: {
|
|
1965
|
-
evaluateStreamingChunks: true,
|
|
1966
|
-
canSanitize: true,
|
|
1967
|
-
},
|
|
1968
|
-
|
|
1969
|
-
async evaluateInput({ input }: GuardrailInputPayload) {
|
|
1970
|
-
// Check for SSN patterns in user input
|
|
1971
|
-
const ssnPattern = /\b\d{3}-\d{2}-\d{4}\b/g;
|
|
1972
|
-
if (input.textInput && ssnPattern.test(input.textInput)) {
|
|
1973
|
-
return {
|
|
1974
|
-
action: GuardrailAction.SANITIZE,
|
|
1975
|
-
modifiedText: input.textInput.replace(ssnPattern, '[SSN REDACTED]'),
|
|
1976
|
-
reason: 'PII detected in user input',
|
|
1977
|
-
reasonCode: 'PII_SSN',
|
|
1978
|
-
};
|
|
1979
|
-
}
|
|
1980
|
-
return { action: GuardrailAction.ALLOW };
|
|
1981
|
-
},
|
|
1982
|
-
|
|
1983
|
-
async evaluateOutput({ chunk }: GuardrailOutputPayload) {
|
|
1984
|
-
const text =
|
|
1985
|
-
chunk.type === 'TEXT_DELTA'
|
|
1986
|
-
? chunk.textDelta ?? ''
|
|
1987
|
-
: chunk.type === 'FINAL_RESPONSE'
|
|
1988
|
-
? chunk.finalResponseText ?? ''
|
|
1989
|
-
: '';
|
|
1990
|
-
|
|
1991
|
-
if (text.toLowerCase().includes('password')) {
|
|
1992
|
-
return {
|
|
1993
|
-
action: GuardrailAction.BLOCK,
|
|
1994
|
-
reason: 'Output contains potentially sensitive credential information',
|
|
1995
|
-
reasonCode: 'CREDENTIAL_LEAK',
|
|
1996
|
-
};
|
|
1997
|
-
}
|
|
1998
|
-
return { action: GuardrailAction.ALLOW };
|
|
1999
|
-
},
|
|
2000
|
-
};
|
|
2001
|
-
|
|
2002
|
-
const agent = new AgentOS();
|
|
2003
|
-
const config = await createTestAgentOSConfig();
|
|
2004
|
-
await agent.initialize({ ...config, guardrailService: piiGuardrail });
|
|
2005
|
-
```
|
|
2006
|
-
|
|
2007
|
-
> **For production PII redaction**, use the built-in `createPiiRedactionGuardrail()` extension
|
|
2008
|
-
> instead of hand-rolled regex. It provides four-tier detection (regex + NLP + BERT NER +
|
|
2009
|
-
> LLM-as-judge), streaming support, and configurable redaction styles. See
|
|
2010
|
-
> [GUARDRAILS_USAGE.md](./docs/safety/GUARDRAILS_USAGE.md) for full examples.
|
|
2011
|
-
|
|
2012
|
-
### Channel Adapters
|
|
2013
|
-
|
|
2014
|
-
```typescript
|
|
2015
|
-
import {
|
|
2016
|
-
EXTENSION_KIND_MESSAGING_CHANNEL,
|
|
2017
|
-
type ExtensionManifest,
|
|
2018
|
-
type IChannelAdapter,
|
|
2019
|
-
type ChannelPlatform,
|
|
2020
|
-
} from '@framers/agentos';
|
|
2021
|
-
|
|
2022
|
-
// Implement a custom channel adapter
|
|
2023
|
-
const myAdapter: IChannelAdapter = {
|
|
2024
|
-
platform: 'webchat' as ChannelPlatform,
|
|
2025
|
-
capabilities: new Set(['text', 'rich_text', 'images', 'typing_indicator']),
|
|
2026
|
-
|
|
2027
|
-
async connect() { /* establish connection */ },
|
|
2028
|
-
async disconnect() { /* clean up */ },
|
|
2029
|
-
async sendMessage(channelId, message) { /* send outbound */ },
|
|
2030
|
-
onMessage(handler) { /* register inbound handler */ },
|
|
2031
|
-
getConnectionInfo() {
|
|
2032
|
-
return { status: 'connected', connectedSince: new Date().toISOString() };
|
|
2033
|
-
},
|
|
2034
|
-
};
|
|
2035
|
-
|
|
2036
|
-
// Register via extension manifest
|
|
2037
|
-
const manifest: ExtensionManifest = {
|
|
2038
|
-
packs: [{
|
|
2039
|
-
factory: async () => ({
|
|
2040
|
-
name: 'my-channels',
|
|
2041
|
-
descriptors: [{
|
|
2042
|
-
id: 'webchat-adapter',
|
|
2043
|
-
kind: EXTENSION_KIND_MESSAGING_CHANNEL,
|
|
2044
|
-
payload: myAdapter,
|
|
2045
|
-
}],
|
|
2046
|
-
}),
|
|
2047
|
-
}],
|
|
2048
|
-
};
|
|
2049
|
-
```
|
|
2050
|
-
|
|
2051
|
-
### Voice Calls
|
|
2052
|
-
|
|
2053
|
-
```typescript
|
|
2054
|
-
import { CallManager, type IVoiceCallProvider } from '@framers/agentos';
|
|
2055
|
-
|
|
2056
|
-
const callManager = new CallManager();
|
|
2057
|
-
|
|
2058
|
-
// Initiate an outbound call
|
|
2059
|
-
const call = await callManager.initiateCall({
|
|
2060
|
-
provider: 'twilio',
|
|
2061
|
-
to: '+1234567890',
|
|
2062
|
-
from: '+0987654321',
|
|
2063
|
-
agentId: 'support-agent',
|
|
2064
|
-
});
|
|
2065
|
-
|
|
2066
|
-
// Monitor call state transitions
|
|
2067
|
-
call.on('stateChange', (newState) => {
|
|
2068
|
-
console.log(`Call ${call.id}: ${newState}`);
|
|
2069
|
-
// initiated -> ringing -> answered -> active -> speaking <-> listening -> completed
|
|
2070
|
-
});
|
|
2071
|
-
|
|
2072
|
-
// Handle call completion
|
|
2073
|
-
call.on('completed', (summary) => {
|
|
2074
|
-
console.log('Duration:', summary.durationMs);
|
|
2075
|
-
console.log('Transcript:', summary.transcript);
|
|
2076
|
-
});
|
|
2077
|
-
```
|
|
2078
|
-
|
|
2079
|
-
---
|
|
2080
|
-
|
|
2081
|
-
## Package Exports
|
|
2082
|
-
|
|
2083
|
-
AgentOS provides 112 export paths for fine-grained imports. Key entry points:
|
|
2084
|
-
|
|
2085
|
-
```typescript
|
|
2086
|
-
// Main entry -- all public types and classes
|
|
2087
|
-
import { AgentOS, AgentOSResponseChunkType, /* ... */ } from '@framers/agentos';
|
|
2088
|
-
|
|
2089
|
-
// Configuration
|
|
2090
|
-
import { createAgentOSConfig, createTestAgentOSConfig } from '@framers/agentos/config/AgentOSConfig';
|
|
766
|
+
// Orchestration (workflow, graph, mission builders)
|
|
767
|
+
import { workflow, AgentGraph, mission, START, END } from '@framers/agentos/orchestration';
|
|
2091
768
|
|
|
2092
769
|
// Safety primitives
|
|
2093
770
|
import { CircuitBreaker, CostGuard, StuckDetector } from '@framers/agentos/safety/runtime';
|
|
2094
771
|
|
|
2095
772
|
// Guardrails
|
|
2096
|
-
import { GuardrailAction } from '@framers/agentos/safety/guardrails';
|
|
2097
|
-
import { ParallelGuardrailDispatcher } from '@framers/agentos/safety/guardrails';
|
|
773
|
+
import { GuardrailAction, ParallelGuardrailDispatcher } from '@framers/agentos/safety/guardrails';
|
|
2098
774
|
|
|
2099
|
-
//
|
|
2100
|
-
import type { ITool, ToolExecutionResult } from '@framers/agentos/core/tools';
|
|
2101
|
-
|
|
2102
|
-
// HITL
|
|
2103
|
-
import type { IHumanInteractionManager } from '@framers/agentos/orchestration/hitl';
|
|
2104
|
-
|
|
2105
|
-
// RAG
|
|
775
|
+
// RAG and GraphRAG
|
|
2106
776
|
import { VectorStoreManager, EmbeddingManager, RetrievalAugmentor } from '@framers/agentos/rag';
|
|
2107
777
|
import { GraphRAGEngine } from '@framers/agentos/rag/graphrag';
|
|
2108
778
|
|
|
2109
779
|
// Skills
|
|
2110
780
|
import { SkillRegistry, SkillLoader } from '@framers/agentos/skills';
|
|
2111
781
|
|
|
2112
|
-
//
|
|
2113
|
-
import {
|
|
2114
|
-
|
|
2115
|
-
|
|
2116
|
-
import {
|
|
2117
|
-
import { createCodeSafetyGuardrail } from '@framers/agentos-ext-code-safety';
|
|
2118
|
-
import { createGroundingGuardrail } from '@framers/agentos-ext-grounding-guard';
|
|
782
|
+
// Tools
|
|
783
|
+
import type { ITool, ToolExecutionResult } from '@framers/agentos/core/tools';
|
|
784
|
+
|
|
785
|
+
// HITL
|
|
786
|
+
import type { IHumanInteractionManager } from '@framers/agentos/orchestration/hitl';
|
|
2119
787
|
|
|
2120
|
-
// Deep imports
|
|
788
|
+
// Deep imports via wildcard (up to 4 levels)
|
|
2121
789
|
import { SomeType } from '@framers/agentos/safety/runtime/CircuitBreaker';
|
|
2122
|
-
import { SomeConfig } from '@framers/agentos/config/ToolOrchestratorConfig';
|
|
2123
790
|
```
|
|
2124
791
|
|
|
2125
|
-
Wildcard exports support paths up to 4 levels deep:
|
|
2126
|
-
- `./*` -- `dist/*.js`
|
|
2127
|
-
- `./*/*` -- `dist/*/*.js`
|
|
2128
|
-
- `./*/*/*` -- `dist/*/*/*.js`
|
|
2129
|
-
- `./*/*/*/*` -- `dist/*/*/*/*.js`
|
|
2130
|
-
|
|
2131
792
|
---
|
|
2132
793
|
|
|
2133
|
-
##
|
|
794
|
+
## Documentation
|
|
2134
795
|
|
|
2135
|
-
|
|
2136
|
-
|
|
2137
|
-
|
|
|
2138
|
-
|
|
2139
|
-
| [`AGENCY_API.md`](./docs/orchestration/AGENCY_API.md) | `agency()`
|
|
2140
|
-
| [`
|
|
2141
|
-
| [`
|
|
2142
|
-
| [`
|
|
2143
|
-
| [`
|
|
2144
|
-
| [`
|
|
2145
|
-
| [`
|
|
2146
|
-
| [`RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) | Vector store setup, embedding models, data
|
|
796
|
+
| Guide | What it covers |
|
|
797
|
+
|-------|---------------|
|
|
798
|
+
| [`ARCHITECTURE.md`](./docs/architecture/ARCHITECTURE.md) | Full system architecture, data flow diagrams, layer breakdown |
|
|
799
|
+
| [`HIGH_LEVEL_API.md`](./docs/getting-started/HIGH_LEVEL_API.md) | `generateText`, `streamText`, `generateObject`, `agent` reference |
|
|
800
|
+
| [`AGENCY_API.md`](./docs/orchestration/AGENCY_API.md) | `agency()` -- all strategies, HITL, guardrails, RAG, nested agencies |
|
|
801
|
+
| [`UNIFIED_ORCHESTRATION.md`](./docs/orchestration/UNIFIED_ORCHESTRATION.md) | Orchestration layer overview (workflow, graph, mission) |
|
|
802
|
+
| [`WORKFLOW_DSL.md`](./docs/orchestration/WORKFLOW_DSL.md) | `workflow()` DSL reference |
|
|
803
|
+
| [`AGENT_GRAPH.md`](./docs/architecture/AGENT_GRAPH.md) | `AgentGraph` builder reference |
|
|
804
|
+
| [`MISSION_API.md`](./docs/orchestration/MISSION_API.md) | `mission()` goal-driven orchestration |
|
|
805
|
+
| [`CHECKPOINTING.md`](./docs/orchestration/CHECKPOINTING.md) | Persistent checkpointing and fault recovery |
|
|
806
|
+
| [`COGNITIVE_MECHANISMS.md`](./docs/memory/COGNITIVE_MECHANISMS.md) | 8 cognitive memory mechanisms, 30+ APA citations |
|
|
807
|
+
| [`RAG_MEMORY_CONFIGURATION.md`](./docs/memory/RAG_MEMORY_CONFIGURATION.md) | Vector store setup, embedding models, data sources |
|
|
2147
808
|
| [`MULTIMODAL_RAG.md`](./docs/memory/MULTIMODAL_RAG.md) | Image, audio, and document RAG pipelines |
|
|
2148
|
-
| [`
|
|
2149
|
-
| [`
|
|
2150
|
-
| [`
|
|
2151
|
-
| [`
|
|
2152
|
-
| [`
|
|
2153
|
-
| [`
|
|
2154
|
-
| [`
|
|
2155
|
-
| [`
|
|
2156
|
-
| [`
|
|
2157
|
-
| [`
|
|
2158
|
-
| [`
|
|
2159
|
-
| [`
|
|
2160
|
-
| [`
|
|
2161
|
-
| [`
|
|
2162
|
-
| [`CLIENT_SIDE_STORAGE.md`](./docs/memory/CLIENT_SIDE_STORAGE.md) | Browser-compatible storage adapters |
|
|
2163
|
-
| [`SQL_STORAGE_QUICKSTART.md`](./docs/getting-started/SQL_STORAGE_QUICKSTART.md) | SQLite/Postgres setup with `@framers/sql-storage-adapter` |
|
|
2164
|
-
| [`RELEASING.md`](./docs/getting-started/RELEASING.md) | Release process and semantic versioning |
|
|
2165
|
-
|
|
2166
|
-
---
|
|
2167
|
-
|
|
2168
|
-
## Key Design Patterns
|
|
2169
|
-
|
|
2170
|
-
1. **Interface-driven design** -- All major components define interface contracts (`IAgentOS`, `ITool`, `IGuardrailService`, `IChannelAdapter`, `IVoiceCallProvider`, `IVectorStore`, etc.). Implementations are swappable.
|
|
2171
|
-
|
|
2172
|
-
2. **Streaming-first** -- Core interaction methods return `AsyncGenerator<AgentOSResponse>` for token-level streaming with natural backpressure. Consumers process chunks as they arrive.
|
|
2173
|
-
|
|
2174
|
-
3. **Extension system** -- Pluggable components via `ExtensionDescriptor` with 12 kinds, priority stacking, lifecycle hooks, and secret management. Extensions can be loaded from npm packages, local modules, or inline factories.
|
|
2175
|
-
|
|
2176
|
-
4. **Multi-provider** -- LLM providers, vector stores, voice providers, and channel adapters all support multiple backend implementations with runtime switching.
|
|
2177
|
-
|
|
2178
|
-
5. **Safety layering** -- Defense-in-depth: input guardrails, output guardrails, circuit breakers, cost guards, tool execution guards, stuck detection, action deduplication, and HITL approval gates.
|
|
2179
|
-
|
|
2180
|
-
6. **Observability** -- OpenTelemetry integration throughout the stack with distributed tracing, custom metrics, cost tracking, and structured logging via pino.
|
|
809
|
+
| [`MEMORY_SCALING.md`](./docs/memory/MEMORY_SCALING.md) | 4-tier vector storage scaling path |
|
|
810
|
+
| [`GUARDRAILS_USAGE.md`](./docs/safety/GUARDRAILS_USAGE.md) | Guardrail implementation patterns |
|
|
811
|
+
| [`SAFETY_PRIMITIVES.md`](./docs/safety/SAFETY_PRIMITIVES.md) | Circuit breaker, cost guard, stuck detection |
|
|
812
|
+
| [`HUMAN_IN_THE_LOOP.md`](./docs/safety/HUMAN_IN_THE_LOOP.md) | Approval workflows, clarification, escalation |
|
|
813
|
+
| [`PLANNING_ENGINE.md`](./docs/orchestration/PLANNING_ENGINE.md) | ReAct reasoning, task planning |
|
|
814
|
+
| [`STRUCTURED_OUTPUT.md`](./docs/orchestration/STRUCTURED_OUTPUT.md) | JSON schema validation, entity extraction |
|
|
815
|
+
| [`AGENT_COMMUNICATION.md`](./docs/architecture/AGENT_COMMUNICATION.md) | Inter-agent messaging and handoffs |
|
|
816
|
+
| [`PLATFORM_SUPPORT.md`](./docs/architecture/PLATFORM_SUPPORT.md) | 37 channel platform capabilities |
|
|
817
|
+
| [`OBSERVABILITY.md`](./docs/observability/OBSERVABILITY.md) | OpenTelemetry setup, tracing, metrics |
|
|
818
|
+
| [`COST_OPTIMIZATION.md`](./docs/safety/COST_OPTIMIZATION.md) | Token usage, caching, model routing |
|
|
819
|
+
| [`SKILLS.md`](./docs/extensions/SKILLS.md) | SKILL.md format, skill authoring guide |
|
|
820
|
+
| [`EMERGENT_CAPABILITIES.md`](./docs/architecture/EMERGENT_CAPABILITIES.md) | Runtime tool forging, tiered promotion |
|
|
821
|
+
| [`SQL_STORAGE_QUICKSTART.md`](./docs/getting-started/SQL_STORAGE_QUICKSTART.md) | SQLite/Postgres setup |
|
|
822
|
+
| [`ECOSYSTEM.md`](./docs/architecture/ECOSYSTEM.md) | Extension ecosystem and official packs |
|
|
2181
823
|
|
|
2182
824
|
---
|
|
2183
825
|
|
|
@@ -2191,8 +833,6 @@ pnpm run build
|
|
|
2191
833
|
pnpm run test
|
|
2192
834
|
```
|
|
2193
835
|
|
|
2194
|
-
**Available scripts:**
|
|
2195
|
-
|
|
2196
836
|
| Script | Purpose |
|
|
2197
837
|
|--------|---------|
|
|
2198
838
|
| `pnpm run build` | Clean, compile TypeScript, resolve aliases, fix ESM imports |
|
|
@@ -2222,6 +862,6 @@ See the [Contributing Guide](https://github.com/framersai/agentos/blob/master/CO
|
|
|
2222
862
|
<img src="https://raw.githubusercontent.com/framersai/agentos/master/assets/frame-logo-green-no-tagline.svg" alt="Frame.dev" height="40" />
|
|
2223
863
|
</a>
|
|
2224
864
|
|
|
2225
|
-
**Built by [Frame.dev](https://frame.dev)**
|
|
865
|
+
**Built by [Frame.dev](https://frame.dev)**
|
|
2226
866
|
|
|
2227
867
|
</div>
|