beddel 0.2.3 → 1.0.0

package/README.md

@@ -1,662 +1,205 @@
-# Beddel
+# Beddel Protocol
 
-**Secure, Declarative, and Extensible Agent Runtime**
+[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
+[![npm version](https://img.shields.io/badge/npm-1.0.0-brightgreen.svg)](https://www.npmjs.com/package/beddel)
+[![TypeScript](https://img.shields.io/badge/TypeScript-5.x-blue.svg)](https://www.typescriptlang.org/)
+[![AI SDK](https://img.shields.io/badge/AI%20SDK-v6-purple.svg)](https://sdk.vercel.ai/)
 
-[![NPM Version](https://img.shields.io/npm/v/beddel.svg)](https://www.npmjs.com/package/beddel)
-[![License](https://img.shields.io/npm/l/beddel.svg)](https://github.com/botanarede/beddel-alpha/blob/main/LICENSE)
+**Beddel Protocol** is a declarative **Sequential Pipeline Executor** that parses YAML workflow definitions and executes steps sequentially. Built on the Vercel AI SDK v6, it provides native streaming support and extensible primitives.
 
-Beddel is a security-first toolkit for building and running declarative YAML agents with enterprise-grade isolation, automatic schema validation, and extensible architecture.
+## Features
 
-## Quick Start
-
-```yaml
-# agents/my-agent.yaml
-agent:
-  id: my-agent
-  version: 1.0.0
-  protocol: beddel-declarative-protocol/v2.0
-
-metadata:
-  name: "My Custom Agent"
-  description: "A simple custom agent"
-  route: "/agents/my-agent"
-
-schema:
-  input:
-    type: "object"
-    properties:
-      message:
-        type: "string"
-    required: ["message"]
-  output:
-    type: "object"
-    properties:
-      response:
-        type: "string"
-    required: ["response"]
-
-logic:
-  workflow:
-    - name: "process"
-      type: "genkit-joke"
-      action:
-        type: "joke"
-        prompt: "{{message}}"
-        result: "result"
-
-output:
-  schema:
-    response: "$result.texto"
-```
-
-```typescript
-import { agentRegistry } from "beddel";
-
-// Execute the agent
-const result = await agentRegistry.executeAgent(
-  "my-agent.execute",
-  { message: "Hello world" },
-  { gemini_api_key: process.env.GEMINI_API_KEY },
-  context
-);
-```
+- 🔄 **Sequential Pipeline Execution** — Define workflows as YAML, execute steps in order
+- 🌊 **Native Streaming** — First-class `streamText` support with `useChat` compatibility
+- 🔌 **Extensible Primitives** — Register custom step types, tools, and callbacks
+- 🔒 **Security First** — YAML parsing with `FAILSAFE_SCHEMA` prevents code execution
+- 📦 **Bundle Separation** — Three entry points for server, client, and full API access
 
 ## Installation
 
 ```bash
 npm install beddel
+# or
+pnpm add beddel
+# or
+yarn add beddel
 ```
 
-Requires Node.js 18+.
-
----
-
-## Architecture Overview
-
-```
-┌─────────────────────────────────────────────────────────────────────┐
-│                         Beddel Runtime                              │
-│                                                                     │
-│  ┌──────────────────────────────────────────────────────────────┐  │
-│  │                    Agent Registry                             │  │
-│  │                                                               │  │
-│  │  ┌─────────────────┐    ┌────────────────────────────────┐   │  │
-│  │  │  Built-in       │    │  Custom Agents                 │   │  │
-│  │  │  Agents         │    │  /agents/*.yaml                │   │  │
-│  │  │                 │    │                                │   │  │
-│  │  │ • joker         │    │  Automatically discovered      │   │  │
-│  │  │ • translator    │    │  and registered at startup     │   │  │
-│  │  │ • image         │    │                                │   │  │
-│  │  └─────────────────┘    └────────────────────────────────┘   │  │
-│  │                                                               │  │
-│  │         Priority: Custom Agents > Built-in Agents            │  │
-│  └──────────────────────────────────────────────────────────────┘  │
-│                                │                                    │
-│                                ▼                                    │
-│  ┌──────────────────────────────────────────────────────────────┐  │
-│  │              Declarative Agent Interpreter                    │  │
-│  │                                                               │  │
-│  │   • YAML Parsing (FAILSAFE schema)                           │  │
-│  │   • Schema Validation (Zod)                                  │  │
-│  │   • Workflow Execution                                       │  │
-│  │   • Genkit Integration (Gemini Flash)                        │  │
-│  └──────────────────────────────────────────────────────────────┘  │
-│                                │                                    │
-│                                ▼                                    │
-│  ┌──────────────────────────────────────────────────────────────┐  │
-│  │               Secure Runtime (isolated-vm)                    │  │
-│  │                                                               │  │
-│  │   • Memory limits         • V8 isolate                       │  │
-│  │   • Execution timeouts    • No Node.js access                │  │
-│  │   • Audit logging         • Multi-tenant isolation           │  │
-│  └──────────────────────────────────────────────────────────────┘  │
-└─────────────────────────────────────────────────────────────────────┘
-```
-
----
-
-## Custom Agents
-
-Beddel supports custom agents that you define in your application's `/agents` directory. Custom agents are automatically discovered and registered at startup.
-
-### Directory Structure
-
-```
-your-app/
-├── agents/                          # Custom agents directory
-│   ├── my-agent.yaml                # Simple YAML-only agent
-│   ├── my-chat.yaml                 # Agent with TypeScript code-behind
-│   ├── my-chat.ts                   # TypeScript implementation  
-│   ├── calculator/
-│   │   └── calculator.yaml          # Agent in subdirectory
-│   └── translator-custom/
-│       └── translator-custom.yaml   # Override built-in
-│
-└── packages/beddel/src/agents/      # Built-in agents (sharded structure)
-    ├── joker/
-    │   ├── joker.yaml               # Agent definition
-    │   ├── joker.handler.ts         # Server-only handler
-    │   ├── joker.schema.ts          # Zod validation
-    │   ├── joker.types.ts           # TypeScript types
-    │   └── index.ts                 # Public exports
-    ├── translator/
-    │   └── ...
-    ├── image/
-    │   └── ...
-    ├── mcp-tool/
-    │   └── ...
-    ├── gemini-vectorize/
-    │   └── ...
-    ├── chromadb/
-    │   └── ...
-    ├── gitmcp/
-    │   └── ...
-    ├── rag/
-    │   └── ...
-    └── chat/
-        └── ...
-```
-
-### Agent Loading Flow
-
-```
-┌─────────────────────────────────────────────────────────────┐
-│                  Agent Loading Sequence                      │
-│                                                              │
-│  1. AgentRegistry constructor()                              │
-│     │                                                        │
-│     ├──▶ 2. registerBuiltinAgents()                         │
-│     │       ├── joker.execute                                │
-│     │       ├── translator.execute                           │
-│     │       └── image.generate                               │
-│     │                                                        │
-│     └──▶ 3. loadCustomAgents()                              │
-│             │                                                │
-│             ├── Recursively scans /agents/**/*.yaml         │
-│             │   └── Registers each agent                     │
-│             │       └── Custom agents override built-ins     │
-│             │                                                │
-│             └── Scans /agents/**/*.ts                        │
-│                 └── Dynamically imports TypeScript modules   │
-│                     └── Registers exported functions         │
-│                                                              │
-│  Priority: Custom Agents > Built-in Agents                   │
-└─────────────────────────────────────────────────────────────┘
-```
-
-### Creating a Custom Agent
-
-1. **Create the YAML file** in `/agents`:
-
-```yaml
-# agents/greeting.yaml
-agent:
-  id: greeting
-  version: 1.0.0
-  protocol: beddel-declarative-protocol/v2.0
-
-metadata:
-  name: "Greeting Agent"
-  description: "Generates personalized greetings"
-  category: "utility"
-  route: "/agents/greeting"
-
-schema:
-  input:
-    type: "object"
-    properties:
-      name:
-        type: "string"
-        minLength: 1
-        maxLength: 100
-    required: ["name"]
-
-  output:
-    type: "object"
-    properties:
-      greeting:
-        type: "string"
-    required: ["greeting"]
-
-logic:
-  workflow:
-    - name: "generate-greeting"
-      type: "genkit-joke"
-      action:
-        type: "joke"
-        prompt: "Create a warm, friendly greeting for {{name}}"
-        result: "greetingResult"
-
-output:
-  schema:
-    greeting: "$greetingResult.texto"
-```
-
-2. **The agent is automatically registered** when your application starts.
-
-3. **Execute via GraphQL or directly** (GraphQL Yoga, introspection always enabled). GraphiQL is available at `GET /api/graphql`:
-
-```graphql
-mutation Execute($methodName: String!, $params: JSON!, $props: JSON!) {
-  executeMethod(methodName: $methodName, params: $params, props: $props) {
-    success
-    data
-    error
-    executionTime
-  }
-}
-```
-
-Example variables:
-
-```json
-{
-  "methodName": "greeting.execute",
-  "params": { "name": "Alice" },
-  "props": { "gemini_api_key": "your-api-key" }
-}
-```
-
-### Custom Agents with TypeScript Code-Behind
-
-For more complex logic, create custom agents with TypeScript implementations:
+## Quick Start
 
-#### 1. Create the TypeScript implementation:
+### 1. Create API Route
 
 ```typescript
-// agents/my-chat.ts
-export const chatHandler = async ({ input, variables, context }) => {
-  // Full TypeScript power available here
-  context.log("Processing chat message with custom TypeScript logic");
-  
-  const message = input.message || "";
-  const history = input.messages || [];
-  
-  // Add user message to history
-  history.push({ role: "user", content: message });
-  
-  // Your custom logic here - call external APIs, use Beddel helpers, etc.
-  const response = `Processed: "${message}"`;
-  
-  history.push({ role: "assistant", content: response });
-  
-  return { 
-    response,
-    history,
-    timestamp: new Date().toISOString()
-  };
-};
-```
-
-#### 2. Create the YAML configuration:
+// app/api/beddel/chat/route.ts
+import { createBeddelHandler } from 'beddel/server';
 
-```yaml
-# agents/my-chat.yaml
-agent:
-  id: my-chat
-  version: 1.0.0
-  protocol: beddel-declarative-protocol/v2.0
-
-metadata:
-  name: "Custom Chat Agent"
-  description: "Chat agent with TypeScript code-behind"
-  route: "/agents/my-chat"
-
-schema:
-  input:
-    type: "object"
-    properties:
-      message:
-        type: "string"
-        minLength: 1
-    required: ["message"]
-
-  output:
-    type: "object"
-    properties:
-      response:
-        type: "string"
-    required: ["response"]
-
-logic:
-  workflow:
-    - name: process
-      type: custom-action
-      action:
-        function: "my-chat/chatHandler"  # Format: filename/exportedFunctionName
-        result: response
-
-output:
-  schema:
-    response: "$response.response"
-```
-
-#### 3. Function Arguments
-
-Custom functions receive a standardized arguments object:
-
-```typescript
-{
-  input: Record<string, any>,      // Validated input from the request
-  variables: Record<string, any>,  // Current workflow variables
-  action: Record<string, any>,     // Action configuration from YAML
-  context: ExecutionContext        // Logging and error handling
-}
+export const POST = createBeddelHandler({
+  agentsPath: 'src/agents'  // Optional, default: 'src/agents'
+});
 ```
 
-### Overriding Built-in Agents
-
-Create a custom agent with the same route to override a built-in:
+### 2. Create YAML Agent
 
 ```yaml
-# agents/joker-custom.yaml
-agent:
-  id: joker
-  protocol: beddel-declarative-protocol/v2.0
-
+# src/agents/assistant.yaml
 metadata:
-  route: "/agents/joker"  # Same route overrides built-in
-  # ... your custom implementation
-```
-
----
-
-## Built-in Agents
-
-| Agent | Method | Description | Required Props |
-|-------|--------|-------------|----------------|
-| **Joker** | `joker.execute` | Generates short, original jokes | `gemini_api_key` |
-| **Translator** | `translator.execute` | Translates text between languages | `gemini_api_key` |
-| **Image Generator** | `image.generate` | Creates images (watercolor, neon, sketch) | `gemini_api_key` |
+  name: "Streaming Assistant"
+  version: "1.0.0"
 
-### Joker Agent
-
-```typescript
-const result = await agentRegistry.executeAgent(
-  "joker.execute",
-  {},
-  { gemini_api_key: "..." },
-  context
-);
-// Returns: { response: "Why did the..." }
+workflow:
+  - id: "chat-interaction"
+    type: "llm"
+    config:
+      model: "gemini-2.0-flash-exp"
+      stream: true
+      system: "You are a helpful assistant."
+      messages: "$input.messages"
 ```
 
-### Translator Agent
+### 3. Set Environment Variable
 
-```typescript
-const result = await agentRegistry.executeAgent(
-  "translator.execute",
-  {
-    texto: "Hello, world!",
-    idioma_origem: "en",
-    idioma_destino: "pt"
-  },
-  { gemini_api_key: "..." },
-  context
-);
-// Returns: { texto_traduzido: "Olá, mundo!", metadados: {...} }
+```bash
+GEMINI_API_KEY=your_api_key_here
+```
+
+### 4. Use with React (useChat)
+
+```tsx
+'use client';
+import { useChat } from '@ai-sdk/react';
+
+export default function Chat() {
+  const { messages, input, handleInputChange, handleSubmit } = useChat({
+    api: '/api/beddel/chat',
+    body: { agentId: 'assistant' },
+  });
+
+  return (
+    <div>
+      {messages.map((m) => (
+        <div key={m.id}>{m.role}: {m.content}</div>
+      ))}
+      <form onSubmit={handleSubmit}>
+        <input value={input} onChange={handleInputChange} />
+        <button type="submit">Send</button>
+      </form>
+    </div>
+  );
+}
 ```
 
-### Image Generator Agent
-
-```typescript
-const result = await agentRegistry.executeAgent(
-  "image.generate",
-  {
-    descricao: "A sunset over mountains",
-    estilo: "watercolor",
-    resolucao: "1024x1024"
-  },
-  { gemini_api_key: "..." },
-  context
-);
-// Returns: { image_url, image_base64, media_type, ... }
-```
+## Entry Points
 
----
+| Import Path | Purpose | Environment |
+|-------------|---------|-------------|
+| `beddel` | Full API: `loadYaml`, `WorkflowExecutor`, registries | Server only |
+| `beddel/server` | `createBeddelHandler` for Next.js API routes | Server only |
+| `beddel/client` | Type-only exports (browser-safe) | Client/Server |
 
-## Core Components
+> ⚠️ **Important:** Never import `beddel` or `beddel/server` in client components. Use `beddel/client` for type imports.
 
-### Package Map
+## Extensibility
 
-| Capability | Module | Description |
-|------------|--------|-------------|
-| **YAML Parsing** | `SecureYamlParser` | FAILSAFE schema, depth/size limits, UTF-8 validation |
-| **Sandboxed Execution** | `IsolatedRuntimeManager` | `isolated-vm` with security profiles |
-| **Agent Interpretation** | `DeclarativeAgentInterpreter` | Executes YAML agents with Genkit |
-| **Agent Registry** | `AgentRegistry` | Manages built-in and custom agents |
-| **Security** | `SecurityScanner`, `ThreatDetectionEngine` | Static scanning, scoring |
-| **Compliance** | `GDPRCompliance`, `LGPDCompliance` | Audit, anonymization |
-| **Performance** | `PerformanceMonitor`, `AutoScaler` | Metrics, scaling |
-| **Multi-tenancy** | `TenantManager`, `ITenantProvider` | Provider-agnostic tenant isolation |
+Beddel follows the **Expansion Pack Pattern** for extensibility:
 
-### Secure YAML Parsing
+### Register Custom Primitives
 
 ```typescript
-import { SecureYamlParser } from "beddel";
+import { registerPrimitive } from 'beddel';
 
-const parser = new SecureYamlParser({
-  maxDepth: 200,
-  filename: "agent-manifest",
+registerPrimitive('http-fetch', async (config, context) => {
+  const response = await fetch(config.url);
+  return { data: await response.json() };
 });
-
-const manifest = parser.parseSecure(yamlContent);
 ```
 
-### Sandboxed Execution
+### Register Custom Tools
 
 ```typescript
-import { IsolatedRuntimeManager } from "beddel";
+import { registerTool } from 'beddel';
+import { z } from 'zod';
 
-const runtime = new IsolatedRuntimeManager();
-
-const result = await runtime.execute({
-  code: `({ value: 42 })`,
-  securityProfile: "ultra-secure",
-  timeout: 2000,
+registerTool('weatherLookup', {
+  description: 'Get weather for a city',
+  parameters: z.object({ city: z.string() }),
+  execute: async ({ city }) => fetchWeather(city),
 });
 ```
 
-### Agent Registry API
+### Register Lifecycle Callbacks
 
 ```typescript
-import { agentRegistry } from "beddel";
-
-// Get all registered agents
-const agents = agentRegistry.getAllAgents();
+import { registerCallback } from 'beddel';
 
-// Get a specific agent
-const agent = agentRegistry.getAgent("joker.execute");
-
-// Execute an agent
-const result = await agentRegistry.executeAgent(
-  "joker.execute",
-  input,
-  props,
-  context
-);
-
-// Load custom agents from a specific path
-agentRegistry.loadCustomAgents("/path/to/custom/agents");
+registerCallback('persistConversation', async ({ text, usage }) => {
+  await db.saveMessage(text, usage);
+});
 ```
 
----
-
-## Schema Validation
-
-All agents must declare `schema.input` and `schema.output` blocks. The runtime compiles these into Zod schemas and validates:
-
-- **Before execution**: Input must match `schema.input`
-- **After execution**: Output must match `schema.output`
+## YAML Workflow Structure
 
 ```yaml
-schema:
-  input:
-    type: "object"
-    properties:
-      name:
-        type: "string"
-        minLength: 1
-        maxLength: 100
-      age:
-        type: "number"
-    required: ["name"]
-
-  output:
-    type: "object"
-    properties:
-      greeting:
-        type: "string"
-    required: ["greeting"]
-```
-
-Supported types: `string`, `number`, `boolean`, `object`, `array`
-
----
-
-## Multi-Tenant Architecture
-
-Beddel provides a provider-agnostic multi-tenant system with a dynamic registry for external providers.
-
-### Built-in Providers
-
-| Provider | Type | Use Case |
-|----------|------|----------|
-| `InMemoryTenantProvider` | `memory` | Testing, development |
-
-### External Providers
-
-Provider implementations that handle credentials (Firebase, Supabase, etc.) should be implemented in the consuming application. Use `ProviderRegistry` to register custom providers:
-
-```typescript
-import { ProviderRegistry } from "beddel";
-import { FirebaseTenantProvider } from "./providers/FirebaseTenantProvider";
+metadata:
+  name: "Agent Name"
+  version: "1.0.0"
 
-// Register during application bootstrap
-ProviderRegistry.register('firebase', () => new FirebaseTenantProvider());
+workflow:
+  - id: "step-1"
+    type: "llm"
+    config:
+      model: "gemini-2.0-flash-exp"
+      stream: true
+      system: "System prompt"
+      messages: "$input.messages"
+      tools:
+        - name: "calculator"
+      onFinish: "callbackName"
+    result: "stepOutput"
 ```
 
-### Quick Start
+### Variable Resolution
 
-```typescript
-import { TenantManager } from "beddel";
-
-const manager = TenantManager.getInstance();
-
-// Initialize tenant
-const result = await manager.initializeTenant({
-  tenantId: "tenant-123",
-  securityProfile: "tenant-isolated",
-  dataRetentionDays: 365,
-  lgpdEnabled: true,
-  gdprEnabled: true,
-  provider: "memory", // or registered external provider like "firebase"
-  providerConfig: {}
-});
+| Pattern | Description | Example |
+|---------|-------------|---------|
+| `$input.*` | Access request input | `$input.messages` |
+| `$stepResult.varName.*` | Access step result | `$stepResult.llmOutput.text` |
 
-// Execute in tenant context
-await manager.executeInTenant("tenant-123", "operation", data, async () => {
-  const app = manager.getTenantApp("tenant-123");
-  const db = app.getDatabase();
-  await db.collection("users").add({ name: "Alice" });
-});
-```
+## Built-in Tools
 
-### Core Interfaces
+| Tool | Description |
+|------|-------------|
+| `calculator` | Evaluate mathematical expressions |
+| `getCurrentTime` | Get current ISO timestamp |
 
-- `ITenantProvider` — Abstract provider for tenant management
-- `ITenantApp` — Isolated tenant instance
-- `ITenantDatabase` — Database operations
-- `ITenantCollection` — Collection operations
-- `ITenantDocument` — Document CRUD
-- `ProviderRegistry` — Dynamic provider registration
+## AI SDK v6 Compatibility
 
----
+Beddel is fully compatible with Vercel AI SDK v6:
 
-## Security
+- **Frontend:** `useChat` sends `UIMessage[]` with `{ parts: [...] }` format
+- **Backend:** `streamText`/`generateText` expects `ModelMessage[]` with `{ content: ... }`
+- **Automatic Conversion:** `convertToModelMessages()` bridges the gap
+- **Streaming:** `toUIMessageStreamResponse()` returns the correct format for `useChat`
 
-### Multi-Layer Security Model
+## Technology Stack
 
-1. **Parser Layer**: FAILSAFE schema, size limits, sanitization
-2. **Runtime Layer**: V8 isolate, memory limits, timeouts
-3. **Application Layer**: Authentication, rate limiting, tenant isolation
+| Category | Technology | Version |
+|----------|------------|---------|
+| Language | TypeScript | 5.x |
+| Runtime | Node.js / Edge | 20+ |
+| AI Core | `ai` | 6.x |
+| AI Provider | `@ai-sdk/google` | 3.x |
+| Validation | `zod` | 3.x |
+| YAML Parser | `js-yaml` | 4.x |
 
-### Security Profiles
+## Documentation
 
-```typescript
-const result = await runtime.execute({
-  code: userCode,
-  securityProfile: "tenant-isolated", // ultra-secure | tenant-isolated
-  timeout: 5000,
-  memoryLimit: 128,
-});
-```
-
----
-
-## Project Structure
-
-```
-src/
-├── agents/                  # Agent registry and sharded agent modules
-│   ├── registry/            # Agent registry
-│   │   ├── agentRegistry.ts
-│   │   └── index.ts
-│   ├── joker/               # Joker agent
-│   │   ├── joker.yaml
-│   │   ├── joker.handler.ts
-│   │   ├── joker.schema.ts
-│   │   ├── joker.types.ts
-│   │   └── index.ts
-│   ├── translator/          # Translator agent
-│   ├── image/               # Image generator agent
-│   ├── mcp-tool/            # MCP tool agent
-│   ├── gemini-vectorize/    # Embeddings agent
-│   ├── chromadb/            # Vector storage agent
-│   ├── gitmcp/              # GitHub docs agent
-│   ├── rag/                 # RAG agent
-│   ├── chat/                # Chat orchestrator agent
-│   └── index.ts             # Public exports
-├── runtime/                 # Execution environments
-│   ├── declarativeAgentRuntime.ts
-│   ├── workflowExecutor.ts  # Workflow step execution
-│   ├── schemaCompiler.ts
-│   └── index.ts
-├── shared/                  # Client-safe types and utilities
-│   ├── types/
-│   └── utils/
-├── client/                  # Client-safe exports
-├── parser/                  # Secure YAML parsing
-├── security/                # Scanning and threat detection
-├── compliance/              # GDPR/LGPD utilities
-├── performance/             # Monitoring and autoscaling
-├── tenant/                  # Provider-agnostic multi-tenant management
-│   ├── TenantManager.ts     # Singleton tenant orchestrator
-│   ├── interfaces.ts        # ITenantProvider, ITenantApp, etc.
-│   ├── providerFactory.ts   # Provider instantiation
-│   ├── providerRegistry.ts  # Dynamic provider registration
-│   └── providers/           # Built-in provider implementations
-│       └── InMemoryTenantProvider.ts
-└── index.ts                 # Public exports
-```
-
----
-
-## Development
-
-```bash
-# Build
-pnpm --filter beddel build
-
-# Test
-pnpm --filter beddel test
-
-# Lint
-pnpm --filter beddel lint
-```
+Detailed documentation is available in [`docs/`](./docs/):
 
----
+- [API Reference](./docs/architecture/api-reference.md)
+- [Components](./docs/architecture/components.md)
+- [Core Workflows](./docs/architecture/core-workflows.md)
+- [High-Level Architecture](./docs/architecture/high-level-architecture.md)
 
 ## License
 
-MIT License. See [LICENSE](LICENSE) for details.
+[MIT](LICENSE)