@memgrafter/flatagents 0.8.1 → 0.9.0

package/MACHINES.md

@@ -0,0 +1,103 @@
+# FlatAgents + FlatMachines Reference
+
+> **Target: <1000 tokens.** LLM-optimized. See `flatagent.d.ts`, `flatmachine.d.ts`, `profiles.d.ts` for schemas.
+>
+> **Versioning:** All specs and SDKs use lockstep versioning.
+
+## Concepts
+
+**FlatAgent**: Single LLM call. Model + prompts + output schema. No orchestration.
+**FlatMachine**: State machine orchestrating agents. States, transitions, conditions, loops, error handling.
+
+| Need | Use |
+|------|-----|
+| Single LLM call | FlatAgent |
+| Multi-step/branching/retry/errors | FlatMachine |
+| Parallel execution | `machine: [a, b, c]` |
+| Dynamic parallelism | `foreach` |
+| Background tasks | `launch` |
+
+## Model Profiles
+
+```yaml
+# profiles.yml — agents reference by name
+spec: flatprofiles
+spec_version: "0.9.0"
+data:
+  model_profiles:
+    fast: { provider: cerebras, name: zai-glm-4.6, temperature: 0.6 }
+    smart: { provider: anthropic, name: claude-3-opus-20240229 }
+  default: fast        # Fallback
+  # override: smart    # Force all
+```
+
+Agent model field: `"fast"` | `{ profile: "fast", temperature: 0.9 }` | `{ provider: x, name: y }`
+Resolution: default → profile → overrides → override
+
+## State Fields
+
+| Field | Purpose |
+|-------|---------|
+| `type` | `initial` (entry) / `final` (exit+output) |
+| `agent` | Agent to call |
+| `machine` | Machine(s) — string or `[array]` for parallel |
+| `foreach` | Array expr for dynamic parallelism (`as`: item var, `key`: result key) |
+| `launch` / `launch_input` | Fire-and-forget machine(s) |
+| `input` | Map input to agent/machine |
+| `output_to_context` | Map `output.*` to `context.*` |
+| `execution` | `{ type: retry, backoffs: [2,8,16], jitter: 0.1 }` |
+| `on_error` | State name or `{ default: x, ErrorType: y }` |
+| `transitions` | `[{ condition: "expr", to: state }, { to: default }]` |
+| `mode` | `settled` (all) / `any` (first) for parallel |
+| `timeout` | Seconds (0=forever) |
+
+## Patterns
+
+**Execution types**: `default` | `retry` (backoffs, jitter) | `parallel` (n_samples) | `mdap_voting` (k_margin, max_candidates)
+
+**Transitions**: `condition: "context.score >= 8"` with `to: state`. Last without condition = default.
+
+**Loops**: Transition `to: same_state`. Machine has `max_steps` safety.
+
+**Errors**: `on_error: state` or per-type. Context gets `last_error`, `last_error_type`.
+
+**Parallel machines**:
+```yaml
+machine: [review_a, review_b]  # Results keyed by name
+mode: settled  # or "any"
+```
+
+**Foreach**:
+```yaml
+foreach: "{{ context.items }}"
+as: item
+machine: processor
+```
+
+**Launch** (fire-and-forget):
+```yaml
+launch: background_task
+launch_input: { data: "{{ context.data }}" }
+```
+
+## Context Variables
+
+`context.*` (all states), `input.*` (initial), `output.*` (in output_to_context), `item`/`as` (foreach)
+
+## Hooks
+
+`on_machine_start`, `on_machine_end`, `on_state_enter`, `on_state_exit`, `on_transition`, `on_error`, `on_action`
+
+```python
+class MyHooks(MachineHooks):
+    def on_action(self, action: str, context: dict) -> dict:
+        if action == "fetch": context["data"] = api_call()
+        return context
+```
+
+## Persistence
+
+```yaml
+persistence: { enabled: true, backend: local }  # local | memory
+```
+Resume: `machine.execute(resume_from=execution_id)`

package/README.md

@@ -1,221 +1,323 @@
-# FlatAgents TypeScript SDK
+# FlatAgents
 
-TypeScript SDK for FlatAgents - Declarative LLM orchestration with YAML.
+Define LLM agents in YAML. Run them anywhere.
 
-## Installation
+**For LLM/machine readers:** see [MACHINES.md](./MACHINES.md) for comprehensive reference.
 
-```bash
-npm install flatagents
-```
+## Why?
 
-## Quick Start
+- **Composition over inheritance** — compose stateless agents and checkpointable machines
+- **Compact structure** — easy for LLMs to read and generate
+- **Simple hook interfaces** — escape hatches without complexity; webhook ready
+- **Inspectable** — every agent and machine is readable config
+- **Language-agnostic** — reduce code in any particular runtime
+- **Common TypeScript interface** — single schema for agents, single schema for machines
+- **Limitations** — machine topologies can get complex at scale
 
-### Single Agent Call
+*Inspired by Kubernetes manifests and character card specifications.*
 
-```typescript
-import { FlatAgent } from 'flatagents';
+## Versioning
 
-const agent = new FlatAgent('agent.yml');
-const result = await agent.call({ query: "Hello World" });
-console.log(result.output);
-```
+All specs (`flatagent.d.ts`, `flatmachine.d.ts`, `profiles.d.ts`) and SDKs (Python, JS) use **lockstep versioning**. A single version number applies across the entire repository.
 
-### State Machine Execution
+## Core Concepts
+
+Use machines to write flatagents and flatmachines, they are designed for LLMs.
 
-```typescript
-import { FlatMachine } from 'flatagents';
+| Term | What it is |
+|------|------------|
+| **FlatAgent** | A single LLM call: model + prompts + output schema |
+| **FlatMachine** | A state machine that orchestrates multiple agents, actions, and state machines |
 
-const machine = new FlatMachine({
-  config: 'machine.yml',
-  hooks: customHooks,
-  persistence: new MemoryBackend()
-});
+Use FlatAgent alone for simple tasks. Use FlatMachine when you need multi-step workflows, branching, or error handling.
+
+## Examples
 
-const result = await machine.execute({ input: "Hello" });
-console.log(result);
+| Example | What it demonstrates |
+|---------|---------------------|
+| [helloworld](./sdk/examples/helloworld/python) | Minimal setup — single agent, single state machine |
+| [writer_critic](./sdk/examples/writer_critic/python) | Multi-agent loop — writer drafts, critic reviews, iterates |
+| [story_writer](./sdk/examples/story_writer/python) | Multi-step creative workflow with chapter generation |
+| [human-in-the-loop](./sdk/examples/human-in-the-loop/python) | Pause execution for human approval via hooks |
+| [error_handling](./sdk/examples/error_handling/python) | Error recovery and retry patterns at state machine level |
+| [dynamic_agent](./sdk/examples/dynamic_agent/python) | On-the-fly agent generation from runtime context |
+| [character_card](./sdk/examples/character_card/python) | Loading agent config from character card format |
+| [mdap](./sdk/examples/mdap/python) | MDAP voting execution — multi-sample consensus |
+| [gepa_self_optimizer](./sdk/examples/gepa_self_optimizer/python) | Self-optimizing prompts via reflection and critique |
+| [research_paper_analysis](./sdk/examples/research_paper_analysis/python) | Document analysis with structured extraction |
+| [multi_paper_synthesizer](./sdk/examples/multi_paper_synthesizer/python) | Cross-document synthesis with dynamic machine launching |
+| [support_triage_json](./sdk/examples/support_triage_json/python) | JSON input/output with classification pipeline |
+| [parallelism](./sdk/examples/parallelism/python) | Parallel machines, dynamic foreach, fire-and-forget launches |
+
+## Quick Start
+
+```bash
+pip install flatagents[all]
 ```
 
-## Core Concepts
+```python
+from flatagents import FlatAgent
+
+agent = FlatAgent(config_file="reviewer.yml")
+result = await agent.call(query="Review this code...")
+print(result.output)
+```
 
-### FlatAgent
-A single LLM call configured in YAML:
+## Example Agent
 
+**reviewer.yml**
 ```yaml
 spec: flatagent
-spec_version: "1.0"
+spec_version: "0.8.2"
+
 data:
-  name: my_agent
-  model:
-    name: gpt-4o-mini
-    provider: openai
-  system: "You are a helpful assistant."
-  user: "{{ input.query }}"
+  name: code-reviewer
+
+  model: "smart-expensive"  # Reference profile from profiles.yml
+
+  system: |
+    You are a senior code reviewer. Analyze code for bugs,
+    style issues, and potential improvements.
+
+  user: |
+    Review this code:
+    {{ input.code }}
+
   output:
-    response:
+    issues:
+      type: list
+      items:
+        type: str
+      description: "List of issues found"
+    rating:
       type: str
-      description: "The response"
+      enum: ["good", "needs_work", "critical"]
+      description: "Overall code quality"
 ```
 
-### FlatMachine
-A state machine that orchestrates agents:
+**What the fields mean:**
 
-```yaml
-spec: flatmachine
-spec_version: "1.0"
-data:
-  name: my_workflow
-  context:
-    result: ""
-  states:
-    initial:
-      type: initial
-      agent: agent.yml
-      transitions:
-        - to: final
-    final:
-      type: final
-      output:
-        result: "{{ context.result }}"
-```
+- **spec/spec_version** — Format identifier and version
+- **data.name** — Agent identifier
+- **data.model** — Profile name, inline config, or profile with overrides
+- **data.system** — System prompt (sets behavior)
+- **data.user** — User prompt template (uses Jinja2, `{{ input.* }}` for runtime values)
+- **data.output** — Structured output schema (the runtime extracts these fields)
 
-## Key Features
+## Model Profiles
 
-### Parallel Execution
+Centralize model configurations in `profiles.yml` and reference them by name:
+
+**profiles.yml**
 ```yaml
-states:
-  parallel_review:
-    machine: [legal_review, tech_review, finance_review]
-    transitions:
-      - to: synthesize
+spec: flatprofiles
+spec_version: "0.8.2"
+
+data:
+  model_profiles:
+    fast-cheap:
+      provider: cerebras
+      name: zai-glm-4.6
+      temperature: 0.6
+      max_tokens: 2048
+
+    smart-expensive:
+      provider: anthropic
+      name: claude-3-opus-20240229
+      temperature: 0.3
+      max_tokens: 4096
+
+  default: fast-cheap      # Fallback when agent has no model
+  # override: smart-expensive  # Uncomment to force all agents
 ```
 
-### Dynamic Parallelism (Foreach)
+**Agent usage:**
 ```yaml
-states:
-  process_all:
-    foreach: "{{ context.documents }}"
-    as: doc
-    machine: processor.yml
-    transitions:
-      - to: aggregate
+# String shorthand — profile lookup
+model: "fast-cheap"
+
+# Profile with overrides
+model:
+  profile: "fast-cheap"
+  temperature: 0.9
+
+# Inline config (no profile)
+model:
+  provider: openai
+  name: gpt-4
+  temperature: 0.3
 ```
 
-### Retry with Backoff
+Resolution order (low → high): default profile → named profile → inline overrides → override profile
+
+## Output Types
+
 ```yaml
-states:
-  robust_call:
-    agent: agent.yml
-    execution:
-      type: retry
-      backoffs: [2, 8, 16, 35]
-      jitter: 0.1
+output:
+  answer:      { type: str }
+  count:       { type: int }
+  score:       { type: float }
+  valid:       { type: bool }
+  raw:         { type: json }
+  items:       { type: list, items: { type: str } }
+  metadata:    { type: object, properties: { key: { type: str } } }
 ```
 
-### Conditional Transitions
-```yaml
-states:
-  check_result:
-    agent: evaluator.yml
-    transitions:
-      - condition: "context.score >= 8"
-        to: success
-      - to: retry
+Use `enum: [...]` to constrain string values.
+
+## Multi-Agent Workflows
+
+For orchestration, use FlatMachine ([full docs in MACHINES.md](./MACHINES.md)):
+
+```python
+from flatagents import FlatMachine
+
+machine = FlatMachine(config_file="workflow.yml")
+result = await machine.execute(input={"query": "..."})
 ```
 
-### Error Handling
-```yaml
-states:
-  risky_state:
-    agent: agent.yml
-    on_error: error_handler
+FlatMachine provides: state transitions, conditional branching, loops, retry with backoff, and error recovery—all in YAML.
+
+## Features
+
+- Checkpoint and restore
+- Python SDK (TypeScript SDK in progress)
+- [MACHINES.md](./MACHINES.md) — LLM-optimized reference docs
+- Decider agents and machines
+- On-the-fly agent and machine definitions
+- Webhook hooks for remote state machine handling
+- Metrics and logging
+- Error recovery and exception handling at the state machine level
+- Parallel machine execution (`machine: [a, b, c]`)
+- Dynamic parallelism with `foreach`
+- Fire-and-forget launches for background tasks
+
+## Planned
+
+- Distributed execution — cross-network machine peering, inter-machine strategies
+- SQL persistence backend
+- TypeScript SDK
+- `max_depth` config to limit machine launch nesting
+- Checkpoint pruning to prevent storage explosion
+- `$root/` path prefix — resolve agent/machine refs from workspace root, not config dir
+- Input size validation — warn when prompt exceeds model context window
+- Serialization warnings — flag non-JSON-serializable context values before checkpoint
+
+## Specs
+
+TypeScript definitions are the source of truth:
+- [`flatagent.d.ts`](./flatagent.d.ts)
+- [`flatmachine.d.ts`](./flatmachine.d.ts)
+- [`profiles.d.ts`](./profiles.d.ts)
+
+## Python SDK
+
+```bash
+pip install flatagents[litellm]
 ```
 
-### Persistence & Checkpointing
-```yaml
-persistence:
-  enabled: true
-  backend: local  # or memory
+### LLM Backends
+
+```python
+from flatagents import LiteLLMBackend, AISuiteBackend
+
+# LiteLLM (default)
+agent = FlatAgent(config_file="agent.yml")
+
+# AISuite
+backend = AISuiteBackend(model="openai:gpt-4o")
+agent = FlatAgent(config_file="agent.yml", backend=backend)
 ```
 
-## Hooks
+### Hooks
+
+Extend machine behavior with Python hooks:
+
+```python
+from flatagents import FlatMachine, MachineHooks
 
-Extend with custom logic:
+class CustomHooks(MachineHooks):
+    def on_state_enter(self, state: str, context: dict) -> dict:
+        context["entered_at"] = time.time()
+        return context
 
-```typescript
-class CustomHooks implements MachineHooks {
-  async onStateEnter(state: string, context: any) {
-    console.log(`Entering ${state}`);
-    return context;
-  }
+    def on_action(self, action: str, context: dict) -> dict:
+        if action == "fetch_data":
+            context["data"] = fetch_from_api()
+        return context
 
-  async onError(state: string, error: Error, context: any) {
-    console.error(`Error in ${state}:`, error);
-    return "recovery_state";
-  }
-}
+machine = FlatMachine(config_file="machine.yml", hooks=CustomHooks())
 ```
 
-## MCP Integration
+**Available hooks**: `on_machine_start`, `on_machine_end`, `on_state_enter`, `on_state_exit`, `on_transition`, `on_error`, `on_action`
+
+### Execution Types
 
 ```yaml
-data:
-  mcp:
-    servers:
-      filesystem:
-        command: "npx"
-        args: ["@modelcontextprotocol/server-filesystem", "/path/to/files"]
-    tool_filter:
-      allow: ["filesystem:*"]
+execution:
+  type: retry              # retry | parallel | mdap_voting
+  backoffs: [2, 8, 16, 35] # Seconds between retries
+  jitter: 0.1              # ±10% random variation
 ```
 
-## Examples
+| Type | Use Case |
+|------|----------|
+| `default` | Single call |
+| `retry` | Rate limit handling with backoff |
+| `parallel` | Multiple samples (`n_samples`) |
+| `mdap_voting` | Consensus voting (`k_margin`, `max_candidates`) |
 
-- **Helloworld**: Simple agent that builds "Hello World" one character at a time
-- **Parallelism**: Machine arrays, foreach loops, and fire-and-forget patterns
-- **Human-in-the-loop**: Custom hooks for interactive approval flows
-- **Peering**: Parent-child machine communication with result backends
+### Schema Validation
 
-## API Reference
+```python
+from flatagents import validate_flatagent_config, validate_flatmachine_config
 
-### FlatAgent
-```typescript
-class FlatAgent {
-  constructor(config: AgentConfig | string);
-  async call(input: Record<string, any>): Promise<{content: string, output: any}>;
-}
+warnings = validate_flatagent_config(config)
+warnings = validate_flatmachine_config(config)
 ```
 
-### FlatMachine
-```typescript
-class FlatMachine {
-  constructor(options: MachineOptions);
-  async execute(input?: Record<string, any>): Promise<any>;
-  async resume(executionId: string): Promise<any>;
-}
-```
+### Logging & Metrics
 
-### Execution Types
-- `DefaultExecution`: Simple single execution
-- `RetryExecution`: Retry with exponential backoff
+```python
+from flatagents import setup_logging, get_logger
+
+setup_logging(level="INFO")  # Respects FLATAGENTS_LOG_LEVEL env var
+logger = get_logger(__name__)
+```
 
-### Persistence Backends
-- `MemoryBackend`: In-memory storage
-- `LocalFileBackend`: File-based with atomic writes
+**Env vars**: `FLATAGENTS_LOG_LEVEL` (`DEBUG`/`INFO`/`WARNING`/`ERROR`), `FLATAGENTS_LOG_FORMAT` (`standard`/`json`/`simple`)
 
-## Testing
+For OpenTelemetry metrics:
 
 ```bash
-npm test
-npm run typecheck
+pip install flatagents[metrics]
+export FLATAGENTS_METRICS_ENABLED=true
 ```
 
-## Building
+Metrics are enabled by default and print to stdout every 5s. Redirect to file or use OTLP for production:
 
 ```bash
-npm run build
-npm run dev  # watch mode
+# Metrics print to stdout by default
+python your_script.py
+
+# Save to file
+python your_script.py >> metrics.log 2>&1
+
+# Disable if needed
+FLATAGENTS_METRICS_ENABLED=false python your_script.py
+
+# Send to OTLP collector for production
+OTEL_METRICS_EXPORTER=otlp \
+OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 \
+python your_script.py
 ```
 
-## License
+**Env vars for metrics**:
 
-MIT
+| Variable | Default | Purpose |
+|----------|---------|---------|
+| `FLATAGENTS_METRICS_ENABLED` | `true` | Enable OpenTelemetry metrics |
+| `OTEL_METRICS_EXPORTER` | `console` | `console` (stdout) or `otlp` (production) |
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | — | OTLP collector endpoint |
+| `OTEL_METRIC_EXPORT_INTERVAL` | `5000` / `60000` | Export interval in ms (5s for console, 60s for otlp) |
+| `OTEL_SERVICE_NAME` | `flatagents` | Service name in metrics |

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@memgrafter/flatagents",
-  "version": "0.8.1",
+  "version": "0.9.0",
   "description": "TypeScript SDK for FlatAgents - Declarative LLM orchestration with YAML",
   "main": "dist/index.js",
   "module": "dist/index.mjs",
@@ -17,7 +17,10 @@
   },
   "files": [
     "dist",
-    "schemas"
+    "schemas",
+    "MACHINES.md",
+    "AGENTS.md",
+    "CLAUDE.md"
   ],
   "scripts": {
     "build": "tsup",

package/schemas/flatagent.d.ts

@@ -149,7 +149,7 @@
  * The profile field specifies which profile name to use as base.
  */
 
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 
 export interface AgentWrapper {
   spec: "flatagent";

package/schemas/flatagent.slim.d.ts

@@ -1,4 +1,4 @@
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 export interface AgentWrapper {
     spec: "flatagent";
     spec_version: string;

package/schemas/flatagents-runtime.d.ts

@@ -13,6 +13,8 @@
  *   - ResultBackend: InMemoryResultBackend (MUST)
  *   - ExecutionType: Default, Retry, Parallel, MDAPVoting (MUST)
  *   - MachineHooks: Base interface (MUST)
+ *   - RegistrationBackend: SQLiteRegistrationBackend (MUST), MemoryRegistrationBackend (SHOULD)
+ *   - WorkBackend: SQLiteWorkBackend (MUST), MemoryWorkBackend (SHOULD)
  *
  * OPTIONAL IMPLEMENTATIONS:
  * -------------------------
@@ -314,6 +316,158 @@ export interface LaunchIntent {
     launched: boolean;
 }
 
+/**
+ * REGISTRATION BACKEND:
+ * ---------------------
+ * Worker lifecycle management for distributed execution.
+ *
+ * SDKs MUST provide:
+ *   - SQLiteRegistrationBackend: For local deployments
+ *
+ * SDKs SHOULD provide:
+ *   - MemoryRegistrationBackend: For testing
+ *
+ * Implementation notes:
+ *   - Time units: Python reference SDK uses seconds for all interval values
+ *   - Stale threshold: SDKs SHOULD default to 2× heartbeat_interval if not specified
+ */
+export interface RegistrationBackend {
+    /**
+     * Register a new worker.
+     * Creates a new worker record with status "active".
+     */
+    register(worker: WorkerRegistration): Promise<WorkerRecord>;
+
+    /**
+     * Update worker's last_heartbeat timestamp.
+     * Can optionally update metadata.
+     */
+    heartbeat(worker_id: string, metadata?: Record<string, any>): Promise<void>;
+
+    /**
+     * Update worker status.
+     * Status values (string, not enum for extensibility):
+     *   - "active": Worker is running and healthy
+     *   - "terminating": Worker received shutdown signal
+     *   - "terminated": Worker exited cleanly
+     *   - "lost": Worker failed heartbeat, presumed dead
+     */
+    updateStatus(worker_id: string, status: string): Promise<void>;
+
+    /**
+     * Get a worker record by ID.
+     * @returns The worker record, or null if not found
+     */
+    get(worker_id: string): Promise<WorkerRecord | null>;
+
+    /**
+     * List workers matching filter criteria.
+     */
+    list(filter?: WorkerFilter): Promise<WorkerRecord[]>;
+}
+
+export interface WorkerRegistration {
+    worker_id: string;
+    host?: string;
+    pid?: number;
+    capabilities?: string[];  // e.g., ["gpu", "paper-analysis"]
+    pool_id?: string;         // Worker pool grouping
+    started_at: string;
+}
+
+export interface WorkerRecord extends WorkerRegistration {
+    status: string;           // See status values in RegistrationBackend.updateStatus
+    last_heartbeat: string;
+    current_task_id?: string;
+}
+
+export interface WorkerFilter {
+    status?: string | string[];
+    capability?: string;
+    pool_id?: string;
+    stale_threshold_seconds?: number;  // Filter workers with old heartbeats
+}
+
+/**
+ * WORK BACKEND:
+ * -------------
+ * Work distribution via named pools with atomic claim.
+ *
+ * SDKs MUST provide:
+ *   - SQLiteWorkBackend: For local deployments
+ *
+ * SDKs SHOULD provide:
+ *   - MemoryWorkBackend: For testing
+ *
+ * Implementation notes:
+ *   - Atomic claim: SDKs MUST ensure no two workers can claim the same job
+ *   - Test requirements: Include concurrent claim race condition tests
+ */
+export interface WorkBackend {
+    /**
+     * Get a named work pool.
+     * Creates the pool if it doesn't exist.
+     */
+    pool(name: string): WorkPool;
+}
+
+export interface WorkPool {
+    /**
+     * Add work item to the pool.
+     * @param item - The work data (will be JSON serialized)
+     * @param options.max_retries - Max retry attempts before poisoning (default: 3)
+     * @returns The item ID
+     */
+    push(item: any, options?: { max_retries?: number }): Promise<string>;
+
+    /**
+     * Atomically claim next available item.
+     * MUST be atomic - no two workers can claim the same job.
+     * @returns The claimed item, or null if pool is empty
+     */
+    claim(worker_id: string): Promise<WorkItem | null>;
+
+    /**
+     * Mark item as complete.
+     * Sets status to "done" and stores result.
+     */
+    complete(item_id: string, result?: any): Promise<void>;
+
+    /**
+     * Mark item as failed.
+     * Increments attempts. If attempts >= max_retries, marks as "poisoned".
+     * Otherwise returns to "pending" status for retry.
+     */
+    fail(item_id: string, error?: string): Promise<void>;
+
+    /**
+     * Get pool depth (unclaimed pending items).
+     */
+    size(): Promise<number>;
+
+    /**
+     * Release all jobs claimed by a worker.
+     * Used for stale worker cleanup.
+     * @returns Number of jobs released
+     */
+    releaseByWorker(worker_id: string): Promise<number>;
+}
+
+export interface WorkItem {
+    id: string;
+    data: any;
+    claimed_by?: string;
+    claimed_at?: string;
+    attempts: number;
+    max_retries: number;  // default: 3
+}
+
+// Job status values (string):
+// - "pending": Available for claim
+// - "claimed": Currently being processed
+// - "done": Successfully completed
+// - "poisoned": Failed max_retries times, will not be retried
+
 export interface BackendConfig {
     /** Checkpoint storage. Default: memory */
     persistence?: "memory" | "local" | "redis" | "postgres" | "s3";
@@ -323,24 +477,36 @@ export interface BackendConfig {
 
     /** Inter-machine results. Default: memory */
     results?: "memory" | "redis";
+
+    /** Worker registration. Default: memory */
+    registration?: "memory" | "sqlite" | "redis";
+
+    /** Work pool. Default: memory */
+    work?: "memory" | "sqlite" | "redis";
+
+    /** Path for sqlite backends (registration and work share this) */
+    sqlite_path?: string;
 }
 
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 
 /**
  * Wrapper interface for JSON schema generation.
  * Groups all runtime interfaces that SDKs must implement.
  */
 export interface SDKRuntimeWrapper {
-  spec: "flatagents-runtime";
-  spec_version: typeof SPEC_VERSION;
-  execution_lock?: ExecutionLock;
-  persistence_backend?: PersistenceBackend;
-  result_backend?: ResultBackend;
-  execution_config?: ExecutionConfig;
-  machine_hooks?: MachineHooks;
-  llm_backend?: LLMBackend;
-  machine_invoker?: MachineInvoker;
-  backend_config?: BackendConfig;
-  machine_snapshot?: MachineSnapshot;
+    spec: "flatagents-runtime";
+    spec_version: typeof SPEC_VERSION;
+    execution_lock?: ExecutionLock;
+    persistence_backend?: PersistenceBackend;
+    result_backend?: ResultBackend;
+    execution_config?: ExecutionConfig;
+    machine_hooks?: MachineHooks;
+    llm_backend?: LLMBackend;
+    machine_invoker?: MachineInvoker;
+    backend_config?: BackendConfig;
+    machine_snapshot?: MachineSnapshot;
+    registration_backend?: RegistrationBackend;
+    work_backend?: WorkBackend;
 }
+

package/schemas/flatagents-runtime.schema.json

@@ -11,7 +11,7 @@
         },
         "spec_version": {
           "type": "string",
-          "const": "0.8.1"
+          "const": "0.9.0"
         },
         "execution_lock": {
           "$ref": "#/definitions/ExecutionLock"
@@ -39,6 +39,12 @@
         },
         "machine_snapshot": {
           "$ref": "#/definitions/MachineSnapshot"
+        },
+        "registration_backend": {
+          "$ref": "#/definitions/RegistrationBackend"
+        },
+        "work_backend": {
+          "$ref": "#/definitions/WorkBackend"
         }
       },
       "required": [
@@ -51,7 +57,7 @@
     "ExecutionLock": {
       "type": "object",
       "additionalProperties": false,
-      "description": "FlatAgents Runtime Interface Spec ==================================\n\nThis file defines the runtime interfaces that SDKs MUST implement to be considered compliant. These are NOT configuration schemas (see flatagent.d.ts and flatmachine.d.ts for those).\n\nREQUIRED IMPLEMENTATIONS:\n-------------------------   - ExecutionLock: NoOpLock (MUST), LocalFileLock (SHOULD)   - PersistenceBackend: MemoryBackend (MUST), LocalFileBackend (SHOULD)   - ResultBackend: InMemoryResultBackend (MUST)   - ExecutionType: Default, Retry, Parallel, MDAPVoting (MUST)   - MachineHooks: Base interface (MUST)\n\nOPTIONAL IMPLEMENTATIONS:\n-------------------------   - Distributed backends (Redis, Postgres, etc.)   - LLMBackend (SDK may use native provider SDKs)\n\nEXECUTION LOCKING:\n------------------ Prevents concurrent execution of the same machine instance.\n\nSDKs MUST provide:   - NoOpLock: For when locking is handled externally or disabled\n\nSDKs SHOULD provide:   - LocalFileLock: For single-node deployments using fcntl/flock\n\nDistributed deployments should implement Redis/Consul/etcd locks.\n\nPERSISTENCE BACKEND:\n-------------------- Storage backend for machine checkpoints.\n\nSDKs MUST provide:   - MemoryBackend: For testing and ephemeral runs\n\nSDKs SHOULD provide:   - LocalFileBackend: For durable local storage with atomic writes\n\nRESULT BACKEND:\n--------------- Inter-machine communication via URI-addressed results.\n\nURI format: flatagents://{execution_id}/{path}   - path is typically \"result\" or \"checkpoint\"\n\nSDKs MUST provide:   - InMemoryResultBackend: For single-process execution\n\nEXECUTION TYPES:\n---------------- Execution strategy for agent calls.\n\nSDKs MUST implement all four types:   - default: Single call, no retry   - retry: Configurable backoffs with jitter   - parallel: Run N samples, return all successes   - mdap_voting: Multi-sample with consensus voting\n\nMACHINE HOOKS:\n-------------- Extension points for machine execution. All methods are optional and can be sync or async.\n\nSDKs SHOULD provide:   - WebhookHooks: Send events to HTTP endpoint   - CompositeHooks: Combine multiple hook implementations\n\nLLM BACKEND (OPTIONAL):\n----------------------- Abstraction over LLM providers.\n\nThis interface is OPTIONAL - SDKs may use provider SDKs directly. Useful for:   - Unified retry/monitoring across providers   - Provider-agnostic code   - Testing with mock backends\n\nMACHINE INVOKER:\n---------------- Interface for invoking peer machines. Used internally by FlatMachine for `machine:` and `launch:` states.\n\nBACKEND CONFIGURATION:\n---------------------- Backend configuration for machine settings.\n\nExample in YAML:   settings:     backends:       persistence: local       locking: none       results: memory"
+      "description": "FlatAgents Runtime Interface Spec ==================================\n\nThis file defines the runtime interfaces that SDKs MUST implement to be considered compliant. These are NOT configuration schemas (see flatagent.d.ts and flatmachine.d.ts for those).\n\nREQUIRED IMPLEMENTATIONS:\n-------------------------   - ExecutionLock: NoOpLock (MUST), LocalFileLock (SHOULD)   - PersistenceBackend: MemoryBackend (MUST), LocalFileBackend (SHOULD)   - ResultBackend: InMemoryResultBackend (MUST)   - ExecutionType: Default, Retry, Parallel, MDAPVoting (MUST)   - MachineHooks: Base interface (MUST)   - RegistrationBackend: SQLiteRegistrationBackend (MUST), MemoryRegistrationBackend (SHOULD)   - WorkBackend: SQLiteWorkBackend (MUST), MemoryWorkBackend (SHOULD)\n\nOPTIONAL IMPLEMENTATIONS:\n-------------------------   - Distributed backends (Redis, Postgres, etc.)   - LLMBackend (SDK may use native provider SDKs)\n\nEXECUTION LOCKING:\n------------------ Prevents concurrent execution of the same machine instance.\n\nSDKs MUST provide:   - NoOpLock: For when locking is handled externally or disabled\n\nSDKs SHOULD provide:   - LocalFileLock: For single-node deployments using fcntl/flock\n\nDistributed deployments should implement Redis/Consul/etcd locks.\n\nPERSISTENCE BACKEND:\n-------------------- Storage backend for machine checkpoints.\n\nSDKs MUST provide:   - MemoryBackend: For testing and ephemeral runs\n\nSDKs SHOULD provide:   - LocalFileBackend: For durable local storage with atomic writes\n\nRESULT BACKEND:\n--------------- Inter-machine communication via URI-addressed results.\n\nURI format: flatagents://{execution_id}/{path}   - path is typically \"result\" or \"checkpoint\"\n\nSDKs MUST provide:   - InMemoryResultBackend: For single-process execution\n\nEXECUTION TYPES:\n---------------- Execution strategy for agent calls.\n\nSDKs MUST implement all four types:   - default: Single call, no retry   - retry: Configurable backoffs with jitter   - parallel: Run N samples, return all successes   - mdap_voting: Multi-sample with consensus voting\n\nMACHINE HOOKS:\n-------------- Extension points for machine execution. All methods are optional and can be sync or async.\n\nSDKs SHOULD provide:   - WebhookHooks: Send events to HTTP endpoint   - CompositeHooks: Combine multiple hook implementations\n\nLLM BACKEND (OPTIONAL):\n----------------------- Abstraction over LLM providers.\n\nThis interface is OPTIONAL - SDKs may use provider SDKs directly. Useful for:   - Unified retry/monitoring across providers   - Provider-agnostic code   - Testing with mock backends\n\nMACHINE INVOKER:\n---------------- Interface for invoking peer machines. Used internally by FlatMachine for `machine:` and `launch:` states.\n\nBACKEND CONFIGURATION:\n---------------------- Backend configuration for machine settings.\n\nExample in YAML:   settings:     backends:       persistence: local       locking: none       results: memory"
     },
     "PersistenceBackend": {
       "type": "object",
@@ -154,6 +160,28 @@
             "redis"
           ],
           "description": "Inter-machine results. Default: memory"
+        },
+        "registration": {
+          "type": "string",
+          "enum": [
+            "memory",
+            "sqlite",
+            "redis"
+          ],
+          "description": "Worker registration. Default: memory"
+        },
+        "work": {
+          "type": "string",
+          "enum": [
+            "memory",
+            "sqlite",
+            "redis"
+          ],
+          "description": "Work pool. Default: memory"
+        },
+        "sqlite_path": {
+          "type": "string",
+          "description": "Path for sqlite backends (registration and work share this)"
         }
       },
       "additionalProperties": false
@@ -238,6 +266,16 @@
         "launched"
       ],
       "additionalProperties": false
+    },
+    "RegistrationBackend": {
+      "type": "object",
+      "additionalProperties": false,
+      "description": "REGISTRATION BACKEND:\n--------------------- Worker lifecycle management for distributed execution.\n\nSDKs MUST provide:   - SQLiteRegistrationBackend: For local deployments\n\nSDKs SHOULD provide:   - MemoryRegistrationBackend: For testing\n\nImplementation notes:   - Time units: Python reference SDK uses seconds for all interval values   - Stale threshold: SDKs SHOULD default to 2× heartbeat_interval if not specified"
+    },
+    "WorkBackend": {
+      "type": "object",
+      "additionalProperties": false,
+      "description": "WORK BACKEND:\n------------- Work distribution via named pools with atomic claim.\n\nSDKs MUST provide:   - SQLiteWorkBackend: For local deployments\n\nSDKs SHOULD provide:   - MemoryWorkBackend: For testing\n\nImplementation notes:   - Atomic claim: SDKs MUST ensure no two workers can claim the same job   - Test requirements: Include concurrent claim race condition tests"
     }
   }
 }

package/schemas/flatagents-runtime.slim.d.ts

@@ -102,12 +102,62 @@ export interface LaunchIntent {
     input: Record<string, any>;
     launched: boolean;
 }
+export interface RegistrationBackend {
+    register(worker: WorkerRegistration): Promise<WorkerRecord>;
+    heartbeat(worker_id: string, metadata?: Record<string, any>): Promise<void>;
+    updateStatus(worker_id: string, status: string): Promise<void>;
+    get(worker_id: string): Promise<WorkerRecord | null>;
+    list(filter?: WorkerFilter): Promise<WorkerRecord[]>;
+}
+export interface WorkerRegistration {
+    worker_id: string;
+    host?: string;
+    pid?: number;
+    capabilities?: string[];
+    pool_id?: string;
+    started_at: string;
+}
+export interface WorkerRecord extends WorkerRegistration {
+    status: string;
+    last_heartbeat: string;
+    current_task_id?: string;
+}
+export interface WorkerFilter {
+    status?: string | string[];
+    capability?: string;
+    pool_id?: string;
+    stale_threshold_seconds?: number;
+}
+export interface WorkBackend {
+    pool(name: string): WorkPool;
+}
+export interface WorkPool {
+    push(item: any, options?: {
+        max_retries?: number;
+    }): Promise<string>;
+    claim(worker_id: string): Promise<WorkItem | null>;
+    complete(item_id: string, result?: any): Promise<void>;
+    fail(item_id: string, error?: string): Promise<void>;
+    size(): Promise<number>;
+    releaseByWorker(worker_id: string): Promise<number>;
+}
+export interface WorkItem {
+    id: string;
+    data: any;
+    claimed_by?: string;
+    claimed_at?: string;
+    attempts: number;
+    max_retries: number;
+}
 export interface BackendConfig {
     persistence?: "memory" | "local" | "redis" | "postgres" | "s3";
     locking?: "none" | "local" | "redis" | "consul";
     results?: "memory" | "redis";
+    registration?: "memory" | "sqlite" | "redis";
+    work?: "memory" | "sqlite" | "redis";
+    sqlite_path?: string;
 }
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 export interface SDKRuntimeWrapper {
     spec: "flatagents-runtime";
     spec_version: typeof SPEC_VERSION;
@@ -120,4 +170,6 @@ export interface SDKRuntimeWrapper {
     machine_invoker?: MachineInvoker;
     backend_config?: BackendConfig;
     machine_snapshot?: MachineSnapshot;
+    registration_backend?: RegistrationBackend;
+    work_backend?: WorkBackend;
 }

package/schemas/flatmachine.d.ts

@@ -256,7 +256,7 @@
  * pending_launches    - Outbox pattern (v0.4.0)
  */
 
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 
 export interface MachineWrapper {
   spec: "flatmachine";

package/schemas/flatmachine.slim.d.ts

@@ -1,4 +1,4 @@
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 export interface MachineWrapper {
     spec: "flatmachine";
     spec_version: string;

package/schemas/profiles.d.ts

@@ -107,7 +107,7 @@
  * base_url          - Custom base URL for the API (e.g., for local models or proxies)
  */
 
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 
 export interface ProfilesWrapper {
   spec: "flatprofiles";

package/schemas/profiles.slim.d.ts

@@ -1,4 +1,4 @@
-export const SPEC_VERSION = "0.8.1";
+export const SPEC_VERSION = "0.9.0";
 export interface ProfilesWrapper {
     spec: "flatprofiles";
     spec_version: string;