@amitdeshmukh/ax-crew 7.0.0 → 8.0.0

package/CHANGELOG.md

@@ -1,5 +1,22 @@
 # Changelog
 
+## [8.0.0] - 2025-01-17
+
+### Added
+- **ACE (Agentic Context Engineering) Support** - Per-agent learning from human feedback
+  - Agents can learn from real-time feedback and update their behavior
+  - Playbook persistence to JSON files or custom storage via callbacks
+  - Teacher/Student model architecture for distilling feedback into rules
+  - Feedback routing across agent dependency chains (`applyTaskFeedback`)
+  - New `ACEConfig` interface with `teacher`, `persistence`, `options`, and `metric` fields
+- New type exports: `ACEConfig`, `ACETeacherConfig`, `ACEPersistenceConfig`, `ACEOptionsConfig`, `ACEMetricConfig`
+- Export `AxCrewOptions` type for consumer type safety
+- ACE examples: `ace-customer-support.ts` and `ace-flight-finder.ts`
+- ACE documentation section in README with configuration options and usage examples
+
+### Changed
+- ACE configuration is presence-based (no redundant `enabled` boolean needed)
+
 ## [7.0.0] - 2025-12-27
 
 ### Breaking

package/README.md

@@ -851,6 +851,110 @@ npm run dev examples/telemetry-demo.ts
 4. **Custom Attributes**: Extend traces with custom attributes specific to your use case
 5. **Performance**: Telemetry adds minimal overhead when properly configured
 
+### ACE Support (Agentic Context Engineering)
+
+AxCrew integrates [Agentic Context Engineering (ACE)](https://www.youtube.com/watch?v=elgYgPo_vY4) from the Ax framework, enabling agents to learn and improve from human feedback. ACE maintains a "playbook" of learned rules that guide agent behavior, which can be persisted across sessions.
+
+#### Key Features
+
+- **Online Learning**: Agents learn from real-time feedback during conversations
+- **Playbook Persistence**: Save learned rules to JSON files or custom storage
+- **Teacher/Student Model**: Use a separate "teacher" LLM to distill feedback into actionable rules
+- **Feedback Routing**: Distribute feedback across agent dependency chains automatically
+
+#### Configuration
+
+Add the `ace` field to any agent configuration:
+
+```typescript
+{
+  name: "SupportAgent",
+  description: "Customer support agent",
+  signature: "ticket:string -> supportResponse:string, decision:string",
+  provider: "google-gemini",
+  providerKeyName: "GEMINI_API_KEY",
+  ai: { model: "gemini-flash-latest", temperature: 0.7 },
+  ace: {
+    teacher: {
+      provider: "google-gemini",
+      providerKeyName: "GEMINI_API_KEY",
+      ai: { model: "gemini-flash-latest" }
+    },
+    options: {
+      maxEpochs: 1,
+      allowDynamicSections: true
+    },
+    persistence: {
+      playbookPath: "playbooks/support-agent.json",
+      autoPersist: true
+    },
+    metric: {
+      primaryOutputField: "supportResponse"
+    }
+  }
+}
+```
+
+#### ACE Configuration Options
+
+| Field | Type | Description |
+|-------|------|-------------|
+| `teacher` | object | Teacher model config (provider, model, apiURL) |
+| `persistence.playbookPath` | string | File path to save/load playbook |
+| `persistence.autoPersist` | boolean | Auto-save playbook after updates |
+| `persistence.onPersist` | function | Custom callback for saving playbook |
+| `persistence.onLoad` | function | Custom callback for loading playbook |
+| `options.maxEpochs` | number | Training epochs for offline compile |
+| `options.allowDynamicSections` | boolean | Allow playbook to create new sections |
+| `metric.primaryOutputField` | string | Output field to evaluate for quality |
+| `compileOnStart` | boolean | Run offline compile on agent init |
+
+#### Usage: Applying Feedback
+
+```typescript
+import { AxCrew, AxCrewFunctions } from '@amitdeshmukh/ax-crew';
+
+const crew = new AxCrew(config, AxCrewFunctions);
+await crew.addAgentsToCrew(['SupportAgent']);
+
+const agent = crew.agents.get('SupportAgent');
+
+// Run the agent
+const result = await agent.forward({ ticket: "Customer wants refund after 45 days" });
+const taskId = result._taskId;
+
+// Apply feedback to teach the agent
+await crew.applyTaskFeedback({
+  taskId,
+  feedback: "For loyal customers (5+ years), extend return window to 60 days",
+  strategy: "all"  // Apply to all agents involved in this task
+});
+
+// View the learned playbook
+const playbook = agent.getPlaybook?.();
+console.log(playbook);
+```
+
+#### Feedback Strategies
+
+| Strategy | Description |
+|----------|-------------|
+| `"all"` | Apply feedback to all agents involved in the task |
+| `"primary"` | Apply only to the primary (entry) agent |
+| `"leaf"` | Apply only to leaf agents (no sub-agents) |
+
+#### Examples
+
+See the ACE examples for complete demonstrations:
+
+- [`ace-customer-support.ts`](examples/ace-customer-support.ts) - Learn edge-case handling beyond standard policies
+- [`ace-feedback-routing.ts`](examples/ace-feedback-routing.ts) - Flight assistant with preference learning
+
+```bash
+# Run the customer support demo
+npx tsx examples/ace-customer-support.ts
+```
+
 ## Changelog
 
 See [CHANGELOG.md](CHANGELOG.md) for a list of changes and version updates.

package/dist/agents/ace.d.ts

@@ -0,0 +1,134 @@
+/**
+ * ACE (Agentic Context Engineering) integration for AxCrew
+ *
+ * This module provides helpers to build and manage AxACE optimizers for agents,
+ * enabling offline compilation and online learning from feedback.
+ *
+ * Reference: https://axllm.dev/ace/
+ */
+import { AxACE, type AxMetricFn } from "@ax-llm/ax";
+import type { AxAI } from "@ax-llm/ax";
+import type { ACEConfig, ACEPersistenceConfig, ACEMetricConfig, FunctionRegistryType } from "../types.js";
+export type { AxACE, AxMetricFn };
+/**
+ * Create an empty playbook structure
+ */
+export declare const createEmptyPlaybook: () => ACEPlaybook;
+/**
+ * Playbook types (mirroring AxACEPlaybook structure)
+ */
+export interface ACEBullet {
+    id: string;
+    section: string;
+    content: string;
+    helpfulCount: number;
+    harmfulCount: number;
+    createdAt: string;
+    updatedAt: string;
+    metadata?: Record<string, unknown>;
+}
+export interface ACEPlaybook {
+    version: number;
+    sections: Record<string, ACEBullet[]>;
+    stats: {
+        bulletCount: number;
+        helpfulCount: number;
+        harmfulCount: number;
+        tokenEstimate: number;
+    };
+    updatedAt: string;
+    description?: string;
+}
+/**
+ * Render a playbook into markdown instruction block for injection into prompts.
+ * Mirrors the AxACE renderPlaybook function.
+ */
+export declare const renderPlaybook: (playbook: Readonly<ACEPlaybook>) => string;
+/**
+ * Build an AxACE optimizer for an agent
+ *
+ * @param studentAI - The agent's AI instance (used as student)
+ * @param cfg - ACE configuration
+ * @returns Configured AxACE optimizer
+ */
+export declare const buildACEOptimizer: (studentAI: AxAI, cfg: ACEConfig) => AxACE;
+/**
+ * Load initial playbook from file, callback, or inline config
+ *
+ * @param cfg - Persistence configuration
+ * @returns Loaded playbook or undefined
+ */
+export declare const loadInitialPlaybook: (cfg?: ACEPersistenceConfig) => Promise<any | undefined>;
+/**
+ * Persist playbook to file or via callback
+ *
+ * @param pb - Playbook to persist
+ * @param cfg - Persistence configuration
+ */
+export declare const persistPlaybook: (pb: any, cfg?: ACEPersistenceConfig) => Promise<void>;
+/**
+ * Resolve metric function from registry or create equality-based metric
+ *
+ * @param cfg - Metric configuration
+ * @param registry - Function registry to search
+ * @returns Metric function or undefined
+ */
+export declare const resolveMetric: (cfg: ACEMetricConfig | undefined, registry: FunctionRegistryType) => AxMetricFn | undefined;
+/**
+ * Run offline ACE compilation
+ *
+ * @param args - Compilation arguments
+ * @returns Compilation result with optimized program
+ */
+export declare const runOfflineCompile: (args: {
+    program: any;
+    optimizer: AxACE;
+    metric: AxMetricFn;
+    examples: any[];
+    persistence?: ACEPersistenceConfig;
+}) => Promise<any>;
+/**
+ * Apply online update with feedback
+ *
+ * @param args - Update arguments
+ * @returns Curator delta (operations applied)
+ */
+export declare const runOnlineUpdate: (args: {
+    optimizer: AxACE;
+    example: any;
+    prediction: any;
+    feedback?: string;
+    persistence?: ACEPersistenceConfig;
+    tokenBudget?: number;
+    debug?: boolean;
+}) => Promise<any>;
+/**
+ * Use LLM to analyze feedback and generate playbook operations.
+ *
+ * This leverages AxGen with a proper signature (like AxACE's reflector/curator)
+ * to properly categorize feedback and extract actionable insights.
+ *
+ * IMPORTANT: The prompt explicitly tells the LLM to preserve specificity.
+ *
+ * @param ai - The AI instance to use for analysis
+ * @param feedback - User feedback string
+ * @param debug - Whether to log debug info
+ * @returns Promise of curator operations
+ */
+export declare const analyzeAndCategorizeFeedback: (ai: AxAI, feedback: string, debug?: boolean) => Promise<Array<{
+    type: "ADD" | "UPDATE" | "REMOVE";
+    section: string;
+    content: string;
+}>>;
+/**
+ * Add feedback to playbook using LLM analysis.
+ *
+ * Uses the AI to properly understand and categorize the feedback,
+ * then applies it as a curator operation.
+ *
+ * @param playbook - The playbook to update (mutated in place)
+ * @param feedback - User feedback string to add
+ * @param ai - AI instance for smart categorization
+ * @param debug - Whether to log debug info
+ */
+export declare const addFeedbackToPlaybook: (playbook: ACEPlaybook, feedback: string, ai: AxAI, debug?: boolean) => Promise<void>;