stagent 0.1.10 → 0.1.12

package/README.md

@@ -12,7 +12,7 @@ npx stagent
 
 Open [localhost:3000](http://localhost:3000).
 
-**Profiles & Policies** · **Blueprints & Schedules** · **Open Source**
+**Profiles & Policies** · **Blueprints & Schedules** · **Built-in Playbook** · **Open Source**
 
 <img src="https://raw.githubusercontent.com/navam-io/stagent/main/public/readme/home-list.png" alt="Stagent home workspace" width="1200" />
 
@@ -54,7 +54,12 @@ Stagent ships a shared runtime registry that routes tasks, schedules, and workfl
 | 🚨 | **[Ambient Approvals](#ambient-approvals)** | Shell-level approval prompts that keep Inbox as the durable supervision queue |
 | 🔒 | **[Tool Permissions](#tool-permission-persistence)** | Trusted-tool policies with explicit "Always Allow" rules |
 | 📋 | **[Kanban Board](#kanban-board-operations)** | Inline editing, bulk operations, and persistent board state |
-| 🤖 | **[AI Assist → Workflows](#ai-assist--workflow-creation)** | Bridge task assist recommendations into governed workflow execution *(in progress)* |
+| 🤖 | **[AI Assist → Workflows](#ai-assist--workflow-creation)** | Bridge task assist recommendations into governed workflow execution |
+| 🧬 | **[Agent Self-Improvement](#agent-self-improvement)** | Agents learn patterns from execution history with human-approved context evolution |
+| 🎯 | **[Tool Permission Presets](#tool-permission-presets)** | Pre-configured permission bundles (read-only, git-safe, full-auto) with layered apply/remove |
+| 📦 | **[Workflow Context Batching](#workflow-context-batching)** | Workflow-scoped proposal buffering with batch approve/reject for learned context |
+| 🧪 | **[E2E Test Automation](#e2e-test-automation)** | API-level end-to-end test suite covering both runtimes, 4 profiles, and 4 workflow patterns |
+| 📖 | **[Playbook](#playbook)** | Built-in documentation with usage-stage awareness, adoption heatmap, and guided learning journeys |
 
 ---
 
@@ -71,6 +76,8 @@ Stagent ships a shared runtime registry that routes tasks, schedules, and workfl
 - **Provider runtime abstraction** — Tasks, schedules, workflows, task assist, and health checks route through shared runtime adapters instead of provider-specific entry points
 - **Reusable agent profiles** — Profiles define instructions, allowed tools, runtime tuning, and MCP configs for repeated use
 - **Permission pre-check** — Saved "Always Allow" patterns bypass the notification loop for trusted tools
+- **Learned context loop** — Pattern extraction → human approval → versioned context injection creates a supervised self-improvement cycle
+- **Permission presets** — Layered preset bundles (read-only ⊂ git-safe ⊂ full-auto) that compose with individual "Always Allow" patterns
 
 ---
 
@@ -113,7 +120,7 @@ Claude Agent SDK integration with the `canUseTool` polling pattern remains the d
 OpenAI Codex App Server is integrated as Stagent's second governed runtime. Codex-backed tasks preserve project working directories, document context, resumable thread IDs, inbox approval requests, user questions, and provider-labeled logs. The same runtime can also power task assist, scheduled firings, and workflow child tasks.
 
 #### Agent Profiles
-Profile-backed execution with specialist definitions for different job types. Each profile packages instructions, allowed tools, runtime tuning, and MCP server configuration so teams can reuse behavior intentionally instead of relying on ad hoc prompts. Workflow steps and schedules can reference profiles directly, and runtimes can be selected independently when provider support differs.
+Profile-backed execution with specialist definitions for different job types. Each profile packages instructions, allowed tools, max turns, and output format so teams can reuse behavior intentionally instead of relying on ad hoc prompts. Workflow steps and schedules can reference profiles directly, and runtimes can be selected independently when provider support differs.
 
 <img src="https://raw.githubusercontent.com/navam-io/stagent/main/public/readme/profiles-list.png" alt="Stagent agent profiles" width="1200" />
 
@@ -143,10 +150,16 @@ AI-powered task creation: generate improved descriptions, break tasks into sub-t
 | <img src="https://raw.githubusercontent.com/navam-io/stagent/main/public/readme/dashboard-create-form-empty.png" alt="Empty task creation form" width="380" /> | <img src="https://raw.githubusercontent.com/navam-io/stagent/main/public/readme/dashboard-create-form-ai-assist.png" alt="AI Assist suggestions panel" width="380" /> | <img src="https://raw.githubusercontent.com/navam-io/stagent/main/public/readme/dashboard-create-form-ai-applied.png" alt="AI suggestions applied to form" width="380" /> |
 
 #### AI Assist → Workflow Creation
-*(In progress)* Bridge from AI task assist to workflow engine: when task assist recommends a multi-step plan, a "Create as Workflow" button converts the recommendation into a validated workflow definition with per-step profile assignments, dependency ordering, and pattern selection across all six workflow types. The `WorkflowConfirmationSheet` lets operators review and edit steps, profiles, and configuration before creating the workflow. A keyword-based profile suggestion fallback ensures steps get reasonable profile assignments even without the AI classifier.
+Bridge from AI task assist to workflow engine: when task assist recommends a multi-step plan, a "Create as Workflow" button converts the recommendation into a validated workflow definition with per-step profile assignments, dependency ordering, and pattern selection across all six workflow types. The `WorkflowConfirmationSheet` lets operators review and edit steps, profiles, and configuration before creating the workflow. A keyword-based profile suggestion fallback ensures steps get reasonable profile assignments even without the AI classifier.
 
 <img src="https://raw.githubusercontent.com/navam-io/stagent/main/public/readme/dashboard-workflow-confirm.png" alt="Workflow creation from AI Assist" width="1200" />
 
+#### Agent Self-Improvement
+Agents learn from execution history through a human-approved instruction evolution loop. After each task completion, the pattern extractor analyzes logs and proposes context updates — concise behavioral rules the agent should follow in future runs. Operators approve, reject, or edit proposals before they take effect. Learned context is versioned with rollback support and size-limited summarization to prevent unbounded growth. A sweep agent can audit the codebase for improvement opportunities and create prioritized tasks from its findings.
+
+#### Workflow Context Batching
+During workflow execution, the pattern extractor buffers context proposals into a learning session instead of creating individual notifications per proposal. When the workflow completes, all proposals are surfaced as a single batch for review. Operators can approve all, reject all, or review individually — reducing notification noise from multi-step workflows while preserving human oversight. The batch review component integrates into the existing pending approval host.
+
 #### Session Management
 Resume failed or cancelled agent tasks with one click. Tracks retry counts (limit: 3), detects expired sessions, and provides atomic claim to prevent duplicate runs.
 
@@ -176,6 +189,11 @@ Automatic text extraction on upload for five file types: text, PDF (pdf-parse),
 #### Agent Document Context
 Documents linked to a task are automatically injected into the agent's prompt as context. The context builder aggregates extracted text from all linked documents, giving agents access to uploaded reference material without manual copy-paste.
 
+### Knowledge
+
+#### Playbook
+Built-in documentation system at `/playbook` with usage-stage awareness that adapts content to your experience level (new, early, active, power user). Browse feature reference docs and guided learning journeys organized by persona (Personal, Work, Power User, Developer). Adoption heatmap tracks which features you've explored, while journey cards show progress through multi-step learning paths. Markdown rendering with automatic internal link resolution, table of contents, related docs, and screengrab embedding.
+
 ### Platform
 
 #### Tool Permission Persistence
@@ -184,6 +202,9 @@ Documents linked to a task are automatically injected into the agent's prompt as
 #### Ambient Approvals
 Pending permission requests now surface through a shell-level approval presenter on any route, so operators can respond without leaving the page they are working on. Inbox remains the durable queue and source of truth, while the ambient surface provides the fast path for active supervision.
 
+#### Tool Permission Presets
+Pre-configured permission bundles that reduce friction for common tool approval patterns. Three layered presets — read-only (file reads, glob, grep), git-safe (adds git operations), and full-auto (adds write, edit, bash) — compose with existing "Always Allow" patterns. Presets are layered: enabling git-safe automatically includes read-only patterns; removing git-safe only strips its unique additions. Risk badges indicate the trust level of each preset. Manage presets from the Settings page alongside individual tool permissions.
+
 #### Schedules
 Time-based scheduling for agent tasks with human-friendly intervals (`5m`, `2h`, `1d`) and raw 5-field cron expressions. One-shot and recurring modes with pause/resume lifecycle, expiry limits, and max firings. Each firing creates a child task through the shared execution pipeline, and schedules can now target a runtime explicitly. Scheduler runs as a poll-based engine started via Next.js instrumentation hook.
 
@@ -219,10 +240,13 @@ Configuration hub with provider-aware sections: Claude authentication (API key o
 The `npx stagent` entry point boots a Next.js server from the published npm package. It is built from `bin/cli.ts` into `dist/cli.js` using tsup, and serves as the primary distribution channel — no clone required.
 
 #### Database
-SQLite with WAL mode via better-sqlite3 + Drizzle ORM. Eight tables: `projects`, `tasks`, `workflows`, `agent_logs`, `notifications`, `documents`, `schedules`, `settings`. Self-healing bootstrap — tables are created on startup if missing.
+SQLite with WAL mode via better-sqlite3 + Drizzle ORM. Ten tables: `projects`, `tasks`, `workflows`, `agent_logs`, `notifications`, `documents`, `schedules`, `settings`, `learned_context`, `usage_ledger`. Self-healing bootstrap — tables are created on startup if missing.
 
 #### App Shell
-Responsive sidebar with collapsible icon-only mode, custom Stagent logo, tooltip navigation, dark/light/system theme, and OKLCH hue 250 blue-indigo color palette. Built on shadcn/ui (New York style) with PWA manifest and app icons. Routes: Home, Dashboard, Projects, Documents, Workflows, Profiles, Schedules, Inbox, Monitor, Settings.
+Responsive sidebar with collapsible icon-only mode, custom Stagent logo, tooltip navigation, dark/light/system theme, and OKLCH hue 250 blue-indigo color palette. Built on shadcn/ui (New York style) with PWA manifest and app icons. Routes: Home, Dashboard, Inbox, Monitor, Projects, Workflows, Documents, Profiles, Schedules, Cost & Usage, Playbook, Settings.
+
+#### E2E Test Automation
+API-level end-to-end test suite built on Vitest with 120-second timeouts and sequential execution. Five test files cover single-task execution, sequence workflows, parallel workflows, blueprints, and cross-runtime scenarios across both Claude and Codex backends. Tests skip gracefully when runtimes are not configured, preventing CI failures. Run with `npm run test:e2e`.
 
 ---
 
@@ -251,46 +275,57 @@ npm run dev            # Next.js dev server (Turbopack)
 npm run build:cli      # Build CLI → dist/cli.js
 npm test               # Run Vitest
 npm run test:coverage  # Coverage report
+npm run test:e2e       # E2E integration tests (requires runtime credentials)
 ```
 
 ### Project Structure
 
 ```
+docs/                     # Playbook markdown docs + manifest.json
 src/
 ├── app/                  # Next.js App Router pages
-│   ├── dashboard/        # Project overview
+│   ├── dashboard/        # Task kanban board
 │   ├── projects/[id]/    # Project detail
+│   ├── tasks/            # Task detail + creation (redirects to dashboard)
+│   ├── profiles/         # Agent profile gallery + detail + creation
 │   ├── documents/        # Document browser
-│   ├── workflows/        # Workflow management
+│   ├── workflows/        # Workflow management + blueprints
 │   ├── schedules/        # Schedule management
+│   ├── costs/            # Cost & usage dashboard
+│   ├── playbook/         # Documentation & learning journeys
 │   ├── inbox/            # Notifications
 │   ├── monitor/          # Log streaming
 │   └── settings/         # Configuration
 ├── components/
 │   ├── dashboard/        # Homepage widgets + charts
 │   ├── tasks/            # Board, cards, panels
-│   ├── workflows/        # Workflow UI
+│   ├── profiles/         # Profile gallery, detail, form, learned context
+│   ├── workflows/        # Workflow UI + blueprints + swarm
 │   ├── documents/        # Document browser + upload
+│   ├── costs/            # Cost dashboard + filters
+│   ├── playbook/         # Playbook docs + journeys + adoption
 │   ├── schedules/        # Schedule management
 │   ├── monitoring/       # Log viewer
 │   ├── notifications/    # Inbox + permission actions
-│   ├── settings/         # Auth, permissions, data mgmt
+│   ├── settings/         # Auth, permissions, budgets, data mgmt
 │   ├── shared/           # App shell, sidebar
 │   └── ui/               # shadcn/ui primitives
 └── lib/
-    ├── agents/           # Runtime adapters, provider integrations, profiles
+    ├── agents/           # Runtime adapters, profiles, learned context, pattern extraction
     ├── db/               # Schema, migrations
+    ├── docs/             # Playbook reader, adoption, usage-stage, journey tracker
     ├── documents/        # Preprocessing + context builder
     ├── workflows/        # Engine + types + blueprints
     ├── schedules/        # Scheduler engine + interval parser
     ├── settings/         # Auth, permissions, helpers
+    ├── usage/            # Metering ledger + pricing registry
     ├── constants/        # Status transitions, colors
     ├── queries/          # Chart data aggregation
     ├── validators/       # Zod schemas
     └── utils/            # Shared helpers
 ```
 
-### API Endpoints (48 routes)
+### API Endpoints (52 routes)
 
 | Domain | Endpoint | Method | Purpose |
 |--------|----------|--------|---------|
@@ -324,7 +359,7 @@ src/
 | **Profiles** | `/api/profiles` | GET | List agent profiles |
 | | `/api/profiles/[id]` | GET/PUT/DELETE | Profile CRUD |
 | | `/api/profiles/[id]/test` | POST | Run behavioral tests on a profile |
-| | `/api/profiles/[id]/context` | GET | Profile context for agent execution |
+| | `/api/profiles/[id]/context` | GET/POST/PATCH | Learned context: version history, manual add, approve/reject/rollback |
 | | `/api/profiles/import` | POST | Import profile from GitHub URL |
 | **Notifications** | `/api/notifications` | GET/POST | List and create notifications |
 | | `/api/notifications/[id]` | PATCH/DELETE | Update and delete notification |
@@ -338,7 +373,10 @@ src/
 | | `/api/settings/test` | POST | Provider-aware runtime connectivity test |
 | | `/api/settings/budgets` | GET/POST | Budget configuration |
 | | `/api/permissions` | GET/POST/DELETE | Tool permission patterns |
+| | `/api/permissions/presets` | GET/POST/DELETE | Permission preset bundles |
+| **Context** | `/api/context/batch` | POST | Batch approve/reject context proposals |
 | **Monitoring** | `/api/logs/stream` | GET | SSE agent log stream |
+| **Playbook** | `/api/playbook/status` | GET | Playbook adoption status and usage stage |
 | **Platform** | `/api/command-palette/recent` | GET | Recent command palette items |
 | | `/api/data/clear` | POST | Clear all data |
 | | `/api/data/seed` | POST | Seed sample data |
@@ -357,7 +395,7 @@ All 14 features shipped across three layers:
 | **Core** | Project management, task board, agent integration, inbox notifications, monitoring dashboard |
 | **Polish** | Homepage dashboard, UX fixes, workflow engine, AI task assist, content handling, session management |
 
-### Post-MVP — Complete (25 features)
+### Post-MVP — Complete (31 features)
 
 | Category | Feature | What shipped |
 |----------|---------|-------------|
@@ -369,6 +407,9 @@ All 14 features shipped across three layers:
 | **Agent Intelligence** | Multi-Agent Routing | Profile registry (4 profiles), task classifier, per-step profile assignment |
 | | Autonomous Loop Execution | 4 stop conditions, iteration context chaining, pause/resume, loop status view |
 | | Multi-Agent Swarm | Mayor → worker pool → refinery orchestration with retryable stages |
+| | AI Assist → Workflows | Bridge task assist into workflow engine with profile assignment and pattern selection |
+| | Agent Self-Improvement | Pattern extraction from logs, human-approved context evolution, versioned rollback |
+| | Workflow Context Batching | Workflow-scoped proposal buffering with batch approve/reject |
 | **Agent Profiles** | Agent Profile Catalog | 13 domain-specific profiles, GitHub import, behavioral testing, MCP passthrough |
 | | Workflow Blueprints | 8 templates, gallery, YAML editor, dynamic forms, GitHub import, lineage tracking |
 | **UI Enhancement** | Ambient Approvals | Shell-level approval presenter on any route for fast supervision |
@@ -382,27 +423,17 @@ All 14 features shipped across three layers:
 | | Board Context Persistence | Persisted filters, sort order, and project selection across sessions |
 | **Platform** | Scheduled Prompt Loops | Cron + human-friendly intervals, one-shot/recurring, pause/resume lifecycle |
 | | Tool Permission Persistence | "Always Allow" patterns, pre-check bypass, Settings management |
+| | Tool Permission Presets | 3 layered presets (read-only, git-safe, full-auto) with risk badges |
 | | Provider Runtimes | Shared runtime registry with Claude Code and OpenAI Codex App Server adapters |
 | | OpenAI Codex Runtime | Codex App Server integration with inbox approvals, logs, and thread resumption |
-| | npm Publish Readiness | `npx stagent` distribution channel with CLI bundling and package config |
 | | Cross-Provider Profiles | Profile compatibility layer ensuring profiles work across Claude and Codex runtimes |
 | | Parallel Fork/Join | 2-5 concurrent research branches with synthesis step |
+| **Runtime Quality** | E2E Test Automation | API-level test suite covering both runtimes, 4 profiles, 4 workflow patterns |
+| **Knowledge** | Playbook | Built-in documentation with usage-stage awareness, adoption heatmap, guided learning journeys |
 | **Governance** | Usage Metering Ledger | Provider-normalized token and spend tracking across all execution paths |
 | | Spend Budget Guardrails | Per-project and global budgets with enforcement and alerts |
 | | Cost & Usage Dashboard | Summary cards, trend views, provider/model breakdowns, budget audit visibility |
 
-### In Progress
-
-| Feature | Description |
-|---------|-------------|
-| **AI Assist → Workflow Creation** | Bridge AI task assist recommendations into the workflow engine with profile assignment and pattern selection |
-
-### Planned
-
-| Feature | Description |
-|---------|-------------|
-| **Agent Self-Improvement** | Agents learn patterns and update context with human approval |
-
 ---
 
 ## Contributing

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "stagent",
-  "version": "0.1.10",
+  "version": "0.1.12",
   "description": "Governed AI agent workspace for supervised local execution, workflows, documents, and provider runtimes.",
   "keywords": [
     "ai",
@@ -35,7 +35,7 @@
   "bugs": {
     "url": "https://github.com/navam-io/stagent/issues"
   },
-  "homepage": "https://github.com/navam-io/stagent#readme",
+  "homepage": "https://stagent.io",
   "scripts": {
     "dev": "next dev --turbopack",
     "build": "next build",
@@ -43,6 +43,7 @@
     "test": "vitest run",
     "test:watch": "vitest",
     "test:coverage": "vitest run --coverage",
+    "test:e2e": "vitest run --config vitest.config.e2e.ts",
     "test:ui": "vitest --ui",
     "prepublishOnly": "npm run build:cli"
   },
@@ -51,7 +52,6 @@
   },
   "dependencies": {
     "@anthropic-ai/claude-agent-sdk": "^0.2.71",
-    "@anthropic-ai/sdk": "^0.78.0",
     "@dnd-kit/core": "^6.3.1",
     "@dnd-kit/sortable": "^10.0.0",
     "@dnd-kit/utilities": "^3.2.2",

package/src/__tests__/e2e/blueprint.test.ts

@@ -0,0 +1,63 @@
+/**
+ * E2E: Blueprint instantiation and execution.
+ *
+ * Tests that blueprints can be listed, instantiated with variables,
+ * and executed as workflows with variable resolution.
+ */
+
+import {
+  setupE2E,
+  teardownE2E,
+  testProjectId,
+  claudeAvailable,
+} from "./setup";
+import {
+  listBlueprints,
+  instantiateBlueprint,
+  executeWorkflow,
+  pollWorkflowUntilDone,
+} from "./helpers";
+
+beforeAll(async () => {
+  await setupE2E();
+});
+
+afterAll(async () => {
+  await teardownE2E();
+});
+
+describe("Blueprint — Gallery & Instantiation", () => {
+  it("lists available blueprints", async () => {
+    const { ok, data } = await listBlueprints();
+    expect(ok).toBe(true);
+    expect(Array.isArray(data)).toBe(true);
+    expect(data!.length).toBeGreaterThan(0);
+  });
+
+  it.skipIf(!claudeAvailable)(
+    "instantiates and executes documentation-generation blueprint",
+    async () => {
+      // Instantiate with variables
+      const { ok: instOk, data: instData } = await instantiateBlueprint(
+        "documentation-generation",
+        {
+          target: "src/index.ts and src/utils.ts",
+          docType: "API Documentation",
+        },
+        testProjectId
+      );
+      expect(instOk).toBe(true);
+
+      const workflow = instData?.workflow;
+      expect(workflow).toBeTruthy();
+      expect(workflow!.status).toBe("draft");
+
+      // Execute the instantiated workflow
+      const exec = await executeWorkflow(workflow!.id);
+      expect(exec.status).toBe(202);
+
+      const result = await pollWorkflowUntilDone(workflow!.id);
+      expect(result.status).toBe("completed");
+    }
+  );
+});

package/src/__tests__/e2e/cross-runtime.test.ts

@@ -0,0 +1,77 @@
+/**
+ * E2E: Cross-runtime comparison.
+ *
+ * Tests that the same task produces valid results on both Claude Code
+ * and Codex runtimes, verifying runtime parity.
+ */
+
+import {
+  setupE2E,
+  teardownE2E,
+  testProjectId,
+  claudeAvailable,
+  codexAvailable,
+} from "./setup";
+import {
+  createTask,
+  executeTask,
+  pollTaskUntilDone,
+  updateTask,
+} from "./helpers";
+
+beforeAll(async () => {
+  await setupE2E();
+});
+
+afterAll(async () => {
+  await teardownE2E();
+});
+
+describe("Cross-Runtime Comparison", () => {
+  const bothAvailable = () => claudeAvailable && codexAvailable;
+
+  it.skipIf(!bothAvailable())(
+    "same task produces valid results on both runtimes",
+    async () => {
+      const taskPrompt =
+        "Describe the TypeScript code in src/index.ts. List the exported functions and any bugs.";
+
+      // Create and execute on Claude
+      const { data: claudeTask } = await createTask({
+        title: "Cross-runtime test (Claude)",
+        description: taskPrompt,
+        projectId: testProjectId,
+        agentProfile: "general",
+      });
+      await updateTask(claudeTask!.id, { status: "queued" });
+      await executeTask(claudeTask!.id);
+
+      // Create and execute on Codex
+      const { data: codexTask } = await createTask({
+        title: "Cross-runtime test (Codex)",
+        description: taskPrompt,
+        projectId: testProjectId,
+        assignedAgent: "codex",
+        agentProfile: "general",
+      });
+      await updateTask(codexTask!.id, { status: "queued" });
+      await executeTask(codexTask!.id);
+
+      // Wait for both
+      const [claudeResult, codexResult] = await Promise.all([
+        pollTaskUntilDone(claudeTask!.id),
+        pollTaskUntilDone(codexTask!.id),
+      ]);
+
+      // Both should complete
+      expect(claudeResult.status).toBe("completed");
+      expect(codexResult.status).toBe("completed");
+
+      // Both should produce non-empty results
+      expect(claudeResult.result).toBeTruthy();
+      expect(codexResult.result).toBeTruthy();
+      expect(claudeResult.result!.length).toBeGreaterThan(50);
+      expect(codexResult.result!.length).toBeGreaterThan(50);
+    }
+  );
+});