flow-frame-core 0.1.6 → 0.1.7

package/README.md

@@ -43,6 +43,38 @@ Detailed documentation is available in the `docs/` folder:
 -   **[Email Automation](docs/email_automation.md)**: Gmail polling and filtering.
 -   **[Workflows](docs/workflows.md)**: How to build and run workflows.
 
+## 🔧 Tool Reference (for Agent Integration)
+
+`flow-frame-core` is designed to be used as a tool library by AI agents. The complete tool glossary is in **[TOOLS.md](./TOOLS.md)**, covering every function and REST endpoint organized by capability:
+
+| Category | Tools | Integration |
+|----------|-------|-------------|
+| **LLM / AI** | `runPrompt`, `runPromptGrok`, `runImagePromptGrok` | Import or `POST /api/prompts/chat` |
+| **Prompt Chains** | `executeChain` — serial/parallel steps with auto-repair | Import |
+| **Workflow Engine** | `executeFlow` — 25+ node types in a DAG | Import or REST |
+| **JSON Utilities** | `extractJSON`, `cleanJSONString`, `containsJSON` | Import |
+| **Schema & Code Gen** | `inferSchema`, `generateJsTransformFromPrompt` | Import |
+| **Prompt Optimization** | `runAutoPromptSearch` — iterative prompt refinement | Import |
+| **Image Analysis** | `classifyImageQuery`, `findImageInImage` | Import or REST |
+| **Audio / TTS** | `processAudio` — ElevenLabs TTS | Import or `POST /generateAudio` |
+| **PDF Processing** | `extractPdf` — text, images, page renders | Import or `GET /pdf/extract` |
+| **Email** | `extractGmailBodyText`, polling endpoints | Import or REST |
+| **Web Scraping** | `crawl` — full-site depth-first crawl | Import or `POST /scrape` |
+| **Storage** | `getItem`, `setItem`, profile management | Import or REST |
+| **Queue** | `queueManager` — serial job queue with events | Import or REST |
+| **File System** | 30+ file utilities (read, write, download, zip) | Import or REST |
+| **Templates** | `generateTextFromTemplate` — LLM variable generation | Import or REST |
+| **Browser / Desktop** | Chrome control, mouse, keyboard, app management | Import or REST |
+| **Screenshots** | `captureScreenshotBase64`, `captureFullScreenshot` | Import or REST |
+| **Self-Learning Vision** | Ingest, recognize, discover screens, transition graphs | Import or REST |
+| **UI Planning** | `UiPlanner`, `StepByStepAiPlanner` — NL → automation steps | Import |
+| **Workflows CRUD** | Create, load, save, validate `.workflow` files | REST |
+| **Extensions** | Plugin discovery and execution via `api.json` | Import or REST |
+| **Media** | Video/audio dimensions, YOLO training data | Import or REST |
+| **Config** | File-based config CRUD with backup/validation | REST |
+
+See **[TOOLS.md](./TOOLS.md)** for full function signatures, parameters, return types, and REST endpoint details.
+
 ## 🌟 Key Features
 
 -   **Agentic UI Planning**: Deeply integrated AI that maps web pages and uses `robotjs` to simulate human clicks and typing.

package/TOOLS.md

@@ -0,0 +1,1080 @@
+# Flow-Frame Tool Reference
+
+> Complete glossary of every capability in `flow-frame-core` that can be used as a tool by an external agent (e.g. Woodbury).
+
+**Package**: `flow-frame-core`
+**Install**: `npm install flow-frame-core`
+**Runtime**: Node.js 18+, ESM (`"type": "module"`)
+**Two integration modes**: Import functions directly (programmatic), or run the server and call HTTP endpoints (REST API).
+
+---
+
+## Table of Contents
+
+- [1. LLM / AI](#1-llm--ai)
+- [2. Prompt Chains](#2-prompt-chains)
+- [3. Workflow Engine](#3-workflow-engine)
+- [4. JSON Utilities](#4-json-utilities)
+- [5. Schema & Code Generation](#5-schema--code-generation)
+- [6. Prompt Optimization](#6-prompt-optimization)
+- [7. Image Analysis](#7-image-analysis)
+- [8. Audio / TTS](#8-audio--tts)
+- [9. PDF Processing](#9-pdf-processing)
+- [10. Email](#10-email)
+- [11. Web Scraping](#11-web-scraping)
+- [12. Storage](#12-storage)
+- [13. Queue Management](#13-queue-management)
+- [14. File System Utilities](#14-file-system-utilities)
+- [15. Template & Variable Generation](#15-template--variable-generation)
+- [16. Browser & Desktop Automation](#16-browser--desktop-automation)
+- [17. Screenshot & Screen Capture](#17-screenshot--screen-capture)
+- [18. Self-Learning Vision Pipeline](#18-self-learning-vision-pipeline)
+- [19. UI Automation Planning](#19-ui-automation-planning)
+- [20. Workflow File Management](#20-workflow-file-management)
+- [21. Extension System](#21-extension-system)
+- [22. Media Processing](#22-media-processing)
+- [23. Configuration Management](#23-configuration-management)
+- [24. Constants & Selectors](#24-constants--selectors)
+- [REST API Quick Reference](#rest-api-quick-reference)
+
+---
+
+## 1. LLM / AI
+
+Send prompts to OpenAI or Groq models. All functions auto-route based on model name prefix.
+
+### `runPrompt(messages, model, images?, jsonMode?, timeout?)`
+
+**Import**: `import { runPrompt } from 'flow-frame-core/services/runPrompt.js'`
+
+The unified LLM call. Routes to Groq for non-GPT models, OpenAI otherwise.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `messages` | `{role, content}[]` | required | Chat message array |
+| `model` | `string` | required | Model ID (e.g. `'gpt-4o'`, `'meta-llama/llama-4-scout-17b-16e-instruct'`) |
+| `images` | `string[]` | `[]` | Base64 image data URLs to attach |
+| `jsonMode` | `boolean` | `false` | Request JSON output format |
+| `timeout` | `number` | `600000` | Timeout in ms |
+
+**Returns**: `string` — the model's response text.
+
+**REST**: `POST /api/prompts/chat`
+```json
+{ "messages": [...], "model": "gpt-4o", "jsonMode": false }
+→ { "success": true, "result": "..." }
+```
+
+---
+
+### `runPromptGrok(messages, model, jsonMode?, maxRetries?, baseDelay?)`
+
+**Import**: `import { runPromptGrok } from 'flow-frame-core'`
+
+Groq-specific with exponential backoff retry on 503 capacity errors.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `messages` | `{role, content}[]` | required | Chat messages |
+| `model` | `string` | required | Groq model ID |
+| `jsonMode` | `boolean` | `false` | JSON response mode |
+| `maxRetries` | `number` | `3` | Max retry attempts |
+| `baseDelay` | `number` | `15000` | Base backoff delay in ms |
+
+**Returns**: `string` — response content.
+
+---
+
+### `runImagePromptGrok(prompt, base64_image, model?)`
+
+**Import**: `import { runImagePromptGrok } from 'flow-frame-core'`
+
+Send a text prompt with an image to Groq's vision model.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `prompt` | `string` | required | Text prompt |
+| `base64_image` | `string` | required | Base64 data URL of the image |
+| `model` | `string` | `'meta-llama/llama-4-scout-17b-16e-instruct'` | Vision model |
+
+**Returns**: `string` — response text.
+
+**REST**: `POST /analyze-image-grok`
+```json
+{ "prompt": "Describe this image", "base64_image": "data:image/png;base64,..." }
+→ { "result": "..." }
+```
+
+---
+
+### `processLLMResponse(response, model, provider?)`
+
+**Import**: `import { processLLMResponse } from 'flow-frame-core/services/runPrompt.js'`
+
+Parse JSON from raw LLM output. On failure, asks the same model to repair the output.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `response` | `string` | Raw LLM text |
+| `model` | `string` | Model to use for repair |
+| `provider` | `string` | `'grok'` or `'openai'` |
+
+**Returns**: Parsed object or repaired JSON.
+
+---
+
+## 2. Prompt Chains
+
+Execute multi-step prompt sequences with validation and auto-repair.
+
+### `executeChain({ chainDef, prompts, input?, preamble?, precontext?, serverUrl?, model?, logFn?, promptExecutor? })`
+
+**Import**: `import { executeChain } from 'flow-frame-core/services/chainExecutor.js'`
+
+Runs a chain of prompts in sequence or parallel, accumulating context across steps.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `chainDef` | `object` | required | `{ steps: Step[] }` — each step is `{ promptKey, expectations? }` or an array (parallel block) |
+| `prompts` | `object` | required | Map of prompt keys to prompt text strings |
+| `input` | `string\|object` | `undefined` | Initial input/context |
+| `preamble` | `string` | `undefined` | System preamble prepended to each step |
+| `precontext` | `string` | `undefined` | Additional context |
+| `serverUrl` | `string` | `undefined` | Base URL for API (mutually exclusive with `promptExecutor`) |
+| `model` | `string` | `'gpt-4'` | LLM model ID |
+| `logFn` | `function` | `console.log` | Logging callback |
+| `promptExecutor` | `function` | `null` | Direct prompt execution function (bypasses HTTP) |
+
+**Returns**: `{ result: object, history: StepResult[] }`
+
+Each step with `expectations` triggers an LLM QA validation loop — if the output doesn't match expectations, a repair agent rewrites it (max 2 retries).
+
+**Parallel blocks**: Wrap steps in an array to run them concurrently:
+```js
+{
+  steps: [
+    { promptKey: 'step1' },           // serial
+    [{ promptKey: 'a' }, { promptKey: 'b' }],  // parallel
+    { promptKey: 'step3' }            // serial
+  ]
+}
+```
+
+---
+
+## 3. Workflow Engine
+
+Execute visual node-graph workflows (DAGs).
+
+### `executeFlow(nodes, edges, workflows, modelContext, initialCtx?, callback?, globalContext?)`
+
+**Import**: `import { executeFlow } from 'flow-frame-core'`
+
+The core workflow runtime. Topologically sorts a node-edge graph and executes each node.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `nodes` | `Node[]` | Array of node objects with `{ id, type, data }` |
+| `edges` | `Edge[]` | Array of `{ source, sourceHandle, target, targetHandle }` |
+| `workflows` | `object` | Named workflow configs for subgraph references |
+| `modelContext` | `object` | Configuration: `{ defaultServer?, loadConfiguration?, loadPromptLibrary?, promptExecutor?, extensionController? }` |
+| `initialCtx` | `object` | `{}` — pre-populated node values |
+| `callback` | `function` | `(nodeId, progress) => void` |
+| `globalContext` | `object` | `{}` — shared mutable state across subgraphs |
+
+#### Supported Node Types
+
+| Node Type | Purpose |
+|-----------|---------|
+| **Data** | |
+| `inputNode` | Reads named input or default value |
+| `setNode` | Sets a key on modelContext |
+| `getNode` | Reads a key from modelContext |
+| `propertyNode` | Deep property access (dot notation) |
+| `arrayItemNode` | Array index access |
+| `inputArrayNode` | Parse raw JSON array |
+| `globalNode` | Write to shared globalContext |
+| `globalGetNode` | Read from globalContext (dot notation + default) |
+| `resultNode` | Capture subgraph output |
+| `outputNode` | POST result to a URL |
+| **AI / Prompts** | |
+| `promptNode` | LLM prompt with system/user/images handles |
+| `autoPromptOptimizerNode` | Find best prompt via iterative search |
+| `uiPlannerNode` | AI-powered UI automation planning |
+| `uiPlannerExecNode` | Execute a UI plan's steps |
+| **Control Flow** | |
+| `switchNode` | Pattern-match input → route to handles |
+| `ifNode` | Conditional branch (inline subgraphs) |
+| `ifWorkflowNode` | Conditional branch (named workflow configs) |
+| `ifGraphNode` | Conditional branch (automation ID match) |
+| `forEachNode` | Iterate array → execute named graph per item |
+| `forLoopNode` | Iterate array → execute inline bodyGraph per item |
+| `pauseNode` | Sleep N seconds |
+| **Network** | |
+| `fetchNode` | HTTP request (method, timeout, body, response type) |
+| `gmailNode` | Read filtered emails |
+| `pollerNode` | Poll emails + LLM-route to labeled subgraphs |
+| **Composition** | |
+| `workflowNode` | Execute named workflow |
+| `smartAutomationNode` | Execute named smart automation |
+| `graphNode` | Load + execute named workflow config |
+| **Code** | |
+| `execJSNode` | Run arbitrary JS (inputs a-g, access to utils/fs/path/$global) |
+| **Extensions** | |
+| `extensionNode` | Resolve + run an extension by capability ID |
+| **UI Map** | |
+| `uiMapSelectorNode` | Generate UI map from workflow files |
+| `uiMapProcessorNode` | Pass-through |
+| `uiMapChatNode` | Return generated context |
+| `uiDependencyGraphNode` | Parse bodyGraph JSON |
+
+### `computeExecutionOrder(nodes, edges)`
+
+Kahn's topological sort on a node graph. Returns ordered node array.
+
+### `getSubgraph(startId, nodes, edges)`
+
+BFS extraction of reachable nodes/edges from a start node.
+
+---
+
+## 4. JSON Utilities
+
+Robust JSON extraction from messy LLM output.
+
+### `extractJSON(input)`
+
+**Import**: `import { extractJSON } from 'flow-frame-core'`
+
+Strips markdown fences, prose preambles, and parses JSON. Returns objects as-is.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `input` | `string\|object` | Raw text or object |
+
+**Returns**: Parsed object, or `null` on failure.
+
+### `cleanJSONString(str)`
+
+Strips markdown code fences, text wrappers, normalizes escapes to isolate raw JSON.
+
+### `safeExtractJSON(input, options?)`
+
+Wraps `extractJSON` with configurable fallback: `{ returnOriginalOnFail?, defaultValue?, throwOnError? }`.
+
+### `containsJSON(str)`
+
+Returns `true` if string contains parseable JSON.
+
+### `extractMultipleJSON(str)`
+
+Finds all JSON objects in a string. Returns `object[]`.
+
+---
+
+## 5. Schema & Code Generation
+
+### `inferSchema(prompt, model)`
+
+**Import**: `import { inferSchema } from 'flow-frame-core/services/schemaInference.js'`
+
+Uses OpenAI to generate a JSON Schema (draft-07) describing what a prompt's output should look like.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `prompt` | `string` | The prompt to analyze |
+| `model` | `string` | OpenAI model to use |
+
+**Returns**: Parsed JSON Schema object.
+
+---
+
+### `generateJsTransformFromPrompt(opts)`
+
+**Import**: `import { generateJsTransformFromPrompt } from 'flow-frame-core/services/generateJsTransformFromPrompt.js'`
+
+Uses an LLM to generate a sandboxed JavaScript transform function from a natural language description.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `opts.prompt` | `string` | required | Natural language description of the transform |
+| `opts.inputSchema` | `object` | `undefined` | JSON Schema for input validation |
+| `opts.outputSchema` | `object` | `undefined` | JSON Schema for output validation |
+| `opts.examples` | `{input, expected}[]` | `[]` | Test cases |
+| `opts.allowNet` | `boolean` | `false` | Allow network access in sandbox |
+| `opts.maxExecMs` | `number` | `5000` | Sandbox execution timeout |
+| `opts.model` | `string` | `'meta-llama/llama-4-maverick'` | LLM model |
+| `opts.retries` | `number` | `3` | Retry count on failure |
+
+**Returns**: `{ code: string, tests: object[], notes: string, plan: string, attempts: number, transform: Function }`
+
+---
+
+## 6. Prompt Optimization
+
+### `runAutoPromptSearch(taskConfig?, opts?)`
+
+**Import**: `import { runAutoPromptSearch } from 'flow-frame-core/services/autoPromptOptimizer.js'`
+
+Iterative prompt optimization: generates candidate system prompts, evaluates them on a dataset, feeds top performers back.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `taskConfig` | `TaskConfig` | `TASK_CONFIG` | Task definition with dataset |
+| `opts.maxIterations` | `number` | `5` | Search iterations |
+| `opts.promptsPerRound` | `number` | `3` | Candidates per round |
+| `opts.topPromptsToFeedBack` | `number` | `2` | Best prompts to feed back |
+| `opts.model` | `string` | `DEFAULT_MODEL` | LLM model |
+
+**Returns**: `{ bestPrompt: { name, text, avgScore }, allResults: object[] }`
+
+#### Helpers
+
+- `createTaskConfig(overrides)` — merge overrides onto base template
+- `addDatasetExamples(taskConfig, examples)` — append examples to dataset
+- `buildDatasetExample({ id, input, expectedOutput })` — construct one example
+
+---
+
+## 7. Image Analysis
+
+Vision-model queries on images.
+
+### `classifyImageQuery(model, question, image)` / `classifyImageQuestion({...})`
+
+**Import**: `import { classifyImageQuery } from 'flow-frame-core'`
+
+Ask a yes/no question about an image.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `model` | `string` | Vision model ID |
+| `question` | `string` | Yes/no question |
+| `image` | `string` | Base64 data URL |
+
+**Returns**: `{ answer: 'yes'|'no', confidence: number, reason: string }`
+
+### `generalClassifyImageQuery(model, question, image)` / `generalClassifyImageQuestion({...})`
+
+Ask any question about an image. Returns arbitrary JSON as specified in the question.
+
+### `findImageInImageQuery(model, imageA, imageB)` / `findImageInImage({...})`
+
+Determine if imageA appears inside imageB.
+
+**Returns**: `{ found: boolean, confidence: number, box: { x_center_rel, y_center_rel, width_rel, height_rel }, reason: string }`
+
+**REST**: `POST /api/prompts/image-check`
+```json
+{ "base64_image": "data:image/...", "model": "..." }
+→ { "success": true, "result": { "hasText": true, "text": "...", "hasIcon": false } }
+```
+
+---
+
+## 8. Audio / TTS
+
+### `processAudio(filePath, text, voice, keepExisting?)`
+
+**Import**: `import { processAudio } from 'flow-frame-core/services/audioService.js'`
+
+Generate speech audio from text via ElevenLabs and save to disk.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `filePath` | `string` | required | Output MP3 path |
+| `text` | `string` | required | Text to synthesize |
+| `voice` | `string` | required | ElevenLabs voice ID |
+| `keepExisting` | `boolean` | `true` | Skip if file exists |
+
+**Returns**: `{ audioPath: string, srtPath?: string }`
+
+### `generateAudio(text, voice, options?)`
+
+Low-level: calls ElevenLabs API, returns raw buffer + alignment data.
+
+**Returns**: `{ audioBuffer: Buffer, alignment: object }`
+
+**REST**: `POST /generateAudio`
+```json
+{ "text": "Hello world", "filePath": "output.mp3", "voiceId": "..." }
+→ { "ok": true }
+```
+
+---
+
+## 9. PDF Processing
+
+### `extractPdf({ pdfPath?, pdfBuffer?, outDir, extractText?, extractImages?, renderPages?, pageRenderScale?, password? })`
+
+**Import**: `import { extractPdf } from 'flow-frame-core/services/extractPdf.js'`
+
+Extract text, images, and page renders from a PDF.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `pdfPath` | `string` | — | Path to PDF file |
+| `pdfBuffer` | `Buffer` | — | PDF as buffer (alternative to path) |
+| `outDir` | `string` | required | Output directory |
+| `extractText` | `boolean` | `true` | Extract text per page |
+| `extractImages` | `boolean` | `true` | Extract embedded images |
+| `renderPages` | `boolean` | `false` | Render pages to PNG |
+| `pageRenderScale` | `number` | `2` | Render scale factor |
+| `password` | `string` | — | PDF password |
+
+**Returns**: `{ pdfPath, numPages, textByPage: string[], images: string[], renderedPages: string[] }`
+
+**REST**: `GET /pdf/extract?pdfPath=...&outDir=...`
+**REST**: `POST /pdf/extract-folder` — batch extract from a folder.
+
+---
+
+## 10. Email
+
+### `extractGmailBodyText(message)`
+
+**Import**: `import { extractGmailBodyText } from 'flow-frame-core'`
+
+Extract plain text from a Gmail API message object. Handles base64url decoding, MIME parts, HTML stripping.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `message` | `object` | Gmail API message object |
+
+**Returns**: `string` — extracted plain text.
+
+### Gmail Polling (server mode only)
+
+The server runs a `FilterGmailPoller` singleton that exposes:
+
+| Endpoint | Method | Description |
+|----------|--------|-------------|
+| `/api/emails` | GET | Paginated filtered inbox |
+| `/api/emails/:id` | GET | Specific email by ID |
+| `/api/stats` | GET | Email statistics |
+| `/api/polling/start` | POST | Start polling |
+| `/api/polling/stop` | POST | Stop polling |
+| `/api/polling/status` | GET | Auth + polling state |
+| `/api/config/filter` | PUT | Update filter config |
+
+---
+
+## 11. Web Scraping
+
+### `crawl(startUrl)`
+
+**Import**: `import { crawl } from 'flow-frame-core/scraper.js'`
+
+Depth-first crawl an entire website, staying on the same hostname.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `startUrl` | `string` | Starting URL |
+
+**Returns**: `Record<string, string>` — map of `{ url: pageText }`.
+
+**REST**: `POST /scrape`
+```json
+{ "domain": "https://example.com" }
+→ { "startUrl": "...", "pages": [...] }
+```
+
+---
+
+## 12. Storage
+
+JSON file-based key-value store with named profiles.
+
+**Import**: `import { getItem, setItem, ... } from 'flow-frame-core'`
+
+| Function | Params | Returns | Description |
+|----------|--------|---------|-------------|
+| `getItem(key)` | `string` | `any\|null` | Read a value by key |
+| `setItem(key, value)` | `string, any` | `void` | Write a value by key |
+| `listStorageProfiles()` | — | `string[]` | List available profiles |
+| `getStorageProfile()` | — | `string` | Current active profile |
+| `setStorageProfile(name)` | `string` | `void` | Switch active profile |
+| `readStorageProfile(name)` | `string` | `object\|null` | Read a profile's full content |
+| `saveStorageProfile(name, content)` | `string, any` | `void` | Save a profile |
+| `deleteStorageProfile(name)` | `string` | `boolean` | Delete a profile (not `'default'`) |
+| `isDocker()` | — | `boolean` | Detect Docker environment |
+
+**REST endpoints**:
+- `GET /storage-profiles` → `{ profiles, active }`
+- `GET /storage-profiles/:name` → profile content
+- `POST /storage-profiles` `{ name, content }` → create/update
+- `DELETE /storage-profiles/:name` → delete
+- `POST /storage-profiles/switch` `{ name }` → switch active
+- `POST /get-key-value` `{ key }` → `{ value }`
+- `POST /setObject` `{ key, value }` → `{ [key]: value }`
+
+---
+
+## 13. Queue Management
+
+Serial job queue with progress tracking.
+
+**Import**: `import queueManager from 'flow-frame-core'`
+
+| Method | Params | Returns | Description |
+|--------|--------|---------|-------------|
+| `enqueue(fn)` | `(signal, progress) => Promise` | `number` (job ID) | Add job to queue |
+| `status(id)` | `number` | `{ id, status, progress }\|null` | Check job status |
+| `cancel(id)` | `number` | `boolean` | Cancel a running job |
+| `list()` | — | `Job[]` | List all jobs |
+| `waitForJobId(id, cb)` | `number, function` | `Promise` | Wait for job completion |
+
+Events: `start`, `progress`, `done`, `aborted`, `error`.
+
+**REST**:
+- `GET /operations` → all queued jobs
+- `GET /operations/:jobId/status` → job status
+- `POST /operations/:jobId/cancel` → cancel job
+
+---
+
+## 14. File System Utilities
+
+**Import**: `import { ... } from 'flow-frame-core'` (re-exported from utils.js)
+
+| Function | Params | Returns | Description |
+|----------|--------|---------|-------------|
+| `ensureDir(dirPath)` | `string` | `Promise<void>` | Create directory recursively |
+| `readProjectFile(filePath)` | `string` | `any[]` | Read JSON array or NDJSON file |
+| `saveProjectFile(filePath, array)` | `string, any[]` | `void` | Write array as NDJSON |
+| `downloadFile(url, destPath, onProgress?)` | `string, string, fn?` | `Promise<void>` | Download a file |
+| `downloadFiles(urls, destFolder)` | `string[], string` | `Promise<string[]>` | Download multiple files |
+| `deleteFileSync(filePath)` | `string` | `void` | Delete a file |
+| `getFiles(dirPath)` | `string` | `Promise<string[]>` | List directory entries |
+| `readJsonFiles(dirPath)` | `string` | `Promise<string[]>` | List `.json` file paths |
+| `findFilesWithString(dir, str)` | `string, string` | `string[]` | Recursive file name search |
+| `getDownloadsFolder()` | — | `string` | OS Downloads folder path |
+| `unzipFile(zipPath, extractTo)` | `string, string` | `Promise<void>` | Extract zip file |
+| `sanitizeFileName(name)` | `string` | `string` | Clean illegal characters |
+| `sanitizeFolderName(name)` | `string` | `string` | Clean folder name |
+| `getLastModifiedTimestampSync(path)` | `string` | `number` | File mtime in ms |
+| `getSortedFilesByUpdatedAt(dir)` | `string` | `Promise<string[]>` | PNGs sorted by mtime |
+| `createZipFromFolders(root, folders, name, opts?)` | various | `Promise<string>` | Create zip archive |
+| `getLocalIp()` | — | `string` | Local IPv4 address |
+| `applyTemplate(template, data)` | `string, object` | `string` | Fill `{key}` placeholders |
+| `getMediaFileDuration(path)` | `string` | `Promise<number>` | Media duration in seconds |
+| `getVideoDimensions(path)` | `string` | `Promise<{width,height}>` | Video pixel dimensions |
+| `getVideoDuration(path)` | `string` | `Promise<{duration,width,height}>` | Full video info |
+
+**REST**:
+- `POST /readDir` `{ dirPath }` → directory listing
+- `POST /readFiles` `{ filePaths }` → file contents
+- `POST /readFile` `{ filePath, encoding? }` → single file content
+- `POST /delete-files` `{ filePaths }` → per-file success/failure
+- `POST /clear-directory` `{ directory }` → delete all files in dir
+- `POST /moveFiles` `{ sourcePaths, destinationDir }` → move files
+- `POST /waitForFiles` `{ directory, filePrefixes, timeout? }` → long-poll for file presence
+- `POST /getFiles` `{ directory, filePrefixes, expectedCounts? }` → check file existence
+- `POST /wait-for-file-stable` `{ file_path, min_size_mb?, max_wait_minutes? }` → wait for file to stop changing
+
+---
+
+## 15. Template & Variable Generation
+
+Generate realistic fake data and fill templates.
+
+### `generateTextFromTemplate(template, entities, options?)`
+
+**Import**: `import { generateTextFromTemplate } from 'flow-frame-core/services/variableGenerator.js'`
+
+Full pipeline: generate entity values via LLM, then fill a template.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `template` | `string` | Template with `{{KEY}}` placeholders |
+| `entities` | `EntityDef[]` | `[{ key, type, description }]` |
+| `options.model` | `string` | LLM model for generation |
+| `options.reuseProbability` | `number` | 0-1 chance of reusing existing pool value |
+
+**Returns**: `{ email: string, valuesByKey: object }`
+
+### `fillTemplate(template, valuesByKey)`
+
+Simple `{{KEY}}` replacement without LLM.
+
+### `generateEntityValues(entities, options?)`
+
+Generate values for all entities, managing the value pool.
+
+**Returns**: `{ valuesByKey: object, pools: object }`
+
+**REST**: `POST /generate-text-from-template`
+
+---
+
+## 16. Browser & Desktop Automation
+
+Control the OS desktop, browser windows, mouse, and keyboard.
+
+### Browser Control
+
+**Import**: `import { BrowserController } from 'flow-frame-core'`
+
+| Method | Params | Description |
+|--------|--------|-------------|
+| `BrowserController.openChrome({ url })` | `string` | Open URL in Chrome |
+| `BrowserController.closeChrome()` | — | Close Chrome |
+| `BrowserController.closeChromeTab({ domain })` | `string` | Close tab by domain |
+| `BrowserController.bringAppToFront({ appName })` | `string` | Focus an app window |
+| `BrowserController.readFile({ filePath })` | `string` | Read a local file |
+
+**REST**:
+- `POST /openChrome` `{ url }`
+- `POST /closeChrome`
+- `POST /closeChromeTab` `{ domain }`
+- `POST /bringAppToFront` `{ appName }`
+- `POST /open-app` `{ appName }` — launch any app
+- `POST /close-app` `{ appName, force? }` — kill any app
+
+### Mouse & Keyboard (requires robotjs)
+
+**Import**: `import { moveMouse, mouseClick, ... } from 'flow-frame-core/operations.js'`
+
+| Function | Params | Description |
+|----------|--------|-------------|
+| `moveMouse(position)` | `{left, top, height, width}` | Move mouse to element center (with browser chrome offset) |
+| `moveMouseDesktop(position)` | `{left, top, height, width}` | Move mouse (no offset) |
+| `moveMouseRelative(offset)` | `{left, top}` | Relative mouse move |
+| `mouseClick()` | — | Left click at current position |
+| `pressEscapeAsync()` | — | Press Escape key |
+| `pressKey(params)` | `{keys, ctrl?, shift?, alt?}` | Press key with modifiers |
+| `typeText(params)` | `{keys, ctrl?, shift?, alt?}` | Type key combo |
+| `pasteText(text)` | `string` | Clipboard paste |
+| `clearAllText()` | — | Select-all + delete |
+| `pressReturn()` | — | Press Enter |
+| `scroll(x?, y?)` | `number, number` | Scroll mouse wheel |
+| `fileModalOperate(filePath)` | `string` | Navigate OS file picker |
+
+---
+
+## 17. Screenshot & Screen Capture
+
+### `captureScreenshotBase64(region?)`
+
+**Import**: `import { captureScreenshotBase64 } from 'flow-frame-core/inference/capturescreenshot.js'`
+
+Capture screen (or region) as base64 PNG data URL.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `region` | `{x, y, width, height}` | Optional crop region |
+
+**Returns**: `string` — `data:image/png;base64,...`
+
+### `captureFullScreenshot(targetFolder)`
+
+Save full-screen capture as a PNG file.
+
+**Returns**: `string` — filename.
+
+### `captureScreenshot(targetFolder, filename, region?)`
+
+Save a named screenshot file.
+
+**REST**: `POST /take-screenshot` `{ appName }` → `{ filename, filePath }`
+
+---
+
+## 18. Self-Learning Vision Pipeline
+
+Computer vision system for recognizing and learning UI screens.
+
+### `ingestScreenshot({ imagePath, meta?, tags? })`
+
+**Import**: `import { ingestScreenshot } from 'flow-frame-core/services/self-learning/injest.js'`
+
+Ingest a screenshot: copies to store, computes perceptual hashes, logs event.
+
+**Returns**: Event object with `{ aHash, dHash, width, height, ... }`.
+
+### `recognizeScreen(imagePath, options?)`
+
+**Import**: `import { recognizeScreen } from 'flow-frame-core/services/self-learning/recognize.js'`
+
+Match a screenshot against known screen signatures.
+
+| Param | Type | Default | Description |
+|-------|------|---------|-------------|
+| `imagePath` | `string` | required | Path to screenshot |
+| `options.topK` | `number` | `5` | Candidates to evaluate |
+| `options.unknownThreshold` | `number` | — | Min score for a match |
+
+**Returns**: `{ screenId, confidence, bestScore, candidates, reason? }`
+
+### `discoverScreens(options?)`
+
+**Import**: `import { discoverScreens } from 'flow-frame-core/services/self-learning/discover.js'`
+
+Cluster ingested screenshots and auto-generate screen signatures.
+
+**Returns**: `{ learned, catalogs, clusters, message }`
+
+### `recordTransition({ takeScreenshot, recognizeScreen, performAction, action, meta? })`
+
+Record a before/after screen state transition.
+
+### `buildTransitionGraph()` / `findPath(graph, start, goal)`
+
+Build a directed graph of screen transitions; find shortest path between screens.
+
+### Perceptual Hashing
+
+| Function | Description |
+|----------|-------------|
+| `aHashHex(imagePath)` | Average hash (64 hex chars) |
+| `dHashHex(imagePath)` | Difference hash (64 hex chars) |
+| `hammingHex(a, b)` | Hamming distance between hashes |
+| `preScoreFromDistances(aDist, dDist)` | 0-1 similarity score |
+
+### Image Utilities
+
+| Function | Description |
+|----------|-------------|
+| `fileToDataUrl(imagePath)` | Image file → base64 data URL |
+| `cropToDataUrl({ imagePath, bbox })` | Crop region → base64 data URL |
+| `getImageSize(imagePath)` | `{ width, height }` |
+
+**REST**:
+- `POST /recognize-screen-signature` `{ filePath }` → recognition result
+- `POST /api/self-learning/injest` `{ imagePath, metadata?, tags? }` → event
+- `POST /api/self-learning/discover` `{ ...options }` → discovery result
+- `POST /api/self-learning/enhance` `{ userPrompt, model? }` → LLM response
+
+---
+
+## 19. UI Automation Planning
+
+AI-powered planning that converts natural language into executable desktop automation steps.
+
+### `UiPlanner` class
+
+**Import**: `import { UiPlanner } from 'flow-frame-core/services/uiPlanner.js'`
+
+```js
+const planner = new UiPlanner(uiMap, pathFinder, macroRegistry, automationFiles);
+const plan = await planner.plan('Upload a video titled "Hello"', 'home', {}, 'gpt-4o');
+```
+
+**`plan(request, startStateId?, context?, model?)`** returns `{ steps: ExecutableStep[] }`.
+
+### `StepByStepAiPlanner` class
+
+**Import**: `import { StepByStepAiPlanner } from 'flow-frame-core/services/stepByStepAiPlanner.js'`
+
+Extends `UiPlanner` with dependency-graph-aware planning. Identifies key automation steps by ID rather than generating field-level goals.
+
+### `generateUIMap(files)`
+
+**Import**: `import { generateUIMap } from 'flow-frame-core/services/uiMapService.js'`
+
+Build a UI state machine map from automation recording files.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `files` | `string[]` | Paths to `.automation` files |
+
+**Returns**: `{ version, states, edges }`
+
+**REST**: `POST /generate-ui-map` `{ files }` → `{ uiMap }`
+
+---
+
+## 20. Workflow File Management
+
+CRUD for workflow/automation files.
+
+### Programmatic
+
+**Import**: `import { WorkflowController } from 'flow-frame-core'`
+
+| Method | Params | Description |
+|--------|--------|-------------|
+| `loadConfiguration({ name, configFolder })` | `string, string` | Load a `.workflow` file |
+| `getWorkflowConfig({ name, workflowDictionary })` | `string, object` | Lookup from in-memory dictionary |
+
+### `buildWorkflowDirectory(jsonInput, targetDir)`
+
+Materialize a `{ files: [{ path, content }] }` structure to disk.
+
+### `runWorkflowCLI(targetDir, workflowJsonPath?, parameters?)`
+
+Run `node src/cli.js` in a target directory.
+
+### `installDependencies(targetDir)`
+
+Run `npm install` in a target directory.
+
+**REST**:
+- `GET /configurations` → list all config files
+- `GET /loadConfiguration?name=...` → load config
+- `POST /saveConfiguration` `{ name, nodes, edges }` → save
+- `POST /buildWorkflowDirectory` `{ jsonInput, targetDir }`
+- `POST /runWorkflowCLI` `{ targetDir, workflowJsonPath?, parameters? }`
+- `POST /installDependencies` `{ targetDir }`
+- Full step-workflow CRUD at `/api/workflows/*` (see REST API section)
+
+---
+
+## 21. Extension System
+
+Discover and run pluggable npm-based extensions.
+
+### `scanExtensions(extensionFolders)`
+
+**Import**: `import { scanExtensions } from 'flow-frame-core/extensionUtils.js'`
+
+Scan directories for subdirectories containing `api.json` files.
+
+| Param | Type | Description |
+|-------|------|-------------|
+| `extensionFolders` | `string\|string[]` | Paths to scan |
+
+**Returns**: `{ location, ...apiJsonData }[]`
+
+### `ExtensionController`
+
+**Import**: `import { ExtensionController } from 'flow-frame-core'`
+
+| Method | Description |
+|--------|-------------|
+| `getExtensions({ extensionFolders })` | Scan and return extensions |
+| `getExtensionFolders()` | Get configured folders |
+| `installDependencies({ targetDir })` | npm install in extension dir |
+| `runWorkflowCLI({ targetDir, workflowJsonPath, parameters })` | Run extension CLI |
+
+**REST**: `GET /get-extensions` → `{ extensions: [...] }`
+
+---
+
+## 22. Media Processing
+
+### Video/Audio
+
+| Function | Params | Returns | Description |
+|----------|--------|---------|-------------|
+| `getMediaFileDuration(path)` | `string` | `Promise<number>` | Duration in seconds |
+| `getVideoDimensions(path)` | `string` | `Promise<{width,height}>` | Pixel dimensions |
+| `getVideoDuration(path)` | `string` | `Promise<{duration,width,height}>` | Full video info |
+| `processFile(targetFile)` | `string` | `void` | Process project file through media pipeline |
+| `fix(text)` | `string` | `string` | Sanitize for filename/prompt use |
+
+**REST**:
+- `POST /get-video-dimensions` `{ filePath }` → dimensions
+- `POST /getAudioDurations` `{ audioFiles, videos }` → duration map
+- `GET /image-dimensions/:name` → `{ width, height }`
+- `POST /get-media-dimensions` `{ imageName }` or `{ videoName, timestamp }` → dimensions
+
+### YOLO Training Data
+
+**REST**:
+- `GET /fetch-training-folders` → folder names
+- `GET /fetch-training-data/:folder/:size` → YAML config as JSON
+- `GET /list-images/:folder/:size/:split` → image filenames
+- `GET /fetch-labels/:folder/:size/:split/:image` → YOLO bounding boxes
+
+---
+
+## 23. Configuration Management
+
+File-based config CRUD.
+
+**REST**:
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/config/files` | List all config files + folder tree |
+| GET | `/api/config/file/:name` | Read a config file |
+| POST | `/api/config/file/:name` | Save/update (with optional backup) |
+| POST | `/api/config/create` | Create new config file |
+| DELETE | `/api/config/file/:name` | Delete (with optional backup) |
+| PUT | `/api/config/rename/:name` | Rename a config file |
+| POST | `/api/config/validate/:name` | Validate file content |
+| GET | `/api/config/info` | Config folder metadata |
+
+---
+
+## 24. Constants & Selectors
+
+**Import**: `import { KEYS, MODES, SUNO_SELECTORS, YOUTUBE_SELECTORS } from 'flow-frame-core'`
+
+### `MODES`
+
+Operating mode constants:
+- `MIDJOURNEY`, `MIDJOURNEY_IMAGES_TO_VIDEO`, `MIDJOURNEY_DOWNLOAD_MEDIA`, `MIDJOURNEY_DOWNLOAD_PUBLIC_MEDIA`
+- `CLEAR_MIDJOURNEY_IMAGES`
+- `SUNO`, `SUNO_FILM`
+- `YOUTUBE`
+- `FLOW_FRAME`
+
+### `KEYS`
+
+80+ storage key constants for all configuration, paths, and state:
+- Folder paths: `workflow_folder`, `ui_map_folder`, `prompt_lib_folder`, `media_folder`, `audio_folder`, `video_folder`, `config_folder`, `learning_dir`, `extension_folders`, `training_jobs_folder`
+- LLM config: `default_llm_model`, `default_llm_multi_modal_model`
+- Mode/state: `mode`, `procedure`, `sourceFile`, `current_folder`, `images_folder`
+- Suno state: `suno_state`, `suno_current_name`
+- UI automation: `boxes`, `element_selectors`, `chrome_requests`
+- And many more.
+
+### `SUNO_SELECTORS` / `YOUTUBE_SELECTORS`
+
+CSS selector maps for Suno and YouTube Studio UI elements.
+
+---
+
+## REST API Quick Reference
+
+All endpoints assume `Content-Type: application/json` unless noted. Server runs on port configured in `peers.ts` (default 3000).
+
+### Core
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/` | Health check |
+| GET | `/health` | Detailed health (uptime, memory, WS count) |
+| GET | `/health/detailed` | Full process info |
+
+### LLM
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| POST | `/api/prompts/chat` | Run LLM prompt |
+| POST | `/api/prompts/image-check` | Analyze image for text/icons |
+| POST | `/analyze-image-grok` | Vision model image analysis |
+
+### Workflows
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/configurations` | List workflow configs |
+| GET | `/loadConfiguration?name=` | Load a workflow |
+| POST | `/saveConfiguration` | Save a workflow |
+| GET | `/api/workflows` | List automation workflows |
+| GET | `/api/workflows/:id` | Get workflow |
+| POST | `/api/workflows/:id` | Save workflow |
+| DELETE | `/api/workflows/:id` | Delete workflow |
+| POST | `/api/workflows/:id/execute` | Execute workflow steps |
+| POST | `/api/workflows/:id/validate` | Validate workflow |
+| POST | `/api/workflows/:id/duplicate` | Duplicate workflow |
+| POST | `/api/workflows/import` | Import workflow file |
+
+### Storage & Config
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| POST | `/get-key-value` | Get storage value |
+| POST | `/setObject` | Set storage value |
+| GET | `/storage-profiles` | List profiles |
+| POST | `/storage-profiles/switch` | Switch profile |
+| GET | `/api/config/files` | List config files |
+| GET | `/api/config/file/:name` | Read config |
+| POST | `/api/config/file/:name` | Save config |
+
+### Files
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| POST | `/readDir` | List directory |
+| POST | `/readFile` | Read file |
+| POST | `/readFiles` | Read multiple files |
+| POST | `/delete-files` | Delete files |
+| POST | `/moveFiles` | Move files |
+| POST | `/waitForFiles` | Wait for files to appear |
+| POST | `/wait-for-file-stable` | Wait for file to stabilize |
+
+### Browser & Desktop
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| POST | `/openChrome` | Open Chrome URL |
+| POST | `/closeChrome` | Close Chrome |
+| POST | `/closeChromeTab` | Close Chrome tab |
+| POST | `/bringAppToFront` | Focus app window |
+| POST | `/open-app` | Launch application |
+| POST | `/close-app` | Kill application |
+| POST | `/take-screenshot` | Capture screenshot |
+| POST | `/recognize-screen-signature` | Recognize UI screen |
+
+### Media
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| POST | `/generate-audio` | Generate audio from text |
+| POST | `/generateAudio` | TTS via ElevenLabs |
+| POST | `/get-video-dimensions` | Video dimensions |
+| POST | `/getAudioDurations` | Media durations |
+| GET | `/image-dimensions/:name` | Image dimensions |
+| GET | `/api/media` | List media files |
+| GET | `/api/media/:filename` | Serve media file |
+| POST | `/api/media/upload` | Upload media |
+| DELETE | `/api/media/:filename` | Delete media |
+
+### PDF
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/pdf/extract` | Extract from PDF |
+| POST | `/pdf/extract-folder` | Extract from PDF folder |
+
+### Scraping
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| POST | `/scrape` | Crawl a website |
+
+### Self-Learning
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| POST | `/api/self-learning/injest` | Ingest screenshot |
+| POST | `/api/self-learning/discover` | Discover screens |
+| POST | `/api/self-learning/enhance` | LLM enhance prompt |
+| GET | `/api/self-learning/media-folder` | Get media folder |
+| POST | `/api/self-learning/media-folder` | Set media folder |
+
+### UI Maps & Prompt Libraries
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/api/ui-maps` | List UI maps |
+| GET | `/api/ui-maps/:id` | Get UI map |
+| POST | `/api/ui-maps/:id` | Save UI map |
+| DELETE | `/api/ui-maps/:id` | Delete UI map |
+| POST | `/generate-ui-map` | Generate from files |
+| GET | `/api/prompt-libs` | List prompt libraries |
+| GET | `/api/prompt-libs/:id` | Get prompt library |
+| POST | `/api/prompt-libs/:id` | Save prompt library |
+| DELETE | `/api/prompt-libs/:id` | Delete prompt library |
+
+### Email
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/api/emails` | Filtered inbox |
+| GET | `/api/emails/:id` | Email by ID |
+| GET | `/api/stats` | Email stats |
+| POST | `/api/polling/start` | Start polling |
+| POST | `/api/polling/stop` | Stop polling |
+
+### Extensions
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/get-extensions` | List extensions |
+| POST | `/buildWorkflowDirectory` | Build from JSON |
+| POST | `/runWorkflowCLI` | Run workflow CLI |
+| POST | `/installDependencies` | npm install |
+
+### Training Data
+
+| Method | Path | Purpose |
+|--------|------|---------|
+| GET | `/fetch-training-folders` | List training folders |
+| GET | `/fetch-training-data/:folder/:size` | Dataset config |
+| GET | `/list-images/:folder/:size/:split` | List images |
+| GET | `/fetch-labels/:folder/:size/:split/:image` | YOLO labels |

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "flow-frame-core",
-    "version": "0.1.6",
+    "version": "0.1.7",
     "type": "module",
     "main": "dist/index.js",
     "types": "dist/index.d.ts",
@@ -10,6 +10,7 @@
     "files": [
         "dist",
         "README.md",
+        "TOOLS.md",
         "LICENSE"
     ],
     "keywords": [