@newsails/veil-cli 1.0.1

package/PLAN/13-operations.md

@@ -0,0 +1,212 @@
+# 13 — Operations
+
+## Failure & Recovery
+
+### LLM Call Failure
+- Retry up to `retry.maxAttempts` times with `retry.delaySeconds` delay (exponential backoff via `openai` SDK)
+- If exhausted and `onExhausted: "fail"` → task status = failed
+- If exhausted and `onExhausted: "wait"` → task status = waiting (human can retry)
+
+### Iteration/Duration Limits
+- `maxIterations` reached → task failed, error = "Max iterations exceeded"
+- `maxDurationSeconds` reached → task failed, error = "Max duration exceeded"
+- Both configurable globally (settings.json) and per-agent per-mode (agent.json)
+
+### Startup Recovery
+On server start:
+1. Open SQLite database, run pending migrations
+2. Scan for tasks with status `pending` or `processing`
+3. Set all to `failed` with error: "Runtime terminated unexpectedly"
+4. Auto-start all agents with `daemon.enabled: true`
+5. Begin accepting API requests
+
+### Graceful Shutdown
+On `veil stop` or SIGTERM/SIGINT:
+1. Stop accepting new HTTP connections
+2. Stop all cron schedulers
+3. Wait for active agent loops to reach a safe point (tool boundary)
+4. Leave Ably presence, close Ably connection
+5. Close SQLite database (checkpoints WAL)
+6. Delete runtime.pid, exit 0
+7. Failsafe: force exit after 10s timeout
+
+### Multiple Instances
+- Multiple one-shot instances: fully supported (SQLite WAL handles concurrent writes)
+- Multiple server instances on same port: OS port conflict error
+- Multiple server instances on different ports: supported, shared SQLite
+- Concurrent memory file writes: last-write-wins in v1
+
+---
+
+## Testing Strategy (v1)
+
+### Scope
+**Smoke test + Unit tests + API tests** — fastest path to ship with confidence.
+
+### Smoke Test (`test/smoke.js`) — Run First
+A fast automated test run immediately after `npm install`. Catches ~14 of the 20 real bugs found during testing before any manual effort. Runs with `npm test`.
+
+What it verifies:
+1. Server starts without errors (Node version check passes, all deps resolve)
+2. `GET /health` returns 200
+3. Example agent (`examples/agents/hello/`) loads and validates against schema
+4. `POST /agents/hello/chat` returns a response (mock LLM)
+5. Server shuts down cleanly
+
+```bash
+# package.json
+{ "scripts": { "test": "node test/smoke.js && node --test test/unit/ test/api/" } }
+```
+
+### Unit Tests
+- Agentic loop logic (mock LLM responses via fetch mock)
+- System prompt assembly (7-layer composition)
+- Tool argument validation (ajv)
+- Permission evaluation (deny/ask/allow resolution)
+- Settings resolution (merge order — including field name constants)
+- Agent discovery (file lookup logic with `getProjectAgentsDir(cwd)`)
+- SQLite queries (in-memory test database)
+- `context.js` singleton (set/get, throws if uninitialized)
+- `id.js` ID generation (no nanoid, crypto-based)
+
+### API Tests
+- All REST endpoints (happy path + error cases)
+- Authentication (with/without secret)
+- Pagination
+- Task lifecycle (create → process → finish)
+- Session lifecycle (create → message → close)
+
+### Integration Test (`test/integration.js`)
+Boots the full server with mock LLM. Exercises task lifecycle end-to-end. Catches wiring bugs that unit tests miss (circular deps, wrong exports, signature mismatches). Should pass before any release.
+
+### Mock LLM
+For tests that need LLM responses: mock native `fetch` (not an openai SDK mock) to return predefined responses with tool calls. No real LLM calls in CI. Pattern:
+```javascript
+global.fetch = async (url, options) => {
+  return { ok: true, json: async () => ({ choices: [{ message: { content: 'Hello!', tool_calls: [] } }] }) };
+};
+```
+
+### Test Runner
+Node.js built-in test runner (`node:test`) — zero external deps.
+
+---
+
+## Compatibility
+
+### Claude Code Compatibility (via Import)
+VeilCLI imports Claude Code project structures via `veil import`:
+
+| Claude Code | VeilCLI Equivalent |
+|-------------|---------------------|
+| `CLAUDE.md` | Converted to `.veil/AGENT.md` via `veil import` |
+| `.claude/` | Read as `.veil/` |
+| `.claude/skills/` | Read as skills |
+| `settings.json` | All Claude Code fields supported |
+| MCP server definitions | Fully supported |
+| Permission format | Fully supported |
+| Template variables | Fully supported |
+
+### Import Adapters
+
+`veil import --type=<adapter> --source <path>`
+
+| Adapter | Source | Transform |
+|---------|--------|-----------|
+| `claude-code` | Claude Code project | CLAUDE.md → AGENT.md, .claude/ → .veil/, generates agent.json with mode config |
+| `openclaw` | OpenClaw project | Maps OpenClaw agent definitions to VeilCLI format |
+| `raw` | Any folder | Copies as-is, creates minimal agent.json |
+
+**Interactive import:** Since VeilCLI agents have modes (chat/task/daemon) which Claude Code doesn't, the import tool asks follow-up questions:
+- Which modes should this agent support?
+- What tools should be available per mode?
+- What model should it use?
+- Any permission overrides?
+
+---
+
+## Veil.com Integration (v1)
+
+### API Surface
+| Endpoint | Purpose |
+|----------|---------|
+| `POST /api/v1/validate-token` | Validate veil_token |
+| `GET /api/v1/registry/agents` | List available agents |
+| `GET /api/v1/registry/agents/:name` | Download agent |
+| `GET /api/v1/registry/tools` | List available tools |
+| `GET /api/v1/registry/tools/:name` | Download tool |
+| `POST /api/v1/ably/token` | Get scoped Ably TokenRequest |
+
+### Version Checking
+- Each cached agent/tool has a `version.json` with SHA256 hash
+- On startup (if connected), VeilCLI checks hashes against Veil.com manifest
+- If different → auto-update global agents/tools
+- Project-local agents are never auto-updated (user owns them)
+
+### No Usage Analytics
+VeilCLI does NOT send usage data to Veil.com. Token counts and costs are tracked locally in SQLite. The AI provider (OpenRouter, etc.) handles their own analytics.
+
+---
+
+## Images & Files
+
+### Image Handling
+- **Vision:** Images sent as base64 in messages (Claude, GPT-4V, Gemini support)
+- **From URL:** `web_fetch` auto-converts image URLs to base64
+- **From file:** `read_file` on image files returns base64
+- **Generation:** Via standard LLM API (models that support image generation)
+- **MCP:** Available for users who want DALL-E, Stable Diffusion, etc.
+
+### File/URL Resolution
+Task input supports multiple file sources:
+
+| Type | Format | Example |
+|------|--------|---------|
+| Local file | `./path/to/file` | `./src/auth.js` |
+| Absolute path | `/path/to/file` | `/home/user/project/src/auth.js` |
+| HTTP URL | `https://...` | `https://raw.githubusercontent.com/...` |
+| Data URL | `data:...` | `data:text/plain;base64,SGVsbG8=` |
+
+Unified resolver with mime type detection. URLs fetched and cached for the session duration.
+
+---
+
+## Modularity & Extensibility
+
+The codebase is designed to be modular, making it easy to add or modify features without touching unrelated code.
+
+### Source Code Boundaries
+| Module | Responsibility | Can change without affecting |
+|--------|---------------|----------------------------|
+| `core/loop.js` | Agentic loop | API routes, CLI |
+| `core/registry.js` | Tool registration | Agent loading, prompts |
+| `core/prompt.js` | System prompt assembly | Tool execution, storage |
+| `core/memory.js` | Memory read/write/search/export | Loop, tools |
+| `core/queue.js` | Agent message queue | Storage, prompts |
+| `api/` | REST transport | Core logic |
+| `cli/` | CLI transport | Core logic |
+| `infrastructure/` | SQLite, Ably, MCP, cron | Core logic |
+| `tools/` | Individual tool implementations | Each tool is independent |
+| `utils/context.js` | CWD singleton | All modules (no circular deps) |
+| `utils/paths.js` | Path construction | Prevents inline `.veil/` string bugs |
+| `settings/fields.js` | Field name constants | Prevents field name drift across modules |
+| `llm/client.js` | Native fetch LLM client | Replaces openai SDK, no version drift |
+
+### Extension Points
+- **New mode:** Add a new key to `modes` in agent.json, handle in `loop.js`
+- **New built-in tool:** Add file to `tools/`, register in `registry.js`
+- **New transport:** Add folder alongside `api/` and `cli/` (e.g., `grpc/`)
+- **New storage backend:** Replace `infrastructure/database.js` (core uses abstract queries)
+- **New MCP transport:** Add to `infrastructure/mcp.js`
+
+### Implementation Standards (enforced by code review + tests)
+
+1. **Schema-first loading:** `agent.json` and `settings.json` are validated against their JSON schemas using ajv at load time. Any invalid file produces a clear error and the agent/setting is not used. Never use unvalidated config.
+
+2. **No `process.cwd()` in modules:** Only `src/cli/index.js` may call `process.cwd()`. All other modules receive `cwd` via `context.getCwd()`. Enforced by grep in CI: `grep -r 'process.cwd()' src/ --exclude='src/cli/*'` must return empty.
+
+3. **Settings field names from constants:** All references to settings field names (`base_url`, `api_key`, etc.) must use `src/settings/fields.js` exports. No inline string literals.
+
+4. **Locked dependencies:** `package.json` uses exact versions. CI runs `npm ci`. Before merging, run `node -e "const p = require('./package.json'); const used = []; /* grep all requires */"` to audit all `require()`d packages are listed.
+
+5. **Example agent in CI:** `examples/agents/hello/agent.json` is validated against the schema in the smoke test. If it fails, the build fails. This is the canary for schema changes.

package/PLAN/README.md

@@ -0,0 +1,23 @@
+# VeilCLI — Specification v3.0
+
+VeilCLI is an **enterprise autonomous agent runtime** — a locally-running Node.js server that manages autonomous AI agents via a REST API.
+
+---
+
+## Table of Contents
+
+| # | Section | Description |
+|---|---------|-------------|
+| 01 | [Vision](01-vision.md) | Identity, philosophy, non-goals |
+| 02 | [Tech Stack](02-tech-stack.md) | Language, dependencies, confirmed choices |
+| 03 | [Agents](03-agents.md) | Agent definition, modes (chat/task/daemon/subagent), agent.json, sub-agents |
+| 04 | [Runtime](04-runtime.md) | Agentic loop, model delegation, context engineering, system prompts |
+| 05 | [Tools](05-tools.md) | Built-in tools, custom JS tools, MCP, skills, discovery |
+| 06 | [Communication](06-communication.md) | A2A messaging, sub-agent spawning, task system |
+| 07 | [Storage](07-storage.md) | SQLite, retention, file-based config |
+| 08 | [API & CLI](08-api-cli.md) | REST API endpoints, CLI commands |
+| 09 | [Permissions & Hooks](09-permissions.md) | Permission system, PreToolUse/PostToolUse |
+| 10 | [Ably](10-ably.md) | Remote access via Ably Realtime |
+| 11 | [File Formats](11-file-formats.md) | agent.json, config.json, settings.json, AGENT.md, SKILL.md, tool.json |
+| 12 | [Folder Structure](12-folder-structure.md) | Project-level, global |
+| 13 | [Operations](13-operations.md) | Failure/recovery, testing, compatibility, import, Veil.com |

package/README.md

@@ -0,0 +1,128 @@
+# VeilCLI
+
+**Enterprise autonomous agent runtime — local, API-first, multi-model.**
+
+VeilCLI lets you define AI agents as simple files, run them as a local REST server, and control them through a clean HTTP API or interactive CLI. Agents can chat, run multi-step tasks with tools, spawn sub-agents, schedule cron jobs, and maintain persistent memory — all without external services.
+
+---
+
+## Key Features
+
+- **4 agent modes** — Chat, Task, Daemon (cron), Subagent
+- **24 built-in tools** — bash, file I/O, web search/fetch, memory, multi-agent communication, todos, and more
+- **Persistent memory** — per-agent and project-wide Markdown memory files
+- **Multi-agent orchestration** — parallel fan-out, durable subscriptions, sync/async spawning
+- **Layered config** — global → project → local → CLI flags
+- **Any OpenAI-compatible LLM** — OpenRouter, local Ollama, Azure, etc.
+- **Zero external dependencies** — SQLite storage, native fetch, Node.js only
+
+---
+
+## Requirements
+
+- **Node.js ≥ 18.3.0**
+- An OpenAI-compatible API key (e.g. [OpenRouter](https://openrouter.ai))
+
+---
+
+## Installation
+
+```bash
+npm install -g @newsails/veil-cli
+```
+
+---
+
+## Quick Start
+
+```bash
+# 1. Set up a workspace
+mkdir my-workspace && cd my-workspace
+
+# 2. Save your API key (Optional)
+veil login --key sk-or-v1-...
+
+# 3. Start the server
+veil start
+
+# 4. Chat with the default agent
+curl -X POST http://localhost:5050/agents/hello/chat \
+  -H "Content-Type: application/json" \
+  -d '{"message": "Hello!"}'
+
+# 5. Run a task with tools
+curl -X POST http://localhost:5050/agents/assistant/task \
+  -H "Content-Type: application/json" \
+  -d '{"input": "List all .js files in the current directory"}'
+```
+
+→ See **[docs/guide/01-quickstart.md](docs/guide/01-quickstart.md)** for the full walkthrough.
+
+---
+
+## Documentation
+
+### API Reference
+| Doc | Contents |
+|-----|----------|
+| [docs/api/README.md](docs/api/README.md) | Overview, auth, error codes |
+| [docs/api/01-system.md](docs/api/01-system.md) | `/health`, `/status`, `/shutdown` |
+| [docs/api/02-agents.md](docs/api/02-agents.md) | List and inspect agents |
+| [docs/api/03-chat.md](docs/api/03-chat.md) | Chat endpoint, multi-turn sessions |
+| [docs/api/04-tasks.md](docs/api/04-tasks.md) | Task lifecycle, events, polling |
+| [docs/api/05-sessions.md](docs/api/05-sessions.md) | Session management and messages |
+| [docs/api/06-daemons.md](docs/api/06-daemons.md) | Daemon control endpoints |
+
+### Project Guide
+| Doc | Contents |
+|-----|----------|
+| [docs/guide/README.md](docs/guide/README.md) | Guide index |
+| [docs/guide/01-quickstart.md](docs/guide/01-quickstart.md) | Install, first agent, first request |
+| [docs/guide/02-folder-structure.md](docs/guide/02-folder-structure.md) | `.veil/` layout explained |
+| [docs/guide/03-configuration.md](docs/guide/03-configuration.md) | `settings.json` + `auth.json` full reference |
+| [docs/guide/04-agents.md](docs/guide/04-agents.md) | `agent.json`, `AGENT.md`, all 4 modes |
+| [docs/guide/05-cli.md](docs/guide/05-cli.md) | `veil` CLI commands reference |
+| [docs/guide/06-tools.md](docs/guide/06-tools.md) | All 24 built-in tools |
+| [docs/guide/07-permissions.md](docs/guide/07-permissions.md) | Permissions system |
+| [docs/guide/08-memory.md](docs/guide/08-memory.md) | Memory + context compaction |
+| [docs/guide/09-multi-agent.md](docs/guide/09-multi-agent.md) | Orchestration patterns |
+| [docs/guide/10-daemons.md](docs/guide/10-daemons.md) | Daemon / cron agents |
+
+---
+
+## CLI Reference (Quick)
+
+```
+veil start [--port 5050] [--folder ./workspace] [--secret <token>]
+veil run --agent hello --input "What time is it?"
+veil stop
+veil status
+veil agents list
+veil agents inspect <name>
+veil login --key <api-key> [--global]
+```
+
+→ Full reference: [docs/guide/05-cli.md](docs/guide/05-cli.md)
+
+---
+
+## Project Layout
+
+```
+VeilCli/
+  cli/          ← Veil CLI entry point
+  api/          ← Express REST server + routes
+  core/         ← agent loader, loop, router, memory, compaction
+  tools/        ← 24 built-in tools
+  infrastructure/ ← SQLite database + scheduler
+  utils/        ← settings, paths, context
+  schemas/      ← JSON schemas for agent.json and settings.json
+  system-prompts/ ← base system prompt layers
+  examples/     ← reference agent definitions
+```
+
+---
+
+## License
+
+MIT

package/REPORT.md

@@ -0,0 +1,174 @@
+# Dead Code & Unused Dependencies Audit Report
+
+**Project:** VeilCLI  
+**Date:** 2026-03-04  
+**Scope:** Production code in `core/`, `utils/`, `infrastructure/`, `api/`, `tools/`, `llm/`, `settings/`
+
+---
+
+## Summary
+
+| Category | Count |
+|----------|-------|
+| Unused Exports | 14 |
+| Dead Internal Functions | 2 |
+| Orphaned Database Columns | 3 |
+| Undocumented/Untested Routes | 0 |
+| Orphaned Tools | 0 |
+| Unused Dependencies | 0 |
+
+---
+
+## Unused Exports
+
+Functions/constants exported from modules but never imported elsewhere in production code.
+
+| File | Export | Reason |
+|------|--------|--------|
+| `core/agent.js:182` | `loadAgentFromFolder` | Only used internally by `loadAgent()` in same file — no external imports |
+| `core/agent.js:182` | `resolveAgentFolder` | Only used internally in `loadAgentFromFolder()` — no external imports |
+| `core/compaction.js:177` | `applyObservationMasking` | Only called internally by `manageContext()` — no external imports |
+| `core/compaction.js:178` | `applyToolResultClearing` | Only called internally by `manageContext()` — no external imports |
+| `core/compaction.js:179` | `estimateContextUsage` | Only called internally by `manageContext()` — no external imports |
+| `core/compaction.js:180` | `compactMessages` | Only called internally by `manageContext()` — no external imports |
+| `core/loop.js:493` | `checkPermission` | Only used internally in `runLoop()` — no external imports |
+| `core/registry.js:248` | `registerBuiltin` | Only called by `loadBuiltinTools()` in same file — no external imports |
+| `core/registry.js:255` | `toolToOpenAIFormat` | Only used internally by `buildToolsForLLM()` — no external imports |
+| `core/registry.js:256` | `getBuiltinNames` | Never imported anywhere — exposed but unused |
+| `core/queue.js:99` | `hasPendingMessages` | Never imported anywhere — exposed but unused |
+| `core/queue.js:99` | `drainNonFollowup` | Never imported anywhere — exposed but unused |
+| `core/queue.js:99` | `drainFollowup` | Never imported anywhere — exposed but unused |
+| `core/queue.js:99` | `postCorrelatedResponse` | Never imported anywhere — exposed but unused |
+| `core/memory.js:99` | `readMemory` | Never imported anywhere (tools use direct fs access) — exposed but unused |
+| `core/memory.js:99` | `searchMemory` | Never imported anywhere (memory_search.js uses its own implementation) — exposed but unused |
+| `utils/settings.js:134` | `deepMerge` | Only used internally in `loadSettings()` — no external imports |
+| `utils/settings.js:134` | `getDefaults` | Only used internally in `loadSettings()` — no external imports |
+| `utils/paths.js:214` | `getBackupsDir` | Never imported anywhere — exposed but unused |
+| `utils/paths.js:210` | `getGlobalSkillsDir` | Never imported anywhere — exposed but unused |
+| `infrastructure/database.js:464` | `runMigrations` | Only called internally by `getDb()` — no external imports |
+
+---
+
+## Dead Internal Functions
+
+Private helpers defined but never called within their own file.
+
+| File | Function | Reason |
+|------|----------|--------|
+| `core/memory.js:51` | `exportOldMemory` | Called by `writeMemory()` but `writeMemory()` itself is never called (tools use direct fs access) |
+
+> **Note:** Most internal helper functions ARE called within their files. The `core/memory.js` module appears to be legacy — the actual `memory_write.js` tool uses direct filesystem operations instead of calling the core memory module.
+
+---
+
+## Orphaned Database Columns
+
+Columns defined in migrations but never read or written by production code.
+
+| Table | Column | Reason |
+|-------|--------|--------|
+| `sessions` | `compaction_count` | No camelCase (`compactionCount`) usage in production code; only in `.OLD/` legacy code |
+| `sessions` | `estimated_cost` | No camelCase (`estimatedCost`) usage in production code; only in `.OLD/` legacy code |
+| `tasks` | `estimated_cost` | No camelCase (`estimatedCost`) usage in production code; only in `.OLD/` legacy code |
+
+> **Note:** The `token_cache` column IS used — written via `tokenCache` in `core/loop.js:307` and `core/router.js`.
+
+---
+
+## Undocumented/Untested Routes
+
+All API routes are documented and tested.
+
+| Route | Documented | Tested |
+|-------|------------|--------|
+| All routes in `api/routes/*.js` | ✅ `docs/api/*.md` | ✅ `test/integration/*.test.js` |
+
+**Verification:**
+- `/settings` — documented in `08-settings.md`, tested in `17-settings-api.test.js`
+- `/memory/*` — documented in `07-memory.md`, tested in `16-memory-api.test.js`
+- All other routes covered by existing docs and tests
+
+---
+
+## Orphaned Tools
+
+All tools in `tools/` are properly registered and reachable.
+
+| Tool | Status |
+|------|--------|
+| All 24 tools in `tools/*.js` | ✅ Loaded via `core/registry.js:loadBuiltinTools()` |
+
+**Verification:** 
+- `loadBuiltinTools()` in `core/registry.js:37-48` iterates over all `.js` files in the `tools/` directory
+- Each tool exports `{ schema, execute }` and is automatically registered
+- No orphaned tools found
+
+---
+
+## Unused Dependencies
+
+All dependencies in `package.json` are imported somewhere in production code.
+
+| Package | Usage |
+|---------|-------|
+| `ajv` | `core/agent.js`, `core/registry.js`, `api/routes/settings.js` |
+| `ajv-formats` | `core/registry.js`, `api/routes/settings.js` |
+| `better-sqlite3` | `infrastructure/database.js` |
+| `cors` | `api/server.js` |
+| `express` | `api/server.js`, `api/routes/*.js` |
+| `node-cron` | `infrastructure/scheduler.js` |
+| `ws` | `api/server.js` (WebSocket server) |
+
+---
+
+## Recommendations
+
+### High Priority (Remove Dead Code)
+
+1. **`core/queue.js`** — The entire module appears unused. The loop uses `drainMessageQueue()` defined inline in `core/loop.js:106-114`. Consider:
+   - Removing `core/queue.js` entirely, OR
+   - Refactoring `core/loop.js` to use `core/queue.js` functions
+
+2. **`core/memory.js`** — The module is bypassed by tools (`memory_read.js`, `memory_write.js`, `memory_search.js` all use direct fs access). Consider:
+   - Having tools use this module, OR
+   - Removing the module
+
+3. **Orphaned DB columns** (`compaction_count`, `estimated_cost`) — These columns are never written. Either:
+   - Implement cost tracking, OR
+   - Remove via a migration
+
+### Low Priority (API Surface Cleanup)
+
+4. **Unexported internal helpers** — Many functions are exported but only used internally. Consider reducing module.exports to only externally-used functions:
+   - `core/agent.js`: Remove `loadAgentFromFolder`, `resolveAgentFolder` from exports
+   - `core/compaction.js`: Only export `manageContext`
+   - `core/registry.js`: Remove `registerBuiltin`, `toolToOpenAIFormat`, `getBuiltinNames`
+   - `utils/settings.js`: Remove `deepMerge`, `getDefaults`
+   - `utils/paths.js`: Remove `getBackupsDir`, `getGlobalSkillsDir`
+
+---
+
+## Files Analyzed
+
+```
+core/agent.js, core/cancel.js, core/compaction.js, core/events.js, 
+core/loop.js, core/memory.js, core/prompt.js, core/queue.js, 
+core/registry.js, core/router.js
+
+utils/context.js, utils/id.js, utils/paths.js, utils/settings.js
+
+infrastructure/database.js, infrastructure/scheduler.js
+
+llm/client.js
+
+settings/fields.js
+
+api/middleware.js, api/server.js
+api/routes/agents.js, api/routes/chat.js, api/routes/daemons.js,
+api/routes/memory.js, api/routes/sessions.js, api/routes/settings.js,
+api/routes/system.js, api/routes/tasks.js
+
+tools/*.js (24 files)
+
+migrations/001-initial.sql, migrations/002-debuggability.sql
+```

package/TODO.md

@@ -0,0 +1,45 @@
+## TODO
+
+<!-- - SYSTEM PROMPT URGENT FIXES:
+  + Make it smaller
+  + Remove full date+time, just date (Breaks cache at every call) -->
+- API do not support generating/receiving multi modal (images, audio, video, ...)
+<!-- - Create a Models.json file with all the models and their data to be used as default, until we build a dedicated endpoint to get models data from main domain veil.com (Use OpenRouter API to get the default models data) -->
+<!-- - Expose the chat completion endpoint as a standalone function, so apps based on Veil can use it for independent AI usage -->
+- Compaction, it should be more robust, clear and configurable with Multi methods compaction, Methods:
+  + Context summarization (based on Claude's Automatic Context Compaction approach):
+    > **How it works:**
+      - Monitor token usage per turn via `context_token_threshold`
+      - When threshold exceeded, inject a summary prompt as a user turn
+      - Model generates a summary wrapped in `<summary></summary>` tags
+      - Clear conversation history and resume with only the summary
+      - Continue task with compressed context
+    > **Configuration options:**
+      - `enabled`: Boolean to enable/disable compaction
+      - `context_token_threshold`: Token count that triggers compaction (default: 100k, low: 5k-20k for sequential tasks, medium: 50k-100k for multi-phase, high: 100k-150k for context-heavy tasks)
+      - `model`: Optional cheaper/faster model for summarization (e.g., `claude-haiku-4-5`)
+      - `summary_prompt`: Custom prompt to guide what info to preserve in summaries
+    > **Config hierarchy (lowest wins):**
+      - Main level: `~/.veil/settings.json`
+      - Project level: `PROJECT/.veil/settings.json`
+      - Agent level: `AGENT/agent.json`
+      - NOTE: All used variables should be configurable (including summary prompt, model, ...)
+    > **When to use:**
+      - Sequential processing (multiple items one after another)
+      - Multi-phase workflows with natural checkpoints
+      - Batch operations with independent items
+      - Extended analysis sessions
+    > **When NOT to use:**
+      - Short tasks (<50k-100k tokens)
+      - Tasks requiring full audit trails
+      - Highly iterative refinement needing exact details from all steps
+    > **Limitations:**
+      - Information loss is inherent (mitigate via custom summary prompts)
+      - Higher thresholds preserve more detail but cost more per call
+    > Add Compaction endpoint to the sessions API to force run this method (usually for testing)
+  + Files Clearing:
+    > Clearing attached files after a specific negative index threshold (exp: Truncate all tool calls before last 10 messages)
+  + Long Tool Call results truncation:
+    > Truncating long toolcall results after a specific negative index threshold (exp: Truncate all tool calls before last 10 messages)
+    > Can be configured with a custom threshold per tool type (per tool key)
+  + NOTE: Clear code to be able to add new methods easily