neoagent 2.4.4-beta.10 → 2.4.4-beta.12

package/docs/models.md

@@ -0,0 +1,64 @@
+# Models and providers
+
+NeoAgent supports local models, API-key providers, and several account-backed
+providers. Provider credentials stay on the server.
+
+## Configure a model
+
+Open **Settings > AI Providers** to add credentials and choose models. You can
+also run `neoagent setup` to configure common values from the terminal.
+
+### Local models
+
+[Ollama](https://ollama.com/) does not require a hosted-model API key. Set its
+server URL with `OLLAMA_URL`, then select an available Ollama model in NeoAgent.
+The Ollama process may run on the NeoAgent host or another reachable machine.
+
+### API-key providers
+
+NeoAgent includes providers for Anthropic, OpenAI, Google Gemini, xAI, MiniMax,
+NVIDIA NIM, OpenRouter, and OpenAI-compatible endpoints. Available models are
+loaded through the configured provider rather than maintained as a fixed list
+in the documentation.
+
+### Account-backed providers
+
+The CLI can authenticate supported developer subscriptions:
+
+```bash
+neoagent login github-copilot
+neoagent login openai-codex
+neoagent login claude-code
+neoagent login grok-oauth
+```
+
+These login flows are separate from ordinary API-key providers.
+
+## Model assignment
+
+The default model is selected in settings. Individual agents and scheduled
+tasks can override it. A model used for tools must support the tool-calling
+contract expected by its NeoAgent provider.
+
+Some features use separate providers:
+
+- Embeddings for memory search use a configured supported embedding provider
+  and fall back to lexical retrieval when embeddings are unavailable.
+- Recording transcription uses Deepgram when enabled.
+- Image generation and analysis depend on the selected provider and model.
+
+## Credential handling
+
+API keys and account tokens are stored under the NeoAgent runtime directory on
+the server. Do not put credentials in task prompts, skills, screenshots, issue
+reports, or chat messages.
+
+Use the environment CLI when terminal administration is more appropriate:
+
+```bash
+neoagent env list
+```
+
+`env list` masks secrets. The `env get`, `env set`, and `env unset` commands
+accept a variable name; `env get` prints the selected value, so avoid running it
+in recorded terminals or shared shells.

package/docs/operations.md

@@ -1,28 +1,33 @@
-# Operations
+# Operations and troubleshooting
 
-## Service State
+Run service commands on the machine hosting NeoAgent.
+
+## Service commands
 
 ```bash
-neoagent status      # install root, config path, runtime paths, release channel, service state
-neoagent logs        # first stop for unexpected behavior in UI, messaging, OAuth, or tasks
+neoagent status
+neoagent start
+neoagent stop
 neoagent restart
+neoagent logs
 ```
 
-## Release Channels
+`status` is the first check for install, service, configuration, and health
+problems. `logs` tails the server logs. Logs from a laptop or client do not
+describe a NeoAgent instance running on another server.
 
-```bash
-neoagent channel           # show current channel
-neoagent channel stable    # switch to stable releases (recommended for most installs)
-neoagent channel beta      # switch to prerelease builds
-```
-
-## Updates
+## Release channels and updates
 
 ```bash
+neoagent channel
+neoagent channel stable
+neoagent channel beta
 neoagent update
 ```
 
-Follows the configured release channel. On git installs, pulls the latest source and reinstalls dependencies when they change. On npm installs, reinstalls the global package from the matching npm tag. Verifies the bundled web client and restarts the service.
+Stable is intended for normal installations. Beta receives prerelease builds.
+`update` follows the selected channel, updates the package or Git checkout,
+verifies the bundled client, and restarts the service.
 
 ## Recovery
 
@@ -30,32 +35,43 @@ Follows the configured release channel. On git installs, pulls the latest source
 neoagent fix
 ```
 
-Use when NeoAgent is in a broken state after a self-edit or corrupted install. On git installs: backs up runtime data, saves local tracked changes, resets source files, reinstalls dependencies, and restarts.
+Use `fix` for a damaged installation or failed source update. It preserves
+runtime data, repairs application source and dependencies, and restarts the
+service. It is not a substitute for a backup.
 
-For configuration-only issues:
+For configuration problems:
 
 ```bash
 neoagent setup
 neoagent restart
 ```
 
-## Troubleshooting
+## Backups
 
-| Symptom | First step |
-|---|---|
-| Service won't start | `neoagent logs` — look for startup errors |
-| UI loads but agent doesn't respond | Confirm an AI provider key is set in **Settings → AI Providers** |
-| OAuth integration fails | Confirm `PUBLIC_URL` is reachable and the redirect URI matches |
-| Messaging not delivering | Check credentials in the messaging tab; confirm `PUBLIC_URL` for webhook-based platforms |
-| Tasks not running | Confirm the task is enabled; check **Runs** and **Logs** for error output |
-| Broken after update | `neoagent fix` — resets source, reinstalls, restarts |
-
-## Runtime Data
+Stop the service or take an application-consistent filesystem snapshot, then
+back up:
 
 | Path | Contents |
 |---|---|
-| `~/.neoagent/.env` | Server config and secrets |
-| `~/.neoagent/data/` | Database, session data, logs, update status |
-| `~/.neoagent/agent-data/` | Skills, memory, daily data |
+| `~/.neoagent/.env` | Server configuration and secrets |
+| `~/.neoagent/data/` | SQLite database, sessions, logs, update state |
+| `~/.neoagent/agent-data/` | Skills, memory files, daily data |
+
+If `NEOAGENT_HOME` is set, these paths live under that directory instead.
+Protect backups as credentials and personal data.
 
-Back up these paths before moving a server or doing manual repairs.
+## Common failures
+
+| Symptom | Check |
+|---|---|
+| Service does not start | `neoagent status`, then `neoagent logs` |
+| Chat has no response | Provider connection, selected model, provider quota |
+| Browser or shell unavailable | QEMU installation and first runtime boot |
+| OAuth fails | Reachable HTTPS `PUBLIC_URL` and exact callback URI |
+| Messaging receives nothing | Channel credentials, webhook, and allowlists |
+| Task does not run | Enabled state, trigger account, next run, **Runs** |
+| Android action fails | ADB visibility, selected device, tool approval |
+| Paired device unavailable | Companion or extension connection on that machine |
+
+When reporting a bug, include the NeoAgent version, server OS, installation
+type, runtime profile, exact reproduction steps, and sanitized server logs.

package/docs/persistence.md

@@ -0,0 +1,49 @@
+# Persistence and migrations
+
+NeoAgent uses a single SQLite database opened through
+`server/db/database.js`. Services share that instance and use synchronous
+`better-sqlite3` statements and transactions.
+
+## Stored state
+
+The database owns:
+
+- Users, sessions, agents, and settings
+- Conversations, messages, runs, steps, events, and model usage
+- Tasks, triggers, webhook deliveries, and widgets
+- Integration connections and encrypted credentials
+- Memory, facts, entities, source documents, and retrieval telemetry
+- Recordings, transcripts, health records, and device registrations
+- Tool policies and approval history
+
+Larger artifacts and editable agent data use runtime directories under
+`NEOAGENT_HOME`.
+
+## Schema changes
+
+Base schema creation lives in the database module. Incremental migrations live
+in `lib/migrations.js` and `lib/schema_migrations.js`. A migration must be
+idempotent and preserve existing user data.
+
+Do not create tables or run `ALTER TABLE` from feature services. Do not open a
+second SQLite database instance.
+
+FTS5 and JSON1 use graceful fallbacks where the host SQLite build may not
+provide an extension. New behavior must preserve those fallback paths unless
+the installation requirements are deliberately changed.
+
+## Runtime paths
+
+`runtime/paths.js` is the source of truth for application, configuration, data,
+agent-data, log, update, and PID paths. Services must use those exports rather
+than hard-coded home-directory paths.
+
+Startup can migrate legacy in-repository `.env`, `data`, and `agent-data`
+locations into the runtime home when the destination is empty.
+
+## Backup implications
+
+The database and agent-data directories should be backed up as one logical
+state. Integration credentials and session material make backups sensitive.
+Restoring only the database can leave referenced files or editable skills out
+of sync.

package/docs/recordings-and-health.md

@@ -0,0 +1,41 @@
+# Recordings and health
+
+Recordings and health data are optional features of the NeoAgent clients. Both
+store their processed data on the NeoAgent backend.
+
+## Recordings
+
+The web client can capture microphone and screen audio. The Android client can
+record microphone audio through a foreground service.
+
+Recordings upload in chunks and move through recording, processing, completed,
+failed, or cancelled states. When Deepgram is configured, NeoAgent transcribes
+the audio and stores timestamped segments that can be searched from the
+operator interface or by the agent.
+
+Optional recording insights can produce summaries, action items, and event
+suggestions after transcription.
+
+The agent can list recordings, inspect a session, and search transcripts. It
+can also extract transcripts and metadata from supported public social-video
+URLs when the required host tool is installed.
+
+## Health Connect
+
+The Android app can read Health Connect records after the user grants Android
+permissions. Supported data includes steps, heart rate, sleep, exercise, and
+weight.
+
+The app uploads granted records to the user's NeoAgent backend. Stored metrics
+appear in **Health** and are available to the owning agent through the health
+tool.
+
+Health Connect access can be revoked in Android settings. Revoking access stops
+future reads but does not remove records already uploaded to NeoAgent; delete
+stored server data separately when required.
+
+## Privacy
+
+Audio, transcripts, and health records are sensitive personal data. Protect the
+server account, use HTTPS for remote clients, restrict agents and integrations,
+and include these data stores in backup and deletion procedures.

package/docs/runtime-and-tools.md

@@ -0,0 +1,52 @@
+# Runtime and tool execution
+
+The runtime manager selects where browser, shell, desktop, and Android actions
+execute. Tool security is checked before dispatch, independent of the model
+that requested the action.
+
+## Browser and shell
+
+The default browser and CLI backend is a per-user isolated runtime managed by
+the server. It exposes a guest agent authenticated with a generated token and
+stores artifacts through the server artifact service.
+
+Browser settings can select a paired Chrome extension. CLI settings can select
+a paired desktop companion. If the requested paired backend is unavailable,
+the effective backend falls back to the isolated runtime.
+
+## Desktop companion
+
+Desktop control and selected CLI commands are sent through the authenticated
+desktop companion registry. Non-PTY desktop commands can run through an
+isolated worker process that has no imports from the main server process.
+
+This process separation protects server memory; it does not restrict what the
+paired desktop account itself can access.
+
+## Android
+
+The Android provider is created per user but executes through ADB on the
+NeoAgent host. Device selection, screenshots, UI observation, input, intents,
+application installation, and shell commands therefore operate outside the
+browser and CLI runtime.
+
+## Workspace files
+
+File tools use server-side path validation and the user's workspace boundary.
+They do not accept arbitrary paths merely because a model generated them.
+
+## Tool security hook
+
+`before_tool_call` runs before tool execution. The security hook maps sensitive
+tools to categories and applies the user's deny, approval, session allow, or
+persistent allow policy.
+
+Approval waits are delivered through Socket.IO. The model receives a structured
+blocked result for denial or timeout and should not treat the tool as executed.
+
+## Boundaries not provided
+
+The runtime does not provide destination-level network egress filtering.
+Read-only tools can execute without approval. Paired machines and ADB devices
+remain trusted operator resources. See [Security](security-boundaries.md) for
+deployment guidance.

package/docs/security-boundaries.md

@@ -1,122 +1,76 @@
-# Security Boundaries
+# Security and permissions
 
-NeoAgent processes untrusted content from web pages, email, messaging
-platforms, integrations, MCP servers, files, and other external systems. That
-content may attempt prompt injection.
+NeoAgent reads untrusted content from websites, email, messages, files,
+integrations, and MCP servers. Any of that content can contain prompt
+injection. Model instructions and injection warnings help, but authorization
+must come from server-enforced boundaries and operator choices.
 
-Prompt instructions, warning text added to tool results, and injection
-detection heuristics are defense-in-depth signals for the model. They are not
-authorization boundaries. The boundaries below are enforced in server-side
-code and cannot be bypassed by any model-generated content.
+## Tool permissions
 
-## What Is Enforced
+Sensitive tools are grouped into categories for shell commands, file writes,
+privileged Android actions, desktop control, browser evaluation, network
+writes, and skill or widget mutation.
 
-### Infrastructure isolation
+Each category can be:
 
-| Capability | Enforced boundary |
+| Policy | Result |
 |---|---|
-| Browser, default backend | Runs in a per-user Docker container with resource limits (`no-new-privileges`, no host filesystem mount). |
-| Browser, paired extension | Commands are sent to the explicitly paired Chrome extension. Controls that browser profile; outside the container boundary. |
-| Shell, VM backend | `execute_command` runs in the same per-user container as the default browser backend. |
-| Shell, desktop companion | Commands route through `ShellWorkerPool` — a `child_process.fork()` pool of isolated worker processes (`server/services/cli/shell_worker.js`). Workers carry **no server imports**: no database handle, no JWT secret, no app state. A compromised command result cannot read main-process memory. |
-| Workspace file tools | `read_file`, `write_file`, `edit_file`, directory listing, and file search are restricted by server-side path checks to the user's workspace directory. |
-| Android | `android_shell` and related tools run from the NeoAgent host via ADB against the selected device or emulator. |
-| Official integrations | OAuth credentials remain server-side. Accounts configured as read-only have write tools blocked by server-side access checks. |
-| Runtime API | The container endpoint is published on loopback. A guest token authenticates requests between the server and the runtime. |
+| Deny | The tool does not run |
+| Require approval | The run pauses for a user decision |
+| Allow | The tool runs for the current session |
+| Always allow | The stored policy permits future runs |
 
-### Tool policy system
+The default is approval for sensitive categories. Skill and widget mutation is
+denied by default. Users can also select a global default, always-ask, or
+allow-all mode.
 
-The `before_tool_call` hook fires before every tool execution in both the
-sequential and parallel engine paths. Two handlers are registered at startup
-and cannot be removed by agent code:
+Approval prompts time out after 30 seconds. A denied or timed-out call is
+reported to the model as blocked rather than executed.
 
-**Priority 5 — per-category policy check** (`server/services/security/tool_security_hook.js`)
+## Where tools run
 
-Every tool is mapped to one of seven sensitive categories:
-
-| Category | Tools |
-|---|---|
-| `shell` | `execute_command` |
-| `file_write` | `write_file`, `edit_file` |
-| `android_privileged` | `android_shell`, `android_install_apk`, `android_open_intent`, `android_open_app` |
-| `desktop_control` | `desktop_click`, `desktop_type`, `desktop_press_key`, `desktop_drag`, `desktop_launch_app`, `desktop_observe` |
-| `browser_privileged` | `browser_evaluate` |
-| `network_write` | `http_request` (POST/PUT/PATCH/DELETE only) |
-| `skill_mutation` | `create_skill`, `update_skill`, `delete_skill`, `create_ai_widget`, `update_ai_widget`, `delete_ai_widget` |
-
-Read-only tools (`think`, `web_search`, `browser_navigate`, `read_file`, etc.)
-are in a static `SAFE_TOOLS` set and bypass all policy checks with a single
-`Set.has()` call.
-
-Per-category policy (stored in `tool_policies` table, one row per user per
-category):
-
-| Policy | Effect |
-|---|---|
-| `deny` | Tool call is blocked immediately. The model receives a structured error and can explain the restriction to the user. |
-| `require_approval` | Execution suspends; the user receives a real-time prompt (bottom sheet + optional push notification). Decision options: Deny / Allow once / Allow session / Always allow. |
-| `allow` | Tool runs without interruption. Grants expire at session end. |
-| `allow_always` | Tool runs without interruption. Persisted to DB; never expires. |
-
-Default policy for all categories is `require_approval`, except `skill_mutation` which defaults to `deny`.
-
-**Priority 10 — approval gate** (`server/services/security/approval_gate_service.js`)
-
-When a category policy is `require_approval`, execution is suspended and a
-`tool:approval_required` event is emitted to the user's Socket.IO room. The
-agent loop awaits the decision with a 30-second timeout. If no decision is
-received, the tool is denied and the model is informed of the timeout.
-
-### Global security mode
-
-Users can override all per-category settings with a global mode (stored in
-`user_settings` under key `tool_security_mode`):
-
-| Mode | Behaviour |
+| Capability | Runtime |
 |---|---|
-| `default` | Per-category policies apply (recommended). |
-| `allow_all` | All tool policy and approval checks are bypassed. The agent runs without interruption. |
-| `always_ask` | Every sensitive tool requires approval regardless of per-category settings. |
-
-### Agent-loop feedback
-
-When a tool is blocked, the model receives a structured result:
-
-```json
-{ "tool": "execute_command", "status": "blocked", "blocked_by": "policy|user_denied|approval_timeout", "reason": "..." }
-```
-
-The `reason` field contains an actionable message the model can surface to the
-user (e.g., pointing to Settings → Tool Permissions, or acknowledging that the
-user denied the call and it should not retry).
-
-## What Is Not Enforced
-
-- **No per-tool allowlist at the schema/selection level.** A tool blocked by
-  policy is still listed in the tool catalog. The block fires at execution
-  time, not at selection time.
-- **No mandatory approval for safe tools.** Tools in `SAFE_TOOLS` always
-  execute without a policy check.
-- **No network egress filtering.** The `http_request` tool (non-write methods)
-  can reach any URL reachable from the server.
-- **Prompt-injection detection is heuristic.** The detection layer may warn the
-  model but cannot prevent every injection attempt.
-
-## Deployment Guidance
-
-- Run NeoAgent under a dedicated, unprivileged OS account.
-- Prefer the container-backed browser and CLI defaults (VM backend).
-- Do not select the desktop shell backend on a machine containing data the
-  agent should not access.
-- Use `require_approval` or `deny` for `shell` and `android_privileged` on
-  machines where you run personal or work data alongside NeoAgent.
-- Treat a paired browser extension as giving the agent control of that browser
-  profile. Use a dedicated profile.
-- Treat an ADB-connected Android device as accessible to the agent, including
-  capabilities exposed by `adb shell`.
-- Keep integration accounts read-only unless write access is required.
-- The `allow_all` global mode is suitable only for trusted automated
-  environments where the operator controls all input. Do not use it for
-  open-ended agentic sessions that browse the web or read email.
-
-For vulnerability reporting, see [Security Policy](https://github.com/NeoLabs-Systems/NeoAgent/blob/main/SECURITY.md).
+| Default browser and shell | Per-user isolated runtime managed by the NeoAgent host |
+| Workspace files | Server-enforced user workspace paths |
+| Paired Chrome extension | The paired Chrome profile and machine |
+| Paired desktop companion | The paired desktop account |
+| Android | The selected host-attached ADB device or emulator |
+| Integrations | NeoAgent server using stored account credentials |
+
+The paired extension and desktop companion are access grants, not isolation
+boundaries. Android commands do not run in the browser and shell VM.
+
+## Account and integration controls
+
+- Credentials remain on the server.
+- Official integration accounts can be set to read-only.
+- Users and agents have separate application data and assignments.
+- Messaging allowlists restrict which chats and senders can trigger runs.
+- A guest token authenticates communication with the isolated runtime.
+
+## Important limitations
+
+- Read-only tools do not require approval by default.
+- Outbound network access is not filtered by destination.
+- A paired browser can access the signed-in state of that Chrome profile.
+- A paired desktop process acts with the permissions of its OS account.
+- ADB can expose broad access to the selected Android device.
+- Prompt-injection detection cannot identify every malicious instruction.
+- Multi-user application isolation does not make NeoAgent suitable for
+  mutually hostile tenants on a shared host.
+
+## Deployment guidance
+
+- Run NeoAgent as a dedicated unprivileged OS account.
+- Use HTTPS for remote access.
+- Keep shell, desktop, Android, and write categories on approval until the
+  workflow is understood.
+- Use dedicated Chrome and desktop profiles for paired control.
+- Keep integrations read-only unless writes are required.
+- Do not use allow-all for open-ended runs that browse the web or read messages.
+- Back up secrets and user data securely and test account recovery.
+
+The implementation details are documented in
+[Runtime and tool execution](runtime-and-tools.md). Report vulnerabilities
+through [SECURITY.md](https://github.com/NeoLabs-Systems/NeoAgent/blob/main/SECURITY.md).

package/docs/skills.md

@@ -1,59 +1,46 @@
-# Skills
+# Skills and MCP
 
-Skills are Markdown files that instruct the agent on how to use a local capability or follow a multi-step workflow. They load at runtime from `~/.neoagent/agent-data/skills/`, so edits take effect without restarting the service.
+Skills and MCP servers both extend the agent, but they solve different
+problems.
 
-## Built-in Skills
+## Skills
 
-| Skill | Description |
-|---|---|
-| `browser.md` | Web browsing and scraping with Puppeteer |
-| `cli.md` | Shell commands in a persistent PTY terminal |
-| `files.md` | Read, write, and search host files |
-| `memory.md` | Store and recall long-term facts |
-| `messaging.md` | Send via Telegram, WhatsApp, Discord, Slack, Matrix, Teams, Google Chat, or webhooks |
-| `system-stats.md` | CPU, memory, and disk usage |
-| `weather.md` | Current weather via wttr.in |
-| `ip-info.md` | Public IP and geolocation |
-| `port-check.md` | Check if a TCP port is open |
-| `ping-host.md` | Ping a host |
-| `process-monitor.md` | List running processes |
-| `disk-usage.md` | Directory size breakdown |
-| `find-large-files.md` | Locate large files |
-| `docker-status.md` | Docker container status |
-| `tail-log.md` | Tail any log file |
-| `news-hackernews.md` | Fetch top Hacker News stories |
-| `qr-code.md` | Generate QR codes |
-| `pdf-toolkit.md` | Inspect, extract, merge, split, compress PDFs |
-| `git-summary.md` | Summarize git status, branches, commits, and diffs |
-| `csv-toolkit.md` | Inspect and transform CSV or TSV files |
-| `markdown-workbench.md` | Clean up, outline, and convert Markdown documents |
+A skill is a Markdown instruction set for a repeatable workflow. Skills can
+teach an agent how to use existing tools, commands, file formats, or local
+conventions. They do not add a new security boundary or bypass tool approvals.
 
-## Creating a Custom Skill
+Manage skills in **Skills**. NeoAgent includes a catalog of system, network,
+document, data, creative, and development workflows. Installed skills are
+editable and removable.
 
-Add a Markdown file to `~/.neoagent/agent-data/skills/`:
+Custom skills live in the NeoAgent agent-data directory and are loaded without
+restarting the server. Keep them focused, name the actual tools they require,
+and never store secrets in them.
 
-```markdown
-# My Skill Name
+Use a skill when:
 
-Brief description of what this skill does and when to use it.
+- the workflow repeats;
+- the agent already has the required underlying tools; and
+- written procedure is enough to make the work reliable.
 
-## Usage
+## MCP servers
 
-Step-by-step instructions or exact commands the agent should follow.
-```
+The [Model Context Protocol](https://modelcontextprotocol.io/) connects NeoAgent
+to external tool servers. Configure servers in **MCP**, choose the supported
+authentication method, and inspect the connection state before assigning the
+tools to an agent.
 
-The agent reads all `.md` files in the skills directory on each turn. Keep skills focused. Include exact tool names or shell commands when precision matters. Don't store secrets in skill files.
+Use MCP when an external system exposes a maintained MCP server or when a
+capability requires a real API implementation rather than instructions.
 
-**When to use a custom skill** — repeated workflows involving specific local paths, commands, or multi-step procedures. For connecting to third-party services, prefer official integrations or MCP tools instead.
+## Choosing the right extension
 
-## Skill Store
+Prefer, in order:
 
-The **Skills** section in the UI includes a built-in catalog from `/api/store`. Skills install as Markdown files into the runtime skills directory — editable and removable after installation.
+1. An official NeoAgent integration for supported account services
+2. A maintained MCP server
+3. A skill using existing tools
+4. Browser or shell automation when no structured interface exists
 
-Catalog categories: system, network, info, document, data, and git helpers.
-
-## MCP Tools
-
-External tools connect via the [Model Context Protocol](https://modelcontextprotocol.io). Configure MCP servers in **Settings → MCP**. Connected tools appear alongside built-in skills automatically.
-
-Use official integrations or MCP tools when they exist — they are more reliable than browser automation or shell scraping and easier to audit.
+Structured integrations are easier to restrict, audit, and keep stable than UI
+automation.