neoagent 2.3.1-beta.91 → 2.3.1-beta.94

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (43) hide show
  1. package/README.md +2 -2
  2. package/com.neoagent.plist +1 -1
  3. package/docs/automation.md +82 -41
  4. package/docs/capabilities.md +85 -91
  5. package/docs/configuration.md +86 -103
  6. package/docs/getting-started.md +37 -35
  7. package/docs/index.md +24 -30
  8. package/docs/integrations.md +46 -63
  9. package/docs/migration.md +41 -146
  10. package/docs/operations.md +23 -20
  11. package/docs/skills.md +20 -20
  12. package/docs/why-neoagent.md +15 -13
  13. package/extensions/chrome-browser/background.mjs +10 -2
  14. package/flutter_app/DESIGN_SYSTEM.md +166 -0
  15. package/flutter_app/lib/features/notifications/notification_interceptor.dart +12 -2
  16. package/flutter_app/lib/main_app_shell.dart +27 -11
  17. package/flutter_app/lib/main_chat.dart +27 -18
  18. package/flutter_app/lib/main_controller.dart +32 -3
  19. package/flutter_app/lib/main_shared.dart +11 -11
  20. package/flutter_app/lib/main_spacing.dart +15 -0
  21. package/flutter_app/lib/main_theme.dart +10 -0
  22. package/flutter_app/lib/src/backend_client.dart +2 -10
  23. package/lib/manager.js +260 -134
  24. package/package.json +1 -1
  25. package/server/db/database.js +1 -1
  26. package/server/public/.last_build_id +1 -1
  27. package/server/public/flutter_bootstrap.js +1 -1
  28. package/server/public/main.dart.js +74719 -73512
  29. package/server/routes/agents.js +4 -1
  30. package/server/routes/auth.js +1 -4
  31. package/server/routes/memory.js +3 -3
  32. package/server/routes/settings.js +11 -2
  33. package/server/routes/skills.js +12 -0
  34. package/server/routes/tasks.js +22 -2
  35. package/server/routes/triggers.js +41 -32
  36. package/server/services/ai/engine.js +15 -5
  37. package/server/services/ai/taskAnalysis.js +1 -0
  38. package/server/services/ai/toolRunner.js +12 -0
  39. package/server/services/android/controller.js +3 -3
  40. package/server/services/memory/manager.js +18 -7
  41. package/server/services/messaging/manager.js +15 -1
  42. package/server/services/tasks/integration_runtime.js +1 -1
  43. package/server/services/tasks/task_repository.js +3 -6
package/README.md CHANGED
@@ -34,7 +34,7 @@ npm install -g neoagent
34
34
  neoagent install
35
35
  ```
36
36
 
37
- Runs as a `launchd` user service on macOS and `systemd --user` on Linux.
37
+ Opens at **http://localhost:3333** when complete. Runs as a `launchd` user service on macOS and `systemd --user` on Linux.
38
38
 
39
39
  ## Manage
40
40
 
@@ -47,7 +47,7 @@ neoagent logs
47
47
 
48
48
  ## Links
49
49
 
50
- [Docs](https://neolabs-systems.github.io/NeoAgent/docs/) | [Capabilities](docs/capabilities.md) | [Why NeoAgent](docs/why-neoagent.md) | [Issues](https://github.com/NeoLabs-Systems/NeoAgent/issues)
50
+ [Docs](https://neolabs-systems.github.io/NeoAgent/docs/) | [Getting Started](docs/getting-started.md) | [Configuration](docs/configuration.md) | [Capabilities](docs/capabilities.md) | [Why NeoAgent](docs/why-neoagent.md) | [Issues](https://github.com/NeoLabs-Systems/NeoAgent/issues)
51
51
 
52
52
  ---
53
53
 
@@ -17,7 +17,7 @@
17
17
  <key>EnvironmentVariables</key>
18
18
  <dict>
19
19
  <key>PATH</key>
20
- <string>/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
20
+ <string>/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
21
21
  <key>HOME</key>
22
22
  <string>__HOME__</string>
23
23
  <key>NODE_ENV</key>
@@ -1,53 +1,94 @@
1
1
  # Automation
2
2
 
3
- NeoAgent is built for proactive work: tasks that run later, repeat on a schedule, use tools, and notify you when there is something useful to report.
3
+ Tasks run on a schedule or integration trigger, use the same tools as chat, and can deliver results through any connected messaging platform.
4
4
 
5
- ## Tasks
5
+ ## Creating a Task
6
6
 
7
- Use the **Tasks** section in the UI to create automations. Tasks can be time-based (Cron) or integration-triggered (for example email, messaging, and weather-event triggers). A task has:
7
+ Open **Tasks** in the UI and fill in:
8
8
 
9
- | Field | Purpose |
9
+ | Field | Description |
10
10
  |---|---|
11
- | Name | Human-readable task label |
12
- | Cron expression | Five-field cron schedule |
13
- | Prompt | Self-contained instruction for the future run |
14
- | Enabled state | Active or paused |
15
- | Model override | Optional model for this task |
11
+ | **Name** | Human-readable label |
12
+ | **Cron** | Five-field schedule expression |
13
+ | **Prompt** | Self-contained instruction for the future run |
14
+ | **Enabled** | Active or paused |
15
+ | **Model** | Optional per-task model override |
16
16
 
17
- Scheduled prompts should include the condition for notifying you. For example, ask NeoAgent to message you only when a monitored thing changes, fails, or needs attention.
17
+ ## Cron Expressions
18
18
 
19
- ## Tool Capabilities
19
+ ```
20
+ ┌───── minute (0–59)
21
+ │ ┌──── hour (0–23)
22
+ │ │ ┌─── day of month (1–31)
23
+ │ │ │ ┌── month (1–12)
24
+ │ │ │ │ ┌─ day of week (0–7, both 0 and 7 = Sunday)
25
+ │ │ │ │ │
26
+ * * * * *
27
+ ```
20
28
 
21
- Automation can use the same tool surface as normal chat runs:
29
+ Common patterns:
30
+
31
+ | Expression | Runs |
32
+ |---|---|
33
+ | `0 9 * * *` | Daily at 9:00 AM |
34
+ | `0 9 * * 1-5` | Weekdays at 9:00 AM |
35
+ | `0 8 * * 1` | Every Monday at 8:00 AM |
36
+ | `0 18 * * 5` | Every Friday at 6:00 PM |
37
+ | `0 9 1 * *` | First of every month at 9:00 AM |
38
+ | `0 */4 * * *` | Every 4 hours |
39
+ | `*/30 * * * *` | Every 30 minutes |
40
+
41
+ ## Writing Good Task Prompts
42
+
43
+ Prompts run unattended. Be specific about what to check and when to notify — tasks that always send a message become noise, tasks that only notify on a condition are useful.
44
+
45
+ **Daily news digest**
46
+ ```
47
+ Search Hacker News for the top 5 stories today. Send me a brief summary of each via Telegram, including the title and link.
48
+ ```
49
+
50
+ **Price monitor**
51
+ ```
52
+ Check the price of Bitcoin and Ethereum on CoinGecko. If either has changed more than 5% in the last 24 hours, send me a Telegram message with the current prices and percentage change. If there are no significant changes, do nothing.
53
+ ```
54
+
55
+ **Server health check**
56
+ ```
57
+ Run `df -h` and `free -m`. If disk usage on any partition is above 85% or available memory is below 500MB, send me a Telegram alert with the details. Otherwise do nothing.
58
+ ```
59
+
60
+ **Weekly email digest**
61
+ ```
62
+ Search my Gmail for unread emails from the last 7 days. Group them by sender domain and summarize the main topics. Send the summary to my Telegram.
63
+ ```
64
+
65
+ ## Tool Access
66
+
67
+ Automation can use everything available in chat:
22
68
 
23
69
  | Capability | Examples |
24
70
  |---|---|
25
- | Browser | Navigate, click, type, extract page content, take screenshots, evaluate page JavaScript |
26
- | Files | Read, write, search, and summarize host files through skills |
27
- | CLI | Run shell commands in a persistent terminal through skills |
28
- | Memory | Store durable facts and retrieve useful context |
29
- | Messaging | Send a proactive result through a connected platform |
30
- | MCP | Use tools exposed by configured remote MCP servers |
31
- | Official integrations | Use structured OAuth-backed app tools where available |
32
- | Recordings | List, open, and search recording transcripts |
33
- | Health | Read synced Android Health Connect metrics as summaries |
34
- | Android | Drive a server-attached emulator or device through UI and ADB tools |
35
- | Subagents | Spawn async helper agents inside a longer run |
36
- | Outputs | Generate artifacts, Grok images, Mermaid graphs, and markdown tables |
37
-
38
- Prefer official integrations and structured MCP tools over browser automation when both can answer the task. They are usually less brittle and easier to audit.
39
-
40
- See [Capabilities](capabilities.md) for the broader tool inventory.
41
-
42
- ## Safety Expectations
43
-
44
- NeoAgent runs on your server and can touch real files, messaging surfaces, browser sessions, and connected services. Keep scheduled prompts narrow and self-contained.
45
-
46
- For sensitive automations:
47
-
48
- - Use allowlists for messaging platforms.
49
- - Keep secrets in server config, not prompts or skills.
50
- - Prefer read-only checks unless the task explicitly needs to mutate data.
51
- - Ask for notification only when a condition is met to avoid noisy repeated messages.
52
- - Review run history in **Runs** and service logs in **Logs** when behavior is surprising.
53
- - Remember that browser, CLI, Android runtime, and local file tools run on the NeoAgent server or configured worker, not necessarily on your current laptop.
71
+ | **Browser** | Navigate, extract content, screenshot, evaluate JavaScript |
72
+ | **CLI** | Shell commands in a persistent terminal |
73
+ | **Files** | Read, write, search host files |
74
+ | **Memory** | Store and retrieve long-term facts |
75
+ | **Messaging** | Send results through any connected platform |
76
+ | **MCP** | Tools from configured remote MCP servers |
77
+ | **Integrations** | Gmail, Calendar, Notion, Slack, and other OAuth tools |
78
+ | **Recordings** | Search and read transcripts |
79
+ | **Health** | Read synced Android Health Connect metrics |
80
+ | **Android** | Control an emulator or device |
81
+ | **Subagents** | Spawn parallel helpers inside a longer run |
82
+ | **Outputs** | Artifacts, images, Mermaid graphs, markdown tables |
83
+
84
+ Prefer official integrations and MCP tools over browser automation when both can do the job they are more reliable and easier to audit.
85
+
86
+ ## Safety
87
+
88
+ NeoAgent runs on your server and can touch real files, messaging surfaces, connected accounts, and browser sessions. Keep prompts narrow.
89
+
90
+ - Use messaging allowlists to restrict which chats receive automated messages
91
+ - Keep secrets in server config, not in prompts or skill files
92
+ - Prefer read-only checks unless the task explicitly needs to write data
93
+ - Review run history in **Runs** and service logs in **Logs** when behavior is unexpected
94
+ - Browser, CLI, Android, and file tools run on the NeoAgent server — not necessarily your current laptop
@@ -1,124 +1,118 @@
1
1
  # Capabilities
2
2
 
3
- This page lists the product surfaces that are easy to miss from the short README. It is based on the current server routes, agent tool registry, Flutter sections, Android bridge code, and integration providers in this repository.
3
+ NeoAgent can operate real software on the machine it runs on a browser, Android emulators and devices, and a terminal. It connects to your accounts, remembers context across sessions, and runs scheduled automations without you being present.
4
4
 
5
5
  ## Operator UI
6
6
 
7
- The Flutter client exposes the main operator surfaces:
8
-
9
- | Section | What it is for |
7
+ | Section | What it's for |
10
8
  |---|---|
11
- | Chat | Normal agent runs with tools, memory, integrations, and messaging |
12
- | Runs | Live and historical run steps, including browser, Android, CLI, messaging, tasks, MCP, and subagent work |
13
- | Logs | Service logs and diagnostics from the server you are connected to |
14
- | Tasks | Schedule-triggered and integration-triggered automations |
15
- | Skills | Built-in and custom reusable workflows |
16
- | Integrations | OAuth account connections for structured app tools |
17
- | MCP | Remote MCP server registration and tool discovery |
18
- | Memory | Long-term memory, core memory, and session search |
19
- | Devices | Server-side browser and Android runtime controls |
20
- | Recordings | Recording sessions, transcripts, segments, and playback |
21
- | Health | Android Health Connect sync status and synced metrics |
22
- | Settings | AI providers, model routing, runtime settings, messaging, and service controls |
23
-
24
- ## Recordings
9
+ | **Chat** | Interactive agent runs with full tool access, memory, integrations, and messaging |
10
+ | **Runs** | Live and historical step-by-step execution browser, Android, CLI, messaging, tasks, MCP, subagents |
11
+ | **Tasks** | Schedule-triggered and integration-triggered automations |
12
+ | **Skills** | Built-in and custom reusable workflows |
13
+ | **Integrations** | OAuth account connections for structured app tools |
14
+ | **MCP** | Remote MCP server registration and tool discovery |
15
+ | **Memory** | Long-term memory, core facts, and session search |
16
+ | **Devices** | Server-side browser and Android runtime controls |
17
+ | **Recordings** | Audio sessions, transcripts, segment playback, AI insights |
18
+ | **Health** | Android Health Connect sync status and synced metrics |
19
+ | **Settings** | AI providers, model routing, runtime settings, messaging credentials |
20
+ | **Logs** | Service logs and diagnostics |
25
21
 
26
- NeoAgent records audio as server-side sessions with one or more sources. The web client can record browser microphone and screen audio, and the Android app can record phone microphone audio through a foreground service.
27
-
28
- Recording sessions support:
22
+ ## Agent Tools
29
23
 
30
- - Chunked uploads with per-source sequence checks.
31
- - Sources, chunks, transcript segments, session status, and playback URLs.
32
- - Statuses for recording, processing, completed, failed, and cancelled sessions.
33
- - Retry transcription and delete transcript segment actions.
34
- - Full session deletion with storage cleanup.
35
- - Agent tools for listing, opening, and searching transcripts: `recordings_list`, `recordings_get`, and `recordings_search`.
36
- - A keyless social video extractor for public YouTube, TikTok, Instagram, and X URLs through `social_video_extract`, with metadata-first fetch, captions-first transcript strategy, STT fallback, and representative frame extraction.
24
+ What the agent can use in chat and automation runs:
37
25
 
38
- Transcription uses Deepgram when `DEEPGRAM_API_KEY` is configured. The default speech model is `nova-3`, and the default language mode is `multi`. When `auto_recording_insights` is enabled in AI settings, NeoAgent can generate structured recording insights such as a summary, action items, and events.
26
+ | Area | Capabilities |
27
+ |---|---|
28
+ | **CLI** | PTY-capable `execute_command` with stdin, timeout, stdout/stderr, exit code |
29
+ | **Browser** | Navigate, click, type, extract, screenshot, evaluate JavaScript |
30
+ | **Android** | UI observation, input, screenshots, app launch, intent launch, APK install, `adb shell` |
31
+ | **Web search** | Brave Search API |
32
+ | **Files** | Read, write, edit, list, search |
33
+ | **HTTP** | Direct requests |
34
+ | **Memory** | Semantic search, session search, daily logs, core memory, API key name lookup |
35
+ | **Skills** | Create, list, update, delete persistent skills |
36
+ | **Tasks** | Create, update, delete, and one-time run automations |
37
+ | **MCP** | Add/remove MCP servers, use dynamic MCP tools |
38
+ | **Subagents** | Spawn, wait for, and cancel async helpers inside a run |
39
+ | **Images** | Generate with Grok, analyze with vision models |
40
+ | **Recordings** | List, inspect, search transcripts |
41
+ | **Social video** | Extract transcript and metadata from public YouTube, TikTok, Instagram, and X URLs |
42
+ | **Health** | Read synced mobile health metrics and summaries |
43
+ | **Outputs** | Markdown tables, Mermaid graphs, downloadable artifacts |
39
44
 
40
45
  ## Android Control
41
46
 
42
- NeoAgent can let the AI control an Android emulator or device attached to the NeoAgent server or configured worker. This is the Android capability in the comparison: the agent can observe and operate Android, not only run an Android companion app.
43
-
44
- Android control supports:
45
-
46
- - Starting and stopping the managed Android emulator.
47
- - Listing ADB-connected devices and installed apps.
48
- - Taking screenshots and UIAutomator XML dumps.
49
- - Observing visible UI nodes.
50
- - Opening apps and Android intents.
51
- - Tapping, long pressing, typing, swiping, and pressing Android navigation keys.
52
- - Waiting for text, resource IDs, descriptions, or classes to appear.
53
- - Installing `.apk` and universal `.apks` bundles.
54
- - Running `adb shell` commands when higher-level tools are not enough.
55
-
56
- These actions run where the NeoAgent backend is running. If NeoAgent is deployed on a remote server, the AI controls the Android runtime attached to that server, not the laptop where you are reading the docs.
47
+ The agent operates Android it is not an app that runs on Android. Controls run on a server-attached emulator or physical device over ADB.
57
48
 
58
- ## Android App And Health
49
+ - Start and stop the managed emulator
50
+ - List connected devices and installed apps
51
+ - Screenshot and UIAutomator XML dump
52
+ - Observe visible UI nodes
53
+ - Open apps and launch Android intents
54
+ - Tap, long press, type, swipe, press navigation keys
55
+ - Wait for text, resource IDs, descriptions, or classes to appear
56
+ - Install `.apk` and `.apks` bundles
57
+ - Run `adb shell` commands directly
59
58
 
60
- The Flutter Android app is still useful as a client. It can sign in to the same self-hosted backend, run chat and operator UI flows, sync Health Connect data, and record audio locally.
59
+ These actions run on the NeoAgent server. If NeoAgent is deployed remotely, it controls the Android runtime on that machine not your local device.
61
60
 
62
- Android app capabilities include:
63
-
64
- - `NEOAGENT_BACKEND_URL` build/run configuration for real devices.
65
- - Health Connect permission flow and background sync.
66
- - Microphone recording through an Android foreground service.
67
- - Boot restore hooks for recording services when Android allows them.
61
+ ## Recordings
68
62
 
69
- ## Health Data
63
+ Audio sessions are recorded server-side. The web client captures browser microphone and screen audio; the Android app records phone microphone audio via a foreground service.
70
64
 
71
- Health data comes from the Android app through `/api/mobile/health`. NeoAgent stores sync runs and normalized metric samples. The built-in metric aliases include steps, heart rate, sleep sessions, exercise sessions, and weight.
65
+ - Chunked uploads with per-source sequence tracking
66
+ - Session statuses: recording, processing, completed, failed, cancelled
67
+ - Transcript segment retry and deletion
68
+ - Transcript search across sessions
69
+ - Agent tools: `recordings_list`, `recordings_get`, `recordings_search`
70
+ - Social video extraction via `social_video_extract` — title, description, transcript, and a representative frame from YouTube, TikTok, Instagram, and X URLs
72
71
 
73
- The agent tool `read_health_data` returns summaries and recent samples. It is designed to answer questions such as recent step totals or available health metrics without dumping every raw record.
72
+ Transcription uses Deepgram (`nova-3` model, multi-language by default). Enable `auto_recording_insights` in AI settings to generate summaries, action items, and events automatically after transcription.
74
73
 
75
- ## Integrations And Messaging
74
+ ## Runtime Modes
76
75
 
77
- NeoAgent has two separate integration layers:
76
+ | Profile | What runs where |
77
+ |---|---|
78
+ | `trusted-host` | CLI and Android run on the host; browser runs in the VM or paired extension |
79
+ | `secure-vm` | CLI, browser, and Android all run inside the isolated VM |
78
80
 
79
- - Official integrations expose structured tools for Google Workspace, Microsoft 365, Notion, Slack, Figma, Home Assistant, Trello, Weather, Spotify, and a separate personal WhatsApp connection.
80
- - Messaging platforms let the agent talk through WhatsApp, Telegram, Discord, Slack, Google Chat, Teams, Matrix, Signal, iMessage/BlueBubbles, IRC, Twitch, LINE, Mattermost, configurable webhook bridges, and Telnyx Voice.
81
+ Production deployments can require `secure-vm` and a strong `NEOAGENT_VM_GUEST_TOKEN` (32+ characters).
81
82
 
82
- Official integration examples include Gmail thread search and send mail, Google Calendar events, Drive upload/download/export/share links, Docs create/append/replace, Sheets read/update/append/create, Microsoft Outlook/Calendar/OneDrive/Teams tools, Notion search/page/block/database tools, Slack conversation/message tools, Figma file/node/comment/image tools, Home Assistant entity/config reads and service calls, Trello board/list/card/comment/search tools, Weather current/forecast tools plus weather-event task triggers, Spotify playback/search/control tools, and a personal WhatsApp integration with isolated chat read/send tools and per-account read-only versus read/write access.
83
+ The browser always runs in isolation either the local VM or a paired Chrome extension on a remote machine. To pair an extension: download `/api/browser-extension/download` from NeoAgent, unzip it, enable Developer Mode in `chrome://extensions`, load the folder, then pair after signing in.
83
84
 
84
- Messaging examples include Telegram and Discord messages, Slack channel replies, Matrix room messages, Google Chat and Teams webhook delivery, Signal bridge delivery, iMessage/BlueBubbles sends, WhatsApp text and media sends, Telnyx inbound voice, Telnyx outbound calls, and scheduled-task call delivery.
85
+ ## Integrations and Messaging
85
86
 
86
- ## Agent Tools
87
+ NeoAgent has two separate layers:
87
88
 
88
- NeoAgent's agent tool surface includes more than basic chat:
89
+ **Official integrations** structured OAuth-backed tools the agent can use:
89
90
 
90
- | Area | Examples |
91
+ | Provider | Tools |
91
92
  |---|---|
92
- | CLI | PTY-capable `execute_command` with stdin, timeout, stdout, stderr, exit code, and duration |
93
- | Browser | Navigate, click, type, extract, screenshot, and evaluate page JavaScript |
94
- | Android control | UI observation, input, screenshots, app launch, intent launch, APK install, and shell commands |
95
- | Web search | Brave Search API through `web_search` |
96
- | Files | Read, write, edit, list, and search files |
97
- | HTTP | Direct HTTP requests |
98
- | Memory | Semantic memory, session search, daily logs, API key name reads, and core memory |
99
- | Skills | Create, list, update, and delete persistent skills |
100
- | Tasks | Schedule-triggered and integration-triggered automations, one-time runs, model overrides, and optional Telnyx call delivery |
101
- | MCP | Add, list, and remove MCP servers, plus dynamic MCP tool use |
102
- | Subagents | Spawn, list, wait for, and cancel async subagents inside a run |
103
- | Output | Generate markdown tables and Mermaid graphs |
104
- | Images | Generate images with Grok and analyze local image files with a vision-capable model |
105
- | Recordings | List, inspect, and search recording transcripts |
106
- | Social video | Extract title, description, transcript, and a representative frame from public social video URLs |
107
- | Health | Read synced mobile health metrics |
108
-
109
- Generated binary or text artifacts can be promoted into user-scoped artifact storage under `~/.neoagent/data/artifacts` and served through authenticated `/api/artifacts/:id/content` URLs.
93
+ | Google Workspace | Gmail, Calendar, Drive, Docs, Sheets |
94
+ | Microsoft 365 | Outlook, Calendar, OneDrive, Teams |
95
+ | Notion | Pages, databases, blocks, search |
96
+ | Slack | Messages, conversations, search |
97
+ | Figma | Files, nodes, comments, rendered images |
98
+ | Home Assistant | Entity state, service calls |
99
+ | Trello | Boards, lists, cards, comments |
100
+ | Spotify | Playback, search, queue |
101
+ | Weather | Current conditions and forecasts (no API key needed) |
102
+ | Personal WhatsApp | Per-account read and send |
110
103
 
111
- ## Runtime Modes
104
+ **Messaging platforms** — channels for communicating with the agent:
112
105
 
113
- Runtime settings let operators choose where higher-risk work runs:
106
+ WhatsApp, Telegram, Discord, Slack, Google Chat, Teams, Matrix, Signal, iMessage/BlueBubbles, IRC, Twitch, LINE, Mattermost, Telnyx Voice, plus webhook bridges for Feishu, Nextcloud Talk, Nostr, Synology Chat, Tlon, Zalo, WeChat, and WebChat.
114
107
 
115
- | Profile | Runtime shape |
116
- |---|---|
117
- | `trusted-host` | CLI and Android tools run on the host; browser runs in the VM or paired extension |
118
- | `secure-vm` | CLI, browser, and Android tools run through the local VM backend |
108
+ Each official integration account can be set to **Read/Write** (default) or **Read Only**. Write tools are blocked server-side for read-only accounts.
109
+
110
+ ## Android App and Health
119
111
 
120
- Production policy can require the secure VM profile and a strong VM guest token.
112
+ The Flutter Android app connects to the same self-hosted backend and can:
121
113
 
122
- These controls matter operationally: the browser, Android emulator, local files, and shell commands run wherever the NeoAgent backend, VM, or paired browser extension is running, not necessarily on the computer where you are reading the docs. Logs from a different server or remote browser may not match the logs on the local machine.
114
+ - Run chat and operator UI
115
+ - Sync Android Health Connect data (steps, heart rate, sleep, exercise, weight)
116
+ - Record microphone audio via a foreground service
123
117
 
124
- For extension-only remote browser control, download `/api/browser-extension/download` from NeoAgent, unzip it on the remote machine, load the folder in `chrome://extensions`, and pair after logging in. The extension uses Chrome's debugger permission for full browser control, so Chrome will show its normal debugging warning while attached. The popup can check whether the server has a newer extension bundle, but unpacked Developer Mode installs still need a manual download and reload.
118
+ Synced health data is available through the `read_health_data` agent tool and the **Health** UI section.