devicely 2.3.3 → 2.3.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (106) hide show
  1. package/README.md +168 -249
  2. package/bin/devicely.js +1 -1
  3. package/config/apps_presets.conf +19 -0
  4. package/lib/actionSchema.js +1 -0
  5. package/lib/advanced-logger.js +1 -1
  6. package/lib/aiProviders.js +1 -1
  7. package/lib/aiProvidersConfig.js +1 -1
  8. package/lib/androidDeviceDetection.js +1 -1
  9. package/lib/api/v1/auth.js +1 -0
  10. package/lib/api/v1/deviceResolver.js +1 -0
  11. package/lib/api/v1/index.js +1 -0
  12. package/lib/api/v1/operations.js +1 -0
  13. package/lib/api/v1/passcode.js +1 -0
  14. package/lib/api/v1/resultNormalizer.js +1 -0
  15. package/lib/api/v1/screenCapture.js +1 -0
  16. package/lib/api/v1/uiTree.js +1 -0
  17. package/lib/appListing.js +1 -0
  18. package/lib/appMappings.js +1 -1
  19. package/lib/cognitiveLocator.js +1 -1
  20. package/lib/commanderService.js +1 -1
  21. package/lib/deviceContextProvider.js +1 -0
  22. package/lib/deviceDetection.js +1 -1
  23. package/lib/deviceDriver.js +1 -0
  24. package/lib/deviceHealthMonitor.js +1 -0
  25. package/lib/devices.js +1 -1
  26. package/lib/doctor.js +1 -1
  27. package/lib/encryption.js +1 -1
  28. package/lib/executor.js +1 -1
  29. package/lib/frontend/asset-manifest.json +5 -7
  30. package/lib/frontend/index.html +1 -1
  31. package/lib/frontend/static/js/main.bd89f5ef.js +2 -0
  32. package/lib/hybridAI.js +1 -1
  33. package/lib/intelligentLocatorService.js +1 -1
  34. package/lib/lightweightAI.js +1 -1
  35. package/lib/localBuiltInAI.js +1 -1
  36. package/lib/localBuiltInAI_backup.js +1 -1
  37. package/lib/localBuiltInAI_simple.js +1 -1
  38. package/lib/locatorStrategy.js +1 -1
  39. package/lib/logger.js +1 -1
  40. package/lib/mcp/audit.js +1 -0
  41. package/lib/mcp/client.js +1 -0
  42. package/lib/mcp/clientConfigs.js +1 -0
  43. package/lib/mcp/format.js +1 -0
  44. package/lib/mcp/http.js +1 -0
  45. package/lib/mcp/index.js +1 -0
  46. package/lib/mcp/policy.js +1 -0
  47. package/lib/mcp/prompts.js +1 -0
  48. package/lib/mcp/resources.js +1 -0
  49. package/lib/mcp/schemas.js +1 -0
  50. package/lib/mcp/stdio.js +3 -0
  51. package/lib/mcp/tools/act.js +1 -0
  52. package/lib/mcp/tools/apps.js +1 -0
  53. package/lib/mcp/tools/devices.js +1 -0
  54. package/lib/mcp/tools/diagnostics.js +1 -0
  55. package/lib/mcp/tools/flow.js +1 -0
  56. package/lib/mcp/tools/multi.js +1 -0
  57. package/lib/mcp/tools/observe.js +1 -0
  58. package/lib/mcp/tools/recordings.js +1 -0
  59. package/lib/mjpegStream.js +1 -0
  60. package/lib/package.json +9 -1
  61. package/lib/quick-start-logger.js +1 -1
  62. package/lib/recordingSteps.js +1 -0
  63. package/lib/recordingStore.js +1 -0
  64. package/lib/replayEngine.js +1 -0
  65. package/lib/scriptLoader.js +1 -1
  66. package/lib/server.js +1 -1
  67. package/lib/tensorflowAI.js +1 -1
  68. package/lib/tinyAI.js +1 -1
  69. package/lib/universalSessionManager.js +1 -1
  70. package/lib/virtualDevices/androidEmulatorProvider.js +1 -0
  71. package/lib/virtualDevices/index.js +1 -0
  72. package/lib/virtualDevices/iosSimulatorCommands.js +1 -0
  73. package/lib/virtualDevices/iosSimulatorProvider.js +1 -0
  74. package/lib/virtualDevices/parsers.js +1 -0
  75. package/lib/virtualDevices/ports.js +1 -0
  76. package/lib/virtualDevices/toolchain.js +1 -0
  77. package/lib/virtualDevices/wdaSimulator.js +1 -0
  78. package/package.json +8 -14
  79. package/scripts/shell/android_device_control.enc +1 -1
  80. package/scripts/shell/connect_android_usb_multi_final.enc +1 -1
  81. package/scripts/shell/connect_ios_usb_multi_final.enc +1 -1
  82. package/scripts/shell/connect_ios_wireless_multi_final.enc +1 -1
  83. package/lib/.env.example +0 -20
  84. package/lib/aiProviders.js.strategic-backup +0 -657
  85. package/lib/devices.js.strategic-backup +0 -57
  86. package/lib/encryption.js.strategic-backup +0 -61
  87. package/lib/eslint.config.js +0 -1
  88. package/lib/executor.js.strategic-backup +0 -107
  89. package/lib/frontend/static/css/main.ed39ba12.css +0 -2
  90. package/lib/frontend/static/js/main.bb9425de.js +0 -3
  91. package/lib/jest.config.js +0 -1
  92. package/lib/logger-demo.js +0 -2
  93. package/lib/logger-integration-examples.js +0 -102
  94. package/lib/public/asset-manifest.json +0 -13
  95. package/lib/public/index.html +0 -1
  96. package/lib/public/static/css/main.ed39ba12.css +0 -2
  97. package/lib/public/voice-test.html +0 -156
  98. package/lib/server.js.strategic-backup +0 -6298
  99. package/lib/tensorflowAI.js.strategic-backup +0 -717
  100. package/lib/tests/__mocks__/logger.js +0 -12
  101. package/lib/tests/integration/deviceSync.test.js +0 -74
  102. package/lib/tests/setup/testSetup.js +0 -1
  103. package/lib/tests/unit/commanderService.test.js +0 -70
  104. package/lib/tsconfig.json +0 -18
  105. /package/lib/{public → frontend}/static/css/main.1bbcfef2.css +0 -0
  106. /package/lib/frontend/static/js/{main.bb9425de.js.LICENSE.txt → main.bd89f5ef.js.LICENSE.txt} +0 -0
package/README.md CHANGED
@@ -1,321 +1,240 @@
1
1
  <div align="center">
2
2
 
3
- # 📱✨ Devicely
3
+ # Devicely
4
4
 
5
- ### **Control one device, and every connected device follows in real-time - with live screen sync on the web**
5
+ ### Tap once. Every iPhone, iPad and Android follows.
6
6
 
7
- *Ever wondered what happens when you combine device mirroring with AI-powered automation? Meet the mobile orchestration platform that curious developers are exploring.*
7
+ Control a fleet of real iOS and Android devices from one screen — by hand, by voice, or through AI agents over MCP. Runs locally on your machine, against your own devices.
8
8
 
9
- ### 🤔 **Intrigued? Try it yourself:**
9
+ [![npm version](https://img.shields.io/npm/v/devicely?style=for-the-badge&logo=npm&color=CB3837)](https://www.npmjs.com/package/devicely)
10
+ [![npm downloads](https://img.shields.io/npm/dm/devicely?style=for-the-badge&color=CB3837)](https://www.npmjs.com/package/devicely)
11
+ [![Platforms](https://img.shields.io/badge/devices-iOS%20%7C%20iPadOS%20%7C%20Android-lightgrey?style=for-the-badge)](#supported-platforms)
12
+ [![MCP](https://img.shields.io/badge/MCP-30%20tools-8A2BE2?style=for-the-badge)](#for-ai-agents-mcp-server)
13
+ [![License](https://img.shields.io/badge/license-free%20%7C%20closed%20source-blue?style=for-the-badge)](#license)
14
+
15
+ <img src="https://raw.githubusercontent.com/sureshkumarm8/Devicely/main/DevicelyDemo.gif" alt="One tap on the commander device is mirrored on every follower device" width="900">
16
+
17
+ [Quick start](#quick-start) · [For QA teams](#for-qa-teams) · [For AI agents](#for-ai-agents-mcp-server) · [CLI](#cli-reference) · [Supported platforms](#supported-platforms) · [Privacy](#privacy-and-security)
18
+
19
+ </div>
20
+
21
+ ---
22
+
23
+ ## Quick start
10
24
 
11
25
  ```bash
12
26
  npm install -g devicely
13
- devicely start
27
+ devicely doctor # checks Node, adb, Xcode and device drivers
28
+ devicely start # starts Devicely and opens the Command Center
14
29
  ```
15
- **Visit [https://devicely-ai.vercel.app/](https://devicely-ai.vercel.app/) and see what happens when devices work together.**
16
30
 
17
- [![NPM Version](https://img.shields.io/badge/npm-v2.3.0-CB3837?style=for-the-badge)](https://www.npmjs.com/package/devicely)
18
- [![License](https://img.shields.io/badge/license-MIT-green?style=for-the-badge)](LICENSE)
19
- [![Platform](https://img.shields.io/badge/platform-iOS%20%7C%20Android-lightgrey?style=for-the-badge)](https://devicely-ai.vercel.app)
31
+ The Command Center opens at **[devicely-ai.vercel.app](https://devicely-ai.vercel.app)**. Plug in a device over USB, or click **Connect via QR** to pair one over Wi-Fi.
20
32
 
21
- [🚀 Quick Start](#-quick-start) • [✨ Key Features](#-the-devicely-advantage) • [🌌 Infinite Possibilities](#-glimpse-of-the-infinite) • [📟 CLI Reference](#-cli-reference)
33
+ **No phone handy?** Boot an Android emulator or iOS simulator with the server:
22
34
 
23
- ---
35
+ ```bash
36
+ devicely emulators list # shows installed AVDs and simulators
37
+ devicely start --launch <name> # starts Devicely and boots that virtual device
38
+ ```
24
39
 
25
- ### 🤷‍♀️ **What if mobile testing was more like conducting an orchestra?**
40
+ **Requirements:** Node.js 18+ on macOS or Linux. Android needs `adb` (platform-tools) on your `PATH`. iOS and iPadOS devices need a Mac with Xcode — see [iOS setup](#ios-setup).
26
41
 
27
- Instead of manually testing the same flow across multiple devices, what if you could:
42
+ ---
28
43
 
29
- - **Speak naturally** to your devices: *"Take a screenshot"* → all devices respond
30
- - **Touch one screen** and watch others mirror your actions in real-time
31
- - **Record once**, replay everywhere - from 1 device to 100+
32
- - **See everything** happening across your device fleet on one web dashboard
44
+ ## Why Devicely
33
45
 
34
- *Turns out, when devices can follow a leader and understand natural language, mobile development gets pretty interesting.*
46
+ Testing one flow on ten devices usually means doing it ten times by hand, renting cloud devices by the minute, or maintaining per-device scripts. Devicely takes a different approach: **you drive one device, and the rest of your fleet does the same thing at the same moment.**
35
47
 
36
- </div>
48
+ | | Devicely | Cloud device labs | Script frameworks (Appium, Maestro, Detox) | Screen mirroring (scrcpy, Vysor) |
49
+ | :--- | :---: | :---: | :---: | :---: |
50
+ | Live 1-to-many gestures (one tap → every device) | ✅ | ❌ | ❌ | ❌ |
51
+ | iOS and Android mixed in one live session | ✅ | ❌ | ❌ | ❌ |
52
+ | Uses your own devices, on your own machine | ✅ | ❌ | ✅ | ✅ |
53
+ | No code needed (UI, voice, plain English) | ✅ | Partial | ❌ | ✅ (one device) |
54
+ | Built-in MCP server for AI agents | ✅ | Varies | Varies | ❌ |
55
+ | Exports locators to test code | ✅ | — | ✅ | ❌ |
37
56
 
38
- ---
57
+ Devicely doesn't replace your test framework. It shortens the slow parts around it: exploring, reproducing bugs across devices, smoke-testing builds, and finding the locators you'll put in your tests.
39
58
 
40
59
  ---
41
60
 
42
- ## 🌟 **What makes this different?**
43
-
44
- ### 👑 **Commander Mode** - *Like screen sharing, but for device control*
45
- * One device leads, others follow in real-time
46
- * Touch the leader's screen → all followers mirror the action
47
- * Switch leaders with a simple dropdown
48
- * Watch it all happen on your web dashboard
49
- * **[NEW in v2.3.0]** Enhanced WDA API for iOS device control
50
- * **[NEW in v2.3.0]** Element caching for faster command execution
51
-
52
- ### 📱 **QR-Based Device Enrollment** - *Connect devices in seconds, not minutes*
53
- * **[NEW in v2.3.0]** Scan QR code to enroll iOS or Android devices
54
- * **[NEW in v2.3.0]** Automatic IP address detection and configuration
55
- * **[NEW in v2.3.0]** Works with both USB and wireless connections
56
- * iOS: WebDriverAgent auto-detection
57
- * Android: Automatic ADB setup with privacy controls
58
- * No more manual IP configuration or device.conf editing
59
-
60
- ### 🔄 **Live Sync** - *What if all your devices moved together?*
61
- * Commands reach all devices simultaneously via WebSockets
62
- * iOS and Android understand the same instructions differently (as they should)
63
- * See live status updates from every connected device
64
- * No waiting for one device to finish before the next starts
65
- * **[NEW in v2.3.0]** Improved wireless connectivity with retry logic
66
-
67
- ### 📹 **Record & Replay - Zero-Code Automation**
68
- * **Visual Recording:** Capture complex workflows with simple button clicks
69
- * **AI-Generated Scripts:** Voice commands auto-convert to replayable recordings
70
- * **Fleet Replay:** Run recorded flows on single devices or entire laboratories
71
- * **Edit & Refine:** Full editor for recordings with add/remove/modify capabilities
72
-
73
- ### 🤖 **AI Integration** - *Talk to your devices like a person*
74
- * Say *"Open Instagram and find search"* - it just works
75
- * Choose from 7 AI providers with latest models:
76
- - **OpenAI:** GPT-4o (Aug 2024)
77
- - **Google:** Gemini 1.5 Pro/Flash
78
- - **Anthropic:** Claude 3.5 Sonnet (Oct 2024)
79
- - **Groq:** Llama 3.3 70B (ultra-fast LPU inference)
80
- - **Cohere:** Command R+ (enterprise)
81
- - **Mistral:** Mistral Large 2
82
- - **GitHub:** Copilot powered
83
- * **[NEW in v2.3.0]** API-only mode - no local models, optimized for speed
84
- * Devices understand context - no hunting for element IDs
85
- * Same command, different results on iOS vs Android (intelligently)
86
-
87
- ### 🎤 **Voice Control** - *Because typing is so 2020*
88
- * Speak naturally, pause when you're thinking (2-second detection)
89
- * Chain commands: *"launch settings, scroll up, launch camera"*
90
- * Watch your words turn into actions in real-time
91
- * Works across as many devices as you want to connect
61
+ ## For QA teams
92
62
 
93
- ---
63
+ ### Commander Mode — one-to-many control
94
64
 
95
- ## ✨ **Other things you might find useful**
65
+ - **Live mirror of the commander device**, with every follower shown alongside it.
66
+ - **Every gesture is broadcast**: tap, long-press, swipe (speed preserved) and typed text go to all followers at once.
67
+ - **Element-aware, not pixel-blind.** Followers find the element the commander touched — by accessibility ID, label, type and position in the UI tree — so a tap on an iPhone hits the same button on an iPad or a Pixel, even when the layouts differ.
68
+ - **Swap the commander** from a dropdown on the mirror, without restarting the session.
69
+ - **Include or exclude individual followers** with one click.
70
+ - **Fast iOS mirroring** over WebDriverAgent MJPEG: about 170 KB per frame, compared with about 5.8 MB for a screenshot poll. Each iOS device gets its own WDA port, so several iPhones don't collide.
71
+ - **Full-screen mirror** with zoom and fit-to-window, and a **D-pad** for directional scrolling and swiping.
96
72
 
97
- ### ⚡ **Quick Actions Panel**
98
- * **Type Text to Send:** Rapid text input without typing full commands
99
- * **Element to Tap:** Click by name or coordinates (x,y) instantly
100
- * **App ID to Launch:** Launch apps with autocomplete dropdown
101
- * **History Navigation:** Arrow keys cycle through previous inputs
73
+ ### Recordings that survive UI changes
102
74
 
103
- ### 🎮 **Unified Command Center**
104
- * **Cross-Platform:** Native support for **iOS 14+** and **Android 5+**
105
- * **Massive Scale:** Control 5, 50, or 100+ devices with zero overhead
106
- * **Live Feedback:** Real-time logs and visual confirmations via WebSockets
107
- * **API Key Management:** Secure local storage with show/hide toggles
75
+ - **Steps are recorded as locators** (`accessibilityId`, `resourceId`, text, type, bounds), not raw coordinates.
76
+ - **Auto-wait**: on replay, each step waits up to 10 s for its element to appear, so animations and slow networks don't break the run.
77
+ - **Coordinate fallback**: if an element can't be found, Devicely taps its scaled position instead and marks the step `fallback`, so you can see what drifted.
78
+ - **Replay on one device or the whole fleet in parallel.**
79
+ - **Manage recordings** in a list or grid view with search and tag filters. A details drawer shows each step's locator, timing and result; the editor lets you add, edit, reorder and delete steps.
108
80
 
109
- ### 🔍 **Smart UI Inspector**
110
- * **Visual Element Scan:** See all clickable, enabled, visible elements instantly
111
- * **Point-and-Click Control:** Direct device interaction through inspector
112
- * **Developer Export:** Clean selectors and coordinates for automation scripts
81
+ ### Plain English and voice
113
82
 
114
- ### 🌌 **The bigger picture**
115
- * Transform 2-hour manual testing into 5-minute parallel runs
116
- * Turn your device collection into a web-accessible lab
117
- * Reproduce bugs instantly across different devices/OS versions
118
- * Mix iOS and Android devices - they all speak the same language
83
+ Type or say *"launch settings, scroll down, open camera"*. Devicely turns it into the right action for each platform — for example `com.apple.Preferences` on iOS and `com.android.settings` on Android — and runs it on every selected device at once. Voice input waits for a 2-second pause before it runs, so you can speak naturally.
119
84
 
120
- ---
85
+ Choose your AI provider in **Settings → AI Config**:
121
86
 
122
- ## 🔍 **How it works** *(in about 60 seconds)*
87
+ | Provider | Notes |
88
+ | :--- | :--- |
89
+ | **Ollama** | Runs locally. No API key, and no prompt leaves your machine. |
90
+ | **Google Gemini**, **Groq**, **Cohere** | Offer free tiers — the easiest way to start. |
91
+ | **OpenAI**, **Anthropic Claude**, **Mistral AI** | Bring your own API key. |
92
+ | **GitHub Copilot** | Uses your Copilot subscription. |
123
93
 
124
- ### **Step 1: Get it running**
125
- ```bash
126
- npm install -g devicely
127
- devicely start
128
- ```
94
+ Each provider lists its current models in the settings screen.
95
+
96
+ ### Quick actions, inspector and code export
129
97
 
130
- ### **Step 2: Connect and explore**
131
- 1. Open [https://devicely-ai.vercel.app/](https://devicely-ai.vercel.app/)
132
- 2. Connect your iOS/Android devices (as many as you want)
133
- 3. Try saying: *"Take screenshot on all devices"*
134
- 4. Watch what happens
98
+ - **Quick Actions panel**: send text, tap an element by name or `x,y`, and launch apps with autocomplete. Arrow keys cycle through what you typed before.
99
+ - **UI Inspector**: hover over any element to see its accessibility ID, resource ID, XPath and bounds.
100
+ - **One-click export** of a selector as a Devicely command, **WebdriverIO**, **Appium Python** or **Appium Java** code.
135
101
 
136
- ### **Step 3: Discover the AI layer** *(optional, but fun)*
137
- - Settings → AI Config → Choose a provider (Gemini/Groq are free)
138
- - Now you can talk to your devices in plain English
139
- - *"Launch settings, scroll down, open camera"* - just works
102
+ ### Devices that stay connected
140
103
 
141
- **💡 Curious tip:** The more devices you connect, the more interesting it gets.
104
+ - **QR enrollment**: scan a code from the Command Center to pair a device over Wi-Fi — no IP addresses to type.
105
+ - **Health monitoring** spots an unplugged cable, a reboot or a driver failure within seconds and cleans up the session.
106
+ - **Auto-reconnect** brings a device back as soon as it reappears on USB or Wi-Fi. It's on by default; set `DEVICELY_AUTO_RECONNECT=0` to turn it off.
107
+ - **Virtual devices**: start and stop Android emulators and iOS simulators from the UI or the CLI, next to your physical devices.
142
108
 
143
109
  ---
144
110
 
145
- ## 🎯 Usage Examples
111
+ ## For AI agents (MCP server)
112
+
113
+ Devicely includes a [Model Context Protocol](https://modelcontextprotocol.io) server, so coding agents can see and control real devices.
146
114
 
147
- ### Natural Language Commands
148
115
  ```bash
149
- # Simple actions
150
- "take screenshot"
151
- "launch chrome"
152
- "go to google.com"
116
+ devicely mcp install claude-code # prints the `claude mcp add` command to run
117
+ devicely mcp install cursor # also: vscode, windsurf, gemini, claude-desktop
118
+ devicely mcp doctor # checks the backend, token and transports
119
+ ```
153
120
 
154
- # Multi-step sequences
155
- "launch settings scroll up launch camera go to youtube.com press home"
121
+ Or add it to Claude Code by hand:
156
122
 
157
- # Cross-platform (automatically uses correct package IDs)
158
- "launch settings" # → com.apple.Preferences (iOS) + com.android.settings (Android)
123
+ ```bash
124
+ claude mcp add devicely -- npx -y devicely mcp
159
125
  ```
160
126
 
161
- ### Voice Commands
162
- 1. Click 🎤 microphone button
163
- 2. Speak: "launch settings scroll up launch camera"
164
- 3. Pause for 2 seconds
165
- 4. Command executes on all selected devices!
166
-
167
- ### Recording & Replay
168
- 1. Click **Record** button
169
- 2. Perform actions on device
170
- 3. Click **Stop Recording**
171
- 4. Save with name
172
- 5. Replay anytime on any device(s)
173
-
174
- ### Commander Mode
175
- 1. Connect 2+ devices
176
- 2. First device auto-becomes commander
177
- 3. Click on commander screen to broadcast actions
178
- 4. Use top-left dropdown on mirror to swap commander
179
- 5. All devices stay synchronized automatically
127
+ Two transports are available: **stdio** (`devicely mcp`, which starts the backend if it isn't running) and **Streamable HTTP**, protected by a Bearer token. Run `devicely mcp doctor` to see the HTTP endpoint.
180
128
 
181
- ---
129
+ ### 30 tools
182
130
 
183
- ## 🤖 AI Provider Comparison
184
-
185
- Devicely supports **7 leading AI providers** with multiple models to choose from:
186
-
187
- | Provider | Models | Speed | Free Tier | Best For |
188
- |----------|--------|-------|-----------|----------|
189
- | **🤖 OpenAI** | GPT-4 Turbo, GPT-4, GPT-3.5 | 🔥🔥 | ❌ | General purpose, accuracy |
190
- | **✨ Google Gemini** | 2.0 Flash, 1.5 Pro, 1.5 Flash | 🔥🔥🔥 | ✅ Yes | Cost-effective, multimodal |
191
- | **🧠 Anthropic Claude** | 3.5 Sonnet, 3 Opus, 3 Haiku | 🔥🔥 | ❌ | Complex reasoning, long context |
192
- | **🐙 GitHub Copilot** | GPT-4, GPT-3.5 | 🔥🔥 | ❌ | Developers with subscription |
193
- | **⚡ Groq** | Llama 3 70B, Mixtral, Gemma | 🔥🔥🔥 | ✅ Yes | **Ultra-fast inference** |
194
- | **🌊 Cohere** | Command R+, Command R | 🔥🔥 | ✅ Yes | Enterprise-grade |
195
- | **🌬️ Mistral AI** | Large, Medium, Small | 🔥🔥 | ❌ | EU compliance, open-source |
196
-
197
- ### 💡 Recommendations
198
- - **Free Users:** Start with Gemini 2.0 Flash or Groq Llama 3
199
- - **Speed Critical:** Use Groq (fastest inference available)
200
- - **Best Quality:** Claude 3.5 Sonnet or GPT-4 Turbo
201
- - **Cost Effective:** Gemini Flash or GPT-3.5 Turbo
202
-
203
- Get API Keys:
204
- - **OpenAI:** https://platform.openai.com/api-keys
205
- - **Gemini:** https://makersuite.google.com/app/apikey
206
- - **Claude:** https://console.anthropic.com/account/keys
207
- - **Groq:** https://console.groq.com/keys
208
- - **Cohere:** https://dashboard.cohere.com/api-keys
209
- - **Mistral:** https://console.mistral.ai/api-keys
131
+ | Group | Tools |
132
+ | :--- | :--- |
133
+ | **Observe** | `devicely_snapshot`, `devicely_find_elements`, `devicely_wait_for`, `devicely_get_device_logs`, `devicely_get_action_log` |
134
+ | **Act** | `devicely_tap`, `devicely_long_press`, `devicely_swipe`, `devicely_type_text`, `devicely_press_key`, `devicely_open_url` |
135
+ | **Apps** | `devicely_launch_app`, `devicely_close_app`, `devicely_list_apps`, `devicely_install_app`, `devicely_uninstall_app` |
136
+ | **Devices** | `devicely_list_devices`, `devicely_connect_device`, `devicely_disconnect_device`, `devicely_restart_device`, `devicely_set_passcode`, `devicely_doctor` |
137
+ | **Fleet and flows** | `devicely_commander`, `devicely_run_commands`, `devicely_run_natural_language` |
138
+ | **Virtual devices** | `devicely_launch_virtual_device`, `devicely_stop_virtual_device` |
139
+ | **Recordings** | `devicely_save_recording`, `devicely_list_recordings`, `devicely_replay_recording` |
210
140
 
211
- ---
141
+ **Prompts** for common jobs: `smoke_test`, `reproduce_bug` and `write_test`.
142
+ **Resources** with live context: `devicely://devices`, `devicely://apps`, `devicely://docs/command-language` and `devicely://docs/agent-guide`.
212
143
 
213
- ## 📟 CLI Reference
144
+ ### Safety controls
214
145
 
215
- Devicely comes with a powerful command-line interface for quick actions:
146
+ Set these in **Settings → AI Agents** (stored in `~/.devicely/mcp.json`):
216
147
 
217
- | Command | Description |
218
- | :--- | :--- |
219
- | `devicely start` | Launch the web-based Command Center. |
220
- | `devicely list` | List all connected iOS and Android devices. |
221
- | `devicely doctor` | Verify system requirements (ADB, WDA, etc.). |
222
- | `devicely exec "<cmd>"` | Execute a natural language command on all devices. |
148
+ - **Destructive tools are off by default**: `install_app`, `uninstall_app`, `restart_device`, `set_passcode` and `stop_virtual_device`. When enabled, they can also require confirmation.
149
+ - **Limit agents to specific devices** (`allowedDevices`), turn off individual tools, and redact sensitive values from output.
150
+ - **Every agent action is logged** to an audit trail.
151
+
152
+ ### REST API
153
+
154
+ The same capabilities are available over HTTP at `/api/v1`, using the same Bearer token — useful for CI pipelines and custom harnesses. View or rotate the token with `devicely mcp token [--rotate]`; it's saved in `~/.devicely/mcp-token`.
155
+
156
+ ### Example
157
+
158
+ > *"Take a snapshot of the Pixel, check that the Login button is visible, tap it, and confirm the welcome screen appears."*
159
+
160
+ The agent calls `devicely_snapshot` and `devicely_find_elements`, taps with `devicely_tap`, and checks the result with `devicely_wait_for`.
223
161
 
224
162
  ---
225
163
 
226
- ## 🆕 Latest Features (v2.2.11 - February 2026)
227
-
228
- ### 🐛 **Commander Mode Error Fix (v2.2.11)**
229
- - **Fixed**: "Error starting commander mode" when devices are already connected
230
- - **Improved**: Commander mode now gracefully handles already-active sessions
231
- - **Enhanced**: Better error messaging and automatic session management
232
- - **Smart**: Auto-switches commander between devices without throwing errors
233
-
234
- ### 👑 Commander Mode with Quick Swap (NEW!)
235
- - **Real-Time Mirroring:** Live screen mirroring of commander device with follower devices displayed in background
236
- - **Commander Dropdown:** Top-left dropdown on mirror screen for quick commander device changes
237
- - **One-Click Swap:** Select any connected device to instantly make it the new commander
238
- - **Visual Indicators:** Commander badge (👑) on device cards and mirror screen
239
- - **Auto-Synchronization:** WebSocket-based real-time updates across all components
240
- - **Touch Broadcasting:** Tap commander screen, all followers execute the same action
241
- - **Seamless Transitions:** Smooth commander swaps with automatic follower re-assignment
242
- - **Button Updates:** "Connect All" and "Select All" for clearer bulk operations
243
-
244
- ### 🤖 Multi-AI Provider Support
245
- - **7 AI Providers:** OpenAI, Gemini, Claude, Copilot, Groq, Cohere, Mistral AI
246
- - **Model Selection:** Choose specific models for each provider (GPT-4 Turbo, Claude 3.5 Sonnet, Llama 3 70B, etc.)
247
- - **Smart Defaults:** Auto-selects best model if not specified (marked with ⭐)
248
- - **Revamped UI:** Modern Settings interface with provider cards, model dropdowns, and help sections
249
- - **Free Tiers:** Gemini, Groq, and Cohere offer generous free tiers for testing
250
- - **Performance Hints:** UI shows speed/cost comparisons (Groq = fastest, Claude = best reasoning)
251
-
252
- ### ✨ Enhanced AI Capabilities
253
- - **Multi-Platform Intelligence:** Single AI command generates platform-specific actions for iOS and Android
254
- - **Simultaneous Execution:** All devices execute at the same time, not sequentially
255
- - **Error Visibility:** AI errors (quota limits, auth failures) shown inline with results
256
- - **Response Cleaning:** Automatically removes markdown and explanations from AI output
257
- - **Cross-Platform Apps:** AI generates both iOS and Android app bundles (Settings, Camera, Chrome, etc.)
258
-
259
- ### 🎤 Improved Voice Input
260
- - **Extended Listening:** 2-second silence detection for natural speech
261
- - **Continuous Speech:** Speak multiple phrases with natural pauses
262
- - **Live Transcription:** See your words accumulate in real-time
263
- - **Complex Commands:** "launch settings scroll up launch camera go to google.com press home" works flawlessly
264
- - **No Duplicates:** Fixed transcript accumulation for clean output
265
-
266
- ### 🔐 Settings Enhancements
267
- - **Revamped AI Config Tab:** Professional card-based design with dark mode support
268
- - **Dynamic API Key Input:** Field changes based on selected provider
269
- - **Show/Hide Toggles:** Individual toggles for all 7 provider keys
270
- - **Status Display:** Green success box showing current provider, model, and key preview
271
- - **Quick Tips Section:** In-UI recommendations for choosing providers
272
- - **Close Button:** Easy navigation back to main screen
273
- - **Dark Mode:** Proper color schemes for light and dark themes
274
-
275
- ### 📝 Recording Improvements
276
- - **AI Recording Editor:** Edit AI-generated recordings like manual ones
277
- - **Format Conversion:** Seamlessly handles both string and array command formats
278
- - **Save & Replay:** Edit, save, and replay AI-generated commands
279
- - **Status Persistence:** Recording status preserved when navigating between screens
164
+ ## CLI reference
165
+
166
+ | Command | What it does |
167
+ | :--- | :--- |
168
+ | `devicely start` | Start the server and open the Command Center |
169
+ | `devicely start -p <port> --no-browser` | Use a custom port without opening a browser (CI, headless machines) |
170
+ | `devicely start --usb-only` / `--wifi-only` | Only detect devices on USB, or only on Wi-Fi |
171
+ | `devicely start --launch <name>` | Boot a virtual device when the server starts |
172
+ | `devicely start --debug` | Turn on debug logging |
173
+ | `devicely list [--verbose]` | List connected iOS and Android devices |
174
+ | `devicely exec "<command>" [-d <name>]` | Run a command on all devices, or on one with `-d` |
175
+ | `devicely doctor` | Check Node, adb, Xcode and driver setup |
176
+ | `devicely emulators list` | List Android emulators and iOS simulators (alias: `devicely virtual`) |
177
+ | `devicely emulators start <name> [--headless]` | Boot a virtual device |
178
+ | `devicely emulators stop <name>` | Shut down a virtual device |
179
+ | `devicely mcp [--url <url>] [--no-autostart]` | Run the MCP stdio server |
180
+ | `devicely mcp doctor` | Check MCP readiness |
181
+ | `devicely mcp token [--rotate]` | Show or rotate the agent token |
182
+ | `devicely mcp install <client>` | Configure `claude-code`, `claude-desktop`, `cursor`, `vscode`, `windsurf` or `gemini` |
183
+
184
+ Example commands for `exec`:
185
+
186
+ ```bash
187
+ devicely exec "launch settings"
188
+ devicely exec "launch settings scroll down launch camera"
189
+ devicely exec "open url https://github.com" -d "Pixel 8"
190
+ ```
280
191
 
281
192
  ---
282
193
 
283
- ## 📜 License & Privacy
194
+ ## Supported platforms
284
195
 
285
- **Proprietary & Confidential.**
286
- Devicely is currently in a controlled rollout phase. This NPM package provides the binary execution environment. Source code remains private under phase 2 & 3 of our roadmap.
196
+ | | Supported |
197
+ | :--- | :--- |
198
+ | **Host machine** | macOS and Linux, Node.js 18+. Windows isn't supported yet. |
199
+ | **Android** | Phones, tablets and emulators via ADB and UIAutomator2. Wireless debugging needs Android 11+. |
200
+ | **iOS / iPadOS** | iPhones, iPads and simulators via WebDriverAgent. Needs a Mac with Xcode. iOS 16+ also requires Developer Mode. |
287
201
 
288
- **Privacy First:**
289
- - All API keys stored locally on your machine
290
- - No telemetry or data collection
291
- - Commands executed directly on your devices
292
- - No cloud dependencies for core functionality
202
+ ### iOS setup
293
203
 
294
- For enterprise inquiries or commercial access, contact: **devicelyai@gmail.com**
204
+ Physical iOS devices are controlled through WebDriverAgent, which must be signed with your Apple ID or developer team before it can run on a device. Run `devicely doctor` and follow the in-app setup guide; it walks you through installing the iOS tools, enabling Developer Mode and trusting the computer. Android devices and iOS simulators don't need signing.
295
205
 
296
206
  ---
297
207
 
298
- <div align="center">
208
+ ## Privacy and security
299
209
 
300
- ---
210
+ - **Runs locally.** The Devicely server runs on your machine and talks to your devices directly. Your apps, screenshots and test data aren't uploaded anywhere.
211
+ - **Local AI is an option.** Use Ollama to keep prompts and screen data on your machine. With a cloud AI provider, your commands go to that provider under your own API key.
212
+ - **API keys stay on your machine.**
213
+ - **Agents need a token.** MCP over HTTP and the REST API require a 32-byte Bearer token.
214
+ - **Analytics:** the local server sends no telemetry. The hosted dashboard at [devicely-ai.vercel.app](https://devicely-ai.vercel.app) uses anonymous Vercel page analytics; it connects to your local server and doesn't send device data to Vercel.
301
215
 
302
216
  ---
303
217
 
304
- ## 🤔 **Still curious?**
218
+ ## What's new in 2.3.4
305
219
 
306
- Mobile development has always been about juggling multiple devices, hunting for the right selectors, and repeating the same tests over and over. What if there was a more interesting way?
220
+ - **Recordings, rebuilt**: locator-based steps, 10 s auto-wait, coordinate fallback, list and grid views, a details drawer and a step editor.
221
+ - **MCP server and REST API**: 30 tools, prompts and resources, stdio and Streamable HTTP transports, safety policy and an audit log, plus one-command setup for six AI clients.
222
+ - **Faster iOS mirroring** with WebDriverAgent MJPEG streaming and a dedicated WDA port per device.
223
+ - **Device health monitoring and auto-reconnect.**
224
+ - **Virtual devices**: Android emulators and iOS simulators from the UI, CLI and MCP.
225
+ - **Passcode tool**: sets a PIN on Android over ADB and gives guided steps on iOS.
307
226
 
308
- ### **Give it a try:**
227
+ ---
309
228
 
310
- ```bash
311
- npm install -g devicely
312
- devicely start
313
- ```
229
+ ## License
314
230
 
315
- [📦 NPM Package](https://www.npmjs.com/package/devicely) • [🌐 Try it live](https://devicely-ai.vercel.app) • [📧 Questions: devicelyai@gmail.com](mailto:devicelyai@gmail.com)
231
+ **Free to use. Closed source.** Devicely is free to install and use. The source code is proprietary: you may not copy, modify or redistribute it. See [LICENSE](https://github.com/sureshkumarm8/Devicely/blob/main/LICENSE).
316
232
 
317
- *Built by developers who got tired of the same old mobile testing routine.*
233
+ - **Questions and enterprise inquiries:** [devicelyai@gmail.com](mailto:devicelyai@gmail.com)
234
+ - **Report an issue:** [github.com/sureshkumarm8/Devicely/issues](https://github.com/sureshkumarm8/Devicely/issues)
235
+
236
+ <div align="center">
318
237
 
319
- **⭐ Star this repo if you find it interesting!**
238
+ **If Devicely saves you time, [star it on GitHub](https://github.com/sureshkumarm8/Devicely).**
320
239
 
321
240
  </div>