devicely 2.3.4 → 2.3.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +168 -249
- package/bin/devicely.js +1 -1
- package/lib/actionSchema.js +1 -1
- package/lib/advanced-logger.js +1 -1
- package/lib/aiProviders.js +1 -1
- package/lib/aiProvidersConfig.js +1 -1
- package/lib/androidDeviceDetection.js +1 -1
- package/lib/api/v1/auth.js +1 -0
- package/lib/api/v1/deviceResolver.js +1 -0
- package/lib/api/v1/index.js +1 -0
- package/lib/api/v1/operations.js +1 -0
- package/lib/api/v1/passcode.js +1 -0
- package/lib/api/v1/resultNormalizer.js +1 -0
- package/lib/api/v1/screenCapture.js +1 -0
- package/lib/api/v1/uiTree.js +1 -0
- package/lib/appListing.js +1 -0
- package/lib/appMappings.js +1 -1
- package/lib/cognitiveLocator.js +1 -1
- package/lib/commanderService.js +1 -1
- package/lib/deviceContextProvider.js +1 -1
- package/lib/deviceDetection.js +1 -1
- package/lib/deviceDriver.js +1 -0
- package/lib/deviceHealthMonitor.js +1 -0
- package/lib/devices.js +1 -1
- package/lib/doctor.js +1 -1
- package/lib/encryption.js +1 -1
- package/lib/executor.js +1 -1
- package/lib/frontend/asset-manifest.json +5 -7
- package/lib/frontend/index.html +1 -1
- package/lib/frontend/static/js/main.bd89f5ef.js +2 -0
- package/lib/hybridAI.js +1 -1
- package/lib/intelligentLocatorService.js +1 -1
- package/lib/lightweightAI.js +1 -1
- package/lib/localBuiltInAI.js +1 -1
- package/lib/localBuiltInAI_backup.js +1 -1
- package/lib/localBuiltInAI_simple.js +1 -1
- package/lib/locatorStrategy.js +1 -1
- package/lib/logger.js +1 -1
- package/lib/mcp/audit.js +1 -0
- package/lib/mcp/client.js +1 -0
- package/lib/mcp/clientConfigs.js +1 -0
- package/lib/mcp/format.js +1 -0
- package/lib/mcp/http.js +1 -0
- package/lib/mcp/index.js +1 -0
- package/lib/mcp/policy.js +1 -0
- package/lib/mcp/prompts.js +1 -0
- package/lib/mcp/resources.js +1 -0
- package/lib/mcp/schemas.js +1 -0
- package/lib/mcp/stdio.js +3 -0
- package/lib/mcp/tools/act.js +1 -0
- package/lib/mcp/tools/apps.js +1 -0
- package/lib/mcp/tools/devices.js +1 -0
- package/lib/mcp/tools/diagnostics.js +1 -0
- package/lib/mcp/tools/flow.js +1 -0
- package/lib/mcp/tools/multi.js +1 -0
- package/lib/mcp/tools/observe.js +1 -0
- package/lib/mcp/tools/recordings.js +1 -0
- package/lib/mjpegStream.js +1 -0
- package/lib/package.json +8 -1
- package/lib/quick-start-logger.js +1 -1
- package/lib/recordingSteps.js +1 -0
- package/lib/recordingStore.js +1 -0
- package/lib/replayEngine.js +1 -0
- package/lib/scriptLoader.js +1 -1
- package/lib/server.js +1 -1
- package/lib/tensorflowAI.js +1 -1
- package/lib/tinyAI.js +1 -1
- package/lib/universalSessionManager.js +1 -1
- package/lib/virtualDevices/androidEmulatorProvider.js +1 -0
- package/lib/virtualDevices/index.js +1 -0
- package/lib/virtualDevices/iosSimulatorCommands.js +1 -0
- package/lib/virtualDevices/iosSimulatorProvider.js +1 -0
- package/lib/virtualDevices/parsers.js +1 -0
- package/lib/virtualDevices/ports.js +1 -0
- package/lib/virtualDevices/toolchain.js +1 -0
- package/lib/virtualDevices/wdaSimulator.js +1 -0
- package/package.json +8 -15
- package/scripts/shell/android_device_control.enc +1 -1
- package/lib/.env.example +0 -20
- package/lib/aiProviders.js.strategic-backup +0 -657
- package/lib/devices.js.strategic-backup +0 -57
- package/lib/encryption.js.strategic-backup +0 -61
- package/lib/eslint.config.js +0 -1
- package/lib/executor.js.strategic-backup +0 -107
- package/lib/frontend/static/css/main.ed39ba12.css +0 -2
- package/lib/frontend/static/js/main.a940844e.js +0 -3
- package/lib/jest.config.js +0 -1
- package/lib/logger-demo.js +0 -2
- package/lib/logger-integration-examples.js +0 -102
- package/lib/public/asset-manifest.json +0 -13
- package/lib/public/index.html +0 -1
- package/lib/public/static/css/main.ed39ba12.css +0 -2
- package/lib/public/voice-test.html +0 -156
- package/lib/server.js.strategic-backup +0 -6298
- package/lib/tensorflowAI.js.strategic-backup +0 -717
- package/lib/tests/__mocks__/logger.js +0 -12
- package/lib/tests/integration/deviceSync.test.js +0 -74
- package/lib/tests/setup/testSetup.js +0 -1
- package/lib/tests/unit/commanderService.test.js +0 -70
- package/lib/tsconfig.json +0 -18
- /package/lib/{public → frontend}/static/css/main.1bbcfef2.css +0 -0
- /package/lib/frontend/static/js/{main.a940844e.js.LICENSE.txt → main.bd89f5ef.js.LICENSE.txt} +0 -0
package/README.md
CHANGED
|
@@ -1,321 +1,240 @@
|
|
|
1
1
|
<div align="center">
|
|
2
2
|
|
|
3
|
-
#
|
|
3
|
+
# Devicely
|
|
4
4
|
|
|
5
|
-
###
|
|
5
|
+
### Tap once. Every iPhone, iPad and Android follows.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Control a fleet of real iOS and Android devices from one screen — by hand, by voice, or through AI agents over MCP. Runs locally on your machine, against your own devices.
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
[](https://www.npmjs.com/package/devicely)
|
|
10
|
+
[](https://www.npmjs.com/package/devicely)
|
|
11
|
+
[](#supported-platforms)
|
|
12
|
+
[](#for-ai-agents-mcp-server)
|
|
13
|
+
[](#license)
|
|
14
|
+
|
|
15
|
+
<img src="https://raw.githubusercontent.com/sureshkumarm8/Devicely/main/DevicelyDemo.gif" alt="One tap on the commander device is mirrored on every follower device" width="900">
|
|
16
|
+
|
|
17
|
+
[Quick start](#quick-start) · [For QA teams](#for-qa-teams) · [For AI agents](#for-ai-agents-mcp-server) · [CLI](#cli-reference) · [Supported platforms](#supported-platforms) · [Privacy](#privacy-and-security)
|
|
18
|
+
|
|
19
|
+
</div>
|
|
20
|
+
|
|
21
|
+
---
|
|
22
|
+
|
|
23
|
+
## Quick start
|
|
10
24
|
|
|
11
25
|
```bash
|
|
12
26
|
npm install -g devicely
|
|
13
|
-
devicely
|
|
27
|
+
devicely doctor # checks Node, adb, Xcode and device drivers
|
|
28
|
+
devicely start # starts Devicely and opens the Command Center
|
|
14
29
|
```
|
|
15
|
-
**Visit [https://devicely-ai.vercel.app/](https://devicely-ai.vercel.app/) and see what happens when devices work together.**
|
|
16
30
|
|
|
17
|
-
[
|
|
18
|
-
[](LICENSE)
|
|
19
|
-
[](https://devicely-ai.vercel.app)
|
|
31
|
+
The Command Center opens at **[devicely-ai.vercel.app](https://devicely-ai.vercel.app)**. Plug in a device over USB, or click **Connect via QR** to pair one over Wi-Fi.
|
|
20
32
|
|
|
21
|
-
|
|
33
|
+
**No phone handy?** Boot an Android emulator or iOS simulator with the server:
|
|
22
34
|
|
|
23
|
-
|
|
35
|
+
```bash
|
|
36
|
+
devicely emulators list # shows installed AVDs and simulators
|
|
37
|
+
devicely start --launch <name> # starts Devicely and boots that virtual device
|
|
38
|
+
```
|
|
24
39
|
|
|
25
|
-
|
|
40
|
+
**Requirements:** Node.js 18+ on macOS or Linux. Android needs `adb` (platform-tools) on your `PATH`. iOS and iPadOS devices need a Mac with Xcode — see [iOS setup](#ios-setup).
|
|
26
41
|
|
|
27
|
-
|
|
42
|
+
---
|
|
28
43
|
|
|
29
|
-
|
|
30
|
-
- **Touch one screen** and watch others mirror your actions in real-time
|
|
31
|
-
- **Record once**, replay everywhere - from 1 device to 100+
|
|
32
|
-
- **See everything** happening across your device fleet on one web dashboard
|
|
44
|
+
## Why Devicely
|
|
33
45
|
|
|
34
|
-
|
|
46
|
+
Testing one flow on ten devices usually means doing it ten times by hand, renting cloud devices by the minute, or maintaining per-device scripts. Devicely takes a different approach: **you drive one device, and the rest of your fleet does the same thing at the same moment.**
|
|
35
47
|
|
|
36
|
-
|
|
48
|
+
| | Devicely | Cloud device labs | Script frameworks (Appium, Maestro, Detox) | Screen mirroring (scrcpy, Vysor) |
|
|
49
|
+
| :--- | :---: | :---: | :---: | :---: |
|
|
50
|
+
| Live 1-to-many gestures (one tap → every device) | ✅ | ❌ | ❌ | ❌ |
|
|
51
|
+
| iOS and Android mixed in one live session | ✅ | ❌ | ❌ | ❌ |
|
|
52
|
+
| Uses your own devices, on your own machine | ✅ | ❌ | ✅ | ✅ |
|
|
53
|
+
| No code needed (UI, voice, plain English) | ✅ | Partial | ❌ | ✅ (one device) |
|
|
54
|
+
| Built-in MCP server for AI agents | ✅ | Varies | Varies | ❌ |
|
|
55
|
+
| Exports locators to test code | ✅ | — | ✅ | ❌ |
|
|
37
56
|
|
|
38
|
-
|
|
57
|
+
Devicely doesn't replace your test framework. It shortens the slow parts around it: exploring, reproducing bugs across devices, smoke-testing builds, and finding the locators you'll put in your tests.
|
|
39
58
|
|
|
40
59
|
---
|
|
41
60
|
|
|
42
|
-
##
|
|
43
|
-
|
|
44
|
-
### 👑 **Commander Mode** - *Like screen sharing, but for device control*
|
|
45
|
-
* One device leads, others follow in real-time
|
|
46
|
-
* Touch the leader's screen → all followers mirror the action
|
|
47
|
-
* Switch leaders with a simple dropdown
|
|
48
|
-
* Watch it all happen on your web dashboard
|
|
49
|
-
* **[NEW in v2.3.0]** Enhanced WDA API for iOS device control
|
|
50
|
-
* **[NEW in v2.3.0]** Element caching for faster command execution
|
|
51
|
-
|
|
52
|
-
### 📱 **QR-Based Device Enrollment** - *Connect devices in seconds, not minutes*
|
|
53
|
-
* **[NEW in v2.3.0]** Scan QR code to enroll iOS or Android devices
|
|
54
|
-
* **[NEW in v2.3.0]** Automatic IP address detection and configuration
|
|
55
|
-
* **[NEW in v2.3.0]** Works with both USB and wireless connections
|
|
56
|
-
* iOS: WebDriverAgent auto-detection
|
|
57
|
-
* Android: Automatic ADB setup with privacy controls
|
|
58
|
-
* No more manual IP configuration or device.conf editing
|
|
59
|
-
|
|
60
|
-
### 🔄 **Live Sync** - *What if all your devices moved together?*
|
|
61
|
-
* Commands reach all devices simultaneously via WebSockets
|
|
62
|
-
* iOS and Android understand the same instructions differently (as they should)
|
|
63
|
-
* See live status updates from every connected device
|
|
64
|
-
* No waiting for one device to finish before the next starts
|
|
65
|
-
* **[NEW in v2.3.0]** Improved wireless connectivity with retry logic
|
|
66
|
-
|
|
67
|
-
### 📹 **Record & Replay - Zero-Code Automation**
|
|
68
|
-
* **Visual Recording:** Capture complex workflows with simple button clicks
|
|
69
|
-
* **AI-Generated Scripts:** Voice commands auto-convert to replayable recordings
|
|
70
|
-
* **Fleet Replay:** Run recorded flows on single devices or entire laboratories
|
|
71
|
-
* **Edit & Refine:** Full editor for recordings with add/remove/modify capabilities
|
|
72
|
-
|
|
73
|
-
### 🤖 **AI Integration** - *Talk to your devices like a person*
|
|
74
|
-
* Say *"Open Instagram and find search"* - it just works
|
|
75
|
-
* Choose from 7 AI providers with latest models:
|
|
76
|
-
- **OpenAI:** GPT-4o (Aug 2024)
|
|
77
|
-
- **Google:** Gemini 1.5 Pro/Flash
|
|
78
|
-
- **Anthropic:** Claude 3.5 Sonnet (Oct 2024)
|
|
79
|
-
- **Groq:** Llama 3.3 70B (ultra-fast LPU inference)
|
|
80
|
-
- **Cohere:** Command R+ (enterprise)
|
|
81
|
-
- **Mistral:** Mistral Large 2
|
|
82
|
-
- **GitHub:** Copilot powered
|
|
83
|
-
* **[NEW in v2.3.0]** API-only mode - no local models, optimized for speed
|
|
84
|
-
* Devices understand context - no hunting for element IDs
|
|
85
|
-
* Same command, different results on iOS vs Android (intelligently)
|
|
86
|
-
|
|
87
|
-
### 🎤 **Voice Control** - *Because typing is so 2020*
|
|
88
|
-
* Speak naturally, pause when you're thinking (2-second detection)
|
|
89
|
-
* Chain commands: *"launch settings, scroll up, launch camera"*
|
|
90
|
-
* Watch your words turn into actions in real-time
|
|
91
|
-
* Works across as many devices as you want to connect
|
|
61
|
+
## For QA teams
|
|
92
62
|
|
|
93
|
-
|
|
63
|
+
### Commander Mode — one-to-many control
|
|
94
64
|
|
|
95
|
-
|
|
65
|
+
- **Live mirror of the commander device**, with every follower shown alongside it.
|
|
66
|
+
- **Every gesture is broadcast**: tap, long-press, swipe (speed preserved) and typed text go to all followers at once.
|
|
67
|
+
- **Element-aware, not pixel-blind.** Followers find the element the commander touched — by accessibility ID, label, type and position in the UI tree — so a tap on an iPhone hits the same button on an iPad or a Pixel, even when the layouts differ.
|
|
68
|
+
- **Swap the commander** from a dropdown on the mirror, without restarting the session.
|
|
69
|
+
- **Include or exclude individual followers** with one click.
|
|
70
|
+
- **Fast iOS mirroring** over WebDriverAgent MJPEG: about 170 KB per frame, compared with about 5.8 MB for a screenshot poll. Each iOS device gets its own WDA port, so several iPhones don't collide.
|
|
71
|
+
- **Full-screen mirror** with zoom and fit-to-window, and a **D-pad** for directional scrolling and swiping.
|
|
96
72
|
|
|
97
|
-
###
|
|
98
|
-
* **Type Text to Send:** Rapid text input without typing full commands
|
|
99
|
-
* **Element to Tap:** Click by name or coordinates (x,y) instantly
|
|
100
|
-
* **App ID to Launch:** Launch apps with autocomplete dropdown
|
|
101
|
-
* **History Navigation:** Arrow keys cycle through previous inputs
|
|
73
|
+
### Recordings that survive UI changes
|
|
102
74
|
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
75
|
+
- **Steps are recorded as locators** (`accessibilityId`, `resourceId`, text, type, bounds), not raw coordinates.
|
|
76
|
+
- **Auto-wait**: on replay, each step waits up to 10 s for its element to appear, so animations and slow networks don't break the run.
|
|
77
|
+
- **Coordinate fallback**: if an element can't be found, Devicely taps its scaled position instead and marks the step `fallback`, so you can see what drifted.
|
|
78
|
+
- **Replay on one device or the whole fleet in parallel.**
|
|
79
|
+
- **Manage recordings** in a list or grid view with search and tag filters. A details drawer shows each step's locator, timing and result; the editor lets you add, edit, reorder and delete steps.
|
|
108
80
|
|
|
109
|
-
###
|
|
110
|
-
* **Visual Element Scan:** See all clickable, enabled, visible elements instantly
|
|
111
|
-
* **Point-and-Click Control:** Direct device interaction through inspector
|
|
112
|
-
* **Developer Export:** Clean selectors and coordinates for automation scripts
|
|
81
|
+
### Plain English and voice
|
|
113
82
|
|
|
114
|
-
|
|
115
|
-
* Transform 2-hour manual testing into 5-minute parallel runs
|
|
116
|
-
* Turn your device collection into a web-accessible lab
|
|
117
|
-
* Reproduce bugs instantly across different devices/OS versions
|
|
118
|
-
* Mix iOS and Android devices - they all speak the same language
|
|
83
|
+
Type or say *"launch settings, scroll down, open camera"*. Devicely turns it into the right action for each platform — for example `com.apple.Preferences` on iOS and `com.android.settings` on Android — and runs it on every selected device at once. Voice input waits for a 2-second pause before it runs, so you can speak naturally.
|
|
119
84
|
|
|
120
|
-
|
|
85
|
+
Choose your AI provider in **Settings → AI Config**:
|
|
121
86
|
|
|
122
|
-
|
|
87
|
+
| Provider | Notes |
|
|
88
|
+
| :--- | :--- |
|
|
89
|
+
| **Ollama** | Runs locally. No API key, and no prompt leaves your machine. |
|
|
90
|
+
| **Google Gemini**, **Groq**, **Cohere** | Offer free tiers — the easiest way to start. |
|
|
91
|
+
| **OpenAI**, **Anthropic Claude**, **Mistral AI** | Bring your own API key. |
|
|
92
|
+
| **GitHub Copilot** | Uses your Copilot subscription. |
|
|
123
93
|
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
devicely start
|
|
128
|
-
```
|
|
94
|
+
Each provider lists its current models in the settings screen.
|
|
95
|
+
|
|
96
|
+
### Quick actions, inspector and code export
|
|
129
97
|
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
3. Try saying: *"Take screenshot on all devices"*
|
|
134
|
-
4. Watch what happens
|
|
98
|
+
- **Quick Actions panel**: send text, tap an element by name or `x,y`, and launch apps with autocomplete. Arrow keys cycle through what you typed before.
|
|
99
|
+
- **UI Inspector**: hover over any element to see its accessibility ID, resource ID, XPath and bounds.
|
|
100
|
+
- **One-click export** of a selector as a Devicely command, **WebdriverIO**, **Appium Python** or **Appium Java** code.
|
|
135
101
|
|
|
136
|
-
###
|
|
137
|
-
- Settings → AI Config → Choose a provider (Gemini/Groq are free)
|
|
138
|
-
- Now you can talk to your devices in plain English
|
|
139
|
-
- *"Launch settings, scroll down, open camera"* - just works
|
|
102
|
+
### Devices that stay connected
|
|
140
103
|
|
|
141
|
-
|
|
104
|
+
- **QR enrollment**: scan a code from the Command Center to pair a device over Wi-Fi — no IP addresses to type.
|
|
105
|
+
- **Health monitoring** spots an unplugged cable, a reboot or a driver failure within seconds and cleans up the session.
|
|
106
|
+
- **Auto-reconnect** brings a device back as soon as it reappears on USB or Wi-Fi. It's on by default; set `DEVICELY_AUTO_RECONNECT=0` to turn it off.
|
|
107
|
+
- **Virtual devices**: start and stop Android emulators and iOS simulators from the UI or the CLI, next to your physical devices.
|
|
142
108
|
|
|
143
109
|
---
|
|
144
110
|
|
|
145
|
-
##
|
|
111
|
+
## For AI agents (MCP server)
|
|
112
|
+
|
|
113
|
+
Devicely includes a [Model Context Protocol](https://modelcontextprotocol.io) server, so coding agents can see and control real devices.
|
|
146
114
|
|
|
147
|
-
### Natural Language Commands
|
|
148
115
|
```bash
|
|
149
|
-
#
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
116
|
+
devicely mcp install claude-code # prints the `claude mcp add` command to run
|
|
117
|
+
devicely mcp install cursor # also: vscode, windsurf, gemini, claude-desktop
|
|
118
|
+
devicely mcp doctor # checks the backend, token and transports
|
|
119
|
+
```
|
|
153
120
|
|
|
154
|
-
|
|
155
|
-
"launch settings scroll up launch camera go to youtube.com press home"
|
|
121
|
+
Or add it to Claude Code by hand:
|
|
156
122
|
|
|
157
|
-
|
|
158
|
-
|
|
123
|
+
```bash
|
|
124
|
+
claude mcp add devicely -- npx -y devicely mcp
|
|
159
125
|
```
|
|
160
126
|
|
|
161
|
-
|
|
162
|
-
1. Click 🎤 microphone button
|
|
163
|
-
2. Speak: "launch settings scroll up launch camera"
|
|
164
|
-
3. Pause for 2 seconds
|
|
165
|
-
4. Command executes on all selected devices!
|
|
166
|
-
|
|
167
|
-
### Recording & Replay
|
|
168
|
-
1. Click **Record** button
|
|
169
|
-
2. Perform actions on device
|
|
170
|
-
3. Click **Stop Recording**
|
|
171
|
-
4. Save with name
|
|
172
|
-
5. Replay anytime on any device(s)
|
|
173
|
-
|
|
174
|
-
### Commander Mode
|
|
175
|
-
1. Connect 2+ devices
|
|
176
|
-
2. First device auto-becomes commander
|
|
177
|
-
3. Click on commander screen to broadcast actions
|
|
178
|
-
4. Use top-left dropdown on mirror to swap commander
|
|
179
|
-
5. All devices stay synchronized automatically
|
|
127
|
+
Two transports are available: **stdio** (`devicely mcp`, which starts the backend if it isn't running) and **Streamable HTTP**, protected by a Bearer token. Run `devicely mcp doctor` to see the HTTP endpoint.
|
|
180
128
|
|
|
181
|
-
|
|
129
|
+
### 30 tools
|
|
182
130
|
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
|
188
|
-
|
|
189
|
-
|
|
|
190
|
-
|
|
|
191
|
-
|
|
|
192
|
-
| **🐙 GitHub Copilot** | GPT-4, GPT-3.5 | 🔥🔥 | ❌ | Developers with subscription |
|
|
193
|
-
| **⚡ Groq** | Llama 3 70B, Mixtral, Gemma | 🔥🔥🔥 | ✅ Yes | **Ultra-fast inference** |
|
|
194
|
-
| **🌊 Cohere** | Command R+, Command R | 🔥🔥 | ✅ Yes | Enterprise-grade |
|
|
195
|
-
| **🌬️ Mistral AI** | Large, Medium, Small | 🔥🔥 | ❌ | EU compliance, open-source |
|
|
196
|
-
|
|
197
|
-
### 💡 Recommendations
|
|
198
|
-
- **Free Users:** Start with Gemini 2.0 Flash or Groq Llama 3
|
|
199
|
-
- **Speed Critical:** Use Groq (fastest inference available)
|
|
200
|
-
- **Best Quality:** Claude 3.5 Sonnet or GPT-4 Turbo
|
|
201
|
-
- **Cost Effective:** Gemini Flash or GPT-3.5 Turbo
|
|
202
|
-
|
|
203
|
-
Get API Keys:
|
|
204
|
-
- **OpenAI:** https://platform.openai.com/api-keys
|
|
205
|
-
- **Gemini:** https://makersuite.google.com/app/apikey
|
|
206
|
-
- **Claude:** https://console.anthropic.com/account/keys
|
|
207
|
-
- **Groq:** https://console.groq.com/keys
|
|
208
|
-
- **Cohere:** https://dashboard.cohere.com/api-keys
|
|
209
|
-
- **Mistral:** https://console.mistral.ai/api-keys
|
|
131
|
+
| Group | Tools |
|
|
132
|
+
| :--- | :--- |
|
|
133
|
+
| **Observe** | `devicely_snapshot`, `devicely_find_elements`, `devicely_wait_for`, `devicely_get_device_logs`, `devicely_get_action_log` |
|
|
134
|
+
| **Act** | `devicely_tap`, `devicely_long_press`, `devicely_swipe`, `devicely_type_text`, `devicely_press_key`, `devicely_open_url` |
|
|
135
|
+
| **Apps** | `devicely_launch_app`, `devicely_close_app`, `devicely_list_apps`, `devicely_install_app`, `devicely_uninstall_app` |
|
|
136
|
+
| **Devices** | `devicely_list_devices`, `devicely_connect_device`, `devicely_disconnect_device`, `devicely_restart_device`, `devicely_set_passcode`, `devicely_doctor` |
|
|
137
|
+
| **Fleet and flows** | `devicely_commander`, `devicely_run_commands`, `devicely_run_natural_language` |
|
|
138
|
+
| **Virtual devices** | `devicely_launch_virtual_device`, `devicely_stop_virtual_device` |
|
|
139
|
+
| **Recordings** | `devicely_save_recording`, `devicely_list_recordings`, `devicely_replay_recording` |
|
|
210
140
|
|
|
211
|
-
|
|
141
|
+
**Prompts** for common jobs: `smoke_test`, `reproduce_bug` and `write_test`.
|
|
142
|
+
**Resources** with live context: `devicely://devices`, `devicely://apps`, `devicely://docs/command-language` and `devicely://docs/agent-guide`.
|
|
212
143
|
|
|
213
|
-
|
|
144
|
+
### Safety controls
|
|
214
145
|
|
|
215
|
-
|
|
146
|
+
Set these in **Settings → AI Agents** (stored in `~/.devicely/mcp.json`):
|
|
216
147
|
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
221
|
-
|
|
222
|
-
|
|
148
|
+
- **Destructive tools are off by default**: `install_app`, `uninstall_app`, `restart_device`, `set_passcode` and `stop_virtual_device`. When enabled, they can also require confirmation.
|
|
149
|
+
- **Limit agents to specific devices** (`allowedDevices`), turn off individual tools, and redact sensitive values from output.
|
|
150
|
+
- **Every agent action is logged** to an audit trail.
|
|
151
|
+
|
|
152
|
+
### REST API
|
|
153
|
+
|
|
154
|
+
The same capabilities are available over HTTP at `/api/v1`, using the same Bearer token — useful for CI pipelines and custom harnesses. View or rotate the token with `devicely mcp token [--rotate]`; it's saved in `~/.devicely/mcp-token`.
|
|
155
|
+
|
|
156
|
+
### Example
|
|
157
|
+
|
|
158
|
+
> *"Take a snapshot of the Pixel, check that the Login button is visible, tap it, and confirm the welcome screen appears."*
|
|
159
|
+
|
|
160
|
+
The agent calls `devicely_snapshot` and `devicely_find_elements`, taps with `devicely_tap`, and checks the result with `devicely_wait_for`.
|
|
223
161
|
|
|
224
162
|
---
|
|
225
163
|
|
|
226
|
-
##
|
|
227
|
-
|
|
228
|
-
|
|
229
|
-
|
|
230
|
-
|
|
231
|
-
-
|
|
232
|
-
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
- **Multi-Platform Intelligence:** Single AI command generates platform-specific actions for iOS and Android
|
|
254
|
-
- **Simultaneous Execution:** All devices execute at the same time, not sequentially
|
|
255
|
-
- **Error Visibility:** AI errors (quota limits, auth failures) shown inline with results
|
|
256
|
-
- **Response Cleaning:** Automatically removes markdown and explanations from AI output
|
|
257
|
-
- **Cross-Platform Apps:** AI generates both iOS and Android app bundles (Settings, Camera, Chrome, etc.)
|
|
258
|
-
|
|
259
|
-
### 🎤 Improved Voice Input
|
|
260
|
-
- **Extended Listening:** 2-second silence detection for natural speech
|
|
261
|
-
- **Continuous Speech:** Speak multiple phrases with natural pauses
|
|
262
|
-
- **Live Transcription:** See your words accumulate in real-time
|
|
263
|
-
- **Complex Commands:** "launch settings scroll up launch camera go to google.com press home" works flawlessly
|
|
264
|
-
- **No Duplicates:** Fixed transcript accumulation for clean output
|
|
265
|
-
|
|
266
|
-
### 🔐 Settings Enhancements
|
|
267
|
-
- **Revamped AI Config Tab:** Professional card-based design with dark mode support
|
|
268
|
-
- **Dynamic API Key Input:** Field changes based on selected provider
|
|
269
|
-
- **Show/Hide Toggles:** Individual toggles for all 7 provider keys
|
|
270
|
-
- **Status Display:** Green success box showing current provider, model, and key preview
|
|
271
|
-
- **Quick Tips Section:** In-UI recommendations for choosing providers
|
|
272
|
-
- **Close Button:** Easy navigation back to main screen
|
|
273
|
-
- **Dark Mode:** Proper color schemes for light and dark themes
|
|
274
|
-
|
|
275
|
-
### 📝 Recording Improvements
|
|
276
|
-
- **AI Recording Editor:** Edit AI-generated recordings like manual ones
|
|
277
|
-
- **Format Conversion:** Seamlessly handles both string and array command formats
|
|
278
|
-
- **Save & Replay:** Edit, save, and replay AI-generated commands
|
|
279
|
-
- **Status Persistence:** Recording status preserved when navigating between screens
|
|
164
|
+
## CLI reference
|
|
165
|
+
|
|
166
|
+
| Command | What it does |
|
|
167
|
+
| :--- | :--- |
|
|
168
|
+
| `devicely start` | Start the server and open the Command Center |
|
|
169
|
+
| `devicely start -p <port> --no-browser` | Use a custom port without opening a browser (CI, headless machines) |
|
|
170
|
+
| `devicely start --usb-only` / `--wifi-only` | Only detect devices on USB, or only on Wi-Fi |
|
|
171
|
+
| `devicely start --launch <name>` | Boot a virtual device when the server starts |
|
|
172
|
+
| `devicely start --debug` | Turn on debug logging |
|
|
173
|
+
| `devicely list [--verbose]` | List connected iOS and Android devices |
|
|
174
|
+
| `devicely exec "<command>" [-d <name>]` | Run a command on all devices, or on one with `-d` |
|
|
175
|
+
| `devicely doctor` | Check Node, adb, Xcode and driver setup |
|
|
176
|
+
| `devicely emulators list` | List Android emulators and iOS simulators (alias: `devicely virtual`) |
|
|
177
|
+
| `devicely emulators start <name> [--headless]` | Boot a virtual device |
|
|
178
|
+
| `devicely emulators stop <name>` | Shut down a virtual device |
|
|
179
|
+
| `devicely mcp [--url <url>] [--no-autostart]` | Run the MCP stdio server |
|
|
180
|
+
| `devicely mcp doctor` | Check MCP readiness |
|
|
181
|
+
| `devicely mcp token [--rotate]` | Show or rotate the agent token |
|
|
182
|
+
| `devicely mcp install <client>` | Configure `claude-code`, `claude-desktop`, `cursor`, `vscode`, `windsurf` or `gemini` |
|
|
183
|
+
|
|
184
|
+
Example commands for `exec`:
|
|
185
|
+
|
|
186
|
+
```bash
|
|
187
|
+
devicely exec "launch settings"
|
|
188
|
+
devicely exec "launch settings scroll down launch camera"
|
|
189
|
+
devicely exec "open url https://github.com" -d "Pixel 8"
|
|
190
|
+
```
|
|
280
191
|
|
|
281
192
|
---
|
|
282
193
|
|
|
283
|
-
##
|
|
194
|
+
## Supported platforms
|
|
284
195
|
|
|
285
|
-
|
|
286
|
-
|
|
196
|
+
| | Supported |
|
|
197
|
+
| :--- | :--- |
|
|
198
|
+
| **Host machine** | macOS and Linux, Node.js 18+. Windows isn't supported yet. |
|
|
199
|
+
| **Android** | Phones, tablets and emulators via ADB and UIAutomator2. Wireless debugging needs Android 11+. |
|
|
200
|
+
| **iOS / iPadOS** | iPhones, iPads and simulators via WebDriverAgent. Needs a Mac with Xcode. iOS 16+ also requires Developer Mode. |
|
|
287
201
|
|
|
288
|
-
|
|
289
|
-
- All API keys stored locally on your machine
|
|
290
|
-
- No telemetry or data collection
|
|
291
|
-
- Commands executed directly on your devices
|
|
292
|
-
- No cloud dependencies for core functionality
|
|
202
|
+
### iOS setup
|
|
293
203
|
|
|
294
|
-
|
|
204
|
+
Physical iOS devices are controlled through WebDriverAgent, which must be signed with your Apple ID or developer team before it can run on a device. Run `devicely doctor` and follow the in-app setup guide; it walks you through installing the iOS tools, enabling Developer Mode and trusting the computer. Android devices and iOS simulators don't need signing.
|
|
295
205
|
|
|
296
206
|
---
|
|
297
207
|
|
|
298
|
-
|
|
208
|
+
## Privacy and security
|
|
299
209
|
|
|
300
|
-
|
|
210
|
+
- **Runs locally.** The Devicely server runs on your machine and talks to your devices directly. Your apps, screenshots and test data aren't uploaded anywhere.
|
|
211
|
+
- **Local AI is an option.** Use Ollama to keep prompts and screen data on your machine. With a cloud AI provider, your commands go to that provider under your own API key.
|
|
212
|
+
- **API keys stay on your machine.**
|
|
213
|
+
- **Agents need a token.** MCP over HTTP and the REST API require a 32-byte Bearer token.
|
|
214
|
+
- **Analytics:** the local server sends no telemetry. The hosted dashboard at [devicely-ai.vercel.app](https://devicely-ai.vercel.app) uses anonymous Vercel page analytics; it connects to your local server and doesn't send device data to Vercel.
|
|
301
215
|
|
|
302
216
|
---
|
|
303
217
|
|
|
304
|
-
##
|
|
218
|
+
## What's new in 2.3.4
|
|
305
219
|
|
|
306
|
-
|
|
220
|
+
- **Recordings, rebuilt**: locator-based steps, 10 s auto-wait, coordinate fallback, list and grid views, a details drawer and a step editor.
|
|
221
|
+
- **MCP server and REST API**: 30 tools, prompts and resources, stdio and Streamable HTTP transports, safety policy and an audit log, plus one-command setup for six AI clients.
|
|
222
|
+
- **Faster iOS mirroring** with WebDriverAgent MJPEG streaming and a dedicated WDA port per device.
|
|
223
|
+
- **Device health monitoring and auto-reconnect.**
|
|
224
|
+
- **Virtual devices**: Android emulators and iOS simulators from the UI, CLI and MCP.
|
|
225
|
+
- **Passcode tool**: sets a PIN on Android over ADB and gives guided steps on iOS.
|
|
307
226
|
|
|
308
|
-
|
|
227
|
+
---
|
|
309
228
|
|
|
310
|
-
|
|
311
|
-
npm install -g devicely
|
|
312
|
-
devicely start
|
|
313
|
-
```
|
|
229
|
+
## License
|
|
314
230
|
|
|
315
|
-
|
|
231
|
+
**Free to use. Closed source.** Devicely is free to install and use. The source code is proprietary: you may not copy, modify or redistribute it. See [LICENSE](https://github.com/sureshkumarm8/Devicely/blob/main/LICENSE).
|
|
316
232
|
|
|
317
|
-
|
|
233
|
+
- **Questions and enterprise inquiries:** [devicelyai@gmail.com](mailto:devicelyai@gmail.com)
|
|
234
|
+
- **Report an issue:** [github.com/sureshkumarm8/Devicely/issues](https://github.com/sureshkumarm8/Devicely/issues)
|
|
235
|
+
|
|
236
|
+
<div align="center">
|
|
318
237
|
|
|
319
|
-
|
|
238
|
+
**If Devicely saves you time, [star it on GitHub](https://github.com/sureshkumarm8/Devicely).**
|
|
320
239
|
|
|
321
240
|
</div>
|