ft-scout 9.0.4 → 9.0.6
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.firebase/hosting.d2Vi.cache +8 -7
- package/AUTONOMOUS_FEATURES.md +192 -0
- package/README.md +67 -35
- package/bin/src/commands/agent.d.ts.map +1 -1
- package/bin/src/commands/agent.js +21 -8
- package/bin/src/commands/agent.js.map +1 -1
- package/bin/src/commands/login.d.ts.map +1 -1
- package/bin/src/commands/login.js +3 -1
- package/bin/src/commands/login.js.map +1 -1
- package/bin/src/engine/actionVerifier.d.ts.map +1 -1
- package/bin/src/engine/actionVerifier.js +9 -1
- package/bin/src/engine/actionVerifier.js.map +1 -1
- package/bin/src/engine/agentEngine.d.ts.map +1 -1
- package/bin/src/engine/agentEngine.js +319 -13
- package/bin/src/engine/agentEngine.js.map +1 -1
- package/bin/src/engine/browser/browserController.d.ts.map +1 -1
- package/bin/src/engine/browser/browserController.js +53 -13
- package/bin/src/engine/browser/browserController.js.map +1 -1
- package/bin/src/engine/computer/computerController.d.ts.map +1 -1
- package/bin/src/engine/computer/computerController.js +564 -296
- package/bin/src/engine/computer/computerController.js.map +1 -1
- package/bin/src/engine/computer/computerDiagnostic.d.ts.map +1 -0
- package/bin/src/engine/computer/computerDiagnostic.js +71 -0
- package/bin/src/engine/computer/computerDiagnostic.js.map +1 -0
- package/bin/src/engine/computer/index.d.ts.map +1 -1
- package/bin/src/engine/computer/index.js +1 -0
- package/bin/src/engine/computer/index.js.map +1 -1
- package/bin/src/engine/computer/mockComputer.d.ts.map +1 -1
- package/bin/src/engine/computer/mockComputer.js +104 -8
- package/bin/src/engine/computer/mockComputer.js.map +1 -1
- package/bin/src/engine/computer/scoutDriver.cs +1161 -0
- package/bin/src/engine/computer/scout_driver.exe +0 -0
- package/bin/src/engine/runtime/agentRuntime.d.ts.map +1 -1
- package/bin/src/engine/runtime/agentRuntime.js +375 -20
- package/bin/src/engine/runtime/agentRuntime.js.map +1 -1
- package/bin/src/engine/safety.d.ts.map +1 -1
- package/bin/src/engine/safety.js +17 -6
- package/bin/src/engine/safety.js.map +1 -1
- package/bin/src/engine/toolRegistry.d.ts.map +1 -1
- package/bin/src/engine/toolRegistry.js +209 -0
- package/bin/src/engine/toolRegistry.js.map +1 -1
- package/bin/src/engine/workspace/agentWorkspace.d.ts.map +1 -1
- package/bin/src/engine/workspace/agentWorkspace.js +21 -0
- package/bin/src/engine/workspace/agentWorkspace.js.map +1 -1
- package/bin/src/index.js +5 -4
- package/bin/src/index.js.map +1 -1
- package/bin/src/utils/auth.d.ts.map +1 -1
- package/bin/src/utils/auth.js +14 -1
- package/bin/src/utils/auth.js.map +1 -1
- package/bin/src/utils/branding.js +1 -1
- package/bin/src/utils/holoseneAuth.d.ts.map +1 -1
- package/bin/src/utils/holoseneAuth.js +169 -41
- package/bin/src/utils/holoseneAuth.js.map +1 -1
- package/firebase.json +8 -0
- package/package.json +1 -1
- package/web/app.js +3 -3
- package/web/styles.css +2 -2
|
@@ -1,8 +1,9 @@
|
|
|
1
|
+
web_cli.html,1789281384143,d251153dac472e4e4ea21d00f881b575e81a63822da13006e720c376cfbe24af
|
|
2
|
+
styles.css,1789281384145,f1c7c32071cbb851d4f0a3023375765c3a5630106878c5fcad8456adb98e138a
|
|
1
3
|
ScoutFavicon.png,1786186569805,41a9ca9abf7acd2976e78c72a72285b49cc0e64b2b5eeb2c2b88c69628fc618a
|
|
2
|
-
paid_plans.html,
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
app.js,1789193833797,a5dc6defb03d9db5398c62ef23a0e6b06dc635bda9c13851861714b08c102a07
|
|
4
|
+
paid_plans.html,1789283167535,0708329edc4938577250c92fb1a294d09ea38778b8edec990b0b80adc839e49f
|
|
5
|
+
index.html,1789282851111,f7812b28faad4cc0ebf3062272dfadc297ab257d3d73ff254b69780e5401934a
|
|
6
|
+
authentication.html,1789281384176,678dbfb5438ff13b0a6f8d759c1af77d5aac926ff44ea7815f181e80674d183a
|
|
7
|
+
app.js,1789284464838,bfddc8f457e7a68a6ff158425cd5b23132e8e15700c7ecd22850fbe0455ff7c0
|
|
8
|
+
admin.html,1789281384176,72bb3b4ab7dbbe6ff32bfb5a51e8c79ef0bece4e2f1fc91218296df17e9b34f4
|
|
9
|
+
oauth.html,1789285036283,f53de5de706d1ce49d2af18e07e1fce39f01fe6a5c65ce525ba678809a1c5bfb
|
|
@@ -0,0 +1,192 @@
|
|
|
1
|
+
# 🚀 Scout Autonomous Agent — Features & Architecture Guide
|
|
2
|
+
|
|
3
|
+
Scout is an advanced, multi-modal autonomous agent platform built directly into FT-CLI. It bridges traditional developer terminal workflows with modern web automation and full operating-system desktop control (GUI).
|
|
4
|
+
|
|
5
|
+
---
|
|
6
|
+
|
|
7
|
+
## 📑 Table of Contents
|
|
8
|
+
|
|
9
|
+
1. [Architectural Overview](#architectural-overview)
|
|
10
|
+
2. [Desktop & Computer-Use (GUI Control)](#1-desktop--computer-use-gui-control)
|
|
11
|
+
3. [Browser Automation & Web Scraping](#2-browser-automation--web-scraping)
|
|
12
|
+
4. [Multi-Step Planning & Cognitive Scratchpad](#3-multi-step-planning--cognitive-scratchpad)
|
|
13
|
+
5. [Action Verification & Self-Healing Engine](#4-action-verification--self-healing-engine)
|
|
14
|
+
6. [Safety & Human-in-the-Loop Risk Classifier](#5-safety--human-in-the-loop-risk-classifier)
|
|
15
|
+
7. [Developer & Workspace Baseline Operations](#6-developer--workspace-baseline-operations)
|
|
16
|
+
8. [Subscription Tier Capabilities (Free vs Pro / Enterprise)](#subscription-tier-capabilities)
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Architectural Overview
|
|
21
|
+
|
|
22
|
+
Scout executes a continuous **Perceive ➔ Plan ➔ Act ➔ Verify ➔ Recover** operational loop:
|
|
23
|
+
|
|
24
|
+
```mermaid
|
|
25
|
+
flowchart TD
|
|
26
|
+
Goal[User High-Level Objective] --> Planner[TaskPlanner & DAG Decomposition]
|
|
27
|
+
Planner --> Scratchpad[Cognitive Scratchpad & State Checkpoint]
|
|
28
|
+
Scratchpad --> SelectTool[Tool Selection Hierarchy]
|
|
29
|
+
SelectTool --> RiskCheck{RiskClassifier & Safety Gate}
|
|
30
|
+
RiskCheck -- High/Critical Risk --> Approval[ApprovalManager: Human Confirmation]
|
|
31
|
+
Approval -- Approved --> Dispatcher[Agent Runtime Dispatcher]
|
|
32
|
+
RiskCheck -- Low/Medium Risk --> Dispatcher
|
|
33
|
+
Dispatcher --> Execute[Execution: Native Computer / Browser / CLI Tools]
|
|
34
|
+
Execute --> Verify[ActionVerifier: Post-Execution Verification]
|
|
35
|
+
Verify -- Success --> Checkpoint[Save Task State & Next Step]
|
|
36
|
+
Verify -- Failure --> Recovery[Error Recovery Strategy: Retry / Alternative]
|
|
37
|
+
Recovery --> Dispatcher
|
|
38
|
+
Checkpoint --> Done{All Steps Complete?}
|
|
39
|
+
Done -- No --> Scratchpad
|
|
40
|
+
Done -- Yes --> Complete[Task Complete Notification]
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## 1. 🖥️ Desktop & Computer-Use (GUI Control)
|
|
46
|
+
|
|
47
|
+
Scout provides deep operating system control powered by the high-performance native `scout_driver.exe` engine (Win32 API interop) and Node.js controller integration (`SystemComputerController`).
|
|
48
|
+
|
|
49
|
+
### Core Features
|
|
50
|
+
|
|
51
|
+
- **Multi-Monitor Screen Perception:**
|
|
52
|
+
- `computer_screenshot` / `screenshot`: Captures high-resolution screen frames to perceive visible windows, buttons, and visual state.
|
|
53
|
+
- **Precision Mouse Automation:**
|
|
54
|
+
- `computer_click`: Sends left, right, or middle mouse clicks to exact screen coordinates `(x, y)`.
|
|
55
|
+
- `computer_double_click`: Opens files, selects words, or triggers desktop shortcuts via double-click.
|
|
56
|
+
- `computer_mouse_move`: Translates the cursor smoothly or instantly across display bounds.
|
|
57
|
+
- `computer_mouse_down` & `computer_mouse_up`: Granular mouse button holds and releases for complex interactions.
|
|
58
|
+
- `computer_drag`: Drags items from `(fromX, fromY)` to destination coordinates `(toX, toY)`.
|
|
59
|
+
- `computer_scroll`: Dispatches vertical and horizontal mouse-wheel scroll deltas.
|
|
60
|
+
- **Keyboard & Shortcut Dispatch:**
|
|
61
|
+
- `computer_type`: Emits text strings directly into focused input controls or editor windows.
|
|
62
|
+
- `computer_key_press`: Presses single functional keys (Enter, Escape, Tab, Backspace, Arrow keys, etc.).
|
|
63
|
+
- `computer_hotkey`: Emits multi-key chords (e.g. `["Control", "s"]` to save, `["Alt", "F4"]` to close, `["Alt", "Tab"]` to switch tasks).
|
|
64
|
+
- **Application & Window Orchestration:**
|
|
65
|
+
- `computer_open_app`: Launches desktop applications (VS Code, Notepad, File Explorer, Terminal, etc.).
|
|
66
|
+
- `computer_close_app`: Gracefully terminates or closes targeted desktop processes and windows.
|
|
67
|
+
- `computer_switch_window`: Brings specific applications into foreground focus by matching title or process.
|
|
68
|
+
- `computer_inspect_ui`: Enumerates active open desktop windows and highlights the current foreground window.
|
|
69
|
+
- `computer_wait`: Pauses execution (100ms - 30,000ms) to allow GUI windows, animations, and transitions to settle.
|
|
70
|
+
- `computer_diagnostic`: Collects display resolution, DPI scaling metrics, and driver connection status.
|
|
71
|
+
|
|
72
|
+
---
|
|
73
|
+
|
|
74
|
+
## 2. 🌐 Browser Automation & Web Scraping
|
|
75
|
+
|
|
76
|
+
Implemented in [`browserController.ts`](file:///c:/Users/rishi/FT-CLI/bin/src/engine/browser/browserController.ts), Scout includes an intelligent browser automation controller capable of navigating websites, analyzing semantic DOM trees, and extracting data.
|
|
77
|
+
|
|
78
|
+
### Core Features
|
|
79
|
+
|
|
80
|
+
- **Page Navigation & State:**
|
|
81
|
+
- `browser_navigate`: Loads HTTP/HTTPS URLs, handles navigation history, and constructs interactive DOM trees.
|
|
82
|
+
- `browser_inspect`: Discovers interactive elements (links, buttons, form fields) and assigns lightweight reference IDs (e.g., `[ref=1]`).
|
|
83
|
+
- **Web Form Interaction:**
|
|
84
|
+
- `browser_click`: Clicks web elements identified by CSS selector, visible label text, or ref ID.
|
|
85
|
+
- `browser_type`: Enters query strings or credentials into active form fields and textareas.
|
|
86
|
+
- `browser_select`: Selects options from dropdown menus (`<select>`) by visible text or value.
|
|
87
|
+
- `browser_scroll`: Scrolls viewport up or down to trigger infinite scroll or lazy-loaded assets.
|
|
88
|
+
- **Data Scraping & File Downloads:**
|
|
89
|
+
- `browser_content`: Strips out HTML markup and extracts clean structured text, tables, and article content.
|
|
90
|
+
- `browser_screenshot`: Captures a visual snapshot of the current web page viewport.
|
|
91
|
+
- `browser_download`: Automatically downloads invoices, reports, PDFs, or files from web links and saves them directly to the workspace `downloads/` directory.
|
|
92
|
+
|
|
93
|
+
#### Example Implementation (`browserController.ts`):
|
|
94
|
+
```typescript
|
|
95
|
+
// Content Extraction & Document Download
|
|
96
|
+
public async extract_content(selector?: string): Promise<BrowserActionResult> {
|
|
97
|
+
const page = this.mockPages.get(this.currentUrl);
|
|
98
|
+
const text = page
|
|
99
|
+
? page.html.replace(/<[^>]+>/g, ' ').replace(/\s+/g, ' ').trim()
|
|
100
|
+
: `Content of ${this.currentUrl}`;
|
|
101
|
+
return { success: true, output: text, url: this.currentUrl, title: this.currentTitle };
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
public async download_file(urlOrSelector: string, destPath?: string): Promise<BrowserActionResult> {
|
|
105
|
+
// Discovers links, downloads files, and writes to workspace disk
|
|
106
|
+
const fileName = path.basename(targetUrl).includes('.') ? path.basename(targetUrl) : 'downloaded_invoice.pdf';
|
|
107
|
+
const finalPath = destPath ? path.resolve(this.workspaceDir, destPath) : path.resolve(this.workspaceDir, 'downloads', fileName);
|
|
108
|
+
fs.mkdirSync(path.dirname(finalPath), { recursive: true });
|
|
109
|
+
// Stream data and persist to disk...
|
|
110
|
+
}
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## 3. 🧠 Multi-Step Planning & Cognitive Scratchpad
|
|
116
|
+
|
|
117
|
+
Complex tasks require structured reasoning, plan tracking, and persistent state across agent steps.
|
|
118
|
+
|
|
119
|
+
- **Dynamic Task Planning (`TaskPlanner`):**
|
|
120
|
+
- Parses open-ended goals into structured steps with explicit dependencies.
|
|
121
|
+
- Dynamically adapts when intermediate steps succeed, fail, or reveal new context.
|
|
122
|
+
- **Cognitive Scratchpad Memory (`scratchpad`):**
|
|
123
|
+
- Exclusive to **Pro & Enterprise** tiers.
|
|
124
|
+
- Provides working memory via `write`, `read`, `append`, and `clear` operations.
|
|
125
|
+
- Persisted locally to `.ft/scratchpad.md` to prevent LLM context overflow during long executions.
|
|
126
|
+
- **Task State Checkpoints (`TaskStateManager`):**
|
|
127
|
+
- Saves full task execution histories and artifact logs to `.ft/tasks/<task_id>.json`.
|
|
128
|
+
- Enables clean task resumption, history reviews, and rollback capabilities.
|
|
129
|
+
|
|
130
|
+
---
|
|
131
|
+
|
|
132
|
+
## 4. 🔍 Action Verification & Self-Healing Engine
|
|
133
|
+
|
|
134
|
+
Scout doesn't simply fire tools and assume success. Every action is verified through [`actionVerifier.ts`](file:///c:/Users/rishi/FT-CLI/bin/src/engine/actionVerifier.ts).
|
|
135
|
+
|
|
136
|
+
### Verification Types
|
|
137
|
+
1. `file_exists`: Verifies target file is present on the filesystem and contains non-empty contents.
|
|
138
|
+
2. `app_active`: Verifies that an opened application is running and holds foreground focus.
|
|
139
|
+
3. `url_matches`: Verifies the browser navigated to the expected target domain/path.
|
|
140
|
+
4. `element_exists`: Confirms targeted DOM elements rendered on the page.
|
|
141
|
+
|
|
142
|
+
### Autonomous Recovery Strategy
|
|
143
|
+
When verification fails:
|
|
144
|
+
- **`RETRY`**: Re-attempts the action if caused by transient loading latency.
|
|
145
|
+
- **`ALTERNATIVE`**: Selects a secondary tool or selector if the primary approach is blocked.
|
|
146
|
+
- **`ABORT`**: Gracefully yields to the user if irreversible roadblocks occur.
|
|
147
|
+
|
|
148
|
+
### Intent Preservation
|
|
149
|
+
If a user requests GUI action (e.g. *"Open Notepad and type my notes"*), Scout strictly honors the intent and will **not** cheat by performing silent background file writes unless explicitly permitted.
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## 5. 🛡️ Safety & Human-in-the-Loop Risk Classifier
|
|
154
|
+
|
|
155
|
+
Implemented in [`safety.ts`](file:///c:/Users/rishi/FT-CLI/bin/src/engine/safety.ts), Scout protects user systems from accidental disruption or destructive operations.
|
|
156
|
+
|
|
157
|
+
| Risk Level | Triggering Tools / Patterns | Behavior |
|
|
158
|
+
| :--- | :--- | :--- |
|
|
159
|
+
| **LOW** | `read_file`, `browser_inspect`, `computer_screenshot`, `file_info` | Auto-executed silently. |
|
|
160
|
+
| **MEDIUM** | `write_file`, `edit_file`, `browser_type`, `browser_click` | Executed with standard logging. |
|
|
161
|
+
| **HIGH** | `computer_close_app`, destructive hotkeys (`Alt+F4`), process termination | **Execution pauses.** Modal/CLI prompt requires user approval. |
|
|
162
|
+
| **CRITICAL** | System-wide deletion (`rmdir /s /q`, formatting), registry edits | **Blocked / Strict explicit consent required.** |
|
|
163
|
+
|
|
164
|
+
---
|
|
165
|
+
|
|
166
|
+
## 6. 💼 Developer & Workspace Baseline Operations
|
|
167
|
+
|
|
168
|
+
In addition to autonomous GUI and web tools, Scout provides a full suite of developer tools:
|
|
169
|
+
|
|
170
|
+
- **File Operations:** `read_file`, `write_file`, `edit_file`, `multi_edit_file`, `file_info`.
|
|
171
|
+
- **Search & Inspection:** `grep_search`, `glob_search`, `tree_view`, `list_dir`.
|
|
172
|
+
- **Command & Version Control:** `run_command`, `git_diff`.
|
|
173
|
+
- **Interaction:** `ask_user`, `task_completed`.
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## Subscription Tier Capabilities
|
|
178
|
+
|
|
179
|
+
| Feature Capability | Free Tier | Pro Plan | Enterprise Plan |
|
|
180
|
+
| :--- | :---: | :---: | :---: |
|
|
181
|
+
| **Code Reading & File Editing** | ✅ Full | ✅ Full | ✅ Full |
|
|
182
|
+
| **File / Directory Creation** | ✅ Full | ✅ Full | ✅ Full |
|
|
183
|
+
| **Terminal & Build Commands** | ✅ Full | ✅ Full | ✅ Full |
|
|
184
|
+
| **AI Cognitive Scratchpad** | ❌ Blocked | ✅ Full | ✅ Full |
|
|
185
|
+
| **Autonomous Browser Automation** | ❌ Blocked | ✅ Full | ✅ Full |
|
|
186
|
+
| **Autonomous Computer & GUI Control** | ❌ Blocked | ✅ Full | ✅ Full |
|
|
187
|
+
| **Multi-App System Automation** | ❌ Blocked | ✅ Full | ✅ Full |
|
|
188
|
+
| **Dedicated Concurrency & Support** | Community | Priority | 24/7 Dedicated |
|
|
189
|
+
|
|
190
|
+
---
|
|
191
|
+
|
|
192
|
+
*Authored by Front Terrain Engineering. For upgrades or enterprise deployment, visit [https://scout.frontterrain.com](https://scout.frontterrain.com).*
|
package/README.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# scout (FrontTerrain CLI v9.0.
|
|
1
|
+
# scout (FrontTerrain CLI v9.0.6)
|
|
2
2
|
|
|
3
3
|
**FrontTerrain (`scout`)** — The Autonomous AI Principal Software Engineer & Codebase Co-Pilot in your terminal.
|
|
4
4
|
|
|
@@ -228,37 +228,67 @@ Scout is built as a high-agency autonomous operator capable of operating externa
|
|
|
228
228
|
|
|
229
229
|
## Scout Autonomous Tool Registry
|
|
230
230
|
|
|
231
|
-
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
|
235
|
-
|
|
236
|
-
| `
|
|
237
|
-
| `
|
|
238
|
-
| `
|
|
239
|
-
| `
|
|
240
|
-
| `
|
|
241
|
-
| `
|
|
242
|
-
| `
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
|
246
|
-
|
|
247
|
-
| `
|
|
248
|
-
| `
|
|
249
|
-
| `
|
|
250
|
-
| `
|
|
251
|
-
| `
|
|
252
|
-
| `
|
|
253
|
-
| `
|
|
254
|
-
| `
|
|
255
|
-
|
|
256
|
-
|
|
257
|
-
|
|
|
258
|
-
|
|
259
|
-
| `
|
|
260
|
-
| `
|
|
261
|
-
| `
|
|
231
|
+
> 📘 **Full Architecture & Specifications**: For complete deep-dive documentation, flowcharts, and schemas, see [AUTONOMOUS_FEATURES.md](./AUTONOMOUS_FEATURES.md).
|
|
232
|
+
|
|
233
|
+
### Desktop & Computer-Use Control Tools
|
|
234
|
+
| Tool | Description | Tier Availability |
|
|
235
|
+
|---|---|:---:|
|
|
236
|
+
| `computer` | Unified multi-action desktop dispatcher (click, drag, type, hotkey, open, etc.) | Pro / Enterprise |
|
|
237
|
+
| `computer_screenshot` | Capture display screen image and inspect active foreground window | Pro / Enterprise |
|
|
238
|
+
| `computer_click` | Move cursor and click mouse at coordinate (left, right, middle) | Pro / Enterprise |
|
|
239
|
+
| `computer_double_click` | Double-click mouse at specified screen coordinate | Pro / Enterprise |
|
|
240
|
+
| `computer_right_click` | Right-click mouse at screen coordinate to open context menus | Pro / Enterprise |
|
|
241
|
+
| `computer_mouse_move` | Move mouse cursor to specific display (x, y) coordinates | Pro / Enterprise |
|
|
242
|
+
| `computer_mouse_down` | Hold down mouse button for drag operations | Pro / Enterprise |
|
|
243
|
+
| `computer_mouse_up` | Release mouse button | Pro / Enterprise |
|
|
244
|
+
| `computer_drag` | Drag mouse cursor from `(fromX, fromY)` to `(toX, toY)` | Pro / Enterprise |
|
|
245
|
+
| `computer_scroll` | Scroll vertical or horizontal mouse wheel deltas | Pro / Enterprise |
|
|
246
|
+
| `computer_type` | Ensure target window focus and type text keystrokes via OS keyboard | Pro / Enterprise |
|
|
247
|
+
| `computer_key_press` | Press individual keyboard key (`enter`, `tab`, `escape`, `backspace`) | Pro / Enterprise |
|
|
248
|
+
| `computer_hotkey` | Trigger key combination (`["ctrl", "s"]`, `["alt", "f4"]`, `["alt", "tab"]`) | Pro / Enterprise |
|
|
249
|
+
| `computer_open_app` | Launch desktop application (Notepad, Excel, Chrome, Terminal) and focus window | Pro / Enterprise |
|
|
250
|
+
| `computer_close_app` | Gracefully close application or window by title/executable | Pro / Enterprise |
|
|
251
|
+
| `computer_switch_window` | Switch foreground active window to target application | Pro / Enterprise |
|
|
252
|
+
| `computer_inspect_ui` | Inspect active window coordinates, title, and UI state | Pro / Enterprise |
|
|
253
|
+
| `computer_wait` | Pause execution to allow desktop applications or animations to settle | Pro / Enterprise |
|
|
254
|
+
| `computer_diagnostic` | Retrieve system display resolution, DPI scaling, and Scout driver health | Pro / Enterprise |
|
|
255
|
+
|
|
256
|
+
### Browser Automation & Web Scraping Tools
|
|
257
|
+
| Tool | Description | Tier Availability |
|
|
258
|
+
|---|---|:---:|
|
|
259
|
+
| `browser_navigate` | Open URL, handle navigation history, and construct semantic DOM tree | Pro / Enterprise |
|
|
260
|
+
| `browser_inspect` | Inspect clickable buttons, links, inputs, and assign element ref IDs (`[ref=1]`) | Pro / Enterprise |
|
|
261
|
+
| `browser_click` | Click element by CSS selector, visible label text, or ref ID | Pro / Enterprise |
|
|
262
|
+
| `browser_type` | Type text strings into active form fields and search inputs | Pro / Enterprise |
|
|
263
|
+
| `browser_select` | Choose option from dropdown `<select>` menus by value or visible text | Pro / Enterprise |
|
|
264
|
+
| `browser_scroll` | Scroll webpage up or down to load lazy-loaded elements | Pro / Enterprise |
|
|
265
|
+
| `browser_content` | Extract clean, structured visible text and tabular data from page | Pro / Enterprise |
|
|
266
|
+
| `browser_screenshot` | Capture visual viewport image of active browser page | Pro / Enterprise |
|
|
267
|
+
| `browser_download` | Automatically download web files, invoices, PDFs to workspace `downloads/` | Pro / Enterprise |
|
|
268
|
+
|
|
269
|
+
### Cognitive Scratchpad & Reasoning Tools
|
|
270
|
+
| Tool | Description | Tier Availability |
|
|
271
|
+
|---|---|:---:|
|
|
272
|
+
| `scratchpad` | Private working memory for multi-step plans, hypotheses, and state (`write`, `read`, `append`, `clear`) | Pro / Enterprise |
|
|
273
|
+
|
|
274
|
+
### Workspace & Engineering Tools (Developer Baseline)
|
|
275
|
+
| Tool | Description | Tier Availability |
|
|
276
|
+
|---|---|:---:|
|
|
277
|
+
| `read_file` | Read content & line ranges of workspace files | All Tiers (Free, Pro, Enterprise) |
|
|
278
|
+
| `write_file` | Create new files or overwrite existing source code | All Tiers (Free, Pro, Enterprise) |
|
|
279
|
+
| `edit_file` | Search & replace exact target code snippets | All Tiers (Free, Pro, Enterprise) |
|
|
280
|
+
| `multi_edit_file` | Apply batch snippet replacements in a single atomic tool call | All Tiers (Free, Pro, Enterprise) |
|
|
281
|
+
| `run_command` | Execute live terminal commands (`npm test`, `tsc`, `pytest`) | All Tiers (Free, Pro, Enterprise) |
|
|
282
|
+
| `list_dir` | List contents of workspace directories | All Tiers (Free, Pro, Enterprise) |
|
|
283
|
+
| `tree_view` | Generate visual ASCII directory tree structures for workspace paths | All Tiers (Free, Pro, Enterprise) |
|
|
284
|
+
| `file_info` | Get file metadata, exact line counts, size, & modification timestamps | All Tiers (Free, Pro, Enterprise) |
|
|
285
|
+
| `grep_search` | String/regex pattern search across entire codebase | All Tiers (Free, Pro, Enterprise) |
|
|
286
|
+
| `glob_search` | Filter workspace files by file glob patterns | All Tiers (Free, Pro, Enterprise) |
|
|
287
|
+
| `fetch_url` | Fetch web API documentation, specs, or external web URLs | All Tiers (Free, Pro, Enterprise) |
|
|
288
|
+
| `git_diff` | Query uncommitted workspace git diff directly inside execution loop | All Tiers (Free, Pro, Enterprise) |
|
|
289
|
+
| `ask_user` | Request missing secret credentials or API keys | All Tiers (Free, Pro, Enterprise) |
|
|
290
|
+
| `speak_text` | Read text phrases out loud using Text-to-Speech (TTS) voice synthesis | All Tiers (Free, Pro, Enterprise) |
|
|
291
|
+
| `task_completed` | Conclude autonomous goal execution with verification and report | All Tiers (Free, Pro, Enterprise) |
|
|
262
292
|
|
|
263
293
|
---
|
|
264
294
|
|
|
@@ -301,12 +331,14 @@ Inside an active `scout agent` interactive session, type any slash command to tr
|
|
|
301
331
|
|
|
302
332
|
---
|
|
303
333
|
|
|
304
|
-
## What's New in v9.0.
|
|
305
|
-
- **Autonomous Computer-Use & Desktop App Operator**: Native desktop application control (Notepad, Excel, VS Code, Browser) with full mouse, keyboard, hotkey, and window focus management across Windows, macOS, and Linux.
|
|
334
|
+
## What's New in v9.0.6
|
|
335
|
+
- **Autonomous Computer-Use & Desktop App Operator**: Native desktop application control (Notepad, Excel, VS Code, Browser) with full mouse, keyboard, hotkey, and window focus management across Windows, macOS, and Linux powered by the native `scout_driver.exe` engine.
|
|
336
|
+
- **Autonomous Browser Automation & Document Download Engine**: Fully automated web browsing (`browser_navigate`, `browser_inspect`, `browser_click`, `browser_type`, `browser_select`, `browser_scroll`, `browser_content`, `browser_screenshot`, `browser_download`) with DOM extraction and automatic file downloads to `downloads/`.
|
|
337
|
+
- **Cognitive Scratchpad & Working Memory (`scratchpad`)**: Long-horizon multi-step reasoning memory (`write`, `read`, `append`, `clear`) saved to `.ft/scratchpad.md`, keeping model context concise and focused.
|
|
306
338
|
- **Strict Execution Intent Alignment (`USER INTENT > TOOL CONVENIENCE`)**: Differentiates between outcome and execution requirements; strictly blocks silent filesystem shortcuts when application interaction is requested.
|
|
307
339
|
- **Bounded Computer Recovery Loop**: Structured action feedback (`{ success, action, error, recoverable }`) with automated re-observation, window refocusing, and retries.
|
|
308
340
|
- **Action Verifier & Anti-Substitute Path Rejection**: Verifies true destination (resolving Windows OneDrive/Desktop and `~/Desktop`) and rejects substitute files created in the workspace root.
|
|
309
|
-
- **Interactive Web Dashboard & Sync Server (`scout dashboard`)**: Launch a local web GUI to visually inspect repository maps, audit logs, plan details,
|
|
341
|
+
- **Interactive Web Dashboard & Sync Server (`scout dashboard`)**: Launch a local web GUI to visually inspect repository maps, audit logs, plan details, live agent controls, and subscription tiers.
|
|
310
342
|
- **Developer Authentication Guard (`scout login` / `signup` / `auth`)**: Unified account management with Firebase and Holosene token authentication, securing command execution and session persistence.
|
|
311
343
|
- **Quota & Subscription Management (`scout quota` / `plan`)**: Easily inspect real-time usage quotas, track billing renewal cycles, switch between tiers, or configure BYOK (Bring-Your-Own-Key) access.
|
|
312
344
|
- **Automated Model Benchmark Suite (`scout benchmark`)**: Run live end-to-end latency and throughput benchmarks (tokens/sec) across all configured LLM providers (Nemotron, Qwen, Gemini, GPT).
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["agent.ts"],"names":[],"mappings":"AA6BA,MAAM,WAAW,YAAY;IACzB,EAAE,EAAE,MAAM,CAAC;IACX,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,GAAG,QAAQ,GAAG,KAAK,CAAC;IACpC,WAAW,EAAE,MAAM,CAAC;CACvB;AAKD,wBAAgB,cAAc,CAAC,MAAM,EAAE,YAAY,EAAE,QAKpD;AAED,wBAAgB,cAAc,IAAI,YAAY,EAAE,CAO/C;AAED,MAAM,WAAW,YAAY;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC5B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAAC;IACpC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,EAAE,GAAG,SAAS,CAAC;IACpC,IAAI,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC3B,GAAG,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,SAAS,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC/B,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC3B,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC9B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC7B,GAAG,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC5B,MAAM,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;CAC/B;AAED,wBAAgB,wBAAwB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAsDlE;AAED,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAG3E;AAED,MAAM,WAAW,gBAAgB;IAC7B,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,UAAU,GAAG,QAAQ,GAAG,QAAQ,GAAG,MAAM,GAAG,MAAM,GAAG,KAAK,CAAC;IACvE,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,WAAW,CAAC,EAAE,MAAM,EAAE,GAAG,SAAS,CAAC;CACtC;AAED,wBAAgB,qBAAqB,CAAC,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,GAAG,gBAAgB,CAoF5F;AAED,wBAAgB,yBAAyB,CACrC,UAAU,EAAE,MAAM,EAClB,aAAa,EAAE,MAAM,GAAG,SAAS,EACjC,SAAS,EAAE,MAAM,EAAE,GACpB,MAAM,EAAE,CAgGV;AAED,wBAAsB,iBAAiB,CAAC,cAAc,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,CAMxF;AAED,wBAAsB,WAAW,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,YAAY,
|
|
1
|
+
{"version":3,"file":"agent.d.ts","sourceRoot":"","sources":["agent.ts"],"names":[],"mappings":"AA6BA,MAAM,WAAW,YAAY;IACzB,EAAE,EAAE,MAAM,CAAC;IACX,KAAK,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,GAAG,QAAQ,GAAG,KAAK,CAAC;IACpC,WAAW,EAAE,MAAM,CAAC;CACvB;AAKD,wBAAgB,cAAc,CAAC,MAAM,EAAE,YAAY,EAAE,QAKpD;AAED,wBAAgB,cAAc,IAAI,YAAY,EAAE,CAO/C;AAED,MAAM,WAAW,YAAY;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC5B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAAC;IACpC,GAAG,CAAC,EAAE,MAAM,GAAG,MAAM,EAAE,GAAG,SAAS,CAAC;IACpC,IAAI,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC3B,GAAG,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IACzB,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,SAAS,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC/B,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,KAAK,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC3B,QAAQ,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC9B,IAAI,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAC1B,MAAM,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC7B,GAAG,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC1B,KAAK,CAAC,EAAE,OAAO,GAAG,SAAS,CAAC;IAC5B,MAAM,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;CAC/B;AAED,wBAAgB,wBAAwB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,EAAE,CAsDlE;AAED,wBAAgB,uBAAuB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,SAAS,CAG3E;AAED,MAAM,WAAW,gBAAgB;IAC7B,SAAS,EAAE,MAAM,CAAC;IAClB,UAAU,EAAE,UAAU,GAAG,QAAQ,GAAG,QAAQ,GAAG,MAAM,GAAG,MAAM,GAAG,KAAK,CAAC;IACvE,UAAU,CAAC,EAAE,MAAM,GAAG,SAAS,CAAC;IAChC,WAAW,CAAC,EAAE,MAAM,EAAE,GAAG,SAAS,CAAC;CACtC;AAED,wBAAgB,qBAAqB,CAAC,OAAO,EAAE,MAAM,EAAE,SAAS,EAAE,MAAM,EAAE,GAAG,gBAAgB,CAoF5F;AAED,wBAAgB,yBAAyB,CACrC,UAAU,EAAE,MAAM,EAClB,aAAa,EAAE,MAAM,GAAG,SAAS,EACjC,SAAS,EAAE,MAAM,EAAE,GACpB,MAAM,EAAE,CAgGV;AAED,wBAAsB,iBAAiB,CAAC,cAAc,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,GAAG,MAAM,CAAC,CAMxF;AAED,wBAAsB,WAAW,CAAC,SAAS,CAAC,EAAE,MAAM,EAAE,OAAO,CAAC,EAAE,YAAY,iBA4nC3E"}
|
|
@@ -8,7 +8,7 @@ import { loadContext, recordHistoryAction, saveAgentSession, getDirectoryFiles }
|
|
|
8
8
|
import { renderMarkdown, safeNote } from '../utils/markdown.js';
|
|
9
9
|
import { normalizeLanguage } from '../utils/language.js';
|
|
10
10
|
import { createPullRequest } from '../engine/git.js';
|
|
11
|
-
import { consumeCredit, checkTierAccess } from '../utils/auth.js';
|
|
11
|
+
import { consumeCredit, checkTierAccess, loadAuthConfig, isProOrHigher } from '../utils/auth.js';
|
|
12
12
|
import { handleUndo } from './undo.js';
|
|
13
13
|
import { AgentExecutionLoop, buildDirectoryTree, AGENT_TOOLS } from '../engine/agentEngine.js';
|
|
14
14
|
import { verifyAndSelfHealFiles, executeSmartCommand, runProjectVerification } from '../engine/verifier.js';
|
|
@@ -281,10 +281,14 @@ export async function handleAgent(goalInput, options) {
|
|
|
281
281
|
const selectedMode = await promptInteractionMode();
|
|
282
282
|
isAudioMode = selectedMode === 'audio';
|
|
283
283
|
}
|
|
284
|
+
const authCfg = loadAuthConfig();
|
|
285
|
+
const userTier = (authCfg.currentTier || 'free').toLowerCase();
|
|
286
|
+
const isPro = isProOrHigher(userTier);
|
|
284
287
|
const agentLoop = new AgentExecutionLoop({
|
|
285
288
|
autoApprove: options?.yes,
|
|
286
289
|
maxSteps: options?.steps || options?.maxSteps || 30,
|
|
287
|
-
language: targetLang
|
|
290
|
+
language: targetLang,
|
|
291
|
+
tier: userTier,
|
|
288
292
|
});
|
|
289
293
|
if (options?.resume) {
|
|
290
294
|
const loaded = agentLoop.getStateManager().loadTaskState(options.resume);
|
|
@@ -301,9 +305,13 @@ export async function handleAgent(goalInput, options) {
|
|
|
301
305
|
}
|
|
302
306
|
const modelInfo = getActiveModelInfo();
|
|
303
307
|
const fallbackPipeline = modelInfo.fallbackChain.slice(1).join(' ➔ ');
|
|
308
|
+
const tierBadge = isPro
|
|
309
|
+
? chalk.bold.magenta(`[Plan: ${userTier.toUpperCase()} - Autonomous & Scratchpad Enabled]`)
|
|
310
|
+
: chalk.bold.yellow(`[Plan: FREE - Basic Dev Ops (Edit/Create/Run)]`);
|
|
304
311
|
const welcomeBanner = `
|
|
305
|
-
${chalk.bold.cyan('Scout Autonomous AI Agent')} ${targetLang ? `(${targetLang})` : ''} [${isAudioMode ? '🎤 Audio Mode' : '📝 Text Mode'}]
|
|
312
|
+
${chalk.bold.cyan('Scout Autonomous AI Agent')} ${targetLang ? `(${targetLang})` : ''} [${isAudioMode ? '🎤 Audio Mode' : '📝 Text Mode'}] ${tierBadge}
|
|
306
313
|
${chalk.bold('Active Model:')} ${chalk.bold.green(modelInfo.currentActiveModel)}
|
|
314
|
+
${chalk.bold('Capabilities:')} ${isPro ? chalk.green('Autonomous Computer/Browser Control + AI Scratchpad Working Memory') : chalk.dim('Basic Dev Ops (Code editing, creating files/folders, running commands - Max 5 steps)')}
|
|
307
315
|
${chalk.bold('Fallback Pipeline:')} ${chalk.dim(fallbackPipeline)}
|
|
308
316
|
|
|
309
317
|
${chalk.dim('Type your goal, or use slash commands:')}
|
|
@@ -439,20 +447,25 @@ ${chalk.cyan('/exit')} - Exit session
|
|
|
439
447
|
}
|
|
440
448
|
if (trimmedInput.toLowerCase() === '/help') {
|
|
441
449
|
const helpText = `
|
|
442
|
-
### 🛠️
|
|
450
|
+
### 🛠️ Basic Developer Tools (Available on All Plans)
|
|
443
451
|
- **read_file**: Read content & line ranges of workspace files.
|
|
444
|
-
- **write_file**: Create new files
|
|
452
|
+
- **write_file**: Create new files and directories.
|
|
445
453
|
- **edit_file**: Search & replace exact target code snippets.
|
|
446
454
|
- **multi_edit_file**: Apply batch snippet replacements in a single step.
|
|
447
|
-
- **run_command**: Execute
|
|
455
|
+
- **run_command**: Execute terminal commands (\`npm test\`, \`npm run build\`).
|
|
448
456
|
- **tree_view**: Generate visual ASCII directory tree structures.
|
|
449
457
|
- **file_info**: Get file statistics, size, lines, & modification time.
|
|
450
458
|
- **grep_search**: String/regex pattern search across codebase.
|
|
451
459
|
- **glob_search**: Filter workspace files by file pattern.
|
|
452
|
-
- **fetch_url**: Fetch web API documentation or URL content.
|
|
453
460
|
- **git_diff**: Retrieve uncommitted workspace git diff.
|
|
454
461
|
- **ask_user**: Request missing secret credentials or API keys.
|
|
455
|
-
- **task_completed**: Conclude
|
|
462
|
+
- **task_completed**: Conclude task execution with teardown summary.
|
|
463
|
+
|
|
464
|
+
### ⚡ Pro & Enterprise Exclusive Features
|
|
465
|
+
- **scratchpad**: Persistent working memory & reasoning scratchpad for multi-phase tasks.
|
|
466
|
+
- **computer_***: Autonomous desktop & GUI app control (screenshot, click, type, open app).
|
|
467
|
+
- **browser_***: Autonomous browser automation (navigate, scrape, interact).
|
|
468
|
+
- **Autonomous Loops**: Extended multi-step execution loops (30+ steps).
|
|
456
469
|
|
|
457
470
|
### 🚀 Interactive Slash Commands
|
|
458
471
|
- \`/help\`: Show this tools & commands guide.
|