@testdriverai/agent 7.11.240-test → 7.11.241-canary

package/ai/skills/testdriver-exec/SKILL.md

@@ -1,317 +1,153 @@
 ---
 name: testdriver:exec
-description: Execute shell or PowerShell commands in the sandbox
+description: Run a shell or PowerShell command in the sandbox and get its output.
 ---
 <!-- Generated from exec.mdx. DO NOT EDIT. -->
 
-## Overview
-
-Run shell commands (Linux) or PowerShell commands (Windows) in the sandbox.
-
-## Syntax
+Run a command in the sandbox, for example to create test files, install software, or launch an app. `exec()` returns the command's output.
 
 ```javascript
-await testdriver.exec(language, code, timeout, silent)
+const files = await testdriver.exec({ language: 'sh', code: 'ls ~/Downloads' });
 ```
 
-## Parameters
+- On Linux sandboxes, use `'sh'`. The command runs in Bash.
+- On Windows sandboxes, use `'pwsh'`. The command runs in PowerShell.
 
-<ParamField path="language" type="string" required>
-  The language to run: `'sh'` (Shell/Linux) or `'pwsh'` (PowerShell/Windows)
-</ParamField>
+## Signature
 
-<ParamField path="code" type="string" required>
-  The code or the command to run
-</ParamField>
+```typescript
+testdriver.exec(options: ExecOptions): Promise<string>
+```
 
-<ParamField path="timeout" type="number" required>
-  The timeout in milliseconds
-</ParamField>
+## Parameters
 
-<ParamField path="silent" type="boolean" default="false">
-  If `true`, TestDriver does not show the output
+<ParamField path="options" type="ExecOptions" required>
+  <Expandable title="properties">
+    <ParamField path="code" type="string" required>
+      The command or script to run. It can span several lines.
+    </ParamField>
+    <ParamField path="language" type="'sh' | 'pwsh'" default="pwsh">
+      The shell to run it in. The default is `'pwsh'`, so set `'sh'` on Linux, which is the default sandbox. On a Windows sandbox, `'sh'` switches to `'pwsh'` with a warning.
+    </ParamField>
+    <ParamField path="timeout" type="number" default="300000">
+      How long the command may run, in milliseconds. The default is 5 minutes.
+    </ParamField>
+    <ParamField path="silent" type="boolean" default="false">
+      Keep the command and its output out of the test log.
+    </ParamField>
+  </Expandable>
 </ParamField>
 
-## Returns
-
-`Promise<string>` - The command output
+Commands run in one shell session that stays open for the whole test. A `cd` or an environment variable you set carries over to the next `exec()` call.
 
-## Shell Execution (Linux)
+## Returns
 
-Run shell commands in the Linux sandbox.
+`Promise<string>`. The command's standard output, with leading and trailing whitespace trimmed. Output over 5 MB is cut off.
 
-### Basic Commands
+## Errors
 
-```javascript
-// List files
-const files = await testdriver.exec('sh', 'ls -la', 5000);
+`exec()` throws when:
 
-// Check current directory
-const pwd = await testdriver.exec('sh', 'pwd', 5000);
-
-// Run a script
-await testdriver.exec('sh', './setup.sh', 60000);
-```
+- The command exits with a non-zero code. The message is `Command failed with exit code <code>: <stderr>`.
+- The command runs longer than `timeout`.
 
-### File Operations
+To allow a failure, catch the error:
 
 ```javascript
-// Create a file
-await testdriver.exec('sh', 'echo "Hello World" > test.txt', 5000);
-
-// Read a file
-const content = await testdriver.exec('sh', 'cat test.txt', 5000);
-
-// Copy files
-await testdriver.exec('sh', 'cp source.txt dest.txt', 5000);
-
-// Delete files
-await testdriver.exec('sh', 'rm test.txt', 5000);
+try {
+  await testdriver.exec({ language: 'sh', code: 'pkill -f my-app' });
+} catch (error) {
+  // pkill exits with 1 when no process matched
+}
 ```
 
-## PowerShell Execution (Windows)
-
-Execute PowerShell commands in the Windows sandbox.
+## Examples
 
-### Software Installation
+### Create a file the test needs
 
 ```javascript
-// Install npm package globally
-await testdriver.exec('pwsh', 'npm install -g http-server', 30000);
-
-// Install via Chocolatey
-await testdriver.exec('pwsh', 'choco install firefox -y', 60000);
-
-// Download and run installer
-await testdriver.exec('pwsh', `
-  Invoke-WebRequest -Uri "https://example.com/setup.exe" -OutFile "C:\\setup.exe"
-  Start-Process -FilePath "C:\\setup.exe" -ArgumentList "/S" -Wait
-`, 120000);
+await testdriver.exec({
+  language: 'sh',
+  code: 'echo "name,email" > ~/Desktop/contacts.csv',
+});
 ```
 
-### File Operations
+### Read a value and check it
 
 ```javascript
-// Create a file
-await testdriver.exec('pwsh', `
-  Set-Content -Path "C:\\test.txt" -Value "Hello World"
-`, 5000);
-
-// Read a file
-const content = await testdriver.exec('pwsh', 
-  'Get-Content -Path "C:\\test.txt"', 
-  5000
-);
-
-// Copy files
-await testdriver.exec('pwsh', 
-  'Copy-Item -Path "C:\\source.txt" -Destination "C:\\dest.txt"', 
-  5000
-);
-
-// Delete files
-await testdriver.exec('pwsh', 
-  'Remove-Item -Path "C:\\test.txt"', 
-  5000
-);
-
-// List directory
-const files = await testdriver.exec('pwsh', 
-  'Get-ChildItem -Path "C:\\Users\\testdriver\\Documents"', 
-  5000
-);
+const version = await testdriver.exec({ language: 'sh', code: 'node --version' });
+expect(version).toMatch(/^v\d+/);
 ```
 
-### Process Management
+### Install software on Windows
 
 ```javascript
-// List running processes
-const processes = await testdriver.exec('pwsh', 'Get-Process', 5000);
-
-// Start application
-await testdriver.exec('pwsh', `
-  Start-Process "C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe" -ArgumentList "--start-maximized", "https://example.com"
-`, 5000);
-
-// Kill a process
-await testdriver.exec('pwsh', 'Stop-Process -Name "chrome" -Force', 5000);
-
-// Wait for process
-await testdriver.exec('pwsh', 'Start-Process notepad -Wait', 30000);
+await testdriver.exec({
+  language: 'pwsh',
+  code: 'choco install firefox -y',
+  timeout: 600000,
+  silent: true,
+});
 ```
 
-### Environment Variables
+### Launch an app on Windows
 
 ```javascript
-// Set environment variable (session)
-await testdriver.exec('pwsh', '$env:MY_VAR = "value"', 5000);
-
-// Get environment variable
-const value = await testdriver.exec('pwsh', '$env:MY_VAR', 5000);
-
-// Set persistent environment variable
-await testdriver.exec('pwsh', `
-  [Environment]::SetEnvironmentVariable("MY_VAR", "value", "User")
-`, 5000);
+await testdriver.exec({ language: 'pwsh', code: 'Start-Process notepad' });
+await testdriver.focusApplication('Notepad');
 ```
 
-### Network Operations
+To open Chrome, VS Code, Electron apps, or installers, use [`provision`](/provision) instead. It launches the app and waits for it.
 
-```javascript
-// Test connectivity
-const pingResult = await testdriver.exec('pwsh', 
-  'Test-NetConnection google.com', 
-  10000
-);
-
-// Download file
-await testdriver.exec('pwsh', `
-  Invoke-WebRequest -Uri "https://example.com/file.zip" -OutFile "C:\\Downloads\\file.zip"
-`, 30000);
-
-// Check if port is open
-const portOpen = await testdriver.exec('pwsh', 
-  'Test-NetConnection -ComputerName localhost -Port 3000', 
-  5000
-);
-```
+## Positional form
 
-## Silent Execution
+Older tests pass the options as arguments:
 
-Suppress output for background operations:
+```typescript
+testdriver.exec(language: 'sh' | 'pwsh', code: string, timeout?: number, silent?: boolean): Promise<string>
+```
 
 ```javascript
-// Silent installation
-await testdriver.exec('pwsh', 'npm install -g some-package', 30000, true);
+await testdriver.exec('sh', 'ls ~/Downloads', 5000);
+```
 
-// Start background process
-await testdriver.exec('pwsh', 'Start-Process notepad', 5000, true);
+It behaves the same. Use the options object in new tests.
 
-// Run setup script silently
-await testdriver.exec('pwsh', '.\\setup.ps1', 60000, true);
-```
+<Note>
+  `language: 'js'` is also accepted. It runs the code in Node on your own machine, not in the sandbox, and returns the value you assign to a `result` variable. Write the JavaScript directly in your test instead.
+</Note>
 
-## Best Practices
-
-<Check>
-  **Use appropriate timeouts**
-  
-  ```javascript
-  // Quick operations: 5000ms
-  await testdriver.exec('sh', 'ls -la', 5000);
-  
-  // Installations: 30000-60000ms
-  await testdriver.exec('pwsh', 'npm install -g package', 30000);
-  
-  // Downloads or complex operations: 60000-120000ms
-  await testdriver.exec('pwsh', 'Install-Module Something', 120000);
-  ```
-</Check>
-
-<Check>
-  **Handle errors gracefully**
-  
-  ```javascript
-  try {
-    await testdriver.exec('pwsh', 'Some-Command', 5000);
-  } catch (error) {
-    console.error('Command failed:', error.message);
-    // Fallback or retry logic
-  }
-  ```
-</Check>
-
-<Check>
-  **Use silent mode for background tasks**
-  
-  ```javascript
-  // Silent install
-  await testdriver.exec('pwsh', 'npm install -g tool', 30000, true);
-  
-  // Background service
-  await testdriver.exec('pwsh', 'Start-Service MyService', 5000, true);
-  ```
-</Check>
-
-<Warning>
-  **Escape strings properly in PowerShell**
-  
-  Use proper escaping for special characters:
-  
-  ```javascript
-  // Use backticks for newlines
-  await testdriver.exec('pwsh', `
-    Write-Host "Line 1\`nLine 2"
-  `, 5000);
-  
-  // Use single quotes to avoid variable expansion
-  await testdriver.exec('pwsh', 
-    "Write-Host 'Text with $special chars'", 
-    5000
-  );
-  ```
-</Warning>
-
-## Complete Example
+## Complete example
 
-```javascript
-import { beforeAll, afterAll, describe, it } from 'vitest';
-import TestDriver from 'testdriverai';
+```javascript tests/download.test.mjs
+import { describe, expect, it } from 'vitest';
+import { TestDriver } from 'testdriverai/vitest/hooks';
 
-describe('Code Execution', () => {
-  let testdriver;
+describe('Downloads', () => {
+  it('opens a file created in the sandbox', async (context) => {
+    const testdriver = TestDriver(context);
+    await testdriver.provision.chrome({ url: 'https://testdriver-sandbox.vercel.app/login' });
 
-  beforeAll(async () => {
-    client = new TestDriver(process.env.TD_API_KEY);
-    await testdriver.auth();
-    await testdriver.connect();
-  });
+    await testdriver.exec({
+      language: 'sh',
+      code: 'echo "<h1>Hello from exec</h1>" > /tmp/hello.html',
+    });
 
-  afterAll(async () => {
-    await testdriver.disconnect();
-  });
+    const contents = await testdriver.exec({ language: 'sh', code: 'cat /tmp/hello.html' });
+    expect(contents).toContain('Hello from exec');
 
-  it('should execute shell commands on Linux', async () => {
-    // List directory
-    const files = await testdriver.exec('sh', 'ls -la', 5000);
-    console.log('Files:', files);
-    
-    // Create a file
-    await testdriver.exec('sh', 'echo "Hello World" > test.txt', 5000);
-    
-    // Read the file
-    const content = await testdriver.exec('sh', 'cat test.txt', 5000);
-    expect(content).toContain('Hello World');
-  });
+    await testdriver.pressKeys(['ctrl', 'l']);
+    await testdriver.type('file:///tmp/hello.html');
+    await testdriver.pressKeys(['enter']);
 
-  it('should install and use tools on Windows', async () => {
-    // Install tool
-    await testdriver.exec('pwsh', 'npm install -g http-server', 30000, true);
-    
-    // Create HTML file
-    await testdriver.exec('pwsh', `
-      Set-Content -Path "C:\\index.html" -Value "<h1>Test Page</h1>"
-    `, 5000);
-    
-    // Start server in background
-    await testdriver.exec('pwsh', `
-      Start-Process pwsh -ArgumentList "-Command", "http-server C:\\ -p 8080"
-    `, 5000, true);
-    
-    await new Promise(r => setTimeout(r, 3000));
-    
-    // Launch browser
-    await testdriver.exec('pwsh', `
-      Start-Process chrome -ArgumentList "http://localhost:8080"
-    `, 5000);
-    
-    await testdriver.focusApplication('Google Chrome');
+    const result = await testdriver.assert('the page shows "Hello from exec"');
+    expect(result).toBe(true);
   });
 });
 ```
 
-## Related Methods
+## Related
 
-- [`focusApplication()`](/focus-application) - Focus apps before exec
-- [`find()`](/find) - Locate elements visually
-- [`type()`](/type) - Type text into inputs
+- [Launch apps](/provision): open Chrome, VS Code, and installers
+- [focusApplication()](/focus-application): bring a launched app to the front
+- [Customize devices](/customizing-devices): set up the sandbox before your test