@testdriverai/agent 7.11.239-test → 7.11.241-canary

package/ai/skills/testdriver-assert/SKILL.md

@@ -1,314 +1,118 @@
 ---
 name: testdriver:assert
-description: Make AI-powered assertions about screen state
+description: Check a plain-language statement about the screen, and fail the test if it's false.
 ---
 <!-- Generated from assert.mdx. DO NOT EDIT. -->
 
-## Overview
+Check that something is true on screen. The AI looks at the current screen and decides whether your statement holds.
 
-Make assertions about the screen state with natural language and AI. The AI examines the screen. It makes sure that your assertion is true.
+```javascript
+await testdriver.assert('the dashboard shows a welcome message');
+```
 
-## Syntax
+For writing good assertions, see [Make assertions](/making-assertions).
 
-```javascript
-await testdriver.assert(assertion)
-await testdriver.assert(assertion, options)
+## Signature
+
+```typescript
+testdriver.assert(assertion: string, options?: AssertOptions): Promise<boolean>
 ```
 
 ## Parameters
 
 <ParamField path="assertion" type="string" required>
-  Natural language description of what should be true
+  A statement about the screen that should be true, such as `'the cart shows 1 item'`.
 </ParamField>
 
-<ParamField path="options" type="object">
-  Optional configuration
-  
+<ParamField path="options" type="AssertOptions">
   <Expandable title="properties">
-    <ParamField path="ai" type="object">
-      AI sampling configuration for this assert call (overrides global `ai` config from constructor).
-      
-      <Expandable title="properties">
-        <ParamField path="temperature" type="number">
-          Controls randomness. `0` = deterministic, higher = more creative. Default: model default.
-        </ParamField>
-        
-        <ParamField path="top" type="object">
-          Sampling parameters
-          
-          <Expandable title="properties">
-            <ParamField path="p" type="number">
-              Top-P (nucleus sampling). Range: 0-1.
-            </ParamField>
-            
-            <ParamField path="k" type="number">
-              Top-K sampling. `1` = most deterministic.
-            </ParamField>
-          </Expandable>
-        </ParamField>
-      </Expandable>
+    <ParamField path="threshold" type="number" default="0.05">
+      How much the screen may differ from a cached passing run (0 to 1) and still reuse its result. Defaults to the client's `cache.thresholds.assert`, which is `0.05`. `-1` turns caching off for this call. See [Caching](/caching).
+    </ParamField>
+    <ParamField path="cacheKey" type="string">
+      Store and look up the cached result under this key instead of the client's.
+    </ParamField>
+    <ParamField path="ai" type="AIConfig">
+      Sampling settings for this call, such as `{ temperature: 0 }`. Overrides the client's [`ai` option](/options#param-ai).
+    </ParamField>
+    <ParamField path="os" type="string">
+      The OS to store the cached result under. Set automatically from the client.
+    </ParamField>
+    <ParamField path="resolution" type="string">
+      The resolution to store the cached result under. Set automatically from the client.
     </ParamField>
   </Expandable>
 </ParamField>
 
 ## Returns
 
-`Promise<boolean>` - `true` if the assertion passes. It throws an error if the assertion fails.
+`Promise<boolean>`. Resolves to `true` when the statement is true.
+
+## Errors
+
+When the statement is false, `assert()` throws, so the test fails at that line. The error message starts with `AI Assertion failed:`, followed by your statement and the AI's explanation.
+
+Inside `ai()`, assertions don't throw. They resolve to `false` so the AI can keep working.
 
 ## Examples
 
-### Basic Assertions
+### Check the result of an action
 
 ```javascript
-// Verify page elements
-await testdriver.assert('the login page is displayed');
-await testdriver.assert('submit button is visible');
-await testdriver.assert('error message is shown');
-
-// Verify text content
-await testdriver.assert('the page title is "Welcome"');
-await testdriver.assert('username field contains "john.doe"');
-await testdriver.assert('success message says "Account created"');
-
-// Verify states
-await testdriver.assert('the form is empty');
-await testdriver.assert('the checkbox is checked');
-await testdriver.assert('the dropdown shows "United States"');
-
-// Verify visual appearance
-await testdriver.assert('the button is blue');
-await testdriver.assert('the loading spinner is displayed');
-await testdriver.assert('the modal dialog is open');
+await testdriver.find('Add to cart button').click();
+await testdriver.assert('the cart icon shows 1 item');
 ```
 
-## Best Practices
-
-<Check>
-  **Be specific in assertions**
-  
-  More specific assertions are more reliable:
-  
-  ```javascript
-  // ❌ Too vague
-  await testdriver.assert('button is visible');
-  
-  // ✅ Specific
-  await testdriver.assert('blue submit button is visible below the form');
-  ```
-</Check>
-
-<Check>
-  **Assert state changes**
-  
-  Make sure of the state before and after actions:
-  
-  ```javascript
-  // Before
-  await testdriver.assert('cart is empty');
-  
-  // Action
-  const addBtn = await testdriver.find('add to cart');
-  await addBtn.click();
-  
-  // After
-  await testdriver.assert('cart contains 1 item');
-  ```
-</Check>
-
-<Check>
-  **Use with test framework assertions**
-  
-  Combine AI assertions with traditional test assertions:
-  
-  ```javascript
-  // AI assertion
-  const result = await testdriver.assert('success message is displayed');
-  
-  // Framework assertion
-  expect(result).toBeTruthy();
-  
-  // Extract for detailed comparison
-  const message = await testdriver.extract('the success message text');
-  expect(message).toContain('successfully');
-  ```
-</Check>
-
-## Polling Assertions
-
-For conditions that may take time to become true:
+### Check text in a field
 
 ```javascript
-async function waitForAssertion(testdriver, assertion, timeout = 30000) {
-  const startTime = Date.now();
-  
-  while (Date.now() - startTime < timeout) {
-    try {
-      await testdriver.assert(assertion);
-      return true; // Assertion passed
-    } catch (error) {
-      // Assertion failed, wait and retry
-      await new Promise(r => setTimeout(r, 1000));
-    }
-  }
-  
-  throw new Error(`Assertion timeout: "${assertion}"`);
-}
-
-// Usage
-await waitForAssertion(testdriver, 'page has finished loading', 30000);
-await waitForAssertion(testdriver, 'results are displayed', 10000);
+await testdriver.find('Username input field').click();
+await testdriver.type('standard_user');
+await testdriver.assert('the username field contains "standard_user"');
 ```
 
-## Use Cases
-
-<AccordionGroup>
-  <Accordion title="Form Validation">
-    ```javascript
-    // Try to submit empty form
-    const submitBtn = await testdriver.find('submit button');
-    await submitBtn.click();
-    
-    // Verify validation errors
-    await testdriver.assert('email field shows "required" error');
-    await testdriver.assert('password field shows "required" error');
-    
-    // Verify form not submitted
-    await testdriver.assert('still on the form page');
-    ```
-  </Accordion>
-  
-  <Accordion title="Page Navigation">
-    ```javascript
-    const loginBtn = await testdriver.find('login button');
-    await loginBtn.click();
-    
-    // Verify navigation
-    await testdriver.assert('user dashboard is displayed');
-    await testdriver.assert('welcome message shows user name');
-    await testdriver.assert('logout button is visible');
-    ```
-  </Accordion>
-  
-  <Accordion title="Dynamic Content">
-    ```javascript
-    const loadBtn = await testdriver.find('load more button');
-    await loadBtn.click();
-    
-    // Poll for content using helper
-    await waitForAssertion(testdriver, 'more than 10 items are shown', 10000);
-    await testdriver.assert('load more button is still visible');
-    ```
-  </Accordion>
-  
-  <Accordion title="Visual States">
-    ```javascript
-    // Verify hover effect
-    const button = await testdriver.find('primary button');
-    await button.hover();
-    
-    await testdriver.assert('button background is darker');
-    
-    // Verify button is enabled
-    await testdriver.assert('submit button is enabled');
-    ```
-  </Accordion>
-  
-  <Accordion title="Multi-Step Workflows">
-    ```javascript
-    // Step 1
-    await testdriver.assert('step 1 is active');
-    const nextBtn = await testdriver.find('next button');
-    await nextBtn.click();
-    
-    // Step 2
-    await testdriver.assert('step 2 is active');
-    await testdriver.assert('step 1 is completed');
-    await nextBtn.click();
-    
-    // Step 3
-    await testdriver.assert('step 3 is active');
-    await testdriver.assert('step 2 is completed');
-    ```
-  </Accordion>
-</AccordionGroup>
-
-## Complete Example
+### Use it with expect()
+
+`assert()` already fails the test when the statement is false. An `expect()` makes the check visible to anyone reading the test:
 
 ```javascript
-import { beforeAll, afterAll, describe, it, expect } from 'vitest';
-import TestDriver from 'testdriverai';
+const loggedIn = await testdriver.assert('the dashboard is visible');
+expect(loggedIn).toBe(true);
+```
 
-describe('Assertions', () => {
-  let testdriver;
+### Wait for something before asserting
 
-  beforeAll(async () => {
-    client = new TestDriver(process.env.TD_API_KEY);
-    await testdriver.auth();
-    await testdriver.connect();
-  });
+`assert()` checks the screen once. For something that takes time to appear, wait for it with [`find()`](/find) first:
 
-  afterAll(async () => {
-    await testdriver.disconnect();
-  });
+```javascript
+await testdriver.find('Order confirmation heading', { timeout: 30000 });
+await testdriver.assert('the confirmation shows an order number');
+```
 
-  it('should validate login flow', async () => {
-    await testdriver.focusApplication('Google Chrome');
-    
-    // Verify initial state
-    await testdriver.assert('the login page is displayed');
-    await testdriver.assert('username field is empty');
-    await testdriver.assert('password field is empty');
-    
-    // Fill form
-    const usernameField = await testdriver.find('username input');
-    await usernameField.click();
-    await testdriver.type('testuser');
-    
-    await testdriver.pressKeys(['tab']);
-    await testdriver.type('password123');
-    
-    // Verify fields filled
-    await testdriver.assert('username field contains "testuser"');
-    await testdriver.assert('password field is not empty');
-    
-    // Submit
-    await testdriver.pressKeys(['enter']);
-    
-    // Poll for success
-    await waitForAssertion(testdriver, 'user dashboard is displayed', 10000);
-    
-    // Verify logged in state
-    await testdriver.assert('welcome message is shown');
-    await testdriver.assert('logout button is visible');
-    
-    // Verify login page is gone
-    await testdriver.assert('login form is displayed', false, true);
-  });
+## Complete example
+
+```javascript tests/login.test.mjs
+import { describe, expect, it } from 'vitest';
+import { TestDriver } from 'testdriverai/vitest/hooks';
+
+describe('Login', () => {
+  it('shows an error when the form is empty', async (context) => {
+    const testdriver = TestDriver(context);
+    await testdriver.provision.chrome({ url: 'https://testdriver-sandbox.vercel.app/login' });
+
+    await testdriver.assert('the login form is empty');
+
+    await testdriver.find('Sign in button below the password field').click();
 
-  it('should show validation errors', async () => {
-    // Try empty submission
-    const submitBtn = await testdriver.find('submit button');
-    await submitBtn.click();
-    
-    // Verify multiple errors
-    const errors = [
-      'email field shows error',
-      'name field shows error',
-      'password field shows error'
-    ];
-    
-    for (const error of errors) {
-      const result = await testdriver.assert(error);
-      expect(result).toBeTruthy();
-    }
-    
-    // Verify not submitted
-    await testdriver.assert('confirmation page is shown', false, true);
+    const result = await testdriver.assert('an error says the fields are required');
+    expect(result).toBe(true);
   });
 });
 ```
 
-## Related Methods
+## Related
 
-- [`extract()`](/extract) - Extract information for detailed assertions
-- [`find()`](/find) - Locate elements to verify
+- [Make assertions](/making-assertions): write assertions that catch real regressions
+- [extract()](/extract): read a value off the screen to compare yourself
+- [find()](/find): wait for an element before asserting
+- [Caching](/caching): how assertion results are reused