codeceptjs 4.2.0 → 4.2.1-beta.1

package/docs/helpers/Decision.md

@@ -0,0 +1,146 @@
+---
+permalink: /helpers/Decision
+editLink: false
+sidebar: auto
+title: Decision
+---
+
+<!-- Generated by documentation.js. Update this documentation by updating the source code. -->
+
+## Decision
+
+**Extends Helper**
+
+Asserts statements about the current page with a decision model.
+
+Decision models (like [Jev][1] or [Clef][2])
+do not generate text. They read the page state and return the probability that a statement is true.
+A statement passes when its probability reaches the configured `confidence`.
+
+```js
+I.decide('top level navigation is available');
+I.decide([
+  'checkout form has all required fields',
+  'success message is shown',
+  'submit button enabled',
+  'cancel button present',
+]);
+I.decideVisually('sidebar is shown');
+```
+
+The model receives the page URL, title and ARIA snapshot.
+Helpers without ARIA snapshots (anything but Playwright) send minified HTML instead.
+`decideVisually` also sends a screenshot, so it requires a model with image input.
+
+This helper must be enabled together with a web helper (Playwright, Puppeteer, WebDriver).
+
+## Configuration
+
+Decision model is created with the OpenRouter provider from `@openrouter/ai-sdk-provider`
+and configured in the `ai.decisionModel` section of the config:
+
+```js
+import { createOpenRouter } from '@openrouter/ai-sdk-provider'
+
+const openrouter = createOpenRouter({
+  apiKey: process.env.OPENROUTER_API_KEY,
+})
+
+export default {
+  ai: {
+    decisionModel: {
+      model: openrouter.evaluationModel('typesafe/jev-1.13'),
+      confidence: 0.8,
+    },
+  },
+  helpers: {
+    Playwright: { url: 'http://localhost' },
+    Decision: {},
+  },
+}
+```
+
+*   `model` - decision model used by `decide`.
+*   `confidence` (default: `0.7`) - minimal probability, between 0 and 1, for a statement to pass.
+*   `timeout` (default: `15000`) - request timeout in ms.
+*   `maxLength` (default: `12000`) - maximal length of ARIA snapshot or HTML sent to the model.
+
+`decideVisually` uses a separate decision model with image input, configured as `visualModel`:
+
+```js
+decisionModel: {
+  model: openrouter.evaluationModel('typesafe/jev-1.13'),
+  visualModel: openrouter.evaluationModel('cloudflare/clef'),
+},
+```
+
+The helper has one option:
+
+*   `mode` (default: `assert`) - how decisions are executed:
+    *   `assert` - request the model and fail the step when a statement is not confirmed.
+    *   `report` - request the model but never fail, even on API errors. Results and errors are added to the step as a comment.
+    *   `skip` - do not request the model, every decision passes.
+
+```js
+Decision: { mode: process.env.CI ? 'assert' : 'skip' }
+```
+
+Decisions work without the `--ai` flag.
+Browse all decision models at [OpenRouter][3].
+
+## Methods
+
+### Parameters
+
+*   `config`   
+
+### decide
+
+Asserts that a statement (or each of statements) is true for the current page.
+The decision model receives page URL, title and ARIA snapshot.
+Multiple statements are checked in a single request and all of them must pass.
+
+```js
+I.decide('user is logged in');
+I.decide(['submit button enabled', 'cancel button present']);
+const probability = await I.decide('cart is empty');
+```
+
+#### Parameters
+
+*   `statements` **([string][4] | [Array][5]<[string][4]>)** statement or list of statements to verify.
+
+Returns **[Promise][6]<([number][7] | [Array][5]<[number][7]> | [undefined][8])>** probability of each statement, `undefined` in `skip` mode.
+
+### decideVisually
+
+Asserts that a statement (or each of statements) is true for the current page using a screenshot.
+The decision model receives page URL, title, ARIA snapshot and a screenshot.
+Requires `visualModel` with image input. Experimental.
+
+```js
+I.decideVisually('sidebar is shown');
+I.decideVisually(['logo is in the header', 'page uses dark theme']);
+```
+
+#### Parameters
+
+*   `statements` **([string][4] | [Array][5]<[string][4]>)** statement or list of statements to verify.
+
+Returns **[Promise][6]<([number][7] | [Array][5]<[number][7]> | [undefined][8])>** probability of each statement, `undefined` in `skip` mode.
+
+[1]: https://openrouter.ai/typesafe/jev-1.13
+
+[2]: https://openrouter.ai/cloudflare/clef
+
+[3]: https://openrouter.ai/models?output_modalities=decisions
+
+[4]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String
+
+[5]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Array
+
+[6]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise
+
+[7]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Number
+
+[8]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/undefined

package/docs/helpers/GraphQL.md

@@ -49,25 +49,6 @@ this.helpers['GraphQL']._executeQuery({
 
 *   `config`  
 
-### _executeQuery
-
-Executes query via axios call
-
-#### Parameters
-
-*   `request` **[object][2]** 
-
-### _prepareGraphQLRequest
-
-Prepares request for axios call
-
-#### Parameters
-
-*   `operation` **[object][2]** 
-*   `headers` **[object][2]** 
-
-Returns **[object][2]** graphQLRequest
-
 ### amBearerAuthenticated
 
 Adds a header for Bearer authentication
@@ -79,7 +60,7 @@ I.amBearerAuthenticated(secret('heregoestoken'))
 
 #### Parameters
 
-*   `accessToken` **([string][3] | CodeceptJS.Secret)** Bearer access token
+*   `accessToken` **([string][2] | CodeceptJS.Secret)** Bearer access token
 
 ### haveRequestHeaders
 
@@ -87,7 +68,7 @@ Sets request headers for all requests of this test
 
 #### Parameters
 
-*   `headers` **[object][2]** headers list
+*   `headers` **[object][3]** headers list
 
 ### sendMutation
 
@@ -113,10 +94,10 @@ I.sendMutation(`
 
 #### Parameters
 
-*   `mutation` **[String][3]** 
-*   `variables` **[object][2]?** that may go along with the mutation
-*   `options` **[object][2]?** are additional query options 
-*   `headers` **[object][2]?**  
+*   `mutation` **[String][2]** 
+*   `variables` **[object][3]?** that may go along with the mutation
+*   `options` **[object][3]?** are additional query options 
+*   `headers` **[object][3]?**  
 
 Returns **any** Promise<any>
 
@@ -138,15 +119,34 @@ const user = response.data.data;
 
 #### Parameters
 
-*   `query` **[String][3]**&#x20;
-*   `variables` **[object][2]?** that may go along with the query
-*   `options` **[object][2]?** are additional query options 
-*   `headers` **[object][2]?**  
+*   `query` **[String][2]**&#x20;
+*   `variables` **[object][3]?** that may go along with the query
+*   `options` **[object][3]?** are additional query options 
+*   `headers` **[object][3]?**  
 
 Returns **any** Promise<any>
 
+### _executeQuery
+
+Executes query via axios call
+
+#### Parameters
+
+*   `request` **[object][3]**&#x20;
+
+### _prepareGraphQLRequest
+
+Prepares request for axios call
+
+#### Parameters
+
+*   `operation` **[object][3]**&#x20;
+*   `headers` **[object][3]**&#x20;
+
+Returns **[object][3]** graphQLRequest
+
 [1]: https://github.com/axios/axios
 
-[2]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object
+[2]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String
 
-[3]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/String
+[3]: https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Object

package/docs/helpers/GraphQLDataFactory.md

@@ -160,26 +160,6 @@ Data of created records are collected and used in the end of a test for the clea
 
 *   `config`  
 
-### _requestCreate
-
-Executes request to create a record to the GraphQL endpoint.
-Can be replaced from a custom helper.
-
-#### Parameters
-
-*   `operation` **[string][4]** 
-*   `variables` **any** to be sent along with the query
-
-### _requestDelete
-
-Executes request to delete a record to the GraphQL endpoint.
-Can be replaced from a custom helper.
-
-#### Parameters
-
-*   `operation` **[string][4]** 
-*   `data` **any** of the record to be deleted.
-
 ### mutateData
 
 Generates a new record using factory, sends a GraphQL mutation to store it.
@@ -215,6 +195,26 @@ I.mutateMultiple('createUser', 3, { age: 25 });
 *   `times` **[number][5]** 
 *   `params` **any** 
 
+### _requestCreate
+
+Executes request to create a record to the GraphQL endpoint.
+Can be replaced from a custom helper.
+
+#### Parameters
+
+*   `operation` **[string][4]** 
+*   `variables` **any** to be sent along with the query
+
+### _requestDelete
+
+Executes request to delete a record to the GraphQL endpoint.
+Can be replaced from a custom helper.
+
+#### Parameters
+
+*   `operation` **[string][4]** 
+*   `data` **any** of the record to be deleted.
+
 [1]: https://github.com/rosiejs/rosie
 
 [2]: https://www.npmjs.com/package/faker