saffron-ai 1.3.2 → 1.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +158 -0
- package/dist-pkg/cli.js +411 -394
- package/package.json +1 -1
- package/skills/saffron/SKILL.md +2 -2
- package/skills/saffron/references/api.md +37 -1
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "saffron-ai",
|
|
3
|
-
"version": "1.3.
|
|
3
|
+
"version": "1.3.3",
|
|
4
4
|
"description": "End-to-end tests written in plain language: an AI agent records each scenario once, replays run at zero tokens, and broken replays heal into proposals you review",
|
|
5
5
|
"scripts": {
|
|
6
6
|
"test": "vitest run",
|
package/skills/saffron/SKILL.md
CHANGED
|
@@ -3,14 +3,14 @@ name: saffron
|
|
|
3
3
|
description: Create, translate, or edit Saffron test scenarios in .saffron or .feature files, browser UI tests and HTTP API tests (@api) alike, including validating their recordings and replay. Use when asked to author or fix these test files in a Saffron project. Do not activate for ordinary application development, generic Cucumber projects, or requests only to run existing tests, inspect reports, review proposals, or configure Saffron.
|
|
4
4
|
license: Saffron Free Use License v1.0 (see the saffron-ai package LICENSE)
|
|
5
5
|
metadata:
|
|
6
|
-
saffron-ai: "1.3.
|
|
6
|
+
saffron-ai: "1.3.3"
|
|
7
7
|
author: saffron-ai
|
|
8
8
|
homepage: https://saffron-ai.io
|
|
9
9
|
---
|
|
10
10
|
|
|
11
11
|
# Saffron
|
|
12
12
|
|
|
13
|
-
This skill describes saffron-ai 1.3.
|
|
13
|
+
This skill describes saffron-ai 1.3.3 (see the compatibility page: https://saffron-ai.io/docs/compatibility).
|
|
14
14
|
|
|
15
15
|
Apply this workflow only when asked to create, translate, or edit Saffron
|
|
16
16
|
scenarios in `.saffron` or `.feature` files. A Saffron mention or an
|
|
@@ -95,6 +95,37 @@ Feature: Customers API
|
|
|
95
95
|
sends the value and the recording keeps the token.
|
|
96
96
|
- **Outlines** work as for browser scenarios: write `<column>` in every
|
|
97
97
|
step whose request or check depends on the row.
|
|
98
|
+
- **Several requests at once, each with its own values** (an idempotency
|
|
99
|
+
test, two registrations racing) are one step with a table: a header
|
|
100
|
+
row, then a row per request, the first column naming it. Saffron sends
|
|
101
|
+
the request once per row, every row before any answers, and keeps each
|
|
102
|
+
response with its row whatever order they arrive in. Values all rows
|
|
103
|
+
share can come from a Background step's table, which sends nothing:
|
|
104
|
+
|
|
105
|
+
```gherkin
|
|
106
|
+
Background:
|
|
107
|
+
Given no-show registrations use these shared request details:
|
|
108
|
+
| reservation | {data:no-show.chargeable} |
|
|
109
|
+
| API key | {env:NOSHOW_API_KEY} |
|
|
110
|
+
| idempotency key | {unique:registration} |
|
|
111
|
+
|
|
112
|
+
Scenario: Correlate every response to its concurrent request
|
|
113
|
+
When I send these no-show registrations concurrently as "registrations":
|
|
114
|
+
| request | correlation ID |
|
|
115
|
+
| original | {unique:original} |
|
|
116
|
+
| duplicate | {unique:duplicate} |
|
|
117
|
+
Then every response in "registrations" should have status 200 or 202
|
|
118
|
+
And every response should echo its own request's correlation ID
|
|
119
|
+
And every 202 response should have a numeric "Retry-After" header
|
|
120
|
+
And at least one response should have status 200
|
|
121
|
+
And all 200 responses should have the same "/paymentReference"
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
Each `Then` says its scope: every response, those of one status
|
|
125
|
+
("every 200 response"), a named one ("the duplicate"), or at least
|
|
126
|
+
some. A filtered check holds when no response has that status, so pair
|
|
127
|
+
"every 200 response ..." with "at least one response should have
|
|
128
|
+
status 200". "The response" after a group names none of them, and fails.
|
|
98
129
|
|
|
99
130
|
## What gets recorded, and what to check in `saffron diff`
|
|
100
131
|
|
|
@@ -104,6 +135,8 @@ Feature: Customers API
|
|
|
104
135
|
| `expectStatus` | the status of the latest response in that step's session, exact (`404`) or a class (`2xx`) |
|
|
105
136
|
| `expectJson` | field checks by JSON Pointer, typed (`/months` equals `3` as a number) or present/absent; an entry of a list with `item` (`{"path": "/messages", "item": [{"path": "/name", "equals": "ann"}]}`), wherever it stands |
|
|
106
137
|
| `expectSchema` | the latest response against the pinned contract: the operation's response schema, or a named component schema |
|
|
138
|
+
| `request` with `group` | sent once per row of its step's table; `<row:column>` is a row's own cell, the first column names it |
|
|
139
|
+
| `expectGroup` | the group's responses: every one, those answering `status`, or the `members` named; each holds `statuses`, `json`, `header` or the contract with its own row's values; `same` a field they all share, non-empty; `atLeast` how many must hold it |
|
|
107
140
|
|
|
108
141
|
Before accepting a proposal, check that:
|
|
109
142
|
|
|
@@ -116,7 +149,10 @@ Before accepting a proposal, check that:
|
|
|
116
149
|
and "should match the contract" became an `expectSchema`;
|
|
117
150
|
- an error case expects the status the step names;
|
|
118
151
|
- an entry of a list is an `item` check, not a position (`/0/name`),
|
|
119
|
-
unless the step names the position ("the first order")
|
|
152
|
+
unless the step names the position ("the first order");
|
|
153
|
+
- a group's request writes each row's own value as `<row:column>`, and a
|
|
154
|
+
`Then` about every response is an `expectGroup` with no `members` or
|
|
155
|
+
`atLeast` it does not state, never a check of one response.
|
|
120
156
|
|
|
121
157
|
The agent verifies each check against the response before it records it,
|
|
122
158
|
so a recorded check held at recording time; review whether it is the
|