@schwabyio/ata 0.0.0-stage → 0.20.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 schwabyio
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,353 @@
-# Temporary Holding Version
+# ata
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The command-line runner of Accelerator Test Automation. Tests for HTTP APIs are plain YAML
+files that live in your repository beside the code they test, and read and diff as text
+in a pull request. `ata` runs them from a terminal or in CI, and writes JUnit, HTML and
+JSON reports. Accelerator, the desktop app, edits and runs the same files on the same engine.
+
+## Install
+
+You need Node.js 22.12 or later.
+
+```sh
+npm install -g @schwabyio/ata
+```
+
+Or add it to a project's `package.json`, and run it there with `npx ata`:
+
+```sh
+npm install --save-dev @schwabyio/ata
+```
+
+The package has no dependencies to install: everything `ata` uses is bundled into it.
+
+## A first project
+
+A project is a folder holding a `collections/` folder, an `environments/` one and a
+`settings.yml`. The smallest project `ata` runs is three files:
+
+```yaml
+# collections/health.yml
+id: health
+steps:
+  - name: service is up
+    GET: '{{baseUrl}}/health'
+    tests: |
+      ata.expectResponseStatusCodeToBe(200)
+```
+
+```yaml
+# environments/local.yml
+vars:
+  baseUrl: http://localhost:8080
+```
+
+```yaml
+# settings.yml: how ata runs the project
+environmentType: local
+```
+
+Then, from the project folder:
+
+```sh
+ata all
+```
+
+## Collections
+
+A collection is a list of requests, called steps. They run in order, and a later step can
+use a value an earlier one saved, like `sessionId` here:
+
+```yaml
+# collections/checkout.yml
+id: checkout
+headers:
+  Authorization: 'Bearer {{apiKey}}'
+steps:
+  - name: create session
+    POST: '{{baseUrl}}/sessions'
+    body:
+      json: |
+        { "amount": 1200 }
+    tests: |
+      ata.expectResponseStatusCodeToBe(201)
+      ata.expectResponseBodyToHaveProperty('id', 'sessionId', 'setAsCollectionVariable')
+
+  - name: read it back
+    GET: '{{baseUrl}}/sessions/{{sessionId}}'
+    tests: |
+      ata.expectResponseStatusCodeToBe(200)
+      ata.expectResponseBodyToHaveProperty('amount', 1200)
+```
+
+An environment supplies the variables for one target. A secret's value is never written
+in a file: it comes from the process environment variable of the same name, or from a
+`.env` file that is not committed.
+
+```yaml
+# environments/staging.yml
+vars:
+  baseUrl: https://staging.example.com
+  apiKey: { secret: true }
+```
+
+## Running
+
+```sh
+ata get                          # list what ata all would run
+ata all                          # run every collection
+ata smoke,checkout               # run these collections, in this order
+ata payments                     # run every collection in collections/payments/
+ata all --tags smoke --generateJUnitResults
+ata all --environmentType staging --limitConcurrency 8
+ata all --json                   # print the results as JSON instead of the table
+```
+
+Collections run in parallel up to `limitConcurrency`, each in a worker thread of its own.
+The steps within a collection always run in order.
+
+The exit code is `0` when everything passed, `1` when something failed, and `2` when
+`ata` could not run at all, such as with a bad setting or an unknown collection.
+`ata get` exits `1` when a collection is broken, or has a `use:`, an `extends:` or a
+file its body sends that a run would stop at.
+
+## Settings
+
+Settings come from `settings.yml`, then `ATA_*` environment variables, then flags on the
+command line, each overriding the one before. An unknown setting is an error.
+`ata --help` lists them all, and `ata get` shows where each one came from:
+
+| Setting                  | Default        | What it does                                                   |
+| ------------------------ | -------------- | -------------------------------------------------------------- |
+| `environmentType`        | none           | The environment to run against: `environments/<name>.yml`      |
+| `limitConcurrency`       | `1`            | How many collections run at once                               |
+| `timeoutCollection`      | `3600000`      | Milliseconds a collection may take before it is stopped        |
+| `bail`                   | `false`        | Stop a collection at its first failing step                    |
+| `tags`                   | none           | Run only what has one of these tags                            |
+| `notTags`                | none           | Leave out what has one of these tags                           |
+| `generateJUnitResults`   | `false`        | Write `<testResultsBasePath>/junit/junit.xml`                  |
+| `generateJsonResults`    | `false`        | Write `<testResultsBasePath>/json/results.json`                |
+| `generateHtmlResults`    | `false`        | Write `<testResultsBasePath>/html/`, a summary and a page each |
+| `autoOpenTestResultHtml` | `false`        | Write the HTML report and open it when the run ends            |
+| `testResultsBasePath`    | `test-results` | Where reports go: relative to the project folder, or absolute  |
+
+A collection stopped at `timeoutCollection` still runs its teardown, for up to 60 seconds,
+so it cleans up after itself; the failure says how the teardown went.
+
+The results folder is emptied before every run, so everything in it is from the last one.
+`ata` only ever deletes a folder that holds nothing but its own reports.
+
+In a monorepo, projects that share a global project (`uses:` in `project.yml`) share its
+`settings.yml` too. It comes before the project's own, so a project changes a shared
+setting by setting it again in its own file. Each project still needs a `settings.yml`,
+even an empty one.
+
+## Project rules
+
+A `rules.yml` beside `project.yml` writes down how a project's files are named, laid out
+and written, so everyone working on it, people and coding agents alike, keeps it the
+same:
+
+```yaml
+# rules.yml
+ids:
+  collections: kebab-case # or a pattern, such as '{folder}-[a-z0-9-]+'
+layout:
+  folders: required # every collection in a folder of collections/
+docs:
+  collections: required
+tags:
+  allowed: [smoke, regression]
+steps:
+  names: required
+tests:
+  only: [ata] # tests call the ata.* functions, and nothing else
+  statusCode: required
+guide: |
+  Name each step for what it proves.
+```
+
+```sh
+ata rules                        # the rules in effect, where each came from, what each means, the guide
+ata lint                         # check every file against them
+ata lint --json                  # the findings as JSON, for tools and agents
+```
+
+`ata lint` prints each finding at its file and line, with the rule it breaks, and exits
+`1` when it finds one. Rules never change a run: a file that breaks one still runs. A
+global project's `rules.yml` is shared like its `settings.yml`, and a project changes or
+turns off (`null`) any rule in its own file. Every rule is in
+[SPEC.md §1.4](https://github.com/schwabyio/accelerator/blob/main/SPEC.md#14-rulesyml).
+
+## In CI
+
+```yaml
+# .github/workflows/api-tests.yml (steps)
+- uses: actions/setup-node@v4
+  with:
+    node-version: 22
+- run: npm install -g @schwabyio/ata
+- run: ata lint
+- run: ata all --environmentType staging --generateJUnitResults
+  env:
+    apiKey: ${{ secrets.STAGING_API_KEY }}
+```
+
+## From Playwright and other code
+
+The package is also a library. A test written in code can run a collection or a request
+set, then use the values it saved, such as a token or the id of a user it created, in a
+browser test.
+
+### Playwright
+
+`@schwabyio/ata/playwright` gives you Playwright's `test` with an `ata` fixture. Point it
+at the project in `playwright.config.ts`:
+
+```ts
+// playwright.config.ts
+import { defineConfig } from '@playwright/test'
+import type { AcceleratorConfig } from '@schwabyio/ata/playwright'
+
+export default defineConfig<{}, AcceleratorConfig>({
+  use: {
+    accelerator: { project: '../api-tests', environment: 'staging' }
+  }
+})
+```
+
+```ts
+// tests/dashboard.spec.ts
+import { test, expect } from '@schwabyio/ata/playwright'
+
+test('a new user sees their dashboard', async ({ page, ata }) => {
+  // requests/create-user.yml, run as a use: step would run it
+  const user = await ata.use('create-user', { plan: 'pro' })
+
+  await page.goto(`/users/${user.values.userId}`)
+  await expect(page.getByRole('heading')).toHaveText('Welcome')
+
+  // collections/billing.yml, starting from what create-user saved
+  await ata.run('billing', { vars: user.values })
+})
+```
+
+- Each `ata.run` and `ata.use` call is a step in Playwright's report. Each request is a
+  step inside it, linked to its step in the YAML, with the request and response attached.
+- A run that fails fails the test at the line that called it, with what failed. With
+  `soft: true` the test goes on and fails at the end, as `expect.soft` does.
+- `project` is relative to the folder `playwright.config.ts` is in, and defaults to it.
+  `environment`, `flags`, `bail` and `timeoutCollection` go beside it. The project opens
+  once per worker, so an environment's flag command runs once per worker.
+- A run counts toward the test's timeout, and one still going when the test ends is
+  stopped.
+- To use this `test` with fixtures of your own, combine them with Playwright's
+  `mergeTests`.
+
+It needs `@playwright/test` 1.51 or later, which installing `ata` does not install.
+
+**When the project is in another repository**, check it out beside the app's and read
+its path from an environment variable, so a checkout anywhere else can say where it is:
+
+```ts
+// playwright.config.ts
+export default defineConfig<{}, AcceleratorConfig>({
+  use: {
+    accelerator: { project: process.env.ATA_PROJECT ?? '../api-tests' }
+  }
+})
+```
+
+`project` names the folder holding `collections/`. A project that `uses:` a global
+project needs the repository checked out whole, since the global project is found from
+it. In CI, check out both:
+
+```yaml
+# .github/workflows/e2e.yml (steps)
+- uses: actions/checkout@v4
+  with:
+    path: web
+- uses: actions/checkout@v4
+  with:
+    repository: your-org/api-tests
+    path: api-tests
+    token: ${{ secrets.API_TESTS_TOKEN }} # when that repository is private
+- uses: actions/setup-node@v4
+  with:
+    node-version: 22
+- run: npm ci && npx playwright install --with-deps
+  working-directory: web
+- run: npx playwright test
+  working-directory: web
+  env:
+    ATA_PROJECT: ${{ github.workspace }}/api-tests
+    apiKey: ${{ secrets.STAGING_API_KEY }}
+```
+
+### Any other code
+
+```js
+import { openProject } from '@schwabyio/ata'
+
+const project = await openProject('api-tests', { environment: 'staging' })
+const login = await project.use('login', { username: 'alice' })
+const checkout = await project.run('checkout', { vars: login.values })
+if (!checkout.passed) throw new Error(checkout.failures)
+```
+
+**`openProject(folder, options)`** opens the project in `folder`, the one holding
+`collections/`. The options are `environment`, `flags`, `bail` and `timeoutCollection`.
+Each one you leave out comes from `settings.yml`, the same way `ata` reads it, but here
+the project does not need a `settings.yml`.
+
+**`project.run(collection, options)`** runs a collection the way `ata` runs it: setup,
+then the steps once for each data row, then teardown. Name the collection by its id or
+its place in `collections/` (`checkout/sessions`). `tags` and `notTags` don't apply,
+since you name what runs, and a collection or step with `exclude: true` runs too. The options:
+
+| Option     | What it does                                                                                                                         |
+| ---------- | ------------------------------------------------------------------------------------------------------------------------------------ |
+| `steps`    | Run only these steps, named as reports name them or numbered from 1. They run in the file's order, and setup and teardown still run. |
+| `vars`     | Values the run starts with. They sit over the environment and under a data row.                                                      |
+| `bail`     | Stop at the first failing step.                                                                                                      |
+| `signal`   | An `AbortSignal` that stops the run. Its teardown still runs after it, for up to 60 seconds.                                         |
+| `onResult` | `(result, step) => void`, called as each request finishes.                                                                           |
+
+**`project.use(name, params, options)`** runs reusable requests as a `use:` step would:
+`login` is `requests/login.yml`, in the project or its global project. `params` are what
+`with:` gives, and a param you leave out takes its default. The options are those of
+`run`, without `steps`.
+
+Both resolve to the same outcome, however the steps fare:
+
+| Field      | What it holds                                                                                                                                                   |
+| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `passed`   | `true` when nothing failed or errored and the run finished.                                                                                                     |
+| `values`   | Every value the run set or captured, by name: `ata.set`, and checks that save what they read, such as `'setAsCollectionVariable'`. Secrets are not hidden here. |
+| `summary`  | Totals, and every request's result as ata's JSON report holds it, with secrets shown as `[secret: NAME]`.                                                       |
+| `steps`    | Where each result's step is: its file and line.                                                                                                                 |
+| `failures` | What went wrong, as `ata` prints it under Failures. Empty when the run passed.                                                                                  |
+| `error`    | Why the run did not start or finish: a file that will not load, `timeoutCollection`, or a cancel. Otherwise `null`.                                             |
+
+They reject only when a collection, step or reusable requests file doesn't exist, or a
+value isn't a string, number, boolean or null. Types ship with the package.
+
+## The file format
+
+[SPEC.md](https://github.com/schwabyio/accelerator/blob/main/SPEC.md) specifies every file,
+every key, and every rule that makes a file invalid. It is written for people and coding
+agents alike, and ends with a complete project to start from. It also ships in this
+package as `dist/SPEC.md`, and `ata`'s messages cite its sections, as in "(SPEC.md §2.5)".
+
+[FUNCTIONS.md](https://github.com/schwabyio/accelerator/blob/main/FUNCTIONS.md) documents every
+`ata` function that `tests` and `before.script` can call, with examples. It ships as
+`dist/FUNCTIONS.md`.
+
+[SETUP-AND-TEARDOWN.md](https://github.com/schwabyio/accelerator/blob/main/SETUP-AND-TEARDOWN.md)
+is an overview of setup and teardown: how they run, when to use them, and when plain steps
+or reusable requests do better.
+
+## License
+
+MIT. The packages bundled into `ata` keep their own licenses, collected in
+`dist/THIRD_PARTY_NOTICES.txt`.