@cloudbeat/atlas-cli 0.3.0 → 0.4.0

package/dist/docs/ai-agents.md

@@ -0,0 +1,90 @@
+# Atlas for AI Agents
+
+**Describe the test. Let the agent build it.**
+
+Load testing has always had a gap between wanting a test and having one. Someone
+has to click through the flow, capture the requests, work out which values are
+per-session tokens, wire up the data files, size the run, and then interpret a
+report full of percentiles. It is a day of work for a thirty-second question:
+*will this hold up on launch day?*
+
+Atlas closes that gap. Every part of it — recording, test plans, data files,
+machines, runs, results — is reachable through one command-line tool that speaks
+JSON, and it ships with the guidance an AI coding assistant needs to use it
+properly. Point your assistant at Atlas and ask in plain language:
+
+> Record a checkout on our staging site, then run it with 500 users for ten
+> minutes and tell me where it breaks.
+
+It records the flow in a real browser, turns the recording into a genuine load
+test, checks the plan will actually report, smoke-runs it, asks you to confirm
+the real run, and comes back with what broke and at what point.
+
+<div class="tip custom-block" style="padding-top: 8px">
+
+New here? [Set it up](/agents/setup) in two commands, then see
+[what it can do](/agents/capabilities).
+
+</div>
+
+## Why this works with Atlas
+
+**One tool, every surface.** The agent does not need infrastructure credentials,
+a database, or your cloud account. Everything goes through the same CLI you use,
+so your permissions, your account's quota and your machine limits apply exactly
+as they do in the browser.
+
+**Machine-readable by design.** Every command takes `--json`. Failures print
+JSON on standard error, so clean output stays clean. `atlas schema --json`
+describes the entire command tree, so an agent can discover the surface instead
+of guessing at flags.
+
+**It knows the traps.** Load testing fails quietly. A data file that was never
+uploaded silently removes an entire thread group — with *zero* errors to explain
+it. A plan whose structure defeats reporting runs perfectly and tells you
+nothing. Fast responses that are really login pages look like excellent
+performance. The bundled guidance teaches an assistant to recognise all of these
+before you act on a number that means nothing.
+
+**The documentation travels with it.** `atlas docs` prints this documentation
+offline, so an agent in a sandbox or a CI container still has the manual.
+
+## What you get
+
+| | |
+|---|---|
+| **Recording without clicking** | The agent opens a real browser, reads the page as an accessibility tree, clicks through your flow and names each step — or hands the window to you for the parts only you can do, like a password or an MFA prompt. |
+| **A real test, not a replay** | It spots the values every simulated user would otherwise repeat — credentials, search terms, product ids — and moves them into a data file, so you are testing your application rather than your cache. |
+| **Plans that actually report** | Before anything runs, `atlas plan check` catches the three failures that produce an empty dashboard an hour later. |
+| **Sizing you can defend** | It reads your real capacity and explains what 500 users means in machines, because each machine runs the whole plan. |
+| **Results in sentences** | Not just p95 numbers: which step degraded, at what user count, and what the errors actually say. |
+| **CI that fails honestly** | Pass/fail thresholds, JUnit output and exit codes, written into your pipeline by the same assistant. |
+
+## Guardrails
+
+An agent that can start a load test can spend money and put load on production.
+The bundled guidance is explicit about this, and a good assistant will:
+
+- **Confirm before any real run.** Machines are powered on for a test and billed
+  to your account. Smoke tests are cheap and encouraged; a 20-machine run is not
+  something to start on an assumption.
+- **Never stop a run it did not start.** Someone else's test is someone else's.
+- **Ask rather than guess** about the things that change the result: which
+  project, how many users, how long, ramp-up, fixed machines or a cluster.
+- **Report what happened, not what should have.** Including failed thresholds, a
+  thread group that vanished, and its own uncertainty.
+- **Leave your files as it found them.** It downloads before it edits, and it
+  restores anything it changed for debugging — think time included.
+
+Everything an agent can do, you can do by hand with the same commands. Nothing
+here is required, and nothing here is hidden from you.
+
+## Which assistants
+
+| Assistant | How it reads the guidance |
+|---|---|
+| **Claude Code** | Skills — loaded automatically when your request is about load testing |
+| **GitHub Copilot, Cursor, Codex, Gemini CLI, Windsurf** | `AGENTS.md` in your project root |
+| **Anything else** | `atlas schema --json`, `atlas docs`, and `--json` on every command |
+
+[Set it up →](/agents/setup)

package/dist/docs/auth.md

@@ -0,0 +1,74 @@
+# Signing In
+
+## For everyday use
+
+```sh
+atlas login --url https://atlas.cloudbeat.io -u you@company.com
+```
+
+You'll be prompted for your password. The session lasts **8 hours**.
+
+Credentials are stored in `~/.atlas/config.json`, readable only by you.
+
+```sh
+atlas whoami     # who am I, and in which account
+atlas logout     # forget the credential
+```
+
+## For CI: API tokens
+
+A password prompt is no use in a pipeline, and storing a human's password is
+worse. Create an **API token** instead.
+
+On your Atlas instance, go to **Profile → API tokens** and create one. The token
+is shown **once** — copy it then.
+
+```sh
+atlas login --token atlas_pat_...
+```
+
+Or, better in CI, pass it as an environment variable and don't store it at all:
+
+```sh
+export ATLAS_TOKEN=atlas_pat_...
+```
+
+### What a token can and can't do
+
+A token acts as **the person who created it**, with the same permissions and in
+one fixed account. If that person's access is reduced or removed, the token
+follows immediately.
+
+Two things it deliberately **cannot** do:
+
+- create more tokens
+- change a password
+
+So a leaked pipeline token can't mint replacements for itself or lock you out.
+Revoking it on the Profile page stops it working at once.
+
+If you belong to several accounts, switch to the one the pipeline should use
+*before* creating the token. The token is fixed to whichever account was active
+when it was made.
+
+A token is a credential. Put it in your CI system's secret store, not in the
+repository, and not in a build log.
+
+## Settings and precedence
+
+Command-line options win over environment variables, which win over the saved
+configuration file.
+
+| Variable | Meaning |
+|---|---|
+| `ATLAS_URL` | your Atlas instance |
+| `ATLAS_TOKEN` | API token |
+| `ATLAS_PROJECT` | project to use when `--project` is omitted |
+| `ATLAS_DEBUG` | set to any value to print full error details |
+
+To avoid repeating yourself:
+
+```sh
+atlas config set url https://atlas.cloudbeat.io
+atlas config set project SHOP
+```

package/dist/docs/automation.md

@@ -0,0 +1,150 @@
+# Building Your Own Integration
+
+The same surfaces the bundled skills use are there for your own scripts, bots
+and agents. No SDK, no separate API client: the CLI is the integration point,
+and everything it does is scriptable.
+
+## JSON everywhere
+
+```sh
+atlas projects --json
+atlas capacity --json
+atlas results SHOP-20260917-101500 --json
+```
+
+Every command takes `--json`. On failure it prints the error as JSON on
+**standard error**, so standard out stays parseable:
+
+```json
+{"error":{"message":"Project 'SHOPP' not found","code":"not_found"}}
+```
+
+Exit codes are the contract, and they do not change with `--json`:
+
+| Code | Meaning |
+|---|---|
+| `0` | Success, and any thresholds passed |
+| `1` | A `--gate` threshold was breached — the test ran, the numbers failed |
+| `2` | A usage or authentication problem: bad flags, unknown project, not signed in |
+| `3` | The run failed or timed out, or Atlas could not be reached |
+
+`1` and `3` are deliberately distinct: a breached threshold is a result, an
+unreachable server is not.
+
+## Discovering the surface
+
+Rather than hard-coding flags, read them:
+
+```sh
+atlas schema --json
+```
+
+This returns every command, argument and option, with its description. Two
+fields worth knowing: `takesValue` says whether the flag expects a value, and
+`valueOptional` whether that value may be omitted. Neither means the flag itself
+is mandatory.
+
+## Documentation, offline
+
+```sh
+atlas docs                  # list the topics
+atlas docs troubleshooting  # print one
+```
+
+The documentation is bundled inside the package, so it works with no network —
+in CI, in a container, in a sandboxed agent.
+
+## Checking a plan without running it
+
+```sh
+atlas plan check ./checkout.jmx -p SHOP --json
+```
+
+Read-only and advisory: it never rewrites your plan. Useful as a pre-commit
+hook or a pipeline step, because it exits non-zero when the plan would produce
+no usable results.
+
+## Driving the recorder from code
+
+The recorder can be controlled step by step, which is what lets an agent click
+through a flow. It is also a perfectly good scripting interface:
+
+```sh
+atlas-rec start https://shop.example.com -o flow.jmx -n "Checkout"
+atlas-rec snapshot                              # the page, as an accessibility tree
+atlas-rec tx new "Search"                       # name the step that follows
+atlas-rec click --role button --name "Search"
+atlas-rec fill "laptop" --selector "#q"
+atlas-rec wait --selector ".results"
+atlas-rec state --full                          # everything captured so far
+atlas-rec stop                                  # writes the plan and the capture
+```
+
+The window stays visible and a person can click in the same session — the two
+mix freely, so a script can pause for a human to sign in and then carry on.
+
+The control channel is **local to your machine**: it listens on the loopback
+interface, on an ephemeral port, and requires a per-session token that is kept
+out of the process list. It only accepts commands while a recording you started
+is open.
+
+Afterwards, a capture can be re-read and rebuilt without recording again:
+
+```sh
+atlas-rec inspect flow.capture.json    # transactions, hosts, response codes
+atlas-rec suggest flow.capture.json    # per-session values, with evidence
+atlas-rec generate flow.capture.json -o flow.jmx --correlations rules.json
+```
+
+The same capture always produces the same plan, so regenerating is safe to
+repeat and easy to review.
+
+## Authentication for automation
+
+Use an API token, never a password:
+
+```sh
+export ATLAS_TOKEN=atlas_pat_...
+export ATLAS_URL=https://atlas.cloudbeat.io
+export ATLAS_PROJECT=SHOP
+atlas run --servers 4 --gate "error_rate<1,p95<800" --junit results.xml
+```
+
+A token acts as its creating user, in one account. Revoking it takes effect
+immediately, and a token cannot mint further tokens or change a password — so a
+leaked CI credential cannot entrench itself. See
+[Signing In](/cli/authentication).
+
+## A worked example
+
+Everything together — gate a deployment on a load test:
+
+```sh
+#!/usr/bin/env bash
+set -euo pipefail
+
+# Refuse to run a plan that could not report
+atlas plan check -p SHOP --jmx checkout.jmx --json >/dev/null
+
+# `|| rc=$?` keeps `set -e` from aborting on a breached threshold, which is a
+# result we want to act on rather than a crash
+rc=0
+atlas run -p SHOP --jmx checkout.jmx --servers 4 \
+  --args "-Jthreads=125 -Jrampup=300" \
+  --gate "error_rate<1,p95<800" \
+  --junit results.xml --json > run.json || rc=$?
+
+case $rc in
+  0) echo "passed: $(jq -r .testId run.json)" ;;
+  1) echo "thresholds breached"; jq '.gates[] | select(.passed | not)' run.json; exit 1 ;;
+  *) echo "the test did not run (exit $rc)"; exit "$rc" ;;
+esac
+```
+
+With `--wait` — which `--gate` and `--junit` imply — the JSON on standard out
+carries `testId`, `state`, `durationSec`, the overall figures and every gate
+result. It is written **before** a breached threshold sets the exit code, so
+`run.json` is complete either way.
+
+[Running in CI](/cli/ci) has the pipeline-ready version, including GitHub
+Actions.

package/dist/docs/ci.md

@@ -0,0 +1,105 @@
+# Running in CI
+
+The point of running Atlas from CI is to catch a performance regression the same
+way you catch a failing unit test — automatically, on the build that caused it.
+
+## A complete example
+
+```sh
+export ATLAS_URL=https://atlas.cloudbeat.io
+export ATLAS_TOKEN=atlas_pat_...
+
+# Only if the plan or its data live in this repository and may have changed.
+atlas files upload -p SHOP ./checkout.jmx ./data/users.csv
+
+atlas run -p SHOP --jmx checkout.jmx --servers 5 \
+          --gate "error_rate<1,p95<2000" \
+          --junit results.xml
+```
+
+If your test plan is maintained in Atlas rather than in the repository, drop the
+upload step — the run command works on its own.
+
+## Pass/fail thresholds
+
+`--gate` fails the command when a threshold is breached, checked against the
+run's overall totals.
+
+| Metric | Meaning |
+|---|---|
+| `error_rate` | percentage of failed requests |
+| `avg` | average response time, ms |
+| `p90`, `p95`, `p99` | percentile response times, ms |
+| `max` | slowest response, ms |
+| `throughput` | requests per second |
+| `requests` | total requests |
+
+Use `<`, `<=`, `>` or `>=`. Separate several with commas, or repeat `--gate`.
+
+```sh
+--gate "error_rate<1,p95<2000,throughput>50"
+```
+
+Setting a gate implies `--wait` — there is nothing to check until the test has
+finished.
+
+`error_rate` and a percentile. Averages hide the tail: a p95 catches the slow
+requests a handful of users actually felt, which an average smooths away.
+
+## Exit codes
+
+| Code | Meaning |
+|---|---|
+| `0` | the test ran and every threshold passed |
+| `1` | the test ran but a `--gate` threshold was breached |
+| `2` | something was wrong with the request — bad option, unknown project, not signed in |
+| `3` | the test failed or timed out, or the server could not be reached |
+
+The distinction that matters: **`1` means your system was too slow, `2` and `3`
+mean the test never produced a verdict.** Treating them the same makes a broken
+pipeline look like a performance regression.
+
+## JUnit report
+
+```sh
+atlas run -p SHOP --servers 5 --junit results.xml
+```
+
+Writes a standard JUnit file with one test case per threshold, so CI shows each
+one as a separate pass or fail. The run's headline numbers are included as
+properties. Point your CI's test reporter at it and gates appear alongside your
+other tests.
+
+## Example: GitHub Actions
+
+```yaml
+- name: Load test
+  env:
+    ATLAS_URL: https://atlas.cloudbeat.io
+    ATLAS_TOKEN: ${{ secrets.ATLAS_TOKEN }}
+  run: |
+    npm install -g @cloudbeat/atlas-cli
+    atlas run -p SHOP --servers 5 \
+              --gate "error_rate<1,p95<2000" \
+              --junit results.xml
+
+- name: Publish results
+  if: always()
+  uses: actions/upload-artifact@v4
+  with:
+    name: load-test
+    path: results.xml
+```
+
+## Things worth knowing
+
+**Load tests take minutes, not seconds.** Give the job a generous timeout, and
+consider running it on a schedule or on merges to main rather than on every
+push.
+
+**Machines may be busy.** If another test holds the generators you asked for,
+your run is queued and starts when they free up. Set `--timeout` to bound how
+long you're willing to wait.
+
+**A queued or failed run is not a regression.** That's the exit-code distinction
+above — don't let a `3` open a performance bug.

package/dist/docs/clusters.md

@@ -0,0 +1,48 @@
+# Clusters
+
+A **cluster** is elastic capacity: instead of fixed machines that sit there
+between tests, units are created when a test starts and destroyed when it ends.
+
+You pay for what a test uses, and you're not limited by how many machines you
+happened to set up in advance.
+
+## When to use one
+
+Use a cluster when your load varies a lot between tests, or when you want a
+large run occasionally without keeping that capacity around. Use fixed
+[servers](/generators/adding) when you want predictable, always-available
+machines — or when your test needs to come from a specific network location.
+
+## Setting one up
+
+A cluster runs on a Docker host you provide. Atlas talks to it through a small
+**controller** you install on that host, which connects outbound — so the host
+needs no inbound access, exactly like a load generator.
+
+Go to **Fleet → Clusters**, create a cluster, and run the install command it
+gives you on your Docker host. When the controller connects, the cluster shows
+as ready.
+
+## Running on a cluster
+
+On the **Run Test** tab choose **Cluster** instead of Servers, and say how many
+units you want:
+
+```bash
+atlas run -p SHOP --cluster my-cluster --units 10
+```
+
+Each unit is one JMeter instance, equivalent to one load generator. Ten units
+running a 100-thread plan is 1,000 concurrent users.
+
+## While it runs
+
+Units register themselves as they come up, then the test deploys and starts as
+usual. When the test finishes, every unit is destroyed.
+
+If a run ends unexpectedly, check the Clusters tab to confirm nothing was left
+behind.
+
+Clusters are newer than fixed servers. If a run needs to be reliable — a
+scheduled regression test, a customer demo — fixed machines are the safer
+choice today.

package/dist/docs/commands.md

@@ -0,0 +1,155 @@
+# Commands
+
+Every command takes `--json`, and `-p, --project <name>` where a project
+applies. Run `atlas <command> --help` for the full list of options. A failure
+under `--json` prints the error as JSON on standard error.
+
+## Overview
+
+| Command | What it does |
+|---|---|
+| `atlas login` | sign in with a password or an API token |
+| `atlas logout` | forget the stored credential |
+| `atlas whoami` | show who you're signed in as |
+| `atlas projects` | list the projects you can access |
+| `atlas files list` | list a project's files |
+| `atlas files upload <files...>` | upload test plans and data files |
+| `atlas files download <filename>` | download one file |
+| `atlas files rm <filename>` | delete a file |
+| `atlas run` | start a test |
+| `atlas status <testId>` | check whether a test is still running |
+| `atlas results <testId>` | show the aggregation report |
+| `atlas stop <testId>` | stop a running test |
+| `atlas tests` | list a project's recent runs |
+| `atlas timeseries <testId>` | active users, response time and error rate over time |
+| `atlas errors <testId>` | the distinct errors a test produced |
+| `atlas capacity` | what you can run on: machines, clusters and usage limits |
+| `atlas plan check [file]` | check a test plan will run and report before you use it |
+| `atlas record [url]` | record a browser session |
+| `atlas schema` | describe every command and option (for scripts and AI agents) |
+| `atlas skills list` / `install` | install the AI-assistant guidance into your project |
+| `atlas docs [topic]` | read the documentation without leaving the terminal |
+| `atlas config get` / `set` | default instance URL and project |
+
+## Files
+
+```sh
+atlas files upload -p SHOP ./checkout.jmx ./data/users.csv
+atlas files list -p SHOP
+atlas files download -p SHOP checkout.jmx
+atlas files rm -p SHOP old-data.csv --yes
+```
+
+Uploading a file whose name already exists replaces it. `rm` asks for
+confirmation, and refuses without `--yes` when there is no terminal — so a
+pipeline can't delete a file by accident.
+
+See [Test Files](/guide/test-files) for how files are laid out and divided
+between machines.
+
+## Running
+
+```sh
+atlas run -p SHOP --jmx checkout.jmx --servers 5 --wait
+```
+
+| Option | Meaning |
+|---|---|
+| `--servers <n>` | use any n available load generators |
+| `--server <name>` | pick one by name; repeat for several |
+| `--cluster <name> --units <n>` | run on n units in a cluster |
+| `--jmx <filename>` | which plan to run (needed only if the project holds more than one) |
+| `--args "<jmeter args>"` | extra JMeter arguments |
+| `--no-split` | send whole data files to every machine |
+| `--wait` | follow the run and print the report at the end |
+| `--timeout <duration>` | how long `--wait` waits, e.g. `30m` (default `2h`) |
+| `--gate <thresholds>` | fail on a threshold breach — see [Running in CI](/cli/ci) |
+| `--junit <file>` | write a JUnit report |
+
+Without `--wait` the command returns as soon as the test is submitted and prints
+the test ID.
+
+## Checking on a run
+
+```sh
+atlas status SHOP-20260818-194201
+atlas results SHOP-20260818-194201
+atlas results SHOP-20260818-194201 --json
+atlas stop SHOP-20260818-194201
+```
+
+`results` prints the aggregation report — one row per request, plus totals.
+
+`stop` reports what actually happened rather than always claiming success. If a
+machine couldn't be reached, it says so and the run stays open, so a test can't
+keep running unnoticed.
+
+## Checking a test plan
+
+```sh
+atlas plan check ./checkout.jmx -p SHOP     # a local file, against the project
+atlas plan check -p SHOP --jmx checkout.jmx # the plan already in the project
+```
+
+Catches the problems that otherwise show up as an empty dashboard after the run:
+a data file the plan reads but the project doesn't have, a plan Atlas can't
+attach its reporting to, a structure broken by hand-editing, or no enabled thread
+group. It exits non-zero when the plan wouldn't produce results, so CI can stop
+before spending machine time.
+
+It also tells you whether the plan's user count can be set at run time, and
+reports the things Atlas fixes for you — data-file paths from another machine,
+and GUI listeners left switched on.
+
+## Analysing a run
+
+```sh
+atlas tests -p SHOP --limit 10          # recent runs, newest first
+atlas timeseries SHOP-20260818-194201   # users, response time and errors over time
+atlas errors SHOP-20260818-194201       # every distinct error, with its share
+atlas errors SHOP-20260818-194201 --full --label "01.01 /home"
+```
+
+`timeseries` shows the shape of a run — where users peaked and where response
+times started to climb. `errors` shows one row per distinct failure with the
+request and response behind it, which is usually enough to tell a broken test
+plan from a struggling application.
+
+## What you can run on
+
+```sh
+atlas capacity
+```
+
+Machines and clusters available to you, how many are free, and how much of your
+usage allowance is left. Worth checking before sizing a large test.
+
+## AI assistants
+
+```sh
+atlas skills list              # what this version carries
+atlas skills install           # into this project (.claude/skills/ + AGENTS.md)
+atlas skills install --user    # for every project on this machine
+atlas skills install --force   # replace files you have already edited
+```
+
+Installs the guidance an AI coding assistant needs to drive Atlas properly.
+See [Atlas for AI Agents](/agents/) for what that enables, and
+[Setting It Up](/agents/setup) for the details.
+
+## Recording
+
+```sh
+atlas record https://example.com -o flow.jmx
+atlas record https://example.com -o flow.jmx --upload -p SHOP
+```
+
+| Option | Meaning |
+|---|---|
+| `-o, --out <file>` | output file (an existing plan is added to, not replaced) |
+| `-n, --name <name>` | name for the recorded scenario |
+| `--cookies <file>` | preload cookies to skip a login step |
+| `--profile <dir>` | reuse a browser profile for a persistent session |
+| `--upload -p <project>` | upload the finished plan into a project |
+
+See [Recorder](/recorder/).