apify-test-tools 0.10.2-beta.2 → 0.10.2-beta.21

package/README.md

@@ -1,8 +1,71 @@
 # Apify Test Tools
 
-[Contributing link](CONTRIBUTING.md)
+Build, test and release Apify Actors from a Git repository or local source code. The package has three parts:
 
-## Getting Started
+1. **A vitest library** for writing platform tests that run your Actors on Apify and check the results:
+
+    ```ts
+    testActor('myteam/web-scraper', 'scrapes the start URL', async ({ expect, run }) => {
+        const runResult = await run({ input: { startUrls: [{ url: 'https://apify.com' }] } });
+        await expect(runResult).toFinishWith({ datasetItemCount: { min: 1, max: 10 } });
+    });
+    ```
+
+2. **A CLI** that finds the Actors a change affects, builds them, and reports the results. It runs anywhere, so you
+   can build your own pipeline on it in any CI:
+
+    ```bash
+    npx apify-test-tools get-affected-actors --base origin/master \
+      | jq '{ actors: (.functionallyChangedActors + .testChangedActors) }' \
+      | npx apify-test-tools build --version-number 0.99
+    ```
+
+3. **Ready-made GitHub Actions workflows** built on the CLI and the library, so a GitHub repo needs only a few lines
+   per workflow:
+
+    ```yaml
+    jobs:
+        buildDevelAndTest:
+            uses: apify/apify-test-tools/.github/workflows/public_pr-build-test.yaml@workflows-v0
+            secrets:
+                TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}
+                # One entry per secret apify-test-tools.config.json names, today each `tokenEnvVar`
+                CONFIG_SECRETS: |
+                    { "APIFY_TOKEN_MYTEAM": "${{ secrets.APIFY_TOKEN_MYTEAM }}" }
+    ```
+
+Most repos only need the library and the workflows: the workflows call the CLI for you. Use the CLI directly only to
+build your own pipeline, e.g. on a CI platform other than GitHub, or to build and test locally.
+
+## Only what changed is retested
+
+Builds and platform tests cost time and money, so the tool skips everything a change can't affect:
+
+- **Only affected Actors.** A changed file only counts for the Actors whose build context it falls in, minus their
+  `.dockerignore`. In a monorepo, a change to one Actor doesn't build or test the others.
+- **Cosmetic changes skip tests.** README, CHANGELOG and formatting-only schema changes don't trigger tests on a PR;
+  they're only rebuilt when released.
+- **Only new changes on a PR.** Each push to a PR retests only the Actors that changed since its last green run, not
+  the whole PR again.
+- **Only their tests.** Only the tests of the Actors that were built run.
+
+See [Change detection](docs/cli.md#change-detection) for the exact rules.
+
+This README covers what you need to get a repo running. The full reference lives in `docs/`:
+
+- [Vitest library API](docs/lib.md) — `testActor`, `testStandbyActor`, `RunTestResult`, matchers, running tests locally
+- [CLI API](docs/cli.md) — every command, change detection, releasing, building locally
+- [Workflow API](docs/workflows.md) — every GitHub workflow, its inputs and secrets, versioning
+
+[Contributing](CONTRIBUTING.md)
+
+## Getting started
+
+Your Actors can be written in any programming language, e.g. Python: anything the platform builds from a Dockerfile
+works. Only the tests and this tooling run on Node.js, so all a repo needs is a `package.json` at the root with
+`apify-test-tools` and `vitest` in its `devDependencies`, and tests written in TypeScript or JavaScript. If you use the
+[PR workflow](docs/workflows.md#public_pr-build-testyaml), also define the `build-check`, `lint` and `test` scripts
+its static checks run, e.g. with your language's own linter and unit tests.
 
 ### 1. Install the package
 
@@ -10,118 +73,87 @@
 npm i -D apify-test-tools
 ```
 
-- Requires `vitest` version `3.2.0` or later (uses [annotate](https://vitest.dev/guide/test-context.html#annotate))
+- Requires `vitest` version `3.2.4` or later (uses [annotate](https://vitest.dev/guide/test-context.html#annotate))
 - Make sure `target` and `module` in your `tsconfig.json`'s `compilerOptions` are set to `ES2022`
 - Every Actor must already have at least one build under its default build tag (usually `latest`), built
   from a Git repository version whose URL points to this repository. Build it once manually on the
-  platform before CI takes over. See [Repository check](#repository-check).
+  platform before CI takes over. See [Repository check](docs/cli.md#repository-check).
 
 ### 2. Create the config file
 
-Every repo that uses `apify-test-tools` must have an `apify-test-tools.config.json` file at the root. It lists the actors in the repo, their folders, their full names on the platform, and which env var holds the token to use for each. A top-level `mode` field picks the file's layout; leave it out to use the default `"legacy"` layout.
+Every repo must have an `apify-test-tools.config.json` file at the root that lists its Actors. See
+[Config file](#config-file) below.
 
-See [Config file](#config-file) for the available modes and every field.
+### 3. Set up Actor folders
 
-### 3. Set up actor folders
+Each Actor in the config must have a `.actor/actor.json` file. The `dockerContextDir` field in `actor.json` defines
+the build context boundary — this is what the tool uses to decide which files can affect the Actor's build (see
+[Change detection](docs/cli.md#change-detection)).
 
-Each actor in the config must have a `.actor/actor.json` file. The `dockerContextDir` field in `actor.json` defines the build context boundary — this is what the tool uses to determine which files can affect the actor's build.
+This example has two Actors that share the code at the repo root, and one standalone Actor with its own code:
 
 ```
 my-repo
 ├── apify-test-tools.config.json
+├── Dockerfile                      <- shared by web-scraper and email-sender
+├── package.json
+├── src/                            <- shared by web-scraper and email-sender
 ├── actors
 │   ├── web-scraper
-│   │   ├── .actor
-│   │   │   └── actor.json
-│   │   └── src/
+│   │   └── .actor
+│   │       └── actor.json          <- points to the repo root
 │   └── email-sender
+│       └── .actor
+│           └── actor.json          <- points to the repo root
+├── standalone-actors
+│   └── orchestrator
 │       ├── .actor
-│       │   └── actor.json
+│       │   └── actor.json          <- default context: its own folder
+│       ├── Dockerfile
+│       ├── package.json
 │       └── src/
 └── test
-    ├── unit
     └── platform
-        ├── core                    <- Core (hourly) tests
-        │   └── core.test.ts
-        ├── some.test.ts            <- Daily tests can be anywhere inside platform/
-        └── some-other.test.ts
+        ├── web-scraper.test.ts
+        ├── email-sender.test.ts
+        └── orchestrator
+            └── orchestrator.test.ts
 ```
 
-For a single-actor repo, set `"folder": "."` in the config and place `.actor/actor.json` at the repo root.
-
-### Change detection
-
-When a PR is opened or code is pushed, the tool determines which actors need to be built and tested based on the changed files. For each changed file, for each actor:
-
-1. **Sibling exclusion** — files inside another actor's `folder` are excluded first. This prevents an actor with broad context from being triggered by changes that belong to a sibling actor.
-2. **CHANGELOG classification** — a `CHANGELOG.md` file is always `cosmetic` (only triggers a release build, not tests), for every actor, regardless of context or folder. (See [issue #106](https://github.com/apify/apify-test-tools/issues/106).)
-3. **Context matching** — the file must fall within one of the actor's context paths (`dockerContextDir` from `actor.json` by default, or `overrideActorContext` from config if set). Files outside every context path are skipped.
-4. **Hardcoded ignore list, context-aware** — the file path is first "hoisted" relative to the context path it matched (e.g. a standalone actor's own `.eslintrc` is checked as just `.eslintrc`, not the full repo-root-relative path), then checked against repo-level dev file patterns (`.vscode/`, `.gitignore`, `.husky/`, `.eslintrc`, `eslint.config.mjs`, `.prettierrc`, `.editorconfig`). There's no hardcoded special-casing for legacy `code/`/`shared/` layouts — repos that need those directories treated as top-level must list them explicitly in `overrideActorContext`.
-5. **`.dockerignore` filtering** — if a `.dockerignore` exists at the root of the actor's `dockerContextDir`, matching files are ignored. Patterns are resolved relative to `dockerContextDir`, matching Docker's own behavior. Test files are the exception: `build --test-files-glob <glob>` (relative to the repo root) skips this check for matching files, so a changed test triggers a build, and with it its tests, even when `.dockerignore` lists it. The PR workflow passes `test/platform/**`. Context matching still applies, so tests outside an actor's `dockerContextDir` (e.g. a root `test/` with standalone actors) need to be listed in its `overrideActorContext`; `build` logs a warning for changed test files outside every actor's context.
-6. **README classification** — a `README.md` file is `cosmetic` (only triggers a release build, not tests) if it lives inside the actor's own `folder`; otherwise it's ignored entirely, since it isn't documentation for this actor.
-7. **Cosmetic JSON classification** — `.json` files inside the actor's own `.actor/` directory with only cosmetic schema changes (whitespace, key ordering) only trigger a release build.
-8. **Functional** — everything else triggers both build and tests.
+The Actors that share code point their `dockerfile` and `dockerContextDir` at the repo root. Both paths are relative
+to the `.actor` folder:
 
-### 4. Create test directories
-
-```bash
-mkdir -p test/platform/core
+```json
+{
+    "actorSpecification": 1,
+    "name": "web-scraper",
+    "version": "0.1",
+    "dockerfile": "../../../Dockerfile",
+    "dockerContextDir": "../../.."
+}
 ```
 
-- Core (hourly) tests go in `test/platform/core`
-- Daily tests go anywhere in `test/platform`
-
-### 5. Set up GitHub workflows
-
-See the [GitHub workflows](#github-worklows) section below.
-
-## Config file
-
-`apify-test-tools.config.json` lives at the repo root and tells the tool which actors live in the repo, how to identify them, and which token to use.
+A change to the shared `src/` then builds and tests both of them, while a change in `standalone-actors/orchestrator`
+counts only for the orchestrator, even though it lies inside the others' context. The orchestrator leaves
+`dockerContextDir` out, so its context is its own folder.
 
-### Actor fields
+For a single-Actor repo, set `"folder": "."` in the config and place `.actor/actor.json` at the repo root.
 
-Every actor in the config is described by these fields, whatever the mode:
+### 4. Write tests
 
-| Field                  | Required | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             |
-| ---------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `folder`               | yes      | Relative path from repo root to the actor's own project directory — the folder that directly contains `.actor/actor.json` (i.e. `<folder>/.actor/actor.json`), the actor's README/CHANGELOG, and its source. Use `"."` for a single-actor repo where `.actor/` is at the root.                                                                                                                                                                                                                          |
-| `actorFullName`        | yes      | Full actor identifier in `owner/name` format (e.g. `"apify/web-scraper"`). This is the source of truth for the actor name — the `name` field in `actor.json` is not used.                                                                                                                                                                                                                                                                                                                               |
-| `tokenEnvVar`          | yes      | Name of the environment variable holding the Apify API token for this actor. In [grouped mode](#grouped-mode) it's inherited from the group unless the actor sets its own. No fallback — if the env var is not set at build time, the build fails.                                                                                                                                                                                                                                                      |
-| `overrideActorContext` | no       | Array of paths (relative to repo root) that define which files are relevant to this actor. When set, replaces the `dockerContextDir` from `actor.json` for change detection. Useful when an actor depends on shared packages outside its Docker build context. Entries must not be prefixes of one another (e.g. `["", "code"]` or `["actors", "actors/foo"]` are rejected). The actor's own `folder` is always part of its context — if none of the listed entries reach it, it's added automatically. |
+Put platform tests in `test/platform`. How you organize them inside is up to you. See
+[Writing tests](#writing-tests) below.
 
-### Modes
+### 5. Set up CI
 
-The file's layout is picked by the top-level `mode` field. Each mode is a different way of writing the same thing: every mode resolves to a list of actors with the fields above, and the same rules apply to all of them — no two actors may share a `folder`, and no two entries may point at the same `actorFullName`. If `mode` is left out, [`"legacy"`](#legacy-mode) is used, so existing config files keep working unchanged.
+On GitHub, add the [GitHub workflows](#github-workflows) below. On another CI platform, call the
+[CLI](docs/cli.md) from your own pipeline; the workflows are a good template for the steps.
 
-### Legacy mode
-
-The default. A flat `actors` array where every entry carries all of its settings. Good for repos with a few actors, or where every actor has different settings.
-
-```json
-{
-    "mode": "legacy",
-    "actors": [
-        {
-            "folder": "actors/web-scraper",
-            "actorFullName": "myteam/web-scraper",
-            "tokenEnvVar": "APIFY_TOKEN_MYTEAM"
-        },
-        {
-            "folder": "actors/email-sender",
-            "actorFullName": "myteam/email-sender",
-            "tokenEnvVar": "APIFY_TOKEN_MYTEAM",
-            "overrideActorContext": ["actors/email-sender", "packages/shared"]
-        }
-    ]
-}
-```
-
-`actors` must hold at least one entry. Every entry takes all the [actor fields](#actor-fields) directly, and `tokenEnvVar` is required on each one.
-
-### Grouped mode
+## Config file
 
-Actors are organized into `groups`, and each group sets `tokenEnvVar` and `overrideActorContext` once for all of its actors. Good for repos where many actors share the same token or context.
+`apify-test-tools.config.json` lives at the repo root and tells the tool which Actors live in the repo, where they
+are, and which token builds them. Actors are organized into groups, and each group sets the token once for all of its
+Actors:
 
 ```json
 {
@@ -129,72 +161,58 @@ Actors are organized into `groups`, and each group sets `tokenEnvVar` and `overr
     "groups": {
         "myteam": {
             "tokenEnvVar": "APIFY_TOKEN_MYTEAM",
-            "overrideActorContext": ["packages/shared"],
             "actors": [
                 { "folder": "actors/web-scraper", "actorFullName": "myteam/web-scraper" },
+                { "folder": "actors/email-sender", "actorFullName": "myteam/email-sender" },
                 {
-                    "folder": "actors/email-sender",
-                    "actorFullName": "myteam/email-sender",
-                    "overrideActorContext": ["actors/email-sender", "packages/mailer"]
+                    "folder": "standalone-actors/orchestrator",
+                    "actorFullName": "myteam/orchestrator",
+                    "overrideActorContext": ["standalone-actors/orchestrator", "test/platform/orchestrator"]
                 }
             ]
-        },
-        "partner": {
-            "tokenEnvVar": "APIFY_TOKEN_PARTNER",
-            "actors": [{ "folder": "actors/partner-crawler", "actorFullName": "partner/crawler" }]
         }
     }
 }
 ```
 
-`groups` is an object with at least one group. The group key (`myteam`, `partner` above) is only a label for readability; it has no effect on the result, and it does not need to match any folder or username. Each group needs a `tokenEnvVar` and at least one actor, and can optionally set `overrideActorContext`.
-
-An actor inside a group can set its own `tokenEnvVar` or `overrideActorContext`, and that value replaces the group's for that actor only. Values are replaced, not merged: in the example above, `myteam/email-sender` gets `["actors/email-sender", "packages/mailer"]`, not `packages/shared` as well.
-
-## Github worklows
-
-The reusable workflows live in this repo, alongside the package they call. They are the
-`public_`-prefixed files in `.github/workflows`; everything else there is this repo's own CI.
-Reference them at the `@workflows-v0` major tag, never at `@master` — see
-[Versioning and releases](#versioning-and-releases).
+The group key (`myteam` above) is only a label for readability; it doesn't need to match any folder or username. Each
+group needs a `tokenEnvVar` and at least one Actor. Add another group for Actors that need a different token, e.g.
+Actors owned by another account.
 
-There should be 4 GH workflow files in `.github/workflows`, plus an optional fifth for Claude reviews.
+The orchestrator sets `overrideActorContext` because its tests live outside its own folder: without it, a change to
+only `test/platform/orchestrator` wouldn't test it. The other two already have the whole repo, tests included, as
+their context.
 
-### `platform-tests-core.yaml`
+| Field                  | Where          | Required | Description                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            |
+| ---------------------- | -------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
+| `folder`               | Actor          | yes      | Relative path from repo root to the Actor's own project directory — the folder that directly contains `.actor/actor.json` (i.e. `<folder>/.actor/actor.json`), the Actor's README/CHANGELOG, and its source. Use `"."` for a single-Actor repo where `.actor/` is at the root.                                                                                                                                                                                                                         |
+| `actorFullName`        | Actor          | yes      | Full Actor identifier in `owner/name` format (e.g. `"apify/web-scraper"`). This is the source of truth for the Actor name — the `name` field in `actor.json` is not used.                                                                                                                                                                                                                                                                                                                              |
+| `tokenEnvVar`          | group or Actor | yes      | Name of the environment variable holding the Apify API token that builds the group's Actors. No fallback — if the env var is not set at build time, the build fails.                                                                                                                                                                                                                                                                                                                                   |
+| `platformTestsGlob`    | top level      | no       | Glob from the repo root of the platform tests and their helpers. Default `test/platform/**`. A changed file that matches it is a test change, also when `.dockerignore` lists it: the PR workflow builds and tests the Actor whose context holds it, and a release skips it. See [Change detection](docs/cli.md#change-detection).                                                                                                                                                                     |
+| `overrideActorContext` | group or Actor | no       | Array of paths (relative to repo root) that define which files are relevant to the Actor. When set, replaces the `dockerContextDir` from `actor.json` for change detection. Useful when an Actor depends on shared packages outside its Docker build context. Entries must not be prefixes of one another (e.g. `["", "code"]` or `["actors", "actors/foo"]` are rejected). The Actor's own `folder` is always part of its context — if none of the listed entries reach it, it's added automatically. |
 
-```yaml
-name: Platform tests - Core
+A value set on an Actor replaces the group's for that Actor only. Values are replaced, not merged: if the `myteam`
+group also set `overrideActorContext`, the orchestrator would still get only the two paths it lists.
 
-on:
-    schedule:
-        # Runs at the start of every hour
-        - cron: '0 * * * *'
-    workflow_dispatch:
+No two Actors may share a `folder`, and no two entries may point at the same `actorFullName`.
 
-jobs:
-    platformTestsCore:
-        uses: apify/apify-test-tools/.github/workflows/public_platform-tests.yaml@workflows-v0
-        with:
-            subtest: core
-        secrets: inherit
-```
+## GitHub workflows
 
-### `platform-tests-daily.yaml`
+The workflows take every change straight to production, with tests in between:
 
-```yaml
-name: Platform tests - Daily
+- **Every push to a PR** builds the Actors the PR changes into a test version (`0.99`) that users never run, and runs
+  their platform tests against those builds. Only the Actors whose files changed are built and tested, and a later
+  push only re-tests what changed since the last green run.
+- **Every merge to `master`** builds the changed Actors into their default version (`latest`), which is what users
+  run, and announces the release on Slack. There is no separate release step.
+- **Scheduled runs** (optional) run the tests against the released builds to catch what breaks without a code change,
+  like a website changing its layout.
 
-on:
-    schedule:
-        # Runs at 00:00 UTC every day
-        - cron: '0 0 * * *'
-    workflow_dispatch:
+These are GitHub Actions workflows. On another CI platform, build the same flow with the [CLI](docs/cli.md).
 
-jobs:
-    platformTestsDaily:
-        uses: apify/apify-test-tools/.github/workflows/public_platform-tests.yaml@workflows-v0
-        secrets: inherit
-```
+Add these files to `.github/workflows` in your repo. They call the reusable workflows in this repo, pinned at the
+`@workflows-v0` major tag — never at `@master`. Every input, the optional Claude workflows, stale issue closer, and how the tag moves are
+in the [Workflow API](docs/workflows.md).
 
 ### `pr-build-devel-test.yaml`
 
@@ -208,7 +226,14 @@ on:
 jobs:
     buildDevelAndTest:
         uses: apify/apify-test-tools/.github/workflows/public_pr-build-test.yaml@workflows-v0
-        secrets: inherit
+        secrets:
+            TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}
+            # One entry per secret apify-test-tools.config.json names, today each `tokenEnvVar`
+            CONFIG_SECRETS: |
+                {
+                    "APIFY_TOKEN_MYTEAM": "${{ secrets.APIFY_TOKEN_MYTEAM }}",
+                    "APIFY_TOKEN_PARTNER": "${{ secrets.APIFY_TOKEN_PARTNER }}"
+                }
 ```
 
 ### `release-latest.yaml`
@@ -223,504 +248,105 @@ on:
 jobs:
     buildLatest:
         uses: apify/apify-test-tools/.github/workflows/public_push-build-latest.yaml@workflows-v0
-        secrets: inherit
+        secrets:
+            SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
+            # One entry per secret apify-test-tools.config.json names, today each `tokenEnvVar`
+            CONFIG_SECRETS: |
+                {
+                    "APIFY_TOKEN_MYTEAM": "${{ secrets.APIFY_TOKEN_MYTEAM }}",
+                    "APIFY_TOKEN_PARTNER": "${{ secrets.APIFY_TOKEN_PARTNER }}"
+                }
 ```
 
-This workflow runs `apify-test-tools release --base-commit ${{ github.event.before }}`. See
-[Releasing Actors](#releasing-actors-release) for how the released range is determined and what to
-pass if you call the CLI from somewhere else.
-
-### `claude-review.yaml`
+### Scheduled tests (optional)
 
-Optional. Reviews a PR against the shared guidelines when you add the trigger label, and again on
-every push while that label is on.
+How often to run which tests is up to you. `test-files-glob` is a glob from the repo root, see
+[Splitting the tests](docs/workflows.md#splitting-the-tests). For example, you can run the daily tests once a day:
 
 ```yaml
-name: Claude review
+name: Platform tests - Daily
 
 on:
-    pull_request:
-        types: [labeled, synchronize]
+    schedule:
+        # Runs at 00:00 UTC every day
+        - cron: '0 0 * * *'
+    workflow_dispatch:
 
 jobs:
-    review:
-        uses: apify/apify-test-tools/.github/workflows/public_review.yaml@workflows-v0
-        secrets: inherit
+    platformTestsDaily:
+        uses: apify/apify-test-tools/.github/workflows/public_platform-tests.yaml@workflows-v0
+        with:
+            test-files-glob: 'test/platform/{daily,daily-without-pr}/**'
+        secrets:
+            TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}
+            SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
 ```
 
-The review instructions live in `.github/review-prompt.md` in this repo and are fetched at run time,
-because a reusable workflow doesn't get its own repo checked out. `prompt-ref` selects which ref to
-fetch them from and defaults to `workflows-v0`, so the instructions match the workflow you're calling — point
-it at a branch only to test a prompt change.
-
-### Secrets
-
-Callers pass `secrets: inherit`. The workflows do not turn every inherited secret into job-wide
-environment variables, so a secret is only visible to the step that needs it:
-
-| Secret                                                                    | Reaches                                                                                                                                                                                                                      |
-| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `NPM_TOKEN`                                                               | dependency install steps only, as both `NPM_TOKEN` and `NODE_AUTH_TOKEN`. Use a read-only token: npm granular tokens can be read-only, classic automation tokens can publish.                                                |
-| the Actor tokens named by `tokenEnvVar` in `apify-test-tools.config.json` | the `build`, `release`, and `delete-old-builds` steps only                                                                                                                                                                   |
-| `TESTER_APIFY_TOKEN`                                                      | the vitest step, and the Claude investigation step: the Apify MCP server's bearer token for reading the failing run, its log and its storages, and Claude's token for re-running it to measure how often the failure happens |
-| `SLACK_TOKEN_TESTS_BOT` / `SLACK_TOKEN_RELEASES_BOT`                      | the reporting and release steps only                                                                                                                                                                                         |
-
-The Actor tokens are the one set that cannot be listed in the workflow, because each Actor names its
-own token via `tokenEnvVar` in `apify-test-tools.config.json`. Those steps pass
-`${{ toJSON(secrets) }}` as `ALL_SECRETS` and run the command through
-`.github/scripts/run-with-apify-tokens.mjs`, which reads that same config file to decide which
-secrets to pass on:
+and a few critical tests every hour:
 
 ```yaml
-- name: Build
-  env:
-      ALL_SECRETS: ${{ toJSON(secrets) }}
-  run: |
-      node "${{ steps.setup.outputs.scripts-path }}/run-with-apify-tokens.mjs" \
-        npx apify-test-tools build --target-branch ...
-```
-
-The wrapper passes only the tokens the config declares and drops `ALL_SECRETS`, so neither `npx` nor
-anything under `node_modules` sees the blob. Nothing is written to `$GITHUB_ENV`, so the tokens stay
-inside that one command rather than leaking into later steps. Reading the same file
-`apify-test-tools` reads means the two can't drift, and a secret that merely looks like an Actor
-token is not passed just because of its name.
-
-A token the config declares but the repo hasn't set is a warning, not a failure: a repo can carry an
-Actor whose token isn't configured and still build fine as long as that Actor never changes, and
-`apify-test-tools` raises a precise error naming the Actor at the point it actually needs the token.
-
-`scripts-path` comes from the setup action (give the step `id: setup`) and points at this repo's
-`.github/scripts/` directory inside the runner's action checkout, so workflows can run these helpers
-without checking this repo out again. The caller's workspace holds the caller's repo, not this one.
-
-Two tidier-looking alternatives don't work, so don't reach for them:
-
-- **Exporting to `$GITHUB_ENV`** would let the steps call `npx` directly with no wrapper, but
-  `$GITHUB_ENV` applies to every later step in the job. In `pr-build-test` the vitest step runs after
-  the build, so it would inherit Actor tokens it has no use for.
-- **Returning the tokens as a step output** would be scoped correctly, but the runner refuses to set
-  an output whose value contains a registered secret. It logs `Skip output <name> since it may
-contain secret` and leaves the output empty, so anything reading it downstream gets nothing.
-
-The `unitTest` job runs static checks and needs `NPM_TOKEN` only. No job runs `npm ci` with Apify or
-Slack credentials in scope, so a postinstall script in the dependency tree cannot read them.
-
-## Versioning and releases
-
-The workflows and the npm package live in one repo but ship on their own schedules:
-
-| What                                         | Consumers get it through                   | Ships when              |
-| -------------------------------------------- | ------------------------------------------ | ----------------------- |
-| the workflows and the CLI they call          | the `@workflows-v0` tag in `uses:`         | every master push       |
-| the lib (`testActor`, `describe`, matchers…) | `apify-test-tools` in their `package.json` | a stable release on npm |
-
-The CLI is bundled with its dependencies into `.github/cli/apify-test-tools.mjs` (`npm run bundle`,
-run by the pre-commit hook and checked in CI), and the setup action runs it from there. Nothing is
-installed for it, so a tag is a complete statement: these workflows and this CLI, with the exact
-dependency versions from `package-lock.json` at that commit. The consumer's own `node_modules` is
-left alone, so their tests keep the lib version they locked.
-
-- **Workflow or CLI change** — merge it. `workflows-v0` moves once checks and tests pass, and it goes
-  live. No release needed, even when a workflow starts using a new CLI feature: both are in the
-  same commit.
-- **Lib change** — merge it, then cut a release when you want it out. Consumers pick it up when
-  they update `apify-test-tools`.
-
-The CLI and the lib in one job can be different versions now, so the formats between them are a
-contract: the `ACTOR_BUILDS` JSON that `build` writes and the lib reads, and the vitest JSON report
-`report-tests` reads. Add fields freely; renaming or removing one needs a new major tag.
-
-`workflows-v0` moving on every master push means `@workflows-v0` is as live as `@master` was —
-there's no staging step. What the tag buys you is a `workflows-v1` for breaking workflow changes,
-so repos migrate one at a time instead of all at once, and a way to roll back by pointing the tag at
-an earlier commit (**Move major version tag** workflow). Bump `MAJOR_TAG` in
-`.github/workflows/_move_major_tag.yaml` to cut the next major; the old tag then freezes where it is
-and keeps working, CLI included, since the bundle is part of the frozen commit.
-
-The tag tracks the **workflows'** contract, not the npm package version. They move independently on
-purpose, so `@workflows-v0` is expected to stay `@workflows-v0` after the package reaches 1.0 —
-bump it when a workflow breaks its callers, not when the package does. Because `uses:` cannot take an expression, every ref
-into this repo repeats the tag literally; `check-major-tag-refs.mjs` fails the build if `MAJOR_TAG`
-and those refs disagree, which is the mistake that would otherwise ship silently during a bump.
-
-### Testing workflow changes
-
-- Point [testing-repo-for-github-actions](https://github.com/apify-store/testing-repo-for-github-actions)
-  at your branch (`uses: ...@your-branch`). It has real attached Actors and tests. The branch carries
-  its own CLI bundle, so one branch tests the workflows and the CLI together.
-- To change the composite action itself, repoint the `uses:` refs inside the reusable workflows at
-  your branch as well, and change them back before merging.
-- Make sure the shell code actually works on your laptop first.
-- After merging, watch the workflow on a real project before moving on.
+name: Platform tests - Hourly
 
-## Writing tests
-
-### Test structure
-
-`testActor` runs the actor and provides extended `expect` and `run` inside the callback.
-
-```ts
-import { describe, testActor } from 'apify-test-tools';
-
-describe('test', () => {
-    testActor(actorId, 'actor test 1', async ({ expect, run }) => {
-        const runResult = await run({ input });
-
-        // your checks
-    });
-
-    testActor(actorId, 'actor test 2', async ({ expect, run }) => {
-        const runResult = await run({ input });
-
-        // your checks
-    });
-});
-```
-
----
-
-### Validating basic run attributes
-
-`toFinishWith` validates common run properties in a single call:
-
-```ts
-await expect(runResult).toFinishWith({
-    datasetItemCount: 100,
-});
-```
-
-You can also specify a range:
-
-```ts
-await expect(runResult).toFinishWith({
-    datasetItemCount: { min: 80, max: 120 },
-});
-```
-
-Here is full example of what you can validate with `toFinishWith`
-
-```ts
-await expect(runResult).toFinishWith({
-    // These are default
-    status: 'SUCCEEDED',
-    duration: {
-        min: 600, // 0.6 sec
-        max: 600_000, // 10 min
-    },
-    failedRequests: 0,
-    requestsRetries: { max: 3 },
-    forbiddenLogs: ['ReferenceError', 'TypeError'],
-
-    // only datasetItemCount is required
-    datasetItemCount: { min: 80, max: 120 },
-
-    // optional
-    chargedEventCounts: {
-        'actor-start': 1,
-        'place-scraped': 9,
-    },
-});
-```
-
----
-
-### Custom validations
-
-```ts
-expect(place.title, `London Eye's title`).toEqual('lastminute.com London Eye');
-```
-
----
-
-### Custom validation functions
-
-You can create your own functions wrapping a common validation logic in e.g. `test/platform/utils.ts` and import it in test files.
-
-```ts
-import { ExpectStatic } from 'apify-test-tools'
-
-export const validateItem = (expect: ExpectStatic, item: any) {
-    expect(item.title, 'Item title').toBeString();
-}
-```
-
----
-
-### Test options
-
-You can pass options as the fourth argument to `testActor`:
-
-```ts
-testActor(
-    actorId,
-    'slow actor test',
-    async ({ expect, run }) => {
-        const runResult = await run({ input });
-        await expect(runResult).toFinishWith({ datasetItemCount: 100 });
-    },
-    {
-        timeout: 2 * 60 * 60 * 1000, // 2 hours (default is 1 hour)
-        retry: 3, // retry up to 3 times (default is 1)
-    },
-);
-```
-
----
-
-### Using prefilled input
-
-If the actor has a prefilled input on the platform, you can merge it with your test input:
+on:
+    schedule:
+        # Runs at the start of every hour
+        - cron: '0 * * * *'
+    workflow_dispatch:
 
-```ts
-testActor(actorId, 'with prefilled input', async ({ expect, run }) => {
-    const runResult = await run({
-        prefilledInput: true,
-        input: { maxItems: 10 }, // merged on top of the prefilled input
-    });
-    await expect(runResult).toFinishWith({ datasetItemCount: 10 });
-});
+jobs:
+    platformTestsHourly:
+        uses: apify/apify-test-tools/.github/workflows/public_platform-tests.yaml@workflows-v0
+        with:
+            test-files-glob: 'test/platform/{hourly,hourly-without-pr}/**'
+        secrets:
+            TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}
+            SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
 ```
 
----
-
-### Testing an existing run
+### Secrets
 
-You can skip starting a new run and validate an existing one by passing `runId`:
+Set these as repository or organization secrets and pass them in each job's `secrets:` block, as in the examples
+above. The Actor tokens go packed in one JSON object, `CONFIG_SECRETS`, because their names differ per repo:
 
-```ts
-testActor(actorId, 'validate existing run', async ({ expect, run }) => {
-    const runResult = await run({ runId: 'some-run-id' });
-    await expect(runResult).toFinishWith({ datasetItemCount: 100 });
-});
-```
+| Secret                                                                    | Needed for                                                        |
+| ------------------------------------------------------------------------- | ----------------------------------------------------------------- |
+| the Actor tokens named by `tokenEnvVar` in `apify-test-tools.config.json` | building Actors and deleting their old builds                     |
+| `TESTER_APIFY_TOKEN`                                                      | running the platform tests                                        |
+| `SLACK_BOT_TOKEN`                                                         | announcing releases and reporting scheduled test results to Slack |
+| `NPM_TOKEN`                                                               | optional, installing private npm packages. Use a read-only token  |
 
----
+Repos in the `apify` org can pass `secrets: inherit` instead. Each secret only reaches the step that needs it. See
+[Secrets](docs/workflows.md#secrets) for both, and for passing `CONFIG_SECRETS` without listing the tokens.
 
-### Accessing run data
+## Writing tests
 
-`RunTestResult` provides methods to access the run's data:
+Platform tests use `describe` and `testActor` from `apify-test-tools` instead of the ones from `vitest`. `testActor`
+registers a test that only runs when that Actor was built for it (or when you ask for all tests), and gives the
+callback an extended `expect` and a `run` function that starts the Actor.
 
 ```ts
-testActor(actorId, 'check dataset items', async ({ expect, run }) => {
-    const runResult = await run({ input });
-
-    // Access dataset items
-    const { items } = await runResult.getDataset();
-    expect(items[0].title).toBeNonEmptyString();
-
-    // Access run log
-    const log = await runResult.getLog();
-    expect(log).toContain('Crawl finished');
-
-    // Access crawler statistics
-    const stats = await runResult.getStatistics();
-    expect(stats?.requestsFinished).toBeGreaterThan(0);
-
-    // Access key-value store
-    const kvs = runResult.getKeyValueStoreClient();
-    const record = await kvs.getRecord('OUTPUT');
-
-    // Access run info (refreshed from API)
-    const runInfo = await runResult.getRunInfo();
-});
-```
-
----
+import { describe, testActor } from 'apify-test-tools';
 
-### Testing standby actors
+describe('web-scraper', () => {
+    testActor('myteam/web-scraper', 'scrapes the start URL', async ({ expect, run }) => {
+        const runResult = await run({ input: { startUrls: [{ url: 'https://apify.com' }] } });
 
-Use `testStandbyActor` for actors that support standby mode:
+        // Status SUCCEEDED, a sane duration, no failed requests, no errors in the log, and the item count
+        await expect(runResult).toFinishWith({ datasetItemCount: { min: 1, max: 10 } });
 
-```ts
-import { describe, testStandbyActor } from 'apify-test-tools';
-
-describe('standby tests', () => {
-    testStandbyActor(actorId, 'standby request', async ({ expect, callStandby }) => {
-        const { data, status } = await callStandby({
-            input: { query: 'test' },
-            path: '/search',
-            headers: { 'Content-Type': 'application/json' },
-        });
-
-        expect(status).toBe(200);
-        expect(data.results).toBeNonEmptyArray();
+        const { items } = await runResult.getDataset();
+        expect(items[0].title, 'Page title').toBeNonEmptyString();
     });
 });
 ```
 
----
-
-### Custom matchers
-
-`testActor` extends `expect` with the following custom matchers:
-
-- `toBeArray()` / `toBeEmptyArray()` / `toBeNonEmptyArray()`
-- `toBeString()` / `toBeNonEmptyString()` / `toStartWith(prefix)`
-- `toBeNumber()` / `toBeWholeNumber()` / `toBeWithinRange(min, max)`
-- `toBeBoolean()` / `toBeTrue()` / `toBeFalse()`
-- `toBeObject()` / `toBeNonEmptyObject()`
-- `toFinishWith(options)` - validates run status, duration, dataset, logs, etc.
-
-## CLI (`apify-test-tools` bin)
-
-The package includes a CLI binary used by CI workflows to build Actors, detect changes, and report test results. You can also run it locally.
-
-### Dry runs (`--dry-run`)
+Platform tests are skipped unless `TESTER_APIFY_TOKEN` is set, so your regular unit test run stays unaffected.
 
-Every command that changes something (`build`, `build-from-local`, `release`, `delete-old-builds`, `report-tests`) takes `--dry-run`. A dry run:
-
-- **Changes nothing.** No versions or builds are created, nothing is uploaded or deleted, and no Slack message is posted.
-- **Does every read and check the real run does**, so it fails wherever the real run would: change detection, the Apify token and Actor lookup, the [repository check](#repository-check), collecting and zipping local sources, and the Slack token (verified with Slack's read-only `auth.test`).
-- **Needs the same tokens as the real run.** Read-only tokens are enough, since nothing is written. A dry run can't tell whether a token is also allowed to write, because the Apify API has no dry-run mode of its own.
-- **Prints a plan, not results.** Logs start with `[DRY RUN] Would …`. `build` and `build-from-local` write `{ "dryRun": true, "builds": [...] }` to stdout instead of the build array, so the plan can't be passed to tests as `ACTOR_BUILDS` by mistake.
-
-### Running locally
-
-Running the testing library locally is useful when you only want to update the testing code in /test because you can iterate on it without pushing new code to the remote.
-
-If you don't need to change any source files and only iterate on /test code, you can skip steps 1-4. But if you want to test vs changed /src, you have to push that GitHub branch since it needs to build the Actors with that code.
-
-The main local flow is:
-
-1. Switch to a dummy branch that you will push and can later delete
-2. `npm i apify-test-tools@latest -D`
-3. Push your code (changes you want to test)
-4. Build Actors on Apify (with your new code)
-5. Run tests against those builds. You can change tests and run on the same builds.
-
-Run every command from the root of the actor repository you want to work with.
-
-#### 4. Build affected Actors
-
-If you want to test vs existing src code, you can skip this and instead construct the output JSON manually from existing builds only for the Actors you need to test.
-
-Requires `APIFY_TOKEN_<USERNAME>` for all Apify users that own your Actors (e.g. `apify`, `compass`, `lukaskrivka` users). The username is derived from the actor name — uppercased with non-word chars replaced by `_` (e.g. Actor `john.doe/my-actor` needs `APIFY_TOKEN_JOHN_DOE`).
-
-```bash
-APIFY_TOKEN_JOHN_DOE=<token> \
-  npx apify-test-tools build \
-    --target-branch origin/master \
-    --source-branch origin/my-dummy-branch \
-    --dry-run
-```
-
-With `--dry-run`, it prints the plan (the version and Git URL each Actor would be built from). Remove `--dry-run` to actually trigger builds and update the branch names. The command outputs a JSON array of build objects to stdout:
-
-```json
-[{ "buildId": "...", "actorRawId": "...", "buildNumber": "...", "actorFullName": "john.doe/my-actor" }]
-```
-
-#### Build from local source (no push needed)
-
-If you don't want to push a dummy branch just to test a change and wait for all the tests to finish, `build-from-local` builds Actors directly from your local files (zipped into the Actor's `actor-<id>-source` key-value store and built as `TARBALL`, like `apify push`), skipping steps 1-4 above.
-
-```bash
-APIFY_TOKEN_JOHN_DOE=<token> \
-  npx apify-test-tools build-from-local --actors john.doe/my-actor
-```
-
-Pass a hardcoded actor name via `--actors` to build only that Actor (comma-separate multiple names). Omit `--actors` to build all Actors in the repo, or add `--dry-run` to check the token, the Actor and the zipped sources without uploading or building. It outputs the same JSON build array as `build`, so you run tests against it the same way as in step 5 below:
-
-```bash
-# Build from local source and capture output
-BUILDS=$(APIFY_TOKEN_JOHN_DOE=apify_api_xxx \
-    npx apify-test-tools build-from-local --actors apify/my-actor)
-```
-
-Since you already scoped the build to just the Actor(s) you care about, point vitest at a specific test file (or a `-t` name filter) instead of the whole `test/platform` directory — you get feedback on that one test without waiting for the full suite to run.
-
-#### 5. Run tests against the builds
-
-Pass the build output as `ACTOR_BUILDS` and provide `TESTER_APIFY_TOKEN`. The token can point to your own account (if you have enough memory) or you can use the testing account (xRGg9iAfJSymqartk). Platform test suites are skipped unless `TESTER_APIFY_TOKEN` is set, so regular unit test runs stay unaffected.
-
-If you want to run only certain tests, change the `test/platform` to be more specific.
-
-```bash
-ACTOR_BUILDS='<JSON output from build command>' \
-TESTER_APIFY_TOKEN=<token> \
-  npx vitest --run --maxConcurrency 20 --fileParallelism=true --maxWorkers 100 test/platform
-```
-
-#### Full example
-
-Tests only run for Actors in `ACTOR_BUILDS`, so a custom workflow has to build (and pass) the Actors even when only tests changed. Pass `--test-files-glob` so test changes count even when `.dockerignore` lists them.
+To run them locally against the Actors' current default builds:
 
 ```bash
-# Build and capture output
-BUILDS=$(APIFY_TOKEN_JOHN_DOE=apify_api_xxx \
-    npx apify-test-tools build \
-    --target-branch origin/master \
-    --source-branch origin/my-dummy-branch \
-    --test-files-glob 'test/platform/**')
-
-# Run tests with the builds
-ACTOR_BUILDS="$BUILDS" \
-TESTER_APIFY_TOKEN=apify_api_yyy \
-  npx vitest --run --maxConcurrency 20 --fileParallelism=true --maxWorkers 100 test/platform
+npx apify-test-tools get-actor-configs | TESTER_APIFY_TOKEN=<token> npx apify-test-tools test
 ```
 
-### Releasing Actors (`release`)
-
-`release` builds the `latest` (default) version of every Actor changed since the last release and
-posts release notes to Slack. It reads everything else from git: the branch from the checked-out
-`HEAD`, the commits and changed files from the range `<base-commit>..HEAD`, and the repository from
-the `origin` remote.
-
-```bash
-npx apify-test-tools release --base-commit <sha> --release-slack-channel "#releases" --dry-run
-```
-
-#### `--base-commit`
-
-The last commit that is **already released**. Everything after it, up to `HEAD`, is released. It is
-required: git alone can't tell which commits a push brought, and guessing wrong silently leaves Actors
-on old code.
-
-The changed files come from diffing `<base-commit>` and `HEAD` directly, so this works for every
-merge strategy as long as the base commit is the branch tip from **before** the push:
-
-| Merge strategy                 | What lands on the branch          | What to pass                                                           |
-| ------------------------------ | --------------------------------- | ---------------------------------------------------------------------- |
-| Squash and merge               | 1 new commit                      | the push's "before" commit (`HEAD~1` also works)                       |
-| Merge commit                   | the PR's commits + 1 merge commit | the push's "before" commit (`HEAD^1`, the first parent, also works)    |
-| Rebase and merge / direct push | N new commits                     | the push's "before" commit (`HEAD~1` would miss the first N-1 commits) |
-
-On GitHub, the push's "before" commit is `${{ github.event.before }}`, which the reusable workflow
-passes for you. On other CIs use the equivalent, e.g. `$CI_COMMIT_BEFORE_SHA` on GitLab. The checkout
-needs the full history (`fetch-depth: 0` in `actions/checkout`) so the base commit is available locally.
-
-`release` fails instead of guessing when:
-
-- the base commit is all zeros (GitHub sends this for a push that created the branch)
-- the base commit isn't in the local history (shallow checkout) or isn't an ancestor of `HEAD`
-  (the branch was force-pushed)
-
-In those cases, rerun `release` with a `--base-commit` you pick yourself: an ancestor of `HEAD` from
-before the changes you want to release (for a shallow checkout, fetch the full history instead).
-`--actors` only narrows which of the changed Actors get released; it never forces a build, so on its
-own it doesn't get past these errors.
-If `HEAD` is the base commit, there is nothing to release and the command exits successfully.
-
-A failed release isn't retried by the next push: that push's base commit is the failed one's head, so
-its changes are skipped. Re-run the failed workflow run, or release the Actors manually.
-
-#### Repository check
-
-`release` and `build` point each Actor's version at the `origin` remote. Because a build overwrites the
-version's Git URL, building from a fork or a mirror would repoint the Actor. So, before building, the
-remote is compared with the Git URL of the Actor's default version (the one its default build tag
-points to), and the command fails if they differ or if the default version isn't built from a Git
-repository. The URL form doesn't matter (`https://github.com/org/repo` equals
-`git@github.com:org/repo.git`).
-
-To build from a different repository on purpose, e.g. when moving an Actor, pass `--repo-url`. That
-skips the check.
-
-#### Dev mode
-
-For development on `apify-test-tools` itself, use `tsx` directly:
-
-```bash
-cd local-clone && tsx ../bin/main.ts get-actor-configs
-```
+The [Vitest library API](docs/lib.md) covers everything `toFinishWith` checks, the other matchers, standby Actors,
+reading run data, and running tests locally against builds of your own changes.