apify-test-tools 0.10.2-beta.19 → 0.10.2-beta.20

package/docs/cli.md

@@ -71,10 +71,12 @@ changed files. For each changed file, for each Actor:
    get no special treatment: to treat them as top-level, list them in `overrideActorContext`.
 5. **`.dockerignore` filtering** — if a `.dockerignore` exists at the root of the Actor's `dockerContextDir`,
    matching files are ignored. Patterns are resolved relative to `dockerContextDir`, matching Docker's own behavior.
-   Test files are the exception: `get-affected-actors --test-files-glob <glob>` (relative to the repo root) skips
-   this check for matching files, so a changed test triggers a build, and with it its tests, even when
-   `.dockerignore` lists it. The PR workflow passes `test/platform/**`. Context matching still applies, so tests
-   outside an Actor's `dockerContextDir` (e.g. a root `test/` with standalone Actors) need to be listed in its
+   Platform test files are the exception, see the next step.
+   **Platform test files** — a file that matches `platformTestsGlob` from the config file (a glob from the repo root,
+   default `test/platform/**`, helpers included) is a `test` change, also when `.dockerignore` lists it. This is
+   checked before the `.dockerignore` filtering. A test change puts the Actor in `testChangedActors`, so its tests
+   run, but it is neither functional nor cosmetic, so a release doesn't build it. Context matching still applies, so
+   tests outside an Actor's `dockerContextDir` (e.g. a root `test/` with standalone Actors) need to be listed in its
    `overrideActorContext`; `get-affected-actors` logs a warning for changed test files outside every Actor's context.
 6. **README classification** — a `README.md` file is `cosmetic` (only triggers a release build, not tests) if it
    lives inside the Actor's own `folder`; otherwise it's ignored entirely, since it isn't documentation for this
@@ -97,13 +99,12 @@ npx apify-test-tools get-affected-actors --base origin/master --since <sha>
 npx apify-test-tools get-affected-actors --since <sha>
 ```
 
-| Flag                | Default | Meaning                                                                                                                                    |
-| ------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
-| `--base`            |         | Only what `HEAD` changes compared to this branch counts, like `git diff <base>...HEAD`                                                     |
-| `--since`           |         | Only changes after this commit count (see below)                                                                                           |
-| `--test-files-glob` |         | Test files (glob from the repo root) that count as changes even when `.dockerignore` lists them, see [Change detection](#change-detection) |
-| `--actors`          | all     | Only report these Actors                                                                                                                   |
-| `--ignore`          | none    | Never report these Actors                                                                                                                  |
+| Flag       | Default | Meaning                                                                                |
+| ---------- | ------- | -------------------------------------------------------------------------------------- |
+| `--base`   |         | Only what `HEAD` changes compared to this branch counts, like `git diff <base>...HEAD` |
+| `--since`  |         | Only changes after this commit count (see below)                                       |
+| `--actors` | all     | Only report these Actors                                                               |
+| `--ignore` | none    | Never report these Actors                                                              |
 
 Pass `--base`, `--since`, or both:
 
@@ -133,26 +134,33 @@ The output is a JSON document:
     "changedFiles": ["src/main.ts", "CHANGELOG.md"],
     "changelog": "Lines added to CHANGELOG.md, or null",
     "functionallyChangedActors": [{ "actorFullName": "john.doe/my-actor" }],
-    "cosmeticallyChangedActors": [{ "actorFullName": "john.doe/other-actor" }]
+    "cosmeticallyChangedActors": [{ "actorFullName": "john.doe/other-actor" }],
+    "testChangedActors": [{ "actorFullName": "john.doe/my-actor" }]
 }
 ```
 
-Each changed Actor is in exactly one of the lists. `functionallyChangedActors` have at least one functional change,
-so they need a new build and tests. `cosmeticallyChangedActors` only have cosmetic changes (see
-[Change detection](#change-detection)), so they need a new build only when released: the PR workflow builds
-`functionallyChangedActors`, the release workflow builds both.
+`functionallyChangedActors` have at least one functional change, so they need a new build and tests.
+`cosmeticallyChangedActors` have cosmetic changes and no functional one (see [Change detection](#change-detection)), so
+they need a new build only when released. An Actor is in at most one of these two lists. `testChangedActors` have a
+changed platform test (`platformTestsGlob` in the config) in their context, whatever else changed, so their tests need
+to run. A test change alone needs no release. The PR workflow builds `functionallyChangedActors` and
+`testChangedActors`. The release workflow builds `functionallyChangedActors` and `cosmeticallyChangedActors`.
+
+With `--base` and `--since`, an Actor is in `testChangedActors` when its tests changed on the branch and again since
+`--since`, the same rule as for functional changes.
 
 ## Building Actors (`build`)
 
 `build` builds exactly the Actors it is given on stdin, from the checked-out branch as it is on the remote, and
-prints the builds as the same JSON array `build-from-local` prints. A detached HEAD is an error, since there is no
-branch to build from. It doesn't look at git history: which Actors to build is up to you, usually picked from the
+prints the builds as the same JSON array `build-from-local` prints. An Actor listed twice is built once, so the lists
+of `get-affected-actors` can be joined without removing duplicates first. A detached HEAD is an error, since there is
+no branch to build from. It doesn't look at git history: which Actors to build is up to you, usually picked from the
 output of [`get-affected-actors`](#finding-affected-actors-get-affected-actors).
 
 ```bash
-# A PR: the Actors a branch changed functionally, built into the test version
+# A PR: the Actors whose code or platform tests the branch changed, built into the test version
 npx apify-test-tools get-affected-actors --base origin/master \
-  | jq '{ actors: .functionallyChangedActors }' \
+  | jq '{ actors: (.functionallyChangedActors + .testChangedActors) }' \
   | npx apify-test-tools build --version-number 0.99
 
 # A release: every Actor changed since the last release, into the version users run
@@ -285,7 +293,8 @@ the repository locked) on the Actors it is given on stdin, and prints a report o
 ... | npx apify-test-tools build --version-number 0.99 | npx apify-test-tools test > test-report.json
 
 # Scheduled tests: every Actor in the config, on its default build
-npx apify-test-tools get-actor-configs | npx apify-test-tools test test/platform/core > test-report.json
+npx apify-test-tools get-actor-configs \
+    | npx apify-test-tools test --test-files-glob 'test/platform/{hourly,hourly-without-pr}/**' > test-report.json
 
 # Only some of them
 npx apify-test-tools get-actor-configs --actors john.doe/my-actor | npx apify-test-tools test > test-report.json
@@ -309,11 +318,19 @@ annotate the failing tests in a GitHub Actions run. It takes no other vitest opt
 test by name, run vitest directly with the builds in `ACTOR_BUILDS` (see
 [Running tests locally](lib.md#running-tests-locally)).
 
-| Argument / flag        | Default         | Meaning                                                                                                                         |
-| ---------------------- | --------------- | ------------------------------------------------------------------------------------------------------------------------------- |
-| `[filters..]`          | `test/platform` | Test files to run, as vitest filters                                                                                            |
-| `--vitest-output-file` |                 | Also keep vitest's own JSON report there, e.g. to upload it                                                                     |
-| `--reporter`           |                 | [Vitest reporter](https://vitest.dev/guide/reporters) to add to `default` and `json`, e.g. `github-actions`. Repeat it for more |
+| Flag                   | Default            | Meaning                                                                                                                         |
+| ---------------------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------- |
+| `--test-files-glob`    | `test/platform/**` | Glob from the repo root of the test files to run, see below                                                                     |
+| `--vitest-output-file` |                    | Also keep vitest's own JSON report there, e.g. to upload it                                                                     |
+| `--reporter`           |                    | [Vitest reporter](https://vitest.dev/guide/reporters) to add to `default` and `json`, e.g. `github-actions`. Repeat it for more |
+
+`test` takes no test paths as arguments, only `--test-files-glob`. The glob is read the same way as
+`platformTestsGlob` in the config file. Vitest filters are parts of a file path, not globs, so `test` finds the files
+the glob matches and passes vitest as few paths as it can: each folder whose files all match, with a trailing `/`, and
+the other matched files one by one. `test/platform/{daily,hourly}/**` becomes two folders however many tests they
+hold, so the command line stays short. Vitest still applies its own `include` and `exclude`, so helper files in the
+glob run nothing. A glob that matches no file is an error, because vitest without filters would run every test of the
+repo. Quote the glob, so the shell doesn't expand it.
 
 Vitest's own output goes to stderr, followed by a summary that lists every test with the link to its run. Stdout
 gets only the report:
@@ -432,15 +449,15 @@ To build from git, like CI does:
 5. Run tests against those builds. You can change tests and run on the same builds.
 
 ```bash
-npx apify-test-tools get-affected-actors --base origin/master --test-files-glob 'test/platform/**' \
-    | jq '{ actors: .functionallyChangedActors }' \
+npx apify-test-tools get-affected-actors --base origin/master \
+    | jq '{ actors: (.functionallyChangedActors + .testChangedActors) }' \
     | APIFY_TOKEN_JOHN_DOE=apify_api_xxx npx apify-test-tools build --version-number 0.99 > builds.json
 ```
 
 Run it with your dummy branch checked out: it builds the Actors that branch changed functionally, from the branch as
 it is on the remote, into the test version `0.99`. Add `--dry-run` to `build` to print the plan (the version and Git
 URL each Actor would be built from) first, and update `--base` to the branch you compare against. Tests only run for
-Actors in `ACTOR_BUILDS`, so pass `--test-files-glob` to count test changes even when `.dockerignore` lists them.
+Actors in `ACTOR_BUILDS`, so the `jq` filter also builds the Actors whose platform tests changed.
 
 If you want to test against existing builds instead, skip the build and write the build array by hand, only for the
 Actors you need to test.

package/docs/workflows.md

@@ -39,11 +39,12 @@ Validates a pull request. It runs two jobs in parallel.
 **`platformTest`** builds and tests the Actors the PR changes:
 
 1. Finds the affected Actors with [`get-affected-actors`](cli.md#finding-affected-actors-get-affected-actors),
-   comparing the PR branch with its base branch. Changed files under `test/platform/**` count even when
-   `.dockerignore` lists them.
-2. Builds the functionally changed Actors into the test version `0.99` with [`build`](cli.md#building-actors-build).
-   Cosmetic-only changes (README, CHANGELOG, schema formatting) need no tests, so they're not built.
-3. Runs [`test`](cli.md#running-tests-test) on `./test/platform/<test-files-glob>` with those builds, so only the
+   comparing the PR branch with its base branch. Changed files that match `platformTestsGlob` in the config file
+   (default `test/platform/**`) are test changes, also when `.dockerignore` lists them.
+2. Builds the Actors with functional or test changes into the test version `0.99` with
+   [`build`](cli.md#building-actors-build). Cosmetic-only changes (README, CHANGELOG, schema formatting) need no
+   tests, so they're not built.
+3. Runs [`test`](cli.md#running-tests-test) on the files that match `test-files-glob` with those builds, so only the
    tests of the built Actors run. Failing tests are annotated on the PR.
 4. On success, caches the tested commit as the "last validated commit". The next push to the PR then only builds
    and tests the Actors that changed since it (see `--since` in
@@ -62,11 +63,11 @@ Validates a pull request. It runs two jobs in parallel.
 In a repo whose Actors aren't written in JavaScript or TypeScript, define the `build-check`, `lint` and `test`
 scripts, e.g. to run your language's own checks: without them, the TypeScript, Lint and Test checks fail.
 
-| Input                      | Type    | Default | Description                                                                                                              |
-| -------------------------- | ------- | ------- | ------------------------------------------------------------------------------------------------------------------------ |
-| `skip-platform-tests`      | boolean | `false` | Skip the `platformTest` job and run the static checks only                                                               |
-| `test-files-glob`          | string  | `''`    | Which tests to run, relative to `test/platform`. Empty runs everything in it                                             |
-| `require-env-vars-in-sync` | boolean | `true`  | Fail the build unless the env var names of `0.99` match the default version's, see [Env var check](cli.md#env-var-check) |
+| Input                      | Type    | Default            | Description                                                                                                                 |
+| -------------------------- | ------- | ------------------ | --------------------------------------------------------------------------------------------------------------------------- |
+| `skip-platform-tests`      | boolean | `false`            | Skip the `platformTest` job and run the static checks only                                                                  |
+| `test-files-glob`          | string  | `test/platform/**` | Glob of the test files to run, from the repo root. Empty means the default. See [Splitting the tests](#splitting-the-tests) |
+| `require-env-vars-in-sync` | boolean | `true`             | Fail the build unless the env var names of `0.99` match the default version's, see [Env var check](cli.md#env-var-check)    |
 
 Secrets: `TESTER_APIFY_TOKEN`, the Actor tokens in `CONFIG_SECRETS`, and optionally `NPM_TOKEN`. With
 `skip-platform-tests`, only `NPM_TOKEN`.
@@ -83,6 +84,9 @@ on:
 jobs:
     buildDevelAndTest:
         uses: apify/apify-test-tools/.github/workflows/public_pr-build-test.yaml@workflows-v0
+        with:
+            # Leaves out the *-without-pr folders, see "Splitting the tests"
+            test-files-glob: 'test/platform/{daily,hourly}/**'
         secrets:
             TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}
             # One entry per secret apify-test-tools.config.json names, today each `tokenEnvVar`
@@ -101,7 +105,7 @@ Releases the Actors a push changed. Skipped when the head commit message contain
 1. `get-affected-actors --since ${{ github.event.before }}` finds everything the push changed. See
    [Releasing Actors](cli.md#releasing-actors) for how the released range is determined.
 2. `build --version-number default` builds both the functionally and the cosmetically changed Actors into their
-   default version, the one users run.
+   default version, the one users run. A push that only changes platform tests builds nothing.
 3. [`notify`](cli.md#notifying-notify) posts the release (its changelog additions and its commits, linked to
    GitHub) to the report channel. It posts nothing when the push brought no new commits.
 4. `delete-old-builds` deletes old builds of every Actor in the config (see
@@ -146,15 +150,15 @@ Runs the platform tests of every Actor in the config against its current default
 reaches the tests. Both the report of `test` and
 vitest's own JSON report are uploaded as the `vitest-results` artifact. Runs only on `master` or `main`.
 
-| Input                  | Type   | Default | Description                                                                  |
-| ---------------------- | ------ | ------- | ---------------------------------------------------------------------------- |
-| `test-files-glob`      | string | `''`    | Which tests to run, relative to `test/platform`. Empty runs everything in it |
-| `subtest`              | string | `''`    | Deprecated, use `test-files-glob`                                            |
-| `report-slack-channel` | string | `''`    | Slack channel for the results. Empty skips Slack                             |
+| Input                  | Type   | Default            | Description                                                                                                                 |
+| ---------------------- | ------ | ------------------ | --------------------------------------------------------------------------------------------------------------------------- |
+| `test-files-glob`      | string | `test/platform/**` | Glob of the test files to run, from the repo root. Empty means the default. See [Splitting the tests](#splitting-the-tests) |
+| `report-slack-channel` | string | `''`               | Slack channel for the results. Empty skips Slack                                                                            |
 
 Secrets: `TESTER_APIFY_TOKEN` (required), `SLACK_BOT_TOKEN`, and optionally `NPM_TOKEN`.
 
-How often to run which tests is up to you. For example, every test once a day:
+How often to run which tests is up to you, see [Splitting the tests](#splitting-the-tests). For example, the daily
+tests once a day:
 
 ```yaml
 name: Platform tests - Daily
@@ -169,13 +173,14 @@ jobs:
     platformTestsDaily:
         uses: apify/apify-test-tools/.github/workflows/public_platform-tests.yaml@workflows-v0
         with:
+            test-files-glob: 'test/platform/{daily,daily-without-pr}/**'
             report-slack-channel: '#notif-my-repo'
         secrets:
             TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}
             SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
 ```
 
-and a few critical tests, kept in e.g. `test/platform/hourly`, every hour:
+and a few critical tests every hour:
 
 ```yaml
 name: Platform tests - Hourly
@@ -190,13 +195,41 @@ jobs:
     platformTestsHourly:
         uses: apify/apify-test-tools/.github/workflows/public_platform-tests.yaml@workflows-v0
         with:
-            test-files-glob: hourly
+            test-files-glob: 'test/platform/{hourly,hourly-without-pr}/**'
             report-slack-channel: '#notif-my-repo'
         secrets:
             TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}
             SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
 ```
 
+## Splitting the tests
+
+`test-files-glob` is one glob from the repo root, for example `test/platform/**`. Both test workflows take it, and it
+only selects which tests run. Which changed files count as test changes is a separate setting, `platformTestsGlob` in
+the config file (default `test/platform/**`). So on a PR, a change to a `*-without-pr` test or to a shared helper
+still builds and tests the Actors, even though the PR doesn't run that test.
+
+Most repos split `test/platform` into folders by how often the tests run, and by whether a PR runs them:
+
+| Folder                            | Runs                                                            |
+| --------------------------------- | --------------------------------------------------------------- |
+| `test/platform/hourly`            | every hour and on every PR                                      |
+| `test/platform/daily`             | once a day and on every PR                                      |
+| `test/platform/hourly-without-pr` | every hour only, e.g. tests that are too slow or costly for PRs |
+| `test/platform/daily-without-pr`  | once a day only                                                 |
+
+Each workflow then selects its folders with a brace pattern:
+
+| Job                                   | `test-files-glob`                             |
+| ------------------------------------- | --------------------------------------------- |
+| PR (`public_pr-build-test.yaml`)      | `test/platform/{daily,hourly}/**`             |
+| Hourly (`public_platform-tests.yaml`) | `test/platform/{hourly,hourly-without-pr}/**` |
+| Daily (`public_platform-tests.yaml`)  | `test/platform/{daily,daily-without-pr}/**`   |
+
+Write the glob to match files, not folders: `test/platform/hourly` matches no file, `test/platform/hourly/**` matches
+every file in it. A glob that matches no file fails the test step, so a typo doesn't pass as a run with no tests.
+`test` only runs the matched files that vitest sees as test files, so a helper in the glob runs nothing.
+
 ## `public_review.yaml`
 
 Optional. Reviews a PR against the shared guidelines when you add the trigger label, and again on every push while
@@ -290,7 +323,7 @@ jobs:
     platformTestsHourly:
         uses: apify/apify-test-tools/.github/workflows/public_platform-tests.yaml@workflows-v0
         with:
-            test-files-glob: hourly
+            test-files-glob: 'test/platform/{hourly,hourly-without-pr}/**'
             report-slack-channel: '#notif-my-repo'
         secrets:
             TESTER_APIFY_TOKEN: ${{ secrets.TESTER_APIFY_TOKEN }}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "apify-test-tools",
-  "version": "0.10.2-beta.19",
+  "version": "0.10.2-beta.20",
   "type": "module",
   "description": "TBD",
   "repository": {

package/test/bundle/cli-bundle.test.ts

@@ -91,7 +91,7 @@ describe('CLI bundle', () => {
         for (const command of [
             'get-actor-configs',
             'get-affected-actors',
-            'test [filters..]',
+            'test',
             'notify',
             'build',
             'build-from-local',

package/test/e2e/cli-bundle.e2e.test.ts

@@ -165,11 +165,15 @@ describe('sandbox dummies', () => {${actorTests}
 `,
         );
 
-        const tested = await runProcess(process.execPath, [bundlePath, 'test', 'test/platform'], {
-            cwd: repoDir,
-            env: { ...cliEnv, TESTER_APIFY_TOKEN: token! },
-            input: JSON.stringify(builds),
-        });
+        const tested = await runProcess(
+            process.execPath,
+            [bundlePath, 'test', '--test-files-glob', 'test/platform/**'],
+            {
+                cwd: repoDir,
+                env: { ...cliEnv, TESTER_APIFY_TOKEN: token! },
+                input: JSON.stringify(builds),
+            },
+        );
         expect(tested.status, tested.stderr).toBe(0);
         expect(tested.stderr).toContain('https://console.apify.com/view/runs/');
 

package/test/unit/bin/actor-filtering.test.ts

@@ -36,6 +36,14 @@ describe('selectActors', () => {
         ]);
     });
 
+    it('selects an actor listed twice once, in config order', () => {
+        // build relies on this to build an Actor in both functionallyChangedActors and testChangedActors once
+        expect(names(selectActors({ actors: ['owner/c', 'owner/a', 'owner/c'], ignore: [] }, configs))).toStrictEqual([
+            'owner/a',
+            'owner/c',
+        ]);
+    });
+
     it('drops the actors listed in --ignore', () => {
         expect(names(selectActors({ actors: [], ignore: ['owner/b'] }, configs))).toStrictEqual(['owner/a', 'owner/c']);
     });

package/test/unit/bin/affected-actors-cli.test.ts

@@ -72,10 +72,31 @@ const runGetAffectedActors = (...args: string[]) => {
     return JSON.parse(result.stdout);
 };
 
+const names = (actors: { actorFullName: string }[]) =>
+    actors.map(({ actorFullName }) => actorFullName.replace('owner/actor-', '')).sort();
+
+/** All three lists, for the tests about test changes */
+const getAllChangedActors = (...args: string[]) => {
+    const document = runGetAffectedActors(...args);
+    return {
+        functional: names(document.functionallyChangedActors),
+        cosmetic: names(document.cosmeticallyChangedActors),
+        test: names(document.testChangedActors),
+    };
+};
+
+/** Writes the config with actor-a's context also covering `extraContext`. It is committed with the next commit. */
+const writeConfigWithTestsInContext = async (extraContext: string, settings: Record<string, unknown> = {}) => {
+    const actors = CONFIG.actors.map((actor) =>
+        actor.folder === 'actors/a'
+            ? { ...actor, overrideActorContext: [...actor.overrideActorContext!, extraContext] }
+            : actor,
+    );
+    await writeFile(path.join(repoDir, 'apify-test-tools.config.json'), JSON.stringify({ ...settings, actors }));
+};
+
 const getAffectedActors = (...args: string[]) => {
     const document = runGetAffectedActors(...args);
-    const names = (actors: { actorFullName: string }[]) =>
-        actors.map(({ actorFullName }) => actorFullName.replace('owner/actor-', '')).sort();
     return {
         functional: names(document.functionallyChangedActors),
         cosmetic: names(document.cosmeticallyChangedActors),
@@ -191,8 +212,57 @@ describe('get-affected-actors CLI on real git histories', () => {
         expect(getAffectedActors('--base', 'main', '--since', 'e'.repeat(40))).toStrictEqual(wholeBranch);
     });
 
+    it('lists Actors whose platform tests changed, and a test change alone is not functional', async () => {
+        await writeConfigWithTestsInContext('test');
+        await writeFiles('test/', ['.dockerignore']);
+        // A test and its helper, both under the default platformTestsGlob test/platform/**
+        await commit('test(a): change a test and its helper', 'test/platform/a.test.ts', 'test/platform/utils.ts');
+
+        const testOnly = { functional: [], cosmetic: [], test: ['a'] };
+        // A PR runs the tests
+        expect(getAllChangedActors('--base', 'main')).toStrictEqual(testOnly);
+        // A release builds only the functional and cosmetic lists, so it builds nothing
+        expect(getAllChangedActors('--since', git('rev-parse', 'main'))).toStrictEqual(testOnly);
+    });
+
+    it('lists an Actor in both lists when its source and its tests changed', async () => {
+        await writeConfigWithTestsInContext('test');
+        await commit('feat(a): change and test it', 'actors/a/main.ts', 'test/platform/a.test.ts');
+
+        expect(getAllChangedActors('--base', 'main')).toStrictEqual({ functional: ['a'], cosmetic: [], test: ['a'] });
+    });
+
+    it('lists only the tests changed since the last validated commit', async () => {
+        await writeConfigWithTestsInContext('test');
+        const validated = await commit('feat(a): validated change', 'actors/a/main.ts');
+        await commit('test(a): new test', 'test/platform/a.test.ts');
+
+        // The validated build of A has the branch's code, only the tests changed since
+        expect(getAllChangedActors('--base', 'main', '--since', validated)).toStrictEqual({
+            functional: [],
+            cosmetic: [],
+            test: ['a'],
+        });
+    });
+
+    it('reads which files are platform tests from platformTestsGlob in the config', async () => {
+        await writeConfigWithTestsInContext('e2e', { platformTestsGlob: 'e2e/platform/**' });
+        await commit('test(a): new test', 'e2e/platform/a.test.ts');
+        expect(getAllChangedActors('--base', 'main')).toStrictEqual({ functional: [], cosmetic: [], test: ['a'] });
+
+        // Outside the glob, a file in the context is a source change like any other
+        await commit('feat(a): other file', 'e2e/fixtures/data.json');
+        expect(getAllChangedActors('--base', 'main')).toStrictEqual({ functional: ['a'], cosmetic: [], test: ['a'] });
+    });
+
     it('reports nothing for an empty range', async () => {
-        const nothing = { commits: [], changedFiles: [], functionallyChangedActors: [], cosmeticallyChangedActors: [] };
+        const nothing = {
+            commits: [],
+            changedFiles: [],
+            functionallyChangedActors: [],
+            cosmeticallyChangedActors: [],
+            testChangedActors: [],
+        };
         // The branch has no commits of its own yet
         expect(runGetAffectedActors('--base', 'main')).toMatchObject(nothing);
         // A release with nothing new since the last one

package/test/unit/bin/affected-actors.test.ts

@@ -24,9 +24,10 @@ describe('getAffectedActors', () => {
 
     beforeEach(() => {
         vi.spyOn(LoadConfig, 'readConfigFile').mockResolvedValue([actorA, actorB]);
+        vi.spyOn(LoadConfig, 'readRepoSettings').mockResolvedValue({ platformTestsGlob: 'test/platform/**' });
         changedActorsSpy = vi
             .spyOn(DiffChanges, 'getChangedActors')
-            .mockResolvedValue({ functional: [actorA], cosmetic: [actorB] });
+            .mockResolvedValue({ functional: [actorA], cosmetic: [actorB], test: [] });
         vi.spyOn(Git, 'resolveCommitSha').mockResolvedValue(headSha);
         vi.spyOn(Git, 'getChangedFilesBetween').mockResolvedValue(changedFiles);
         vi.spyOn(Git, 'getChangelogAdditions').mockResolvedValue('- Added foo');
@@ -62,6 +63,7 @@ describe('getAffectedActors', () => {
                 // Only the names, the config file stays the source of truth for the rest
                 functionallyChangedActors: [{ actorFullName: 'owner/a' }],
                 cosmeticallyChangedActors: [{ actorFullName: 'owner/b' }],
+                testChangedActors: [],
             });
             expect(Git.getMergeBase).toHaveBeenCalledWith('origin/master', 'HEAD');
             expect(Git.getChangedFilesBetween).toHaveBeenCalledWith(mergeBaseSha, headSha);
@@ -71,6 +73,7 @@ describe('getAffectedActors', () => {
                 filepathsChanged: changedFiles,
                 actorConfigs: [actorA, actorB],
                 range: { fromRef: mergeBaseSha, toRef: headSha },
+                platformTestsGlob: 'test/platform/**',
             });
             expect(findCommitSpy).not.toHaveBeenCalled();
         });
@@ -96,15 +99,20 @@ describe('getAffectedActors', () => {
                 return {
                     functional: result.functionallyChangedActors.map(({ actorFullName }) => actorFullName),
                     cosmetic: result.cosmeticallyChangedActors.map(({ actorFullName }) => actorFullName),
+                    test: result.testChangedActors.map(({ actorFullName }) => actorFullName),
                 };
             };
 
             it('classifies the changes since --since separately', async () => {
-                await getNarrowed({ functional: [actorA], cosmetic: [] }, { functional: [actorA], cosmetic: [] });
+                await getNarrowed(
+                    { functional: [actorA], cosmetic: [], test: [] },
+                    { functional: [actorA], cosmetic: [], test: [] },
+                );
                 expect(changedActorsSpy).toHaveBeenCalledWith({
                     filepathsChanged: sinceFiles,
                     actorConfigs: [actorA, actorB],
                     range: { fromRef: sinceSha, toRef: headSha },
+                    platformTestsGlob: 'test/platform/**',
                 });
             });
 
@@ -116,25 +124,55 @@ describe('getAffectedActors', () => {
 
             it('retests an Actor whose validated change meets a new functional change', async () => {
                 await expect(
-                    getNarrowed({ functional: [actorA], cosmetic: [] }, { functional: [actorA], cosmetic: [] }),
-                ).resolves.toStrictEqual({ functional: ['owner/a'], cosmetic: [] });
+                    getNarrowed(
+                        { functional: [actorA], cosmetic: [], test: [] },
+                        { functional: [actorA], cosmetic: [], test: [] },
+                    ),
+                ).resolves.toStrictEqual({ functional: ['owner/a'], cosmetic: [], test: [] });
             });
 
             it('leaves out Actors that only the branch changed before --since or only --base changed', async () => {
                 await expect(
-                    getNarrowed({ functional: [actorA], cosmetic: [] }, { functional: [actorB], cosmetic: [] }),
-                ).resolves.toStrictEqual({ functional: [], cosmetic: [] });
+                    getNarrowed(
+                        { functional: [actorA], cosmetic: [], test: [] },
+                        { functional: [actorB], cosmetic: [], test: [] },
+                    ),
+                ).resolves.toStrictEqual({ functional: [], cosmetic: [], test: [] });
             });
 
             it('takes the less impactful change when the two disagree', async () => {
                 // A validated functional change, then only cosmetic changes (e.g. a README merged in from base)
                 await expect(
-                    getNarrowed({ functional: [actorA], cosmetic: [] }, { functional: [], cosmetic: [actorA] }),
-                ).resolves.toStrictEqual({ functional: [], cosmetic: ['owner/a'] });
+                    getNarrowed(
+                        { functional: [actorA], cosmetic: [], test: [] },
+                        { functional: [], cosmetic: [actorA], test: [] },
+                    ),
+                ).resolves.toStrictEqual({ functional: [], cosmetic: ['owner/a'], test: [] });
                 // A functional change merged in from base, while the branch only changes the README
                 await expect(
-                    getNarrowed({ functional: [], cosmetic: [actorA] }, { functional: [actorA], cosmetic: [] }),
-                ).resolves.toStrictEqual({ functional: [], cosmetic: ['owner/a'] });
+                    getNarrowed(
+                        { functional: [], cosmetic: [actorA], test: [] },
+                        { functional: [actorA], cosmetic: [], test: [] },
+                    ),
+                ).resolves.toStrictEqual({ functional: [], cosmetic: ['owner/a'], test: [] });
+            });
+
+            it('runs the tests of an Actor whose tests the branch changed and changed again since --since', async () => {
+                await expect(
+                    getNarrowed(
+                        { functional: [actorA], cosmetic: [], test: [actorA, actorB] },
+                        { functional: [], cosmetic: [], test: [actorA] },
+                    ),
+                ).resolves.toStrictEqual({ functional: [], cosmetic: [], test: ['owner/a'] });
+            });
+
+            it('leaves out test changes that only --base or only the validated part of the branch has', async () => {
+                await expect(
+                    getNarrowed(
+                        { functional: [], cosmetic: [], test: [actorA] },
+                        { functional: [], cosmetic: [], test: [actorB] },
+                    ),
+                ).resolves.toStrictEqual({ functional: [], cosmetic: [], test: [] });
             });
 
             it.each([
@@ -172,6 +210,7 @@ describe('getAffectedActors', () => {
                 // Only the names, the config file stays the source of truth for the rest
                 functionallyChangedActors: [{ actorFullName: 'owner/a' }],
                 cosmeticallyChangedActors: [{ actorFullName: 'owner/b' }],
+                testChangedActors: [],
             });
             expect(resolveBaseSpy).toHaveBeenCalledWith('HEAD~2');
             expect(commitsBetweenSpy).toHaveBeenCalledWith(sinceSha, headSha);