apify-test-tools 0.10.2-beta.3 → 0.10.2-beta.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/cli/apify-test-tools.mjs +117 -118
- package/.github/scripts/run-with-apify-tokens.mjs +1 -1
- package/.github/workflows/public_pr-build-test.yaml +25 -25
- package/.github/workflows/public_push-build-latest.yaml +20 -4
- package/CHANGELOG.md +6 -0
- package/CONTRIBUTING.md +4 -1
- package/README.md +104 -56
- package/bin/affected-actors.ts +135 -103
- package/bin/build.ts +53 -38
- package/bin/diff-changes.ts +7 -6
- package/bin/diff-json-schema.ts +5 -6
- package/bin/git.ts +24 -69
- package/bin/main.ts +46 -80
- package/bin/slack.ts +15 -4
- package/bin/stdin-input.ts +65 -0
- package/bin/types.ts +6 -0
- package/dist/bin/affected-actors.d.ts +23 -27
- package/dist/bin/affected-actors.d.ts.map +1 -1
- package/dist/bin/affected-actors.js +111 -65
- package/dist/bin/affected-actors.js.map +1 -1
- package/dist/bin/build.d.ts +11 -5
- package/dist/bin/build.d.ts.map +1 -1
- package/dist/bin/build.js +43 -23
- package/dist/bin/build.js.map +1 -1
- package/dist/bin/diff-changes.d.ts +4 -3
- package/dist/bin/diff-changes.d.ts.map +1 -1
- package/dist/bin/diff-changes.js +4 -4
- package/dist/bin/diff-changes.js.map +1 -1
- package/dist/bin/diff-json-schema.d.ts +2 -2
- package/dist/bin/diff-json-schema.d.ts.map +1 -1
- package/dist/bin/diff-json-schema.js +1 -4
- package/dist/bin/diff-json-schema.js.map +1 -1
- package/dist/bin/git.d.ts +7 -15
- package/dist/bin/git.d.ts.map +1 -1
- package/dist/bin/git.js +23 -61
- package/dist/bin/git.js.map +1 -1
- package/dist/bin/main.js +39 -71
- package/dist/bin/main.js.map +1 -1
- package/dist/bin/slack.d.ts +6 -3
- package/dist/bin/slack.d.ts.map +1 -1
- package/dist/bin/slack.js +13 -1
- package/dist/bin/slack.js.map +1 -1
- package/dist/bin/stdin-input.d.ts +17 -0
- package/dist/bin/stdin-input.d.ts.map +1 -0
- package/dist/bin/stdin-input.js +54 -0
- package/dist/bin/stdin-input.js.map +1 -0
- package/dist/bin/types.d.ts +5 -0
- package/dist/bin/types.d.ts.map +1 -1
- package/dist/test/bundle/cli-bundle.test.js +32 -11
- package/dist/test/bundle/cli-bundle.test.js.map +1 -1
- package/dist/test/e2e/cli-bundle.e2e.test.js +23 -57
- package/dist/test/e2e/cli-bundle.e2e.test.js.map +1 -1
- package/dist/test/e2e/git-build-release.e2e.test.d.ts +2 -0
- package/dist/test/e2e/git-build-release.e2e.test.d.ts.map +1 -0
- package/dist/test/e2e/git-build-release.e2e.test.js +87 -0
- package/dist/test/e2e/git-build-release.e2e.test.js.map +1 -0
- package/dist/test/e2e/helpers.d.ts +37 -0
- package/dist/test/e2e/helpers.d.ts.map +1 -0
- package/dist/test/e2e/helpers.js +57 -0
- package/dist/test/e2e/helpers.js.map +1 -0
- package/dist/test/unit/bin/affected-actors-cli.test.d.ts +2 -0
- package/dist/test/unit/bin/affected-actors-cli.test.d.ts.map +1 -0
- package/dist/test/unit/bin/affected-actors-cli.test.js +161 -0
- package/dist/test/unit/bin/affected-actors-cli.test.js.map +1 -0
- package/dist/test/unit/bin/affected-actors.test.js +68 -87
- package/dist/test/unit/bin/affected-actors.test.js.map +1 -1
- package/dist/test/unit/bin/build-cli.test.d.ts +2 -0
- package/dist/test/unit/bin/build-cli.test.d.ts.map +1 -0
- package/dist/test/unit/bin/build-cli.test.js +44 -0
- package/dist/test/unit/bin/build-cli.test.js.map +1 -0
- package/dist/test/unit/bin/build.test.js +61 -4
- package/dist/test/unit/bin/build.test.js.map +1 -1
- package/dist/test/unit/bin/compiled-cli.d.ts +6 -0
- package/dist/test/unit/bin/compiled-cli.d.ts.map +1 -0
- package/dist/test/unit/bin/compiled-cli.js +16 -0
- package/dist/test/unit/bin/compiled-cli.js.map +1 -0
- package/dist/test/unit/bin/diff-changes.test.js +55 -55
- package/dist/test/unit/bin/diff-changes.test.js.map +1 -1
- package/dist/test/unit/bin/diff.test.js +25 -28
- package/dist/test/unit/bin/diff.test.js.map +1 -1
- package/dist/test/unit/bin/git.test.js +28 -108
- package/dist/test/unit/bin/git.test.js.map +1 -1
- package/dist/test/unit/bin/slack.test.js +19 -2
- package/dist/test/unit/bin/slack.test.js.map +1 -1
- package/dist/test/unit/bin/stdin-input.test.d.ts +2 -0
- package/dist/test/unit/bin/stdin-input.test.d.ts.map +1 -0
- package/dist/test/unit/bin/stdin-input.test.js +50 -0
- package/dist/test/unit/bin/stdin-input.test.js.map +1 -0
- package/dist/test/unit/should-built-and-test.test.js +18 -21
- package/dist/test/unit/should-built-and-test.test.js.map +1 -1
- package/dist/tsconfig.tsbuildinfo +1 -1
- package/package.json +1 -1
- package/test/bundle/cli-bundle.test.ts +47 -11
- package/test/e2e/cli-bundle.e2e.test.ts +47 -77
- package/test/e2e/git-build-release.e2e.test.ts +113 -0
- package/test/e2e/helpers.ts +82 -0
- package/test/unit/bin/affected-actors-cli.test.ts +193 -0
- package/test/unit/bin/affected-actors.test.ts +86 -94
- package/test/unit/bin/build-cli.test.ts +55 -0
- package/test/unit/bin/build.test.ts +71 -3
- package/test/unit/bin/compiled-cli.ts +17 -0
- package/test/unit/bin/diff-changes.test.ts +55 -55
- package/test/unit/bin/diff.test.ts +25 -28
- package/test/unit/bin/git.test.ts +35 -132
- package/test/unit/bin/slack.test.ts +21 -2
- package/test/unit/bin/stdin-input.test.ts +61 -0
- package/test/unit/should-built-and-test.test.ts +19 -22
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
#!/usr/bin/env node
|
|
2
2
|
// Runs apify-test-tools and passes it only secrets needed for the Actors, no other secrets are leaked.
|
|
3
3
|
//
|
|
4
|
-
// node .github/scripts/run-with-apify-tokens.mjs apify-test-tools build --
|
|
4
|
+
// jq ... | node .github/scripts/run-with-apify-tokens.mjs apify-test-tools build --version-number 0.99
|
|
5
5
|
//
|
|
6
6
|
// The step passes the whole secrets map as ALL_SECRETS. The token names come from `tokenEnvVar` in
|
|
7
7
|
// the repo's apify-test-tools.config.json, which is the same file apify-test-tools itself reads, so
|
|
@@ -41,31 +41,39 @@ jobs:
|
|
|
41
41
|
# This is needed to force overriding the cache with new entry at the end
|
|
42
42
|
restore-keys: base_commit-
|
|
43
43
|
|
|
44
|
-
#
|
|
45
|
-
#
|
|
46
|
-
|
|
47
|
-
- name: Build
|
|
48
|
-
id: build
|
|
44
|
+
# Needs no tokens, it only reads git and apify-test-tools.config.json. The setup action checks out
|
|
45
|
+
# the PR branch itself (not GitHub's merge commit), so the range ends at the branch's HEAD.
|
|
46
|
+
- name: Find affected Actors
|
|
49
47
|
env:
|
|
50
|
-
ALL_SECRETS: ${{ toJSON(secrets) }}
|
|
51
48
|
# Branch names are attacker-controlled on fork PRs, so they reach the script as
|
|
52
49
|
# environment variables rather than being interpolated into it.
|
|
53
|
-
HEAD_REF: ${{ github.head_ref }}
|
|
54
50
|
BASE_REF: ${{ github.base_ref }}
|
|
55
|
-
SCRIPTS_PATH: ${{ steps.setup.outputs.scripts-path }}
|
|
56
51
|
run: |
|
|
57
52
|
base_commit=$(cat base_commit.txt 2>/dev/null || echo "")
|
|
58
53
|
echo "Last validated (base) commit to be used for build & test: $base_commit"
|
|
59
54
|
args=(
|
|
60
|
-
--
|
|
61
|
-
--target-branch "origin/$BASE_REF"
|
|
55
|
+
--base "origin/$BASE_REF"
|
|
62
56
|
# TODO: Temporary hotfix until build glob matching aligns with Vitest's test selection glob.
|
|
63
57
|
--test-files-glob 'test/platform/**'
|
|
64
|
-
--use-docker-cache
|
|
65
58
|
)
|
|
66
|
-
[ -n "$base_commit" ] && args+=(--
|
|
67
|
-
|
|
68
|
-
|
|
59
|
+
[ -n "$base_commit" ] && args+=(--since "$base_commit")
|
|
60
|
+
apify-test-tools get-affected-actors "${args[@]}" > affected.json
|
|
61
|
+
|
|
62
|
+
# Cosmetic-only changes need no tests, so only the functionally changed Actors are built, into the
|
|
63
|
+
# test version 0.99 that users never run.
|
|
64
|
+
# run-with-apify-tokens.mjs reads the token names from in apify-test-tools.config.json
|
|
65
|
+
# and hands the build those secrets and nothing else, so the tokens stay in this one command instead of the whole job.
|
|
66
|
+
# The library never sees secrets it doesn't need
|
|
67
|
+
- name: Build
|
|
68
|
+
id: build
|
|
69
|
+
# bash rather than the default shell, for pipefail
|
|
70
|
+
shell: bash
|
|
71
|
+
env:
|
|
72
|
+
ALL_SECRETS: ${{ toJSON(secrets) }}
|
|
73
|
+
SCRIPTS_PATH: ${{ steps.setup.outputs.scripts-path }}
|
|
74
|
+
run: |
|
|
75
|
+
actor_builds=$(jq '{ actors: .functionallyChangedActors }' affected.json \
|
|
76
|
+
| node "$SCRIPTS_PATH/run-with-apify-tokens.mjs" apify-test-tools build --version-number 0.99)
|
|
69
77
|
echo "actor_builds=$actor_builds" | tee -a "$GITHUB_OUTPUT"
|
|
70
78
|
|
|
71
79
|
# Runs the repo's test code and its whole import graph, so it gets the tester token only.
|
|
@@ -77,18 +85,10 @@ jobs:
|
|
|
77
85
|
|
|
78
86
|
# NOTE: This is an optimization that if we did functional changes and later only cosmetic changes (e.g. dev readme), we will compare changes files only for the last commit. This must run after tests because we want to cache the latest commit only if the tests are successful
|
|
79
87
|
- name: Store last validated commit to cache
|
|
80
|
-
env:
|
|
81
|
-
HEAD_REF: ${{ github.head_ref }}
|
|
82
|
-
BASE_REF: ${{ github.base_ref }}
|
|
83
88
|
run: |
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
--source-branch "origin/$HEAD_REF"
|
|
88
|
-
--target-branch "origin/$BASE_REF"
|
|
89
|
-
)
|
|
90
|
-
[ -n "$base_commit" ] && args+=(--base-commit "$base_commit")
|
|
91
|
-
apify-test-tools get-latest-commit "${args[@]}" > ./base_commit.txt
|
|
89
|
+
echo "Old last validated (base) commit: $(cat base_commit.txt 2>/dev/null || echo "")"
|
|
90
|
+
# The HEAD that was just built and tested
|
|
91
|
+
jq -r .head affected.json > base_commit.txt
|
|
92
92
|
echo "New last validated (base) commit: $(cat base_commit.txt)"
|
|
93
93
|
|
|
94
94
|
# Pure static checks against the repo. Needs no Apify or Slack credentials, only the npm
|
|
@@ -25,18 +25,34 @@ jobs:
|
|
|
25
25
|
with:
|
|
26
26
|
npm-token: ${{ secrets.NPM_TOKEN }}
|
|
27
27
|
|
|
28
|
-
# --
|
|
28
|
+
# --since is the last commit that is already released: the push's "before" commit.
|
|
29
29
|
# Everything after it up to HEAD gets released. See the "Releasing Actors" section of the README.
|
|
30
|
+
# Needs no tokens, it only reads git and apify-test-tools.config.json.
|
|
31
|
+
- name: Find changes since the last release
|
|
32
|
+
run: |
|
|
33
|
+
apify-test-tools get-affected-actors --since ${{ github.event.before }} > affected.json
|
|
34
|
+
|
|
35
|
+
# A release ships cosmetic changes too (README, input schema descriptions), so both lists are built,
|
|
36
|
+
# into each Actor's default version: the one its default build tag points to, which users run.
|
|
30
37
|
# run-with-apify-tokens.mjs reads the token names from in apify-test-tools.config.json
|
|
31
38
|
# and hands the build those secrets and nothing else, so the tokens stay in this one command instead of the whole job.
|
|
32
39
|
# The library never sees secrets it doesn't need
|
|
33
|
-
- name:
|
|
40
|
+
- name: Build the default versions
|
|
41
|
+
# bash rather than the default shell, for pipefail
|
|
42
|
+
shell: bash
|
|
34
43
|
env:
|
|
35
44
|
ALL_SECRETS: ${{ toJSON(secrets) }}
|
|
45
|
+
run: |
|
|
46
|
+
jq '{ actors: (.functionallyChangedActors + .cosmeticallyChangedActors) }' affected.json \
|
|
47
|
+
| node "${{ steps.setup.outputs.scripts-path }}/run-with-apify-tokens.mjs" \
|
|
48
|
+
apify-test-tools build --version-number default
|
|
49
|
+
|
|
50
|
+
# Skips the announcement when the push brought no new commits
|
|
51
|
+
- name: Announce the release on Slack
|
|
52
|
+
env:
|
|
36
53
|
SLACK_TOKEN_RELEASES_BOT: ${{ secrets.SLACK_TOKEN_RELEASES_BOT }}
|
|
37
54
|
run: |
|
|
38
|
-
|
|
39
|
-
apify-test-tools release --base-commit ${{ github.event.before }} --release-slack-channel "#delivery-public-actors" --report-slack-channel "${{ inputs.report-slack-channel || format('#notif-{0}', github.event.repository.name) }}" --use-docker-cache
|
|
55
|
+
apify-test-tools notify-release --release-slack-channel "#delivery-public-actors" --report-slack-channel "${{ inputs.report-slack-channel || format('#notif-{0}', github.event.repository.name) }}" < affected.json
|
|
40
56
|
|
|
41
57
|
- name: Delete old builds
|
|
42
58
|
env:
|
package/CHANGELOG.md
CHANGED
|
@@ -9,12 +9,18 @@ All notable changes to this project will be documented in this file.
|
|
|
9
9
|
|
|
10
10
|
- **ci:** Ship the workflows' CLI as a bundle in the action instead of installing it ([#200](https://github.com/apify/apify-test-tools/pull/200)) ([a0ef975](https://github.com/apify/apify-test-tools/commit/a0ef9756123a30a8210590b080a2a289c6a9c17c)) by [@metalwarrior665](https://github.com/metalwarrior665)
|
|
11
11
|
- [**breaking**] Revamp get-affected-actors to be a core logic command ([#190](https://github.com/apify/apify-test-tools/pull/190)) ([ea5f244](https://github.com/apify/apify-test-tools/commit/ea5f244cfd42bf3fa9ffd38cd2ad499dde2e83f8)) by [@metalwarrior665](https://github.com/metalwarrior665)
|
|
12
|
+
- [**breaking**] Add logic-less build command and use command chaining in workflows ([#193](https://github.com/apify/apify-test-tools/pull/193)) ([e5b0b86](https://github.com/apify/apify-test-tools/commit/e5b0b86ee06fca51d4be0ddc81b0a4a469d31603)) by [@metalwarrior665](https://github.com/metalwarrior665)
|
|
13
|
+
- **get-affected-actors:** Three-dot diff, narrowed per Actor with --since ([#204](https://github.com/apify/apify-test-tools/pull/204)) ([fa9a0e7](https://github.com/apify/apify-test-tools/commit/fa9a0e7518764dd4722ec9ecbf67b2a26a5a062e)) by [@metalwarrior665](https://github.com/metalwarrior665)
|
|
12
14
|
|
|
13
15
|
### 🐛 Bug Fixes
|
|
14
16
|
|
|
15
17
|
- **toBeObject:** Reject null ([#191](https://github.com/apify/apify-test-tools/pull/191)) ([6bdae49](https://github.com/apify/apify-test-tools/commit/6bdae4905756e0ae8dd6146942e6fc04dbead48e)) by [@JuanGalilea](https://github.com/JuanGalilea), closes [#88](https://github.com/apify/apify-test-tools/issues/88)
|
|
16
18
|
- Build and test on test-only changes even when .dockerignore lists the tests ([#177](https://github.com/apify/apify-test-tools/pull/177)) ([5cb1b5a](https://github.com/apify/apify-test-tools/commit/5cb1b5ab5f6f43ec279f9624b402e73d55a5974a)) by [@artogahr](https://github.com/artogahr), closes [#145](https://github.com/apify/apify-test-tools/issues/145)
|
|
17
19
|
|
|
20
|
+
### 🚜 Refactor
|
|
21
|
+
|
|
22
|
+
- [**breaking**] Remove build-for-test and release composite commands ([#203](https://github.com/apify/apify-test-tools/pull/203)) ([85efc54](https://github.com/apify/apify-test-tools/commit/85efc54697e98208c827c4f450906e00eebcd9ca)) by [@metalwarrior665](https://github.com/metalwarrior665)
|
|
23
|
+
|
|
18
24
|
|
|
19
25
|
<!-- git-cliff-unreleased-end -->
|
|
20
26
|
## [0.10.1](https://github.com/apify/apify-test-tools/releases/tag/v0.10.1) (2026-10-13)
|
package/CONTRIBUTING.md
CHANGED
|
@@ -94,9 +94,12 @@ npm release relate — the short version:
|
|
|
94
94
|
- the CLI runs from `.github/cli/apify-test-tools.mjs`, a bundle the pre-commit hook rebuilds with
|
|
95
95
|
`npm run bundle`. If you commit with `--no-verify`, run it yourself: the `CLI bundle` check fails
|
|
96
96
|
on a stale bundle, and a stale bundle would hold the tag.
|
|
97
|
-
- `npm test` covers the bundle without network (`test/bundle/`). `npm run test:e2e` runs it against
|
|
97
|
+
- `npm test` covers the bundle without network (`test/bundle/`). `npm run test:e2e` runs it against
|
|
98
98
|
dummy Actors in an isolated sandbox account (`test/e2e/`) and skips itself unless
|
|
99
99
|
`E2E_TEST_APIFY_TOKEN_DBYPODOIHUZABNQTR` is set. CI passes that secret only to the `E2E tests` job.
|
|
100
|
+
The git `build` and release pipeline tests clone
|
|
101
|
+
[testing-repo-for-github-actions](https://github.com/apify-store/testing-repo-for-github-actions) and
|
|
102
|
+
use its `e2e/base` and `e2e/feature` branches as fixtures, so never rebase or delete those.
|
|
100
103
|
|
|
101
104
|
`npm run lint` and `actionlint` (via the `Code checks` workflow) both gate master, so run them before
|
|
102
105
|
pushing workflow changes.
|
package/README.md
CHANGED
|
@@ -57,7 +57,7 @@ When a PR is opened or code is pushed, the tool determines which actors need to
|
|
|
57
57
|
2. **CHANGELOG classification** — a `CHANGELOG.md` file is always `cosmetic` (only triggers a release build, not tests), for every actor, regardless of context or folder. (See [issue #106](https://github.com/apify/apify-test-tools/issues/106).)
|
|
58
58
|
3. **Context matching** — the file must fall within one of the actor's context paths (`dockerContextDir` from `actor.json` by default, or `overrideActorContext` from config if set). Files outside every context path are skipped.
|
|
59
59
|
4. **Hardcoded ignore list, context-aware** — the file path is first "hoisted" relative to the context path it matched (e.g. a standalone actor's own `.eslintrc` is checked as just `.eslintrc`, not the full repo-root-relative path), then checked against repo-level dev file patterns (`.vscode/`, `.gitignore`, `.husky/`, `.eslintrc`, `eslint.config.mjs`, `.prettierrc`, `.editorconfig`). There's no hardcoded special-casing for legacy `code/`/`shared/` layouts — repos that need those directories treated as top-level must list them explicitly in `overrideActorContext`.
|
|
60
|
-
5. **`.dockerignore` filtering** — if a `.dockerignore` exists at the root of the actor's `dockerContextDir`, matching files are ignored. Patterns are resolved relative to `dockerContextDir`, matching Docker's own behavior. Test files are the exception: `
|
|
60
|
+
5. **`.dockerignore` filtering** — if a `.dockerignore` exists at the root of the actor's `dockerContextDir`, matching files are ignored. Patterns are resolved relative to `dockerContextDir`, matching Docker's own behavior. Test files are the exception: `get-affected-actors --test-files-glob <glob>` (relative to the repo root) skips this check for matching files, so a changed test triggers a build, and with it its tests, even when `.dockerignore` lists it. The PR workflow passes `test/platform/**`. Context matching still applies, so tests outside an actor's `dockerContextDir` (e.g. a root `test/` with standalone actors) need to be listed in its `overrideActorContext`; `get-affected-actors` logs a warning for changed test files outside every actor's context.
|
|
61
61
|
6. **README classification** — a `README.md` file is `cosmetic` (only triggers a release build, not tests) if it lives inside the actor's own `folder`; otherwise it's ignored entirely, since it isn't documentation for this actor.
|
|
62
62
|
7. **Cosmetic JSON classification** — `.json` files inside the actor's own `.actor/` directory with only cosmetic schema changes (whitespace, key ordering) only trigger a release build.
|
|
63
63
|
8. **Functional** — everything else triggers both build and tests.
|
|
@@ -226,9 +226,9 @@ jobs:
|
|
|
226
226
|
secrets: inherit
|
|
227
227
|
```
|
|
228
228
|
|
|
229
|
-
This workflow runs `
|
|
230
|
-
[Releasing Actors](#releasing-actors
|
|
231
|
-
pass if you call the CLI from somewhere else.
|
|
229
|
+
This workflow runs one step per command: `get-affected-actors --since ${{ github.event.before }}`, then
|
|
230
|
+
`build --version-number default` on every changed Actor, then `notify-release`. See [Releasing Actors](#releasing-actors) for how the
|
|
231
|
+
released range is determined and what to pass if you call the CLI from somewhere else.
|
|
232
232
|
|
|
233
233
|
### `claude-review.yaml`
|
|
234
234
|
|
|
@@ -261,9 +261,9 @@ environment variables, so a secret is only visible to the step that needs it:
|
|
|
261
261
|
| Secret | Reaches |
|
|
262
262
|
| ------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
263
263
|
| `NPM_TOKEN` | dependency install steps only, as both `NPM_TOKEN` and `NODE_AUTH_TOKEN`. Use a read-only token: npm granular tokens can be read-only, classic automation tokens can publish. |
|
|
264
|
-
| the Actor tokens named by `tokenEnvVar` in `apify-test-tools.config.json` | the
|
|
264
|
+
| the Actor tokens named by `tokenEnvVar` in `apify-test-tools.config.json` | the build and `delete-old-builds` steps only |
|
|
265
265
|
| `TESTER_APIFY_TOKEN` | the vitest step, and the Claude investigation step: the Apify MCP server's bearer token for reading the failing run, its log and its storages, and Claude's token for re-running it to measure how often the failure happens |
|
|
266
|
-
| `SLACK_TOKEN_TESTS_BOT` / `SLACK_TOKEN_RELEASES_BOT` | the reporting and
|
|
266
|
+
| `SLACK_TOKEN_TESTS_BOT` / `SLACK_TOKEN_RELEASES_BOT` | the reporting and announcement steps only |
|
|
267
267
|
|
|
268
268
|
The Actor tokens are the one set that cannot be listed in the workflow, because each Actor names its
|
|
269
269
|
own token via `tokenEnvVar` in `apify-test-tools.config.json`. Those steps pass
|
|
@@ -273,11 +273,13 @@ secrets to pass on:
|
|
|
273
273
|
|
|
274
274
|
```yaml
|
|
275
275
|
- name: Build
|
|
276
|
+
shell: bash
|
|
276
277
|
env:
|
|
277
278
|
ALL_SECRETS: ${{ toJSON(secrets) }}
|
|
278
279
|
run: |
|
|
279
|
-
|
|
280
|
-
|
|
280
|
+
jq '{ actors: .functionallyChangedActors }' affected.json \
|
|
281
|
+
| node "${{ steps.setup.outputs.scripts-path }}/run-with-apify-tokens.mjs" \
|
|
282
|
+
npx apify-test-tools build --version-number 0.99
|
|
281
283
|
```
|
|
282
284
|
|
|
283
285
|
The wrapper passes only the tokens the config declares and drops `ALL_SECRETS`, so neither `npx` nor
|
|
@@ -568,7 +570,7 @@ The package includes a CLI binary used by CI workflows to build Actors, detect c
|
|
|
568
570
|
|
|
569
571
|
### Dry runs (`--dry-run`)
|
|
570
572
|
|
|
571
|
-
Every command that changes something (`build`, `build-from-local`, `release`, `delete-old-builds`, `report-tests`) takes `--dry-run`. A dry run:
|
|
573
|
+
Every command that changes something (`build`, `build-from-local`, `notify-release`, `delete-old-builds`, `report-tests`) takes `--dry-run`. A dry run:
|
|
572
574
|
|
|
573
575
|
- **Changes nothing.** No versions or builds are created, nothing is uploaded or deleted, and no Slack message is posted.
|
|
574
576
|
- **Does every read and check the real run does**, so it fails wherever the real run would: change detection, the Apify token and Actor lookup, the [repository check](#repository-check), collecting and zipping local sources, and the Slack token (verified with Slack's read-only `auth.test`).
|
|
@@ -598,14 +600,15 @@ If you want to test vs existing src code, you can skip this and instead construc
|
|
|
598
600
|
Requires `APIFY_TOKEN_<USERNAME>` for all Apify users that own your Actors (e.g. `apify`, `compass`, `lukaskrivka` users). The username is derived from the actor name — uppercased with non-word chars replaced by `_` (e.g. Actor `john.doe/my-actor` needs `APIFY_TOKEN_JOHN_DOE`).
|
|
599
601
|
|
|
600
602
|
```bash
|
|
601
|
-
|
|
602
|
-
|
|
603
|
-
|
|
604
|
-
--source-branch origin/my-dummy-branch \
|
|
605
|
-
--dry-run
|
|
603
|
+
npx apify-test-tools get-affected-actors --base origin/master \
|
|
604
|
+
| jq '{ actors: .functionallyChangedActors }' \
|
|
605
|
+
| APIFY_TOKEN_JOHN_DOE=<token> npx apify-test-tools build --version-number 0.99 --dry-run
|
|
606
606
|
```
|
|
607
607
|
|
|
608
|
-
|
|
608
|
+
Run it with your dummy branch checked out: it builds the Actors that branch changed functionally, from the branch
|
|
609
|
+
as it is on the remote, into the test version `0.99`. With `--dry-run`, it prints the plan (the version and Git URL
|
|
610
|
+
each Actor would be built from). Remove `--dry-run` to actually trigger builds, and update `--base` to the branch
|
|
611
|
+
you compare against. `build` outputs a JSON array of build objects to stdout:
|
|
609
612
|
|
|
610
613
|
```json
|
|
611
614
|
[{ "buildId": "...", "actorRawId": "...", "buildNumber": "...", "actorFullName": "john.doe/my-actor" }]
|
|
@@ -644,15 +647,13 @@ TESTER_APIFY_TOKEN=<token> \
|
|
|
644
647
|
|
|
645
648
|
#### Full example
|
|
646
649
|
|
|
647
|
-
Tests only run for Actors in `ACTOR_BUILDS`, so a custom workflow has to build (and pass) the Actors even when only tests changed. Pass `--test-files-glob` so test changes count even when `.dockerignore` lists them.
|
|
650
|
+
Tests only run for Actors in `ACTOR_BUILDS`, so a custom workflow has to build (and pass) the Actors even when only tests changed. Pass `--test-files-glob` to `get-affected-actors` so test changes count even when `.dockerignore` lists them.
|
|
648
651
|
|
|
649
652
|
```bash
|
|
650
653
|
# Build and capture output
|
|
651
|
-
BUILDS=$(
|
|
652
|
-
|
|
653
|
-
--
|
|
654
|
-
--source-branch origin/my-dummy-branch \
|
|
655
|
-
--test-files-glob 'test/platform/**')
|
|
654
|
+
BUILDS=$(npx apify-test-tools get-affected-actors --base origin/master --test-files-glob 'test/platform/**' \
|
|
655
|
+
| jq '{ actors: .functionallyChangedActors }' \
|
|
656
|
+
| APIFY_TOKEN_JOHN_DOE=apify_api_xxx npx apify-test-tools build --version-number 0.99)
|
|
656
657
|
|
|
657
658
|
# Run tests with the builds
|
|
658
659
|
ACTOR_BUILDS="$BUILDS" \
|
|
@@ -667,7 +668,7 @@ TESTER_APIFY_TOKEN=apify_api_yyy \
|
|
|
667
668
|
The range always ends at the checked-out `HEAD`, so check out the branch you want to inspect first.
|
|
668
669
|
|
|
669
670
|
```bash
|
|
670
|
-
# A PR: the branch's own
|
|
671
|
+
# A PR: the branch's own changes, narrowed to the Actors that changed since its last validated commit
|
|
671
672
|
npx apify-test-tools get-affected-actors --base origin/master --since <sha>
|
|
672
673
|
|
|
673
674
|
# A release: everything after the last released commit
|
|
@@ -676,29 +677,36 @@ npx apify-test-tools get-affected-actors --since <sha>
|
|
|
676
677
|
|
|
677
678
|
| Flag | Default | Meaning |
|
|
678
679
|
| ------------------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
679
|
-
| `--base` | | Only
|
|
680
|
-
| `--since` | | Only
|
|
680
|
+
| `--base` | | Only what `HEAD` changes compared to this branch counts, like `git diff <base>...HEAD` |
|
|
681
|
+
| `--since` | | Only changes after this commit count (see below) |
|
|
681
682
|
| `--test-files-glob` | | Test files (glob from the repo root) that count as changes even when `.dockerignore` lists them, see [Change detection](#change-detection) |
|
|
682
683
|
| `--actors` | all | Only report these Actors |
|
|
683
684
|
| `--ignore` | none | Never report these Actors |
|
|
684
685
|
|
|
685
686
|
Pass `--base`, `--since`, or both:
|
|
686
687
|
|
|
687
|
-
- **`--base` only** inspects the
|
|
688
|
-
|
|
689
|
-
|
|
690
|
-
- **`--base` and `--since`** inspects
|
|
691
|
-
|
|
688
|
+
- **`--base` only** inspects what the branch changes, the same files GitHub's "Files changed" tab shows: the diff
|
|
689
|
+
from where the branch forked from `--base`, or the last commit of `--base` it merged in. What it merged in from
|
|
690
|
+
`--base` is already validated there, so it doesn't count.
|
|
691
|
+
- **`--base` and `--since`** also inspects what changed after the branch's last validated commit, and reports only
|
|
692
|
+
the Actors that changed in both. An Actor in only one of them is already validated, by this branch or on `--base`.
|
|
693
|
+
When both changed an Actor, the changes can interact, so it is reported, but as functional only when both changes
|
|
694
|
+
are functional. A validated functional change followed by README edits is cosmetic, and so is a functional
|
|
695
|
+
change merged in from `--base` that meets only cosmetic changes of the branch. `--since` doesn't have to be on
|
|
696
|
+
the branch, so a rebase only re-checks the Actors that `--base` changed too. If `--since` is not in the local
|
|
697
|
+
history (a force-push), or it is `HEAD` (a rerun), it is ignored and the whole branch is checked.
|
|
692
698
|
- **`--since` only** inspects everything after it up to `HEAD`. It must be an ancestor of `HEAD` and fails
|
|
693
|
-
otherwise,
|
|
699
|
+
otherwise, see [Releasing Actors](#releasing-actors). If `HEAD` is `--since`, nothing is
|
|
694
700
|
reported.
|
|
695
701
|
|
|
702
|
+
`commits`, `changedFiles` and `changelog` describe the whole range, `--base...HEAD` or `--since..HEAD`: `--since`
|
|
703
|
+
together with `--base` only narrows the Actor lists.
|
|
704
|
+
|
|
696
705
|
The output is a JSON document:
|
|
697
706
|
|
|
698
707
|
```json
|
|
699
708
|
{
|
|
700
709
|
"version": 1,
|
|
701
|
-
"ref": "my-branch",
|
|
702
710
|
"head": "<sha of HEAD>",
|
|
703
711
|
"commits": [{ "sha": "…", "author": "…", "date": "…", "message": "…" }],
|
|
704
712
|
"changedFiles": ["src/main.ts", "CHANGELOG.md"],
|
|
@@ -710,30 +718,71 @@ The output is a JSON document:
|
|
|
710
718
|
|
|
711
719
|
Each changed Actor is in exactly one of the lists. `functionallyChangedActors` have at least one functional change, so
|
|
712
720
|
they need a new build and tests. `cosmeticallyChangedActors` only have cosmetic changes (see
|
|
713
|
-
[Change detection](#change-detection)), so they need a new build only when released:
|
|
714
|
-
`functionallyChangedActors`,
|
|
721
|
+
[Change detection](#change-detection)), so they need a new build only when released: the PR workflow builds
|
|
722
|
+
`functionallyChangedActors`, the release workflow builds both.
|
|
723
|
+
|
|
724
|
+
### Building Actors (`build`)
|
|
715
725
|
|
|
716
|
-
`
|
|
726
|
+
`build` builds exactly the Actors it is given on stdin, from the checked-out branch as it is on the remote, and
|
|
727
|
+
prints the builds as the same JSON array `build-from-local` prints. A detached HEAD is an
|
|
728
|
+
error, since there is no branch to build from. It doesn't look at git history: which Actors to build is
|
|
729
|
+
up to you, usually picked from the output of [`get-affected-actors`](#finding-affected-actors-get-affected-actors).
|
|
717
730
|
|
|
718
|
-
|
|
731
|
+
```bash
|
|
732
|
+
# A PR: the Actors a branch changed functionally, built into the test version
|
|
733
|
+
npx apify-test-tools get-affected-actors --base origin/master \
|
|
734
|
+
| jq '{ actors: .functionallyChangedActors }' \
|
|
735
|
+
| npx apify-test-tools build --version-number 0.99
|
|
736
|
+
|
|
737
|
+
# A release: every Actor changed since the last release, into the version users run
|
|
738
|
+
npx apify-test-tools get-affected-actors --since <sha> \
|
|
739
|
+
| jq '{ actors: (.functionallyChangedActors + .cosmeticallyChangedActors) }' \
|
|
740
|
+
| npx apify-test-tools build --version-number default
|
|
741
|
+
|
|
742
|
+
# One Actor, whatever changed
|
|
743
|
+
echo '{ "actors": [{ "actorFullName": "john.doe/my-actor" }] }' | npx apify-test-tools build --version-number 0.99
|
|
744
|
+
```
|
|
719
745
|
|
|
720
|
-
`
|
|
721
|
-
|
|
722
|
-
`
|
|
723
|
-
|
|
746
|
+
The input is `{ "actors": [{ "actorFullName": "<owner/name>" }] }`; other fields are ignored. An empty `actors` list builds nothing and prints `[]`. Empty or invalid input is an error,
|
|
747
|
+
so a failed command earlier in the pipe can't turn into a successful run that built nothing. In GitHub
|
|
748
|
+
Actions, give such a step `shell: bash` so the pipe also fails on its own (the default shell has no `pipefail`).
|
|
749
|
+
|
|
750
|
+
| Flag | Default | Meaning |
|
|
751
|
+
| ------------------ | ------------------- | ---------------------------------------------------------------------------------------------------------------------- |
|
|
752
|
+
| `--version-number` | required | Version to build, like `0.99`, or `default` for each Actor's default version (the one its default build tag points to) |
|
|
753
|
+
| `--repo-url` | the `origin` remote | Repository to build from, see [Repository check](#repository-check) |
|
|
754
|
+
|
|
755
|
+
`build` never sets or changes build tags. A build gets the tag its version is configured with on the platform,
|
|
756
|
+
so building the default version moves its tag (usually `latest`) to the new build, and building a version
|
|
757
|
+
without a tag leaves every tag where it is. Builds always use the Docker cache.
|
|
758
|
+
|
|
759
|
+
### Releasing Actors
|
|
760
|
+
|
|
761
|
+
A release is three commands: find what changed since the last released commit, build those Actors into
|
|
762
|
+
their default version (the one users run), and announce it.
|
|
724
763
|
|
|
725
764
|
```bash
|
|
726
|
-
npx apify-test-tools
|
|
765
|
+
npx apify-test-tools get-affected-actors --since <sha> > affected.json
|
|
766
|
+
|
|
767
|
+
jq '{ actors: (.functionallyChangedActors + .cosmeticallyChangedActors) }' affected.json \
|
|
768
|
+
| npx apify-test-tools build --version-number default --dry-run
|
|
769
|
+
|
|
770
|
+
npx apify-test-tools notify-release --release-slack-channel "#releases" --dry-run < affected.json
|
|
727
771
|
```
|
|
728
772
|
|
|
729
|
-
|
|
773
|
+
`notify-release` reads the output of `get-affected-actors` on stdin. It posts the changelog additions to
|
|
774
|
+
`--release-slack-channel` and the commits and changed files to `--report-slack-channel`, and posts nothing when
|
|
775
|
+
there are no new commits. It names the release after the `origin` remote (or `--repo-url`) and the author of the
|
|
776
|
+
newest commit, and needs `SLACK_TOKEN_RELEASES_BOT`.
|
|
777
|
+
|
|
778
|
+
#### The last released commit (`--since`)
|
|
730
779
|
|
|
731
|
-
The last commit that is **already released**. Everything after it, up to `HEAD`, is released.
|
|
732
|
-
|
|
733
|
-
|
|
780
|
+
The last commit that is **already released**. Everything after it, up to `HEAD`, is released. Git alone
|
|
781
|
+
can't tell which commits a push brought, and guessing wrong silently leaves Actors on old code, so it has
|
|
782
|
+
to be passed.
|
|
734
783
|
|
|
735
|
-
The changed files come from diffing
|
|
736
|
-
|
|
784
|
+
The changed files come from diffing `--since` and `HEAD` directly, so this works for every merge strategy as
|
|
785
|
+
long as `--since` is the branch tip from **before** the push:
|
|
737
786
|
|
|
738
787
|
| Merge strategy | What lands on the branch | What to pass |
|
|
739
788
|
| ------------------------------ | --------------------------------- | ---------------------------------------------------------------------- |
|
|
@@ -743,26 +792,25 @@ merge strategy as long as the base commit is the branch tip from **before** the
|
|
|
743
792
|
|
|
744
793
|
On GitHub, the push's "before" commit is `${{ github.event.before }}`, which the reusable workflow
|
|
745
794
|
passes for you. On other CIs use the equivalent, e.g. `$CI_COMMIT_BEFORE_SHA` on GitLab. The checkout
|
|
746
|
-
needs the full history (`fetch-depth: 0` in `actions/checkout`) so the
|
|
795
|
+
needs the full history (`fetch-depth: 0` in `actions/checkout`) so the commit is available locally.
|
|
747
796
|
|
|
748
|
-
`
|
|
797
|
+
`get-affected-actors --since` without `--base` fails instead of guessing when:
|
|
749
798
|
|
|
750
|
-
- the
|
|
751
|
-
- the
|
|
799
|
+
- the commit is all zeros (GitHub sends this for a push that created the branch)
|
|
800
|
+
- the commit isn't in the local history (shallow checkout) or isn't an ancestor of `HEAD`
|
|
752
801
|
(the branch was force-pushed)
|
|
753
802
|
|
|
754
|
-
In those cases, rerun
|
|
755
|
-
|
|
756
|
-
|
|
757
|
-
|
|
758
|
-
If `HEAD` is the base commit, there is nothing to release and the command exits successfully.
|
|
803
|
+
In those cases, rerun it with a `--since` you pick yourself: an ancestor of `HEAD` from before the changes
|
|
804
|
+
you want to release (for a shallow checkout, fetch the full history instead). `--actors` only narrows which
|
|
805
|
+
of the changed Actors are reported; it never forces a build, so on its own it doesn't get past these errors.
|
|
806
|
+
If `HEAD` is `--since`, nothing is reported, so nothing is built or announced.
|
|
759
807
|
|
|
760
808
|
A failed release isn't retried by the next push: that push's base commit is the failed one's head, so
|
|
761
809
|
its changes are skipped. Re-run the failed workflow run, or release the Actors manually.
|
|
762
810
|
|
|
763
811
|
#### Repository check
|
|
764
812
|
|
|
765
|
-
`
|
|
813
|
+
`build` points each Actor's version at the `origin` remote. Because a build overwrites the
|
|
766
814
|
version's Git URL, building from a fork or a mirror would repoint the Actor. So, before building, the
|
|
767
815
|
remote is compared with the Git URL of the Actor's default version (the one its default build tag
|
|
768
816
|
points to), and the command fails if they differ or if the default version isn't built from a Git
|