@andrian.yablonskyy/thub-agent 1.0.23 → 1.0.25

package/README.md

@@ -9,7 +9,7 @@ See the [main TestHub repo](https://github.com/andrianyablonskyy/thub) for the f
 ```bash
 npm i -g @andrian.yablonskyy/thub-agent
 # or, one-off in CI:
-npx -y @andrian.yablonskyy/thub-agent run --type sw --image "$IMAGE_URL" --tests "$TESTS_URL" --wait
+npx -y @andrian.yablonskyy/thub-agent run --type sw --download-file "$IMAGE_URL" --git-repo "$TESTS_REPO" --command ./ci/test.sh --wait
 ```
 
 ## Configuration
@@ -50,10 +50,14 @@ Key options for `thub run`:
 | `--group <groupId>` | Restrict scheduling to resources that are members of this group. Falls back to `THUB_GROUP` / `thub config set group <id>`. |
 | `--client <name\|id>` | Run on this specific Client (resource name or id) only; the job waits in that Client's queue even if other matching resources are idle. |
 | `--user <name>` | Free-text job owner — a label, not an identity. Falls back to `THUB_USER` / `thub config set user <name>`. |
-| `--image <url>` | Firmware/build image URL, fetched by the Client. |
-| `--sha256 <hex>` | Expected sha256 of `--image`; the Client verifies it before flashing/running. |
-| `--tests <url>` / `--suite <name>` | Test package and suite. |
-| `--arg <value>` | Extra argument passed through to `run-tests.sh` on the Client (repeatable). |
+| `--command <string>` | **Required.** The task's entry point: a shell command the Client runs (`sh -c`) in the task's work directory — the `--git-repo` checkout, else an empty directory — after preparing its inputs. Its exit code is the verdict. On HW it flashes the board itself (the Client doesn't); it gets `THUB_DUT_STLINK`/`_UART`/`_USB`/`_HOST`/`_CONTAINER`, `THUB_DOWNLOAD_<n>`, `THUB_DOWNLOADS_DIR`, `THUB_GIT_COMMIT`, `THUB_SUITE`, `THUB_META_*`. |
+| `--download-file <url>` | A file the Client downloads before running the command (repeatable, `http(s)`), into the job's `downloads/` directory. |
+| `--docker-image <name>` | SW only: a Docker image the Client runs as the DUT instead of its own `sw.image` (the Client must allow it: `sw.allowJobImages`). |
+| `--git-repo <url> [<branch>\|<tag>\|<commit>]` | A git repository the Client clones (default ref: the default branch); the command runs in the checkout. |
+| `--depth <n>` | With `--git-repo`: commits to fetch, default `1`; `0` = full history. |
+| `--git-options <string>` | With `--git-repo`: extra git options placed between `git` and its subcommand on the Client, e.g. `'-c core.sshCommand="ssh -i ~/.ssh/lab_key -p 2222"'`. Shell-quoted (no shell run). Stored with the job, so reference key files rather than inlining secrets. |
+| `--suite <name>` | Passed to the command as `THUB_SUITE`. |
+| `--arg <value>` | Extra argument for the command, as `"$@"` (repeatable). |
 | `--timeout <dur>` | e.g. `30m`, default `30m`. |
 | `--priority <n>` | 0–100; CI defaults to 50, CLI to 60 so a developer is not starved by a busy pipeline. |
 | `--meta <key=value>` | Arbitrary metadata stored on the job (repeatable) — CI job ids, git coordinates, anything else worth attaching to the run. |
@@ -78,34 +82,45 @@ Exit codes make the Agent usable as a CI step:
 
 ## Examples
 
-**Associate a CI/CD job id with the internal job id:**
+**Run a task** — download the firmware, check out the tests at a tag, flash and test (HW):
 
 ```bash
-thub run --type sw --image "$IMAGE_URL" --tests "$TESTS_URL" \
-  --meta ciJobId="$GITHUB_RUN_ID" --wait
-thub status A-00123 --json | jq '.spec.meta.ciJobId'
+thub run --type hw --board nucleo-f401re \
+  --download-file "$IMAGE_URL" \
+  --git-repo https://github.com/yourorg/firmware-tests.git v1.4.0 \
+  --command 'st-flash --serial "$THUB_DUT_STLINK" --reset write "$THUB_DOWNLOAD_1" 0x08000000 && ./ci/test.sh "$@"' \
+  --arg --junit --wait
 ```
 
-**Pass Git repo/branch/hash/tag to the Client** — these travel as `--meta key=value` and the Client exposes each one to `run-tests.sh` as `THUB_META_<KEY>` (`ciJobId` → `THUB_META_CI_JOB_ID`):
+**SW task in a Docker image of your own:**
 
 ```bash
-thub run --type hw --board nucleo-f401re \
-  --image "$IMAGE_URL" --tests "$TESTS_URL" --suite smoke \
-  --meta repo=yourorg/firmware --meta branch=main --meta sha=a1b2c3d --wait
+thub run --type sw --docker-image alpine:3.20 \
+  --git-repo git@github.com:yourorg/firmware-tests.git main --depth 20 \
+  --command 'make test' --wait
 ```
 
-**Dry-run the pipeline** — proves the Coordinator↔Client plumbing works without real hardware, a real emulator image, or a reachable Artifactory:
+**Associate a CI/CD job id with the internal job id:**
+
+```bash
+thub run --type sw --download-file "$IMAGE_URL" --git-repo "$TESTS_REPO" --command ./ci/test.sh \
+  --meta ciJobId="$GITHUB_RUN_ID" --wait
+thub status A-00123 --json | jq '.spec.meta.ciJobId'
+```
+
+**Pass extra metadata** — `--meta key=value` reaches the command as `THUB_META_<KEY>` (`ciJobId` → `THUB_META_CI_JOB_ID`).
+
+**Dry-run the pipeline** — proves the Coordinator↔Client plumbing works without real hardware, a real emulator image, or reachable downloads:
 
 ```bash
-thub run --type sw --image https://does-not-exist.invalid/app.bin \
-  --tests https://does-not-exist.invalid/tests.tar.gz --suite smoke \
-  --dry-run --wait
+thub run --type sw --download-file https://does-not-exist.invalid/app.bin \
+  --command ./ci/test.sh --dry-run --wait
 ```
 
 **Run on a specific resource group only:**
 
 ```bash
-thub run --type sw --image "$IMAGE_URL" --tests "$TESTS_URL" \
+thub run --type sw --git-repo "$TESTS_REPO" --command ./ci/test.sh \
   --group 548ae4ae-ac5b-401f-acaa-24bbe790e62d --wait
 ```
 
@@ -113,7 +128,7 @@ thub run --type sw --image "$IMAGE_URL" --tests "$TESTS_URL" \
 
 ```bash
 thub run --type hw --board nucleo-f401re \
-  --image "$IMAGE_URL" --tests "$TESTS_URL" --user "Your Name" --wait
+  --git-repo "$TESTS_REPO" --command ./ci/test.sh --user "Your Name" --wait
 ```
 
 ## GitHub Actions
@@ -128,8 +143,9 @@ test-sw:
   steps:
     - run: |
         npx -y @andrian.yablonskyy/thub-agent run --type sw \
-          --image "${{ needs.build.outputs.image_url }}" \
-          --tests "${{ needs.build.outputs.tests_url }}" --suite full --wait
+          --download-file "${{ needs.build.outputs.image_url }}" \
+          --git-repo "$GITHUB_SERVER_URL/$GITHUB_REPOSITORY.git" "$GITHUB_SHA" \
+          --command ./ci/sw-tests.sh --suite full --wait
 ```
 
 If the GitHub job is canceled, the runner sends `SIGINT` to the Agent. In `--wait` mode (CI), that's treated as a cancel request (`POST /jobs/:id/cancel`) before exiting, so abandoned CI jobs don't hold hardware. In interactive mode, Ctrl-C only detaches.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@andrian.yablonskyy/thub-agent",
-  "version": "1.0.23",
+  "version": "1.0.25",
   "description": "TestHub Agent CLI — the single entry point CI/CD and developers use to submit and follow test jobs",
   "bin": {
     "thub": "./src/cli.js"

package/src/cli.js

@@ -17,7 +17,8 @@
 
 const { Command } = require('commander'),
   {
-    ApiClient, EXIT_CODES, ACTIVE_JOB_STATES, exitCodeForJobState, PACKAGES, fetchLatestVersion, isNewer, isValidVersion, formatDateTime
+    ApiClient, EXIT_CODES, ACTIVE_JOB_STATES, exitCodeForJobState, PACKAGES, fetchLatestVersion, isNewer, isValidVersion, formatDateTime,
+    splitArgs
   } = require('@andrian.yablonskyy/thub-common'),
   { resolveConnection, resolveGroup, resolveUser, writeConfigFile, readConfigFile } = require('./config'),
   { parseDurationSec } = require('./duration'),
@@ -54,55 +55,51 @@ function client(){
   return new ApiClient({ baseUrl: url, token, userAgent: `thub-agent/${version}` });
 }
 
-// `--image`: an http(s) URL is a firmware file to download; anything else
-// is taken as a Docker image reference, which only an SW job can run. Caught
-// here so `--type hw --image alpine` explains itself instead of failing
-// the Coordinator's schema check.
-function firmwareFromImage(opts){
-  if (/^https?:\/\//i.test(opts.image)){
-    return { url: opts.image, ...(opts.sha256 ? { sha256: opts.sha256 } : {}) };
-  }
-  if (/^[a-z][a-z0-9+.-]*:\/\//i.test(opts.image)){
-    throw usageError(`--image ${opts.image}: only http(s) URLs can be downloaded as firmware`);
-  }
-  if (opts.type !== 'sw'){
-    throw usageError(
-      `--image ${opts.image} looks like a Docker image, which only an SW job can run (--type sw). ` +
-        'An HW job flashes a firmware file: pass its http(s) URL.'
-    );
-  }
-  if (opts.sha256){
-    throw usageError('--sha256 applies to a firmware URL only; pin a Docker image by digest instead (image@sha256:...)');
+// --command (mandatory), --download-file (repeatable), --docker-image (SW
+// only) and --git-repo <url> [ref] --depth <n> -> the job spec's task fields.
+// Checked here too, so mistakes explain themselves before anything is sent.
+function taskFromOptions(opts){
+  const downloads = opts.downloadFile.map((url) => {
+      if (!/^https?:\/\//i.test(url)){
+        throw usageError(`--download-file ${url}: must be an http(s) URL`);
+      }
+      return { url };
+    }),
+    task = { command: opts.command, args: opts.arg, suite: opts.suite, ...(downloads.length ? { downloads } : {}) };
+
+  if (opts.dockerImage){
+    if (opts.type !== 'sw'){
+      throw usageError('--docker-image only works for SW jobs (--type sw): an HW job runs on the physical board');
+    }
+    task.image = opts.dockerImage;
   }
-  return { image: opts.image };
-}
 
-// --tests (archive URL) or --tests-git (+ at most one of branch/tag/commit).
-function testsFromOptions(opts){
-  const refs = ['branch', 'tag', 'commit'].filter((k) => opts[`tests${k[0].toUpperCase()}${k.slice(1)}`]);
-  if (opts.tests && opts.testsGit){
-    throw usageError('give either --tests (an archive URL) or --tests-git (a repository), not both');
+  if (opts.depth !== undefined && !opts.gitRepo){
+    throw usageError('--depth only applies to --git-repo');
   }
-  if (!opts.tests && !opts.testsGit){
-    throw usageError('missing test sources: --tests <archive-url> or --tests-git <repo> [--tests-branch|--tests-tag|--tests-commit]');
+  if (opts.gitOptions !== undefined && !opts.gitRepo){
+    throw usageError('--git-options only applies to --git-repo');
   }
-  if (opts.tests){
-    if (refs.length){
-      throw usageError(`--tests-${refs[0]} only applies to --tests-git`);
+  if (opts.gitOptions){
+    try {
+      splitArgs(opts.gitOptions);
+    }
+    catch (err){
+      throw usageError(`--git-options: ${err.message}`);
     }
-    return { url: opts.tests };
-  }
-  if (refs.length > 1){
-    throw usageError(`give at most one of --tests-branch, --tests-tag, --tests-commit (got ${refs.map((r) => `--tests-${r}`).join(', ')})`);
   }
-  return {
-    git: {
-      url: opts.testsGit,
-      ...(opts.testsBranch ? { branch: opts.testsBranch } : {}),
-      ...(opts.testsTag ? { tag: opts.testsTag } : {}),
-      ...(opts.testsCommit ? { commit: opts.testsCommit } : {})
+  if (opts.gitRepo){
+    const [url, ref, ...extra] = opts.gitRepo;
+    if (extra.length){
+      throw usageError(`--git-repo takes a URL and at most one branch, tag or commit (got: ${opts.gitRepo.join(' ')})`);
+    }
+    const depth = opts.depth === undefined ? 1 : Number(opts.depth);
+    if (!Number.isInteger(depth) || depth < 0){
+      throw usageError(`--depth ${opts.depth}: must be a whole number (0 = full history)`);
     }
-  };
+    task.git = { url, ...(ref ? { ref } : {}), depth, ...(opts.gitOptions ? { options: opts.gitOptions } : {}) };
+  }
+  return task;
 }
 
 function usageError(message){
@@ -136,28 +133,35 @@ program
       '— purely a label, not an identity. Overrides THUB_USER / config file.'
   )
   .requiredOption(
-    '--image <url|docker-image>',
-    'What the DUT runs: a firmware file URL (http/https, e.g. Artifactory) the Client downloads — HW and SW jobs — ' +
-      'or, for --type sw only, a Docker image to run as the DUT (e.g. alpine, alpine:3.20, registry.lab:5000/emu:1), ' +
-      'pulled from the Client\'s registry or Docker Hub; the Client must allow it (sw.allowJobImages)'
+    '--command <string>',
+    'The task\'s entry point: a shell command the Client runs (sh -c) in the task\'s work directory — the --git-repo ' +
+      'checkout if given — after downloading --download-file files. --arg values arrive as "$@"'
+  )
+  .option(
+    '--download-file <url>',
+    'A file the Client downloads into the work directory before running --command (repeatable; http/https). ' +
+      'Paths are passed as THUB_DOWNLOAD_1.. / THUB_DOWNLOADS_DIR',
+    collectRepeatable,
+    []
+  )
+  .option(
+    '--docker-image <name>',
+    'SW jobs only: a Docker image the Client runs as the DUT (e.g. alpine, alpine:3.20, registry.lab:5000/emu:1), ' +
+      'instead of its own sw.image; pulled from its registry or Docker Hub. The Client must allow it (sw.allowJobImages)'
   )
-  .option('--sha256 <hex>', 'Expected sha256 of a firmware --image URL; the Client verifies it before flashing/running')
   .option(
-    '--tests <url>',
-    'Test sources as an archive URL (tar, tar.gz/.tgz/.bz2/.xz, or zip) the Client downloads and unpacks. ' +
-      'Give this or --tests-git'
+    '--git-repo <url...>',
+    'A git repository the Client clones before running --command, then runs it there: <url> [<branch>|<tag>|<commit>] ' +
+      '(https://, ssh://, git:// or user@host:path; default ref: the default branch)'
   )
-  .option('--tests-git <repo>', 'Test sources as a git repository (https://, ssh://, git:// or user@host:path) the Client fetches')
-  .option('--tests-branch <name>', 'With --tests-git: the branch to check out (default: the repository\'s default branch)')
-  .option('--tests-tag <name>', 'With --tests-git: the tag to check out')
-  .option('--tests-commit <sha>', 'With --tests-git: the commit to check out (7-40 hex digits)')
+  .option('--depth <n>', 'With --git-repo: how many commits to fetch (default 1; 0 = full history)')
   .option(
-    '--run <command>',
-    'Shell command that starts the tests, run in the test sources on the Client (--arg values arrive as "$@"). ' +
-      'Default: the sources\' own run-tests.sh. The Client must allow it (allowJobCommands)'
+    '--git-options <string>',
+    'With --git-repo: extra git options, placed between `git` and its subcommand on the Client (shell-quoted, no shell run), ' +
+      'e.g. \'-c core.sshCommand="ssh -i ~/.ssh/lab_key -p 2222"\'. Stored with the job — reference key files, don\'t inline secrets'
   )
-  .option('--suite <name>', 'Test suite name', 'default')
-  .option('--arg <value>', 'Extra argument passed through to run-tests.sh / --run on the Client (repeatable)', collectRepeatable, [])
+  .option('--suite <name>', 'Test suite name, passed to --command as THUB_SUITE', 'default')
+  .option('--arg <value>', 'Extra argument for --command, as "$@" (repeatable)', collectRepeatable, [])
   .option('--timeout <duration>', 'e.g. 30m, 1h', '30m')
   .option('--priority <n>', 'Priority 0-100', (v) => Number(v))
   .option('--wait', 'Do not detach on job end; exit with the verdict code (used in CI)', false)
@@ -178,8 +182,7 @@ program
   )
   .action(async (opts) => {
     try {
-      const firmware = firmwareFromImage(opts),
-        testSources = testsFromOptions(opts),
+      const task = taskFromOptions(opts),
         c = client(),
         labels = [...(opts.board ? [`board:${opts.board}`] : []), ...opts.label],
         meta = Object.fromEntries(opts.meta.map((kv) => kv.split(/=(.*)/s).slice(0, 2))),
@@ -188,8 +191,7 @@ program
 
         spec = {
           target: { type: opts.type, labels, ...(group ? { group } : {}), ...(opts.client ? { client: opts.client } : {}) },
-          firmware,
-          tests: { ...testSources, ...(opts.run ? { command: opts.run } : {}), suite: opts.suite, args: opts.arg },
+          ...task,
           timeoutSec: parseDurationSec(opts.timeout),
           ...(opts.priority !== undefined ? { priority: opts.priority } : {}),
           ...(user ? { user } : {}),