@andrian.yablonskyy/thub-coordinator 1.0.15 → 1.0.17

package/README.md

@@ -157,11 +157,13 @@ Server-rendered Pug templates styled with Bootstrap 5.3, with a little vanilla J
 | `/admin/agents` | Register/revoke CI and developer agents; token shown once. Shows each agent's last reported version. |
 | `/profile` | Every logged-in user's own account settings: display name, avatar, timezone (renders every dashboard timestamp), theme, idle session timeout, password change. |
 
+Notifications (results of dashboard actions, errors) appear as toasts in the top-right corner and close themselves by severity: **errors stay until closed**, warnings close after **30 s**, info and success after **10 s**. A one-time secret such as a rotated resource token never closes on its own.
+
 **Resource card.** Clicking a resource on `/` or `/resources` opens a card with its name, type, status, host uptime and current task/status with its duration (both from the Client's heartbeats, anchored to the Coordinator's clock; as of the last heartbeat for an OUT_OF_SERVICE resource), Client version, IP addresses, capabilities (reported at registration: HW udev devices with a **missing** flag for any whose device node doesn't exist, relay channels and power control; SW image, source and limits), labels, groups, busy source/reason, last heartbeat and (admins only) the maintenance/rotate-token actions. IP addresses are the Client host's interface addresses except loopback (reported at registration and on every heartbeat) plus the external address — the one the Coordinator last saw the Client connect from, taken from `X-Forwarded-For` when the request came through a proxy `trustProxy` trusts. Admins get a **Cancel** button on it, enabled only while the Client is doing something: it cancels its current job, releases a manual local lock, or aborts a self-update hold on its host (the last two delivered as `unlock` / `cancel-update` heartbeat commands).
 
 The Coordinator's own version (`ver. X.Y.Z`) is shown under the TestHub logo, top left of every page.
 
-**Versions and self-update.** The Coordinator checks the npm registry for the latest Coordinator, Agent and Client every `updates.checkIntervalMin` (default 15 minutes, `0` disables; the older `checkIntervalHours` is ignored; `updates.registry` for a private mirror) and on **Check for updates**. A newer Coordinator shows as an **Update app to X.Y.Z** button in the navbar for admins (a badge under the logo for viewers); Agents and Resources show each one's version with an **update available** or pending **→ vX.Y.Z** badge. Admins request self-updates per agent/resource (**Update**, which becomes **Cancel update**) or with **Update all agents** / **Update all clients**, always to the latest version:
+**Versions and self-update.** The Coordinator checks the npm registry for the latest Coordinator, Agent and Client every `updates.checkIntervalMin` (default 15 minutes, `0` disables; the older `checkIntervalHours` is ignored; `updates.registry` for a private mirror) and immediately with the navbar's **check for updates** button (admins; also on the Agents and Resources pages), which reports in a toast whether the Coordinator is current and which connected resources can update their Client. A newer Coordinator shows as an **Update app to X.Y.Z** button in the navbar for admins (a badge under the logo for viewers); Agents and Resources show each one's version with an **update available** or pending **→ vX.Y.Z** badge. Admins request self-updates per agent/resource (**Update**, which becomes **Cancel update**) or with **Update all agents** / **Update all clients**, always to the latest version:
 - Clients get a `self-update` command on their next heartbeat and install it through their root `thub-client-update` helper, once none of the host's instances is busy.
 - Agents install it at the start of their next run (`GET /api/v1/agents/me/update`), then re-run the command on the new version.
 - The Coordinator itself: the **Update app** button writes `<dataDir>/update-request.json`; the root `thub-coordinator-update.path` unit (installed by `sudo npm i -g`) runs `npm i -g` for it, and the postinstall restarts the service (the dashboard and API are briefly down). Logs: `journalctl -u thub-coordinator-update`. Without that unit, use `thub-admin self-update` on the host.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@andrian.yablonskyy/thub-coordinator",
-  "version": "1.0.15",
+  "version": "1.0.17",
   "description": "TestHub Coordinator — job queue, resource registry, scheduler, heartbeat monitor, log/artifact store and web dashboard",
   "main": "src/server.js",
   "engines": {

package/public/js/toasts.js

@@ -0,0 +1,33 @@
+/**
+ * @file        packages/coordinator/public/js/toasts.js
+ * @description Dashboard: shows flash messages as Bootstrap toasts that close themselves by severity
+ *
+ * @author      Andrian Yablonskyy
+ * @copyright   Copyright (c) 2026 Andrian Yablonskyy. All rights reserved.
+ *
+ * This file is part of TestHub and is proprietary and confidential.
+ * Unauthorized copying, modification, distribution, or use of this file,
+ * via any medium, is strictly prohibited without prior written permission
+ * from AdSystem.PRO.
+ */
+
+'use strict';
+
+(function (){
+  // Auto-close delay per severity; null = stays until closed. Errors never
+  // close on their own, and neither does a toast marked sticky (e.g. a
+  // one-time token the user still has to copy).
+  const DELAY_MS = { danger: null, warning: 30_000, info: 10_000, success: 10_000 };
+
+  document.querySelectorAll('[data-thub-toast]').forEach((el) => {
+    const type = el.dataset.thubToast in DELAY_MS ? el.dataset.thubToast : 'info',
+      delay = el.dataset.sticky ? null : DELAY_MS[type],
+      toast = new bootstrap.Toast(el, { autohide: delay !== null, delay: delay ?? 0 });
+    // "Self-destroying": gone from the page once hidden, not just invisible.
+    el.addEventListener('hidden.bs.toast', () => {
+      toast.dispose();
+      el.remove();
+    });
+    toast.show();
+  });
+})();

package/src/services/jobs.js

@@ -91,11 +91,22 @@ function createJobsService(db, { bus, events, registry, artifacts, config }){
     }
     spec.source = source;
 
-    if (!registry.everSatisfiable(spec.target.type, spec.target.labels, spec.target.group)){
+    // `target.client` (thub run --client) accepts a resource id or name;
+    // store the id so the job stays pinned to the same Client across renames.
+    if (spec.target.client){
+      const resource = registry.get(spec.target.client) || registry.getByName(spec.target.client);
+      if (!resource){
+        throw Object.assign(new Error(`Unknown client "${spec.target.client}"`), { status: 422 });
+      }
+      spec.target.client = resource.id;
+    }
+
+    if (!registry.everSatisfiable(spec.target.type, spec.target.labels, spec.target.group, spec.target.client)){
       throw Object.assign(
         new Error(
           `No registered resource can ever satisfy type=${spec.target.type} labels=${spec.target.labels.join(',')}` +
-            (spec.target.group ? ` group=${spec.target.group}` : '')
+            (spec.target.group ? ` group=${spec.target.group}` : '') +
+            (spec.target.client ? ` client=${spec.target.client}` : '')
         ),
         { status: 422 }
       );

package/src/services/registry.js

@@ -468,21 +468,23 @@ function createRegistryService(db, { bus, events }){
     );
   }
 
-  function findIdleCandidates(type, labels, groupId){
+  function matchesTarget(r, labels, groupId, resourceId){
+    return labels.every((l) => r.labels.includes(l)) &&
+      (!groupId || r.group_ids.includes(groupId)) &&
+      (!resourceId || r.id === resourceId);
+  }
+
+  function findIdleCandidates(type, labels, groupId, resourceId){
     const rows = db
       .prepare('SELECT * FROM resources WHERE status = ? AND type = ?')
       .all(RESOURCE_STATES.IDLE, type)
       .map(rowToResource);
-    return rows.filter(
-      (r) => labels.every((l) => r.labels.includes(l)) && (!groupId || r.group_ids.includes(groupId))
-    );
+    return rows.filter((r) => matchesTarget(r, labels, groupId, resourceId));
   }
 
-  function everSatisfiable(type, labels, groupId){
+  function everSatisfiable(type, labels, groupId, resourceId){
     const rows = db.prepare('SELECT * FROM resources WHERE type = ?').all(type).map(rowToResource);
-    return rows.some(
-      (r) => labels.every((l) => r.labels.includes(l)) && (!groupId || r.group_ids.includes(groupId))
-    );
+    return rows.some((r) => matchesTarget(r, labels, groupId, resourceId));
   }
 
   return {

package/src/services/scheduler.js

@@ -1,6 +1,6 @@
 /**
  * @file        packages/coordinator/src/services/scheduler.js
- * @description Scheduler: matches queued jobs to idle resources by type/labels/group (README §5.4)
+ * @description Scheduler: matches queued jobs to idle resources by type/labels/group/client (README §5.4)
  *
  * @author      Andrian Yablonskyy
  * @copyright   Copyright (c) 2026 Andrian Yablonskyy. All rights reserved.
@@ -34,21 +34,28 @@ function createScheduler(db, { bus, events, registry, config }){
             .map(rowToJob),
 
           assignments = [],
-          claimedResourceIds = new Set();
+          claimedResourceIds = new Set(),
+          // Resources that some queued job is pinned to (`target.client`) —
+          // an unpinned job only takes one of these when nothing else fits,
+          // so it doesn't jump the pinned job's per-client queue.
+          reservedResourceIds = new Set(queued.map((j) => j.spec.target.client).filter(Boolean));
 
         for (const job of queued){
-          const candidates = registry
-            .findIdleCandidates(job.spec.target.type, job.spec.target.labels || [], job.spec.target.group)
-            .filter((r) => !claimedResourceIds.has(r.id));
+          const { type, labels = [], group, client } = job.spec.target,
+            candidates = registry
+              .findIdleCandidates(type, labels, group, client)
+              .filter((r) => !claimedResourceIds.has(r.id));
           if (candidates.length === 0){
             continue;
           }
 
-          // Least-recently-used: spread wear across benches (§5.4).
+          // Unreserved first, then least-recently-used: spread wear across benches (§5.4).
           candidates.sort((a, b) => {
-            const ta = a.last_job_finished_at ? new Date(a.last_job_finished_at).getTime() : 0,
+            const ra = reservedResourceIds.has(a.id) ? 1 : 0,
+              rb = reservedResourceIds.has(b.id) ? 1 : 0,
+              ta = a.last_job_finished_at ? new Date(a.last_job_finished_at).getTime() : 0,
               tb = b.last_job_finished_at ? new Date(b.last_job_finished_at).getTime() : 0;
-            return ta - tb;
+            return ra - rb || ta - tb;
           });
           const resource = candidates[0];
           claimedResourceIds.add(resource.id);

package/src/services/updates.js

@@ -46,7 +46,8 @@ function createUpdatesService({ config }){
       state.latest = { ...state.latest, ...latest };
       state.checkedAt = new Date().toISOString();
       state.error = errors.length ? errors.join('; ') : null;
-      return status();
+      // What this check itself got — `latest` keeps earlier results.
+      return { ...status(), fetched: Object.keys(latest) };
     })().finally(() => {
       inFlight = null;
     });

package/src/web/routes.js

@@ -40,9 +40,11 @@ const AVATAR_MAX_BYTES = 2 * 1024 * 1024,
 function createWebRouter({ services, config }){
   const router = express.Router();
 
-  function flash(req, type, text){
+  // Shown as a toast (layout.pug): `danger` stays until closed, `warning`
+  // closes after 30 s, `info`/`success` after 10 s — unless `sticky`.
+  function flash(req, type, text, { sticky = false } = {}){
     req.session.flash = req.session.flash || [];
-    req.session.flash.push({ type, text });
+    req.session.flash.push({ type, text, ...(sticky ? { sticky } : {}) });
   }
 
   // Where a resource action (maintenance, rotate token) came from — the
@@ -107,7 +109,7 @@ function createWebRouter({ services, config }){
     const { username, password } = req.body,
       user = services.adminUsers.verify(username, password);
     if (!user){
-      return res.status(401).render('login', { title: 'Sign in', error: 'Invalid credentials' });
+      return res.status(401).render('login', { title: 'Sign in', messages: [{ type: 'danger', text: 'Invalid credentials' }] });
     }
     req.session.user = user;
     req.session.cookie.maxAge = user.sessionTimeoutMin * 60 * 1000;
@@ -287,7 +289,8 @@ function createWebRouter({ services, config }){
     const token = generateToken('res');
     services.db.prepare('UPDATE resources SET token_hash = ? WHERE id = ?').run(hashToken(token), req.params.id);
     services.events.record('resource', req.params.id, 'resource.token_rotated', {});
-    flash(req, 'warning', `New resource token (copy it now): ${token}`);
+    // Sticky: shown only once, so it mustn't close before it's copied.
+    flash(req, 'warning', `New resource token (copy it now): ${token}`, { sticky: true });
     res.redirect(returnTo(req, '/resources'));
   });
 
@@ -297,7 +300,10 @@ function createWebRouter({ services, config }){
   function updateAction(fn){
     return async (req, res) => {
       try {
-        flash(req, 'info', await fn(req));
+        // `fn` returns the message, or { type, text } for another severity.
+        const result = await fn(req),
+          { type, text } = typeof result === 'string' ? { type: 'info', text: result } : result;
+        flash(req, type, text);
       }
       catch (err){
         flash(req, 'danger', err.message);
@@ -306,10 +312,45 @@ function createWebRouter({ services, config }){
     };
   }
 
+  // Check now (navbar, Agents and Resources pages): fetch the latest
+  // versions, then report whether the Coordinator and the connected
+  // (heartbeating) Clients are behind — as a warning if anything is.
   router.post('/updates/check', requireAdminRole, updateAction(async () => {
     const s = await services.updates.checkNow(),
-      found = Object.entries(s.latest).map(([app, v]) => `${app} v${v}`).join(', ');
-    return s.error ? `Version check: ${found || 'nothing found'} (errors: ${s.error})` : `Latest versions: ${found}.`;
+      parts = [];
+    if (!s.fetched.length){
+      return { type: 'danger', text: `Update check failed — the npm registry couldn't be reached (${s.error}). Nothing was checked.` };
+    }
+
+    parts.push(s.coordinatorUpdate
+      ? `Coordinator v${s.coordinatorVersion} → v${s.coordinatorUpdate} available (Update app button).`
+      : `Coordinator is up to date (v${s.coordinatorVersion}).`);
+
+    const connected = services.registry.list().filter((r) => r.status !== RESOURCE_STATES.OUT_OF_SERVICE && r.last_heartbeat_at),
+      outdated = connected.filter((r) => r.client_version && isNewer(s.latest.client, r.client_version)),
+      unknown = connected.filter((r) => !r.client_version),
+      names = (list) => list.slice(0, 5).map((r) => `${r.name}${r.client_version ? ` (v${r.client_version})` : ''}`).join(', ') +
+        (list.length > 5 ? `, +${list.length - 5} more` : '');
+    if (!s.latest.client){
+      parts.push('Latest Client version unknown.');
+    }
+    else if (!connected.length){
+      parts.push('No connected resources.');
+    }
+    else if (outdated.length){
+      parts.push(`${outdated.length} of ${connected.length} connected resource(s) can update to Client v${s.latest.client}: ${names(outdated)}.`);
+    }
+    else {
+      parts.push(`All ${connected.length} connected resource(s) run the latest Client (v${s.latest.client}).`);
+    }
+    if (unknown.length){
+      parts.push(`Version unknown (Client too old to report it): ${names(unknown)}.`);
+    }
+    if (s.error){
+      parts.push(`Some checks failed: ${s.error}`);
+    }
+    const behind = s.coordinatorUpdate || outdated.length || unknown.length;
+    return { type: behind || s.error ? 'warning' : 'success', text: parts.join(' ') };
   }));
 
   router.post('/updates/coordinator', requireAdminRole, updateAction(async (req) => {
@@ -392,6 +433,7 @@ function createWebRouter({ services, config }){
       title: job.id,
       active: 'jobs',
       job,
+      pinnedClient: job.spec.target.client ? services.registry.get(job.spec.target.client) : null,
       artifacts,
       jobActive,
       canCancel: req.session.user.role === 'admin' && jobActive

package/test/scheduler.test.js

@@ -1,6 +1,6 @@
 /**
  * @file        packages/coordinator/test/scheduler.test.js
- * @description Tests: scheduler, registry, groups, job id allocation, and admin queue operations
+ * @description Tests: scheduler, registry, groups, client pinning, job id allocation, and admin queue operations
  *
  * @author      Andrian Yablonskyy
  * @copyright   Copyright (c) 2026 Andrian Yablonskyy. All rights reserved.
@@ -349,6 +349,76 @@ test('a job with target.group is rejected (422) when no resource is ever a membe
   );
 });
 
+test('a job with target.client (by name) waits for that client even when another matching one is idle', () => {
+  const { registry, agents, jobs, scheduler } = buildTestServices(),
+    { agent } = agents.create({ name: 'ci', kind: 'ci' }),
+    pinned = registerResource(registry, { name: 'lab-pinned', type: 'sw' }),
+    other = registerResource(registry, { name: 'lab-other', type: 'sw' });
+  registry.heartbeat(other.id, { state: 'idle' }); // pinned is still REGISTERED, not IDLE
+
+  const job = jobs.create({
+    agentId: agent.id,
+    source: 'cli',
+    spec: makeSpec({ target: { type: 'sw', labels: [], client: 'lab-pinned' } })
+  });
+  assert.equal(job.spec.target.client, pinned.id); // name resolved to the stable resource id
+
+  scheduler.runPass();
+  assert.equal(jobs.get(job.id).state, JOB_STATES.QUEUED); // not grabbed by lab-other
+
+  registry.heartbeat(pinned.id, { state: 'idle' });
+  scheduler.runPass();
+  const assigned = jobs.get(job.id);
+  assert.equal(assigned.state, JOB_STATES.ASSIGNED);
+  assert.equal(assigned.resource_id, pinned.id);
+});
+
+test('an unpinned job prefers a resource no queued job is pinned to', () => {
+  const { registry, agents, jobs, scheduler } = buildTestServices(),
+    { agent } = agents.create({ name: 'ci', kind: 'ci' }),
+    pinned = registerResource(registry, { name: 'lab-pinned', type: 'sw' }),
+    other = registerResource(registry, { name: 'lab-other', type: 'sw' });
+  registry.heartbeat(pinned.id, { state: 'idle' });
+  registry.heartbeat(other.id, { state: 'idle' });
+
+  // Unpinned job has higher priority, so it's considered first; it must not
+  // take lab-pinned (which is also least-recently-used) away from the pinned job.
+  const unpinned = jobs.create({ agentId: agent.id, source: 'cli', spec: makeSpec({ priority: 90 }) }),
+    pinnedJob = jobs.create({
+      agentId: agent.id,
+      source: 'cli',
+      spec: makeSpec({ target: { type: 'sw', labels: [], client: pinned.id }, priority: 10 })
+    });
+  scheduler.runPass();
+
+  assert.equal(jobs.get(unpinned.id).resource_id, other.id);
+  assert.equal(jobs.get(pinnedJob.id).resource_id, pinned.id);
+});
+
+test('a job with target.client is rejected (422) when the client is unknown or cannot satisfy the target', () => {
+  const { registry, agents, jobs } = buildTestServices(),
+    { agent } = agents.create({ name: 'ci', kind: 'ci' });
+  registerResource(registry, { name: 'lab-sw', type: 'sw', labels: ['board:a'] });
+
+  assert.throws(
+    () => jobs.create({ agentId: agent.id, source: 'cli', spec: makeSpec({ target: { type: 'sw', client: 'nope' } }) }),
+    /Unknown client "nope"/
+  );
+  assert.throws(
+    () => jobs.create({ agentId: agent.id, source: 'cli', spec: makeSpec({ target: { type: 'hw', client: 'lab-sw' } }) }),
+    /No registered resource/
+  );
+  assert.throws(
+    () =>
+      jobs.create({
+        agentId: agent.id,
+        source: 'cli',
+        spec: makeSpec({ target: { type: 'sw', labels: ['board:b'], client: 'lab-sw' } })
+      }),
+    /No registered resource/
+  );
+});
+
 test('a resource can belong to several groups at once', () => {
   const { registry, groups } = buildTestServices(),
     g1 = groups.create({ name: 'g1' }),

package/views/jobs/show.pug

@@ -22,6 +22,9 @@ block content
     dd.col-sm-10= job.resource ? job.resource.name : '—'
     dt.col-sm-2 Target
     dd.col-sm-10= `${job.spec.target.type} — ${(job.spec.target.labels || []).join(', ')}`
+    if job.spec.target.client
+      dt.col-sm-2 Client
+      dd.col-sm-10= pinnedClient ? pinnedClient.name : job.spec.target.client
     dt.col-sm-2 Created
     dd.col-sm-10= fmtDate(job.created_at)
     if job.message

package/views/layout.pug

@@ -37,6 +37,18 @@ html(lang="en" data-bs-theme=(user && user.theme) || "auto")
                 li.nav-item
                   a.nav-link(href="/admin/agents" class=active === 'agents' ? 'active' : '') Agents
           ul.navbar-nav
+            if user && user.role === 'admin'
+              li.nav-item.d-flex.align-items-center.me-2
+                form(method="post" action="/updates/check" onsubmit="this.querySelector('button').disabled = true; this.querySelector('i').className = 'spinner-border spinner-border-sm';")
+                  input(type="hidden" name="returnTo" value=currentPath)
+                  button.btn.btn-sm.btn-outline-secondary(
+                    type="submit"
+                    aria-label="Check for updates"
+                    data-bs-toggle="tooltip"
+                    data-bs-placement="bottom"
+                    data-bs-title=updates.checkedAt ? `Check for updates now — the Coordinator and connected resources (last checked ${fmtDate(updates.checkedAt)})` : 'Check for updates now — the Coordinator and connected resources'
+                  )
+                    i.bi.bi-arrow-repeat
             if user && user.role === 'admin' && updates.coordinatorUpdate
               li.nav-item.d-flex.align-items-center.me-3
                 if updates.coordinatorPending
@@ -91,13 +103,37 @@ html(lang="en" data-bs-theme=(user && user.theme) || "auto")
                     data-bs-title="Sign out of the dashboard"
                   )
                     i.bi.bi-box-arrow-right
-    main.container.py-4
-      if messages && messages.length
+    //- Flash messages as toasts (public/js/toasts.js): errors stay until
+    //- closed, warnings close after 30 s, info/success after 10 s. Rendered
+    //- visible (.show), so they're readable even if the script fails.
+    if messages && messages.length
+      .toast-container.position-fixed.end-0.p-3(style="top: 3.5rem; z-index: 1090;")
         each m in messages
-          .alert(class=`alert-${m.type || 'info'}`)= m.text
+          - const type = ['danger', 'warning', 'info', 'success'].includes(m.type) ? m.type : 'info'
+          - const icon = { danger: 'x-octagon-fill', warning: 'exclamation-triangle-fill', info: 'info-circle-fill', success: 'check-circle-fill' }[type]
+          .toast.show.border-0(
+            class=`text-bg-${type}`
+            role=type === 'danger' ? 'alert' : 'status'
+            aria-live=type === 'danger' ? 'assertive' : 'polite'
+            aria-atomic="true"
+            data-thub-toast=type
+            data-sticky=m.sticky ? '1' : false
+          )
+            .d-flex
+              .toast-body.d-flex.gap-2
+                i.bi(class=`bi-${icon}`)
+                span= m.text
+              button.btn-close.me-2.m-auto(
+                type="button"
+                class=type === 'danger' || type === 'success' ? 'btn-close-white' : ''
+                data-bs-dismiss="toast"
+                aria-label="Close"
+              )
+    main.container.py-4
       block content
     script(src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js")
     script(src="/js/theme.js")
     script(src="/js/tooltips.js")
+    script(src="/js/toasts.js")
     script(src="/js/copy-to-clipboard.js")
     block scripts

package/views/login.pug

@@ -4,8 +4,6 @@ block content
   .row.justify-content-center
     .col-md-4
       h1.h4.mb-3 Sign in
-      if error
-        .alert.alert-danger= error
       form(method="post" action="/login")
         .mb-3
           label.form-label(for="username") Username