@andrian.yablonskyy/thub-coordinator 1.1.13 → 1.1.15
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -5
- package/package.json +2 -2
- package/scripts/install-default-config.js +3 -1
- package/scripts/install-systemd-unit.js +13 -2
- package/scripts/install-target.js +18 -0
- package/src/config.js +30 -4
- package/src/server.js +19 -6
- package/src/services/updates.js +6 -0
- package/src/tls.js +96 -0
- package/systemd/thub-coordinator.service +2 -0
- package/test/config.test.js +76 -0
- package/test/tls.test.js +141 -0
- package/test/updates.test.js +38 -1
- package/views/help/_agent-cli.pug +4 -4
- package/views/help/_ci.pug +2 -2
- package/views/help/_client-machines.pug +4 -5
- package/views/help/_client-setup.pug +2 -2
- package/views/help/_coordinator.pug +36 -6
- package/views/help/_docker.pug +38 -10
- package/views/help/_env.pug +11 -14
- package/views/help/_git.pug +10 -6
- package/views/help/_kubernetes.pug +169 -0
- package/views/help/index.pug +2 -0
- package/views/layout.pug +4 -3
- package/views/mixins/client-config.pug +1 -1
package/README.md
CHANGED
|
@@ -50,6 +50,12 @@ The Coordinator loads a plain **JSON** config file (no YAML support). Resolution
|
|
|
50
50
|
|
|
51
51
|
`sessionSecret` signs the dashboard's session cookie and the HMAC on artifact download links; `clientJoinKey` is the shared secret Clients self-register with — omit or leave `null` to disable auto-registration entirely. `trustProxy` is Express's [`trust proxy`](https://expressjs.com/en/guide/behind-proxies.html) setting (default `loopback`): which reverse proxies' `X-Forwarded-For` to believe for a Client's external address on the resource card — e.g. `"10.0.0.0/8"` or `1` for a proxy on another host, `false` to always use the socket address. Individual `THUB_LISTEN` / `THUB_PUBLIC_URL` / `THUB_DATA_DIR` / `THUB_SESSION_SECRET` / `THUB_CLIENT_JOIN_KEY` env vars override whatever the file set.
|
|
52
52
|
|
|
53
|
+
**Kubernetes:** a Dockerfile, manifests (one replica, a volume for SQLite, ingress with SSE-friendly settings) and the upgrade, backup and HTTPS notes are in the main README, §13.3, and on the dashboard's Help page (*Running in Kubernetes*). The image starts without `--self-update`: update by deploying a new image.
|
|
54
|
+
|
|
55
|
+
**Database file:** `dbPath` (or `THUB_DB_PATH`). The default is `<dataDir>/thub.db`; a relative path is taken from `dataDir`. Its directory must be writable, because SQLite keeps `-wal`/`-shm` files next to it. Details: main README §13.
|
|
56
|
+
|
|
57
|
+
**HTTPS without a reverse proxy:** set `tls.certFile` / `tls.keyFile` (PEM, readable by the service user; optional `tls.caFile`, `tls.passphrase`), or `THUB_TLS_CERT` / `THUB_TLS_KEY` / `THUB_TLS_CA` / `THUB_TLS_PASSPHRASE`, and an `https://` `publicUrl`. The Coordinator then serves HTTPS itself. Bad files stop the start with the reason, and renewed files are picked up within a minute or on `SIGHUP`. For a self-signed or private CA, start Clients and Agents with `NODE_EXTRA_CA_CERTS`. Details: main README §13.
|
|
58
|
+
|
|
53
59
|
`npm install -g` creates `~/.config/thub/coordinator.json` for you if it doesn't already exist, with all the options above, `dataDir` set to `~/var/lib/thub` and a freshly generated random `sessionSecret` (not the placeholder above), readable only by its owner — a re-install never overwrites it or regenerates the secret. Set `publicUrl` and `clientJoinKey` yourself before relying on auto-registration.
|
|
54
60
|
|
|
55
61
|
## Running it
|
|
@@ -79,9 +85,9 @@ Talks to the Coordinator's SQLite database directly — no running server requir
|
|
|
79
85
|
# THUB_BOOTSTRAP_ADMIN_PASSWORD above).
|
|
80
86
|
thub-admin create-admin alice s3cret --role admin
|
|
81
87
|
|
|
82
|
-
#
|
|
83
|
-
#
|
|
84
|
-
thub-admin agent add ci-firmware
|
|
88
|
+
# A CI token for a pipeline (people get their key with: thub-admin user key);
|
|
89
|
+
# the token is shown once.
|
|
90
|
+
thub-admin agent add ci-firmware
|
|
85
91
|
# -> Agent agt_... created. Token (shown once): agt_...
|
|
86
92
|
|
|
87
93
|
# Mint the shared secret Clients use to self-register, and put it in both
|
|
@@ -102,7 +108,7 @@ thub-admin group add ci-nightly --comment "shared CI pool"
|
|
|
102
108
|
thub-admin group list
|
|
103
109
|
thub-admin group remove <groupId>
|
|
104
110
|
|
|
105
|
-
# Update the Coordinator by hand (the dashboard's "Update app" button does the same through a root helper).
|
|
111
|
+
# Update the Coordinator by hand (with --self-update, the dashboard's "Update app" button does the same through a root helper).
|
|
106
112
|
thub-admin check-update # installed vs latest published version
|
|
107
113
|
thub-admin self-update [--to <x.y.z>] # sudo npm i -g; the postinstall restarts the service
|
|
108
114
|
```
|
|
@@ -166,7 +172,7 @@ The Coordinator's own version (`ver. X.Y.Z`) is shown under the TestHub logo, to
|
|
|
166
172
|
**Versions and self-update.** The Coordinator checks the npm registry for the latest Coordinator, Agent and Client every `updates.checkIntervalMin` (default 15 minutes, `0` disables; the older `checkIntervalHours` is ignored; `updates.registry` for a private mirror) and immediately with the navbar's **check for updates** button (admins; also on the Agents and Resources pages), which reports in a toast whether the Coordinator is current and which connected resources can update their Client. A newer Coordinator shows as an **Update app to X.Y.Z** button in the navbar for admins (a badge under the logo for viewers); Agents and Resources show each one's version with an **update available** or pending **→ vX.Y.Z** badge. Admins request self-updates per agent/resource (**Update**, which becomes **Cancel update**) or with **Update all agents** / **Update all clients**, always to the latest version:
|
|
167
173
|
- Clients get a `self-update` command on their next heartbeat and install it through their root `thub-client-update` helper, once none of the host's instances is busy.
|
|
168
174
|
- Agents install it at the start of their next run (`GET /api/v1/agents/me/update`), then re-run the command on the new version.
|
|
169
|
-
- The Coordinator itself: the **Update app** button writes `<dataDir>/update-request.json`; the root `thub-coordinator-update.path` unit (installed by `sudo npm i -g`) runs `npm i -g` for it, and the postinstall restarts the service (the dashboard and API are briefly down). Logs: `journalctl -u thub-coordinator-update`. Without that unit, use `thub-admin self-update` on the host.
|
|
175
|
+
- The Coordinator itself, only when started with `--self-update` (off by default; under systemd add it to the unit's `ExecStart`, a re-install keeps it): the **Update app** button writes `<dataDir>/update-request.json`; the root `thub-coordinator-update.path` unit (installed by `sudo npm i -g`) runs `npm i -g` for it, and the postinstall restarts the service (the dashboard and API are briefly down). Logs: `journalctl -u thub-coordinator-update`. Without that unit, use `thub-admin self-update` on the host.
|
|
170
176
|
|
|
171
177
|
A request is cleared when the Agent/Client reports the new version.
|
|
172
178
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@andrian.yablonskyy/thub-coordinator",
|
|
3
|
-
"version": "1.1.
|
|
3
|
+
"version": "1.1.15",
|
|
4
4
|
"description": "TestHub Coordinator — job queue, resource registry, scheduler, heartbeat monitor, log/artifact store and web dashboard",
|
|
5
5
|
"main": "src/server.js",
|
|
6
6
|
"engines": {
|
|
@@ -19,7 +19,7 @@
|
|
|
19
19
|
"lint:fix": "eslint . --fix"
|
|
20
20
|
},
|
|
21
21
|
"dependencies": {
|
|
22
|
-
"@andrian.yablonskyy/thub-common": "^1.1.
|
|
22
|
+
"@andrian.yablonskyy/thub-common": "^1.1.4",
|
|
23
23
|
"better-sqlite3": "^12.4.1",
|
|
24
24
|
"express": "^5.1.0",
|
|
25
25
|
"express-session": "^1.18.2",
|
|
@@ -81,7 +81,9 @@ function main(){
|
|
|
81
81
|
paths = targetPaths(user);
|
|
82
82
|
try {
|
|
83
83
|
mkdirOwned(paths.configDir, user);
|
|
84
|
-
|
|
84
|
+
// The database's own directory too, when dbPath puts it elsewhere: the
|
|
85
|
+
// systemd unit's ReadWritePaths needs it to exist.
|
|
86
|
+
for (const dir of [paths.dataDir, ...paths.dataSubdirs, ...(paths.dbDir ? [paths.dbDir] : [])]){
|
|
85
87
|
mkdirOwned(dir, user);
|
|
86
88
|
}
|
|
87
89
|
if (!fs.existsSync(paths.configPath)){
|
|
@@ -34,13 +34,24 @@ const fs = require('node:fs'),
|
|
|
34
34
|
// The checked-in unit only has placeholders — fill in the target user and
|
|
35
35
|
// wherever *this* install's node and server.js actually are, so it works
|
|
36
36
|
// regardless of npm prefix or an nvm-managed Node.
|
|
37
|
+
// `--self-update` (README §10.2) is off by default; once added to the
|
|
38
|
+
// installed unit's ExecStart, a re-install keeps it there.
|
|
39
|
+
function selfUpdateFlag(existingUnit = UNIT_DEST){
|
|
40
|
+
try {
|
|
41
|
+
return /^ExecStart=.*\s--self-update(\s|$)/m.test(fs.readFileSync(existingUnit, 'utf8')) ? ' --self-update' : '';
|
|
42
|
+
}
|
|
43
|
+
catch {
|
|
44
|
+
return '';
|
|
45
|
+
}
|
|
46
|
+
}
|
|
47
|
+
|
|
37
48
|
function renderUnit(user, paths){
|
|
38
49
|
return fs.readFileSync(UNIT_SRC, 'utf8')
|
|
39
50
|
.replace(/^User=.*$/m, `User=${user.name}`)
|
|
40
51
|
.replace(/^Group=.*$/m, `Group=${user.gid}`)
|
|
41
52
|
.replace(/^Environment=THUB_COORDINATOR_CONFIG=.*$/m, `Environment=THUB_COORDINATOR_CONFIG=${paths.configPath}`)
|
|
42
|
-
.replace(/^ExecStart=.*$/m, `ExecStart=${process.execPath} ${SERVER_PATH}`)
|
|
43
|
-
.replace(/^ReadWritePaths=.*$/m, `ReadWritePaths=${paths.dataDir}`);
|
|
53
|
+
.replace(/^ExecStart=.*$/m, `ExecStart=${process.execPath} ${SERVER_PATH}${selfUpdateFlag()}`)
|
|
54
|
+
.replace(/^ReadWritePaths=.*$/m, `ReadWritePaths=${[paths.dataDir, paths.dbDir].filter(Boolean).join(' ')}`);
|
|
44
55
|
}
|
|
45
56
|
|
|
46
57
|
function renderUpdateUnits(user, paths){
|
|
@@ -66,6 +66,9 @@ function targetPaths(user){
|
|
|
66
66
|
configPath,
|
|
67
67
|
defaultDataDir,
|
|
68
68
|
dataDir,
|
|
69
|
+
// The database's directory, when the config's dbPath puts it outside
|
|
70
|
+
// dataDir: the systemd unit must allow writes there too.
|
|
71
|
+
dbDir: configuredDbDir(configPath, dataDir),
|
|
69
72
|
// Written by the dashboard's "Update app" button, watched by
|
|
70
73
|
// thub-coordinator-update.path (README §10.2).
|
|
71
74
|
updateRequestFile: path.join(dataDir, 'update-request.json'),
|
|
@@ -73,6 +76,21 @@ function targetPaths(user){
|
|
|
73
76
|
};
|
|
74
77
|
}
|
|
75
78
|
|
|
79
|
+
function configuredDbDir(configPath, dataDir){
|
|
80
|
+
try {
|
|
81
|
+
const dbPath = JSON.parse(fs.readFileSync(configPath, 'utf8'))?.dbPath;
|
|
82
|
+
if (typeof dbPath !== 'string' || !dbPath.trim()){
|
|
83
|
+
return null;
|
|
84
|
+
}
|
|
85
|
+
const dir = path.dirname(path.resolve(dataDir, dbPath)),
|
|
86
|
+
inside = path.relative(dataDir, dir);
|
|
87
|
+
return inside === '' || (!inside.startsWith('..') && !path.isAbsolute(inside)) ? null : dir;
|
|
88
|
+
}
|
|
89
|
+
catch {
|
|
90
|
+
return null;
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
|
|
76
94
|
function configuredDataDir(configPath){
|
|
77
95
|
try {
|
|
78
96
|
const dataDir = JSON.parse(fs.readFileSync(configPath, 'utf8'))?.dataDir;
|
package/src/config.js
CHANGED
|
@@ -27,6 +27,10 @@ const DEFAULTS = {
|
|
|
27
27
|
// default setup — listening on 127.0.0.1 behind a local reverse proxy.
|
|
28
28
|
trustProxy: 'loopback',
|
|
29
29
|
dataDir: path.join(process.cwd(), '.data'),
|
|
30
|
+
// The SQLite database file. null = <dataDir>/thub.db; a relative path is
|
|
31
|
+
// taken from dataDir. Its directory must be writable: SQLite keeps its
|
|
32
|
+
// -wal/-shm files next to it.
|
|
33
|
+
dbPath: null,
|
|
30
34
|
sessionSecret: 'dev-only-change-me',
|
|
31
35
|
session: {
|
|
32
36
|
// The dashboard session cookie's Secure flag (server.js cookieSecure):
|
|
@@ -34,6 +38,16 @@ const DEFAULTS = {
|
|
|
34
38
|
// Secure only on a connection that is itself HTTPS); true / false force it.
|
|
35
39
|
secureCookie: 'auto'
|
|
36
40
|
},
|
|
41
|
+
// HTTPS served by the Coordinator itself (tls.js): PEM files, readable by
|
|
42
|
+
// its user. No certFile/keyFile = plain HTTP, e.g. behind a reverse proxy
|
|
43
|
+
// that terminates TLS. caFile: intermediate certificates, if not already
|
|
44
|
+
// in certFile; passphrase: for an encrypted key.
|
|
45
|
+
tls: {
|
|
46
|
+
certFile: null,
|
|
47
|
+
keyFile: null,
|
|
48
|
+
caFile: null,
|
|
49
|
+
passphrase: null
|
|
50
|
+
},
|
|
37
51
|
// Shared secret Clients present to self-register (see api/resource.js).
|
|
38
52
|
// null disables auto-registration entirely — set it explicitly to turn it on.
|
|
39
53
|
clientJoinKey: null,
|
|
@@ -127,12 +141,20 @@ function loadConfig(configPath = process.env.THUB_COORDINATOR_CONFIG){
|
|
|
127
141
|
if (process.env.THUB_DATA_DIR){
|
|
128
142
|
config.dataDir = process.env.THUB_DATA_DIR;
|
|
129
143
|
}
|
|
144
|
+
if (process.env.THUB_DB_PATH){
|
|
145
|
+
config.dbPath = process.env.THUB_DB_PATH;
|
|
146
|
+
}
|
|
130
147
|
if (process.env.THUB_SESSION_SECRET){
|
|
131
148
|
config.sessionSecret = process.env.THUB_SESSION_SECRET;
|
|
132
149
|
}
|
|
133
150
|
if (process.env.THUB_CLIENT_JOIN_KEY){
|
|
134
151
|
config.clientJoinKey = process.env.THUB_CLIENT_JOIN_KEY;
|
|
135
152
|
}
|
|
153
|
+
for (const [env, key]of [['THUB_TLS_CERT', 'certFile'], ['THUB_TLS_KEY', 'keyFile'], ['THUB_TLS_CA', 'caFile'], ['THUB_TLS_PASSPHRASE', 'passphrase']]){
|
|
154
|
+
if (process.env[env]){
|
|
155
|
+
config.tls[key] = process.env[env];
|
|
156
|
+
}
|
|
157
|
+
}
|
|
136
158
|
|
|
137
159
|
for (const key of ['requestsPerMinute', 'loginPerMinute']){
|
|
138
160
|
const v = config.rateLimit[key];
|
|
@@ -160,20 +182,24 @@ function loadConfig(configPath = process.env.THUB_COORDINATOR_CONFIG){
|
|
|
160
182
|
const [host, port] = config.listen.split(':');
|
|
161
183
|
config.host = host;
|
|
162
184
|
config.port = Number(port);
|
|
163
|
-
config.dbPath
|
|
185
|
+
if (config.dbPath !== null && (typeof config.dbPath !== 'string' || !config.dbPath.trim())){
|
|
186
|
+
throw new Error(`dbPath must be a file path, or null for <dataDir>/thub.db (got ${JSON.stringify(config.dbPath)}) in ${candidate || 'the config'}`);
|
|
187
|
+
}
|
|
188
|
+
config.dbPath = config.dbPath ? path.resolve(config.dataDir, config.dbPath) : path.join(config.dataDir, 'thub.db');
|
|
164
189
|
config.workDir = path.join(config.dataDir, 'work');
|
|
165
190
|
config.avatarsDir = path.join(config.dataDir, 'avatars');
|
|
166
191
|
|
|
167
192
|
ensureWritableDir(config.dataDir, config.configPath);
|
|
168
193
|
ensureWritableDir(config.avatarsDir, config.configPath);
|
|
194
|
+
ensureWritableDir(path.dirname(config.dbPath), config.configPath, 'dbPath / THUB_DB_PATH');
|
|
169
195
|
|
|
170
196
|
return config;
|
|
171
197
|
}
|
|
172
198
|
|
|
173
|
-
// SQLite opens thub.db (plus its -wal/-shm files)
|
|
199
|
+
// SQLite opens thub.db (plus its -wal/-shm files) in its directory, so an
|
|
174
200
|
// existing but unwritable dataDir — typically one created by an earlier
|
|
175
201
|
// run under sudo — would otherwise fail later as a bare SQLITE_CANTOPEN.
|
|
176
|
-
function ensureWritableDir(dir, configPath){
|
|
202
|
+
function ensureWritableDir(dir, configPath, setting = 'dataDir / THUB_DATA_DIR'){
|
|
177
203
|
try {
|
|
178
204
|
fs.mkdirSync(dir, { recursive: true });
|
|
179
205
|
fs.accessSync(dir, fs.constants.W_OK);
|
|
@@ -181,7 +207,7 @@ function ensureWritableDir(dir, configPath){
|
|
|
181
207
|
catch (err){
|
|
182
208
|
throw new Error(
|
|
183
209
|
`Coordinator data directory ${dir} is not writable by this user (${err.code || err.message}). ` +
|
|
184
|
-
`Fix its ownership, or set
|
|
210
|
+
`Fix its ownership, or set ${setting} (in ${configPath || 'the config'}) to a writable path.`
|
|
185
211
|
);
|
|
186
212
|
}
|
|
187
213
|
}
|
package/src/server.js
CHANGED
|
@@ -21,6 +21,7 @@ const fs = require('node:fs'),
|
|
|
21
21
|
session = require('express-session'),
|
|
22
22
|
|
|
23
23
|
{ loadConfig } = require('./config'),
|
|
24
|
+
{ createServer, readTlsOptions } = require('./tls'),
|
|
24
25
|
{ openDb } = require('./db'),
|
|
25
26
|
{ bus } = require('./services/bus'),
|
|
26
27
|
{ createEventsService } = require('./services/events'),
|
|
@@ -209,9 +210,16 @@ function createApp(config, services){
|
|
|
209
210
|
return app;
|
|
210
211
|
}
|
|
211
212
|
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
213
|
+
// `node server.js [--self-update]`. --self-update turns on the dashboard's
|
|
214
|
+
// "Update app" (an admin installs a newer Coordinator through the root
|
|
215
|
+
// thub-coordinator-update helper, README §10.2). Off by default — e.g. in a
|
|
216
|
+
// container, where a new image is the update.
|
|
217
|
+
function start(configPath, argv = process.argv.slice(2)){
|
|
218
|
+
const config = loadConfig(configPath);
|
|
219
|
+
config.selfUpdate = argv.includes('--self-update');
|
|
220
|
+
// Certificate problems stop the Coordinator before anything else starts.
|
|
221
|
+
readTlsOptions(config.tls, config.configPath || 'the config');
|
|
222
|
+
const services = buildServices(config),
|
|
215
223
|
app = createApp(config, services);
|
|
216
224
|
|
|
217
225
|
ensureBootstrapAdmin(services);
|
|
@@ -219,10 +227,15 @@ function start(configPath){
|
|
|
219
227
|
// Clients and jobs to work on the dashboard with (src/dev/virtual.js).
|
|
220
228
|
services.dev = startDevMode(services, config);
|
|
221
229
|
|
|
222
|
-
const server = app
|
|
223
|
-
|
|
224
|
-
console.log(`
|
|
230
|
+
const { server, scheme } = createServer(app, config);
|
|
231
|
+
server.listen(config.port, config.host, () => {
|
|
232
|
+
console.log(`Config: ${config.configPath || '(built-in defaults)'}; data: ${config.dataDir}; database: ${config.dbPath}`);
|
|
233
|
+
console.log(`TestHub Coordinator listening on ${scheme}://${config.host}:${config.port}`);
|
|
234
|
+
if (scheme === 'https' && !/^https:/.test(config.publicUrl)){
|
|
235
|
+
console.warn(`tls: serving HTTPS, but publicUrl is ${config.publicUrl} — set it to the https:// address Clients and Agents use`);
|
|
236
|
+
}
|
|
225
237
|
console.log(`Coordinator v${require('../package.json').version}, thub-common v${commonVersion()} (${commonPath()})`);
|
|
238
|
+
console.log(config.selfUpdate ? 'Self-update: on (--self-update)' : 'Self-update: off (start with --self-update to update from the dashboard)');
|
|
226
239
|
});
|
|
227
240
|
|
|
228
241
|
return { app, server, services, config };
|
package/src/services/updates.js
CHANGED
|
@@ -120,6 +120,8 @@ function createUpdatesService({ config, pathUnit = UPDATE_PATH_UNIT, fetchLatest
|
|
|
120
120
|
return {
|
|
121
121
|
...state,
|
|
122
122
|
coordinatorVersion,
|
|
123
|
+
// The dashboard's "Update app": only with `--self-update` (server.js).
|
|
124
|
+
selfUpdate: Boolean(config.selfUpdate),
|
|
123
125
|
coordinatorUpdate: isNewer(state.latest.coordinator, coordinatorVersion) ? state.latest.coordinator : null
|
|
124
126
|
};
|
|
125
127
|
}
|
|
@@ -128,6 +130,10 @@ function createUpdatesService({ config, pathUnit = UPDATE_PATH_UNIT, fetchLatest
|
|
|
128
130
|
// thub-coordinator-update helper, whose install restarts this process
|
|
129
131
|
// (so `coordinatorPending` only lasts until then).
|
|
130
132
|
function requestCoordinatorUpdate(requestedBy){
|
|
133
|
+
if (!config.selfUpdate){
|
|
134
|
+
throw Object.assign(new Error('Self-update is off: start the Coordinator with --self-update to update it from the dashboard ' +
|
|
135
|
+
'(README §10.2), or update it by hand — thub-admin self-update, or a new container image'), { status: 403 });
|
|
136
|
+
}
|
|
131
137
|
const target = status().coordinatorUpdate;
|
|
132
138
|
if (!target){
|
|
133
139
|
throw Object.assign(new Error(`Already on the latest version (v${coordinatorVersion}).`), { status: 409 });
|
package/src/tls.js
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file packages/coordinator/src/tls.js
|
|
3
|
+
* @description HTTPS for the Coordinator itself: certificate files from the config, checked at start, reloaded when renewed (README §13)
|
|
4
|
+
*
|
|
5
|
+
* @author Andrian Yablonskyy
|
|
6
|
+
* @copyright Copyright (c) 2026 Andrian Yablonskyy. All rights reserved.
|
|
7
|
+
*
|
|
8
|
+
* This file is part of TestHub and is proprietary and confidential.
|
|
9
|
+
* Unauthorized copying, modification, distribution, or use of this file,
|
|
10
|
+
* via any medium, is strictly prohibited without prior written permission
|
|
11
|
+
* from AdSystem.PRO.
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
'use strict';
|
|
15
|
+
|
|
16
|
+
const fs = require('node:fs'),
|
|
17
|
+
http = require('node:http'),
|
|
18
|
+
https = require('node:https'),
|
|
19
|
+
tls = require('node:tls');
|
|
20
|
+
|
|
21
|
+
// How often renewed certificate files are looked for (Let's Encrypt,
|
|
22
|
+
// cert-manager rotate them in place). SIGHUP reloads at once.
|
|
23
|
+
const WATCH_INTERVAL_MS = 60_000;
|
|
24
|
+
|
|
25
|
+
// The `tls` config section as Node TLS options — or null for plain HTTP
|
|
26
|
+
// (no certFile/keyFile). Throws with a readable reason if the files can't be
|
|
27
|
+
// read or don't make a usable certificate/key pair, so a mistake stops the
|
|
28
|
+
// Coordinator at start instead of failing every connection later.
|
|
29
|
+
function readTlsOptions(cfg, where = 'the config'){
|
|
30
|
+
if (!cfg || (!cfg.certFile && !cfg.keyFile)){
|
|
31
|
+
return null;
|
|
32
|
+
}
|
|
33
|
+
if (!cfg.certFile || !cfg.keyFile){
|
|
34
|
+
throw new Error(`tls: set both certFile and keyFile (or neither, for plain HTTP) in ${where}`);
|
|
35
|
+
}
|
|
36
|
+
const read = (key) => {
|
|
37
|
+
try {
|
|
38
|
+
return fs.readFileSync(cfg[key]);
|
|
39
|
+
}
|
|
40
|
+
catch (err){
|
|
41
|
+
throw new Error(`tls.${key} ${cfg[key]} can't be read by this user (${err.code || err.message}) — check the path and its permissions`);
|
|
42
|
+
}
|
|
43
|
+
},
|
|
44
|
+
options = {
|
|
45
|
+
cert: read('certFile'),
|
|
46
|
+
key: read('keyFile'),
|
|
47
|
+
...(cfg.caFile ? { ca: read('caFile') } : {}),
|
|
48
|
+
...(cfg.passphrase ? { passphrase: cfg.passphrase } : {})
|
|
49
|
+
};
|
|
50
|
+
try {
|
|
51
|
+
tls.createSecureContext(options); // a cert that doesn't match its key, a wrong passphrase…
|
|
52
|
+
}
|
|
53
|
+
catch (err){
|
|
54
|
+
throw new Error(`tls: ${cfg.certFile} / ${cfg.keyFile} aren't a usable certificate and key: ${err.message}`);
|
|
55
|
+
}
|
|
56
|
+
return options;
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
// The Coordinator's server: HTTPS with the configured certificate, else
|
|
60
|
+
// plain HTTP (behind a TLS-terminating proxy, as before). With HTTPS, a
|
|
61
|
+
// renewed certificate is picked up without a restart: its files are checked
|
|
62
|
+
// every minute, and on SIGHUP.
|
|
63
|
+
function createServer(app, config, { log = console } = {}){
|
|
64
|
+
const options = readTlsOptions(config.tls);
|
|
65
|
+
if (!options){
|
|
66
|
+
return { server: http.createServer(app), scheme: 'http' };
|
|
67
|
+
}
|
|
68
|
+
const server = https.createServer(options, app),
|
|
69
|
+
files = [config.tls.certFile, config.tls.keyFile, config.tls.caFile].filter(Boolean),
|
|
70
|
+
reload = (why) => {
|
|
71
|
+
try {
|
|
72
|
+
server.setSecureContext(readTlsOptions(config.tls));
|
|
73
|
+
log.log(`tls: certificate reloaded (${why})`);
|
|
74
|
+
}
|
|
75
|
+
catch (err){
|
|
76
|
+
log.error(`tls: keeping the current certificate — ${err.message}`);
|
|
77
|
+
}
|
|
78
|
+
},
|
|
79
|
+
onChange = (cur, prev) => {
|
|
80
|
+
if (cur.mtimeMs !== prev.mtimeMs){
|
|
81
|
+
reload('file changed');
|
|
82
|
+
}
|
|
83
|
+
},
|
|
84
|
+
onHup = () => reload('SIGHUP');
|
|
85
|
+
for (const f of files){
|
|
86
|
+
fs.watchFile(f, { interval: WATCH_INTERVAL_MS, persistent: false }, onChange);
|
|
87
|
+
}
|
|
88
|
+
process.on('SIGHUP', onHup);
|
|
89
|
+
server.on('close', () => {
|
|
90
|
+
files.forEach((f) => fs.unwatchFile(f, onChange));
|
|
91
|
+
process.off('SIGHUP', onHup);
|
|
92
|
+
});
|
|
93
|
+
return { server, scheme: 'https' };
|
|
94
|
+
}
|
|
95
|
+
|
|
96
|
+
module.exports = { readTlsOptions, createServer };
|
|
@@ -3,6 +3,8 @@ Description=TestHub Coordinator
|
|
|
3
3
|
After=network-online.target
|
|
4
4
|
Wants=network-online.target
|
|
5
5
|
|
|
6
|
+
# Add --self-update to ExecStart to let admins update the Coordinator from
|
|
7
|
+
# the dashboard ("Update app", README §10.2); a re-install keeps it.
|
|
6
8
|
# User, Group, THUB_COORDINATOR_CONFIG, ReadWritePaths and ExecStart are
|
|
7
9
|
# rewritten by scripts/install-systemd-unit.js at install time to the user
|
|
8
10
|
# who ran `sudo npm i -g` and this install's real node/server.js paths.
|
|
@@ -0,0 +1,76 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file packages/coordinator/test/config.test.js
|
|
3
|
+
* @description Tests: the Coordinator config — database file location
|
|
4
|
+
*
|
|
5
|
+
* @author Andrian Yablonskyy
|
|
6
|
+
* @copyright Copyright (c) 2026 Andrian Yablonskyy. All rights reserved.
|
|
7
|
+
*
|
|
8
|
+
* This file is part of TestHub and is proprietary and confidential.
|
|
9
|
+
* Unauthorized copying, modification, distribution, or use of this file,
|
|
10
|
+
* via any medium, is strictly prohibited without prior written permission
|
|
11
|
+
* from AdSystem.PRO.
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
'use strict';
|
|
15
|
+
|
|
16
|
+
const test = require('node:test'),
|
|
17
|
+
assert = require('node:assert/strict'),
|
|
18
|
+
fs = require('node:fs'),
|
|
19
|
+
os = require('node:os'),
|
|
20
|
+
path = require('node:path'),
|
|
21
|
+
{ loadConfig } = require('../src/config'),
|
|
22
|
+
{ buildServices } = require('../src/server');
|
|
23
|
+
|
|
24
|
+
function withConfig(content){
|
|
25
|
+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'thub-cfg-')),
|
|
26
|
+
file = path.join(dir, 'coordinator.json');
|
|
27
|
+
fs.writeFileSync(file, JSON.stringify({ dataDir: path.join(dir, 'data'), updates: { checkIntervalMin: 0 }, ...content }));
|
|
28
|
+
return { dir, file };
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
test('dbPath: <dataDir>/thub.db by default; absolute, relative to dataDir, or THUB_DB_PATH; the database is opened there', () => {
|
|
32
|
+
const plain = withConfig({});
|
|
33
|
+
assert.equal(loadConfig(plain.file).dbPath, path.join(plain.dir, 'data', 'thub.db'));
|
|
34
|
+
|
|
35
|
+
const rel = withConfig({ dbPath: 'db/testhub.sqlite' });
|
|
36
|
+
assert.equal(loadConfig(rel.file).dbPath, path.join(rel.dir, 'data', 'db', 'testhub.sqlite'));
|
|
37
|
+
|
|
38
|
+
const abs = withConfig({}),
|
|
39
|
+
elsewhere = path.join(abs.dir, 'fast-disk', 'thub.db');
|
|
40
|
+
fs.writeFileSync(abs.file, JSON.stringify({ dataDir: path.join(abs.dir, 'data'), dbPath: elsewhere, updates: { checkIntervalMin: 0 } }));
|
|
41
|
+
const config = loadConfig(abs.file);
|
|
42
|
+
assert.equal(config.dbPath, elsewhere);
|
|
43
|
+
buildServices(config).db.close();
|
|
44
|
+
assert.ok(fs.existsSync(elsewhere), 'the database file is created at dbPath');
|
|
45
|
+
assert.ok(!fs.existsSync(path.join(abs.dir, 'data', 'thub.db')));
|
|
46
|
+
|
|
47
|
+
const env = withConfig({ dbPath: 'from-file.db' }),
|
|
48
|
+
saved = process.env.THUB_DB_PATH;
|
|
49
|
+
process.env.THUB_DB_PATH = path.join(env.dir, 'from-env.db');
|
|
50
|
+
try {
|
|
51
|
+
assert.equal(loadConfig(env.file).dbPath, path.join(env.dir, 'from-env.db'));
|
|
52
|
+
}
|
|
53
|
+
finally {
|
|
54
|
+
if (saved === undefined){
|
|
55
|
+
delete process.env.THUB_DB_PATH;
|
|
56
|
+
}
|
|
57
|
+
else {
|
|
58
|
+
process.env.THUB_DB_PATH = saved;
|
|
59
|
+
}
|
|
60
|
+
}
|
|
61
|
+
|
|
62
|
+
assert.throws(() => loadConfig(withConfig({ dbPath: 42 }).file), /dbPath must be a file path, or null/);
|
|
63
|
+
});
|
|
64
|
+
|
|
65
|
+
test('dbPath in a directory this user can\'t write: stops the start, naming the setting', { skip: process.getuid?.() === 0 }, () => {
|
|
66
|
+
const { dir, file } = withConfig({}),
|
|
67
|
+
locked = path.join(dir, 'locked');
|
|
68
|
+
fs.mkdirSync(locked, { mode: 0o500 });
|
|
69
|
+
fs.writeFileSync(file, JSON.stringify({ dataDir: path.join(dir, 'data'), dbPath: path.join(locked, 'thub.db') }));
|
|
70
|
+
try {
|
|
71
|
+
assert.throws(() => loadConfig(file), /locked is not writable by this user .*set dbPath \/ THUB_DB_PATH/);
|
|
72
|
+
}
|
|
73
|
+
finally {
|
|
74
|
+
fs.chmodSync(locked, 0o700);
|
|
75
|
+
}
|
|
76
|
+
});
|
package/test/tls.test.js
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* @file packages/coordinator/test/tls.test.js
|
|
3
|
+
* @description Tests: HTTPS served by the Coordinator itself — certificate checks, serving, renewal
|
|
4
|
+
*
|
|
5
|
+
* @author Andrian Yablonskyy
|
|
6
|
+
* @copyright Copyright (c) 2026 Andrian Yablonskyy. All rights reserved.
|
|
7
|
+
*
|
|
8
|
+
* This file is part of TestHub and is proprietary and confidential.
|
|
9
|
+
* Unauthorized copying, modification, distribution, or use of this file,
|
|
10
|
+
* via any medium, is strictly prohibited without prior written permission
|
|
11
|
+
* from AdSystem.PRO.
|
|
12
|
+
*/
|
|
13
|
+
|
|
14
|
+
'use strict';
|
|
15
|
+
|
|
16
|
+
const test = require('node:test'),
|
|
17
|
+
assert = require('node:assert/strict'),
|
|
18
|
+
fs = require('node:fs'),
|
|
19
|
+
os = require('node:os'),
|
|
20
|
+
path = require('node:path'),
|
|
21
|
+
https = require('node:https'),
|
|
22
|
+
{ spawnSync } = require('node:child_process'),
|
|
23
|
+
{ loadConfig } = require('../src/config'),
|
|
24
|
+
{ buildServices, createApp } = require('../src/server'),
|
|
25
|
+
{ readTlsOptions, createServer } = require('../src/tls');
|
|
26
|
+
|
|
27
|
+
const hasOpenssl = spawnSync('openssl', ['version']).status === 0;
|
|
28
|
+
|
|
29
|
+
// A self-signed certificate for 127.0.0.1 into `dir`, as <name>.crt / <name>.key.
|
|
30
|
+
function makeCert(dir, name){
|
|
31
|
+
const crt = path.join(dir, `${name}.crt`),
|
|
32
|
+
key = path.join(dir, `${name}.key`),
|
|
33
|
+
r = spawnSync('openssl', ['req', '-x509', '-newkey', 'rsa:2048', '-nodes', '-days', '2', '-subj', `/CN=${name}`,
|
|
34
|
+
'-keyout', key, '-out', crt], { encoding: 'utf8' });
|
|
35
|
+
assert.equal(r.status, 0, r.stderr);
|
|
36
|
+
return { crt, key };
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
// GET over HTTPS, trusting only `ca`; resolves { status, headers, subject }.
|
|
40
|
+
function get(port, urlPath, ca, headers = {}){
|
|
41
|
+
return new Promise((resolve, reject) => {
|
|
42
|
+
const req = https.request({ host: '127.0.0.1', port, path: urlPath, ca, headers, checkServerIdentity: () => undefined }, (res) => {
|
|
43
|
+
const subject = res.socket.getPeerCertificate().subject.CN;
|
|
44
|
+
res.resume();
|
|
45
|
+
res.on('end', () => resolve({ status: res.statusCode, headers: res.headers, subject }));
|
|
46
|
+
});
|
|
47
|
+
req.on('error', reject);
|
|
48
|
+
req.end();
|
|
49
|
+
});
|
|
50
|
+
}
|
|
51
|
+
|
|
52
|
+
test('tls config: none = plain HTTP; half a pair, an unreadable file or a mismatched key stop the start, saying why', { skip: !hasOpenssl }, () => {
|
|
53
|
+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'thub-tls-')),
|
|
54
|
+
a = makeCert(dir, 'a'),
|
|
55
|
+
b = makeCert(dir, 'b');
|
|
56
|
+
assert.equal(readTlsOptions({}), null);
|
|
57
|
+
assert.equal(readTlsOptions({ certFile: null, keyFile: null }), null);
|
|
58
|
+
assert.throws(() => readTlsOptions({ certFile: a.crt }), /set both certFile and keyFile/);
|
|
59
|
+
assert.throws(() => readTlsOptions({ certFile: a.crt, keyFile: path.join(dir, 'missing.key') }),
|
|
60
|
+
/tls.keyFile .*missing.key can't be read by this user \(ENOENT\)/);
|
|
61
|
+
assert.throws(() => readTlsOptions({ certFile: a.crt, keyFile: b.key }), /aren't a usable certificate and key/);
|
|
62
|
+
assert.ok(readTlsOptions({ certFile: a.crt, keyFile: a.key }).cert);
|
|
63
|
+
});
|
|
64
|
+
|
|
65
|
+
test('HTTPS: the dashboard and API over TLS, a Secure session cookie, a renewed certificate picked up on SIGHUP', { skip: !hasOpenssl }, async (t) => {
|
|
66
|
+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'thub-tls-')),
|
|
67
|
+
first = makeCert(dir, 'first'),
|
|
68
|
+
renewed = makeCert(dir, 'renewed'),
|
|
69
|
+
live = { crt: path.join(dir, 'live.crt'), key: path.join(dir, 'live.key') },
|
|
70
|
+
file = path.join(dir, 'coordinator.json');
|
|
71
|
+
fs.copyFileSync(first.crt, live.crt);
|
|
72
|
+
fs.copyFileSync(first.key, live.key);
|
|
73
|
+
fs.writeFileSync(file, JSON.stringify({
|
|
74
|
+
dataDir: path.join(dir, 'data'), updates: { checkIntervalMin: 0 }, publicUrl: 'https://thub.example.com',
|
|
75
|
+
tls: { certFile: live.crt, keyFile: live.key }
|
|
76
|
+
}));
|
|
77
|
+
const config = loadConfig(file),
|
|
78
|
+
services = buildServices(config),
|
|
79
|
+
logs = [],
|
|
80
|
+
{ server, scheme } = createServer(createApp(config, services), config, { log: { log: (m) => logs.push(m), error: (m) => logs.push(m) } });
|
|
81
|
+
assert.equal(scheme, 'https');
|
|
82
|
+
await new Promise((r) => server.listen(0, '127.0.0.1', r));
|
|
83
|
+
t.after(() => new Promise((r) => {
|
|
84
|
+
server.closeAllConnections?.();
|
|
85
|
+
server.close(r);
|
|
86
|
+
}));
|
|
87
|
+
const port = server.address().port,
|
|
88
|
+
login = await get(port, '/login', fs.readFileSync(first.crt));
|
|
89
|
+
assert.equal(login.status, 200);
|
|
90
|
+
assert.equal(login.subject, 'first');
|
|
91
|
+
|
|
92
|
+
services.adminUsers.create({ username: 'root', email: 'r@example.com', role: 'admin', password: 'pw' }, { by: 't' });
|
|
93
|
+
const signIn = await new Promise((resolve, reject) => {
|
|
94
|
+
const req = https.request({
|
|
95
|
+
host: '127.0.0.1', port, path: '/login', method: 'POST', ca: fs.readFileSync(first.crt), checkServerIdentity: () => undefined,
|
|
96
|
+
headers: { 'content-type': 'application/x-www-form-urlencoded' }
|
|
97
|
+
}, (res) => {
|
|
98
|
+
res.resume();
|
|
99
|
+
res.on('end', () => resolve(res));
|
|
100
|
+
});
|
|
101
|
+
req.on('error', reject);
|
|
102
|
+
req.end('username=root&password=pw');
|
|
103
|
+
});
|
|
104
|
+
assert.equal(signIn.statusCode, 302);
|
|
105
|
+
assert.match(String(signIn.headers['set-cookie']), /;\s*Secure/i); // the cookie only travels over HTTPS
|
|
106
|
+
|
|
107
|
+
// Renewed in place (cert-manager, certbot): picked up without a restart.
|
|
108
|
+
fs.copyFileSync(renewed.crt, live.crt);
|
|
109
|
+
fs.copyFileSync(renewed.key, live.key);
|
|
110
|
+
process.emit('SIGHUP');
|
|
111
|
+
const after = await get(port, '/login', fs.readFileSync(renewed.crt));
|
|
112
|
+
assert.equal(after.subject, 'renewed');
|
|
113
|
+
assert.ok(logs.some((l) => /certificate reloaded \(SIGHUP\)/.test(l)), logs.join('\n'));
|
|
114
|
+
|
|
115
|
+
// A broken renewal keeps the working certificate.
|
|
116
|
+
fs.writeFileSync(live.key, 'not a key');
|
|
117
|
+
process.emit('SIGHUP');
|
|
118
|
+
assert.equal((await get(port, '/login', fs.readFileSync(renewed.crt))).subject, 'renewed');
|
|
119
|
+
assert.ok(logs.some((l) => /keeping the current certificate/.test(l)));
|
|
120
|
+
});
|
|
121
|
+
|
|
122
|
+
test('tls settings from the environment (THUB_TLS_*) win over the file', () => {
|
|
123
|
+
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'thub-tls-')),
|
|
124
|
+
file = path.join(dir, 'coordinator.json'),
|
|
125
|
+
saved = { ...process.env };
|
|
126
|
+
fs.writeFileSync(file, JSON.stringify({ dataDir: path.join(dir, 'data'), tls: { certFile: '/file.crt', keyFile: '/file.key' } }));
|
|
127
|
+
Object.assign(process.env, { THUB_TLS_CERT: '/env.crt', THUB_TLS_KEY: '/env.key', THUB_TLS_CA: '/env-ca.crt', THUB_TLS_PASSPHRASE: 'pp' });
|
|
128
|
+
try {
|
|
129
|
+
assert.deepEqual(loadConfig(file).tls, { certFile: '/env.crt', keyFile: '/env.key', caFile: '/env-ca.crt', passphrase: 'pp' });
|
|
130
|
+
}
|
|
131
|
+
finally {
|
|
132
|
+
for (const k of ['THUB_TLS_CERT', 'THUB_TLS_KEY', 'THUB_TLS_CA', 'THUB_TLS_PASSPHRASE']){
|
|
133
|
+
if (saved[k] === undefined){
|
|
134
|
+
delete process.env[k];
|
|
135
|
+
}
|
|
136
|
+
else {
|
|
137
|
+
process.env[k] = saved[k];
|
|
138
|
+
}
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
});
|
package/test/updates.test.js
CHANGED
|
@@ -29,7 +29,7 @@ async function setup(){
|
|
|
29
29
|
clock = { t: Date.parse('2026-09-28T12:00:00Z') };
|
|
30
30
|
fs.writeFileSync(pathUnit, '');
|
|
31
31
|
const updates = createUpdatesService({
|
|
32
|
-
config: { dataDir, updates: {} },
|
|
32
|
+
config: { dataDir, updates: {}, selfUpdate: true }, // started with --self-update
|
|
33
33
|
pathUnit,
|
|
34
34
|
fetchLatest: async () => NEWER,
|
|
35
35
|
now: () => clock.t
|
|
@@ -104,3 +104,40 @@ test('on startup, a finished update\'s status is cleared; a failure is kept for
|
|
|
104
104
|
make().start();
|
|
105
105
|
assert.equal(fs.existsSync(statusFile), true);
|
|
106
106
|
});
|
|
107
|
+
|
|
108
|
+
test('self-update is off unless the Coordinator was started with --self-update: no request, no button', async (t) => {
|
|
109
|
+
const dataDir = fs.mkdtempSync(path.join(os.tmpdir(), 'thub-updates-')),
|
|
110
|
+
updates = createUpdatesService({ config: { dataDir, updates: {} }, pathUnit: path.join(dataDir, 'fake.path'), fetchLatest: async () => '99.0.0' });
|
|
111
|
+
fs.writeFileSync(path.join(dataDir, 'fake.path'), '');
|
|
112
|
+
await updates.checkNow();
|
|
113
|
+
assert.equal(updates.status().selfUpdate, false);
|
|
114
|
+
assert.ok(updates.status().coordinatorUpdate); // the notice still shows a newer version
|
|
115
|
+
assert.throws(() => updates.requestCoordinatorUpdate('admin'), /Self-update is off: start the Coordinator with --self-update/);
|
|
116
|
+
assert.ok(!fs.existsSync(path.join(dataDir, 'update-request.json')));
|
|
117
|
+
|
|
118
|
+
// The dashboard: a notice, no "Update app" button.
|
|
119
|
+
const { loadConfig } = require('../src/config'),
|
|
120
|
+
{ buildServices, createApp } = require('../src/server'),
|
|
121
|
+
file = path.join(dataDir, 'coordinator.json');
|
|
122
|
+
fs.writeFileSync(file, JSON.stringify({ dataDir: path.join(dataDir, 'data'), updates: { checkIntervalMin: 0 } }));
|
|
123
|
+
for (const selfUpdate of [false, true]){
|
|
124
|
+
const config = { ...loadConfig(file), selfUpdate },
|
|
125
|
+
services = buildServices(config);
|
|
126
|
+
services.updates.checkNow = async () => {};
|
|
127
|
+
Object.assign(services.updates, { status: () => ({ latest: {}, coordinatorVersion: '1.0.0', coordinatorUpdate: '99.0.0', selfUpdate }) });
|
|
128
|
+
services.adminUsers.create({ username: `root${selfUpdate}`, email: `r${selfUpdate}@example.com`, role: 'admin', password: 'pw' }, { by: 't' });
|
|
129
|
+
const server = createApp(config, services).listen(0, '127.0.0.1');
|
|
130
|
+
await new Promise((r) => server.on('listening', r));
|
|
131
|
+
t.after(() => new Promise((r) => {
|
|
132
|
+
server.closeAllConnections?.();
|
|
133
|
+
server.close(r);
|
|
134
|
+
}));
|
|
135
|
+
const base = `http://127.0.0.1:${server.address().port}`,
|
|
136
|
+
cookie = (await fetch(`${base}/login`, {
|
|
137
|
+
method: 'POST', body: new URLSearchParams({ username: `root${selfUpdate}`, password: 'pw' }), redirect: 'manual'
|
|
138
|
+
})).headers.get('set-cookie').split(';')[0],
|
|
139
|
+
html = await (await fetch(`${base}/runners`, { headers: { cookie } })).text();
|
|
140
|
+
assert.equal(/action="\/updates\/coordinator"/.test(html), selfUpdate, `Update app button with selfUpdate=${selfUpdate}`);
|
|
141
|
+
assert.match(html, /v99\.0\.0 available|Update app to 99\.0\.0/);
|
|
142
|
+
}
|
|
143
|
+
});
|
|
@@ -25,7 +25,7 @@
|
|
|
25
25
|
{ title: 'What to run', rows: [
|
|
26
26
|
['--command <string>', 'Required. Run with sh -c in the work directory. Exit code 0 = PASSED. It clones repositories and runs containers itself (see Using git / Using Docker).', "--command './ci/test.sh'"],
|
|
27
27
|
['--arg <value>', 'Argument for the command, as "$@" (repeatable).', '--arg --junit --arg -v'],
|
|
28
|
-
['--suite <name>', 'Passed as
|
|
28
|
+
['--suite <name>', 'Passed as JOB_SUITE (default: default).', '--suite smoke'],
|
|
29
29
|
['--download-file <url>', 'File downloaded before the command (repeatable, http(s), no credentials). THUB_DOWNLOAD_1…', '--download-file https://…/app.bin'],
|
|
30
30
|
['--env <vars>', 'NAME=value[,NAME=value] for the command (repeatable): how data and secrets reach the job (git tokens, registry passwords). --env NAME takes the value from your shell. Masked, dropped at job end.', '--env TARGET=staging --env API_TOKEN']
|
|
31
31
|
] },
|
|
@@ -70,12 +70,12 @@
|
|
|
70
70
|
thub run --type hw --label board:nucleo-f401re \
|
|
71
71
|
--download-file https://artifactory.example.com/fw-local/app/1.4.0-42/app.bin \
|
|
72
72
|
--env GH_TOKEN \
|
|
73
|
-
--command 'git clone --depth 1 --branch v1.4.0 "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git"
|
|
74
|
-
st-flash --
|
|
73
|
+
--command 'git clone --depth 1 --branch v1.4.0 "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git" src && cd src &&
|
|
74
|
+
st-flash --reset write "$THUB_DOWNLOAD_1" 0x08000000 && ./ci/test.sh "$@"' \
|
|
75
75
|
--arg --junit --timeout 45m --wait
|
|
76
76
|
+code('Reproduce a failure on the exact bench (the job carries your username)').
|
|
77
77
|
thub run --type hw --label board:nucleo-f401re --client lab-hw-01 \
|
|
78
|
-
--download-file "$IMAGE_URL" --env GH_TOKEN --command 'git clone --depth 1 "https://x-access-token:$GH_TOKEN@$TESTS_REPO"
|
|
78
|
+
--download-file "$IMAGE_URL" --env GH_TOKEN --command 'git clone --depth 1 "https://x-access-token:$GH_TOKEN@$TESTS_REPO" src && cd src && ./ci/test.sh'
|
|
79
79
|
# Ctrl-C detaches; the job keeps running. Re-attach:
|
|
80
80
|
thub status M-00126
|
|
81
81
|
+code('Verify a download before using it').
|
package/views/help/_ci.pug
CHANGED
|
@@ -34,8 +34,8 @@
|
|
|
34
34
|
--type hw --label board:nucleo-f401re \
|
|
35
35
|
--download-file "${{ needs.build.outputs.image_url }}" \
|
|
36
36
|
--env GH_TOKEN="${{ secrets.TESTS_READ_TOKEN }}",SHA="$GITHUB_SHA",REPO="$GITHUB_REPOSITORY" \
|
|
37
|
-
--command 'git init -q
|
|
38
|
-
st-flash --
|
|
37
|
+
--command 'git init -q src && cd src && git fetch -q --depth 1 "https://x-access-token:$GH_TOKEN@github.com/$REPO.git" "$SHA" && git checkout -q FETCH_HEAD &&
|
|
38
|
+
st-flash --reset write "$THUB_DOWNLOAD_1" 0x08000000 && ./ci/hw-tests.sh' \
|
|
39
39
|
--suite smoke --timeout 30m --wait \
|
|
40
40
|
--meta runUrl="$GITHUB_SERVER_URL/$GITHUB_REPOSITORY/actions/runs/$GITHUB_RUN_ID"
|
|
41
41
|
ul.small.mb-0
|
|
@@ -53,8 +53,8 @@
|
|
|
53
53
|
thub run --type hw --label board:nucleo-f401re \
|
|
54
54
|
--download-file https://artifactory.example.com/fw-local/app/1.4.0-42/app.bin \
|
|
55
55
|
--env GH_TOKEN \
|
|
56
|
-
--command 'git clone --depth 1 --branch v1.4.0 "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git"
|
|
57
|
-
st-flash --
|
|
56
|
+
--command 'git clone --depth 1 --branch v1.4.0 "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git" src && cd src &&
|
|
57
|
+
st-flash --reset write "$THUB_DOWNLOAD_1" 0x08000000 && ./ci/test.sh' \
|
|
58
58
|
--wait
|
|
59
59
|
|
|
60
60
|
+note('warning').
|
|
@@ -74,10 +74,9 @@
|
|
|
74
74
|
+code('A typical SW job').
|
|
75
75
|
thub run --type sw \
|
|
76
76
|
--env GH_TOKEN --env DOCKER_PASSWORD \
|
|
77
|
-
--command 'git clone --depth 1 "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git"
|
|
78
|
-
export DOCKER_CONFIG="$THUB_WORK_DIR/.docker" &&
|
|
77
|
+
--command 'git clone --depth 1 "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git" src && cd src &&
|
|
79
78
|
echo "$DOCKER_PASSWORD" | docker login registry.lab.local:5000 -u ci --password-stdin &&
|
|
80
|
-
docker run --rm -v "$
|
|
79
|
+
docker run --rm --user "$(id -u):$(id -g)" -v "$THUB_WORK_DIR/src:/work" -w /work registry.lab.local:5000/dut-emulator:2026.08 make test' --wait
|
|
81
80
|
|
|
82
81
|
h3.h6 Checklist for any Client machine
|
|
83
82
|
ul.small.mb-0
|
|
@@ -106,8 +106,8 @@
|
|
|
106
106
|
change it hasn't applied yet. #[strong USB devices → Import to config] fills them from the host's #[code lsusb].
|
|
107
107
|
The card's #[strong Export] and #[strong Import] icons download a Client's config file (never its
|
|
108
108
|
#[code joinKey]) and apply one to it. An older HW Client takes only the file's #[code hw-devices].
|
|
109
|
-
There's no ST-Link serial field: the job's #[code --command] picks the probe
|
|
110
|
-
|
|
109
|
+
There's no ST-Link serial field: the job's #[code --command] picks the probe (#[code st-flash] alone uses the
|
|
110
|
+
only one; with several, #[code st-flash --serial] with a serial from #[code st-info --probe]).
|
|
111
111
|
|
|
112
112
|
h3.h6 Several DUT slots on one machine
|
|
113
113
|
+code('One instance per slot').
|
|
@@ -12,7 +12,8 @@
|
|
|
12
12
|
#[code sudo npm i -g] does the whole setup for the user who ran #[code sudo]. It creates
|
|
13
13
|
#[code ~/.config/thub/coordinator.json] (with a random #[code sessionSecret]) and
|
|
14
14
|
#[code ~/var/lib/thub] (database, avatars). It installs, enables and starts
|
|
15
|
-
#[code thub-coordinator.service], plus the root helper behind the navbar's #[strong Update app] button
|
|
15
|
+
#[code thub-coordinator.service], plus the root helper behind the navbar's #[strong Update app] button
|
|
16
|
+
(off unless the Coordinator runs with #[code --self-update]; see Updates below).
|
|
16
17
|
The unit uses #[code ProtectSystem=strict] with #[code PrivateTmp=yes], so the data directory and a
|
|
17
18
|
private #[code /tmp] (SQLite's temp files) are the only places the service can write.
|
|
18
19
|
|
|
@@ -58,6 +59,12 @@
|
|
|
58
59
|
tr
|
|
59
60
|
td: code session.secureCookie
|
|
60
61
|
td #[code "auto"] (default): the session cookie is Secure (HTTPS-only) whenever #[code publicUrl] is https://. #[code true]/#[code false] force it. Sessions are stored in the database and survive restarts.
|
|
62
|
+
tr
|
|
63
|
+
td: code dbPath
|
|
64
|
+
td The SQLite database file. Default: #[code <dataDir>/thub.db]; a relative path is taken from #[code dataDir]. Its directory must be writable (SQLite's #[code -wal]/#[code -shm] files sit next to it). To move a database, stop the Coordinator and move #[code thub.db] with its #[code -wal]/#[code -shm] files.
|
|
65
|
+
tr
|
|
66
|
+
td: code tls.*
|
|
67
|
+
td #[code certFile] / #[code keyFile] (PEM): HTTPS served by the Coordinator itself, without a reverse proxy. #[code caFile]: intermediates, if not in the cert file; #[code passphrase]: for an encrypted key. Unset (default): plain HTTP. See #[strong HTTPS without a proxy] below.
|
|
61
68
|
tr
|
|
62
69
|
td: code clientJoinKey
|
|
63
70
|
td The shared secret Clients self-register with. Unset or #[code null] turns registration off (#[code 503]).
|
|
@@ -86,7 +93,8 @@
|
|
|
86
93
|
p.small.
|
|
87
94
|
Resolution order: #[code THUB_COORDINATOR_CONFIG] (a path) → #[code ~/.config/thub/coordinator.json] → the bundled
|
|
88
95
|
default. These environment variables override the file: #[code THUB_LISTEN], #[code THUB_PUBLIC_URL],
|
|
89
|
-
#[code THUB_DATA_DIR], #[code THUB_SESSION_SECRET] and #[code
|
|
96
|
+
#[code THUB_DATA_DIR], #[code THUB_DB_PATH], #[code THUB_SESSION_SECRET], #[code THUB_CLIENT_JOIN_KEY] and #[code THUB_TLS_CERT] /
|
|
97
|
+
#[code THUB_TLS_KEY] / #[code THUB_TLS_CA] / #[code THUB_TLS_PASSPHRASE].
|
|
90
98
|
|
|
91
99
|
h3.h6 3. HTTPS reverse proxy
|
|
92
100
|
p.small.
|
|
@@ -112,6 +120,23 @@
|
|
|
112
120
|
client_max_body_size 10m; # log batches, avatars
|
|
113
121
|
}
|
|
114
122
|
|
|
123
|
+
h3.h6 Or: HTTPS without a proxy
|
|
124
|
+
p.small.
|
|
125
|
+
The Coordinator can serve HTTPS itself, e.g. in Kubernetes or on a single host. Give it the certificate files
|
|
126
|
+
(readable by its user) and an #[code https://] #[code publicUrl]:
|
|
127
|
+
+code('coordinator.json').
|
|
128
|
+
{
|
|
129
|
+
"listen": "0.0.0.0:8443",
|
|
130
|
+
"publicUrl": "https://thub.example.com:8443",
|
|
131
|
+
"tls": { "certFile": "/etc/thub/tls/tls.crt", "keyFile": "/etc/thub/tls/tls.key" }
|
|
132
|
+
}
|
|
133
|
+
ul.small
|
|
134
|
+
li A bad path, half a pair or a key that doesn't match the certificate stops the start with the reason.
|
|
135
|
+
li Renewed files are picked up within a minute, or at once on #[code sudo systemctl kill -s HUP thub-coordinator]; a broken renewal keeps the working certificate.
|
|
136
|
+
li Port 443 under systemd needs #[code AmbientCapabilities=CAP_NET_BIND_SERVICE] (a drop-in), or use 8443. certbot's private key is root-only: copy it for the service user in a #[code --deploy-hook].
|
|
137
|
+
li Self-signed or private CA: start Clients and Agents with #[code NODE_EXTRA_CA_CERTS=/path/to/ca.pem] (a Client: #[code sudo systemctl edit thub-client@<instance>] → #[code Environment=NODE_EXTRA_CA_CERTS=…]).
|
|
138
|
+
li Kubernetes: mount the TLS secret and set #[code THUB_TLS_CERT] / #[code THUB_TLS_KEY] to its #[code tls.crt] / #[code tls.key].
|
|
139
|
+
|
|
115
140
|
h3.h6 4. Users, agents and the join key
|
|
116
141
|
p.small.
|
|
117
142
|
#[code thub-admin] works directly on the database. Run it on the Coordinator host as the service user.
|
|
@@ -150,7 +175,12 @@
|
|
|
150
175
|
|
|
151
176
|
h3.h6 Updates
|
|
152
177
|
p.small.mb-0.
|
|
153
|
-
The Coordinator checks npm for new Coordinator, Agent and Client versions. Admins update
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
178
|
+
The Coordinator checks npm for new Coordinator, Agent and Client versions. Admins update Agents and Clients
|
|
179
|
+
from the #[a(href="/admin/agents") Agents] and #[a(href="/runners") Runners] pages; Clients install an update
|
|
180
|
+
only between jobs. Updating the #[strong Coordinator] from the dashboard (#[strong Update app] in the navbar)
|
|
181
|
+
is #[strong off by default]: start it with #[code --self-update] to turn it on. Under systemd:
|
|
182
|
+
#[code sudo sed -i 's|server.js$|server.js --self-update|' /etc/systemd/system/thub-coordinator.service],
|
|
183
|
+
then #[code sudo systemctl daemon-reload && sudo systemctl restart thub-coordinator] (a re-install keeps it).
|
|
184
|
+
Without it, a #[strong vX.Y.Z available] badge shows; update by hand — #[code thub-admin self-update],
|
|
185
|
+
#[code sudo npm i -g …], or a new container image. Manual alternatives for the others:
|
|
186
|
+
#[code thub self-update] and #[code thub-client self-update].
|
package/views/help/_docker.pug
CHANGED
|
@@ -13,11 +13,10 @@
|
|
|
13
13
|
thub run --type sw \
|
|
14
14
|
--env DOCKER_REGISTRY=registry.lab.local:5000,DOCKER_USER=ci --env DOCKER_PASSWORD \
|
|
15
15
|
--env TARGET=staging \
|
|
16
|
-
--command '
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
-
|
|
20
|
-
-e TARGET -e THUB_SUITE -e THUB_JOB_ID \
|
|
16
|
+
--command 'echo "$DOCKER_PASSWORD" | docker login "$DOCKER_REGISTRY" -u "$DOCKER_USER" --password-stdin &&
|
|
17
|
+
docker run --rm --user "$(id -u):$(id -g)" \
|
|
18
|
+
-v "$THUB_WORK_DIR/src:/work" -w /work \
|
|
19
|
+
-e TARGET -e JOB_SUITE -e THUB_JOB_ID \
|
|
21
20
|
"$DOCKER_REGISTRY/team/test-runner:1.4" ./run-tests.sh --junit results/junit.xml' \
|
|
22
21
|
--wait
|
|
23
22
|
table.table.table-sm.small
|
|
@@ -30,8 +29,8 @@
|
|
|
30
29
|
td: code --rm
|
|
31
30
|
td Removes the container when the tests end.
|
|
32
31
|
tr
|
|
33
|
-
td: code -v "$THUB_WORK_DIR:/work" -w /work
|
|
34
|
-
td
|
|
32
|
+
td: code -v "$THUB_WORK_DIR/src:/work" -w /work
|
|
33
|
+
td Only the clone (#[code src/]), not the job directory, which holds the registry login in #[code .docker/]. The Client reads JUnit XML from #[code src/results/] or #[code src/artifacts/] for the test counts.
|
|
35
34
|
tr
|
|
36
35
|
td: code -e NAME
|
|
37
36
|
td Passes a job variable (#[code --env], #[code THUB_*], #[code JOB_*]) into the container.
|
|
@@ -40,7 +39,37 @@
|
|
|
40
39
|
td Files written to #[code /work] stay owned by the Client user, so the workspace can be cleaned up.
|
|
41
40
|
tr
|
|
42
41
|
td: code --device /dev/thub/dut1-uart
|
|
43
|
-
td HW: gives the container the DUT's UART (#[code
|
|
42
|
+
td HW: gives the container the DUT's UART (#[code /dev/thub/dut1-uart]). Use #[code --privileged] only if you must.
|
|
43
|
+
|
|
44
|
+
h3.h6 Clone and test inside a container, with a deploy key and a registry login from the job
|
|
45
|
+
p.small.
|
|
46
|
+
Everything comes from the Agent as #[code --env]: the registry and its credentials, and the private key the
|
|
47
|
+
container clones with. Nothing is set up on the Client host beforehand.
|
|
48
|
+
+code('Reference example').
|
|
49
|
+
# In CI, from its secret store — never typed on the command line:
|
|
50
|
+
export THUB_KEY=… # a CI token (dashboard → CI tokens)
|
|
51
|
+
export DOCKER_PASSWORD=… # the registry password
|
|
52
|
+
export GIT_KEY="$(cat ~/.ssh/thub_deploy)" # a private deploy key with read access to the repository
|
|
53
|
+
|
|
54
|
+
thub run --type sw \
|
|
55
|
+
--env DOCKER_REGISTRY=registry.lab:5000,DOCKER_USERNAME=ci-reader \
|
|
56
|
+
--env DOCKER_PASSWORD --env GIT_KEY \
|
|
57
|
+
--command 'echo "$DOCKER_PASSWORD" | docker login "$DOCKER_REGISTRY" --username "$DOCKER_USERNAME" --password-stdin &&
|
|
58
|
+
mkdir -p "$THUB_WORK_DIR/src" &&
|
|
59
|
+
docker run --rm -e GIT_KEY -e HOME=/tmp --user "$(id -u):$(id -g)" \
|
|
60
|
+
-v "$THUB_WORK_DIR/src:/work" -w /work --entrypoint sh alpine/git -c "
|
|
61
|
+
eval \$(ssh-agent -s) > /dev/null &&
|
|
62
|
+
printf \"%s\n\" \"\$GIT_KEY\" | ssh-add - &&
|
|
63
|
+
GIT_SSH_COMMAND=\"ssh -o StrictHostKeyChecking=accept-new\" git clone --depth 1 git@bitbucket.org:yourorg/web-ui-tests.git . &&
|
|
64
|
+
./run-tests.sh"' \
|
|
65
|
+
--wait
|
|
66
|
+
ul.small
|
|
67
|
+
li Plain values go as #[code --env NAME=value]. Secrets go as #[code --env NAME] alone: the Agent takes the value from its own environment, so it never appears on the command line, in shell history or in CI logs. A multi-line value, such as the key, arrives intact.
|
|
68
|
+
li The Client sets #[code DOCKER_CONFIG=$THUB_WORK_DIR/.docker]: the registry login lands in the job's directory, deleted with the job. Only #[code src/] is mounted into the container, so the login isn't visible in it.
|
|
69
|
+
li #[code -e GIT_KEY] hands the key to the container. #[code --entrypoint sh] is needed because #[code alpine/git]'s own entrypoint is #[code git]. #[code --user "$(id -u):$(id -g)"] (with #[code HOME=/tmp]) keeps the clone owned by the Client user, so it can be deleted with the job; #[code mkdir -p] first so Docker doesn't create #[code src/] as root.
|
|
70
|
+
li In the inner script, #[code \$] is escaped, so the #[strong container] expands the variables, not the Client's shell. #[code ssh-agent] holds the key in memory: it's never written to a file. #[code accept-new] trusts the git server's host key on first use.
|
|
71
|
+
li #[code alpine/git] comes from Docker Hub. The #[code docker login] is for private images such as #[code $DOCKER_REGISTRY/team/test-runner:1.4].
|
|
72
|
+
li Never #[code echo] a secret: the command's output is the job log, which is stored and visible.
|
|
44
73
|
|
|
45
74
|
h3.h6 An emulator as the DUT
|
|
46
75
|
p.small.
|
|
@@ -49,8 +78,7 @@
|
|
|
49
78
|
thub run --type sw \
|
|
50
79
|
--download-file https://artifactory.example.com/fw-local/app/1.4.0-42/app.elf \
|
|
51
80
|
--env DOCKER_REGISTRY=registry.lab.local:5000,DOCKER_USER=ci --env DOCKER_PASSWORD \
|
|
52
|
-
--command '
|
|
53
|
-
echo "$DOCKER_PASSWORD" | docker login "$DOCKER_REGISTRY" -u "$DOCKER_USER" --password-stdin &&
|
|
81
|
+
--command 'echo "$DOCKER_PASSWORD" | docker login "$DOCKER_REGISTRY" -u "$DOCKER_USER" --password-stdin &&
|
|
54
82
|
dut="thub-$THUB_JOB_ID" && trap "docker rm -f $dut >/dev/null" EXIT &&
|
|
55
83
|
docker run -d --name "$dut" -p 127.0.0.1:5555:5555 -v "$THUB_DOWNLOADS_DIR:/downloads:ro" \
|
|
56
84
|
"$DOCKER_REGISTRY/dut-emulator:2026.08" &&
|
package/views/help/_env.pug
CHANGED
|
@@ -10,14 +10,11 @@
|
|
|
10
10
|
const thubVars = [
|
|
11
11
|
['THUB_JOB_ID', 'The job id, e.g. M-00125'],
|
|
12
12
|
['THUB_ARTIFACTS_FILE', 'Where the command may list the artifacts it published elsewhere: a JSON array of {"name", "size", "link", "timestamp"}, shown on the job page'],
|
|
13
|
-
['THUB_WORK_DIR', 'The directory the command
|
|
14
|
-
['
|
|
13
|
+
['THUB_WORK_DIR', 'The job\'s directory (<client workDir>/<jobId>), where the command starts. Holds downloads/, the artifacts list and .docker/ (the job\'s DOCKER_CONFIG); clone into a folder of your own, e.g. src/. JUnit XML in results/ or artifacts/ gives the test counts; deleted when the job ends'],
|
|
14
|
+
['DOCKER_CONFIG', '$THUB_WORK_DIR/.docker — so a docker login stays with the job and is deleted with it (unless --env sets DOCKER_CONFIG)'],
|
|
15
15
|
['THUB_DOWNLOADS_DIR', 'Where the --download-file files are'],
|
|
16
16
|
['THUB_DOWNLOADS / THUB_DOWNLOAD_<n>', 'Local paths of the downloads: all, one per line / each one'],
|
|
17
17
|
['THUB_META_<KEY>', '--meta values; camelCase keys become SNAKE_CASE (ciJobId → THUB_META_CI_JOB_ID)'],
|
|
18
|
-
['THUB_DUT_STLINK / _<n>', 'HW: ST-Link serials (device path if the serial couldn\'t be read); unsuffixed = the first'],
|
|
19
|
-
['THUB_DUT_UART / _<n>', 'HW: UART device paths (/dev/thub/dut<N>-uart); unsuffixed = the first'],
|
|
20
|
-
['THUB_DUT_USB / _<n>', 'HW: DUT USB device paths']
|
|
21
18
|
]
|
|
22
19
|
.table-responsive
|
|
23
20
|
table.table.table-sm.small
|
|
@@ -80,7 +77,7 @@
|
|
|
80
77
|
+code('ci/test.sh (shell)').
|
|
81
78
|
#!/bin/sh
|
|
82
79
|
set -eu
|
|
83
|
-
echo "Job $THUB_JOB_ID on $(hostname), suite ${
|
|
80
|
+
echo "Job $THUB_JOB_ID on $(hostname), suite ${JOB_SUITE}"
|
|
84
81
|
|
|
85
82
|
# optional parameters with defaults
|
|
86
83
|
TARGET="${TARGET:-staging}"
|
|
@@ -90,21 +87,21 @@
|
|
|
90
87
|
printf '%s\n' "$THUB_DOWNLOADS" | while read -r f; do echo "downloaded: $f"; done
|
|
91
88
|
|
|
92
89
|
# HW: flash through the first ST-Link, talk to the second UART
|
|
93
|
-
st-flash --
|
|
94
|
-
python3 -m pytest tests/ --uart "
|
|
90
|
+
st-flash --reset write "$THUB_DOWNLOAD_1" 0x08000000
|
|
91
|
+
python3 -m pytest tests/ --uart "/dev/thub/dut2-uart" --junitxml=results/junit.xml
|
|
95
92
|
|
|
96
93
|
# CI metadata passed with --meta ciJobId=… / --meta sha=…
|
|
97
94
|
echo "CI run ${THUB_META_CI_JOB_ID:-local}, sha ${THUB_META_SHA:-unknown}"
|
|
98
95
|
+code('conftest.py (Python)').
|
|
99
|
-
import os
|
|
96
|
+
import glob, os
|
|
100
97
|
|
|
101
|
-
UARTS =
|
|
98
|
+
UARTS = sorted(glob.glob("/dev/thub/dut*-uart")) # this Client's UARTs, by udev path
|
|
102
99
|
FIRMWARE = os.environ.get("THUB_DOWNLOAD_1")
|
|
103
|
-
RESULTS =
|
|
100
|
+
RESULTS = "results" # in the clone (src/results), where the tests run
|
|
104
101
|
API_TOKEN = os.environ["API_TOKEN"] # from --env API_TOKEN
|
|
105
102
|
+code('Into a container: -e NAME copies a variable from the job').
|
|
106
|
-
docker run --rm -v "$THUB_WORK_DIR:/work" -w /work \
|
|
107
|
-
-e THUB_JOB_ID -e
|
|
103
|
+
docker run --rm --user "$(id -u):$(id -g)" -v "$THUB_WORK_DIR/src:/work" -w /work \
|
|
104
|
+
-e THUB_JOB_ID -e JOB_SUITE -e TARGET -e API_TOKEN \
|
|
108
105
|
python:3.14 ./run-tests.sh
|
|
109
106
|
+code('Submitting the job that feeds these').
|
|
110
107
|
export API_TOKEN=…
|
|
@@ -113,7 +110,7 @@
|
|
|
113
110
|
--env TARGET=production,BRANCH=main --env API_TOKEN --env GH_TOKEN \
|
|
114
111
|
--meta ciJobId="$GITHUB_RUN_ID" --meta sha="$GITHUB_SHA" \
|
|
115
112
|
--suite smoke \
|
|
116
|
-
--command 'git clone --depth 1 --branch "$BRANCH" "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git"
|
|
113
|
+
--command 'git clone --depth 1 --branch "$BRANCH" "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git" src && cd src &&
|
|
117
114
|
./ci/test.sh' --wait
|
|
118
115
|
p.small.text-body-secondary.mb-0.
|
|
119
116
|
Tip: #[code thub run … --dry-run] lists every variable the command would get on the chosen Client, with #[code --env] values as #[code ***].
|
package/views/help/_git.pug
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
export GH_TOKEN=… # GitHub: a fine-grained PAT with read access; GitLab: oauth2 token; Bitbucket: x-token-auth
|
|
12
12
|
thub run --type sw \
|
|
13
13
|
--env GH_TOKEN \
|
|
14
|
-
--command 'git clone --depth 1 --branch main "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git"
|
|
14
|
+
--command 'git clone --depth 1 --branch main "https://x-access-token:$GH_TOKEN@github.com/yourorg/firmware-tests.git" src && cd src &&
|
|
15
15
|
./ci/test.sh' \
|
|
16
16
|
--wait
|
|
17
17
|
p.small.
|
|
@@ -23,19 +23,23 @@
|
|
|
23
23
|
export DEPLOY_KEY_B64="$(base64 < ~/.ssh/thub_deploy | tr -d '\n')"
|
|
24
24
|
thub run --type hw --label board:nucleo-f401re \
|
|
25
25
|
--env DEPLOY_KEY_B64 \
|
|
26
|
-
--command 'umask 077 && echo "$DEPLOY_KEY_B64" | base64 -d > "$THUB_WORK_DIR
|
|
27
|
-
GIT_SSH_COMMAND="ssh -i $THUB_WORK_DIR
|
|
28
|
-
git clone --depth 1 git@github.com:yourorg/firmware-tests.git
|
|
26
|
+
--command 'umask 077 && echo "$DEPLOY_KEY_B64" | base64 -d > "$THUB_WORK_DIR/key" &&
|
|
27
|
+
GIT_SSH_COMMAND="ssh -i $THUB_WORK_DIR/key -o IdentitiesOnly=yes -o StrictHostKeyChecking=accept-new" \
|
|
28
|
+
git clone --depth 1 git@github.com:yourorg/firmware-tests.git src && cd src &&
|
|
29
29
|
./ci/test.sh' \
|
|
30
30
|
--wait
|
|
31
31
|
p.small.
|
|
32
32
|
The job directory, key included, is deleted when the job ends. Add the public key as a read-only deploy key: GitHub,
|
|
33
33
|
Repository → Settings → Deploy keys; GitLab, Settings → Repository → Deploy keys; Bitbucket, Repository settings → Access keys.
|
|
34
34
|
|
|
35
|
+
p.small.
|
|
36
|
+
To clone #[em inside a container] with a key passed the same way (held by #[code ssh-agent], never written to disk), see
|
|
37
|
+
#[a(href="#docker") Using Docker → Clone and test inside a container].
|
|
38
|
+
|
|
35
39
|
h3.h6 Branch, tag, commit, submodules, LFS
|
|
36
40
|
+code('All in the command').
|
|
37
|
-
git clone --depth 1 --branch v1.4.0 "$REPO"
|
|
38
|
-
git init -q
|
|
41
|
+
git clone --depth 1 --branch v1.4.0 "$REPO" src # a branch or tag
|
|
42
|
+
git init -q src && cd src && git remote add origin "$REPO" && git fetch -q --depth 1 origin a1b2c3d && git checkout -q FETCH_HEAD # a commit
|
|
39
43
|
git submodule update --init --recursive --depth 1 && git lfs pull # submodules / LFS
|
|
40
44
|
p.small.
|
|
41
45
|
Pass the repository and ref as #[code --env REPO=…,REF=…] (or #[code --meta]) so the same command works for every branch.
|
|
@@ -0,0 +1,169 @@
|
|
|
1
|
+
+section('kubernetes', 'Running in Kubernetes', 'boxes')
|
|
2
|
+
p.
|
|
3
|
+
The Coordinator runs as one container: the published package in a Node.js image, its SQLite database on a
|
|
4
|
+
persistent volume, TLS at the ingress. Clients and Agents connect from outside at #[code publicUrl].
|
|
5
|
+
Self-update stays off: the image starts the Coordinator without #[code --self-update], and you update it by
|
|
6
|
+
deploying a new image tag.
|
|
7
|
+
|
|
8
|
+
h3.h6 1. Build the image
|
|
9
|
+
+code('Dockerfile').
|
|
10
|
+
# syntax=docker/dockerfile:1
|
|
11
|
+
# docker build --build-arg THUB_VERSION=<version> -t registry.example.com/thub-coordinator:<version> .
|
|
12
|
+
ARG NODE_VERSION=22
|
|
13
|
+
|
|
14
|
+
FROM node:${NODE_VERSION}-bookworm-slim AS build
|
|
15
|
+
ARG THUB_VERSION
|
|
16
|
+
# Only needed if better-sqlite3 has no prebuilt binary for the platform.
|
|
17
|
+
RUN apt-get update && apt-get install -y --no-install-recommends python3 make g++ ca-certificates \
|
|
18
|
+
&& rm -rf /var/lib/apt/lists/*
|
|
19
|
+
WORKDIR /app
|
|
20
|
+
# A local (not global) install: the package's systemd and ~/.config setup is skipped.
|
|
21
|
+
RUN npm pack "@andrian.yablonskyy/thub-coordinator@${THUB_VERSION}" \
|
|
22
|
+
&& tar xzf andrian.yablonskyy-thub-coordinator-*.tgz --strip-components=1 && rm -f *.tgz \
|
|
23
|
+
&& npm install --omit=dev --no-audit --no-fund && npm cache clean --force \
|
|
24
|
+
&& rm -rf test systemd
|
|
25
|
+
|
|
26
|
+
FROM node:${NODE_VERSION}-bookworm-slim
|
|
27
|
+
# tini as PID 1: forwards SIGTERM and reaps zombies.
|
|
28
|
+
RUN apt-get update && apt-get install -y --no-install-recommends tini && rm -rf /var/lib/apt/lists/*
|
|
29
|
+
COPY --from=build /app /app
|
|
30
|
+
RUN ln -s /app/bin/thub-admin.js /usr/local/bin/thub-admin \
|
|
31
|
+
&& mkdir -p /var/lib/thub && chown node:node /var/lib/thub
|
|
32
|
+
ENV NODE_ENV=production THUB_LISTEN=0.0.0.0:8080 THUB_DATA_DIR=/var/lib/thub
|
|
33
|
+
USER 1000:1000
|
|
34
|
+
WORKDIR /var/lib/thub
|
|
35
|
+
EXPOSE 8080
|
|
36
|
+
ENTRYPOINT ["/usr/bin/tini", "--"]
|
|
37
|
+
# No --self-update: a new image is how this Coordinator is updated.
|
|
38
|
+
CMD ["node", "/app/src/server.js"]
|
|
39
|
+
+code('Build and push').
|
|
40
|
+
docker build --build-arg THUB_VERSION=1.1.15 -t registry.example.com/thub-coordinator:1.1.15 .
|
|
41
|
+
docker push registry.example.com/thub-coordinator:1.1.15
|
|
42
|
+
|
|
43
|
+
h3.h6 2. Create the Secret
|
|
44
|
+
+code('Secrets as environment variables (they win over the config file and Settings)').
|
|
45
|
+
kubectl create namespace thub
|
|
46
|
+
kubectl -n thub create secret generic thub-coordinator-secrets \
|
|
47
|
+
--from-literal=THUB_SESSION_SECRET="$(openssl rand -hex 32)" \
|
|
48
|
+
--from-literal=THUB_CLIENT_JOIN_KEY="$(openssl rand -hex 24)" \
|
|
49
|
+
--from-literal=THUB_BOOTSTRAP_ADMIN_PASSWORD='choose-a-strong-password'
|
|
50
|
+
|
|
51
|
+
h3.h6 3. Apply the manifests
|
|
52
|
+
p.small.
|
|
53
|
+
Replace #[code thub.example.com], which must match #[code publicUrl], and the image.
|
|
54
|
+
+code('thub.yaml').
|
|
55
|
+
apiVersion: v1
|
|
56
|
+
kind: Namespace
|
|
57
|
+
metadata: { name: thub }
|
|
58
|
+
---
|
|
59
|
+
# Non-secret settings, mounted as the config file. Settings changed on the
|
|
60
|
+
# dashboard are stored in the database, so this mount can be read-only.
|
|
61
|
+
apiVersion: v1
|
|
62
|
+
kind: ConfigMap
|
|
63
|
+
metadata: { name: thub-coordinator-config, namespace: thub }
|
|
64
|
+
data:
|
|
65
|
+
coordinator.json: |
|
|
66
|
+
{
|
|
67
|
+
"publicUrl": "https://thub.example.com",
|
|
68
|
+
"trustProxy": "loopback, uniquelocal",
|
|
69
|
+
"updates": { "checkIntervalMin": 15 }
|
|
70
|
+
}
|
|
71
|
+
---
|
|
72
|
+
# SQLite needs block storage (ReadWriteOnce); avoid NFS-backed classes.
|
|
73
|
+
apiVersion: v1
|
|
74
|
+
kind: PersistentVolumeClaim
|
|
75
|
+
metadata: { name: thub-coordinator-data, namespace: thub }
|
|
76
|
+
spec:
|
|
77
|
+
accessModes: ["ReadWriteOnce"]
|
|
78
|
+
resources: { requests: { storage: 10Gi } }
|
|
79
|
+
---
|
|
80
|
+
# One replica only: one SQLite file, scheduler in-process. Recreate frees
|
|
81
|
+
# the volume before the new pod opens the database.
|
|
82
|
+
apiVersion: apps/v1
|
|
83
|
+
kind: Deployment
|
|
84
|
+
metadata: { name: thub-coordinator, namespace: thub }
|
|
85
|
+
spec:
|
|
86
|
+
replicas: 1
|
|
87
|
+
strategy: { type: Recreate }
|
|
88
|
+
selector: { matchLabels: { app: thub-coordinator } }
|
|
89
|
+
template:
|
|
90
|
+
metadata: { labels: { app: thub-coordinator } }
|
|
91
|
+
spec:
|
|
92
|
+
securityContext: { runAsNonRoot: true, runAsUser: 1000, runAsGroup: 1000, fsGroup: 1000 }
|
|
93
|
+
containers:
|
|
94
|
+
- name: coordinator
|
|
95
|
+
image: registry.example.com/thub-coordinator:<version>
|
|
96
|
+
ports: [{ name: http, containerPort: 8080 }]
|
|
97
|
+
env:
|
|
98
|
+
- { name: THUB_COORDINATOR_CONFIG, value: /etc/thub/coordinator.json }
|
|
99
|
+
envFrom:
|
|
100
|
+
- secretRef: { name: thub-coordinator-secrets } # THUB_SESSION_SECRET, THUB_CLIENT_JOIN_KEY, …
|
|
101
|
+
volumeMounts:
|
|
102
|
+
- { name: data, mountPath: /var/lib/thub }
|
|
103
|
+
- { name: config, mountPath: /etc/thub, readOnly: true }
|
|
104
|
+
- { name: tmp, mountPath: /tmp } # SQLite temp files (read-only root FS)
|
|
105
|
+
startupProbe: { httpGet: { path: /login, port: http }, periodSeconds: 5, failureThreshold: 24 }
|
|
106
|
+
readinessProbe: { httpGet: { path: /login, port: http }, periodSeconds: 10 }
|
|
107
|
+
livenessProbe: { tcpSocket: { port: http }, periodSeconds: 20 }
|
|
108
|
+
resources: { requests: { cpu: 100m, memory: 192Mi }, limits: { memory: 512Mi } }
|
|
109
|
+
securityContext: { allowPrivilegeEscalation: false, readOnlyRootFilesystem: true, capabilities: { drop: ["ALL"] } }
|
|
110
|
+
volumes:
|
|
111
|
+
- { name: data, persistentVolumeClaim: { claimName: thub-coordinator-data } }
|
|
112
|
+
- { name: config, configMap: { name: thub-coordinator-config } }
|
|
113
|
+
- { name: tmp, emptyDir: { sizeLimit: 256Mi } }
|
|
114
|
+
---
|
|
115
|
+
apiVersion: v1
|
|
116
|
+
kind: Service
|
|
117
|
+
metadata: { name: thub-coordinator, namespace: thub }
|
|
118
|
+
spec:
|
|
119
|
+
selector: { app: thub-coordinator }
|
|
120
|
+
ports: [{ name: http, port: 80, targetPort: http }]
|
|
121
|
+
---
|
|
122
|
+
# TLS at the ingress. Long polls and log streams (SSE) need long timeouts and
|
|
123
|
+
# no buffering; log batches and avatars stay under 10 MB.
|
|
124
|
+
apiVersion: networking.k8s.io/v1
|
|
125
|
+
kind: Ingress
|
|
126
|
+
metadata:
|
|
127
|
+
name: thub-coordinator
|
|
128
|
+
namespace: thub
|
|
129
|
+
annotations:
|
|
130
|
+
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
|
|
131
|
+
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
|
|
132
|
+
nginx.ingress.kubernetes.io/proxy-buffering: "off"
|
|
133
|
+
nginx.ingress.kubernetes.io/proxy-body-size: "10m"
|
|
134
|
+
# cert-manager.io/cluster-issuer: letsencrypt
|
|
135
|
+
spec:
|
|
136
|
+
ingressClassName: nginx
|
|
137
|
+
tls: [{ hosts: ["thub.example.com"], secretName: thub-coordinator-tls }]
|
|
138
|
+
rules:
|
|
139
|
+
- host: thub.example.com
|
|
140
|
+
http:
|
|
141
|
+
paths:
|
|
142
|
+
- path: /
|
|
143
|
+
pathType: Prefix
|
|
144
|
+
backend: { service: { name: thub-coordinator, port: { name: http } } }
|
|
145
|
+
+code('Deploy').
|
|
146
|
+
kubectl apply -f thub.yaml
|
|
147
|
+
kubectl -n thub rollout status deploy/thub-coordinator
|
|
148
|
+
kubectl -n thub logs deploy/thub-coordinator # listening on http://0.0.0.0:8080 · Self-update: off
|
|
149
|
+
|
|
150
|
+
h3.h6 4. First sign-in, then remove the bootstrap password
|
|
151
|
+
+code('It resets the admin password on every start while it\'s set').
|
|
152
|
+
kubectl -n thub patch secret thub-coordinator-secrets --type=json \
|
|
153
|
+
-p='[{"op":"remove","path":"/data/THUB_BOOTSTRAP_ADMIN_PASSWORD"}]'
|
|
154
|
+
kubectl -n thub rollout restart deploy/thub-coordinator
|
|
155
|
+
+code('thub-admin works inside the pod, on the same database').
|
|
156
|
+
kubectl -n thub exec deploy/thub-coordinator -- thub-admin user add alice --email alice@example.com --role admin --password '…'
|
|
157
|
+
kubectl -n thub exec deploy/thub-coordinator -- thub-admin agent add ci-firmware
|
|
158
|
+
kubectl -n thub exec deploy/thub-coordinator -- thub-admin join-key generate
|
|
159
|
+
|
|
160
|
+
h3.h6 Things to know
|
|
161
|
+
ul.small
|
|
162
|
+
li #[strong One replica.] One SQLite file and an in-process scheduler: no scale-out, no HPA. #[code Recreate] means a few seconds of downtime per upgrade; Clients and Agents reconnect by themselves.
|
|
163
|
+
li #[strong Updating:] deploy a new image tag. Migrations run on the new pod's first start. #[strong Update app] stays off (no #[code --self-update]); admins see a #[strong vX.Y.Z available] badge. Agents and Clients still update from the dashboard.
|
|
164
|
+
li #[strong Storage:] block storage (ReadWriteOnce), not NFS. A separate database volume: mount it and set #[code THUB_DB_PATH].
|
|
165
|
+
li #[strong /tmp] is an #[code emptyDir]: SQLite needs a writable temp directory and the root filesystem is read-only.
|
|
166
|
+
li #[strong Backups:] #[code kubectl -n thub exec deploy/thub-coordinator -- node -e "new (require('/app/node_modules/better-sqlite3'))('/var/lib/thub/thub.db').backup('/var/lib/thub/backup.db')"], then #[code kubectl cp].
|
|
167
|
+
li #[strong trustProxy] #[code "loopback, uniquelocal"] trusts the ingress from private ranges; set your pod CIDR if it's elsewhere.
|
|
168
|
+
li #[strong Other ingress controllers:] a read timeout well above 30 s, no response buffering, bodies up to 10 MB.
|
|
169
|
+
li #[strong HTTPS in the pod:] mount the TLS secret, set #[code THUB_TLS_CERT] / #[code THUB_TLS_KEY], listen on 8443, #[code scheme: HTTPS] in the probes, and an HTTPS backend (#[code nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"]) or a #[code LoadBalancer] on 443.
|
package/views/help/index.pug
CHANGED
|
@@ -8,6 +8,7 @@ block content
|
|
|
8
8
|
{ id: 'use-cases', label: 'Use cases' },
|
|
9
9
|
{ id: 'quick-start', label: 'Quick start' },
|
|
10
10
|
{ id: 'coordinator', label: 'Coordinator setup' },
|
|
11
|
+
{ id: 'kubernetes', label: 'Running in Kubernetes' },
|
|
11
12
|
{ id: 'agent-setup', label: 'Agent setup' },
|
|
12
13
|
{ id: 'client-setup', label: 'Client setup' },
|
|
13
14
|
{ id: 'client-machines', label: 'Client machines (HW / SW)' },
|
|
@@ -53,6 +54,7 @@ block content
|
|
|
53
54
|
include _overview
|
|
54
55
|
include _quick-start
|
|
55
56
|
include _coordinator
|
|
57
|
+
include _kubernetes
|
|
56
58
|
include _agent-setup
|
|
57
59
|
include _client-setup
|
|
58
60
|
include _client-machines
|
package/views/layout.pug
CHANGED
|
@@ -29,11 +29,12 @@ html(lang="en" data-bs-theme=(user && user.theme) || "auto")
|
|
|
29
29
|
data-bs-title=`Coordinator v${coordinatorVersion} · thub-common v${commonVersion} (validates job specs)`
|
|
30
30
|
)
|
|
31
31
|
| ver. #{coordinatorVersion}
|
|
32
|
-
|
|
32
|
+
//- Without --self-update (or for non-admins): just the notice.
|
|
33
|
+
if user && updates.coordinatorUpdate && !(can.admin && updates.selfUpdate)
|
|
33
34
|
span.badge.text-bg-warning.ms-1(
|
|
34
35
|
data-bs-toggle="tooltip"
|
|
35
36
|
data-bs-placement="bottom"
|
|
36
|
-
data-bs-title=
|
|
37
|
+
data-bs-title=updates.selfUpdate ? 'An admin can update the Coordinator from the navbar' : 'Update the Coordinator by hand (thub-admin self-update, npm i -g, or a new container image); self-update from the dashboard is off'
|
|
37
38
|
)= `v${updates.coordinatorUpdate} available`
|
|
38
39
|
button.navbar-toggler(type="button" data-bs-toggle="collapse" data-bs-target="#nav")
|
|
39
40
|
span.navbar-toggler-icon
|
|
@@ -91,7 +92,7 @@ html(lang="en" data-bs-theme=(user && user.theme) || "auto")
|
|
|
91
92
|
data-bs-title=updates.checkedAt ? `Check for updates now — the Coordinator and connected runners (last checked ${fmtDate(updates.checkedAt)})` : 'Check for updates now — the Coordinator and connected runners'
|
|
92
93
|
)
|
|
93
94
|
i.bi.bi-arrow-repeat
|
|
94
|
-
if user && can.admin && updates.coordinatorUpdate
|
|
95
|
+
if user && can.admin && updates.selfUpdate && updates.coordinatorUpdate
|
|
95
96
|
li.nav-item.d-flex.align-items-center.me-3
|
|
96
97
|
if updates.coordinatorPending
|
|
97
98
|
//- public/js/update-watch.js polls /updates/status and
|
|
@@ -249,7 +249,7 @@ mixin configPanes(r, returnTo, groupsById)
|
|
|
249
249
|
+configPane(p, 'stlinks')
|
|
250
250
|
+capabilitiesStatus(r)
|
|
251
251
|
+deviceTable('stlinks', 'ST-Link probes', current.stlinks || [], ['Device', 'udev devpath'])
|
|
252
|
-
.form-text Device: a udev index 1–8 (→ /dev/thub/dut<N>-stlink) or a /dev path. Which probe a job uses is up to its #[code --command] (
|
|
252
|
+
.form-text Device: a udev index 1–8 (→ /dev/thub/dut<N>-stlink) or a /dev path. Which probe a job uses is up to its #[code --command] (st-flash, by serial with several probes).
|
|
253
253
|
+saveCapabilities(canEdit)
|
|
254
254
|
+configPane(p, 'uarts')
|
|
255
255
|
+capabilitiesStatus(r)
|