datagrok-tools 6.5.10 → 6.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,17 @@
1
1
  # Datagrok-tools changelog
2
2
 
3
+ ## 6.7.0 (WIP)
4
+
5
+ * `grok login <server>` — keypair authentication, the replacement for the developer key. Generates an EC P-256 key, registers only its public half (in the browser, or with a one-shot `--code` from Profile > Public keys...), and keeps the private half in `~/.grok/keys/<alias>.json`. Logging in signs a server-issued nonce, so nothing reusable crosses the wire; keys can carry an expiry (`--expires`) and are revoked one at a time. `grok publish`, `grok test`, `grok stresstest` and `grok s` use it automatically whenever one is configured for the server, and fall back to the developer key otherwise. For CI, `GROK_PRIVATE_KEY` holds the private JWK (raw or base64) instead of a config file. Needs a server from 1.28 on; see https://datagrok.ai/help/govern/access-control/keypair-authentication
6
+ * `grok s token` — prints a session token for the configured server, so shell scripts stop curling `/users/login/dev` with a long-lived key.
7
+ * `grok config add` — `--key` is now optional: a server reached with a keypair has no developer key to record.
8
+ * Against a server older than 1.28 — which has no keypair endpoints and refuses their paths before routing — the CLI names the version needed instead of reporting a bare 401, and falls back to the developer key when one is still configured for that server.
9
+ * `grok login` reports its progress step by step (spinner on a terminal, one line per step in a log), including while it waits for the browser approval.
10
+
11
+ ## 6.6.0 (2026-09-13)
12
+
13
+ * The developer key is sent in the `Authorization` header (`Dev <key>`) instead of the URL path. `POST /users/login/dev/<key>` and `POST /packages/dev/<key>/<package>` put a long-lived credential into every nginx access log, proxy log and shell history along the way; the key-less routes (`/users/login/dev`, `/packages/dev/<package>`) take it as a header. Servers that predate the header form answer 404/401 and the old URL is used instead, so publishing to an older server still works - but a server from 1.28 on rejects the URL form and asks for datagrok-tools 6.6.0 or later.
14
+
3
15
  ## 6.5.10 (2026-09-11)
4
16
 
5
17
  * `grok s pull` — a bundle file name is capped at 200 bytes plus a digest of the full name, so it stays unique and stable across pulls. A nested view under a long space spells a longer name than a path component may hold (255 bytes on ext4 and NTFS): one TWIG snapshot reached 268 and the write failed with `ENAMETOOLONG`, taking down the whole part rather than the entity — a `--by-namespace` part that only touched the space as a dependency died with it.
package/CLAUDE.md CHANGED
@@ -94,7 +94,9 @@ The `publish` command executes these steps:
94
94
  3. Process environment variables in `/connections/*.json` files (replace `${VAR}`)
95
95
  4. **Process Docker images** (see below)
96
96
  5. Create ZIP archive with archiver-promise (includes `image.json` metadata per container)
97
- 6. Upload to server: `POST ${host}/packages/dev/${devKey}/${packageName}`
97
+ 6. Upload to server: `POST ${host}/packages/dev/${packageName}` with `Authorization: Dev ${devKey}`
98
+ (older servers only know `POST ${host}/packages/dev/${devKey}/${packageName}`; `devKeyFetch` in
99
+ `bin/utils/dev-key.ts` falls back to it on 404/401)
98
100
 
99
101
  **Key flags:**
100
102
  - `--debug` (default) - Package visible only to developer
package/GROK_S.md CHANGED
@@ -52,13 +52,21 @@ servers:
52
52
  key: admin
53
53
  dev:
54
54
  url: https://dev.datagrok.ai/api
55
- key: <developer-key>
55
+ keyFile: /home/me/.grok/keys/dev.json
56
+ login: me
56
57
  ```
57
58
 
58
- - `grok config add --alias <name> --server <url> --key <key>` writes a new entry.
59
+ - `grok login <server>` is the way to add a server: it registers a keypair and writes the
60
+ entry for you. The private key stays in `~/.grok/keys/<alias>.json`.
61
+ See [keypair authentication](../help/govern/access-control/keypair-authentication.md).
62
+ - `grok config add --alias <name> --server <url> [--key <developer-key>]` writes an entry by
63
+ hand. The developer key is deprecated; omit it when the server is reached with a keypair.
59
64
  - Add `--default` to make it the active server.
65
+ - In CI, `GROK_PRIVATE_KEY` (the private JWK, raw or base64) overrides the config file.
60
66
  - Every `grok s ...` command accepts `--host <alias-or-url>` to override the default. The URL
61
67
  is the API base (`https://host/api`, or `http://host:8082` for a bare Datlas).
68
+ - `grok s token` prints a session token for the target server — what a shell script needs
69
+ when it has to call the API with `curl` itself.
62
70
 
63
71
  ## Entity operations
64
72
 
package/README.md CHANGED
@@ -100,6 +100,11 @@ For more information on configuring connections, refer to the [Connections](http
100
100
  - `--suffix`: a string containing package version hash
101
101
 
102
102
  Running `grok publish` is the same as running `grok publish defaultHost --build --debug`.
103
+ - `report` works with user error reports on a Datagrok instance (the instance is a server alias from
104
+ `config.yaml`): `grok report fetch <instance> <number>` downloads the report zip, `grok report read
105
+ <zip|json|instance number>` prints one normalized JSON object (flags `--extract-screenshot`,
106
+ `--extract-d42`, `--extract-client-log`), `grok report resolve <instance> <number>` marks it resolved.
107
+ `ticket`, `comment`, `label`, `attach` talk to JIRA with `JIRA_TOKEN`. See `grok report --help`.
103
108
  - `check` checks package content (function signatures, import statements of external modules,
104
109
  etc.). The check is also run during package publication.
105
110
  - `init` modifies a package template by adding config files for linters, IDE, and so on
@@ -72,7 +72,8 @@ async function processPackage() {
72
72
  let timestamps = {};
73
73
  if (debug) {
74
74
  try {
75
- timestamps = await (await fetch(`${host}/packages/dev/${devKey}/${packageName}/timestamps`)).json();
75
+ timestamps = await (await fetch(`${host}/packages/dev/${packageName}/timestamps`,
76
+ {headers: {Authorization: `Dev ${devKey}`}})).json();
76
77
  if (timestamps['#type'] === 'ApiError') {
77
78
  console.log(timestamps.message);
78
79
  return 1;
@@ -134,8 +135,9 @@ async function processPackage() {
134
135
 
135
136
  //upload
136
137
  const uploadPromise = new Promise((resolve, reject) => {
137
- fetch(`${host}/packages/dev/${devKey}/${packageName}?debug=${debug.toString()}&rebuild=${rebuild.toString()}`, {
138
+ fetch(`${host}/packages/dev/${packageName}?debug=${debug.toString()}&rebuild=${rebuild.toString()}`, {
138
139
  method: 'POST',
140
+ headers: {Authorization: `Dev ${devKey}`},
139
141
  body: zip,
140
142
  }).then((body) => body.json()).then((j) => resolve(j)).catch((err) => {
141
143
  reject(err);
@@ -84,7 +84,9 @@ function config(args) {
84
84
  const nOptions = Object.keys(args).length - 1;
85
85
  const askRegistry = args.registry != null;
86
86
  const interactiveMode = args['_'].length === 1 && (nOptions < 1 || nOptions === 1 && (args.reset || askRegistry) || nOptions === 2 && args.reset && askRegistry);
87
- const hasAddServerCommand = args['_'].length === 2 && args['_'][1] === 'add' && args.server && args.key && args.k && args.alias && nOptions >= 4 && nOptions <= 6;
87
+ // `--key` is optional: a server reached with a keypair (`grok login`) has no
88
+ // developer key to record. minimist mirrors -k into both `key` and `k`.
89
+ const hasAddServerCommand = args['_'].length === 2 && args['_'][1] === 'add' && args.server && args.alias && nOptions >= 2 && nOptions <= 6;
88
90
  if (!interactiveMode && !hasAddServerCommand) return false;
89
91
  if (!_fs.default.existsSync(grokDir)) _fs.default.mkdirSync(grokDir);
90
92
  if (!_fs.default.existsSync(confPath) || args.reset) _fs.default.writeFileSync(confPath, _jsYaml.default.dump(confTemplate));
@@ -100,7 +102,7 @@ function config(args) {
100
102
  }
101
103
  const server = {
102
104
  url: args.server,
103
- key: args.key
105
+ key: args.key ?? ''
104
106
  };
105
107
  if (args.registry != null) {
106
108
  const registry = typeof args.registry === 'string' ? args.registry : defaultRegistry(args.server);
@@ -21,6 +21,7 @@ Commands:
21
21
  docker-gen Generate Celery Docker artifacts from Python functions
22
22
  init Modify a package template
23
23
  link Link \`datagrok-api\` and libraries for local development
24
+ login Log in to a server with a keypair (replaces the developer key)
24
25
  publish Upload a package
25
26
  report Manage user error reports (fetch, resolve, create ticket)
26
27
  run Build, publish, and open in browser
@@ -138,6 +139,29 @@ Options:
138
139
  file exists, plain \`grok api\` keeps it up to date; delete it to
139
140
  opt out again
140
141
  `;
142
+ const HELP_LOGIN = `
143
+ Usage: grok login <server>
144
+
145
+ Log in to a Datagrok server with a keypair. Generates an EC P-256 key, registers
146
+ its public half on your account, and stores the private half in
147
+ ~/.grok/keys/<alias>.json. Nothing reusable is ever copied out of the UI, and the
148
+ key can be given an expiry and revoked on its own.
149
+
150
+ grok login https://dev.datagrok.ai Approve the key in the browser
151
+ grok login dev --code AB12CD34 Use a code from your profile page
152
+ (Profile > Public keys...), no browser
153
+
154
+ Options:
155
+ [--code] [--name] [--expires] [--alias]
156
+
157
+ --code One-shot enrollment code from your profile page. Skips the browser
158
+ --name Key name shown in your profile (default: user@host)
159
+ --expires Days from now, or an ISO date (2027-01-31). Default: never
160
+ --alias Config alias to write (default: the server's first host label)
161
+
162
+ For CI, set GROK_PRIVATE_KEY to the private key JWK (raw or base64) instead of a
163
+ config file. Read more: https://datagrok.ai/help/govern/access-control/keypair-authentication
164
+ `;
141
165
  const HELP_CONFIG = `
142
166
  Usage: grok config
143
167
 
@@ -150,6 +174,7 @@ Options:
150
174
  --server Use to add a server to the config (\`grok config add --alias alias --server url --key key\`)
151
175
  --alias Use in conjunction with the \`server\` option to set the server name
152
176
  --key Use in conjunction with the \`server\` option to set the developer key
177
+ (deprecated - prefer \`grok login\`, which needs no key here)
153
178
  --default Use in conjunction with the \`server\` option to set the added server as default
154
179
  --registry Docker registry URL (default: registry.{server hostname})
155
180
  `;
@@ -371,10 +396,17 @@ Usage: grok report <subcommand> [args]
371
396
  Manage Datagrok user error reports
372
397
 
373
398
  Subcommands:
374
- fetch Download a report zip from a managed instance
399
+ fetch Download a report zip from a managed instance (writes <stem>_meta.json next to it)
375
400
  read Normalize a report (zip or json) into one JSON object on stdout
376
- resolve Mark a report as resolved
377
- ticket Create a JIRA ticket for a report via the Datlas API
401
+ resolve Mark a report as resolved (needs the _meta.json written by fetch)
402
+ ticket Create a JIRA ticket for a report directly in JIRA (no dedup; the key is NOT
403
+ written back to the report — prefer POST /reports/{id}/jira for that)
404
+ comment Add a comment to a JIRA ticket (--body <text> | --body-file <path>)
405
+ label Add labels to a JIRA ticket
406
+ attach Attach a file to a JIRA ticket
407
+
408
+ JIRA subcommands need JIRA_TOKEN (plus JIRA_USER for a user API token); ticket also needs
409
+ --project or $JIRA_PROJECT.
378
410
 
379
411
  Read flags:
380
412
  --extract-screenshot <path> Write the screenshot binary to <path>
@@ -403,6 +435,7 @@ const help = exports.help = {
403
435
  'docker-gen': HELP_DOCKER_GEN,
404
436
  init: HELP_INIT,
405
437
  link: HELP_LINK,
438
+ login: HELP_LOGIN,
406
439
  publish: HELP_PUBLISH,
407
440
  report: HELP_REPORT,
408
441
  run: HELP_RUN,
@@ -0,0 +1,213 @@
1
+ "use strict";
2
+
3
+ var _interopRequireDefault = require("@babel/runtime/helpers/interopRequireDefault");
4
+ Object.defineProperty(exports, "__esModule", {
5
+ value: true
6
+ });
7
+ exports.login = login;
8
+ var _crypto = _interopRequireDefault(require("crypto"));
9
+ var _http = _interopRequireDefault(require("http"));
10
+ var _os = _interopRequireDefault(require("os"));
11
+ var _child_process = require("child_process");
12
+ var color = _interopRequireWildcard(require("../utils/color-utils"));
13
+ var kp = _interopRequireWildcard(require("../utils/keypair"));
14
+ function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r = new WeakMap(), n = new WeakMap(); return (_interopRequireWildcard = function (e, t) { if (!t && e && e.__esModule) return e; var o, i, f = { __proto__: null, default: e }; if (null === e || "object" != typeof e && "function" != typeof e) return f; if (o = t ? n : r) { if (o.has(e)) return o.get(e); o.set(e, f); } for (const t in e) "default" !== t && {}.hasOwnProperty.call(e, t) && ((i = (o = Object.defineProperty) && Object.getOwnPropertyDescriptor(e, t)) && (i.get || i.set) ? o(f, t, i) : f[t] = e[t]); return f; })(e, t); }
15
+ /**
16
+ * `grok login <server>` - generates a keypair, registers its public half on the
17
+ * server, and stores the private half locally. Replaces the developer key: no
18
+ * reusable secret is copied out of the UI, and the key can be given an expiry
19
+ * and revoked on its own.
20
+ */
21
+ async function login(args) {
22
+ if (args['_'].length > 2) return false;
23
+ const target = (args['_'][1] ?? '').toString();
24
+ const name = args.name ?? `${_os.default.userInfo().username}-${_os.default.hostname()}`;
25
+ const expires = parseExpiry(args.expires);
26
+ if (args.expires && !expires) {
27
+ color.error('--expires takes a number of days or an ISO date (2027-01-31)');
28
+ return false;
29
+ }
30
+ let url;
31
+ let alias;
32
+ let keyFile;
33
+ let registered;
34
+ const config = kp.readConfig();
35
+ const {
36
+ publicKey,
37
+ privateKey
38
+ } = kp.generateKeyPair();
39
+ try {
40
+ ({
41
+ url,
42
+ alias
43
+ } = resolveTarget(target, config, args.alias));
44
+ url = await color.step(`Finding the Datagrok API at ${url}`, () => resolveApiRoot(url));
45
+ registered = args.code ? await color.step(`Registering "${name}" with the enrollment code`, () => kp.enrollWithCode(url, args.code, publicKey, name, expires)) : await enrollInBrowser(url, publicKey, name, expires);
46
+ if (registered.login == null) throw new Error(registered.comment ?? registered.message ?? 'The server did not accept the key');
47
+ keyFile = await color.step('Storing the private key', async () => {
48
+ const file = kp.savePrivateKey(alias, privateKey);
49
+ config.servers ??= {};
50
+ config.servers[alias] = {
51
+ ...(config.servers[alias] ?? {}),
52
+ url,
53
+ key: config.servers[alias]?.key ?? '',
54
+ keyFile: file,
55
+ login: registered.login
56
+ };
57
+ config.default ??= alias;
58
+ kp.writeConfig(config);
59
+ return file;
60
+ });
61
+
62
+ // Proves the round trip before the user walks away, rather than at the next
63
+ // `grok publish`: a key that registered but cannot sign is worse than none.
64
+ await color.step('Signing in with the new key', () => kp.keyLogin(url, privateKey));
65
+ } catch (error) {
66
+ color.error(error.message ?? String(error));
67
+ return false;
68
+ }
69
+ color.success(`Logged in to ${url} as ${registered.login}`);
70
+ console.log(` key ${name} (${kp.fingerprint(publicKey)})`);
71
+ console.log(` private key ${keyFile}`);
72
+ console.log(` alias ${alias} - use it as \`grok publish ${alias}\`, \`grok test --host ${alias}\``);
73
+ if (expires) console.log(` expires ${expires}`);
74
+ return true;
75
+ }
76
+ function resolveTarget(target, config, aliasArg) {
77
+ if (target === '') {
78
+ const alias = aliasArg ?? config.default;
79
+ if (!alias || !config.servers?.[alias]) throw new Error('Which server? Pass a URL or a configured alias: grok login https://dev.datagrok.ai/api');
80
+ return {
81
+ url: config.servers[alias].url,
82
+ alias
83
+ };
84
+ }
85
+ const configured = config.servers?.[target];
86
+ if (configured) return {
87
+ url: configured.url,
88
+ alias: aliasArg ?? target
89
+ };
90
+ let parsed;
91
+ try {
92
+ parsed = new URL(target);
93
+ } catch {
94
+ throw new Error(`"${target}" is neither a URL nor a server in your config`);
95
+ }
96
+ return {
97
+ url: parsed.href.replace(/\/$/, ''),
98
+ alias: aliasArg ?? defaultAlias(parsed)
99
+ };
100
+ }
101
+
102
+ /**
103
+ * The API base for [url]. A stand behind nginx serves it at `<origin>/api`, a bare Datlas at
104
+ * the origin itself, and there is no telling which from the URL alone - so ask, rather than
105
+ * guess and fail at the first call.
106
+ *
107
+ * A 200 is not the answer: nginx serves the single-page app for anything it does not route,
108
+ * so the origin of a real stand answers `/info/server` with the app's HTML. Only a JSON body
109
+ * that names the server counts.
110
+ */
111
+ async function resolveApiRoot(url) {
112
+ const candidates = /\/api$/.test(url) ? [url] : [`${url}/api`, url];
113
+ for (const candidate of candidates) {
114
+ try {
115
+ const response = await fetch(`${candidate}/info/server`);
116
+ if (!response.ok) continue;
117
+ const info = JSON.parse(await response.text());
118
+ if (info?.webRoot != null || info?.Version != null) return candidate;
119
+ } catch {/* not JSON, or unreachable: try the next shape */}
120
+ }
121
+ throw new Error(`${url} does not answer as a Datagrok API (tried ${candidates.join(' and ')})`);
122
+ }
123
+ function defaultAlias(url) {
124
+ const host = url.hostname;
125
+ if (host === 'localhost' || host === '127.0.0.1' || host === '::1') return 'local';
126
+ return host.split('.')[0];
127
+ }
128
+
129
+ /** Days from now, or an ISO date, as the ISO instant the server stores. */
130
+ function parseExpiry(value) {
131
+ if (value == null || value === '') return undefined;
132
+ const days = Number(value);
133
+ if (!isNaN(days) && days > 0) return new Date(Date.now() + days * 86400000).toISOString();
134
+ const date = new Date(String(value));
135
+ return isNaN(date.getTime()) ? undefined : date.toISOString();
136
+ }
137
+
138
+ /**
139
+ * Opens the server's enrollment page in a browser and waits for it to call back
140
+ * on a loopback listener. The user authenticates however that stand does -
141
+ * password, SSO, SAML - because the approval happens in the platform UI.
142
+ */
143
+ async function enrollInBrowser(url, publicKey, name, expires) {
144
+ const origin = new URL(url).origin;
145
+ // The page asks for this before it registers anything. It exists only in this terminal, so a
146
+ // link someone else sent has nothing for the user to type — which is the difference between
147
+ // approving one's own `grok login` and handing an attacker a key to one's account.
148
+ const verify = _crypto.default.randomBytes(4).toString('hex').toUpperCase().slice(0, 6);
149
+ let started;
150
+ const listening = new Promise(resolve => started = resolve);
151
+ const approved = new Promise((resolve, reject) => {
152
+ const timeout = setTimeout(() => {
153
+ server.close();
154
+ reject(new Error('Timed out waiting for the browser. Use `grok login <server> --code <code>` ' + 'with a code from your profile page instead.'));
155
+ }, 5 * 60 * 1000);
156
+ const server = _http.default.createServer((req, res) => {
157
+ const params = new URL(req.url ?? '/', 'http://127.0.0.1').searchParams;
158
+ const status = params.get('status');
159
+ res.writeHead(200, {
160
+ 'content-type': 'text/html; charset=utf-8'
161
+ });
162
+ res.end(status === 'ok' ? '<h3>Key registered. You can close this tab and return to the terminal.</h3>' : `<h3>Key registration was cancelled.</h3><p>${escapeHtml(params.get('message') ?? '')}</p>`);
163
+ clearTimeout(timeout);
164
+ server.close();
165
+ if (status === 'ok') resolve({
166
+ login: params.get('login'),
167
+ fingerprint: params.get('fingerprint')
168
+ });else reject(new Error(params.get('message') ?? 'Key registration was cancelled in the browser'));
169
+ });
170
+ server.listen(0, '127.0.0.1', () => {
171
+ const port = server.address().port;
172
+ const enrollUrl = `${origin}/enroll-key?` + new URLSearchParams({
173
+ pk: JSON.stringify(publicKey),
174
+ name,
175
+ ...(expires ? {
176
+ expires
177
+ } : {}),
178
+ verify,
179
+ cb: `http://127.0.0.1:${port}`
180
+ }).toString();
181
+ console.log(`\n Verification code: ${verify}`);
182
+ console.log(` Type it on the page that opens. If the browser does not open, visit:`);
183
+ console.log(` ${enrollUrl}\n`);
184
+ openBrowser(enrollUrl);
185
+ started();
186
+ });
187
+ });
188
+
189
+ // The prints above land before the spinner starts, so the code stays readable on screen.
190
+ await listening;
191
+ return await color.step(`Waiting for approval in the browser at ${origin}`, () => approved);
192
+ }
193
+ function escapeHtml(s) {
194
+ return s.replace(/[&<>"]/g, c => ({
195
+ '&': '&amp;',
196
+ '<': '&lt;',
197
+ '>': '&gt;',
198
+ '"': '&quot;'
199
+ })[c]);
200
+ }
201
+ function openBrowser(url) {
202
+ try {
203
+ // cmd splits an unquoted argument at `&`, and the enrollment URL is all query parameters;
204
+ // rundll32 takes the whole thing and hands it to the default browser.
205
+ const [command, args] = process.platform === 'win32' ? ['rundll32', ['url.dll,FileProtocolHandler', url]] : process.platform === 'darwin' ? ['open', [url]] : ['xdg-open', [url]];
206
+ (0, _child_process.spawn)(command, args, {
207
+ detached: true,
208
+ stdio: 'ignore'
209
+ }).unref();
210
+ } catch {
211
+ // The URL is printed above; a headless box just uses that.
212
+ }
213
+ }
@@ -20,6 +20,8 @@ var color = _interopRequireWildcard(require("../utils/color-utils"));
20
20
  var _check = require("./check");
21
21
  var _pythonCeleryGen = require("../utils/python-celery-gen");
22
22
  var _queueWorkerGen = require("../utils/queue-worker-gen");
23
+ var _devKey = require("../utils/dev-key");
24
+ var keypair = _interopRequireWildcard(require("../utils/keypair"));
23
25
  function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r = new WeakMap(), n = new WeakMap(); return (_interopRequireWildcard = function (e, t) { if (!t && e && e.__esModule) return e; var o, i, f = { __proto__: null, default: e }; if (null === e || "object" != typeof e && "function" != typeof e) return f; if (o = t ? n : r) { if (o.has(e)) return o.get(e); o.set(e, f); } for (const t in e) "default" !== t && {}.hasOwnProperty.call(e, t) && ((i = (o = Object.defineProperty) && Object.getOwnPropertyDescriptor(e, t)) && (i.get || i.set) ? o(f, t, i) : f[t] = e[t]); return f; })(e, t); }
24
26
  // @ts-ignore
25
27
 
@@ -241,18 +243,21 @@ function listRecursive(basePath, rel) {
241
243
  return results;
242
244
  }
243
245
  async function getUserLogin(host, devKey) {
244
- let loginResp;
246
+ let token;
245
247
  try {
246
- loginResp = await (0, _nodeFetch.default)(`${host}/users/login/dev/${devKey}`, {
247
- method: 'POST'
248
- });
248
+ token = await (0, _devKey.keypairToken)(host, devKey);
249
+ if (token == null) {
250
+ const loginResp = await (0, _devKey.devKeyFetch)(`${host}/users/login/dev`, `${host}/users/login/dev/${devKey}`, devKey, {
251
+ method: 'POST'
252
+ });
253
+ if (loginResp.status !== 200) return null;
254
+ token = (await loginResp.json()).token;
255
+ }
249
256
  } catch (e) {
250
257
  color.warn(`Cannot reach server ${host}: ${e.message || e}`);
251
258
  return null;
252
259
  }
253
- if (loginResp.status !== 200) return null;
254
- const loginData = await loginResp.json();
255
- const token = loginData.token;
260
+ if (token == null) return null;
256
261
  try {
257
262
  const userResp = await (0, _nodeFetch.default)(`${host}/users/current`, {
258
263
  headers: {
@@ -485,9 +490,10 @@ async function fallbackImage(img, host, devKey, registry, version, contentHash)
485
490
  async function processPackage(debug, rebuild, host, devKey, packageName, dropDb, suffix, hostAlias, registry, rebuildDocker, skipDockerRebuild) {
486
491
  // Validate server connectivity and dev key
487
492
  let timestamps = {};
488
- let url = `${host}/packages/dev/${devKey}/${packageName}`;
493
+ const url = `${host}/packages/dev/${packageName}`;
494
+ const legacyUrl = `${host}/packages/dev/${devKey}/${packageName}`;
489
495
  try {
490
- const checkResp = await (0, _nodeFetch.default)(url + '/timestamps');
496
+ const checkResp = await (0, _devKey.devKeyFetch)(`${url}/timestamps`, `${legacyUrl}/timestamps`, devKey, {}, host);
491
497
  const checkData = await checkResp.json();
492
498
  if (checkData['#type'] === 'ApiError') {
493
499
  color.error(checkData.message);
@@ -614,15 +620,15 @@ async function processPackage(debug, rebuild, host, devKey, packageName, dropDb,
614
620
  });
615
621
 
616
622
  // Upload
617
- url += `?debug=${debug.toString()}&rebuild=${rebuild.toString()}&dropDb=${(dropDb ?? false).toString()}`;
618
- if (suffix) url += `&suffix=${suffix.toString()}`;
623
+ let query = `?debug=${debug.toString()}&rebuild=${rebuild.toString()}&dropDb=${(dropDb ?? false).toString()}`;
624
+ if (suffix) query += `&suffix=${suffix.toString()}`;
619
625
  await zip.finalize();
620
626
  const zipBuffer = Buffer.concat(chunks);
621
627
  try {
622
- const body = await (0, _nodeFetch.default)(url, {
628
+ const body = await (0, _devKey.devKeyFetch)(url + query, legacyUrl + query, devKey, {
623
629
  method: 'POST',
624
630
  body: zipBuffer
625
- });
631
+ }, host);
626
632
  const log = JSON.parse(await body.text());
627
633
  if (log != undefined) {
628
634
  if (log['#type'] === 'ApiError') {
@@ -726,7 +732,7 @@ async function publishPackage(args) {
726
732
 
727
733
  // Update the developer key
728
734
  if (args.key) key = args.key;
729
- if (key === '') return color.warn('Please provide the key with `--key` option or add it by running `grok config`');
735
+ if (!key && !keypair.keypairFor(url)) return color.warn(`No credentials for ${url}. Run \`grok login ${host}\`, ` + 'or pass a developer key with `--key` (deprecated).');
730
736
 
731
737
  // Get the package name
732
738
  if (!_fs.default.existsSync(packDir)) return color.error('`package.json` doesn\'t exist');
@@ -27,7 +27,7 @@ const ENTITY_TYPES = {
27
27
  reports: 'UserReport'
28
28
  };
29
29
  const ENTITIES = ['users', 'groups', 'functions', 'connections', 'queries', 'scripts', 'packages', 'reports', 'files', 'tables'];
30
- const COMMANDS = ['shares', 'domains', 'raw', 'batch', 'describe', 'healthcheck', 'sync', 'pull', 'push', 'migrate', 'diff', 'bundle'];
30
+ const COMMANDS = ['shares', 'domains', 'raw', 'batch', 'describe', 'healthcheck', 'sync', 'pull', 'push', 'migrate', 'diff', 'bundle', 'token'];
31
31
  const VERBS = ['list', 'count', 'get', 'delete'];
32
32
  async function server(argv) {
33
33
  const args = argv['_'].slice(1);
@@ -61,6 +61,12 @@ async function server(argv) {
61
61
  if (['pull', 'push', 'migrate', 'diff', 'bundle'].includes(entity)) return await (0, _serverMigrate.handleMigrate)(dapi, entity, [verb, ...rest].filter(Boolean), argv, output);
62
62
  if (entity === 'domains') return await (0, _serverDomains.handleDomains)(dapi, verb, rest, argv, output);
63
63
  if (entity === 'batch') return await handleBatch(dapi, argv, verb, rest, output);
64
+ // Shell scripts that used to curl /users/login/dev get a token the same way
65
+ // every other command does, whatever credential the config holds.
66
+ if (entity === 'token') {
67
+ console.log(client.token);
68
+ return true;
69
+ }
64
70
  if (entity === 'raw') return await handleRaw(dapi, verb, rest, argv, output);
65
71
  if (entity === 'describe') return await handleDescribe(dapi, verb ?? rest[0], output);
66
72
  if (entity === 'healthcheck') return await handleHealthcheck(dapi, argv, output);
@@ -49,7 +49,9 @@ async function run(config, args) {
49
49
  processArgs.push('./node-test-loader/register.mjs');
50
50
  processArgs.push('src/package-test-node.ts');
51
51
  processArgs.push(`--apiUrl=${config.url}`);
52
- processArgs.push(`--devKey=${config.key}`);
52
+ // A session token rather than the credential: it is short-lived, and with a keypair
53
+ // there is no reusable secret to put on a command line at all.
54
+ processArgs.push(`--token=${await testUtils.getToken(config.url, config.key)}`);
53
55
  // Explicit even though it's the runner default: the stress baseline must only run
54
56
  // stressTest-marked tests regardless of how the runner's default evolves.
55
57
  processArgs.push('--mode=stress');
package/bin/grok.js CHANGED
@@ -23,6 +23,7 @@ const commands = {
23
23
  'docker-gen': require('./commands/docker-gen').dockerGen,
24
24
  init: require('./commands/init').init,
25
25
  link: require('./commands/link').link,
26
+ login: require('./commands/login').login,
26
27
  publish: require('./commands/publish').publish,
27
28
  report: require('./commands/report').report,
28
29
  run: require('./commands/run').run,
@@ -5,7 +5,9 @@ Object.defineProperty(exports, "__esModule", {
5
5
  });
6
6
  exports.isVerbose = exports.info = exports.fail = exports.error = void 0;
7
7
  exports.log = log;
8
- exports.warn = exports.success = exports.setVerbose = void 0;
8
+ exports.setVerbose = void 0;
9
+ exports.step = step;
10
+ exports.warn = exports.success = void 0;
9
11
  const error = s => console.log('\x1b[31m%s\x1b[0m', s);
10
12
  exports.error = error;
11
13
  const info = s => console.log('\x1b[32m%s\x1b[0m', s);
@@ -42,4 +44,31 @@ function log(s, type = 'plain') {
42
44
  console.log(s);
43
45
  break;
44
46
  }
47
+ }
48
+
49
+ /**
50
+ * One step of a multi-step command: prints its label, ticks a spinner while [action] runs, and
51
+ * replaces the line with the outcome. A spinner needs a terminal to erase lines, so a CI log
52
+ * (or a redirected stdout) gets one plain line per step instead.
53
+ */
54
+ async function step(label, action) {
55
+ const tty = process.stdout.isTTY === true;
56
+ const frames = ['-', '\\', '|', '/'];
57
+ let frame = 0;
58
+ const draw = () => process.stdout.write(`\r ${frames[frame++ % frames.length]} ${label} `);
59
+ if (!tty) console.log(` ${label}...`);
60
+ const timer = tty ? setInterval(draw, 120) : null;
61
+ if (tty) draw();
62
+ const finish = (mark, color, text) => {
63
+ if (timer) clearInterval(timer);
64
+ if (tty) process.stdout.write(`\r\x1b[2K \x1b[${color}m${mark}\x1b[0m ${text}\n`);else if (mark !== '+') console.log(` ${mark} ${text}`);
65
+ };
66
+ try {
67
+ const result = await action();
68
+ finish('+', '32', label);
69
+ return result;
70
+ } catch (e) {
71
+ finish('x', '31', label);
72
+ throw e;
73
+ }
45
74
  }
@@ -0,0 +1,71 @@
1
+ "use strict";
2
+
3
+ Object.defineProperty(exports, "__esModule", {
4
+ value: true
5
+ });
6
+ exports.devKeyFetch = devKeyFetch;
7
+ exports.devKeyHeaders = devKeyHeaders;
8
+ exports.keypairToken = keypairToken;
9
+ var keypair = _interopRequireWildcard(require("./keypair"));
10
+ function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r = new WeakMap(), n = new WeakMap(); return (_interopRequireWildcard = function (e, t) { if (!t && e && e.__esModule) return e; var o, i, f = { __proto__: null, default: e }; if (null === e || "object" != typeof e && "function" != typeof e) return f; if (o = t ? n : r) { if (o.has(e)) return o.get(e); o.set(e, f); } for (const t in e) "default" !== t && {}.hasOwnProperty.call(e, t) && ((i = (o = Object.defineProperty) && Object.getOwnPropertyDescriptor(e, t)) && (i.get || i.set) ? o(f, t, i) : f[t] = e[t]); return f; })(e, t); }
11
+ const fetch = require('node-fetch');
12
+
13
+ /** `Authorization` header carrying the developer key. */
14
+ function devKeyHeaders(key, headers = {}) {
15
+ return {
16
+ ...headers,
17
+ 'Authorization': `Dev ${key}`
18
+ };
19
+ }
20
+
21
+ /** One login per API root per process: every publish step reuses the token. */
22
+ const tokenCache = new Map();
23
+
24
+ /**
25
+ * Bearer token obtained by signing a nonce with the server's registered keypair,
26
+ * or `null` when no keypair is configured for [apiRoot] and the caller should
27
+ * fall back to the developer key.
28
+ */
29
+ async function keypairToken(apiRoot, devKey) {
30
+ const privateKey = keypair.keypairFor(apiRoot, devKey);
31
+ if (!privateKey) return null;
32
+ if (!tokenCache.has(apiRoot)) tokenCache.set(apiRoot, keypair.keyLogin(apiRoot, privateKey));
33
+ try {
34
+ return await tokenCache.get(apiRoot);
35
+ } catch (e) {
36
+ // A server without the keypair endpoints is a reason to use the developer key that is
37
+ // still configured, not to stop: one config usually names stands of both vintages.
38
+ if (e?.name !== 'ServerTooOldError' || !devKey) throw e;
39
+ tokenCache.delete(apiRoot);
40
+ return null;
41
+ }
42
+ }
43
+
44
+ /**
45
+ * Calls [url] as the configured user. With a keypair (`grok login`) that is a
46
+ * session token; otherwise the developer key rides in the `Authorization`
47
+ * header, falling back to [legacyUrl] - which carries the key as a path segment -
48
+ * for servers that predate the header form. Those answer 404 (no such route) or
49
+ * 401 (the route-less path is not on their anonymous allow-list); a server that
50
+ * knows the header form answers neither.
51
+ *
52
+ * [apiRoot] enables the keypair path; without it this stays dev-key only.
53
+ */
54
+ async function devKeyFetch(url, legacyUrl, key, init = {}, apiRoot) {
55
+ if (apiRoot != null) {
56
+ const token = await keypairToken(apiRoot, key);
57
+ if (token) return await fetch(url, {
58
+ ...init,
59
+ headers: {
60
+ ...init.headers,
61
+ 'Authorization': token
62
+ }
63
+ });
64
+ }
65
+ const response = await fetch(url, {
66
+ ...init,
67
+ headers: devKeyHeaders(key, init.headers)
68
+ });
69
+ if (response.status !== 404 && response.status !== 401) return response;
70
+ return await fetch(legacyUrl, init);
71
+ }
@@ -0,0 +1,292 @@
1
+ "use strict";
2
+
3
+ var _interopRequireDefault = require("@babel/runtime/helpers/interopRequireDefault");
4
+ Object.defineProperty(exports, "__esModule", {
5
+ value: true
6
+ });
7
+ exports.ServerTooOldError = exports.MIN_SERVER_VERSION = void 0;
8
+ exports.enrollWithCode = enrollWithCode;
9
+ exports.fingerprint = fingerprint;
10
+ exports.generateKeyPair = generateKeyPair;
11
+ exports.getServerCredentials = getServerCredentials;
12
+ exports.hasKeypair = hasKeypair;
13
+ exports.keyFilePath = keyFilePath;
14
+ exports.keyFromEnv = keyFromEnv;
15
+ exports.keyLogin = keyLogin;
16
+ exports.keypairFor = keypairFor;
17
+ exports.publicPart = publicPart;
18
+ exports.readConfig = readConfig;
19
+ exports.savePrivateKey = savePrivateKey;
20
+ exports.signNonce = signNonce;
21
+ exports.writeConfig = writeConfig;
22
+ var _crypto = _interopRequireDefault(require("crypto"));
23
+ var _fs = _interopRequireDefault(require("fs"));
24
+ var _os = _interopRequireDefault(require("os"));
25
+ var _path = _interopRequireDefault(require("path"));
26
+ var _jsYaml = _interopRequireDefault(require("js-yaml"));
27
+ const grokDir = _path.default.join(_os.default.homedir(), '.grok');
28
+ const confPath = _path.default.join(grokDir, 'config.yaml');
29
+ const keysDir = _path.default.join(grokDir, 'keys');
30
+
31
+ /** Prefix the server wraps a nonce in before checking the signature. */
32
+ const SIGNATURE_PREFIX = 'datagrok-login:';
33
+ /** Generates the keypair `grok login` registers: EC P-256, the JOSE ES256 curve. */
34
+ function generateKeyPair() {
35
+ const {
36
+ publicKey,
37
+ privateKey
38
+ } = _crypto.default.generateKeyPairSync('ec', {
39
+ namedCurve: 'prime256v1'
40
+ });
41
+ return {
42
+ publicKey: publicKey.export({
43
+ format: 'jwk'
44
+ }),
45
+ privateKey: privateKey.export({
46
+ format: 'jwk'
47
+ })
48
+ };
49
+ }
50
+
51
+ /** The public half of a private JWK — what gets registered on the server. */
52
+ function publicPart(privateKey) {
53
+ const {
54
+ d,
55
+ p,
56
+ q,
57
+ dp,
58
+ dq,
59
+ qi,
60
+ ...pub
61
+ } = privateKey;
62
+ return pub;
63
+ }
64
+
65
+ /**
66
+ * RFC 7638 JWK thumbprint, base64url without padding. The server computes the
67
+ * same value from the stored key, so this is what identifies a key at login.
68
+ */
69
+ function fingerprint(jwk) {
70
+ const members = {
71
+ EC: ['crv', 'kty', 'x', 'y'],
72
+ RSA: ['e', 'kty', 'n']
73
+ };
74
+ const order = members[jwk.kty];
75
+ if (!order) throw new Error(`Unsupported key type "${jwk.kty}" - expected EC or RSA`);
76
+ const canonical = '{' + order.map(m => `${JSON.stringify(m)}:${JSON.stringify(jwk[m])}`).join(',') + '}';
77
+ return _crypto.default.createHash('sha256').update(canonical).digest('base64url');
78
+ }
79
+
80
+ /**
81
+ * Signs the login nonce. [audience] is the API root this client dialed: it is part of what is
82
+ * signed, so a server cannot relay the signature to a second stand where the same key is
83
+ * enrolled. ECDSA signatures go out in the raw r||s form (JOSE's, not DER's), which is what
84
+ * the server's verifier expects.
85
+ */
86
+ function signNonce(privateKey, audience, nonce) {
87
+ const key = _crypto.default.createPrivateKey({
88
+ key: privateKey,
89
+ format: 'jwk'
90
+ });
91
+ const options = {
92
+ key
93
+ };
94
+ if (privateKey.kty === 'EC') options.dsaEncoding = 'ieee-p1363';
95
+ return _crypto.default.sign('sha256', Buffer.from(`${SIGNATURE_PREFIX}${audience}:${nonce}`), options).toString('base64url');
96
+ }
97
+
98
+ /** Exchanges a keypair for a session token: ask for a nonce, sign it, log in. */
99
+ async function keyLogin(url, privateKey) {
100
+ const fp = fingerprint(publicPart(privateKey));
101
+ const challenge = await postJson(`${url}/users/login/key/challenge`, {
102
+ fingerprint: fp
103
+ });
104
+ const response = await postJson(`${url}/users/login/key`, {
105
+ fingerprint: fp,
106
+ audience: url,
107
+ nonce: challenge.nonce,
108
+ signature: signNonce(privateKey, url, challenge.nonce)
109
+ });
110
+ if (response.isSuccess !== true) throw new Error(response.comment ?? 'Key login failed');
111
+ return response.token;
112
+ }
113
+
114
+ /** Registers [publicKey] with a one-shot enrollment code from the user profile. */
115
+ async function enrollWithCode(url, code, publicKey, name, expires) {
116
+ return await postJson(`${url}/users/keys/enroll`, {
117
+ code,
118
+ name,
119
+ expires,
120
+ publicKey: JSON.stringify(publicKey)
121
+ });
122
+ }
123
+
124
+ /** The first Datagrok that has the keypair endpoints. */
125
+ const MIN_SERVER_VERSION = exports.MIN_SERVER_VERSION = '1.28';
126
+
127
+ /**
128
+ * Thrown when the server has no keypair endpoints at all. Callers that still hold a developer
129
+ * key catch it and fall back; `grok login` reports it, since there is nothing to fall back to.
130
+ */
131
+ class ServerTooOldError extends Error {
132
+ constructor(server) {
133
+ super(`${server} does not support keypair authentication — it needs Datagrok ` + `${MIN_SERVER_VERSION} or later. Use a developer key for this server ` + '(grok config add --alias <alias> --server <url> --key <key>), or ask its operator to upgrade.');
134
+ this.server = server;
135
+ this.name = 'ServerTooOldError';
136
+ }
137
+ }
138
+ exports.ServerTooOldError = ServerTooOldError;
139
+ async function postJson(url, body) {
140
+ const response = await fetch(url, {
141
+ method: 'POST',
142
+ headers: {
143
+ 'content-type': 'application/json'
144
+ },
145
+ body: JSON.stringify(body)
146
+ });
147
+ // These routes are anonymous on every server that has them. A server that does not answers
148
+ // 404 (no such route) or 401 (unknown paths are refused before routing) — either way, the
149
+ // keypair endpoints are simply not there.
150
+ if (response.status === 404 || response.status === 401) throw new ServerTooOldError(new URL(url).origin);
151
+ const text = await response.text();
152
+ try {
153
+ return JSON.parse(text);
154
+ } catch {
155
+ throw new Error(`Unexpected response from ${url} (status ${response.status}): ${text.slice(0, 200)}`);
156
+ }
157
+ }
158
+ function keyFilePath(alias) {
159
+ return _path.default.join(keysDir, `${alias}.json`);
160
+ }
161
+
162
+ /** Writes the private key readable only by its owner, the way ssh-keygen does. */
163
+ function savePrivateKey(alias, privateKey) {
164
+ _fs.default.mkdirSync(keysDir, {
165
+ recursive: true,
166
+ mode: 0o700
167
+ });
168
+ const file = keyFilePath(alias);
169
+ _fs.default.writeFileSync(file, JSON.stringify(privateKey, null, 2), {
170
+ mode: 0o600
171
+ });
172
+ // mkdirSync/writeFileSync ignore `mode` when the path already exists.
173
+ try {
174
+ _fs.default.chmodSync(keysDir, 0o700);
175
+ _fs.default.chmodSync(file, 0o600);
176
+ } catch {/* Windows has no POSIX modes; ACLs already keep it in the profile. */}
177
+ return file;
178
+ }
179
+ function readConfig() {
180
+ if (!_fs.default.existsSync(confPath)) return {
181
+ servers: {},
182
+ default: ''
183
+ };
184
+ return _jsYaml.default.load(_fs.default.readFileSync(confPath, {
185
+ encoding: 'utf-8'
186
+ })) ?? {
187
+ servers: {},
188
+ default: ''
189
+ };
190
+ }
191
+ function writeConfig(config) {
192
+ _fs.default.mkdirSync(grokDir, {
193
+ recursive: true
194
+ });
195
+ _fs.default.writeFileSync(confPath, _jsYaml.default.dump(config));
196
+ }
197
+
198
+ /**
199
+ * The key at [file], or `undefined` when there is none. A config can name a key file the
200
+ * deployment has not filled in yet — CI writes the entry and the secret separately — and that
201
+ * has to mean "fall back to the developer key". A file that exists but cannot be read or parsed
202
+ * is a different thing, and says so.
203
+ */
204
+ function tryLoadKeyFile(file) {
205
+ const expanded = file.startsWith('~') ? _path.default.join(_os.default.homedir(), file.slice(1)) : file;
206
+ let text;
207
+ try {
208
+ text = _fs.default.readFileSync(expanded, {
209
+ encoding: 'utf-8'
210
+ });
211
+ } catch (error) {
212
+ if (error?.code === 'ENOENT') return undefined;
213
+ throw new Error(`cannot read the private key at ${expanded}: ${error?.message ?? error}`);
214
+ }
215
+ try {
216
+ return parseKey(text);
217
+ } catch (error) {
218
+ throw new Error(`${expanded} is not a private key in JWK form: ${error?.message ?? error}`);
219
+ }
220
+ }
221
+
222
+ /**
223
+ * The credentials for one server: its config entry, with `GROK_PRIVATE_KEY`
224
+ * taking precedence so a CI job can hold the key in a secret rather than on
225
+ * disk. [hostKey] is an alias, a URL, or empty for the default server.
226
+ */
227
+ function getServerCredentials(hostKey) {
228
+ const config = readConfig();
229
+ let host = (hostKey === '' || hostKey == null ? config.default : hostKey).trim();
230
+ let entry;
231
+ let alias;
232
+ let url;
233
+ try {
234
+ url = new URL(host).href;
235
+ if (url.endsWith('/')) url = url.slice(0, -1);
236
+ // Several aliases can name the same server. Prefer one that has a keypair: a
237
+ // dev-key-only entry matching first would silently downgrade the login.
238
+ const matches = Object.keys(config.servers ?? {}).filter(name => config.servers[name].url === url);
239
+ alias = matches.find(name => config.servers[name].keyFile || _fs.default.existsSync(keyFilePath(name))) ?? matches[0];
240
+ entry = alias == null ? undefined : config.servers[alias];
241
+ } catch (error) {
242
+ entry = config.servers?.[host];
243
+ if (entry == null) throw new Error(`Unknown server alias. Please add it to ${confPath}`);
244
+ alias = host;
245
+ url = entry.url;
246
+ }
247
+ const cred = {
248
+ url: url,
249
+ alias,
250
+ key: entry?.key,
251
+ login: entry?.login
252
+ };
253
+ if (process.env.GROK_PRIVATE_KEY) {
254
+ cred.privateKey = parseKey(process.env.GROK_PRIVATE_KEY);
255
+ cred.privateKeySource = 'GROK_PRIVATE_KEY';
256
+ } else if (entry?.keyFile && (cred.privateKey = tryLoadKeyFile(entry.keyFile)) != null) cred.privateKeySource = entry.keyFile;else if (alias && (cred.privateKey = tryLoadKeyFile(keyFilePath(alias))) != null) cred.privateKeySource = keyFilePath(alias);
257
+ if (process.env.GROK_LOGIN) cred.login = process.env.GROK_LOGIN;
258
+ return cred;
259
+ }
260
+
261
+ /** A JWK, raw or base64-encoded — CI secret stores mangle multi-line values. */
262
+ function parseKey(value) {
263
+ const text = value.trim().startsWith('{') ? value : Buffer.from(value.trim(), 'base64').toString('utf-8');
264
+ return JSON.parse(text);
265
+ }
266
+ function hasKeypair(cred) {
267
+ return cred.privateKey != null;
268
+ }
269
+
270
+ /** A private JWK held in an environment variable, raw or base64-encoded. */
271
+ function keyFromEnv(name) {
272
+ const value = process.env[name];
273
+ return value ? parseKey(value) : undefined;
274
+ }
275
+
276
+ /**
277
+ * The private key to use for [url], or `undefined` to fall back to the developer
278
+ * key. An explicitly supplied [devKey] that differs from the one configured for
279
+ * this server means the caller wants *that* identity - a second CI user, say -
280
+ * so the configured keypair must not silently take over.
281
+ */
282
+ function keypairFor(url, devKey) {
283
+ let cred;
284
+ try {
285
+ cred = getServerCredentials(url);
286
+ } catch {
287
+ return undefined;
288
+ }
289
+ if (cred.privateKey == null) return undefined;
290
+ if (devKey && devKey !== cred.key) return undefined;
291
+ return cred.privateKey;
292
+ }
@@ -11,6 +11,7 @@ exports.mapPositionalParams = mapPositionalParams;
11
11
  exports.parseDomainAddress = parseDomainAddress;
12
12
  exports.throwIfApiError = throwIfApiError;
13
13
  var _crypto = require("crypto");
14
+ var _keypair = require("./keypair");
14
15
  /// Docs: [Grok Dapi](/docs/plans/grok-dapi/)
15
16
 
16
17
  function ensureBodyId(body) {
@@ -62,13 +63,34 @@ async function fetchOrRetry(url, opts, retriable, timeoutMs = setting('TIMEOUT',
62
63
  class NodeApiClient {
63
64
  /** Set by `createClient` when the run asked for an admin session, so a re-login restores it. */
64
65
  adminMode = false;
65
- constructor(baseUrl, token, devKey) {
66
+ constructor(baseUrl, token, devKey, privateKey) {
66
67
  this.baseUrl = baseUrl;
67
68
  this.token = token;
68
69
  this.devKey = devKey;
70
+ this.privateKey = privateKey;
69
71
  }
70
- static async login(baseUrl, devKey) {
71
- const res = await fetch(`${baseUrl}/users/login/dev/${devKey}`, {
72
+
73
+ /** [privateKey] from a caller that resolved it by alias; otherwise it is looked up by URL. */
74
+ static async login(baseUrl, devKey, privateKey) {
75
+ privateKey ??= (0, _keypair.keypairFor)(baseUrl, devKey);
76
+ if (privateKey) {
77
+ try {
78
+ return new NodeApiClient(baseUrl, await (0, _keypair.keyLogin)(baseUrl, privateKey), devKey, privateKey);
79
+ } catch (e) {
80
+ // A server without the keypair endpoints is a reason to use the developer key that is
81
+ // still configured, not to stop: the same config often names stands of both vintages.
82
+ if (e?.name !== 'ServerTooOldError' || !devKey) throw e;
83
+ }
84
+ }
85
+ // Servers before 1.28 only knew the key-in-URL form, where it leaked into every
86
+ // access log on the way; they answer 404 or 401 to the key-less route.
87
+ let res = await fetch(`${baseUrl}/users/login/dev`, {
88
+ method: 'POST',
89
+ headers: {
90
+ 'Authorization': `Dev ${devKey}`
91
+ }
92
+ });
93
+ if (res.status === 404 || res.status === 401) res = await fetch(`${baseUrl}/users/login/dev/${devKey}`, {
72
94
  method: 'POST'
73
95
  });
74
96
  const json = (res.headers.get('content-type') ?? '').includes('application/json') ? await res.json() : null;
@@ -79,12 +101,12 @@ class NodeApiClient {
79
101
 
80
102
  /**
81
103
  * A stand serving several isolates can reject a session one of them does not know, and an
82
- * hour-long walk has no way to ask the operator to log in again. The developer key is good
83
- * for a new session, so one is taken rather than losing the run.
104
+ * hour-long walk has no way to ask the operator to log in again. The keypair (or the
105
+ * developer key) is good for a new session, so one is taken rather than losing the run.
84
106
  */
85
107
  async reauthenticate() {
86
- if (!this.devKey) return false;
87
- const fresh = await NodeApiClient.login(this.baseUrl, this.devKey).catch(() => null);
108
+ if (!this.devKey && !this.privateKey) return false;
109
+ const fresh = await NodeApiClient.login(this.baseUrl, this.devKey, this.privateKey).catch(() => null);
88
110
  if (!fresh) return false;
89
111
  this.token = fresh.token;
90
112
  if (this.adminMode) this.token = (await fresh.post('/users/sessions/current/admin'))?.token ?? this.token;
@@ -12,6 +12,7 @@ var _path = _interopRequireDefault(require("path"));
12
12
  var _papaparse = _interopRequireDefault(require("papaparse"));
13
13
  var color = _interopRequireWildcard(require("./color-utils"));
14
14
  var testUtils = _interopRequireWildcard(require("./test-utils"));
15
+ var keypair = _interopRequireWildcard(require("./keypair"));
15
16
  function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r = new WeakMap(), n = new WeakMap(); return (_interopRequireWildcard = function (e, t) { if (!t && e && e.__esModule) return e; var o, i, f = { __proto__: null, default: e }; if (null === e || "object" != typeof e && "function" != typeof e) return f; if (o = t ? n : r) { if (o.has(e)) return o.get(e); o.set(e, f); } for (const t in e) "default" !== t && {}.hasOwnProperty.call(e, t) && ((i = (o = Object.defineProperty) && Object.getOwnPropertyDescriptor(e, t)) && (i.get || i.set) ? o(f, t, i) : f[t] = e[t]); return f; })(e, t); }
16
17
  function hasPlaywrightTests(pkgDir) {
17
18
  const pkgJsonPath = _path.default.join(pkgDir, 'package.json');
@@ -159,12 +160,17 @@ async function runPlaywrightTests(pkgDir, testDir, args, hostKey) {
159
160
  } catch {
160
161
  webUrl = url.replace(/\/api\/?$/, '');
161
162
  }
163
+
164
+ // The second user the multi-user specs need. Its keypair wins over its developer key,
165
+ // the same way the first user's does; both are explicit, so neither picks up the
166
+ // configured identity by accident.
162
167
  let token2 = '';
163
- if (process.env.DATAGROK_DEV_KEY_2 && process.env.DATAGROK_DEV_KEY_2.length > 0) {
168
+ const privateKey2 = keypair.keyFromEnv('DATAGROK_PRIVATE_KEY_2');
169
+ if (privateKey2 || process.env.DATAGROK_DEV_KEY_2) {
164
170
  try {
165
- token2 = await testUtils.getToken(url, process.env.DATAGROK_DEV_KEY_2);
171
+ token2 = privateKey2 ? await keypair.keyLogin(url, privateKey2) : await testUtils.getToken(url, process.env.DATAGROK_DEV_KEY_2);
166
172
  } catch (e) {
167
- color.warn(`Playwright: DATAGROK_DEV_KEY_2 set but failed to exchange for token: ${e.message || e}`);
173
+ color.warn(`Playwright: second-user credentials set but failed to exchange for token: ${e.message || e}`);
168
174
  }
169
175
  }
170
176
  const configPath = _path.default.join(testDir, 'playwright.config.ts');
@@ -5,7 +5,7 @@ Object.defineProperty(exports, "__esModule", {
5
5
  });
6
6
  exports.createClient = createClient;
7
7
  var _nodeDapi = require("./node-dapi");
8
- var _testUtils = require("./test-utils");
8
+ var _keypair = require("./keypair");
9
9
  /**
10
10
  * `--admin` asks the server for an admin session, which lifts the permission filter for this run:
11
11
  * without it a stand-wide pull sees only what the key's own account can, and content in other
@@ -14,14 +14,13 @@ var _testUtils = require("./test-utils");
14
14
  * the command or reach another session.
15
15
  */
16
16
  async function createClient(hostArg, admin = false) {
17
- const {
18
- url,
19
- key
20
- } = (0, _testUtils.getDevKey)(hostArg ?? '');
21
- const client = await _nodeDapi.NodeApiClient.login(url, key);
17
+ // Resolved from the alias the caller named, not from its URL: two aliases can point at the
18
+ // same server, and only this side knows which one was asked for.
19
+ const cred = (0, _keypair.getServerCredentials)(hostArg ?? '');
20
+ const client = await _nodeDapi.NodeApiClient.login(cred.url, cred.key ?? '', cred.privateKey);
22
21
  if (!admin) return client;
23
22
  const token = (await client.post('/users/sessions/current/admin'))?.token;
24
- if (!token) throw new Error(`${url} refused an admin session — the account behind this key cannot start one`);
23
+ if (!token) throw new Error(`${cred.url} refused an admin session — the account behind this key cannot start one`);
25
24
  client.token = token;
26
25
  client.adminMode = true;
27
26
  return client;
@@ -35,6 +35,8 @@ var utils = _utils;
35
35
  var _puppeteer = _interopRequireDefault(require("puppeteer"));
36
36
  var color = _interopRequireWildcard(require("../utils/color-utils"));
37
37
  var _papaparse = _interopRequireDefault(require("papaparse"));
38
+ var _devKey = require("./dev-key");
39
+ var keypair = _interopRequireWildcard(require("./keypair"));
38
40
  function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r = new WeakMap(), n = new WeakMap(); return (_interopRequireWildcard = function (e, t) { if (!t && e && e.__esModule) return e; var o, i, f = { __proto__: null, default: e }; if (null === e || "object" != typeof e && "function" != typeof e) return f; if (o = t ? n : r) { if (o.has(e)) return o.get(e); o.set(e, f); } for (const t in e) "default" !== t && {}.hasOwnProperty.call(e, t) && ((i = (o = Object.defineProperty) && Object.getOwnPropertyDescriptor(e, t)) && (i.get || i.set) ? o(f, t, i) : f[t] = e[t]); return f; })(e, t); }
39
41
  const fetch = require('node-fetch');
40
42
  const grokDir = _path.default.join(_os.default.homedir(), '.grok');
@@ -54,10 +56,25 @@ async function getToken(url, key) {
54
56
  // auth failure (valid JSON with isSuccess=false) is returned immediately, no retry.
55
57
  const maxAttempts = 15;
56
58
  const delayMs = 3000;
59
+ // Keypair login is the supported path; the developer key remains as a fallback
60
+ // for stands and CI secrets that have not been migrated yet — including when a key is
61
+ // configured but that stand does not know it, which must not take the run down while a
62
+ // working dev key is right there.
63
+ let privateKey = keypair.keypairFor(url, key);
57
64
  let lastError;
58
65
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
59
66
  try {
60
- const response = await fetch(`${url}/users/login/dev/${key}`, {
67
+ if (privateKey) {
68
+ try {
69
+ return await keypair.keyLogin(url, privateKey);
70
+ } catch (error) {
71
+ const refused = error?.name === 'ServerTooOldError' || error?.message?.startsWith('Key login failed');
72
+ if (!key || !refused) throw error;
73
+ color.warn(`${url}: ${error.message} Falling back to the developer key.`);
74
+ privateKey = undefined;
75
+ }
76
+ }
77
+ const response = await (0, _devKey.devKeyFetch)(`${url}/users/login/dev`, `${url}/users/login/dev/${key}`, key, {
61
78
  method: 'POST'
62
79
  });
63
80
  const text = await response.text();
@@ -79,12 +96,14 @@ async function getToken(url, key) {
79
96
  throw new Error('Unable to login to server. Check your dev key');
80
97
  } catch (error) {
81
98
  if (error?.message === 'Unable to login to server. Check your dev key') throw error;
99
+ // A rejected signature is a credential problem, not a readiness one.
100
+ if (error?.message?.startsWith('Key login failed')) throw error;
82
101
  lastError = error;
83
102
  if (utils.isConnectivityError(error)) color.warn(`Playwright: server not reachable yet (attempt ${attempt}/${maxAttempts}): ${url}`);
84
103
  if (attempt < maxAttempts) await new Promise(r => setTimeout(r, delayMs));
85
104
  }
86
105
  }
87
- throw lastError ?? new Error(`Unable to exchange dev key for token at ${url}`);
106
+ throw lastError ?? new Error(`Unable to obtain a token from ${url}`);
88
107
  }
89
108
  async function isPackageOnServer(hostKey, packageName) {
90
109
  try {
package/package.json CHANGED
@@ -4,7 +4,7 @@
4
4
  "type": "git",
5
5
  "url": "https://github.com/datagrok-ai/public.git"
6
6
  },
7
- "version": "6.5.10",
7
+ "version": "6.7.0",
8
8
  "description": "Utility to upload and publish packages to Datagrok",
9
9
  "homepage": "https://github.com/datagrok-ai/public/tree/master/tools#readme",
10
10
  "dependencies": {
@@ -36,7 +36,7 @@
36
36
  "update:ivp-parser": "esbuild plugins/ivp-parser.entry.mjs --bundle --format=cjs --platform=node --alias:diff-grok=../libraries/compute-utils/node_modules/diff-grok --outfile=plugins/ivp-parser.bundle.cjs",
37
37
  "debug-source-map": "node build.js --source-maps",
38
38
  "test": "vitest run --project unit",
39
- "test:server": "vitest run --project unit bin/__tests__/node-dapi bin/__tests__/server bin/__tests__/migrate",
39
+ "test:server": "vitest run --project unit bin/__tests__/node-dapi bin/__tests__/server bin/__tests__/migrate bin/__tests__/keypair",
40
40
  "test:watch": "vitest --project unit",
41
41
  "test:integration": "vitest run --project integration",
42
42
  "test:all": "vitest run"