datagrok-tools 6.7.0 → 6.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.devcontainer/entrypoint.sh +0 -0
- package/.turbo/turbo-build.log +5 -0
- package/CHANGELOG.md +2 -0
- package/CLAUDE.md +5 -6
- package/LICENSE.md +16 -0
- package/README.md +5 -3
- package/bin/commands/build.js +123 -139
- package/bin/commands/check.js +3 -4
- package/bin/commands/create.js +37 -11
- package/bin/commands/help.js +59 -17
- package/bin/commands/link.js +10 -0
- package/bin/commands/publish.js +12 -13
- package/bin/commands/setup.js +139 -0
- package/bin/commands/tsc.js +26 -0
- package/bin/grok.js +34 -21
- package/bin/utils/toolchain.js +121 -0
- package/bin/utils/utils.js +9 -0
- package/package-template/.vscode/tasks.json +1 -1
- package/package-template/package.json +9 -14
- package/package-template/tsconfig.json +2 -69
- package/package.json +17 -17
- package/plugins/func-gen-plugin.js +9 -4
- package/turbo.json +9 -0
- package/package-template/.eslintrc.json +0 -38
- package/package-template/ts.webpack.config.js +0 -46
- package/package-template/webpack.config.js +0 -35
|
File without changes
|
package/CHANGELOG.md
CHANGED
|
@@ -22,6 +22,8 @@
|
|
|
22
22
|
|
|
23
23
|
## 6.5.8 (2026-09-10)
|
|
24
24
|
|
|
25
|
+
* `grok setup [--check] [--global]`: gets a public/ checkout ready (Node check, pnpm via corepack at the pinned version, removal of npm-era per-package node_modules and lockfiles, `pnpm install`, global grok version check). `grok tsc ...` runs the workspace TypeScript compiler; `grok build` inside a package or a Turborepo task builds just that package (bundle + metadata + check, or a library emit). `grok --version`.
|
|
26
|
+
* Workspace toolchain: `public/` is a pnpm workspace with a Turborepo task graph and the shared `@datagrok/build-config` package (rspack + swc bundling, TypeScript 7 type-checking). `grok build` is now a front for Turborepo: the package in cwd and its dependencies in order, cached, never prompting; `--all`, `--affected`, `--typecheck`, `--filter`. `grok link` is a no-op inside a workspace (`pnpm install` links everything). `grok check` accepts `workspace:` and `catalog:` for the `datagrok-api` range. `grok publish` recognises packages built by `grok build --skip-check` (no `webpack.config.js`). `grok create` emits the new package shape (runtime deps only, `extends` tsconfig, four scripts); outside a workspace it pins published ranges and adds `@datagrok/build-config` as a devDependency. Command modules load lazily: `grok` starts in ~0.6 s instead of ~6 s. The webpack `FuncGeneratorPlugin` is also used by rspack builds; `@babel/generator` is a declared dependency now.
|
|
25
27
|
* `grok s` — correctness pass from the CLI audit: every server failure now exits 1 with one line on stderr (a 200 carrying an `ApiError` — missing entity, duplicate login or group name, unknown function — used to print as data with exit 0); `--limit`/`--offset` are honoured for users, groups, connections and functions (their public routes ignore paging, so `list` and the new `count` verb go through the internal routers); `files list` works (the public files route only downloads; listing goes through the connection's file route) and prints `path, kind, size`; `tables list|get|delete` no longer throw, and `tables download|get|delete` accept the bare table name, the full `Project:Table` name or the id; `describe <entity|type>` works (registry record plus the fields of a live sample; it called a nonexistent endpoint before); positional `functions run 'F(1, 2)'` maps the values onto the function's inputs instead of silently sending `{"0": 1}` (which ran the function with nulls); `raw` takes API-relative paths (a leading `/api` is accepted), sends a body from `--json <file>` or `--data '<json>'`, and fails on non-2xx — so it, and `healthcheck`, now work against a bare-Datlas `--host` too; `shares list` accepts a name and shows the grants inherited through project links; `groups` name resolution prefers an exact match (`admin` no longer collides with `Administrators`) and `--user` works on `list-members`/`list-memberships`; `connections delete` of an unknown id fails instead of reporting success; a bad `--host` URL or alias reports the reason without the help dump. `users delete` refuses with a pointer at `users block`: the platform has no user deletion, and removing the entity record leaves the login unusable. `functions delete` covers scripts and queries.
|
|
26
28
|
|
|
27
29
|
* GROK-20298: `grok s domains` — entity-mapped domain schemas and their rows from the CLI, addressed as `<schema>` or `<schema>.<table>`: `list`, `get` (manifest, a table's columns, or one row), `query` (`--filter`, `--columns`, `--sort`, `--expand`, `--limit`, `--offset`), `count`, `aggregate` (`--measures 'count,sum(x) as t' --group-by a`), `insert` / `update` (from `--json` or inline `col=value`, `--version` for optimistic concurrency), `delete` (one row, `--filter` bulk, or a whole user schema with `--force`), `upload` (csv, d42, json; `--upsert`, `--no-all-or-nothing`), `download` (csv to stdout or `-O out.csv|out.d42`), `transaction`, `audit`, `capabilities`, `grants` / `grant` / `revoke` (groups by name), `create`, and `apply` (`--dry-run`, `--confirm-destructive`, `--if-version`). Validation errors print per row; a destructive plan awaiting confirmation is printed with the error.
|
package/CLAUDE.md
CHANGED
|
@@ -141,12 +141,11 @@ Tests use Puppeteer for headless browser automation:
|
|
|
141
141
|
|
|
142
142
|
### `grok build` Command
|
|
143
143
|
|
|
144
|
-
|
|
145
|
-
-
|
|
146
|
-
-
|
|
147
|
-
- `--
|
|
148
|
-
-
|
|
149
|
-
- `--no-incremental` - Force full rebuild (default uses `--env incremental`)
|
|
144
|
+
A front for Turborepo inside the `public/` pnpm workspace (`pnpm exec turbo run build ...`), never prompting:
|
|
145
|
+
- `grok build` (from a package directory): the package and everything it depends on, in dependency order, cached
|
|
146
|
+
- `grok build --all`: the whole workspace; `grok build --affected`: everything the diff against `origin/master` touches
|
|
147
|
+
- `--typecheck` adds the type-check task; `--filter <turbo filter>`; `--parallel N` (default 3); `--force` ignores the cache
|
|
148
|
+
- Outside a workspace it explains that `pnpm run build` / `npm run build` in the package is the way
|
|
150
149
|
|
|
151
150
|
### `grok claude` Command
|
|
152
151
|
|
package/LICENSE.md
ADDED
|
@@ -0,0 +1,16 @@
|
|
|
1
|
+
Copyright 2020 Datagrok, INC
|
|
2
|
+
|
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and
|
|
4
|
+
associated documentation files (the "Software"), to deal in the Software without restriction,
|
|
5
|
+
including without limitation the rights to use, copy, modify, merge, publish, distribute,
|
|
6
|
+
sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is
|
|
7
|
+
furnished to do so, subject to the following conditions:
|
|
8
|
+
|
|
9
|
+
The above copyright notice and this permission notice shall be included in all copies or substantial
|
|
10
|
+
portions of the Software.
|
|
11
|
+
|
|
12
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT
|
|
13
|
+
NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
|
14
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES
|
|
15
|
+
OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
|
|
16
|
+
CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
package/README.md
CHANGED
|
@@ -26,7 +26,9 @@ npm install datagrok-tools -g
|
|
|
26
26
|
|
|
27
27
|
A new folder `<package-name>` will be created automatically as well as its contents.
|
|
28
28
|
3. Run `npm install` in your package directory to get the required dependencies (the command is called
|
|
29
|
-
automatically after package creation, if it ran successfully, skip this step).
|
|
29
|
+
automatically after package creation, if it ran successfully, skip this step). Inside the
|
|
30
|
+
[public repository](https://github.com/datagrok-ai/public) the packages form one pnpm workspace: run
|
|
31
|
+
`pnpm install` once at the repository root instead, and build with `grok build`.
|
|
30
32
|
4. Start working on the functionality of your package. Use `grok add` to create function templates.
|
|
31
33
|
5. Once you have completed the work on your package, upload it by running:
|
|
32
34
|
|
|
@@ -92,8 +94,8 @@ For more information on configuring connections, refer to the [Connections](http
|
|
|
92
94
|
```
|
|
93
95
|
|
|
94
96
|
Options:
|
|
95
|
-
-
|
|
96
|
-
|
|
97
|
+
- the package is built locally before upload (`npm run build`, or `pnpm run build` inside the workspace);
|
|
98
|
+
`--skip-build` uploads the existing `dist/` as is, `--rebuild` asks the server to build instead
|
|
97
99
|
- `--debug` or `--release`: boolean flags that determine whether to publish a debug version of the package visible
|
|
98
100
|
only to the developer or a release version accessible by all eligible users and user groups
|
|
99
101
|
- `--key`: a string containing a developer key that is not listed in the config file, e.g., the key for a new server
|
package/bin/commands/build.js
CHANGED
|
@@ -8,74 +8,136 @@ exports.applyFilter = applyFilter;
|
|
|
8
8
|
exports.build = build;
|
|
9
9
|
exports.confirm = confirm;
|
|
10
10
|
exports.discoverPackages = discoverPackages;
|
|
11
|
+
exports.findWorkspaceRoot = findWorkspaceRoot;
|
|
11
12
|
exports.getNestedValue = getNestedValue;
|
|
12
13
|
var _fs = _interopRequireDefault(require("fs"));
|
|
13
14
|
var _path = _interopRequireDefault(require("path"));
|
|
14
|
-
var _util = require("util");
|
|
15
|
-
var _child_process = require("child_process");
|
|
16
15
|
var readline = _interopRequireWildcard(require("readline"));
|
|
17
|
-
var
|
|
16
|
+
var _child_process = require("child_process");
|
|
18
17
|
var color = _interopRequireWildcard(require("../utils/color-utils"));
|
|
18
|
+
var _toolchain = require("../utils/toolchain");
|
|
19
19
|
function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r = new WeakMap(), n = new WeakMap(); return (_interopRequireWildcard = function (e, t) { if (!t && e && e.__esModule) return e; var o, i, f = { __proto__: null, default: e }; if (null === e || "object" != typeof e && "function" != typeof e) return f; if (o = t ? n : r) { if (o.has(e)) return o.get(e); o.set(e, f); } for (const t in e) "default" !== t && {}.hasOwnProperty.call(e, t) && ((i = (o = Object.defineProperty) && Object.getOwnPropertyDescriptor(e, t)) && (i.get || i.set) ? o(f, t, i) : f[t] = e[t]); return f; })(e, t); }
|
|
20
|
-
|
|
20
|
+
/** Walks up from `dir` to the pnpm workspace root (the directory holding pnpm-workspace.yaml). */
|
|
21
|
+
function findWorkspaceRoot(dir) {
|
|
22
|
+
let d = _path.default.resolve(dir);
|
|
23
|
+
for (;;) {
|
|
24
|
+
if (_fs.default.existsSync(_path.default.join(d, 'pnpm-workspace.yaml'))) return d;
|
|
25
|
+
const parent = _path.default.dirname(d);
|
|
26
|
+
if (parent === d) return null;
|
|
27
|
+
d = parent;
|
|
28
|
+
}
|
|
29
|
+
}
|
|
30
|
+
|
|
31
|
+
/**
|
|
32
|
+
* `grok build` in a workspace drives Turborepo: the package in cwd and everything it depends on,
|
|
33
|
+
* in dependency order, cached (`--all`, `--affected`, `--typecheck`, `--filter`). Inside a Turborepo
|
|
34
|
+
* task (TURBO_HASH is set), in a standalone package, or with `--local`, it builds just the package in
|
|
35
|
+
* cwd: an rspack bundle plus `grok check` for a plugin (src/package.ts or rspack.config.js), a
|
|
36
|
+
* TypeScript emit into dist/ for a library.
|
|
37
|
+
*/
|
|
21
38
|
async function build(args) {
|
|
22
|
-
|
|
23
|
-
const
|
|
24
|
-
if (args.
|
|
39
|
+
const cwd = process.cwd();
|
|
40
|
+
const root = findWorkspaceRoot(cwd);
|
|
41
|
+
if (args.local || process.env.TURBO_HASH || !root) return buildLocal(cwd, args);
|
|
42
|
+
return buildWithTurbo(cwd, root, args);
|
|
25
43
|
}
|
|
26
|
-
async function
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
await utils.runScript('npm install', dir, color.isVerbose());
|
|
36
|
-
await utils.runScript(buildCmd, dir, color.isVerbose());
|
|
37
|
-
color.success(`Successfully built ${name}`);
|
|
38
|
-
return true;
|
|
39
|
-
} catch (error) {
|
|
40
|
-
color.error(`Failed to build ${name}`);
|
|
41
|
-
if (error.message) color.error(error.message);
|
|
42
|
-
return false;
|
|
44
|
+
async function buildWithTurbo(cwd, root, args) {
|
|
45
|
+
const filters = [];
|
|
46
|
+
if (args.affected) filters.push('--filter=...[origin/master]');else if (!(args.all || args.recursive)) {
|
|
47
|
+
const pkgPath = _path.default.join(cwd, 'package.json');
|
|
48
|
+
if (!_fs.default.existsSync(pkgPath)) {
|
|
49
|
+
color.error('Not a package directory (no package.json). Use --all or --affected from anywhere in the workspace.');
|
|
50
|
+
return false;
|
|
51
|
+
}
|
|
52
|
+
filters.push(`--filter=${JSON.parse(_fs.default.readFileSync(pkgPath, 'utf8')).name}...`);
|
|
43
53
|
}
|
|
54
|
+
if (args.filter) filters.push(`--filter=${args.filter}`);
|
|
55
|
+
const tasks = ['build'];
|
|
56
|
+
if (args.typecheck) tasks.push('typecheck');
|
|
57
|
+
const cmd = ['exec', 'turbo', 'run', ...tasks, ...filters, `--concurrency=${args.parallel ?? 3}`, '--continue'];
|
|
58
|
+
if (!args.verbose) cmd.push('--output-logs=errors-only');
|
|
59
|
+
if (args.force) cmd.push('--force');
|
|
60
|
+
const r = (0, _child_process.spawnSync)('pnpm', cmd, {
|
|
61
|
+
stdio: 'inherit',
|
|
62
|
+
cwd: root,
|
|
63
|
+
shell: true,
|
|
64
|
+
env: {
|
|
65
|
+
...process.env,
|
|
66
|
+
TURBO_TELEMETRY_DISABLED: '1'
|
|
67
|
+
}
|
|
68
|
+
});
|
|
69
|
+
return r.status === 0;
|
|
44
70
|
}
|
|
45
|
-
async function
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
color.warn('No packages found in the current directory');
|
|
71
|
+
async function buildLocal(dir, args) {
|
|
72
|
+
if (!_fs.default.existsSync(_path.default.join(dir, 'package.json'))) {
|
|
73
|
+
color.error('Not a package directory (no package.json).');
|
|
49
74
|
return false;
|
|
50
75
|
}
|
|
51
|
-
const
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
76
|
+
const isPlugin = _fs.default.existsSync(_path.default.join(dir, 'rspack.config.js')) || ['ts', 'js'].some(e => _fs.default.existsSync(_path.default.join(dir, 'src', `package.${e}`)));
|
|
77
|
+
return isPlugin ? bundle(dir, args) : emitLibrary(dir);
|
|
78
|
+
}
|
|
79
|
+
async function bundle(dir, args) {
|
|
80
|
+
const {
|
|
81
|
+
bundler,
|
|
82
|
+
rspack
|
|
83
|
+
} = (0, _toolchain.toolchain)(dir).buildConfig;
|
|
84
|
+
// --key=value arguments reach a config exported as a function, e.g. `grok build --only=browser`
|
|
85
|
+
const env = {};
|
|
86
|
+
for (const [k, v] of Object.entries(args)) if (!['_', 'local', 'verbose', 'skip-check', 'typecheck', 'all', 'affected', 'filter', 'parallel', 'force'].includes(k)) env[k] = v;
|
|
87
|
+
const cfgPath = _path.default.join(dir, 'rspack.config.js');
|
|
88
|
+
let config = _fs.default.existsSync(cfgPath) ? require(cfgPath) : bundler({
|
|
89
|
+
dir
|
|
90
|
+
});
|
|
91
|
+
if (typeof config === 'function') config = config(env);
|
|
92
|
+
if (config && typeof config.then === 'function') config = await config;
|
|
93
|
+
if (!args['skip-check'] && _fs.default.existsSync(_path.default.join(dir, 'src', 'package.ts'))) {
|
|
94
|
+
const {
|
|
95
|
+
check
|
|
96
|
+
} = require('./check');
|
|
97
|
+
if (check({
|
|
98
|
+
_: ['check'],
|
|
99
|
+
soft: true,
|
|
100
|
+
'no-exit': true
|
|
101
|
+
}) === false) {
|
|
102
|
+
color.error('grok check failed');
|
|
61
103
|
return false;
|
|
62
104
|
}
|
|
63
105
|
}
|
|
64
|
-
const
|
|
65
|
-
const
|
|
66
|
-
|
|
106
|
+
const t0 = Date.now();
|
|
107
|
+
const ok = await new Promise((resolve, reject) => {
|
|
108
|
+
rspack(config, (err, stats) => {
|
|
109
|
+
if (err) return reject(err);
|
|
110
|
+
const text = stats.toString({
|
|
111
|
+
colors: process.stdout.isTTY,
|
|
112
|
+
preset: 'errors-warnings'
|
|
113
|
+
});
|
|
114
|
+
if (text.trim()) console.log(text);
|
|
115
|
+
resolve(!stats.hasErrors());
|
|
116
|
+
});
|
|
117
|
+
});
|
|
118
|
+
const dist = _path.default.join(dir, 'dist', 'package.js');
|
|
119
|
+
const size = _fs.default.existsSync(dist) ? `${(_fs.default.statSync(dist).size / 1024).toFixed(0)} KB` : '';
|
|
120
|
+
console.log(`${ok ? 'bundled' : 'FAILED'} ${_path.default.basename(dir)} in ${((Date.now() - t0) / 1000).toFixed(1)}s ${size}`);
|
|
121
|
+
return ok;
|
|
67
122
|
}
|
|
123
|
+
|
|
124
|
+
/** A library: `tsc -p tsconfig.json` into dist/, then the css/wasm/json assets its sources import. */
|
|
125
|
+
function emitLibrary(dir) {
|
|
126
|
+
(0, _toolchain.ensureLibraryExports)(dir);
|
|
127
|
+
const r = (0, _child_process.spawnSync)(process.execPath, [(0, _toolchain.toolchain)(dir).tsc, '-p', 'tsconfig.json'], {
|
|
128
|
+
stdio: 'inherit',
|
|
129
|
+
cwd: dir
|
|
130
|
+
});
|
|
131
|
+
if (r.status !== 0) return false;
|
|
132
|
+
(0, _toolchain.copyLibraryAssets)(dir);
|
|
133
|
+
return true;
|
|
134
|
+
}
|
|
135
|
+
/** Packages directly under `baseDir` (used by `grok testall`). */
|
|
68
136
|
function discoverPackages(baseDir) {
|
|
69
|
-
const entries = _fs.default.readdirSync(baseDir);
|
|
70
137
|
const packages = [];
|
|
71
|
-
for (const entry of
|
|
138
|
+
for (const entry of _fs.default.readdirSync(baseDir)) {
|
|
72
139
|
if (entry.startsWith('.') || entry === 'node_modules') continue;
|
|
73
140
|
const dir = _path.default.join(baseDir, entry);
|
|
74
|
-
try {
|
|
75
|
-
if (!_fs.default.statSync(dir).isDirectory()) continue;
|
|
76
|
-
} catch (_) {
|
|
77
|
-
continue;
|
|
78
|
-
}
|
|
79
141
|
const packageJsonPath = _path.default.join(dir, 'package.json');
|
|
80
142
|
if (!_fs.default.existsSync(packageJsonPath)) continue;
|
|
81
143
|
try {
|
|
@@ -87,114 +149,36 @@ function discoverPackages(baseDir) {
|
|
|
87
149
|
version: packageJson.version || '0.0.0',
|
|
88
150
|
packageJson
|
|
89
151
|
});
|
|
90
|
-
} catch (_) {
|
|
91
|
-
continue;
|
|
92
|
-
}
|
|
152
|
+
} catch (_) {/* not a package */}
|
|
93
153
|
}
|
|
94
154
|
return packages.sort((a, b) => a.friendlyName.localeCompare(b.friendlyName));
|
|
95
155
|
}
|
|
156
|
+
|
|
157
|
+
/** `field:regex && field:regex` filter over package.json fields. */
|
|
96
158
|
function applyFilter(packages, filterStr) {
|
|
97
|
-
const conditions = filterStr.split('&&').map(s => s.trim())
|
|
98
|
-
const parsedConditions = conditions.map(cond => {
|
|
159
|
+
const conditions = filterStr.split('&&').map(s => s.trim()).map(cond => {
|
|
99
160
|
const colonIdx = cond.indexOf(':');
|
|
100
|
-
|
|
161
|
+
return colonIdx === -1 ? {
|
|
101
162
|
field: cond,
|
|
102
163
|
pattern: new RegExp('.')
|
|
164
|
+
} : {
|
|
165
|
+
field: cond.substring(0, colonIdx).trim(),
|
|
166
|
+
pattern: new RegExp(cond.substring(colonIdx + 1).trim())
|
|
103
167
|
};
|
|
104
|
-
const field = cond.substring(0, colonIdx).trim();
|
|
105
|
-
const pattern = new RegExp(cond.substring(colonIdx + 1).trim());
|
|
106
|
-
return {
|
|
107
|
-
field,
|
|
108
|
-
pattern
|
|
109
|
-
};
|
|
110
|
-
});
|
|
111
|
-
return packages.filter(pkg => {
|
|
112
|
-
for (const cond of parsedConditions) {
|
|
113
|
-
const value = getNestedValue(pkg.packageJson, cond.field);
|
|
114
|
-
if (value === undefined || !cond.pattern.test(String(value))) return false;
|
|
115
|
-
}
|
|
116
|
-
return true;
|
|
117
168
|
});
|
|
169
|
+
return packages.filter(pkg => conditions.every(cond => {
|
|
170
|
+
const value = getNestedValue(pkg.packageJson, cond.field);
|
|
171
|
+
return value !== undefined && cond.pattern.test(String(value));
|
|
172
|
+
}));
|
|
118
173
|
}
|
|
119
174
|
function getNestedValue(obj, path) {
|
|
120
|
-
const parts = path.split('.');
|
|
121
175
|
let current = obj;
|
|
122
|
-
for (const part of
|
|
176
|
+
for (const part of path.split('.')) {
|
|
123
177
|
if (current == null || typeof current !== 'object') return undefined;
|
|
124
178
|
current = current[part];
|
|
125
179
|
}
|
|
126
180
|
return current;
|
|
127
181
|
}
|
|
128
|
-
async function buildParallel(packages, buildCmd, maxParallel) {
|
|
129
|
-
const results = [];
|
|
130
|
-
const headers = ['Plugin', 'Version', 'Build time', 'Bundle size'];
|
|
131
|
-
const widths = [Math.max(headers[0].length, ...packages.map(p => p.friendlyName.length)), Math.max(headers[1].length, ...packages.map(p => p.version.length)), Math.max(headers[2].length, 10), Math.max(headers[3].length, 40)];
|
|
132
|
-
const pad = (s, w) => s + ' '.repeat(Math.max(0, w - s.length));
|
|
133
|
-
console.log(`\nBuilding with ${maxParallel} parallel job(s)...`);
|
|
134
|
-
console.log(headers.map((h, i) => pad(h, widths[i])).join(' | '));
|
|
135
|
-
console.log(widths.map(w => '-'.repeat(w)).join('-+-'));
|
|
136
|
-
const buildOne = async pkg => {
|
|
137
|
-
const start = Date.now();
|
|
138
|
-
let success = true;
|
|
139
|
-
let buildTime = '';
|
|
140
|
-
let bundleSize = '';
|
|
141
|
-
try {
|
|
142
|
-
await execAsync('npm install', {
|
|
143
|
-
cwd: pkg.dir,
|
|
144
|
-
maxBuffer: 10 * 1024 * 1024
|
|
145
|
-
});
|
|
146
|
-
await execAsync(buildCmd, {
|
|
147
|
-
cwd: pkg.dir,
|
|
148
|
-
maxBuffer: 10 * 1024 * 1024
|
|
149
|
-
});
|
|
150
|
-
const elapsed = (Date.now() - start) / 1000;
|
|
151
|
-
buildTime = `${elapsed.toFixed(1)}s`;
|
|
152
|
-
bundleSize = getBundleSize(pkg.dir);
|
|
153
|
-
} catch (error) {
|
|
154
|
-
success = false;
|
|
155
|
-
buildTime = 'Error';
|
|
156
|
-
const raw = (error.stderr || error.stdout || error.message || 'Unknown error').trim();
|
|
157
|
-
bundleSize = raw.replace(/\r?\n/g, ' ').replace(/\s+/g, ' ').substring(0, 40);
|
|
158
|
-
}
|
|
159
|
-
const result = {
|
|
160
|
-
name: pkg.friendlyName,
|
|
161
|
-
version: pkg.version,
|
|
162
|
-
buildTime,
|
|
163
|
-
bundleSize,
|
|
164
|
-
success
|
|
165
|
-
};
|
|
166
|
-
results.push(result);
|
|
167
|
-
const cells = [result.name, result.version, result.buildTime, result.bundleSize];
|
|
168
|
-
const line = cells.map((cell, j) => pad(cell, widths[j])).join(' | ');
|
|
169
|
-
if (success) color.info(line);else color.error(line);
|
|
170
|
-
return result;
|
|
171
|
-
};
|
|
172
|
-
let idx = 0;
|
|
173
|
-
const next = async () => {
|
|
174
|
-
while (idx < packages.length) {
|
|
175
|
-
const pkg = packages[idx++];
|
|
176
|
-
await buildOne(pkg);
|
|
177
|
-
}
|
|
178
|
-
};
|
|
179
|
-
const workers = Array.from({
|
|
180
|
-
length: Math.min(maxParallel, packages.length)
|
|
181
|
-
}, () => next());
|
|
182
|
-
await Promise.all(workers);
|
|
183
|
-
const succeeded = results.filter(r => r.success).length;
|
|
184
|
-
const failed = results.length - succeeded;
|
|
185
|
-
console.log('');
|
|
186
|
-
if (failed === 0) color.success(`All ${results.length} package(s) built successfully`);else color.warn(`${succeeded} succeeded, ${failed} failed`);
|
|
187
|
-
return results;
|
|
188
|
-
}
|
|
189
|
-
function getBundleSize(dir) {
|
|
190
|
-
const bundlePath = _path.default.join(dir, 'dist', 'package.js');
|
|
191
|
-
try {
|
|
192
|
-
const stats = _fs.default.statSync(bundlePath);
|
|
193
|
-
return `${(stats.size / 1024).toFixed(1)} KB`;
|
|
194
|
-
} catch (_) {
|
|
195
|
-
return 'N/A';
|
|
196
|
-
}
|
|
197
|
-
}
|
|
198
182
|
function confirm(message) {
|
|
199
183
|
const rl = readline.createInterface({
|
|
200
184
|
input: process.stdin,
|
package/bin/commands/check.js
CHANGED
|
@@ -19,7 +19,6 @@ var _path = _interopRequireDefault(require("path"));
|
|
|
19
19
|
var _ignoreWalk = _interopRequireDefault(require("ignore-walk"));
|
|
20
20
|
var utils = _interopRequireWildcard(require("../utils/utils"));
|
|
21
21
|
var color = _interopRequireWildcard(require("../utils/color-utils"));
|
|
22
|
-
var testUtils = _interopRequireWildcard(require("../utils/test-utils"));
|
|
23
22
|
var _const = require("../utils/const");
|
|
24
23
|
function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r = new WeakMap(), n = new WeakMap(); return (_interopRequireWildcard = function (e, t) { if (!t && e && e.__esModule) return e; var o, i, f = { __proto__: null, default: e }; if (null === e || "object" != typeof e && "function" != typeof e) return f; if (o = t ? n : r) { if (o.has(e)) return o.get(e); o.set(e, f); } for (const t in e) "default" !== t && {}.hasOwnProperty.call(e, t) && ((i = (o = Object.defineProperty) && Object.getOwnPropertyDescriptor(e, t)) && (i.get || i.set) ? o(f, t, i) : f[t] = e[t]); return f; })(e, t); }
|
|
25
24
|
const warns = ['Latest package version', 'Datagrok API version should contain'];
|
|
@@ -57,7 +56,7 @@ function check(args) {
|
|
|
57
56
|
color.error('File `package.json` not found. Run the command from the package directory');
|
|
58
57
|
return false;
|
|
59
58
|
}
|
|
60
|
-
return runChecks(curDir, args.soft ?? false, false);
|
|
59
|
+
return runChecks(curDir, args.soft ?? false, args['no-exit'] ?? false);
|
|
61
60
|
}
|
|
62
61
|
}
|
|
63
62
|
function runChecks(packagePath, soft = false, noExit = false) {
|
|
@@ -113,7 +112,7 @@ function runChecks(packagePath, soft = false, noExit = false) {
|
|
|
113
112
|
console.log(`Checking package ${_path.default.basename(packagePath)}...`);
|
|
114
113
|
showError(errors);
|
|
115
114
|
if (soft || json.version.startsWith('0') || errors.every(w => warns.some(ww => w.includes(ww)))) return true;
|
|
116
|
-
if (noExit) return false;else
|
|
115
|
+
if (noExit) return false;else process.exit(1);
|
|
117
116
|
}
|
|
118
117
|
color.log(`Checking package ${_path.default.basename(packagePath)}...\t\t\t\u2713 OK`);
|
|
119
118
|
return true;
|
|
@@ -410,7 +409,7 @@ function checkPackageFile(packagePath, json, options) {
|
|
|
410
409
|
if (json.version.includes('beta') && isPublicPackage) warnings.push('File "package.json": public package cannot be beta version.');
|
|
411
410
|
const api = json.dependencies?.['datagrok-api'];
|
|
412
411
|
if (api) {
|
|
413
|
-
if (api === '../../js-api') {} else if (api === 'latest') warnings.push('File "package.json": you should specify Datagrok API version constraint (for example ^1.16.0, >=1.16.0).');else if (options?.isReleaseCandidateVersion === false && !/^(
|
|
412
|
+
if (api === '../../js-api') {} else if (api === 'latest') warnings.push('File "package.json": you should specify Datagrok API version constraint (for example ^1.16.0, >=1.16.0).');else if (options?.isReleaseCandidateVersion === false && !/^(\^|>|<|~|workspace:|catalog:).+/.test(api)) warnings.push('File "package.json": Datagrok API version should starts with > | >= | ~ | ^ | < | <=');
|
|
414
413
|
}
|
|
415
414
|
|
|
416
415
|
// const dt = json.devDependencies?.['datagrok-tools'] ?? json.dependencies?.['datagrok-tools'];
|
package/bin/commands/create.js
CHANGED
|
@@ -21,6 +21,12 @@ const curFolder = _path.default.basename(curDir);
|
|
|
21
21
|
const grokDir = _path.default.join(_os.default.homedir(), '.grok');
|
|
22
22
|
const confPath = _path.default.join(grokDir, 'config.yaml');
|
|
23
23
|
const templateDir = _path.default.join(_path.default.dirname(_path.default.dirname(__dirname)), 'package-template');
|
|
24
|
+
function isPnpmWorkspace(dir) {
|
|
25
|
+
for (let d = _path.default.resolve(dir);; d = _path.default.dirname(d)) {
|
|
26
|
+
if (_fs.default.existsSync(_path.default.join(d, 'pnpm-workspace.yaml'))) return true;
|
|
27
|
+
if (_path.default.dirname(d) === d) return false;
|
|
28
|
+
}
|
|
29
|
+
}
|
|
24
30
|
const confTemplateDir = _path.default.join(_path.default.dirname(_path.default.dirname(__dirname)), 'config-template.yaml');
|
|
25
31
|
const confTemplate = _jsYaml.default.load(_fs.default.readFileSync(confTemplateDir, {
|
|
26
32
|
encoding: 'utf-8'
|
|
@@ -37,7 +43,7 @@ function createDirectoryContents(name, friendlyName, config, templateDir, packag
|
|
|
37
43
|
_fs.default.writeFileSync(copyFilePath, _fs.default.readFileSync(origFilePath, 'base64'), 'base64');
|
|
38
44
|
return false;
|
|
39
45
|
}
|
|
40
|
-
let contents = _fs.default.readFileSync(
|
|
46
|
+
let contents = _fs.default.readFileSync(origFilePath, 'utf8');
|
|
41
47
|
contents = contents.replace(/#{PACKAGE_FRIENDLY_NAME}/g, friendlyName);
|
|
42
48
|
contents = contents.replace(/#{PACKAGE_NAME}/g, name);
|
|
43
49
|
contents = contents.replace(/#{PACKAGE_DETECTORS_NAME}/g, utils.kebabToCamelCase(name));
|
|
@@ -49,20 +55,40 @@ function createDirectoryContents(name, friendlyName, config, templateDir, packag
|
|
|
49
55
|
contents = contents.replace(/#{GROK_HOST_ALIAS}/g, config.default);
|
|
50
56
|
contents = contents.replace(/#{GROK_HOST}/g, /localhost|127\.0\.0\.1/.test(config['servers'][config.default]['url']) ? 'http://localhost:63343/login.html' : new URL(config['servers'][config.default]['url']).origin);
|
|
51
57
|
if (file === 'package.json') {
|
|
52
|
-
//
|
|
58
|
+
// The toolchain (bundler, TypeScript, eslint) comes from the workspace root; a package
|
|
59
|
+
// declares only what it imports. Deploy with `grok publish <alias>`, no per-server scripts.
|
|
53
60
|
const _package = JSON.parse(contents);
|
|
54
61
|
_package['scripts'] = _package['scripts'] ?? {};
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
+
_package.devDependencies = _package.devDependencies ?? {};
|
|
63
|
+
// Outside the public/ pnpm workspace the template's workspace:/catalog: specifiers mean
|
|
64
|
+
// nothing to npm: pin published ranges and bring the toolchain in as a devDependency.
|
|
65
|
+
if (!isPnpmWorkspace(packageDir)) {
|
|
66
|
+
const published = {
|
|
67
|
+
'datagrok-api': `^${require('datagrok-api/package.json').version}`,
|
|
68
|
+
'@datagrok-libraries/test': '^1.4.0',
|
|
69
|
+
'rxjs': '^6.5.5',
|
|
70
|
+
'cash-dom': '^8.1.5',
|
|
71
|
+
'dayjs': '^1.11.13',
|
|
72
|
+
'wu': '^2.1.0',
|
|
73
|
+
'typescript': '^7.0.2'
|
|
74
|
+
};
|
|
75
|
+
for (const sect of ['dependencies', 'devDependencies']) {
|
|
76
|
+
for (const [dep, spec] of Object.entries(_package[sect] ?? {})) {
|
|
77
|
+
if (spec === 'workspace:^' || spec === 'catalog:') _package[sect][dep] = published[dep] ?? 'latest';
|
|
78
|
+
}
|
|
79
|
+
}
|
|
62
80
|
Object.assign(_package.devDependencies, {
|
|
63
|
-
'
|
|
64
|
-
'
|
|
81
|
+
'@datagrok/build-config': '^0.1.0',
|
|
82
|
+
'eslint': '^8.57.1',
|
|
83
|
+
'@typescript-eslint/parser': '^8.39.0',
|
|
84
|
+
'@typescript-eslint/eslint-plugin': '^8.39.0',
|
|
85
|
+
'eslint-config-google': '^0.14.0',
|
|
86
|
+
'typescript': '^5.9.3'
|
|
65
87
|
});
|
|
88
|
+
_fs.default.writeFileSync(_path.default.join(packageDir, '.eslintrc.json'), JSON.stringify({
|
|
89
|
+
root: true,
|
|
90
|
+
extends: './node_modules/@datagrok/build-config/eslintrc.json'
|
|
91
|
+
}, null, 2) + '\n');
|
|
66
92
|
}
|
|
67
93
|
if (eslint) {
|
|
68
94
|
Object.assign(_package.devDependencies, {
|
package/bin/commands/help.js
CHANGED
|
@@ -13,8 +13,10 @@ Datagrok's package management tool
|
|
|
13
13
|
Commands:
|
|
14
14
|
add Add an object template
|
|
15
15
|
api Create wrapper functions
|
|
16
|
-
|
|
17
|
-
|
|
16
|
+
setup Get a public/ checkout ready to build (pnpm, install, npm-era clean-up)
|
|
17
|
+
build Build a package and its dependencies (Turborepo), or just this package
|
|
18
|
+
tsc Run the workspace TypeScript compiler
|
|
19
|
+
check Check package content (function signatures, etc.)
|
|
18
20
|
claude Launch Claude Code in a Datagrok dev container
|
|
19
21
|
config Create and manage config files
|
|
20
22
|
create Create a package
|
|
@@ -209,7 +211,8 @@ Options:
|
|
|
209
211
|
--all Publish all available packages (run in packages directory)
|
|
210
212
|
--refresh Publish all available already loaded packages (run in packages directory)
|
|
211
213
|
--link Link the package to local packages
|
|
212
|
-
--build Builds the package
|
|
214
|
+
--build Builds the package (the default; kept for compatibility)
|
|
215
|
+
--skip-build Upload the existing dist/ without rebuilding
|
|
213
216
|
--release Publish package as release version
|
|
214
217
|
--rebuild-docker Force rebuild Docker images locally before pushing to registry
|
|
215
218
|
--skip-docker-rebuild Skip auto-rebuild when Dockerfile folder has changed
|
|
@@ -308,6 +311,9 @@ https://datagrok.ai/help/develop/how-to/test-packages#local-testing
|
|
|
308
311
|
const HELP_LINK = `
|
|
309
312
|
Usage: grok link
|
|
310
313
|
|
|
314
|
+
In a pnpm workspace checkout (public/) this is a no-op: \`pnpm install\` at the repository root
|
|
315
|
+
links every in-repo dependency (workspace:^). The options below apply to standalone checkouts only.
|
|
316
|
+
|
|
311
317
|
Links \`datagrok-api\`, all necessary libraries and packages for local development.
|
|
312
318
|
Uses \`npm link\` unless the --path option specified.
|
|
313
319
|
By default, it links packages from the parent directory of the repository's root.
|
|
@@ -341,27 +347,61 @@ Example:
|
|
|
341
347
|
⟶
|
|
342
348
|
meta: { role: 'viewer,ml' }
|
|
343
349
|
`;
|
|
350
|
+
const HELP_SETUP = `
|
|
351
|
+
Usage: grok setup [--check] [--global]
|
|
352
|
+
|
|
353
|
+
Gets a public/ checkout (or a fresh worktree) ready to build, and keeps it that way after a pull:
|
|
354
|
+
1. Node 20+ (22 recommended); pnpm through corepack, at the version the workspace pins
|
|
355
|
+
2. removes per-package node_modules left by the npm era and stray package-lock.json files
|
|
356
|
+
3. pnpm install at the workspace root
|
|
357
|
+
4. reports a global grok older than the workspace one
|
|
358
|
+
|
|
359
|
+
--check Report only, change nothing
|
|
360
|
+
--global Also update the global datagrok-tools to the workspace version
|
|
361
|
+
|
|
362
|
+
Examples:
|
|
363
|
+
grok setup After cloning, after a pull that changed dependencies, in a new worktree
|
|
364
|
+
grok setup --check What would change
|
|
365
|
+
`;
|
|
366
|
+
const HELP_TSC = `
|
|
367
|
+
Usage: grok tsc [tsc arguments]
|
|
368
|
+
|
|
369
|
+
Runs the workspace TypeScript compiler (the version @datagrok/build-config pins) with the given arguments,
|
|
370
|
+
so a package never needs its own tsc. After an emitting run in a library, the css/wasm/json assets its
|
|
371
|
+
sources import are mirrored into dist/.
|
|
372
|
+
|
|
373
|
+
Examples:
|
|
374
|
+
grok tsc --noEmit -p tsconfig.json Type-check (what the \`typecheck\` script runs)
|
|
375
|
+
grok tsc -p tsconfig.build.json A library with a custom build config
|
|
376
|
+
`;
|
|
344
377
|
const HELP_BUILD = `
|
|
345
378
|
Usage: grok build
|
|
346
379
|
|
|
347
|
-
|
|
380
|
+
Inside the public/ workspace: build the package in the current directory and everything it depends
|
|
381
|
+
on, in dependency order, through Turborepo (cached: unchanged packages take milliseconds). Never prompts.
|
|
382
|
+
Inside a Turborepo task, in a standalone package, or with --local: build just this package — an rspack
|
|
383
|
+
bundle (src/package.ts -> dist/package.js, function metadata generated, \`grok check --soft\`) for a
|
|
384
|
+
plugin, a TypeScript emit into dist/ for a library.
|
|
348
385
|
|
|
349
386
|
Options:
|
|
350
|
-
[
|
|
351
|
-
|
|
352
|
-
--
|
|
353
|
-
--
|
|
354
|
-
--
|
|
355
|
-
--
|
|
356
|
-
--parallel N Max parallel builds (default:
|
|
357
|
-
--
|
|
387
|
+
[--all] [--affected] [--typecheck] [--filter <turbo filter>] [--parallel N] [--force] [--local] [--skip-check] [-v | --verbose]
|
|
388
|
+
|
|
389
|
+
--all Build the whole workspace
|
|
390
|
+
--affected Build everything the diff against origin/master affects, dependents included
|
|
391
|
+
--typecheck Also run the type-check task
|
|
392
|
+
--filter Extra Turborepo filter (e.g. --filter "@datagrok/chem...")
|
|
393
|
+
--parallel N Max parallel builds (default: 3)
|
|
394
|
+
--force Ignore the build cache
|
|
395
|
+
--local Build only this package, without Turborepo
|
|
396
|
+
--skip-check Local build: skip \`grok check\`
|
|
397
|
+
--verbose Show full build output instead of errors only
|
|
358
398
|
|
|
359
399
|
Examples:
|
|
360
|
-
grok build
|
|
361
|
-
grok build
|
|
362
|
-
grok build
|
|
363
|
-
grok build
|
|
364
|
-
grok build
|
|
400
|
+
grok build This package and its dependencies
|
|
401
|
+
grok build --all Everything
|
|
402
|
+
grok build --affected What my change touches (CI does the same)
|
|
403
|
+
grok build --typecheck Bundle and type-check
|
|
404
|
+
grok build --local Just this package (what the package's build script runs)
|
|
365
405
|
`;
|
|
366
406
|
|
|
367
407
|
// const HELP_MIGRATE = `
|
|
@@ -440,9 +480,11 @@ const help = exports.help = {
|
|
|
440
480
|
report: HELP_REPORT,
|
|
441
481
|
run: HELP_RUN,
|
|
442
482
|
test: HELP_TEST,
|
|
483
|
+
tsc: HELP_TSC,
|
|
443
484
|
testall: HELP_TESTALL,
|
|
444
485
|
migrate: HELP_MIGRATE,
|
|
445
486
|
server: _server.HELP_SERVER,
|
|
446
487
|
s: _server.HELP_SERVER,
|
|
488
|
+
setup: HELP_SETUP,
|
|
447
489
|
help: HELP
|
|
448
490
|
};
|