datagrok-tools 6.7.7 → 6.7.9

package/.devcontainer/PACKAGES_DEV.md

@@ -66,7 +66,7 @@ Host machine
 │  │    │       └── <folder>/     ← bind mount of user's repo   │  │
 │  │    ├── /var/run/docker.sock  ← host Docker for stand mgmt  │  │
 │  │    ├── entrypoint.sh         ← bind-mounted from tools/    │  │
-│  │    ├── Claude Code + MCP plugins (Jira, GitHub)            │  │
+│  │    ├── Claude Code + MCP plugins (GitHub, Jira)            │  │
 │  │    ├── Playwright + Chrome for testing                     │  │
 │  │    └── grok CLI for build/publish/test                     │  │
 │  └────────────────────────────────────────────────────────────┘  │
@@ -197,15 +197,25 @@ Based on `node:22-bookworm-slim`. Pre-installed:
 
 Runs as **root** in compose (entrypoint drops to `node` user for the main process).
 
-## MCP plugins: Jira and GitHub
+## MCP plugins: GitHub and Jira
 
 The agent uses MCP plugins to search for similar issues, read context, and update
-status. Set tokens in `.env` or the agent will ask interactively on first use.
+status. Tickets are GitHub issues (`datagrok-ai/public`); Jira holds only read-only
+`GROK-NNNNN` history. Set tokens in `.env` or the agent will ask interactively on first use.
 
-### Jira (Atlassian)
+### GitHub
 
 ```bash
 # Inside tools-dev, register the MCP server:
+claude mcp add github -s user -- \
+  npx -y @modelcontextprotocol/server-github
+```
+
+Requires `GITHUB_TOKEN` in the environment (already passed from `.env`).
+
+### Jira (optional, read-only history)
+
+```bash
 claude mcp add mcp-atlassian -s user -- \
   npx -y mcp-atlassian \
   --jira-url "$JIRA_URL" \
@@ -213,15 +223,6 @@ claude mcp add mcp-atlassian -s user -- \
   --jira-token "$JIRA_TOKEN"
 ```
 
-### GitHub
-
-```bash
-claude mcp add github -s user -- \
-  npx -y @modelcontextprotocol/server-github
-```
-
-Requires `GITHUB_TOKEN` in the environment (already passed from `.env`).
-
 ## Testing
 
 ### grok test (Puppeteer-based)

package/.turbo/turbo-build.log

@@ -1,5 +1,5 @@
 
-> datagrok-tools@6.7.7 build /home/runner/work/public/public/tools
+> datagrok-tools@6.7.9 build /home/runner/work/public/public/tools
 > node build.js
 
-Successfully compiled 51 files with Babel.
+Successfully compiled 52 files with Babel.

package/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Datagrok-tools changelog
 
+## v.next
+
+* Upgraded Puppeteer to v25, which drops the vulnerable `extract-zip` and `basic-ftp` transitive dependencies; Node.js 22.12+ is now required
+* `grok add app --domain`, `grok api --ui`: scaffold against `@datagrok-libraries/ems`
+* GROK-20884: Added `grok s o errors get <signature|problem>`: each occurrence of an error with its time, user, function and session, so an error incident leads to its calls
+* GROK-20884: Added `grok s o logs container [<name|id>]`: a docker container's log lines cut by `--since`/`--from`/`--to` and `--grep`; no name lists the containers
+* GROK-20884: `grok s o problems get` prints one field per line with every detail in full, and `errors top --limit` returns that many rows instead of the top of every source
+* GROK-20884: Added `grok s observe` (alias `grok s o`): problems and their alerts, problem rules, logger settings with expiring user and group entries, errors, the timeline and cloud logs (CloudWatch and the log archive)
+* GROK-20884: Added `grok s o logs capture <login|group> [--for 30m]`: saves every API request of a user or a group until the entry expires (a `logger set` of `saveHttpRequests`; an entry that expires later keeps its time); `logs capture stop` removes an expiring entry that holds only the capture, `logs capture` lists the active ones
+* GROK-20298: `grok s domains apply` sends the manifest JSON's `version` as the now-required `ifVersion` of a user-managed schema, accepts `--if-version current` to take the registered one, refuses such an apply without either, and forwards `friendlyName`, `description` and `storage` from the JSON for a user-managed schema (a package schema keeps its key set)
+* GROK-20298: `domain-schema.schema.json` accepts the root `storage` key (`{kind: external, connection, schema?, catalog?}`) and the remote-name keys `table`/`column`; `grok api` types an external table's rows with `id` only, an empty expand map, and emits no `<Schema>TransactionOp`/`transaction` for an external schema
+
+## 6.7.8 (2026-10-13)
+
+* Fixed `grok api` (and every command other than `grok s`) exiting with code 1 and its usage text in 6.7.7: the new `--send-email` flag reached every command as `send-email: false`, which `grok api` rejected as an unknown option
+
 ## 6.7.7 (2026-10-09)
 
 * GROK-21156: `grok s users invite [<email>] [--prefix <P>] [--send-email]`: invites a user and prints a link that logs them in (no email sent; the email may be omitted); `--prefix` starts the code with `<P>`; `--send-email` mails the invitation instead

package/CLAUDE.md

@@ -121,6 +121,18 @@ When a package has a `dockerfiles/` directory, `grok publish` automatically mana
 
 **Practical implication:** editing any file inside `dockerfiles/<name>/` will cause `grok publish` to rebuild that container image on the next publish, without needing `--rebuild-docker`.
 
+**Publish traps:**
+- `grok publish` runs `npm run build` / `pnpm run build` itself (`buildPackage`); `--skip-check`
+  only skips checks, and `--rebuild` skips the local build rather than forcing one.
+- With no local image and a transient error on the fallback lookup, CLI <= 6.5.10 published
+  `image: null` and the container went to `error`; 6.5.11+ builds instead.
+- A republish renames a package's containers to `<name>_1`.
+- Outside the pnpm workspace, a fresh `npm install` in a package may pull ts-loader 9.6.x /
+  webpack 5.109, which cannot resolve `datagrok-api/*` (tsc can); reuse a known-good
+  `node_modules` (ts-loader 9.5.7 / webpack 5.106) and junction `node_modules/datagrok-api` to a
+  built js-api (`cmd /c mklink /J`).
+- Long publishes outlive a 10-minute agent tool limit; run them detached and poll a log.
+
 ### Testing Framework
 
 Tests use Puppeteer for headless browser automation:
@@ -220,7 +232,7 @@ Based on `node:22-bookworm-slim`. Pre-installed:
 | `demo` | + world, test_db, northwind |
 | `full` | + grok_spawner, JKG, demo DBs |
 
-See `.devcontainer/PACKAGES_DEV.md` for detailed usage docs, architecture diagram, MCP plugin setup (Jira/GitHub), and troubleshooting.
+See `.devcontainer/PACKAGES_DEV.md` for detailed usage docs, architecture diagram, MCP plugin setup (GitHub/Jira), and troubleshooting.
 
 ### `grok server` / `grok s` Command
 

package/GROK_S.md

@@ -36,6 +36,7 @@ browser or a logged-in session.
 | See what fields an entity type has                          | `grok s describe connections`                          |
 | Hit any undocumented endpoint                               | `grok s raw GET /users/current` / `raw POST <path> --data '{...}'` |
 | Check server + per-module health                            | `grok s healthcheck [--module <name>]`                 |
+| Problems, alerts, problem rules, logger settings, errors, cloud logs | `grok s observe problems list --alert-status open` (alias `grok s o`) |
 | Bulk operations in one round-trip                           | `grok s batch <entity> <verb> --json items.json`       |
 | Move entities dev to prod (bundle, or instance to instance) | `grok s pull ... --out ./bundle` / `grok s migrate ... --from dev --to prod` |
 | Browse / query / edit domain-table rows                    | `grok s domains query grit.issue --filter 'status = "open"'` / `domains insert` / `domains upload` |
@@ -136,6 +137,18 @@ grok s connections test --json conn.json              # test before saving
 
 See `public/packages/Chembl/connections/` for connection JSON examples.
 
+Connection tags over REST: the entity JSON carries `entityTags`
+(`[{id, tag, owner: {id: <All users group id>}}]`, no `#type` on the owner), not `tags`.
+`grok s connections save --json` without `entityTags` **clears** the tags (saveTags deletes rows
+not in the list); `POST /entities/save_tags {id, entityTags}` sets them. Filter with
+`whereTag([...])` (`?tags=x`); `whereSmart('tag = "x"')` returns nothing for connections over REST.
+A `Web` connection's parameters must be wrapped (`{"url": {"value": "https://..."}}`); a bare
+string makes the test route answer 400 "Invalid argument(s): value".
+
+A connection saved over REST goes through `Project.add(uniqueName: true)` and can come back
+renamed `<name>_1` for a package's connection — see `core/docs/PROJECTS_AND_PACKAGES.MD`
+"Package connections on a stand".
+
 ## Group membership
 
 `grok s groups add-members` is idempotent — it resolves each member (by login, name,
@@ -161,6 +174,12 @@ ambiguous the command prints every matching group and exits non-zero — pass a
 disambiguate, or add `--user` (accepted by all four membership verbs) to restrict lookup to
 personal groups.
 
+> **Trap:** `grok s groups add-members <group> <users...>` without `--admin` sets `isAdmin=false`
+> on members that are already group admins (reported as `updated`, not `noop`). Before a bulk add
+> run `grok s groups list-members <group> --admin` and exclude those users, or re-add them with
+> `--admin`. Pass the group UUID — `list-members Administrators` by name has returned
+> "(no results)" while the id worked.
+
 ## User administration
 
 ```bash
@@ -411,12 +430,19 @@ grok s domains audit inventory --limit 50                        # DDL and row e
 grok s domains delete inventory --force                          # purge: data, audit, registry, grants
 ```
 
-`schema.json` may be a full manifest or a partial apply body — only `tables`, `extend`,
-`propertySchemas` and `dropTables` are sent; `name`, `version` and `description` are dropped.
-Named tables replace their definition wholesale; untouched tables stay as the registry has
-them. A plan that drops or narrows anything is refused until `--confirm-destructive` is
-passed, and the plan is printed with the refusal. `--if-version` fails the apply when the
-schema's apply counter (or `ext_version` on a package schema) has moved. Package-deployed
+`schema.json` may be a full manifest or a partial apply body — `tables`, `extend`,
+`propertySchemas`, `dropTables` and `access` are sent, plus `friendlyName`, `description` and
+`storage` when the schema is user-managed (`name` is dropped; `version` becomes `ifVersion` and
+`incarnation` becomes `ifIncarnation` unless `--if-version` says otherwise — `current` takes
+both from the schema registered right now, so a name deleted and re-created since the file was
+written is a conflict, not a fresh `1`). Named tables
+replace their definition wholesale; untouched tables stay as the registry has them. A plan
+that drops or narrows anything is refused until `--confirm-destructive` is passed, and the
+plan is printed with the refusal. `--if-version` fails the apply when the schema's apply
+counter (or `ext_version` on a package schema) has moved; a user-managed schema REQUIRES the
+token (the server answers `version-required` without it) — pass the version the edit was made
+against, or `--if-version current` to take the one registered right now. Dropping a table
+needs Delete on that table, dropping its columns Edit on it. Package-deployed
 schemas cannot be applied to or deleted here — `apply` on one is the user-extension path
 (needs `Extend`), and the manifest is owned by `grok publish`. `delete <schema>` requires
 `--force` because it takes every row with it.
@@ -471,6 +497,44 @@ module the server does not report exits 1. Requires a valid dev key (standard `g
 For an anonymous liveness probe — load balancer, k8s readiness — hit `/admin/health` directly;
 it's on the server's unauthenticated allowlist.
 
+## Problems, logging and errors (`grok s observe`, alias `grok s o`)
+
+Thin commands over the server's routes; `grok s observe` with no command prints every verb and option.
+
+```bash
+grok s o problems list --alert-status open                 # the open alerts; --status, --state, --kind, --since 7d
+grok s o problems get 'health:Jupyter@<server>'            # by id or kind:key; history <id> for its records
+grok s o problems status connection:<connection id> --data '{"status": "muted", "reason": "maintenance", "until": "2026-10-04T06:00:00Z"}'
+grok s o problems ack 'health:Jupyter@<server>' --reason x  # resolve <id> --reason ... closes the alert
+grok s o rules get > rules.json                             # Settings > Alerts problem rules; rules put --json rules.json
+grok s o rules test                                         # evaluates the saved rules once, now
+grok s o logger get exportSettings                          # the logger settings, or one path of them
+grok s o logger set --user alice --set debugFlags='["db"]' --for 30m --reason "ticket 123"   # an expiring entry, from All users
+grok s o logger set --user alice --remove                  # delete that entry
+grok s o logger history --limit 5                           # log-settings-changed audit records with their diff
+grok s o logs capture alice --for 2h --reason "ticket 123" # logger set of saveHttpRequests on an expiring entry; 30m by default
+grok s o logs capture                                       # the active captures; logs capture stop alice removes one
+grok s o errors top --since 7d --by signature --limit 20    # GET /admin/metrics errors
+grok s o errors top --since 24h --source server           # ranked by distinct users: system errors (scheduler, probes) need --source
+grok s o errors get error-incident:<key>                    # each occurrence of a problem's error: time, user, function, session
+grok s o errors get <signature> --since 1d                  # or of a signature from errors top --by signature
+grok s o timeline --session <id>                            # its events and requests of the last 15 min
+grok s o timeline --session <id> --from 2026-10-09T10:00     # 10 min from then; --to sets the end, at most 2 h later
+grok s o logs cloud --since 2h --filter ERROR                # CloudWatch, the instance's own group; --group, --from/--to
+grok s o logs archive list --prefix cw/ --since 1d           # keys in the log archive (WORM bucket); --since filters the returned page only
+grok s o logs archive read <key>                             # one archived object decoded to events
+grok s o logs container                                      # docker containers and their status
+grok s o logs container scripting-python_1 --since 1h --grep 'Kernel (started|shutdown)'   # one container's log lines
+```
+
+Problems need `ManageAlerts`, errors, another session's timeline and `logs cloud`/`logs archive` `ViewTelemetry`,
+`rules test` both, logger and rules changes and `logs capture` `EditPluginsSettings`. `capture` is an expiring
+`userGroupSettings` entry with `saveHttpRequests` on (seeded from the All Users entry, with `usage` saved); on an entry
+that already expires later it keeps that time. `capture stop` removes only an expiring entry that holds nothing but the
+capture; for any other entry use `logger set --user <login> --remove`. Read what it recorded with
+`timeline --session <id>`. `logs container` reads the container's latest `--limit` lines (20000) and only then cuts `--since`/`--from`/`--to` (UTC, matched across the container's own offset) and `--grep`, so a window older than those lines comes back empty: raise `--limit`. `logs archive list --since` filters only the page the server returned (`--limit`), so narrow
+the listing with `--prefix` first. `logs` read with the server's AWS role, or with `--connection <id>`. In durations `m` means minutes (in `grok s pull --since` it means months).
+
 ## Describing an entity type
 
 ```bash
@@ -626,9 +690,12 @@ entities shared with that account, and the target's own users never get their co
 
 ```bash
 grok s pull --since 10y --type project,connection,query,script,group,layout   --out ./bundle --host dev --admin
-grok s push ./bundle --host prod --admin --on-conflict skip
+grok s push ./bundle --host prod --admin --on-conflict adopt
 ```
 
+`adopt` keeps one copy of what both instances already have, such as shared groups and spaces;
+`skip` would leave them alone and fail everything in the bundle that points at them.
+
 **Move a whole instance one space at a time.** Placement is exclusive on the server — one project
 holding an entity takes it from every other — so a single bundle carrying every space has its
 projects competing, and a stand-sized push spends its time taking rows back off each other. Scoped

package/README.md

@@ -106,7 +106,8 @@ For more information on configuring connections, refer to the [Connections](http
   `config.yaml`): `grok report fetch <instance> <number>` downloads the report zip, `grok report read
   <zip|json|instance number>` prints one normalized JSON object (flags `--extract-screenshot`,
   `--extract-d42`, `--extract-client-log`), `grok report resolve <instance> <number>` marks it resolved.
-  `ticket`, `comment`, `label`, `attach` talk to JIRA with `JIRA_TOKEN`. See `grok report --help`.
+  `ticket`, `comment`, `label`, `attach` talk to JIRA with `JIRA_TOKEN`; porting them to GitHub Issues
+  is pending. See `grok report --help`.
 - `check` checks package content (function signatures, import statements of external modules,
   etc.). The check is also run during package publication.
 - `init` modifies a package template by adding config files for linters, IDE, and so on

package/bin/commands/add.js

@@ -16,8 +16,7 @@ function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r
  * answer key): u2 is not on npm, so the dependency is the workspace link, and `u2core` is the one
  * external `@datagrok/build-config` does not know about. The css imports and the tsconfig need
  * nothing — the shared bundler config carries the loaders and the tsconfig extends the base. */
-const u2Dependency = 'workspace:^';
-const u2DgModule = '@datagrok-libraries/u2/src/dg/index.js';
+const emsDgModule = '@datagrok-libraries/ems/src/dg';
 const u2RspackExternal = `'datagrok-api/u2core': 'DG.U2'`;
 const u2RspackConfig = [`const {bundler} = require('@datagrok/build-config');`, '', `module.exports = bundler({externals: {${u2RspackExternal}}});`, ''].join('\n');
 function add(args) {
@@ -110,7 +109,7 @@ function add(args) {
    * existing imports (which webpack's externals depend on). */
   function addU2Imports() {
     const contents = _fs.default.readFileSync(packageEntry, 'utf8');
-    if (contents.includes(u2DgModule)) return;
+    if (contents.includes(emsDgModule)) return;
     const eol = contents.includes('\r\n') ? '\r\n' : '\n';
     const imports = _fs.default.readFileSync(_path.default.join(templateDir, 'entity-template', 'domain-app-imports.js'), 'utf8').trimEnd().split(/\r?\n/);
     const lines = contents.split(/\r?\n/);
@@ -125,7 +124,8 @@ function add(args) {
   function addU2Wiring() {
     const packageObj = JSON.parse(_fs.default.readFileSync(packagePath, 'utf8'));
     packageObj.dependencies = packageObj.dependencies ?? {};
-    packageObj.dependencies['@datagrok-libraries/u2'] ??= u2Dependency;
+    packageObj.dependencies['@datagrok-libraries/ems'] ??= 'workspace:^';
+    packageObj.dependencies['@datagrok-libraries/u2'] ??= 'workspace:^';
     _fs.default.writeFileSync(packagePath, JSON.stringify(packageObj, null, 2), 'utf8');
     if (!_fs.default.existsSync(rspackConfigPath)) _fs.default.writeFileSync(rspackConfigPath, u2RspackConfig, 'utf8');else if (!_fs.default.readFileSync(rspackConfigPath, 'utf8').includes('datagrok-api/u2core')) color.warn(`rspack.config.js is already there — add \`externals: {${u2RspackExternal}}\` to it by hand`);
   }

package/bin/commands/api.js

@@ -24,7 +24,7 @@ const dbFile = 'db.ts';
 const dbUiFile = 'db-ui.ts';
 /** Where `db-ui.ts` takes the u2 domain stack from — the same specifier Grit and Stockroom use
  * (u2 is a relative-path dependency without a `dg` subpath export). */
-const u2DgModule = '@datagrok-libraries/u2/src/dg/index.js';
+const emsDgModule = '@datagrok-libraries/ems/src/dg';
 
 /** Naive English plural of a (snake_case) table name — `issue` → `issues`, `box` →
  * `boxes`, `category` → `categories`. */
@@ -229,7 +229,7 @@ function generateDomainClients(packageDir = curDir, options) {
   _fs.default.writeFileSync(_path.default.join(genDir, dbFile), normEol(content).replace(/\n/g, '\r\n'), 'utf8');
   color.log(`Successfully generated file src/generated/${dbFile}${sep}`, 'success');
   if (ui) {
-    const uiContent = annotationForDbUiFile + `import {domains, DomainTable} from '${u2DgModule}';${sep}` + wrapTokens(`import {`, dbImports, ', ', `} from './${dbFile.replace(/\.ts$/, '')}';`) + sep + sep + uiParts.join(sep);
+    const uiContent = annotationForDbUiFile + `import {domains, DomainTable} from '${emsDgModule}';${sep}` + wrapTokens(`import {`, dbImports, ', ', `} from './${dbFile.replace(/\.ts$/, '')}';`) + sep + sep + uiParts.join(sep);
     _fs.default.writeFileSync(_path.default.join(genDir, dbUiFile), normEol(uiContent).replace(/\n/g, '\r\n'), 'utf8');
     color.log(`Successfully generated file src/generated/${dbUiFile}${sep}`, 'success');
   }
@@ -265,6 +265,10 @@ function findDependencyManifest(packageDir, schemaName) {
 function generateDomainSchemaCode(manifest, manifestPath, emittedTypes, packageDir) {
   const decls = [];
   const systemColumnNames = new Set(domainSystemColumns.map(([name]) => name));
+  // An external table's rows carry `id` (the encoded business key) and nothing else
+  // system-side; nothing is expanded and nothing is written (external-bindings §3).
+  const external = manifest.storage?.kind === 'external';
+  const systemColumns = external ? domainSystemColumns.filter(([name]) => name === 'id') : domainSystemColumns;
   const tableNames = Object.keys(manifest.tables);
   // Keyed by table name; a qualified ref target ('Core.queries', 'grit.issue') lands here
   // under its qualified key so the expand map can list its columns — it gets no accessor.
@@ -427,7 +431,7 @@ function generateDomainSchemaCode(manifest, manifestPath, emittedTypes, packageD
     const columns = tableColumns[tableName];
     const relations = tableRelations[tableName];
     const rowLines = [`/** Row of \`${manifest.name}.${tableName}\`. */`, `export interface ${typeName}Row {`];
-    for (const [name, tsType] of domainSystemColumns) rowLines.push(`  ${name}: ${tsType};`);
+    for (const [name, tsType] of systemColumns) rowLines.push(`  ${name}: ${tsType};`);
     for (const c of columns) rowLines.push(`  ${c.name}${c.required || c.autoNumber ? '' : '?'}: ${c.tsType};`);
     rowLines.push('}');
     decls.push(rowLines.join(sep));
@@ -447,13 +451,13 @@ function generateDomainSchemaCode(manifest, manifestPath, emittedTypes, packageD
       updateLines.push('};');
       decls.push(updateLines.join(sep));
     }
-    decls.push(formatColumnUnion(`${typeName}Column`, [...domainSystemColumns.map(([name]) => name), ...columns.map(c => c.name)]));
+    decls.push(formatColumnUnion(`${typeName}Column`, [...systemColumns.map(([name]) => name), ...columns.map(c => c.name)]));
 
     // Expand map: MUST stay a `type` alias — object-type literals carry the implicit
     // index signature the `TExpand extends {[key: string]: {}}` constraint needs
     // (an interface would fail it).
     const expandEntries = [];
-    for (const c of columns) {
+    for (const c of external ? [] : columns) {
       if (c.ref == null) continue;
       const fields = tableColumns[c.ref].map(tc => `'${c.name}.${tc.name}'?: ${tc.tsType}`);
       expandEntries.push(wrapTokens(`  '${c.name}': {`, fields, '; ', '};', '    '));
@@ -463,15 +467,15 @@ function generateDomainSchemaCode(manifest, manifestPath, emittedTypes, packageD
     // The link shape stays INLINE (structurally DG.DomainRelationLink): generated
     // files must compile against the datagrok-api the package depends on, which may
     // predate that interface.
-    for (const r of relations) expandEntries.push(`  '${r.name}': {${r.name}?: {id: string; name: string}[]};`);
-    for (const childName of tableNames) {
+    for (const r of external ? [] : relations) expandEntries.push(`  '${r.name}': {${r.name}?: {id: string; name: string}[]};`);
+    for (const childName of external ? [] : tableNames) {
       const fks = tableColumns[childName].filter(c => c.ref === tableName);
       if (fks.length === 0) continue;
       const childType = utils.snakeToCamelCase(childName);
       if (fks.length === 1) expandEntries.push(`  'details:${childName}': {${childName}?: ${childType}Row[]};`);else for (const fk of fks) expandEntries.push(`  'details:${childName}.${fk.name}': {${childName}?: ${childType}Row[]};`);
     }
     decls.push(expandEntries.length === 0 ? `export type ${typeName}Expand = {};` : [`/** Expand keys of \`${manifest.name}.${tableName}\` → fields each adds to the row ` + `(consumed by query()/builder). */`, `export type ${typeName}Expand = {`, ...expandEntries, '};'].join(sep));
-    txArms.push(`  {op: 'insert'; table: '${tableName}'; ref?: string; values: DG.DomainTxValues<${typeName}Insert>; ` + `onDuplicate?: 'error'} |`, `  {op: 'update'; table: '${tableName}'; id: string; ` + `values: DG.DomainTxValues<${relations.length > 0 ? `${typeName}Update` : `Partial<${typeName}Row>`}>; ` + `expectedVersion?: number} |`, `  {op: 'delete'; table: '${tableName}'; id: string} |`);
+    if (!external) txArms.push(`  {op: 'insert'; table: '${tableName}'; ref?: string; values: DG.DomainTxValues<${typeName}Insert>; ` + `onDuplicate?: 'error'} |`, `  {op: 'update'; table: '${tableName}'; id: string; ` + `values: DG.DomainTxValues<${relations.length > 0 ? `${typeName}Update` : `Partial<${typeName}Row>`}>; ` + `expectedVersion?: number} |`, `  {op: 'delete'; table: '${tableName}'; id: string} |`);
 
     // No datetime config: the client resolves datetime columns from the domain
     // registry itself (the Dayjs typing in <Table>Row stays true for every client).
@@ -480,10 +484,14 @@ function generateDomainSchemaCode(manifest, manifestPath, emittedTypes, packageD
     clients.push(`  get ${tableProp(tableName)}() {`, `    return grok.dapi.domains.table<${typeName}Row, ${typeName}Insert, ` + `${typeName}Column, ${typeName}Expand` + `${relations.length > 0 ? `, ${typeName}Update` : ''}>('${manifest.name}.${tableName}');`, '  },');
   }
   const schemaType = utils.snakeToCamelCase(manifest.name);
-  const lastArm = txArms.pop();
-  txArms.push(lastArm.replace(/ \|$/, ';'));
-  decls.push([`export type ${schemaType}TransactionOp =`, ...txArms].join(sep));
-  decls.push([`/** Typed clients for the \`${manifest.name}\` domain schema tables ` + `(lazy — no import-time side effects). */`, `export const ${utils.snakeToCamelCase(manifest.name, false)}Db = {`, ...clients, `  transaction<T extends ${schemaType}TransactionOp[]>(ops: [...T]):`, `      Promise<{[K in keyof T]: DG.DomainOpResultFor<T[K]>}> {`, `    return grok.dapi.domains.transaction('${manifest.name}', ops) as`, `      Promise<{[K in keyof T]: DG.DomainOpResultFor<T[K]>}>;`, '  },', '};'].join(sep));
+  const transaction = [];
+  if (!external) {
+    const lastArm = txArms.pop();
+    txArms.push(lastArm.replace(/ \|$/, ';'));
+    decls.push([`export type ${schemaType}TransactionOp =`, ...txArms].join(sep));
+    transaction.push(`  transaction<T extends ${schemaType}TransactionOp[]>(ops: [...T]):`, `      Promise<{[K in keyof T]: DG.DomainOpResultFor<T[K]>}> {`, `    return grok.dapi.domains.transaction('${manifest.name}', ops) as`, `      Promise<{[K in keyof T]: DG.DomainOpResultFor<T[K]>}>;`, '  },');
+  }
+  decls.push([`/** Typed clients for the \`${manifest.name}\` domain schema tables ` + `(lazy — no import-time side effects). */`, `export const ${utils.snakeToCamelCase(manifest.name, false)}Db = {`, ...clients, ...transaction, '};'].join(sep));
   return decls.join(sep.repeat(2)) + sep;
 }
 

package/bin/commands/help.js

@@ -96,7 +96,7 @@ grok add tests
 Please note that entity names may only include letters and numbers
 
 --domain scaffolds a working browse/CRUD app over an entity-mapped domain table
-(\`grok.dapi.domains\`) from the u2 defaults (\`@datagrok-libraries/u2\`): a three-line
+(\`grok.dapi.domains\`) from the EMS defaults (\`@datagrok-libraries/ems\`): a three-line
 app function, a \`src/app.spec.json\` the designer edits, and — for a schema the package
 does not declare yet — a one-table starter \`databases/<schema>/schema.json\`. Give it
 one table (\`--domain tracker.issue\`), a whole schema the package declares in
@@ -140,7 +140,7 @@ Options:
 --verbose         Print detailed output
 --ui              Also generate \`src/generated/db-ui.ts\` — typed u2 handles
                   (\`get<Schema>Db()\` → a \`DomainTable<Row>\` per table, opened in
-                  parallel behind one await) over \`@datagrok-libraries/u2\`. Once the
+                  parallel behind one await) over \`@datagrok-libraries/ems\`. Once the
                   file exists, plain \`grok api\` keeps it up to date; delete it to
                   opt out again
 `;

package/bin/commands/server-domains.js

@@ -21,7 +21,9 @@ function _interopRequireWildcard(e, t) { if ("function" == typeof WeakMap) var r
 /// `grok s domains ...` — entity-mapped domain schemas and their rows (DomainsRouter, `/domains/`).
 /// Every verb addresses a schema as `<schema>` or a table as `<schema>.<table>`.
 
-const APPLY_KEYS = ['tables', 'extend', 'propertySchemas', 'dropTables', 'ifVersion', 'confirmDestructive'];
+const APPLY_KEYS = ['tables', 'extend', 'propertySchemas', 'dropTables', 'access', 'ifVersion', 'ifIncarnation', 'confirmDestructive'];
+/** Accepted by a user-managed schema's apply only; a package schema refuses them as unknown keys. */
+const USER_APPLY_KEYS = ['friendlyName', 'description', 'storage'];
 const PERMISSIONS = ['View', 'Edit', 'Delete', 'Share', 'Extend'];
 const USAGE = `Usage: grok s domains <verb> [args]
   list [<schema>] [--filter <text>]                      Schemas, or the tables of one schema
@@ -210,11 +212,23 @@ async function handleDomains(dapi, verb, rest, argv, output) {
       }
     case 'apply':
       {
-        if (!args[0] || !argv.json) return usage('apply <schema> --json manifest.json [--dry-run] [--confirm-destructive] [--if-version <v>]');
+        if (!args[0] || !argv.json) return usage('apply <schema> --json manifest.json [--dry-run] [--confirm-destructive] [--if-version <v>|current]');
         const a = (0, _nodeDapi.parseDomainAddress)(args[0], {
           table: false
         });
-        const body = applyBody(readJson(argv.json), argv);
+        const json = readJson(argv.json);
+        // One registry read, only when the token or the metadata keys depend on who manages the schema.
+        const s = argv['if-version'] === undefined || argv['if-version'] === 'current' || USER_APPLY_KEYS.some(k => json?.[k] !== undefined) ? await domains.schema(a.schema) : null;
+        const body = applyBody(json, argv, {
+          userManaged: s?.managedBy === 'user'
+        });
+        if (body.ifVersion === 'current') {
+          // Both halves of the token as registered now: a name deleted and re-created
+          // since the file was written must conflict, not pass as a fresh `1`.
+          const m = await domains.manifest(a.schema);
+          body.ifVersion = String(m?.version ?? s.version ?? '1');
+          if (m?.incarnation !== undefined) body.ifIncarnation = String(m.incarnation);
+        } else if (body.ifVersion === undefined && s.managedBy === 'user') return usage(`apply ${a.schema} --json manifest.json --if-version <v>|current — a user-managed schema ` + 'requires the version the edit was made against (the "version" of the manifest JSON is used when present)');
         printJson(await domains.applySchema(a.schema, body, argv['dry-run'] === true), output, r => r?.version);
         return true;
       }
@@ -305,12 +319,15 @@ function aggregateSpec(argv, filter, limit) {
   return spec;
 }
 
-/** The apply payload from a schema.json (or a partial body): only the keys the server accepts, flags folded in. */
-function applyBody(json, argv) {
+/** The apply payload from a schema.json (or a partial body): only the keys the server accepts
+ * (the schema's metadata and storage keys for a user-managed schema only), flags folded in. */
+function applyBody(json, argv, options = {}) {
   const body = {};
-  for (const k of APPLY_KEYS) if (json?.[k] !== undefined) body[k] = json[k];
+  for (const k of options.userManaged ? [...APPLY_KEYS, ...USER_APPLY_KEYS] : APPLY_KEYS) if (json?.[k] !== undefined) body[k] = json[k];
   if (argv['confirm-destructive'] === true) body.confirmDestructive = true;
-  if (argv['if-version'] !== undefined) body.ifVersion = String(argv['if-version']);
+  if (argv['if-version'] !== undefined) body.ifVersion = String(argv['if-version']);else if (json?.version !== undefined) body.ifVersion = String(json.version);
+  // The manifest's `incarnation` rides with its `version` (the server keeps it optional).
+  if (json?.incarnation !== undefined) body.ifIncarnation = String(json.incarnation);
   return body;
 }