datagrok-tools 6.5.6 → 6.5.8

package/GROK_S.md

@@ -21,7 +21,7 @@ browser or a logged-in session.
 | List members of a group                                     | `grok s groups list-members <group>`                   |
 | List the groups a user belongs to                           | `grok s groups list-memberships <user>`                |
 | Share a connection / query / script / project with a group  | `grok s shares add <entity> <group> [--access View\|Edit]` |
-| See who an entity is shared with                            | `grok s shares list <entity-id>`                       |
+| See who an entity is shared with                            | `grok s shares list <entity-id-or-name>`               |
 | Create / update a connection                                | `grok s connections save --json conn.json`             |
 | Test a connection                                           | `grok s connections test <id-or-name>`                 |
 | Run a registered function                                   | `grok s functions run 'Pkg:fn(arg1,arg2)'`             |
@@ -29,9 +29,16 @@ browser or a logged-in session.
 | List / browse files in a file share                         | `grok s files list "System:AppData" -r`                |
 | Upload / download a table (CSV or d42)                      | `grok s tables upload <name> file.csv\|file.d42` / `tables download <name> -O out.csv` |
 | Check whether a package is deployed                         | `grok s packages list --filter "MyPlugin"`             |
-| Hit any undocumented endpoint                               | `grok s raw GET /api/users/current`                    |
+| Install / update / uninstall server plugins                 | `grok s packages install Chem Bio` / `packages update --all` / `packages uninstall Chem` |
+| See installed vs latest plugin versions                     | `grok s packages outdated` / `packages versions Chem`  |
+| Count entities                                              | `grok s users count --filter 'status = "active"'`      |
+| See what fields an entity type has                          | `grok s describe connections`                          |
+| Hit any undocumented endpoint                               | `grok s raw GET /users/current` / `raw POST <path> --data '{...}'` |
 | Check server + per-module health                            | `grok s healthcheck [--module <name>]`                 |
 | Bulk operations in one round-trip                           | `grok s batch <entity> <verb> --json items.json`       |
+| Move entities dev to prod (bundle, or instance to instance) | `grok s pull ... --out ./bundle` / `grok s migrate ... --from dev --to prod` |
+| Browse / query / edit domain-table rows                    | `grok s domains query grit.issue --filter 'status = "open"'` / `domains insert` / `domains upload` |
+| Manage domain schemas (manifest, apply, grants)             | `grok s domains get grit` / `domains apply` / `domains grant grit.issue Chemists --access Edit` |
 
 ## Configuration
 
@@ -48,21 +55,25 @@ servers:
     key: <developer-key>
 ```
 
-- `grok config --server --alias <name> --server <url> --key <key>` writes a new entry.
+- `grok config add --alias <name> --server <url> --key <key>` writes a new entry.
 - Add `--default` to make it the active server.
-- Every `grok s ...` command accepts `--host <alias-or-url>` to override the default.
+- Every `grok s ...` command accepts `--host <alias-or-url>` to override the default. The URL
+  is the API base (`https://host/api`, or `http://host:8082` for a bare Datlas).
 
 ## Entity operations
 
-### List / get / delete
+### List / count / get / delete
 
 ```bash
 grok s users list                              # default: table, 50 rows
 grok s users list --filter "login = 'admin'"   # smart filter
+grok s users list --limit 20 --offset 40       # third page of 20
+grok s users count --filter 'status = "active"'
 grok s connections list --output json
 grok s packages list --filter "name:MyPlugin"  # table shows friendlyName
 grok s groups get <id-or-name>
-grok s connections delete <id>
+grok s users get alice.mendel                  # users: id or login
+grok s connections delete "Admin:MyConnection" # id or namespace:name
 ```
 
 Options that work on every entity:
@@ -75,6 +86,14 @@ Options that work on every entity:
 | `--offset <n>`        | Skip first `n` rows                               |
 | `--host <alias\|url>` | Target a specific server from your config         |
 
+A missing entity, a rejected save (duplicate login or group name), or any other server-side
+failure prints one line on stderr and exits 1 — never an error object on stdout.
+
+`delete` is not available for every entity: **users cannot be deleted** (Datagrok has no user
+deletion; `users delete` refuses and points at `users block`), and `functions delete` covers
+scripts and queries only (package functions go away with their package). `count` has no server
+endpoint for `reports`.
+
 ### Save from JSON
 
 `grok s users save` and `grok s groups save` accept the same JSON shape that the
@@ -125,10 +144,13 @@ grok s groups list-members Chemists                            # all members
 grok s groups list-members Chemists --admin                    # admin members only
 grok s groups list-members Chemists --no-admin                 # non-admin members only
 grok s groups list-memberships alice.mendel                    # groups a user belongs to
+grok s groups list-memberships admin --user                    # the personal group, not "Administrators"
 ```
 
-When a name is ambiguous the command prints every matching group and exits non-zero —
-pass a UUID to disambiguate, or add `--user` to restrict lookup to personal groups.
+Names resolve by substring, an exact `name`/`friendlyName` match wins. When a name is still
+ambiguous the command prints every matching group and exits non-zero — pass a UUID to
+disambiguate, or add `--user` (accepted by all four membership verbs) to restrict lookup to
+personal groups.
 
 ## User administration
 
@@ -145,11 +167,12 @@ flips `status` to `Blocked` and terminates active sessions; unblocking restores
 
 ```bash
 grok s shares add "MyUser:MyConnection" Chemists,Biologists --access Edit
-grok s shares list <entity-uuid>
+grok s shares list "MyUser:MyConnection"
 ```
 
 The entity argument accepts either a UUID or an `"author:name"` pair. `--access`
-defaults to `View`.
+defaults to `View`. `shares list` prints one row per group and permission, including the
+grants inherited through the entity's project links (`inherited: true`).
 
 ## Running functions
 
@@ -159,6 +182,10 @@ grok s functions run 'Pkg:fn({smiles:"CCO", radius:2})'        # named args
 grok s functions run Pkg:fn --json params.json                 # big input from a file
 ```
 
+The server binds arguments by parameter name, so positional values are mapped onto the
+function's inputs in declared order (one extra lookup of the function); more values than inputs
+is an error. Named arguments are sent as they are.
+
 ## Listing functions with filters
 
 `grok s functions list` composes a smart filter from flags so you don't have to hand-roll
@@ -192,7 +219,8 @@ contain colons (the namespace separator).
 
 ```bash
 grok s files list "System:AppData" -r                          # recursive
-grok s files list "System:AppData/MyPlugin"
+grok s files list "System:AppData/MyPlugin"                    # path, kind (file|dir), size, updatedOn
+grok s files list "System:DemoFiles" --output quiet            # paths only, pipe-friendly
 grok s files get  "System:AppData/MyPlugin/config.json"
 grok s files put  ./smiles.csv "System:DemoFiles/smiles.csv"   # upload local file
 grok s files delete "System:AppData/MyPlugin/old.csv"
@@ -214,9 +242,16 @@ grok s tables upload MyTable ./data.d42                         # d42 binary →
 grok s tables upload MyTable ./data.csv --output json           # get ID and markup back
 grok s tables download MyTable                                  # CSV to stdout (pipe-friendly)
 grok s tables download MyTable -O ./data.csv                    # CSV to a local file
-grok s tables download <uuid>                                   # UUID or namespace:name both work
+grok s tables download "Admin:MyTable:MyTable"                  # full name, or the UUID
+grok s tables list --filter MyTable                             # registered tables
+grok s tables get MyTable                                       # the TableInfo record
+grok s tables delete MyTable
 ```
 
+A table lives under a project namespace (the upload prints its full name,
+`Admin:MyTable:MyTable`); the bare name works while only one table carries it, otherwise the
+CLI lists the candidates and asks for the full name or the id.
+
 Upload streams raw bytes — `Content-Type: text/csv` for `.csv` and
 `application/octet-stream` for `.d42` (auto-detected from the file extension). Both
 formats handle large tables without loading the whole file into a JSON envelope.
@@ -226,6 +261,170 @@ table|json|csv|quiet`), which still controls how the upload result is printed.
 Download is CSV-only — the server reads the stored d42 blob and converts. If you
 need the raw d42 bytes, hit `/tables/data/<id>` directly via `grok s raw`.
 
+## Packages
+
+Manage server plugins the same way the Package Manager UI does — the server pulls
+published packages from its configured package repository (npm). This is for
+installing **released** packages on a running server; publishing your own local
+package sources is still `grok publish`.
+
+```bash
+grok s packages install Chem Bio PowerGrid       # install latest of each, one line
+grok s packages install Chem --version 1.14.0    # pin a specific version (single package only)
+grok s packages outdated                         # installed vs registry-latest
+grok s packages update Chem Bio                  # upgrade named packages to latest
+grok s packages update --all                     # upgrade everything outdated
+grok s packages versions Chem                    # published versions w/ current/latest/debug flags
+grok s packages set-version Chem 1.13.0          # activate a specific version (pulls it if needed)
+grok s packages uninstall Chem                   # remove; repository entry stays installable
+grok s packages share Chem Chemists --access View
+```
+
+Semantics worth knowing:
+
+- **`install` without `--version` sets the package to `latest`**, which is an
+  auto-update intent: the server re-resolves it against the registry every ~15
+  minutes, so the package keeps itself current. A pinned `--version` (and `update`,
+  which pins the concrete latest version) stays put until you change it.
+- **Install is synchronous** — the call returns once the version is downloaded,
+  published, and made current. A heavy first-time install (e.g. Chem) can take
+  minutes; if the HTTP call times out, just re-run it — install is idempotent, and
+  a version that finished installing server-side is activated instantly on retry.
+- Multi-package `install`/`update` runs sequentially and prints a per-package
+  status table (`installed` / `noop` / `error`); a partial failure sets exit code 1.
+- **`update` preserves auto-update tracking**: a package whose desired version is
+  `latest` is updated by re-resolving `latest` (it keeps auto-updating); a pinned
+  package gets pinned to the concrete registry-latest version. Note that a
+  locally-published dev build counts as outdated once the registry moves past its
+  base version, so `update --all` will replace it with the registry release.
+- Status semantics: `noop` means "already at the requested latest"; installing an
+  explicit `--version` reports `installed` (with the published-version id) even
+  when that version was already current — the server re-activates it.
+- **`uninstall` of a repository-backed package keeps the package entry** (it shows
+  up as installable again); only locally-published packages are deleted outright.
+  It also clears the desired version, so the package drops out of `outdated` until
+  reinstalled. Neither path cleans up package credentials or package DB schemas.
+- `versions`, `outdated`, and name resolution read the repository catalog, so they
+  need a configured package repository — check with `grok s raw GET /api/packages/repos`.
+- `share` targets the package's project under the hood (same as sharing from the UI),
+  so the whole package — functions, queries, connections — is shared at once. For the
+  same reason `grok s shares list <package-uuid>` won't show the grant — it lives on
+  the package's project, not the package entity itself.
+
+## Domain schemas and rows
+
+Entity-mapped domain tables — the schemas plugins declare in `databases/<schema>/schema.json`
+and the user-managed ones created at runtime — are reachable through one entity, `domains`.
+Every verb takes an address: a bare `<schema>` names a schema, `<schema>.<table>` names a
+table, mirroring `grok.dapi.domains.schema('grit')` and `grok.dapi.domains.table('grit.issue')`.
+Reads return only the rows and columns the key's user may see; writes are validated,
+permission-checked and audited by the server exactly as they are from the UI.
+
+### Browsing
+
+```bash
+grok s domains list                                 # schemas: name, managedBy, version, table count
+grok s domains list grit                            # tables of one schema: security mode, business key, ...
+grok s domains get grit                             # the manifest, as JSON (doubles as an export)
+grok s domains get grit.issue                       # the table's columns (--output json: its manifest section)
+grok s domains get grit.issue <row-id>              # one row
+grok s domains capabilities grit.issue              # what the current user may do on the table
+```
+
+### Querying
+
+```bash
+grok s domains query grit.issue --filter 'status = "open"' --sort '!created_on' --limit 20
+grok s domains query grit.issue --columns title,status --expand project_id --offset 100
+grok s domains count grit.issue --filter 'status = "open"'
+grok s domains aggregate grit.issue --measures 'count,avg(estimate) as mean' --group-by status
+grok s domains aggregate grit.issue --json spec.json             # any DomainAggregateSpec
+grok s domains download grit.issue -O ./issues.csv --filter 'status = "open"'
+grok s domains download grit.issue -O ./issues.d42                # typed DataFrame, 10M-row cap
+```
+
+`--filter` is the domain smart-filter grammar (`status = "open"`, `title contains "crash"`,
+`quantity > 10`); values are bound server-side. `--sort` is a comma list with `!` for
+descending; `--expand` takes `<fk_column>`, `details:<child>` or a relation name. JSON output
+and CSV downloads are capped at 10k rows by the server; a `.d42` download uses the DataFrame
+path and goes up to 10M. `--limit` defaults to 50 for `query` and is unbounded for `download`.
+
+### Writing rows
+
+```bash
+grok s domains insert grit.issue title="Crash on save" status=open project_id=<uuid>
+grok s domains insert grit.issue --json rows.json                # one object or an array
+grok s domains update grit.issue <row-id> status=closed --version 3   # optimistic concurrency
+grok s domains delete grit.issue <row-id>
+grok s domains delete grit.issue --filter 'status = "closed"' --limit 500   # bulk, oldest first
+grok s domains transaction grit --json ops.json                  # ordered ops, one transaction
+```
+
+Inline `col=value` pairs are typed when the value parses as JSON (`quantity=5`,
+`done=true`, `note=null`, `tags=["a","b"]`) and sent as strings otherwise. A business-key
+duplicate is reported as `status: duplicate` with the existing id; add `--error-on-duplicate`
+to fail instead. A validation failure prints one line per offending column and exits 1.
+`delete --filter` removes at most 1000 rows per call and says when more remain.
+`ops.json` is the `DomainsDataSource.transaction` ops list — `{op, table, ref?, values?, id?,
+expectedVersion?}`, with `"$ref"` placeholders for earlier ops' ids.
+
+### Bulk upload
+
+```bash
+grok s domains upload grit.issue ./issues.csv                    # insert
+grok s domains upload grit.issue ./issues.csv --upsert           # merge by business key
+grok s domains upload grit.issue ./issues.d42                    # d42 DataFrame
+grok s domains upload grit.issue ./issues.json --no-all-or-nothing --error-on-duplicate
+```
+
+The format follows the extension (`.csv`, `.d42`, `.json` — a row array, bare or under
+`rows`). The whole batch is one transaction by default; `--no-all-or-nothing` applies the good
+rows and reports the bad ones per row. The report prints `inserted / updated / skipped /
+errors` plus a row table for the failures; any error sets exit code 1.
+
+### Schema lifecycle (user-managed schemas)
+
+```bash
+grok s domains create inventory --friendly-name "Inventory" --description "Lab stock"
+grok s domains apply inventory --json schema.json --dry-run      # the change plan, no writes
+grok s domains apply inventory --json schema.json                # create / alter tables
+grok s domains apply inventory --json schema.json --confirm-destructive --if-version 3
+grok s domains audit inventory --limit 50                        # DDL and row events, newest first
+grok s domains delete inventory --force                          # purge: data, audit, registry, grants
+```
+
+`schema.json` may be a full manifest or a partial apply body — only `tables`, `extend`,
+`propertySchemas` and `dropTables` are sent; `name`, `version` and `description` are dropped.
+Named tables replace their definition wholesale; untouched tables stay as the registry has
+them. A plan that drops or narrows anything is refused until `--confirm-destructive` is
+passed, and the plan is printed with the refusal. `--if-version` fails the apply when the
+schema's apply counter (or `ext_version` on a package schema) has moved. Package-deployed
+schemas cannot be applied to or deleted here — `apply` on one is the user-extension path
+(needs `Extend`), and the manifest is owned by `grok publish`. `delete <schema>` requires
+`--force` because it takes every row with it.
+
+### Grants
+
+```bash
+grok s domains grants grit.issue                                 # direct permission rows
+grok s domains grant grit.issue Chemists,Biologists --access Edit
+grok s domains revoke grit.issue Chemists --access Edit          # omit --access to revoke all
+grok s domains grant grit Chemists --access Extend               # schema-level: may add own tables/columns
+```
+
+Groups resolve by name the same way `groups add-members` does (a login resolves to the
+personal group; ambiguous names list the candidates). Table grants gate row data; schema
+grants gate schema operations (`Edit` to apply, `Delete` to purge, `Share`, `Extend`) and do
+not reach rows. A grant on a table restricted by per-column sharing does not un-hide the
+column — that path (`/domains/grants/column`) is `grok s raw` territory for now.
+
+### Not covered here
+
+Watch/subscribe, facets, saved filters, per-column sharing and row promotion have no verb;
+`grok s raw <METHOD> /api/domains/...` reaches them. Domain schemas and their data do not
+take part in `pull` / `push` / `migrate` — install the package (or re-`apply` the manifest)
+on the target and `upload` the rows.
+
 ## Server health
 
 ```bash
@@ -248,26 +447,50 @@ Hits `GET /public/v1/healthcheck`. Response:
 }
 ```
 
-`services` is the same payload as `/admin/health` (per-`GrokServiceInfo` records).
-Requires a valid dev key (standard `grok s` auth). For an anonymous liveness probe —
-load balancer, k8s readiness — hit `/admin/health` directly; it's on the server's
-unauthenticated allowlist.
+`services` is the same payload as `/admin/health` (per-`GrokServiceInfo` records). A server
+that reports no services (a dev stack) prints `(no services reported)`; `--module` for a
+module the server does not report exits 1. Requires a valid dev key (standard `grok s` auth).
+For an anonymous liveness probe — load balancer, k8s readiness — hit `/admin/health` directly;
+it's on the server's unauthenticated allowlist.
+
+## Describing an entity type
+
+```bash
+grok s describe connections                    # fields of a DataConnection, with types and examples
+grok s describe Project --output json          # by type name; JSON carries the registry record and a sample
+grok s describe users --output quiet           # field names only
+```
+
+The server publishes no JSON schema, so `describe` combines the entity-type registry record
+with the top-level fields of one existing entity of that type (`field`, `type`, `example`).
+Aliases: `users groups connections queries scripts functions packages reports tables projects
+files`; any other registered type name (`Project`, `ViewLayout`) works when at least one
+entity exists.
 
 ## Raw API access
 
 When no dedicated subcommand exists, fall through to `grok s raw`:
 
 ```bash
-grok s raw GET  /api/users/current
-grok s raw GET  /api/packages/dev/MyPlugin
-grok s raw POST /api/admin/reload-settings
+grok s raw GET  /users/current
+grok s raw GET  /packages/dev/MyPlugin
+grok s raw POST /admin/reload-settings
+grok s raw POST /public/v1/functions/Sin/call --data '{"x": 1}'
+grok s raw POST /domains/grants/<id> --json grant.json
 ```
 
+Paths are relative to the API base of the target server (`/users/current` becomes
+`https://host/api/users/current`, or `http://host:8082/users/current` on a bare Datlas); a
+leading `/api` is accepted and dropped, so old `/api/...` paths keep working on both host
+shapes. A body comes from `--json <file>` or `--data '<json>'`. A non-2xx answer, or a
+200 carrying an `ApiError`, prints the message with the HTTP status on stderr and exits 1;
+under `--output json` the stderr line is `{"error", "errorCode", "body"}`.
+
 On **Windows Git Bash**, prefix raw paths with `MSYS_NO_PATHCONV=1` to stop the shell
 from rewriting POSIX paths into Windows paths:
 
 ```bash
-MSYS_NO_PATHCONV=1 grok s raw GET /api/users/current
+MSYS_NO_PATHCONV=1 grok s raw GET /users/current
 ```
 
 ## Batch operations
@@ -292,14 +515,17 @@ Manifest shape:
   "stopOnError": true,
   "transaction": false,
   "operations": [
-    { "id": "op1", "action": "users.save",  "params": {"login": "alice", "firstName": "Alice"} },
-    { "id": "op2", "action": "groups.save", "params": {"name": "Chemists"} }
+    { "id": "op1", "action": "users.create",  "params": {"login": "alice", "firstName": "Alice"} },
+    { "id": "op2", "action": "groups.create", "params": {"name": "Chemists"} }
   ]
 }
 ```
 
-For `files.put`, add `"source": "<local-path>"` and the CLI base64-encodes the file
-into `content` before sending.
+Actions the server accepts: `create | get | delete` for `users`, `groups`, `connections`,
+`functions`, `queries`, `scripts` (`get | delete` for `reports`), `functions.run`
+(`{name, params}`), and `files.list | get | put | delete`. For `files.put`, add
+`"source": "<local-path>"` and the CLI base64-encodes the file into `content` before sending.
+`users.delete` removes the entity record only (see "List / count / get / delete").
 
 ## Scripting pattern
 
@@ -354,13 +580,323 @@ grok s groups add-members Chemists alice.mendeleev bob.curie carol.pauling --use
 grok s groups list-members Chemists --no-admin
 ```
 
-Every subcommand above is idempotent — re-running the whole block is safe.
+Steps 1 and 2 create; re-running them fails with "already exists" (exit 1) unless the JSON
+carries the existing `id`. Step 3 is idempotent (`noop` on a re-run).
+
+## Migrating entities between instances (pull / push / migrate)
+
+Entities built in the UI on one instance — connections, queries, scripts, dashboards,
+spaces, layouts, tables, files, jobs, notebooks, models, and the groups and grants they
+need — are promoted to another instance through a **bundle**: a directory of one JSON file
+per entity that travels by any means (a commit, a PR, a USB stick).
+
+```bash
+grok s pull   <selection> --out ./bundle --host dev      # instance → directory
+grok s bundle ls ./bundle                                # what is in it
+grok s diff   ./bundle --host prod                       # what a push would change
+grok s push   ./bundle --host prod [--dry-run]           # directory → instance
+grok s migrate <selection> --from dev --to prod          # pull + push, temp dir in between
+```
+
+Every entity keeps the **same UUID** on both instances, so a push is idempotent: pushing an
+unchanged bundle a second time writes nothing, and 1.28's built-in server-to-server sync
+recognises what the CLI pushed as its own.
+
+Moving a **whole instance** rather than one team's work needs `--admin` on both ends, so the
+run sees and writes what the key's own account cannot: without it a pull collects only the
+entities shared with that account, and the target's own users never get their content back.
+
+```bash
+grok s pull --since 10y --type project,connection,query,script,group,layout   --out ./bundle --host dev --admin
+grok s push ./bundle --host prod --admin --on-conflict skip
+```
+
+**Move a whole instance one space at a time.** Placement is exclusive on the server — one project
+holding an entity takes it from every other — so a single bundle carrying every space has its
+projects competing, and a stand-sized push spends its time taking rows back off each other. Scoped
+to one space that contention stays inside it:
+
+```bash
+grok s migrate --from dev --to prod --admin --by-namespace          # every space, in turn
+grok s migrate --from dev --to prod --admin --by-namespace --only Chem,Bio
+```
+
+It checks the instance-level prerequisites once before writing anything — users and packages the
+target lacks — and refuses a whole-instance run until they are fixed (`--force` overrides; a run
+already scoped with `--only` is reported but allowed). `--skip` leaves a space out, and a full run
+ends with an `(unowned)` sweep for leaf entities no space holds — a layout under no namespace would
+otherwise never travel. `--no-sweep` turns that off. Each space is a separate pull and push, so a
+failure costs one space rather than the run, and what finished is recorded in a state file
+(`--state`), letting a re-run continue instead of repeating work.
+
+Measured on 1.27.9: three spaces, 1,946 entities, 4m44s, one failure isolated to its space; the
+re-run skipped all three in 8s. The same content as part of a 15,778-entity single bundle did not
+finish at all.
+
+**Name the content types; do not pull tables directly.** A long-lived stand accumulates a loose
+`TableInfo` per ad-hoc import — dev holds 1.76 M of them against 21 K projects, and an admin
+session sees every one. Selecting projects instead pulls each project's tables, views and layouts
+as dependencies, so everything a dashboard needs still travels and the scratch rows stay behind.
+Check what a stand is carrying before deciding:
+
+```bash
+grok s tables list --host dev --admin --limit 1 --output json   # `count` ignores --filter
+```
+
+`--type` takes the aliases `conn`/`connection`, `query`, `script`, `project`/`dashboard`/`space`,
+`view`, `layout`, `table`, `file`, `group`, `job`, `notebook`, `model`, or an exact type name
+(`DataConnection`) — a lowercased type name is not an alias and is rejected.
+
+### Preparing the target
+
+Four things belong to the instance, not to a bundle, and have to be in place before the push.
+`grok s diff` reports missing users; missing packages are reported by the push itself, since the
+check runs against what the push is about to write.
+
+| | Why | How to check |
+|---|---|---|
+| **Users** | Content of a user who does not exist lands under the pushing account | `warn user_missing` |
+| **Packages** | Package functions, scripts and connections carry a name-derived id that only exists once the package is *published* there — installing it from the repository is not enough | `warn package_not_installed` |
+| **Platform shares** | `System:AppData` and `System:DemoFiles` are created by the deployment; entities that read from them need them present | `grok s connections list --host prod` |
+| **Credentials** | Secrets never travel — set them on the target after the push, or pass `--creds` | `needs-credentials` rows |
+
+A freshly deployed stand also stops at its setup wizard, and 1.27.x keeps the session token in
+memory rather than a cookie — so that redirect logs you straight back out. Click the wizard
+through once and normal login works.
+
+Users are **not** part of a bundle. Create them on the target first — content that belongs to
+a user who does not exist there lands under the pushing account instead of theirs.
+
+### Surviving a blip
+
+A retriable answer (429, 502, 503, 504) or a dropped socket is retried with exponential backoff —
+five attempts by default, about half a minute. That covers a busy moment, not a stand that steps
+out for a restart, and a whole-instance walk is long enough to meet one:
+
+```bash
+GROK_HTTP_RETRIES=9 grok s pull --out ./bundle --host dev --admin ...   # ~90s of tolerance
+```
+
+A pull that dies anyway is not wasted — pulls accumulate, so re-running without `--replace`
+merges into the same bundle.
+
+### A bundle remembers where it was pushed
+
+`--on-conflict adopt` records every adoption in the bundle's own `idmap.json`, so a re-push
+writes into the same target rows instead of adopting again. That is what makes a repeated push
+idempotent — but it also means the bundle is no longer neutral: pushed at a *different* target,
+or re-pushed after an aborted attempt, it resolves entities by the ids it learned last time and
+updates rows it would otherwise have created.
+
+Pull into a fresh directory (or delete `idmap.json`) whenever a bundle is aimed at a different
+instance than the one it last adopted against.
+
+### Conflict policies
+
+Measured against a target holding 924 of the bundle's 1,162 names:
+
+| `--on-conflict` | What lands | Cost |
+|---|---|---|
+| `fail` (default) | nothing; every conflict is listed first | safest way to see the damage before writing |
+| `skip` | the non-conflicting entities only | most failures — whatever depended on a skipped entity fails too |
+| `adopt` | conflicting names are written into the target's existing rows | keeps one copy; the usual choice for a real migration |
+| `duplicate` | a second copy of everything that collides | target grows by the whole bundle, tables bring their columns |
+
+### References the source has already lost
+
+A long-lived instance accumulates entities that point at something deleted out from under them —
+most often a saved view whose table is gone. The pull resolves every outside reference against the
+source, so a bundle records which of them the source itself can no longer answer:
+
+```
+Bundle  warn  source_dangling_refs  50 reference(s) point at entities the source itself no longer has
+```
+
+The push then reports whatever depends on one as `skip dead_on_source` rather than as a failure,
+and does not fail the run over it — no target can supply what the source has already lost. A
+reference the source *does* keep but the target lacks stays a real failure (`dependency_missing`),
+because installing the package or widening the pull fixes it.
+
+### Bundle layout
+
+```
+bundle/
+  manifest.json                     # source url + version, pull history, FK-safe order
+  DataConnection/Chem.Chembl.json   # <Type>/<nqName with ':' and '/' as '.'>.json
+  DataQuery/Chem.CompoundsByTarget.json
+  Project/Chem.Dashboard.json
+  UserGroup/Chemists.json           # bare group + member logins, never member ids
+  FileInfo/reports.readme.md.json   # a file without a namespace is named by its path
+  tables/<id>.d42                   # table data (on by default for pulled tables)
+  files/<id>                        # file bytes (only with --include-files)
+  shares/<encoded path>             # share files a datasync table rebuilds itself from
+  idmap.json                        # sourceId -> targetId, written whenever the push resolves an
+                                    # entity to a different id on the target — an adoption, a
+                                    # package entity found by name, or a save that answered with
+                                    # an existing row's id
+```
+
+**Pulls accumulate.** Pulling into an existing bundle merges: entities already there are
+overwritten by id, new ones are added, nothing is removed, and `manifest.pulls[]` keeps one
+entry per invocation. `--replace` clears the directory first.
+
+```bash
+grok s pull Chem:TargetDashboard --out ./release --host dev
+grok s pull --type script --author alice --no-deps --out ./release --host dev
+grok s pull Chemists --out ./release --host dev
+grok s push ./release --host prod
+```
+
+### Selection
+
+| Flag | Selects |
+|---|---|
+| positional `Chem:Dashboard <uuid> ...` | exactly these entities, by nqName or id |
+| `--type conn,query,script,project,dashboard,space,view,layout,table,file,group,job,notebook,model` | which types to list (default: all of them) |
+| `--name <glob>` | free-text search, then a client-side glob on name and friendly name (`Cereal*`, `*demo*`) |
+| `--namespace Chem` | everything **under** the namespace, recursively, plus the space `Chem` itself — without it nothing selected has anywhere to be placed |
+| `--space Chem:Reports` | the same, named by nqName rather than by namespace |
+| `--author alice` | entities authored by a login |
+| `--tag demo` | entities carrying a tag (types whose router has no `tags` param are skipped with a warning) |
+| `--since 2w` / `--since=-30d` / `--since 2026-08-01` | updated since (bare `2w` means `-2w`; the shell eats a leading `-` unless you use `=`) |
+| `--filter "<expr>"` | a smart-filter expression, ANDed with the rest |
+| `--no-deps` | do not follow dependencies (what never travels is still excluded — see below) |
+| `--no-include-data` | do not pull the `.d42` data of the tables that were pulled (data is on by default) |
+| `--include-files` | also pull the bytes of the files that were pulled (off by default) |
+| `--verbose` | print the stack of a runtime failure instead of the one-line message |
+
+`--type space` and `--type dashboard` are both `Project` with a different listing rule, so
+they cannot be combined in one command (neither can `--type project,space`).
+
+### What the walker adds, and what never travels
+
+A selected entity brings its dependencies with it: a project brings its relation children
+(recursively), their views, layouts and tables; a query brings its connection; a datasync
+table brings the connection its creation script opens; a notebook and a model bring their
+tables; every entity brings the non-personal groups that hold grants on it, and a group
+brings its parents and members. A **job** brings nothing — 1.27 does not persist the link
+between a job and the queries it runs, so pull those explicitly.
+
+The table below holds however the entity was chosen: `--no-deps` skips the walk, not these
+rules, and `push` refuses the same connections even if a bundle was hand-edited to carry one
+(`skip(platform_connection)` / `skip(personal_storage)` / `skip(space_files_connection)`).
+
+| Never travels | Row on the report |
+|---|---|
+| Passwords and other password-class connection parameters | `needs-credentials` (see `--creds`) |
+| Users — memberships are replayed by login and group name | `warn(member_not_found)` when a login is not on the target |
+| Package-owned entities — install the package instead | `warn(package_entity)`, `warn(package_not_installed)` |
+| `System:` connections, the personal `Home` share, a space's own `Files` connection | `info(platform_connection)` / `warn(personal_storage)` / `info(space_files_connection)` |
+| A trained model blob | `info(model_blob_skipped)` |
+
+### Credentials: `--creds`
+
+A pushed connection arrives without its secrets. Author a YAML file **for the target** and
+pass it to `push` or `migrate`; the values are merged into the connection's parameters before
+the save, and the server encrypts and masks them itself. Nothing is ever read back into the
+bundle.
+
+```yaml
+# creds.yaml — keys are connection nqNames as the bundle spells them
+Chem:Chembl:
+  password: ${CHEMBL_PROD_PASSWORD}
+Admin:Northwind:
+  password: ${NORTHWIND_PASSWORD}
+```
+
+```bash
+CHEMBL_PROD_PASSWORD=... grok s push ./release --host prod --creds ./creds.yaml
+```
+
+`${VAR}` is resolved from the environment exactly as `grok publish` resolves it in
+`connections/*.json` (the file is parsed as YAML first, so a `${VAR}` written in flow style
+needs quoting: `{password: "${VAR}"}`); a variable that is not set aborts the push before the
+first write. A connection the file covers gets no `needs-credentials` row, and is **always**
+written — a secret is invisible in the payload, so a connection that would otherwise be
+`identical` is planned as `update` with reason `credentials`. That is how a password is
+rotated: re-run the push with a new value.
+
+### Conflicts
+
+An entity whose id is absent on the target but whose name is taken by another id is a
+conflict. `--on-conflict` decides:
+
+| Policy | What happens |
+|---|---|
+| `fail` (default) | nothing is written; every conflict is listed and the command exits 1 |
+| `skip` | the twin is left alone; anything in the bundle that points at it is reported `failed(dependency_skipped)` |
+| `adopt` | the bundle entity is written **into** the twin, and `idmap.json` records `sourceId -> targetId` so every later reference and every later push follows it |
+| `duplicate` | the bundle entity is created under its own id next to the twin (the server renames it `Name_1`). A `UserGroup` cannot be duplicated — group names are unique on 1.27, so that row fails |
+
+### Reading the plan
+
+`diff` and `--dry-run` print the plan without writing (`diff` plans with `skip`, so a
+conflict does not abort it). Actions: `create`, `update` (with the changed top-level keys as
+the detail), `identical` (no write), `skip`, `failed`, plus the `warn` / `info` /
+`needs-credentials` notes. `--output json` emits the 1.28-compatible shape — the same one for
+`diff`, `push` and `migrate`, with `detail` always present (`""` when there is nothing to
+say):
+
+```json
+{
+  "items": [{"name": "Chem:Chembl", "entityType": "DataConnection", "action": "create", "reason": "", "detail": ""}],
+  "counts": {"create": 1, "identical": 4},
+  "status": "ok",
+  "remoteUrl": "https://prod.datagrok.ai/api"
+}
+```
+
+A push exits 1 if any row is `failed`.
+
+### 1.27 limits worth knowing
+
+- A **FileInfo that lives in a share** is a dead row on the target — only stand-alone blobs
+  migrate (`info(file_in_share_not_migratable)`), the same rule the 1.28 sync applies.
+- `metaParams` do not survive a save on connections and queries, so metadata attached there
+  does not travel.
+- `GET /projects/relations` fails on a project that links domain-table rows; the walker falls
+  back to the project's own `relations[]` and reports `warn(relations_degraded)`.
+- Relations are **merged**, never replaced: a link that exists only on the target is kept, so
+  removing a relation from a bundle never unlinks it there (`info(relation_not_removed)`).
+  Unlink it in the UI on the target instead.
+- A space delete leaves its `…:Files` connection behind, and it cannot be deleted through the
+  API. Deleting a group that still holds grants fails — revoke the grants first.
+
+### dev to prod, end to end
+
+```bash
+# 1. see what would come over, from dev, without touching prod
+grok s migrate Chem:TargetDashboard --from dev --to prod --dry-run
+
+# 2. promote it, filling in the target's secrets
+CHEMBL_PROD_PASSWORD=... grok s migrate Chem:TargetDashboard \
+  --from dev --to prod --creds ./creds.yaml --keep
+
+# 3. re-run: everything is `identical`, nothing is written
+grok s migrate Chem:TargetDashboard --from dev --to prod
+
+# 4. or keep the bundle under version control instead
+grok s pull Chem:TargetDashboard --out ./release --host dev
+git add release && git commit -m "release: target dashboard"
+grok s push ./release --host prod --creds ./creds.yaml
+```
+
+`migrate` pulls into a temporary directory and deletes it afterwards; `--keep` keeps it and
+prints its path on **stderr**, so `--output json` stays one parseable document. `--from` is
+only ever read from.
 
 ## Implementation notes
 
-- Source: `public/tools/bin/commands/server.ts`, `public/tools/bin/utils/node-dapi.ts`.
+- Source: `public/tools/bin/commands/server.ts`, `public/tools/bin/utils/node-dapi.ts`;
+  pull / push / migrate in `bin/commands/server-migrate.ts` + `bin/utils/migrate/`.
+  Domain schemas and rows in `bin/commands/server-domains.ts` (`NodeDomainsDataSource` in
+  `node-dapi.ts`); they call the internal `/domains/` router the browser uses.
 - The Node client talks directly to `/public/v1/` — no Dart interop, no browser, no
   logged-in session required. Authentication uses the developer key from the config.
 - If `grok s` is not working, start by running `grok s healthcheck` — it verifies the
   URL, the key, and basic connectivity, and returns per-module status if the server is
-  reachable. Fall back to `grok s raw GET /api/users/current` to isolate auth issues.
+  reachable. Fall back to `grok s raw GET /users/current` to isolate auth issues. A `--host`
+  URL that is not the API base fails at login with the reason (`should end with /api`).
+- Cross-instance sync (1.28 servers): `grok s sync pairs list`, `sync setups list --pair <id>`,
+  `sync setup get <id>`, `sync run <id>`; on 1.27 the routes do not exist and the commands
+  answer "not found".