dataiku-sdk 2.0.0 → 3.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (80) hide show
  1. package/LICENSE +107 -17
  2. package/README.md +144 -17
  3. package/dist/build-metadata.json +1 -1
  4. package/dist/packages/types/src/index.d.ts +39 -4
  5. package/dist/packages/types/src/index.js +31 -2
  6. package/dist/src/cli/commands/api-service.js +62 -16
  7. package/dist/src/cli/commands/bundle.js +49 -15
  8. package/dist/src/cli/commands/code-env.js +210 -49
  9. package/dist/src/cli/commands/code.d.ts +19 -0
  10. package/dist/src/cli/commands/code.js +87 -15
  11. package/dist/src/cli/commands/dataset.js +8 -3
  12. package/dist/src/cli/commands/flow-zone.js +33 -5
  13. package/dist/src/cli/commands/notebook.js +72 -41
  14. package/dist/src/cli/commands/project-git.js +8 -5
  15. package/dist/src/cli/commands/project-library.d.ts +21 -0
  16. package/dist/src/cli/commands/project-library.js +121 -50
  17. package/dist/src/cli/commands/project.js +16 -2
  18. package/dist/src/cli/commands/recipe.js +37 -12
  19. package/dist/src/cli/commands/sql.d.ts +10 -0
  20. package/dist/src/cli/commands/sql.js +39 -17
  21. package/dist/src/cli/commands/webapp.js +67 -13
  22. package/dist/src/cli/contract.d.ts +15 -0
  23. package/dist/src/cli/contract.js +870 -107
  24. package/dist/src/cli/env.d.ts +9 -0
  25. package/dist/src/cli/env.js +20 -2
  26. package/dist/src/cli/flags.js +9 -0
  27. package/dist/src/cli/helpers/cleanup.js +104 -3
  28. package/dist/src/cli/helpers/flow-zone.d.ts +3 -0
  29. package/dist/src/cli/helpers/flow-zone.js +33 -3
  30. package/dist/src/cli/helpers/recipe.d.ts +20 -0
  31. package/dist/src/cli/helpers/recipe.js +158 -3
  32. package/dist/src/cli/output.d.ts +2 -0
  33. package/dist/src/cli/output.js +33 -6
  34. package/dist/src/cli/runtime.d.ts +35 -1
  35. package/dist/src/cli/runtime.js +125 -18
  36. package/dist/src/cli/types.d.ts +7 -0
  37. package/dist/src/cli.js +135 -28
  38. package/dist/src/client.d.ts +60 -0
  39. package/dist/src/client.js +237 -42
  40. package/dist/src/errors.d.ts +8 -2
  41. package/dist/src/errors.js +40 -0
  42. package/dist/src/index.d.ts +4 -3
  43. package/dist/src/index.js +2 -1
  44. package/dist/src/resources/api-services.d.ts +48 -6
  45. package/dist/src/resources/api-services.js +54 -8
  46. package/dist/src/resources/bundles.d.ts +51 -6
  47. package/dist/src/resources/bundles.js +39 -9
  48. package/dist/src/resources/code-envs.d.ts +25 -3
  49. package/dist/src/resources/code-envs.js +104 -2
  50. package/dist/src/resources/datasets.d.ts +20 -0
  51. package/dist/src/resources/datasets.js +232 -52
  52. package/dist/src/resources/jobs.d.ts +35 -10
  53. package/dist/src/resources/jobs.js +92 -17
  54. package/dist/src/resources/notebooks.d.ts +77 -7
  55. package/dist/src/resources/notebooks.js +143 -19
  56. package/dist/src/resources/project-git.d.ts +12 -0
  57. package/dist/src/resources/project-git.js +39 -8
  58. package/dist/src/resources/project-library.d.ts +122 -7
  59. package/dist/src/resources/project-library.js +494 -25
  60. package/dist/src/resources/projects.d.ts +5 -2
  61. package/dist/src/resources/projects.js +18 -2
  62. package/dist/src/resources/scenarios.d.ts +30 -0
  63. package/dist/src/resources/scenarios.js +73 -27
  64. package/dist/src/resources/sql.d.ts +6 -0
  65. package/dist/src/resources/sql.js +22 -2
  66. package/dist/src/resources/webapps.d.ts +35 -4
  67. package/dist/src/resources/webapps.js +54 -5
  68. package/dist/src/schemas.d.ts +2 -2
  69. package/dist/src/schemas.js +1 -1
  70. package/dist/src/skill.d.ts +9 -0
  71. package/dist/src/skill.js +37 -4
  72. package/dist/src/utils/flow-analysis.d.ts +53 -0
  73. package/dist/src/utils/flow-analysis.js +382 -0
  74. package/dist/src/utils/project-archive.d.ts +41 -2
  75. package/dist/src/utils/project-archive.js +159 -16
  76. package/package.json +5 -5
  77. package/packages/types/dist/index.d.ts +39 -4
  78. package/packages/types/dist/index.js +31 -2
  79. package/plugin.json +2 -2
  80. package/skills/dataiku-dss/SKILL.md +75 -48
@@ -5,7 +5,7 @@ description: >-
5
5
  datasets, recipes, jobs, scenarios, folders, notebooks, SQL, variables,
6
6
  code envs, and connections. Discover scoped command metadata with
7
7
  dss commands run --fields RESOURCE.ACTION.
8
- license: MIT
8
+ license: LicenseRef-Dataiku-SDK-Limited-Use-1.0
9
9
  compatibility: >-
10
10
  Requires the dss CLI, network access to a Dataiku DSS instance, configured
11
11
  DATAIKU_URL and DATAIKU_API_KEY credentials, and Bun >= 1.4.0.
@@ -14,19 +14,16 @@ compatibility: >-
14
14
  # Dataiku DSS agent CLI
15
15
 
16
16
  Use `dss` when an agent needs to inspect or change Dataiku DSS resources: projects, datasets, recipes, jobs, scenarios, folders, notebooks, SQL, variables, code envs, and connections.
17
- If the installed `dss` binary is unavailable but the repository checkout is the current workspace, prefer `bun --no-env-file src/cli.ts ...` or the packaged launcher `bun --no-env-file ./bin/dss.js ...`. From another working directory, pass the checkout's absolute `bin/dss.js` path to Bun.
17
+ If the installed `dss` binary is unavailable but the checkout is the workspace, prefer `bun --no-env-file src/cli.ts ...` or `bun --no-env-file ./bin/dss.js ...`; from another working directory, pass the checkout's absolute `bin/dss.js` path to Bun.
18
18
  `--no-env-file` disables Bun's automatic preloading only; the CLI still applies its documented `.env` handling unless `DATAIKU_DISABLE_ENV=1` is set.
19
19
 
20
20
  ## Contract
21
21
 
22
- - Command results write exactly one compact JSON value to stdout; void success is `{ok:true}`. `doctor`, `batch`, and `cleanup` failure reports are their direct result objects on stdout with a nonzero exit code — use the exit code and result fields.
23
- - Dispatch/runtime failures write one compact structured error object on stdout (`type:"error"`, `ok:false`, `error`, `code`, `category`, `exitCode`) with a nonzero exit code; stderr carries JSONL diagnostics only.
24
- - Non-fatal warnings and `--verbose` HTTP traces are JSONL stderr events (`type:"warning"` / `type:"trace"`), flushed before both success and failure output, never prose.
25
- - No prompts, help screens, tables, banners, or prose output are part of the contract.
26
- - Exit codes: 0 success, 1 usage/configuration error, 2 DSS or internal error, 3 transient/retryable DSS error, 4 a failed long-running result or synchronous assertion.
27
- - Every result is compact single-line JSON on stdout; output formatting is not configurable.
22
+ - Command results write exactly one compact JSON value to stdout; void success is `{ok:true}`. `doctor`, `batch`, and `cleanup` failure reports are their direct result objects on stdout — use the exit code and result fields.
23
+ - Dispatch/runtime failures write one compact structured error object on stdout (`type:"error"`, `ok:false`, `error`, `code`, `category`, `exitCode`) with a nonzero exit code; stderr carries JSONL diagnostics only. Warnings and `--verbose` HTTP traces are JSONL stderr events (`type:"warning"` / `type:"trace"`), flushed before success and failure output, never prose.
24
+ - No prompts, help screens, tables, banners, or prose output are part of the contract. Exit codes: 0 success, 1 usage/configuration error, 2 DSS or internal error, 3 transient/retryable DSS error, 4 a failed long-running result or synchronous assertion.
28
25
  - Recipe payload commands write the payload as a JSON string on stdout; with `--output PATH` the exact bytes go to a file and stdout carries the JSON string equal to `PATH`.
29
- - `--fields a,b,c` projects those fields from object or array-of-objects results; dotted paths (`a.b.c`) drill into nested objects, and missing fields become `null`; string and scalar results pass through unchanged.
26
+ - `--fields a,b,c` projects those fields from object or array-of-objects results; dotted paths (`a.b.c`) drill into nested objects, missing fields become `null`; strings and scalars pass through unchanged.
30
27
 
31
28
  ## Discover commands
32
29
 
@@ -40,23 +37,23 @@ dss agent contract --fields protocol,agentContractVersion,cli,stdio,planning,com
40
37
  dss agent contract --fields commands.actions
41
38
  ```
42
39
 
43
- `dss commands run` prints the compact resource/action summary — every resource keyed to its action names, about 1k tokens — and never dumps registry entries to stdout. Bootstrap with the scoped `agent contract` call above: `protocol,agentContractVersion,cli,stdio,planning,compatibility` cover protocol/schema compatibility, stdout/stderr stream semantics, preferred discovery commands, planning rules, and compatibility guarantees in ~250 tokens. Request `schemas` (the JSON schemas) or `commands` (the contract's discovery section) only when you actually need them.
40
+ `dss commands run` prints the compact resource/action summary — every resource keyed to its action names, about 1k tokens — and never dumps registry entries to stdout. Bootstrap with the scoped `agent contract` call above: those six fields cover protocol/schema compatibility, stream semantics, preferred discovery commands, planning rules, and compatibility guarantees in ~250 tokens; request `schemas` or `commands` only when needed.
44
41
 
45
- Before choosing command syntax, look the action up in the command registry: `--fields RESOURCE.ACTION` returns one complete entry, `--fields RESOURCE` returns every action of one resource, and appended `.FIELD` paths return only the metadata you need. Prefer the four-field projection `usage,description,flags,examples` by default — it is the smallest self-sufficient starting point for constructing an invocation. The registry entry is the canonical schema for flags, positional arguments, side effects, auth requirements, output shape, idempotency, dry-run support, structured examples, payload schemas, unsafe outputs, cleanup hints, and exit codes.
42
+ Before choosing command syntax, look the action up in the registry: `--fields RESOURCE.ACTION` returns one complete entry, `--fields RESOURCE` every action of one resource, and appended `.FIELD` paths only the metadata you need. Prefer the four-field projection `usage,description,flags,examples` by default — the smallest self-sufficient starting point for an invocation. Each registry entry is the canonical schema for flags, positional arguments, side effects, auth requirements, output shape, idempotency, dry-run support, structured examples, payload schemas, unsafe outputs, cleanup hints, and exit codes.
46
43
 
47
- `--fields` takes a comma-separated list, so request several actions in one call. Each key is echoed exactly as requested (`--fields dataset.create` returns `{"dataset.create": {...}}`); an empty `--fields` is a usage error, never a silent full dump. The full registry (~220k tokens) is exported only via `--output PATH`: `dss commands run --output PATH` writes the registry, or the selected `--fields` subset, as compact JSON to `PATH`, and stdout carries the compact `{"path":"PATH"}` result — never the registry itself. An unknown resource or action exits with code 1 and a compact JSON error object on stdout containing the valid options.
44
+ `--fields` takes a comma-separated list, so request several actions in one call. Each key is echoed exactly as requested (`--fields dataset.create` returns `{"dataset.create": {...}}`); an empty `--fields` is a usage error, never a silent full dump. The full registry (~220k tokens) is exported only via `--output PATH`: `dss commands run --output PATH` writes the registry, or the selected `--fields` subset, as compact JSON to `PATH`, and stdout carries `{"path":"PATH"}` — never the registry itself. An unknown resource or action exits with code 1 and a compact JSON error object on stdout containing the valid options.
48
45
  Credential lookup order is flags first, then `DATAIKU_*` environment variables, then saved credentials.
49
- Set `DATAIKU_DISABLE_ENV=1` when a test must ignore both `.env` files and `DATAIKU_*` environment variables.
50
- When `.env` loading is enabled, the CLI reads `.env` from the command's current working directory first and then the CLI build/root directory; the invocation directory wins on conflicting keys. Put test-specific `.env` files in the directory where you invoke `dss`.
46
+ Set `DATAIKU_DISABLE_ENV=1` to ignore both `.env` files and `DATAIKU_*` variables.
47
+ With `.env` loading enabled, the CLI reads `.env` from the command's current working directory first, then the CLI root; the invocation directory wins on conflicts, so put test-specific `.env` files where you invoke `dss`.
51
48
  For disposable agent tests, set `DSS_CONFIG_DIR` to a temporary directory so saved credentials never touch the real profile.
52
49
 
53
50
  ## Planning, safety, and generic inputs
54
51
 
55
- - Before any registry entry with `sideEffect:"write"`, run the exact argv with `--plan` first. Planning is local: it returns the derived operation without resolving credentials or calling DSS. Check `destructive`, `idempotency`, `async`, `unsafeOutputs`, and `exitCodes` before execution.
56
- - Use `--dry-run` only when that action's registry entry has `dryRun:true`. `--plan` explains the operation; `--dry-run` exercises the action-specific simulation path. Never add an unsupported flag.
57
- - When a create/upload action advertises `--record-cleanup`, pass `--record-cleanup cleanup.jsonl`. `dss cleanup --file cleanup.jsonl` previews the recorded cleanup steps in reverse order and does not mutate DSS; add `--apply` only after checking the preview.
58
- - For JSON payload actions, follow `inputContract`, `requiredFlags`, and `requiredOneOf`. Prefer `--data-file PATH` or `--stdin` when advertised instead of inline `--data`; this preserves exact JSON across shells and keeps large or sensitive payloads out of process arguments.
59
- - Authenticated actions advertise `--request-timeout MS` and `--retries N`; `--retries` applies to idempotent GET requests. Long-running actions separately advertise controls such as `--timeout MS`, `--poll-interval MS`, and log limits; use only the flags in that action's registry entry.
52
+ - Before any registry entry with `sideEffect:"write"`, run the exact argv with `--plan` first. Planning is local: it returns the derived operation without credentials or DSS calls. Check `destructive`, `idempotency`, `async`, `unsafeOutputs`, and `exitCodes` before executing.
53
+ - Use `--dry-run` only when that action's registry entry has `dryRun:true`. `--plan` explains the operation; `--dry-run` exercises the simulation path. Never add an unsupported flag.
54
+ - When a create/upload action advertises `--record-cleanup`, pass `--record-cleanup cleanup.jsonl`. `dss cleanup --file cleanup.jsonl` previews recorded steps in reverse order without mutating DSS; add `--apply` only after checking it.
55
+ - For JSON payload actions, follow `inputContract`, `requiredFlags`, and `requiredOneOf`. Prefer `--data-file PATH` or `--stdin` when advertised over inline `--data`; this preserves exact JSON across shells and keeps large or sensitive payloads out of argv.
56
+ - Authenticated actions advertise `--request-timeout MS` and `--retries N`; `--retries` applies to idempotent GET requests. Long-running actions advertise `--timeout MS`, `--poll-interval MS`, and log limits; use only the flags in that registry entry.
60
57
  - Before live mutation tests, use `dss fixtures` to discover compatible test resources instead of guessing project objects.
61
58
 
62
59
  ## Authentication
@@ -93,17 +90,16 @@ To persist credentials for later invocations:
93
90
  dss auth login --url https://dss.example.com --api-key YOUR_KEY --project-key MYPROJ
94
91
  ```
95
92
 
96
- The command saves credentials and returns `{"saved":true,"path":"..."}`. `DSS_CONFIG_DIR` wins when set. Otherwise credentials use `XDG_CONFIG_HOME/dataiku/credentials.json`, the `dataiku/credentials.json` directory under `APPDATA` on Windows, or `~/.config/dataiku/credentials.json`.
97
- `auth login` validates by listing accessible projects before saving credentials, so the API key must be allowed to call DSS project-list APIs.
93
+ The command saves credentials and returns `{"saved":true,"path":"..."}`. `DSS_CONFIG_DIR` wins when set; otherwise credentials use `XDG_CONFIG_HOME/dataiku/credentials.json`, the `dataiku/credentials.json` directory under `APPDATA` on Windows, or `~/.config/dataiku/credentials.json`.
94
+ `auth login` validates by listing accessible projects before saving, so the key must be allowed to call DSS project-list APIs.
98
95
 
99
- TLS flags: `--insecure` disables certificate verification; `--ca-cert PATH` adds a PEM CA bundle. Environment equivalents: `NODE_TLS_REJECT_UNAUTHORIZED`, `NODE_EXTRA_CA_CERTS`.
96
+ TLS: `--insecure` disables certificate verification; `--ca-cert PATH` adds a PEM CA bundle. Environment equivalents: `NODE_TLS_REJECT_UNAUTHORIZED`, `NODE_EXTRA_CA_CERTS`.
100
97
 
101
98
  ## Common workflows
102
99
 
103
100
  ```text
104
101
  dss version
105
102
  dss project list
106
- dss dataset list --project-key MYPROJ
107
103
  dss recipe get-payload compute_orders --output code.py --project-key MYPROJ
108
104
  dss recipe diff compute_orders --file code.py --project-key MYPROJ
109
105
  dss recipe set-payload compute_orders --file code.py --project-key MYPROJ
@@ -111,16 +107,16 @@ dss job build-and-wait orders --include-logs --project-key MYPROJ
111
107
  dss sql query --connection analytics --sql "select 1" --project-key MYPROJ
112
108
  dss batch --data-file steps.json
113
109
  ```
114
- For fake-DSS smoke tests, return project lists as JSON arrays such as `[{"projectKey":"MYPROJ","name":"My Project"}]` from `/public/api/projects/`; recipe payload commands read `/public/api/projects/<PROJECT>/recipes/<NAME>?includePayload=true` and expect a JSON object shaped like `{"recipe":{"name":"<NAME>","type":"python"},"payload":"..."}`.
110
+ For fake-DSS smoke tests, return project lists as JSON arrays such as `[{"projectKey":"MYPROJ","name":"My Project"}]` from `/public/api/projects/`; recipe payload commands read `/public/api/projects/<PROJECT>/recipes/<NAME>?includePayload=true` expecting `{"recipe":{"name":"<NAME>","type":"python"},"payload":"..."}`.
115
111
 
116
112
  ## Application release safety
117
113
 
118
114
  Treat app release as explicit validate, compare, version, successor, verify, and permission gates.
119
115
  The public app-manifest `version` field is raw metadata: writing it is NOT a publish transaction.
120
116
  New instances inherit the template's raw `version`; existing instances are never upgraded in
121
- place — create an additive successor (the old instance is preserved), verify it, then retire the
122
- old one as a separate guarded step. Never infer private publish/recreate/rename,
123
- recipient-sharing, or UI-click operations that the public DSS API does not expose.
117
+ place — create an additive successor (the old instance is preserved), verify it, retire it
118
+ separately. Never infer private publish/recreate/rename, recipient-sharing, or UI-click operations
119
+ the public DSS API lacks.
124
120
 
125
121
  ```text
126
122
  dss app validate-manifest --project-key APP_TEMPLATE
@@ -145,54 +141,85 @@ manifest hash for the next `set-manifest-version --expect-hash` guard.
145
141
  A masked `403` absent from both visible lists is rejected as
146
142
  `target_absence_unverifiable` / `permission_or_environment`; lists prove collisions, not absence.
147
143
  DSS exposes no permission-independent public key-availability endpoint or guaranteed
148
- non-overwriting duplicate-key rejection, so no force or server-atomic bypass is supported. Use
144
+ non-overwriting duplicate-key rejection, so no force or server-atomic bypass is supported; use
149
145
  global project visibility. A definitive rejection writes no cleanup entry. An ambiguous POST
150
146
  without a returned future ID or verified incarnation is `INDETERMINATE` and also produces no cleanup
151
147
  entry. Future-addressable cleanup waits for target identity and `creationTag`; the predecessor is
152
148
  never targeted. Ledgers bind the canonical DSS URL, project key, and concrete project incarnation,
153
- and reject legacy, mixed-server, mismatched-server, or unbound app cleanup entries. Static plans
149
+ rejecting legacy, mixed-server, mismatched-server, or unbound app cleanup entries. Static plans
154
150
  expose `preflightExecuted:false` and `preflightWillRunDuringApply:true`.
155
151
 
156
152
  `app set-manifest-version` reports
157
153
  `concurrencyControl:"client-side-non-atomic-stale-read-check"`. `--expect-hash` is a stale-read
158
- guard; the PUT is unconditional. Never treat the hash as a serializing lock. Ambiguous writes report
159
- `outcome:"indeterminate"`.
154
+ guard; the PUT is unconditional. Never treat the hash as a serializing lock; ambiguous writes
155
+ report `outcome:"indeterminate"`.
160
156
 
161
157
  The API key authenticates public REST only. `app verify-instance` reports
162
158
  `status:"API_VERIFIED_UI_PENDING"`; its external SSO gate requires exercising the affected tiles,
163
159
  forms, and actions. Permission snapshots bind identity and permissions to the server/project
164
160
  incarnation and reject mismatches.
165
161
 
162
+ ## Flow maps and visual organization
163
+
164
+ ```text
165
+ dss project map --render mermaid --project-key MYPROJ
166
+ dss flow-zone plan --project-key MYPROJ > flow-zones.json
167
+ dss flow-zone organize --file flow-zones.json --dry-run --project-key MYPROJ
168
+ dss flow-zone organize --file flow-zones.json --project-key MYPROJ
169
+ ```
170
+
171
+ `project map` returns zones, SCC-safe layers, weak components, diagnostics, and a full-flow
172
+ fingerprint; rendering stays in `rendering.content`. DSS exposes zone positions only, never node
173
+ pixel coordinates.
174
+
175
+ `flow-zone plan` output feeds `flow-zone organize`; inspect the dry-run. Organize checks topology
176
+ before and after writes and skips unchanged moves. Audit recipe payloads separately; layout
177
+ commands never fetch or analyze code.
178
+
166
179
  ## Project Git
167
180
 
168
181
  UAT=`create-branch --duplicate-project`; pull=rebase; checkout=switch. Remotes: DSS Git rules and
169
182
  SSH/server credentials. `--password-env`; `future-wait`.
170
183
 
184
+ ## Coding and libraries
185
+
186
+ Use `project-library` for internal `lib/`; `project-git list-libraries` for external Git repos.
187
+ Safe edit: `get-bytes` → `diff` → `put --expect-sha256 HASH`; create with
188
+ `--if-not-exists`. Paths reject traversal.
189
+
190
+ Run Python with `code run --file`; inspect/diff/backup recipes before `recipe run --dry-run`.
191
+ Notebook saves report hashes and accept `--expect-hash`; output clearing uses DSS DELETE, while
192
+ `unload-jupyter --all` composes session deletes. Inspect code-env definitions/logs before updates;
193
+ guard webapp/API settings with `--expect-hash`. `exact:false` means live state was not guessed:
194
+ use command `--dry-run`. Public APIs expose no plugin authoring, notebook execute/checkpoints, or
195
+ webapp/API delete; use project Git, code/recipes, or the DSS UI.
196
+
171
197
  ## Confirming mutations
172
198
 
173
- Mutations print a small JSON ack to stdout and exit 0 on success (e.g. `{"updated":"NAME","resource":"recipe"}`); on failure they print the error envelope to stdout and exit non-zero. The exit code is the source of truth.
199
+ Mutations print a small JSON ack to stdout and exit 0 on success (e.g. `{"updated":"NAME","resource":"recipe"}`); on failure the error envelope appears on stdout with a non-zero exit. The exit code is the source of truth.
174
200
 
175
- - For portable multi-step writes, prefer `dss batch` (payload: a JSON array of argv arrays). It runs fail-fast, returns one envelope with per-step `ok`/`result`/`error`, and exits non-zero if any step fails.
176
- - If separate processes are required, inspect each exit code and stop before launching the next step. Do not rely on shell chaining: syntax differs between POSIX shells, Windows PowerShell 5.1, PowerShell 7, and Command Prompt.
177
- - Never pipe a mutation into a command that prints a fixed string or merges stderr: a pipeline may return the helper's exit code and report a failed mutation as success.
178
- - To branch in code, key off the exit code or the JSON ack on stdout — never a hardcoded label.
201
+ - For portable multi-step writes, prefer `dss batch` (payload: a JSON array of argv arrays): fail-fast, one envelope with per-step `ok`/`result`/`error`, non-zero exit if any step fails.
202
+ - With separate processes, inspect each exit code and stop before the next step; shell-chaining syntax differs across POSIX shells, PowerShell 5.1/7, and Command Prompt.
203
+ - Never pipe a mutation into a command that prints a fixed string or merges stderr: the helper's exit code can mask a failed mutation as success.
204
+ - Branch on the exit code or the JSON ack on stdout — never a hardcoded label.
179
205
 
180
206
  ## Platform & debugging notes
181
207
 
182
208
  - Pass code and SQL via `--file`/`--sql-file`, not inline: shells (especially PowerShell) mangle quotes, `$`, and newlines in multi-line snippets.
183
- - On a non-UTF-8 console (e.g. Windows cp1252), don't print non-ASCII results; write them to a UTF-8 file and read that, or use `--output PATH`.
184
- - Build failures: `dss job log <id> --errors-only` surfaces just error/traceback lines, and `--output PATH` saves the full log to a file. Logs are one long line with JVM noise; the `Error in Python process: At line <N>` marker maps `<N>` straight to your recipe payload's source line.
185
- - Schema changes aren't automatic for code recipes: after changing a python/SQL/prepare recipe's output columns run `dss dataset refresh-schema` (or rebuild) before downstream reads, and `dss dataset validate-build` to catch file-backed misconfig before launching a build. Exception: `dss recipe create --type sync` copies the input schema onto a schemaless dataset output at create time (see `syncOutputSchemaPropagated` in the result), so a fresh sync output builds populated without a manual refresh.
186
- - `dss dataset download` is capped (default 100k rows) and returns `{ path, rows, truncated, limit }`: check `truncated` and raise `--limit N` when you need more — treat it as a sample, not a guaranteed full export. Without `--output` the file lands in the current working directory (and a `dataset_download_default_location` warning names that path); pass `--output PATH` to control the destination. For very large tables, aggregate in SQL or read inside a recipe instead.
187
- - `dss dashboard create` and `dss dashboard update` validate every `INSIGHT` tile before mutation: `insightId` and `targetInsightId` must agree and resolve, and any `datasetSmartName` must resolve. Missing or stale references exit with `validation_failed` before POST/PUT. Cross-project `PROJECT.DATASET` references require the API key to read that dataset project; a `403` blocks the save rather than silently accepting an unverifiable dashboard.
188
- - `dss api-service list-packages SERVICE` first verifies the parent service through its settings. A `not_found` error means the service is missing (verify the service ID and project key); an empty array means the service exists but has no deployable packages. Do not reinterpret `not_found` as an empty package list or retry the lower-level packages route.
189
- - `dss notebook clear-jupyter-outputs NAME --dry-run` returns the full `current` and `next` notebook states. Applying it clears `outputs` and resets `execution_count` to `0` only on code cells, preserving markdown and raw cells unchanged.
190
- - `dss sql query ... --dataset PROJECT.NAME` first queries through the dataset. If DSS rejects that dataset connection as neither SQL nor HDFS, the CLI reads dataset metadata and retries with `params.connection` (using its schema or catalog as the database when available). A readable dataset with no usable connection exits with `validation_failed` and advises `--connection`; dataset metadata `404` and `403` errors propagate as `not_found` and `permission_denied`.
191
- - `dss sql query ... --start-retries N` opts the query-start POST into transient retries with exponential backoff. A lost response can cause DSS to execute the SQL more than once, so use it only for SQL whose repetition is safe; ordinary `--retries N` remains GET-only.
209
+ - On a non-UTF-8 console (Windows cp1252), don't print non-ASCII results; write them to a UTF-8 file or use `--output PATH`.
210
+ - Build failures: `dss job log <id> --errors-only` surfaces error/traceback lines; `--output PATH` saves the full log. Logs are one long line with JVM noise; the `Error in Python process: At line <N>` marker names your payload's source line.
211
+ - Schema changes aren't automatic for code recipes: after changing a python/SQL/prepare recipe's output columns run `dss dataset refresh-schema` (or rebuild) before downstream reads, and `dss dataset validate-build` to catch file-backed misconfig before a build. Exception: `dss recipe create --type sync` copies the input schema onto a schemaless output at create time (`syncOutputSchemaPropagated`), so a fresh sync output builds populated without a manual refresh.
212
+ - `dss dataset download` defaults to 100k rows and returns `{ path, rows, truncated, limit }`; raise `--limit` when truncated. Set `--output` or read the `dataset_download_default_location` path warning. CSV is spreadsheet-safe by default; `--raw-data` preserves formula prefixes. For large tables, aggregate in SQL or use a recipe.
213
+ - `dss dashboard create`/`update` validate every `INSIGHT` tile before mutation: `insightId` and `targetInsightId` must agree and resolve, and any `datasetSmartName` must resolve. Missing or stale references exit with `validation_failed` before POST/PUT. Cross-project `PROJECT.DATASET` references require the API key to read that dataset project; a `403` blocks the save rather than accept an unverifiable dashboard.
214
+ - `dss api-service list-packages SERVICE` first verifies the parent service through its settings. `not_found` means the service is missing (verify the service ID and project key); an empty array means it exists but has no deployable packages. Never reinterpret `not_found` as an empty list or retry the lower-level route.
215
+ - `dss notebook clear-jupyter-outputs NAME --dry-run` plans the official output DELETE; applying it uses that endpoint rather than rewriting notebook cells.
216
+ - `dss sql query ... --dataset PROJECT.NAME` first queries through the dataset. If DSS rejects that connection as neither SQL nor HDFS, the CLI reads dataset metadata and retries with `params.connection` (schema or catalog as the database when available). A readable dataset with no usable connection exits with `validation_failed` and advises `--connection`; dataset metadata `404` and `403` errors propagate as `not_found` and `permission_denied`.
217
+ - `dss sql query ... --start-retries N` opts the query-start POST into transient retries with exponential backoff. A lost response can make DSS execute the SQL more than once, so use it only for repetition-safe SQL; ordinary `--retries N` remains GET-only.
218
+ - `dss sql query` without `--preview` returns full rows on stdout for compatibility; `--preview N` bounds stdout for exploratory reads; `--output PATH` writes full rows to a file instead.
192
219
 
193
220
  ## Error envelope
194
221
 
195
- For dispatch/runtime failures, stdout carries one compact error object:
222
+ Dispatch/runtime failures carry one compact error object on stdout:
196
223
 
197
224
  ```json
198
225
  {
@@ -208,5 +235,5 @@ For dispatch/runtime failures, stdout carries one compact error object:
208
235
  ```
209
236
 
210
237
  `doctor`, `batch`, and `cleanup` report command-level failures as their direct result object on stdout with a nonzero exit code; key off the exit code before interpreting the result.
211
- Use `code`, `category`, `exitCode`, `retryable`, `status`, and `details` for recovery logic. Do not scrape message text when a structured field is available.
212
- Treat `details.body` as sanitized metadata only: it contains at most `requestId`/`request_id`/`errorId`/`elapsedMs` plus locally trusted target/timing fields, not the arbitrary DSS response body. `details.statusText` is canonical text derived from the numeric status, not a remote reason phrase. Base recovery on top-level `code`, `category`, `status`, `retryable`, `requestId`, `hint`, and `details.dssCategory`; never parse or expose assumed server-body fields.
238
+ Recover from `code`, `category`, `exitCode`, `retryable`, `status`, and `details`; never scrape message text when a structured field exists.
239
+ Treat `details.body` as sanitized metadata only: at most `requestId`/`request_id`/`errorId`/`elapsedMs` plus locally trusted target/timing fields, not the DSS response body. `details.statusText` is canonical text derived from the numeric status, not a remote reason phrase. Base recovery on top-level `code`, `category`, `status`, `retryable`, `requestId`, `hint`, and `details.dssCategory`; never parse assumed server-body fields.