dataiku-sdk 3.6.1 → 3.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (127) hide show
  1. package/README.md +14 -6
  2. package/bin/bun-version.js +14 -0
  3. package/bin/dss.js +11 -15
  4. package/dist/build-metadata.json +1 -1
  5. package/dist/packages/types/src/index.d.ts +74 -1
  6. package/dist/packages/types/src/index.js +53 -1
  7. package/dist/src/cli/coerce.d.ts +1 -0
  8. package/dist/src/cli/coerce.js +19 -15
  9. package/dist/src/cli/command-syntax.json +3521 -416
  10. package/dist/src/cli/commands/analysis.js +19 -1
  11. package/dist/src/cli/commands/bundle.js +22 -0
  12. package/dist/src/cli/commands/connection.js +26 -2
  13. package/dist/src/cli/commands/data-quality.js +33 -1
  14. package/dist/src/cli/commands/dataset.js +35 -4
  15. package/dist/src/cli/commands/discussion.js +15 -0
  16. package/dist/src/cli/commands/doctor.js +1 -1
  17. package/dist/src/cli/commands/flow.d.ts +5 -0
  18. package/dist/src/cli/commands/flow.js +57 -0
  19. package/dist/src/cli/commands/future.js +8 -0
  20. package/dist/src/cli/commands/index.js +6 -0
  21. package/dist/src/cli/commands/job.js +29 -6
  22. package/dist/src/cli/commands/metrics.js +21 -1
  23. package/dist/src/cli/commands/ml-task.d.ts +5 -0
  24. package/dist/src/cli/commands/ml-task.js +86 -16
  25. package/dist/src/cli/commands/mlflow.d.ts +11 -0
  26. package/dist/src/cli/commands/mlflow.js +121 -0
  27. package/dist/src/cli/commands/plugin.js +5 -11
  28. package/dist/src/cli/commands/project-deployer.js +70 -0
  29. package/dist/src/cli/commands/project-library.js +2 -5
  30. package/dist/src/cli/commands/project.js +8 -0
  31. package/dist/src/cli/commands/recipe.d.ts +9 -0
  32. package/dist/src/cli/commands/recipe.js +93 -7
  33. package/dist/src/cli/commands/saved-model.d.ts +1 -1
  34. package/dist/src/cli/commands/saved-model.js +9 -76
  35. package/dist/src/cli/commands/streaming-endpoint.js +30 -1
  36. package/dist/src/cli/commands/webapp.js +8 -0
  37. package/dist/src/cli/commands/wiki.js +22 -0
  38. package/dist/src/cli/commands/workspace.js +8 -0
  39. package/dist/src/cli/contract.d.ts +16 -15
  40. package/dist/src/cli/contract.js +139 -3404
  41. package/dist/src/cli/doctor.d.ts +13 -0
  42. package/dist/src/cli/doctor.js +41 -1
  43. package/dist/src/cli/flags.d.ts +7 -1
  44. package/dist/src/cli/flags.js +78 -6
  45. package/dist/src/cli/helpers/app-successor.js +1 -2
  46. package/dist/src/cli/helpers/documentation.d.ts +21 -0
  47. package/dist/src/cli/helpers/documentation.js +53 -0
  48. package/dist/src/cli/helpers/trained-model.d.ts +26 -0
  49. package/dist/src/cli/helpers/trained-model.js +144 -0
  50. package/dist/src/cli/list-output.d.ts +7 -0
  51. package/dist/src/cli/list-output.js +77 -0
  52. package/dist/src/cli/list-shapes.d.ts +21 -0
  53. package/dist/src/cli/list-shapes.js +111 -0
  54. package/dist/src/cli/output-schemas.d.ts +0 -14
  55. package/dist/src/cli/output-schemas.js +4 -7
  56. package/dist/src/cli/output.d.ts +0 -1
  57. package/dist/src/cli/output.js +27 -19
  58. package/dist/src/cli/plans.d.ts +14 -0
  59. package/dist/src/cli/plans.js +3684 -0
  60. package/dist/src/cli/usage.d.ts +7 -0
  61. package/dist/src/cli/usage.js +20 -1
  62. package/dist/src/cli/version.d.ts +2 -2
  63. package/dist/src/cli/version.js +2 -2
  64. package/dist/src/cli.js +127 -113
  65. package/dist/src/client.d.ts +52 -43
  66. package/dist/src/client.js +267 -96
  67. package/dist/src/errors.d.ts +11 -1
  68. package/dist/src/errors.js +31 -47
  69. package/dist/src/generated/action-output-schemas.json +345 -89
  70. package/dist/src/index.d.ts +5 -1
  71. package/dist/src/index.js +3 -0
  72. package/dist/src/resources/analyses.d.ts +8 -0
  73. package/dist/src/resources/analyses.js +13 -1
  74. package/dist/src/resources/bundles.d.ts +34 -0
  75. package/dist/src/resources/bundles.js +67 -0
  76. package/dist/src/resources/connections.d.ts +9 -0
  77. package/dist/src/resources/connections.js +15 -0
  78. package/dist/src/resources/data-quality.d.ts +20 -0
  79. package/dist/src/resources/data-quality.js +30 -0
  80. package/dist/src/resources/dataset-create.d.ts +19 -0
  81. package/dist/src/resources/dataset-create.js +66 -0
  82. package/dist/src/resources/datasets.d.ts +25 -15
  83. package/dist/src/resources/datasets.js +59 -111
  84. package/dist/src/resources/discussions.d.ts +5 -0
  85. package/dist/src/resources/discussions.js +7 -0
  86. package/dist/src/resources/documentation.d.ts +25 -0
  87. package/dist/src/resources/documentation.js +16 -0
  88. package/dist/src/resources/flow.d.ts +48 -0
  89. package/dist/src/resources/flow.js +65 -0
  90. package/dist/src/resources/futures.d.ts +23 -0
  91. package/dist/src/resources/futures.js +32 -0
  92. package/dist/src/resources/jobs.js +102 -2
  93. package/dist/src/resources/metrics.d.ts +10 -0
  94. package/dist/src/resources/metrics.js +10 -0
  95. package/dist/src/resources/ml-tasks.d.ts +65 -2
  96. package/dist/src/resources/ml-tasks.js +102 -28
  97. package/dist/src/resources/mlflow-extension.d.ts +73 -0
  98. package/dist/src/resources/mlflow-extension.js +105 -0
  99. package/dist/src/resources/projects.d.ts +6 -0
  100. package/dist/src/resources/projects.js +8 -0
  101. package/dist/src/resources/recipe-create.d.ts +65 -0
  102. package/dist/src/resources/recipe-create.js +338 -0
  103. package/dist/src/resources/recipes.d.ts +28 -29
  104. package/dist/src/resources/recipes.js +126 -245
  105. package/dist/src/resources/saved-models.d.ts +10 -6
  106. package/dist/src/resources/saved-models.js +16 -14
  107. package/dist/src/resources/scenarios.js +8 -14
  108. package/dist/src/resources/streaming-endpoints.d.ts +12 -0
  109. package/dist/src/resources/streaming-endpoints.js +15 -0
  110. package/dist/src/resources/trained-model.d.ts +60 -0
  111. package/dist/src/resources/trained-model.js +98 -0
  112. package/dist/src/resources/webapps.d.ts +8 -0
  113. package/dist/src/resources/webapps.js +10 -0
  114. package/dist/src/resources/wiki.d.ts +5 -0
  115. package/dist/src/resources/wiki.js +7 -0
  116. package/dist/src/resources/workspaces.d.ts +2 -0
  117. package/dist/src/resources/workspaces.js +4 -0
  118. package/dist/src/utils/response-file.js +20 -9
  119. package/dist/src/utils/secret-sanitize.d.ts +7 -0
  120. package/dist/src/utils/secret-sanitize.js +46 -0
  121. package/package.json +25 -26
  122. package/plugin.json +1 -1
  123. package/skills/dataiku-dss/SKILL.md +1 -1
  124. package/skills/dataiku-dss/references/authentication.md +3 -1
  125. package/skills/dataiku-dss/references/coding.md +1 -0
  126. package/skills/dataiku-dss/references/discovery.md +4 -6
  127. package/skills/dataiku-dss/references/troubleshooting.md +3 -2
@@ -0,0 +1,60 @@
1
+ import type { DataikuClient } from "../client.js";
2
+ import { type DocumentationTemplate } from "./documentation.js";
3
+ import { type DssTask } from "./futures.js";
4
+ /** Sampling and execution settings of a diagnostics computation (dataikuapi `computationParams`). */
5
+ export interface ModelDiagnosticsComputation {
6
+ /** Records sampled from the test set (DSS default 10000). */
7
+ sampleSize?: number;
8
+ randomState?: number;
9
+ nJobs?: number;
10
+ debugMode?: boolean;
11
+ }
12
+ /** Diagnostics a trained model computes on demand, then serves as results. */
13
+ export type ComputedModelDiagnostic = "subpopulation-analyses" | "partial-dependencies" | "timeseries-residuals";
14
+ /** Every diagnostics result a trained model serves. */
15
+ export type ModelDiagnostic = ComputedModelDiagnostic | "per-timeseries-metrics" | "per-timeseries-evaluation-forecasts";
16
+ /** Diagnostics that are started with POST and return a DSS future. */
17
+ export declare const COMPUTED_MODEL_DIAGNOSTICS: Record<string, true>;
18
+ /** Diagnostics results served by GET (computed ones plus per-series results of forecasting models). */
19
+ export declare const MODEL_DIAGNOSTICS: Record<string, true>;
20
+ /**
21
+ * Request body that starts a diagnostics computation. Subpopulation analyses
22
+ * and partial dependencies take the features to analyse; timeseries
23
+ * residuals take no body.
24
+ */
25
+ export declare function modelDiagnosticsBody(kind: ComputedModelDiagnostic, features: string[], computation?: ModelDiagnosticsComputation): Record<string, unknown> | undefined;
26
+ /** Query options for the scoring JAR export (GET .../scoring-jar). */
27
+ export interface ScoringJarOptions {
28
+ fullClassName?: string;
29
+ includeLibs?: boolean;
30
+ }
31
+ /**
32
+ * One trained model: a visual-ML lab model (`mlTasks.model(...)`) or a
33
+ * saved-model version (`savedModels.version(...)`). Both expose the same
34
+ * routes (user meta, scoring exports, diagnostics, documentation) under
35
+ * different bases. Generated documents are downloaded from the owning
36
+ * resource (`mlTasks.downloadModelDocumentation`, `savedModels.downloadDocumentation`).
37
+ */
38
+ export declare class TrainedModel {
39
+ private readonly client;
40
+ /** Route of the trained model, e.g. `.../models/lab/{a}/{t}/models/{id}`. */
41
+ private readonly modelPath;
42
+ constructor(client: DataikuClient,
43
+ /** Route of the trained model, e.g. `.../models/lab/{a}/{t}/models/{id}`. */
44
+ modelPath: string);
45
+ /** Replace the user metadata: only the `userMeta` field of the model details, edited. */
46
+ setUserMeta(userMeta: Record<string, unknown>): Promise<void>;
47
+ /** GET the Java scoring JAR (license-gated, optimized-scoring models only) as a binary stream. */
48
+ downloadScoringJar(options?: ScoringJarOptions): Promise<Response>;
49
+ /** GET the PMML export (license-gated, PMML-compatible models only) as a stream. */
50
+ downloadScoringPmml(): Promise<Response>;
51
+ /**
52
+ * Start computing subpopulation analyses or partial dependencies over
53
+ * `features`, or timeseries residuals. Returns the DSS future.
54
+ */
55
+ compute(kind: ComputedModelDiagnostic, features?: string[], computation?: ModelDiagnosticsComputation): Promise<DssTask>;
56
+ /** Results computed so far (subpopulation analyses, partial dependencies, residuals, per-series metrics or forecasts). */
57
+ get(kind: ModelDiagnostic): Promise<unknown>;
58
+ /** Start generating the model documentation (docx); the finished result carries `exportId`. */
59
+ generateDocumentation(template?: DocumentationTemplate): Promise<DssTask>;
60
+ }
@@ -0,0 +1,98 @@
1
+ import { ClientValidationError, } from "../errors.js";
2
+ import { requireObject, } from "./base.js";
3
+ import { startDocumentation, } from "./documentation.js";
4
+ import { dssTask, } from "./futures.js";
5
+ /** Diagnostics that are started with POST and return a DSS future. */
6
+ export const COMPUTED_MODEL_DIAGNOSTICS = {
7
+ "subpopulation-analyses": true,
8
+ "partial-dependencies": true,
9
+ "timeseries-residuals": true,
10
+ };
11
+ /** Diagnostics results served by GET (computed ones plus per-series results of forecasting models). */
12
+ export const MODEL_DIAGNOSTICS = {
13
+ ...COMPUTED_MODEL_DIAGNOSTICS,
14
+ "per-timeseries-metrics": true,
15
+ "per-timeseries-evaluation-forecasts": true,
16
+ };
17
+ /**
18
+ * Request body that starts a diagnostics computation. Subpopulation analyses
19
+ * and partial dependencies take the features to analyse; timeseries
20
+ * residuals take no body.
21
+ */
22
+ export function modelDiagnosticsBody(kind, features, computation = {}) {
23
+ if (kind === "timeseries-residuals")
24
+ return undefined;
25
+ if (features.length === 0) {
26
+ throw new ClientValidationError(`${kind} needs at least one feature.`, "missing_required_arg");
27
+ }
28
+ return {
29
+ features,
30
+ computationParams: {
31
+ ...(computation.sampleSize !== undefined ? { sample_size: computation.sampleSize, } : {}),
32
+ ...(computation.randomState !== undefined ? { random_state: computation.randomState, } : {}),
33
+ ...(computation.nJobs !== undefined ? { n_jobs: computation.nJobs, } : {}),
34
+ ...(computation.debugMode !== undefined ? { debug_mode: computation.debugMode, } : {}),
35
+ },
36
+ };
37
+ }
38
+ /**
39
+ * One trained model: a visual-ML lab model (`mlTasks.model(...)`) or a
40
+ * saved-model version (`savedModels.version(...)`). Both expose the same
41
+ * routes (user meta, scoring exports, diagnostics, documentation) under
42
+ * different bases. Generated documents are downloaded from the owning
43
+ * resource (`mlTasks.downloadModelDocumentation`, `savedModels.downloadDocumentation`).
44
+ */
45
+ export class TrainedModel {
46
+ client;
47
+ modelPath;
48
+ constructor(client,
49
+ /** Route of the trained model, e.g. `.../models/lab/{a}/{t}/models/{id}`. */
50
+ modelPath) {
51
+ this.client = client;
52
+ this.modelPath = modelPath;
53
+ }
54
+ /** Replace the user metadata: only the `userMeta` field of the model details, edited. */
55
+ async setUserMeta(userMeta) {
56
+ await this.client.putVoid(`${this.modelPath}/user-meta`, requireObject(userMeta, "userMeta"));
57
+ }
58
+ /** GET the Java scoring JAR (license-gated, optimized-scoring models only) as a binary stream. */
59
+ async downloadScoringJar(options = {}) {
60
+ const query = new URLSearchParams();
61
+ if (options.fullClassName !== undefined)
62
+ query.set("fullClassName", options.fullClassName);
63
+ if (options.includeLibs !== undefined)
64
+ query.set("includeLibs", String(options.includeLibs));
65
+ return this.client.stream(`${this.modelPath}/scoring-jar${query.size > 0 ? `?${query}` : ""}`);
66
+ }
67
+ /** GET the PMML export (license-gated, PMML-compatible models only) as a stream. */
68
+ async downloadScoringPmml() {
69
+ return this.client.stream(`${this.modelPath}/scoring-pmml`);
70
+ }
71
+ /**
72
+ * Start computing subpopulation analyses or partial dependencies over
73
+ * `features`, or timeseries residuals. Returns the DSS future.
74
+ */
75
+ async compute(kind, features = [], computation = {}) {
76
+ if (COMPUTED_MODEL_DIAGNOSTICS[kind] !== true) {
77
+ throw new ClientValidationError(`Computed diagnostic must be one of ${Object.keys(COMPUTED_MODEL_DIAGNOSTICS).join(", ")}.`, "invalid_enum");
78
+ }
79
+ const raw = await this.client.post(`${this.modelPath}/${kind}`, modelDiagnosticsBody(kind, features, computation));
80
+ return dssTask(raw, `Computing ${kind}`);
81
+ }
82
+ /** Results computed so far (subpopulation analyses, partial dependencies, residuals, per-series metrics or forecasts). */
83
+ async get(kind) {
84
+ if (MODEL_DIAGNOSTICS[kind] !== true) {
85
+ throw new ClientValidationError(`Diagnostic must be one of ${Object.keys(MODEL_DIAGNOSTICS).join(", ")}.`, "invalid_enum");
86
+ }
87
+ return this.client.get(`${this.modelPath}/${kind}`);
88
+ }
89
+ /** Start generating the model documentation (docx); the finished result carries `exportId`. */
90
+ async generateDocumentation(template = { kind: "default", }) {
91
+ const base = `${this.modelPath}/generate-documentation-from`;
92
+ return startDocumentation(this.client, {
93
+ default: `${base}-default-template`,
94
+ file: `${base}-custom-template`,
95
+ folder: `${base}-template-in-folder`,
96
+ }, template, "Model documentation generation");
97
+ }
98
+ }
@@ -68,4 +68,12 @@ export declare class WebappsResource extends BaseResource {
68
68
  restartBackendAndWait(webappId: string, projectKey?: string, options?: WebappWaitOptions): Promise<FutureWaitResult>;
69
69
  /** Get a webapp backend's state. */
70
70
  getBackendState(webappId: string, projectKey?: string): Promise<Record<string, unknown>>;
71
+ /**
72
+ * Trust a webapp's code (POST /actions/trust): for the caller, or for every
73
+ * user with `trustForEverybody` (admin). Returns `{canAccess, trustedCodeStatus}`.
74
+ */
75
+ trust(webappId: string, opts?: {
76
+ projectKey?: string;
77
+ trustForEverybody?: boolean;
78
+ }): Promise<Record<string, unknown>>;
71
79
  }
@@ -95,4 +95,14 @@ export class WebappsResource extends BaseResource {
95
95
  const id = encodeURIComponent(webappId);
96
96
  return this.client.get(`/public/api/projects/${this.enc(projectKey)}/webapps/${id}/backend/state`);
97
97
  }
98
+ /**
99
+ * Trust a webapp's code (POST /actions/trust): for the caller, or for every
100
+ * user with `trustForEverybody` (admin). Returns `{canAccess, trustedCodeStatus}`.
101
+ */
102
+ async trust(webappId, opts = {}) {
103
+ const query = opts.trustForEverybody === undefined
104
+ ? ""
105
+ : `?trustForEverybody=${opts.trustForEverybody}`;
106
+ return this.client.post(`/public/api/projects/${this.enc(opts.projectKey)}/webapps/${encodeURIComponent(webappId)}/actions/trust${query}`);
107
+ }
98
108
  }
@@ -14,6 +14,11 @@ export interface WikiArticleUpdateOptions {
14
14
  }
15
15
  export declare class WikiResource extends BaseResource {
16
16
  settings(projectKey?: string): Promise<WikiSettings>;
17
+ /**
18
+ * Replace the wiki properties (home article and taxonomy, PUT /wiki/) with
19
+ * an object obtained from `settings`, edited.
20
+ */
21
+ updateSettings(settings: Record<string, unknown>, projectKey?: string): Promise<void>;
17
22
  list(projectKey?: string): Promise<WikiArticleData[]>;
18
23
  get(articleIdOrName: string, projectKey?: string): Promise<WikiArticleData>;
19
24
  create(opts: WikiArticleCreateOptions): Promise<WikiArticleData>;
@@ -33,6 +33,13 @@ export class WikiResource extends BaseResource {
33
33
  const raw = await this.client.get(`/public/api/projects/${this.enc(projectKey)}/wiki/`);
34
34
  return this.client.safeParse(WikiSettingsSchema, raw, "wiki.settings");
35
35
  }
36
+ /**
37
+ * Replace the wiki properties (home article and taxonomy, PUT /wiki/) with
38
+ * an object obtained from `settings`, edited.
39
+ */
40
+ async updateSettings(settings, projectKey) {
41
+ await this.client.putVoid(`/public/api/projects/${this.enc(projectKey)}/wiki/`, settings);
42
+ }
36
43
  async list(projectKey) {
37
44
  const settings = await this.settings(projectKey);
38
45
  const ids = taxonomyIds(settings.taxonomy);
@@ -56,4 +56,6 @@ export declare class WorkspacesResource extends BaseResource {
56
56
  listObjects(workspaceKey: string): Promise<WorkspaceObject[]>;
57
57
  /** Add an object to a workspace. */
58
58
  addObject(workspaceKey: string, object: WorkspaceObjectInput): Promise<WorkspaceObject>;
59
+ /** Remove an object from a workspace by its workspace object id (from `listObjects`). */
60
+ removeObject(workspaceKey: string, workspaceObjectId: string): Promise<void>;
59
61
  }
@@ -28,4 +28,8 @@ export class WorkspacesResource extends BaseResource {
28
28
  async addObject(workspaceKey, object) {
29
29
  return this.client.post(`/public/api/workspaces/${encodeURIComponent(workspaceKey)}/objects`, object);
30
30
  }
31
+ /** Remove an object from a workspace by its workspace object id (from `listObjects`). */
32
+ async removeObject(workspaceKey, workspaceObjectId) {
33
+ await this.client.del(`/public/api/workspaces/${encodeURIComponent(workspaceKey)}/objects/${encodeURIComponent(workspaceObjectId)}`);
34
+ }
31
35
  }
@@ -1,9 +1,6 @@
1
- import { randomUUID, } from "node:crypto";
2
- import { createWriteStream, } from "node:fs";
1
+ import { closeSync, openSync, } from "node:fs";
3
2
  import { chmod, rename, rm, stat, } from "node:fs/promises";
4
3
  import { basename, dirname, join, } from "node:path";
5
- import { Readable, } from "node:stream";
6
- import { pipeline, } from "node:stream/promises";
7
4
  /** Stream with backpressure; publish only a complete download, preserving existing output on failure. */
8
5
  export async function writeResponseToFile(path, response, permissions = "private") {
9
6
  let mode;
@@ -17,16 +14,30 @@ export async function writeResponseToFile(path, response, permissions = "private
17
14
  mode = 0o666 & ~process.umask();
18
15
  }
19
16
  }
20
- const input = Readable.from(response.body, { objectMode: false, });
21
- const temporaryPath = join(dirname(path), `.${basename(path)}.tmp-${randomUUID()}`);
22
- const output = createWriteStream(temporaryPath, { flags: "wx", mode: 0o600, });
17
+ const temporaryPath = join(dirname(path), `.${basename(path)}.tmp-${crypto.randomUUID()}`);
18
+ // Exclusive, private temp file; Bun.write streams the body into it natively.
19
+ let fd;
23
20
  try {
24
- await pipeline(input, output);
21
+ fd = openSync(temporaryPath, "wx", 0o600);
22
+ }
23
+ catch (error) {
24
+ // Nothing will read the body; release the connection before surfacing the error.
25
+ await response.body?.cancel().catch(() => { });
26
+ throw error;
27
+ }
28
+ try {
29
+ let bytes;
30
+ try {
31
+ bytes = await Bun.write(Bun.file(fd), response);
32
+ }
33
+ finally {
34
+ closeSync(fd);
35
+ }
25
36
  // Keep partial contents private; restore publication permissions only after completion.
26
37
  if (mode !== undefined)
27
38
  await chmod(temporaryPath, mode);
28
39
  await rename(temporaryPath, path);
29
- return output.bytesWritten;
40
+ return bytes;
30
41
  }
31
42
  finally {
32
43
  await rm(temporaryPath, { force: true, });
@@ -46,6 +46,13 @@ export declare function redactUrlUserinfo(text: string): string;
46
46
  * recurse, arrays recurse. Non-string scalars pass through unchanged.
47
47
  */
48
48
  export declare function sanitizeSecrets<T>(value: T, options: SecretSanitizeOptions): T;
49
+ /**
50
+ * Every string value under a credential-like key (any depth), plus bare
51
+ * `key` fields such as `authRealm.key`. Collected from a command's own JSON
52
+ * input so that DSS error text echoing the payload can be scrubbed; error
53
+ * redaction only, so the broad key match errs toward hiding.
54
+ */
55
+ export declare function credentialValues(value: unknown): string[];
49
56
  /**
50
57
  * Scrub secret material from an error's message, stack, and string body.
51
58
  * `DataikuError.message` is built at construction, so all three carriers need
@@ -77,6 +77,52 @@ export function sanitizeSecrets(value, options) {
77
77
  }
78
78
  return value;
79
79
  }
80
+ /** Credential-like key fragments (normalized); values under them are secrets. */
81
+ const CREDENTIAL_KEY_FRAGMENTS = [
82
+ "password",
83
+ "passwd",
84
+ "passphrase",
85
+ "secret",
86
+ "token",
87
+ "apikey",
88
+ "accesskey",
89
+ "privatekey",
90
+ "credential",
91
+ "authorization",
92
+ ];
93
+ /** Keys naming how credentials work (`credentialsMode`, `passwordType`), not secrets. */
94
+ const CREDENTIAL_SETTING_SUFFIXES = ["mode", "type", "policy",];
95
+ /**
96
+ * Every string value under a credential-like key (any depth), plus bare
97
+ * `key` fields such as `authRealm.key`. Collected from a command's own JSON
98
+ * input so that DSS error text echoing the payload can be scrubbed; error
99
+ * redaction only, so the broad key match errs toward hiding.
100
+ */
101
+ export function credentialValues(value) {
102
+ const found = [];
103
+ const visit = (node, sensitive) => {
104
+ if (typeof node === "string") {
105
+ if (sensitive && node.length > 0)
106
+ found.push(node);
107
+ return;
108
+ }
109
+ if (Array.isArray(node)) {
110
+ for (const item of node)
111
+ visit(item, sensitive);
112
+ return;
113
+ }
114
+ if (node === null || typeof node !== "object")
115
+ return;
116
+ for (const [key, item,] of Object.entries(node)) {
117
+ const normalized = normalizeSecretKey(key);
118
+ const setting = CREDENTIAL_SETTING_SUFFIXES.some((suffix) => normalized.endsWith(suffix));
119
+ visit(item, sensitive || (!setting && (normalized === "key"
120
+ || CREDENTIAL_KEY_FRAGMENTS.some((fragment) => normalized.includes(fragment)))));
121
+ }
122
+ };
123
+ visit(value, false);
124
+ return found;
125
+ }
80
126
  /**
81
127
  * Scrub secret material from an error's message, stack, and string body.
82
128
  * `DataikuError.message` is built at construction, so all three carriers need
package/package.json CHANGED
@@ -1,11 +1,13 @@
1
1
  {
2
2
  "name": "dataiku-sdk",
3
- "version": "3.6.1",
3
+ "version": "3.7.0",
4
4
  "description": "Dataiku DSS SDK and CLI for programmatic access to DSS REST APIs",
5
+ "repository": {
6
+ "type": "git",
7
+ "url": "https://github.com/clssck/dataiku-sdk.git"
8
+ },
9
+ "license": "SEE LICENSE IN LICENSE",
5
10
  "type": "module",
6
- "workspaces": [
7
- "packages/*"
8
- ],
9
11
  "main": "dist/src/index.js",
10
12
  "types": "dist/src/index.d.ts",
11
13
  "bin": {
@@ -15,11 +17,12 @@
15
17
  "dist/",
16
18
  "plugin.json",
17
19
  "skills/",
18
- "bin/dss.js"
20
+ "bin/dss.js",
21
+ "bin/bun-version.js"
22
+ ],
23
+ "workspaces": [
24
+ "packages/*"
19
25
  ],
20
- "engines": {
21
- "bun": ">=1.4.0"
22
- },
23
26
  "scripts": {
24
27
  "build": "bun scripts/generate-action-output-schemas.mjs && tsc && bun scripts/generate-build-metadata.mjs",
25
28
  "prepack": "bun scripts/sync-plugin-version.mjs --check && bun scripts/generate-action-output-schemas.mjs --check && tsc && bun scripts/generate-build-metadata.mjs",
@@ -40,30 +43,26 @@
40
43
  "test:integration:rigorous": "cross-env RUN_DATAIKU_INTEGRATION=1 bun test tests/integration-rigorous.test.ts",
41
44
  "test:integration:rigorous:mutating": "cross-env RUN_DATAIKU_INTEGRATION=1 RUN_DATAIKU_INTEGRATION_MUTATING=1 bun test tests/integration-rigorous.test.ts"
42
45
  },
46
+ "dependencies": {
47
+ "@sinclair/typebox": "^0.34.52",
48
+ "node-stream-zip": "^1.16.0"
49
+ },
43
50
  "devDependencies": {
44
- "@oxlint/plugins": "1.81.0",
51
+ "@oxlint/plugins": "1.86.0",
45
52
  "@types/bun": "^1.4.2",
46
53
  "cross-env": "^10.1.0",
47
- "dprint": "^0.57.0",
54
+ "dprint": "^0.60.1",
48
55
  "js-tiktoken": "1.0.21",
49
- "oxlint": "1.81.0",
50
- "typescript": "^7.0.2",
51
- "typescript-language-server": "6.0.0",
52
- "typescript-language-service": "npm:typescript@5.9.3"
53
- },
54
- "dependencies": {
55
- "@sinclair/typebox": "^0.34.52",
56
- "node-stream-zip": "^1.16.0"
56
+ "oxlint": "1.86.0",
57
+ "typescript": "^7.0.2"
57
58
  },
58
- "license": "SEE LICENSE IN LICENSE",
59
- "repository": {
60
- "type": "git",
61
- "url": "https://github.com/clssck/dataiku-sdk.git"
59
+ "bundleDependencies": [
60
+ "@sinclair/typebox"
61
+ ],
62
+ "engines": {
63
+ "bun": ">=1.4.2"
62
64
  },
63
65
  "publishConfig": {
64
66
  "access": "public"
65
- },
66
- "bundleDependencies": [
67
- "@sinclair/typebox"
68
- ]
67
+ }
69
68
  }
package/plugin.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json",
3
3
  "name": "dataiku-sdk",
4
- "version": "3.6.1",
4
+ "version": "3.7.0",
5
5
  "description": "Dataiku DSS SDK and agent-oriented CLI for programmatic access to DSS REST APIs.",
6
6
  "homepage": "https://github.com/clssck/dataiku-sdk",
7
7
  "repository": "https://github.com/clssck/dataiku-sdk",
@@ -6,7 +6,7 @@ description: >-
6
6
  Discover syntax: dss commands run --fields RESOURCE.ACTION.
7
7
  license: LicenseRef-Dataiku-SDK-Limited-Use-1.0
8
8
  compatibility: >-
9
- Requires Bun >= 1.4.0, the CLI, DSS network access, and credentials.
9
+ Requires Bun >= 1.4.2, the CLI, DSS network access, and credentials.
10
10
  ---
11
11
 
12
12
  # Dataiku DSS
@@ -36,4 +36,6 @@ dss auth login
36
36
  ```
37
37
  `auth login` lists projects before saving: the key needs project-list permission. Returns `{"saved":true,"path":"..."}`. Storage: `credentials.json` in `DSS_CONFIG_DIR`, else `XDG_CONFIG_HOME/dataiku`, Windows `APPDATA/dataiku`, or `~/.config/dataiku`.
38
38
 
39
- TLS: `--insecure` disables verification; `--ca-cert PATH` adds a PEM CA bundle. Environment equivalents: `NODE_TLS_REJECT_UNAUTHORIZED`, `NODE_EXTRA_CA_CERTS`.
39
+ TLS: `--insecure` disables verification; `--ca-cert PATH` adds a PEM CA bundle. Environment equivalents: `NODE_TLS_REJECT_UNAUTHORIZED`, `NODE_EXTRA_CA_CERTS`. A subprocess sees only its own env and cwd `.env`; `dss auth login --ca-cert PATH` saves the CA for all processes.
40
+
41
+ Connection `403` the UI allows: check the key's identity (`dss doctor`: `authSource`, login, groups). `connection get` needs admin; `connection info NAME` covers usable connections.
@@ -3,6 +3,7 @@
3
3
  - Git: UAT=`create-branch --duplicate-project`; pull=rebase; checkout=switch. Remotes use DSS Git rules and SSH/server credentials. `--password-env`; `future-wait`.
4
4
  - Internal `lib/`: `project-library`. External Git: `project-git list-libraries`. Safe edit: `get-bytes` → `diff` → `put --expect-sha256 HASH`; create with `--if-not-exists`. Paths reject traversal.
5
5
  - Python: `code run --file`. Inspect/diff/backup recipes before `recipe run --dry-run`.
6
+ - Grouping/prepare: build with `recipe create` flags (`--group-by`, `--aggregate`; `--rename`, `--formula`, `--filter`, ...). Other visual payloads (join): no published schema; template = `recipe get-payload` after create. Join `MANUAL` selection uses top-level `selectedColumns: [{name, table, alias?}]` (`table` = input index). DSS-engine computed columns run as H2 SQL: no GREL `+` concat, use `SQL` mode `||`. Grouping `first` needs an ordering column.
6
7
  - Notebook saves return hashes; guard with `--expect-hash`. Clear outputs via DSS DELETE; `unload-jupyter --all` composes session deletes.
7
8
  - Inspect code-env definitions/logs before updates; guard webapp/API settings with `--expect-hash`. `exact:false`: live state not guessed; use command `--dry-run`.
8
9
  - Plugin authoring: `dss plugin` create-dev, move-to-dev, contents-put, push/pull. No notebook execute/checkpoints or webapp/API delete; use the DSS UI.
@@ -8,11 +8,13 @@
8
8
  - Exits: 0 success, 1 usage/configuration, 2 DSS/internal/permission-or-environment, 3 transient/retryable DSS error, 4 failed long-running result/assertion.
9
9
  - Recipe payload stdout: JSON string. With `--output PATH`: exact bytes to file, JSON string equal to `PATH` on stdout.
10
10
  - General `--fields a,b,c`: object projection, element-wise for object arrays. Dotted paths traverse nested objects; missing fields become `null`; strings/scalars pass through.
11
+ - `*.list`: compact items (ids, kind, next-step fields such as recipe `inputs`/`outputs`); `--full`/`--fields` use DSS objects. `--contains TEXT` filters ids; `--limit N` caps (`list_truncated` warning).
11
12
 
12
13
  ## Discovery
13
14
 
14
15
  ```text
15
16
  dss commands run
17
+ dss commands run --contains "propagate schema"
16
18
  dss agent contract --fields commands.actions.dataset
17
19
  dss commands run --fields dataset.create
18
20
  dss commands run --fields dataset.preview.usage,dataset.preview.description,dataset.preview.flags,dataset.preview.examples
@@ -21,10 +23,6 @@ dss agent contract --fields protocol,agentContractVersion,cli,stdio,planning,com
21
23
  dss agent contract --fields commands.actions
22
24
  ```
23
25
 
24
- `commands run` defaults to a resource→action-name summary (~1.6k tokens), never full entries. For a known resource, project `commands.actions.RESOURCE` from `agent contract` (~64 tokens for dataset). Bootstrap once with the six-field projection above (~250 tokens); fetch `schemas` or `commands` only as needed.
26
+ `commands run`: resource→action summary (~1.6k tokens). Unknown name: `--contains TEXT` gives the top 5 actions with usage. Known resource: `agent contract --fields commands.actions.RESOURCE`. Bootstrap once with the six-field projection (~250 tokens). Global flags (`--fields`, `--verbose`, connection/TLS) appear once in `commands.globalFlags`, not per entry.
25
27
 
26
- Look up syntax before invoking. `--fields RESOURCE` selects every full entry (potentially large); `RESOURCE.ACTION` one complete entry; append `.FIELD` for nested metadata. Reads: prefer `usage,description,flags,examples`. Writes: fetch the full entry directly once, not compact-then-full. It includes flags, positionals, side effects, auth, output, idempotency, dry-run, examples, payload schemas, unsafe outputs, cleanup, and exits.
27
-
28
- Comma-separate selectors to batch lookups. Keys echo selectors: `--fields dataset.create` → `{"dataset.create":{...}}`. Empty `--fields` fails usage validation, never dumps everything. Unknown resources/actions exit 1 with compact JSON errors listing valid options.
29
-
30
- Full registry: `commands run --output PATH` only. With `--fields`, export that subset instead. Files contain compact JSON; stdout is `{"path":"PATH"}`, never registry content.
28
+ Look up syntax before invoking: `RESOURCE.ACTION` is one entry, `.FIELD` nests, commas batch, keys echo selectors. Reads: `usage,description,flags,examples`. Writes: the full entry once (flags, side effects, idempotency, dry-run, payload schemas, cleanup, exits). Unknown flags, resources, and actions exit 1 with the usage line or valid options. Full registry: `commands run --output PATH` only; stdout is `{"path":"PATH"}`.
@@ -4,7 +4,8 @@
4
4
 
5
5
  - Code/SQL: use `--file`/`--sql-file`; shells, especially PowerShell, mangle quotes, `$`, and multiline text. Non-UTF-8 consoles (e.g. Windows cp1252): write non-ASCII results to UTF-8 files or `--output PATH`, not the console.
6
6
  - Build errors: `dss job log <id> --errors-only` selects errors/tracebacks. For full logs, use `--output PATH` without `--errors-only`. Logs may arrive as one JVM-noisy line; `Error in Python process: At line <N>` identifies the payload source line.
7
- - After python/SQL/prepare output-column changes, supply updated columns with `dss dataset refresh-schema NAME --data-file columns.json` or rebuild before downstream reads. Run `dss dataset validate-build` before builds to catch file-backed misconfiguration. Exception: `dss recipe create --type sync` copies input schema to schemaless output at creation (`syncOutputSchemaPropagated`); fresh sync outputs build populated without manual refresh.
7
+ - Schemas: code recipes set output schemas at run time. Visual/SQL query recipes: `recipe create` fills created or empty outputs (`outputSchemaUpdated`); `recipe set-payload` updates empty or DSS-derived ones, warns `recipe_output_schema_outdated` for hand-edited ones. Apply with `dss recipe update-schema NAME` (one recipe), `dss flow propagate-schema DATASET --wait` (every downstream recipe), or build `--auto-update-schema`. `built_dataset_has_no_columns`: the DONE build wrote nothing. Manual columns: `dss dataset refresh-schema NAME --data-file columns.json`. `dss dataset validate-build` catches file-backed misconfiguration.
8
+ - `Inline` (editable) datasets: the public API cannot write rows, so `dataset create` rejects them; use `UploadedFiles` + `dataset upload-file`, or a python recipe output. `code run` is a scenario step; some DSS versions refuse dataset writes there.
8
9
  - `dss dataset download`: default 100k rows; result `{ path, rows, truncated, limit }`. If truncated, raise `--limit`. Specify `--output` or read the `dataset_download_default_location` path warning. CSV is spreadsheet-safe; `--raw-data` retains formula prefixes. For large tables, aggregate in SQL or use a recipe.
9
10
  - `dss dashboard create`/`update`: before POST/PUT, every `INSIGHT` tile needs agreeing/resolvable `insightId` and `targetInsightId`; any `datasetSmartName` must resolve. Missing/stale references: `validation_failed`. Cross-project `PROJECT.DATASET` needs dataset-project read permission; `403` blocks saving an unverifiable dashboard.
10
11
  - `dss api-service list-packages SERVICE` checks parent settings first. `not_found`: missing service; verify service ID/project key. Empty array: existing service, no deployable packages. Never convert `not_found` to an empty list or retry the lower-level route.
@@ -26,4 +27,4 @@ Dispatch/runtime failure: one compact stdout error:
26
27
 
27
28
  `doctor`/`batch`/`cleanup` command failures return direct stdout result objects; check nonzero exit before interpreting. Recover from structured `code`, `category`, `exitCode`, `retryable`, `status`, `details`, never message scraping.
28
29
 
29
- `details.body` is sanitized metadata, **not the DSS response body**: at most `requestId`/`request_id`/`errorId`/`elapsedMs` plus locally trusted target/timing. `details.statusText` derives from numeric status, not remote reason phrases. Use top-level `code`, `category`, `status`, `retryable`, `requestId`, `hint`, and `details.dssCategory`; never assume server-body fields.
30
+ DSS failures: `error` is `STATUS Text: DSS message` (credentials redacted, bounded); `hint` names the next check (resource, project). `details` appears only with content: `elapsedMs`/`target`, `dssErrorType`, `bodyTruncated`, and `retry` when a retry or timeout happened. Refused connections, unknown hosts, bad URLs, and untrusted certificates fail at once as `validation_failed` (exit 2); `transient` (exit 3) means automatic retries already ran.