dataiku-sdk 3.3.0 → 3.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -1
- package/dist/build-metadata.json +1 -1
- package/dist/src/cli/commands/code.js +1 -1
- package/dist/src/cli/commands/dataset.js +1 -1
- package/dist/src/cli/commands/job.js +4 -2
- package/dist/src/cli/commands/macro.js +2 -2
- package/dist/src/cli/commands/project-git.js +1 -1
- package/dist/src/cli/commands/scenario.js +1 -1
- package/dist/src/cli/contract.js +23 -208
- package/dist/src/cli/output-schemas.d.ts +24 -0
- package/dist/src/cli/output-schemas.js +233 -0
- package/dist/src/client.d.ts +10 -1
- package/dist/src/client.js +20 -5
- package/dist/src/errors.js +6 -4
- package/dist/src/generated/action-output-schemas.json +1735 -0
- package/dist/src/resources/datasets.js +37 -4
- package/dist/src/resources/flow-zones.d.ts +6 -2
- package/dist/src/resources/flow-zones.js +6 -3
- package/dist/src/resources/futures.d.ts +2 -0
- package/dist/src/resources/futures.js +38 -24
- package/dist/src/resources/jobs.js +37 -25
- package/dist/src/resources/ml-tasks.js +28 -19
- package/dist/src/resources/plugins.d.ts +0 -2
- package/dist/src/resources/plugins.js +0 -3
- package/dist/src/resources/project-git.d.ts +6 -12
- package/dist/src/resources/project-git.js +23 -48
- package/dist/src/resources/projects.d.ts +4 -2
- package/dist/src/resources/projects.js +33 -33
- package/dist/src/utils/git-reference.d.ts +12 -0
- package/dist/src/utils/git-reference.js +38 -0
- package/package.json +5 -5
- package/plugin.json +1 -1
- package/skills/dataiku-dss/SKILL.md +4 -2
- package/skills/dataiku-dss/references/app-releases.md +2 -3
- package/skills/dataiku-dss/references/authentication.md +6 -6
- package/skills/dataiku-dss/references/mutations.md +1 -1
- package/skills/dataiku-dss/references/troubleshooting.md +2 -2
package/README.md
CHANGED
|
@@ -267,7 +267,7 @@ DSS_TOKENIZER_MODULE=/absolute/path/to/oh-my-pi/packages/natives/native/index.js
|
|
|
267
267
|
|
|
268
268
|
When configured, the native module also accelerates ordinary OpenAI token measurements after parity checks against the pinned JavaScript model. The JS fingerprint and independent cross-model reference checks remain authoritative; reserved special-token validation is preserved. Without this variable, measurements use JavaScript and only the native cross-model case is skipped. A configured missing/incompatible module fails instead of silently falling back. The cross-model gate permits 10% growth over recorded text baselines. Claude counts use ctok reconstructions, not an official tokenizer; counts exclude provider framing. No native dependency is added to the CLI.
|
|
269
269
|
|
|
270
|
-
For a known resource, `dss agent contract --fields commands.actions.dataset` lists action names in **64 tokens**, versus **1,575** for the global index. For known writes, fetch the full action entry directly once: `dataset.create` costs **1,059 tokens**, versus **1,415** for compact discovery followed by the required full entry. Use compact `usage,description,flags,examples` projections for reads when those fields suffice. This is selective disclosure, not lossless compression. Scoped discovery
|
|
270
|
+
For a known resource, `dss agent contract --fields commands.actions.dataset` lists action names in **64 tokens**, versus **1,575** for the global index. For known writes, fetch the full action entry directly once: `dataset.create` costs **1,059 tokens**, versus **1,415** for compact discovery followed by the required full entry. Use compact `usage,description,flags,examples` projections for reads when those fields suffice. This is selective disclosure, not lossless compression. Scoped discovery constructs only requested action entries and materializes schemas only when requested; action-name indexes avoid payload schemas, while full registry exports and recovery hints retain their existing content. Metadata-only paths defer SDK runtime/schema imports.
|
|
271
271
|
|
|
272
272
|
Boolean flags accept bare flags or explicit `=true`/`=false` values, including aliases. Invalid boolean values fail before dispatch. Direct, batch, and local commands share execution-mode resolution; `--plan --dry-run` returns a plan with `plannedAndDryRun:true` when both modes are supported. Unreadable JSON/text payload files produce `validation_failed` usage errors (exit 1) with the flag, path, and filesystem cause.
|
|
273
273
|
|
|
@@ -275,6 +275,8 @@ For raw streaming downloads, `--request-timeout MS` bounds each pending body rea
|
|
|
275
275
|
|
|
276
276
|
The SDK configuration `maxResponseBodyBytes` (default **50 MiB**) also bounds retained non-2xx response bodies. Oversized errors preserve HTTP status, header request IDs, and a UTF-8-safe prefix, with `DataikuError.bodyTruncated: true`; CLI errors expose `details.bodyTruncated: true`. Retry eligibility is unchanged, and error-body reads share the transport attempt deadline.
|
|
277
277
|
|
|
278
|
+
Eligible transient retries honor `Retry-After` delta-seconds and HTTP dates, waiting at least the greater of server guidance and normal jittered backoff. Invalid or past values use normal backoff. Server waits above the existing 30-second automatic backoff cap decline retry and return the original error rather than retrying early. Total request deadlines remain authoritative; this does not enable retries for otherwise non-retryable requests.
|
|
279
|
+
|
|
278
280
|
## Credentials
|
|
279
281
|
|
|
280
282
|
Use environment variables for ephemeral runs. For disposable agent tests, set `DSS_CONFIG_DIR` to a temporary directory so saved credentials never touch your real profile.
|
package/dist/build-metadata.json
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"format":1,"buildRevision":"
|
|
1
|
+
{"format":1,"buildRevision":"c2f0acd7beed8f8256bfa72bfaa99794237a0a24"}
|
|
@@ -1,5 +1,4 @@
|
|
|
1
1
|
import { readFileSync, } from "node:fs";
|
|
2
|
-
import { ScenarioScriptRunWithCleanupFailureError, } from "../../resources/scenarios.js";
|
|
3
2
|
import { num, readStdinText, stripUtf8Bom, } from "../coerce.js";
|
|
4
3
|
import { enqueueCliWarning, } from "../output.js";
|
|
5
4
|
import { UsageError, } from "../usage.js";
|
|
@@ -78,6 +77,7 @@ function reportCleanupFailure(cleanupFailure) {
|
|
|
78
77
|
});
|
|
79
78
|
}
|
|
80
79
|
function unwrapCodeRunError(error) {
|
|
80
|
+
const { ScenarioScriptRunWithCleanupFailureError, } = require("../../resources/scenarios.js");
|
|
81
81
|
if (error instanceof ScenarioScriptRunWithCleanupFailureError) {
|
|
82
82
|
reportCleanupFailure(error.cleanupFailure);
|
|
83
83
|
return error.cause;
|
|
@@ -1,4 +1,3 @@
|
|
|
1
|
-
import { buildDatasetCloneSettings, } from "../../resources/datasets.js";
|
|
2
1
|
import { deepMerge, } from "../../utils/deep-merge.js";
|
|
3
2
|
import { compareStrings, stableHash, } from "../../utils/stable-hash.js";
|
|
4
3
|
import { jsonInput, num, schemaColumnsInput, unknownJsonInput, } from "../coerce.js";
|
|
@@ -538,6 +537,7 @@ export const datasetCommands = {
|
|
|
538
537
|
allowSamePath: f["allow-same-path"] === true,
|
|
539
538
|
};
|
|
540
539
|
const current = await c.datasets.get(a[0], pk);
|
|
540
|
+
const { buildDatasetCloneSettings, } = await import("../../resources/datasets.js");
|
|
541
541
|
const next = buildDatasetCloneSettings(current, a[1], pk ?? c.resolveProjectKey(pk), opts);
|
|
542
542
|
const zoneId = await resolveFlowZoneIdFromFlags(c, f, pk);
|
|
543
543
|
if (executionMode(f).dryRun) {
|
|
@@ -1,6 +1,5 @@
|
|
|
1
1
|
import { mkdir, writeFile, } from "node:fs/promises";
|
|
2
2
|
import { dirname, resolve, } from "node:path";
|
|
3
|
-
import { parseJobLogProgress, } from "../../resources/jobs.js";
|
|
4
3
|
import { jobBuildTargetTypeFromFlags, jobLogFilterFromFlag, maxLogLinesFromFlags, num, plainRecord, } from "../coerce.js";
|
|
5
4
|
import { executionMode, } from "../flags.js";
|
|
6
5
|
import { encodedProjectEndpoint, } from "../output.js";
|
|
@@ -158,7 +157,10 @@ async function jobInspectionSummary(client, jobId, flags) {
|
|
|
158
157
|
logError = error instanceof Error ? error.message : String(error);
|
|
159
158
|
}
|
|
160
159
|
const durationMs = jobDurationMs(details);
|
|
161
|
-
const progress = log
|
|
160
|
+
const progress = log
|
|
161
|
+
? require("../../resources/jobs.js")
|
|
162
|
+
.parseJobLogProgress(log, durationMs)
|
|
163
|
+
: undefined;
|
|
162
164
|
const logLines = log
|
|
163
165
|
? log.split(/\r?\n/).map((line) => line.trim()).filter((line) => line.length > 0)
|
|
164
166
|
: [];
|
|
@@ -1,4 +1,3 @@
|
|
|
1
|
-
import { macroResultMaxBytes, } from "../../resources/macros.js";
|
|
2
1
|
import { jsonInput, num, } from "../coerce.js";
|
|
3
2
|
import { executionMode, } from "../flags.js";
|
|
4
3
|
import { encodedProjectEndpoint, } from "../output.js";
|
|
@@ -211,7 +210,8 @@ export const macroCommands = {
|
|
|
211
210
|
}
|
|
212
211
|
return c.macros.result(a[0], a[1], {
|
|
213
212
|
projectKey: pk,
|
|
214
|
-
maxBytes:
|
|
213
|
+
maxBytes: require("../../resources/macros.js")
|
|
214
|
+
.macroResultMaxBytes(maxBytes),
|
|
215
215
|
});
|
|
216
216
|
},
|
|
217
217
|
usage: "dss macro result <macro_id> <run_id> [--max-bytes N (default 8388608)] [--project-key KEY]",
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { validateGitReferencePath, } from "../../
|
|
1
|
+
import { validateGitReferencePath, } from "../../utils/git-reference.js";
|
|
2
2
|
import { sanitizeErrorSecrets, sanitizeSecrets, } from "../../utils/secret-sanitize.js";
|
|
3
3
|
import { num, parseBooleanOption, requiredStringFlag, } from "../coerce.js";
|
|
4
4
|
import { CommandResultFailure, } from "../output.js";
|
|
@@ -1,4 +1,3 @@
|
|
|
1
|
-
import { scenarioUpdatePreview, } from "../../resources/scenarios.js";
|
|
2
1
|
import { jsonInput, num, parseBooleanOption, } from "../coerce.js";
|
|
3
2
|
import { executionMode, } from "../flags.js";
|
|
4
3
|
import { encodedProjectEndpoint, readIfExists, skipResult, } from "../output.js";
|
|
@@ -153,6 +152,7 @@ export const scenarioCommands = {
|
|
|
153
152
|
const pk = f["project-key"];
|
|
154
153
|
if (executionMode(f).dryRun) {
|
|
155
154
|
const current = await c.scenarios.get(a[0], { projectKey: pk, });
|
|
155
|
+
const { scenarioUpdatePreview, } = await import("../../resources/scenarios.js");
|
|
156
156
|
const preview = scenarioUpdatePreview(current, data);
|
|
157
157
|
return {
|
|
158
158
|
dryRun: true,
|
package/dist/src/cli/contract.js
CHANGED
|
@@ -3,10 +3,11 @@ import * as path from "node:path";
|
|
|
3
3
|
import { validateCredentials, } from "../auth.js";
|
|
4
4
|
import { getCredentialsPath, saveCredentials, } from "../config.js";
|
|
5
5
|
import { DataikuError, } from "../errors.js";
|
|
6
|
+
import actionOutputSchemasMetadata from "../generated/action-output-schemas.json" with { type: "json"
|
|
7
|
+
};
|
|
6
8
|
import { APP_MANIFEST_CONCURRENCY_CONTROL, } from "../resources/applications.js";
|
|
7
|
-
import { encodeGitReferencePath, validateGitReferencePath, } from "../resources/project-git.js";
|
|
8
9
|
import { encodeLibraryPath, PROJECT_LIBRARY_CONCURRENCY_CONTROL, validateLibraryDestinationPath, validateLibraryName, validateLibraryPath, } from "../resources/project-library.js";
|
|
9
|
-
import {
|
|
10
|
+
import { encodeGitReferencePath, validateGitReferencePath, } from "../utils/git-reference.js";
|
|
10
11
|
import { jobBuildTargetTypeFromFlags, json, jsonInput, num, parseBooleanOption, parseJsonObject, recipeInputDatasetsFromFlags, requiredJsonInput, rewritePairsFromFlags, schemaColumnsInput, sha256Hex, stableHash, stringField, textInput, } from "./coerce.js";
|
|
11
12
|
import { parseCodeRunIntegerFlag, resolveCodeInputWithSource, } from "./commands/code.js";
|
|
12
13
|
import { commands, } from "./commands/index.js";
|
|
@@ -433,212 +434,18 @@ function structuredExamples(examples, examplePayload) {
|
|
|
433
434
|
};
|
|
434
435
|
});
|
|
435
436
|
}
|
|
436
|
-
const PROJECT_LIBRARY_ITEM_OUTPUT_SCHEMA = {
|
|
437
|
-
type: "object",
|
|
438
|
-
additionalProperties: false,
|
|
439
|
-
required: ["name",],
|
|
440
|
-
properties: {
|
|
441
|
-
name: { type: "string", },
|
|
442
|
-
path: { type: "string", },
|
|
443
|
-
size: { type: "integer", minimum: 0, },
|
|
444
|
-
mimeType: { type: "string", },
|
|
445
|
-
hasData: { type: "boolean", },
|
|
446
|
-
lastModified: { type: "number", },
|
|
447
|
-
children: { type: "array", items: { type: "object", additionalProperties: true, }, },
|
|
448
|
-
},
|
|
449
|
-
};
|
|
450
|
-
const NOTEBOOK_SAVE_OUTPUT_SCHEMA = {
|
|
451
|
-
type: "object",
|
|
452
|
-
additionalProperties: false,
|
|
453
|
-
required: ["saved", "resource", "created", "hash",],
|
|
454
|
-
properties: {
|
|
455
|
-
saved: { type: "string", },
|
|
456
|
-
resource: { enum: ["jupyter-notebook", "sql-notebook",], },
|
|
457
|
-
created: { type: "boolean", },
|
|
458
|
-
hash: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
459
|
-
},
|
|
460
|
-
};
|
|
461
437
|
/**
|
|
462
|
-
*
|
|
463
|
-
*
|
|
438
|
+
* Discovery output schemas: the generated declarative metadata committed at
|
|
439
|
+
* `src/generated/action-output-schemas.json`. Plain data only, so discovery
|
|
440
|
+
* (`dss commands run`, `dss agent contract`) never loads the TypeBox graph.
|
|
441
|
+
* The content matches `typeBoxCommandOutputSchemas()` exactly; parity and
|
|
442
|
+
* freshness are enforced by tests/cli/discovery-output-schemas.test.ts and
|
|
443
|
+
* `bun run check`.
|
|
464
444
|
*/
|
|
465
|
-
const
|
|
466
|
-
|
|
467
|
-
required: ["saved", "requested", "resource", "created", "hash",],
|
|
468
|
-
properties: {
|
|
469
|
-
...NOTEBOOK_SAVE_OUTPUT_SCHEMA["properties"],
|
|
470
|
-
requested: { type: "string", },
|
|
471
|
-
},
|
|
472
|
-
};
|
|
473
|
-
const COMMAND_OUTPUT_SCHEMAS = {
|
|
474
|
-
"code-env.list": CodeEnvSummaryArraySchema,
|
|
475
|
-
"code-env.get": CodeEnvDetailsSchema,
|
|
476
|
-
"code-env.list-logs": CodeEnvLogSummaryArraySchema,
|
|
477
|
-
"code-env.get-log": {
|
|
478
|
-
oneOf: [
|
|
479
|
-
{
|
|
480
|
-
type: "object",
|
|
481
|
-
required: ["log", "bytes", "truncated", "tailed", "envLang", "envName", "logName",],
|
|
482
|
-
properties: {
|
|
483
|
-
log: { type: "string", },
|
|
484
|
-
bytes: { type: "integer", minimum: 0, },
|
|
485
|
-
truncated: { type: "boolean", },
|
|
486
|
-
tailed: { type: "boolean", },
|
|
487
|
-
envLang: { enum: ["PYTHON", "R",], },
|
|
488
|
-
envName: { type: "string", },
|
|
489
|
-
logName: { type: "string", },
|
|
490
|
-
},
|
|
491
|
-
},
|
|
492
|
-
{
|
|
493
|
-
type: "object",
|
|
494
|
-
required: ["path", "bytes", "truncated", "tailed", "envLang", "envName", "logName",],
|
|
495
|
-
properties: {
|
|
496
|
-
path: { type: "string", },
|
|
497
|
-
bytes: { type: "integer", minimum: 0, },
|
|
498
|
-
truncated: { type: "boolean", },
|
|
499
|
-
tailed: { type: "boolean", },
|
|
500
|
-
envLang: { enum: ["PYTHON", "R",], },
|
|
501
|
-
envName: { type: "string", },
|
|
502
|
-
logName: { type: "string", },
|
|
503
|
-
},
|
|
504
|
-
},
|
|
505
|
-
],
|
|
506
|
-
},
|
|
507
|
-
"code-env.version": CodeEnvVersionForProjectSchema,
|
|
508
|
-
"code-env.usages": CodeEnvUsageArraySchema,
|
|
509
|
-
"code.run": {
|
|
510
|
-
type: "object",
|
|
511
|
-
additionalProperties: false,
|
|
512
|
-
required: [
|
|
513
|
-
"outcome",
|
|
514
|
-
"success",
|
|
515
|
-
"runId",
|
|
516
|
-
"elapsedMs",
|
|
517
|
-
"pollCount",
|
|
518
|
-
"output",
|
|
519
|
-
"logTruncated",
|
|
520
|
-
"maxLogBytes",
|
|
521
|
-
"cleanup",
|
|
522
|
-
],
|
|
523
|
-
properties: {
|
|
524
|
-
outcome: { type: "string", },
|
|
525
|
-
success: { type: "boolean", },
|
|
526
|
-
runId: { type: "string", },
|
|
527
|
-
elapsedMs: { type: "number", minimum: 0, },
|
|
528
|
-
pollCount: { type: "integer", minimum: 0, },
|
|
529
|
-
output: { type: "string", },
|
|
530
|
-
log: { type: "string", },
|
|
531
|
-
logTruncated: { type: "boolean", },
|
|
532
|
-
maxLogBytes: { type: "integer", minimum: 0, },
|
|
533
|
-
timedOut: { const: true, },
|
|
534
|
-
timeoutMs: { type: "integer", minimum: 0, },
|
|
535
|
-
cleanup: {
|
|
536
|
-
type: "object",
|
|
537
|
-
additionalProperties: false,
|
|
538
|
-
required: ["status",],
|
|
539
|
-
properties: {
|
|
540
|
-
status: { enum: ["deleted", "kept", "failed",], },
|
|
541
|
-
error: { type: "string", },
|
|
542
|
-
},
|
|
543
|
-
},
|
|
544
|
-
},
|
|
545
|
-
},
|
|
546
|
-
"notebook.list-jupyter": JupyterNotebookSummaryArraySchema,
|
|
547
|
-
"notebook.get-jupyter": JupyterNotebookContentSchema,
|
|
548
|
-
"notebook.sessions-jupyter": NotebookSessionArraySchema,
|
|
549
|
-
"notebook.list-sql": SqlNotebookSummaryArraySchema,
|
|
550
|
-
"notebook.get-sql": SqlNotebookContentSchema,
|
|
551
|
-
"notebook.history-sql": SqlNotebookHistorySchema,
|
|
552
|
-
"notebook.save-jupyter": NOTEBOOK_SAVE_OUTPUT_SCHEMA,
|
|
553
|
-
"notebook.save-sql": NOTEBOOK_SAVE_SQL_OUTPUT_SCHEMA,
|
|
554
|
-
"project-library.list": { type: "array", items: PROJECT_LIBRARY_ITEM_OUTPUT_SCHEMA, },
|
|
555
|
-
"project-library.get-bytes": {
|
|
556
|
-
type: "object",
|
|
557
|
-
additionalProperties: false,
|
|
558
|
-
required: ["path", "bytes", "sha256",],
|
|
559
|
-
properties: {
|
|
560
|
-
path: { type: "string", },
|
|
561
|
-
bytes: { type: "integer", minimum: 0, },
|
|
562
|
-
sha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
563
|
-
},
|
|
564
|
-
},
|
|
565
|
-
"project-library.put": {
|
|
566
|
-
type: "object",
|
|
567
|
-
additionalProperties: false,
|
|
568
|
-
required: ["updated", "bytes", "sha256",],
|
|
569
|
-
properties: {
|
|
570
|
-
updated: { type: "string", },
|
|
571
|
-
bytes: { type: "integer", minimum: 0, },
|
|
572
|
-
sha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
573
|
-
beforeSha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
574
|
-
},
|
|
575
|
-
},
|
|
576
|
-
"project-library.diff": {
|
|
577
|
-
type: "object",
|
|
578
|
-
additionalProperties: false,
|
|
579
|
-
required: [
|
|
580
|
-
"path",
|
|
581
|
-
"unchanged",
|
|
582
|
-
"added",
|
|
583
|
-
"removed",
|
|
584
|
-
"diff",
|
|
585
|
-
"diffTruncated",
|
|
586
|
-
"localSha256",
|
|
587
|
-
"localBytes",
|
|
588
|
-
"maxLines",
|
|
589
|
-
],
|
|
590
|
-
properties: {
|
|
591
|
-
path: { type: "string", },
|
|
592
|
-
unchanged: { type: "boolean", },
|
|
593
|
-
added: { type: "integer", minimum: 0, },
|
|
594
|
-
removed: { type: "integer", minimum: 0, },
|
|
595
|
-
diff: { type: "string", },
|
|
596
|
-
diffTruncated: { type: "boolean", },
|
|
597
|
-
binary: { type: "boolean", },
|
|
598
|
-
remoteAbsent: { type: "boolean", },
|
|
599
|
-
remoteSha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
600
|
-
remoteBytes: { type: "integer", minimum: 0, },
|
|
601
|
-
localSha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
602
|
-
localBytes: { type: "integer", minimum: 0, },
|
|
603
|
-
maxLines: { type: "integer", minimum: 0, },
|
|
604
|
-
},
|
|
605
|
-
},
|
|
606
|
-
"project.list": ProjectSummaryArraySchema,
|
|
607
|
-
"project.get": ProjectDetailsSchema,
|
|
608
|
-
"project.metadata": ProjectMetadataSchema,
|
|
609
|
-
"dataset.list": DatasetSummaryArraySchema,
|
|
610
|
-
"dataset.get": DatasetDetailsSchema,
|
|
611
|
-
"dataset.schema": DatasetSchemaSchema,
|
|
612
|
-
"recipe.list": RecipeSummaryArraySchema,
|
|
613
|
-
"job.list": JobSummaryArraySchema,
|
|
614
|
-
"job.wait": JobWaitResultSchema,
|
|
615
|
-
"job.monitor": JobWaitResultSchema,
|
|
616
|
-
"scenario.list": ScenarioSummaryArraySchema,
|
|
617
|
-
"scenario.get": ScenarioDetailsSchema,
|
|
618
|
-
"scenario.status": ScenarioStatusSchema,
|
|
619
|
-
"flow-zone.list": FlowZoneArraySchema,
|
|
620
|
-
"flow-zone.get": FlowZoneSchema,
|
|
621
|
-
"sql.query": {
|
|
622
|
-
anyOf: [
|
|
623
|
-
SqlQueryResponseSchema,
|
|
624
|
-
{
|
|
625
|
-
type: "object",
|
|
626
|
-
required: ["queryId", "rowCount", "preview",],
|
|
627
|
-
additionalProperties: true,
|
|
628
|
-
properties: {
|
|
629
|
-
queryId: { type: "string", },
|
|
630
|
-
rowCount: { type: "number", },
|
|
631
|
-
preview: { type: "array", items: true, },
|
|
632
|
-
truncated: { type: "boolean", },
|
|
633
|
-
outputPath: { type: "string", },
|
|
634
|
-
written: { type: "string", },
|
|
635
|
-
},
|
|
636
|
-
},
|
|
637
|
-
],
|
|
638
|
-
},
|
|
639
|
-
};
|
|
445
|
+
const actionOutputSchemas = actionOutputSchemasMetadata
|
|
446
|
+
.schemas;
|
|
640
447
|
function outputJsonSchema(resource, action, shape) {
|
|
641
|
-
const precise =
|
|
448
|
+
const precise = actionOutputSchemas[registryKey(resource, action)];
|
|
642
449
|
if (precise)
|
|
643
450
|
return precise;
|
|
644
451
|
if (shape === "array")
|
|
@@ -1460,6 +1267,7 @@ function buildRegistryEntry(resource, action, meta) {
|
|
|
1460
1267
|
const uniqueRequiredFlags = uniqueStrings(requiredFlags);
|
|
1461
1268
|
const uniqueOptionalFlags = uniqueStrings(optionalFlags);
|
|
1462
1269
|
const unsafe = unsafeOutputs(resource, action, producesLocalFile);
|
|
1270
|
+
let schemas;
|
|
1463
1271
|
return {
|
|
1464
1272
|
resource,
|
|
1465
1273
|
action,
|
|
@@ -1486,7 +1294,9 @@ function buildRegistryEntry(resource, action, meta) {
|
|
|
1486
1294
|
...(requiredOneOf.length > 0 ? { requiredOneOf, } : {}),
|
|
1487
1295
|
...(inputGroups.length > 0 ? { requiredInputGroups: inputGroups, } : {}),
|
|
1488
1296
|
...(payloadSchema ? { payloadSchema, } : {}),
|
|
1489
|
-
schemas
|
|
1297
|
+
get schemas() {
|
|
1298
|
+
return schemas ??= buildCommandSchemas(resource, action, flagMetadata, uniqueRequiredFlags, requiredOneOf, payloadSchema, outputShape, meta.usage);
|
|
1299
|
+
},
|
|
1490
1300
|
...(unsafe ? { unsafeOutputs: unsafe, } : {}),
|
|
1491
1301
|
...(examplePayload !== undefined ? { examplePayload, } : {}),
|
|
1492
1302
|
...(cleanupCommand ? { cleanupCommand, } : {}),
|
|
@@ -1591,9 +1401,14 @@ function commandDefinitions(resourceFilter) {
|
|
|
1591
1401
|
export function buildCommandRegistry(resourceFilter) {
|
|
1592
1402
|
const registry = {};
|
|
1593
1403
|
for (const [resource, actions,] of Object.entries(commandDefinitions(resourceFilter))) {
|
|
1594
|
-
|
|
1404
|
+
const entries = {};
|
|
1405
|
+
registry[resource] = entries;
|
|
1595
1406
|
for (const [action, meta,] of Object.entries(actions)) {
|
|
1596
|
-
|
|
1407
|
+
let entry;
|
|
1408
|
+
Object.defineProperty(entries, action, {
|
|
1409
|
+
enumerable: true,
|
|
1410
|
+
get: () => entry ??= buildRegistryEntry(resource, action, meta),
|
|
1411
|
+
});
|
|
1597
1412
|
}
|
|
1598
1413
|
}
|
|
1599
1414
|
return registry;
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Canonical discovery output schemas, keyed by `resource.action`.
|
|
3
|
+
*
|
|
4
|
+
* TypeBox definitions from `packages/types` plus the inline literal schemas
|
|
5
|
+
* that exist only on the CLI side (code-env log envelope, code.run outcome,
|
|
6
|
+
* project-library payloads, notebook saves, sql.query handoff envelope).
|
|
7
|
+
* `scripts/generate-action-output-schemas.mjs` serializes this map into
|
|
8
|
+
* `src/generated/action-output-schemas.json`; discovery (src/cli/contract.ts)
|
|
9
|
+
* reads that generated JSON as plain data, so the TypeBox schema graph never
|
|
10
|
+
* loads on the discovery path. SDK validation keeps using the TypeBox schema
|
|
11
|
+
* objects from `packages/types` directly.
|
|
12
|
+
*
|
|
13
|
+
* Imported by the generator and the discovery parity test only.
|
|
14
|
+
*/
|
|
15
|
+
/**
|
|
16
|
+
* Canonical TypeBox-derived output schemas, keyed by `resource.action`. This is
|
|
17
|
+
* the single source of truth for `src/generated/action-output-schemas.json`,
|
|
18
|
+
* which discovery reads as plain data so no TypeBox schema graph loads on the
|
|
19
|
+
* discovery path. Regenerate with
|
|
20
|
+
* `bun scripts/generate-action-output-schemas.mjs`; `bun run check` fails when
|
|
21
|
+
* the generated file is stale. SDK validation keeps using the TypeBox schema
|
|
22
|
+
* objects from `packages/types` directly and never depends on the generated file.
|
|
23
|
+
*/
|
|
24
|
+
export declare function typeBoxCommandOutputSchemas(): Record<string, Record<string, unknown>>;
|
|
@@ -0,0 +1,233 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Canonical discovery output schemas, keyed by `resource.action`.
|
|
3
|
+
*
|
|
4
|
+
* TypeBox definitions from `packages/types` plus the inline literal schemas
|
|
5
|
+
* that exist only on the CLI side (code-env log envelope, code.run outcome,
|
|
6
|
+
* project-library payloads, notebook saves, sql.query handoff envelope).
|
|
7
|
+
* `scripts/generate-action-output-schemas.mjs` serializes this map into
|
|
8
|
+
* `src/generated/action-output-schemas.json`; discovery (src/cli/contract.ts)
|
|
9
|
+
* reads that generated JSON as plain data, so the TypeBox schema graph never
|
|
10
|
+
* loads on the discovery path. SDK validation keeps using the TypeBox schema
|
|
11
|
+
* objects from `packages/types` directly.
|
|
12
|
+
*
|
|
13
|
+
* Imported by the generator and the discovery parity test only.
|
|
14
|
+
*/
|
|
15
|
+
const PROJECT_LIBRARY_ITEM_OUTPUT_SCHEMA = {
|
|
16
|
+
type: "object",
|
|
17
|
+
additionalProperties: false,
|
|
18
|
+
required: ["name",],
|
|
19
|
+
properties: {
|
|
20
|
+
name: { type: "string", },
|
|
21
|
+
path: { type: "string", },
|
|
22
|
+
size: { type: "integer", minimum: 0, },
|
|
23
|
+
mimeType: { type: "string", },
|
|
24
|
+
hasData: { type: "boolean", },
|
|
25
|
+
lastModified: { type: "number", },
|
|
26
|
+
children: { type: "array", items: { type: "object", additionalProperties: true, }, },
|
|
27
|
+
},
|
|
28
|
+
};
|
|
29
|
+
const NOTEBOOK_SAVE_OUTPUT_SCHEMA = {
|
|
30
|
+
type: "object",
|
|
31
|
+
additionalProperties: false,
|
|
32
|
+
required: ["saved", "resource", "created", "hash",],
|
|
33
|
+
properties: {
|
|
34
|
+
saved: { type: "string", },
|
|
35
|
+
resource: { enum: ["jupyter-notebook", "sql-notebook",], },
|
|
36
|
+
created: { type: "boolean", },
|
|
37
|
+
hash: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
38
|
+
},
|
|
39
|
+
};
|
|
40
|
+
/**
|
|
41
|
+
* `notebook.save-sql` adds `requested`: on create DSS allocates the persisted
|
|
42
|
+
* id (`saved`), while the requested handle only becomes the display name.
|
|
43
|
+
*/
|
|
44
|
+
const NOTEBOOK_SAVE_SQL_OUTPUT_SCHEMA = {
|
|
45
|
+
...NOTEBOOK_SAVE_OUTPUT_SCHEMA,
|
|
46
|
+
required: ["saved", "requested", "resource", "created", "hash",],
|
|
47
|
+
properties: {
|
|
48
|
+
...NOTEBOOK_SAVE_OUTPUT_SCHEMA["properties"],
|
|
49
|
+
requested: { type: "string", },
|
|
50
|
+
},
|
|
51
|
+
};
|
|
52
|
+
let commandOutputSchemas;
|
|
53
|
+
/**
|
|
54
|
+
* Canonical TypeBox-derived output schemas, keyed by `resource.action`. This is
|
|
55
|
+
* the single source of truth for `src/generated/action-output-schemas.json`,
|
|
56
|
+
* which discovery reads as plain data so no TypeBox schema graph loads on the
|
|
57
|
+
* discovery path. Regenerate with
|
|
58
|
+
* `bun scripts/generate-action-output-schemas.mjs`; `bun run check` fails when
|
|
59
|
+
* the generated file is stale. SDK validation keeps using the TypeBox schema
|
|
60
|
+
* objects from `packages/types` directly and never depends on the generated file.
|
|
61
|
+
*/
|
|
62
|
+
export function typeBoxCommandOutputSchemas() {
|
|
63
|
+
if (commandOutputSchemas)
|
|
64
|
+
return commandOutputSchemas;
|
|
65
|
+
const { CodeEnvDetailsSchema, CodeEnvLogSummaryArraySchema, CodeEnvSummaryArraySchema, CodeEnvUsageArraySchema, CodeEnvVersionForProjectSchema, DatasetDetailsSchema, DatasetSchemaSchema, DatasetSummaryArraySchema, FlowZoneArraySchema, FlowZoneSchema, JobSummaryArraySchema, JobWaitResultSchema, JupyterNotebookContentSchema, JupyterNotebookSummaryArraySchema, NotebookSessionArraySchema, ProjectDetailsSchema, ProjectMetadataSchema, ProjectSummaryArraySchema, RecipeSummaryArraySchema, ScenarioDetailsSchema, ScenarioStatusSchema, ScenarioSummaryArraySchema, SqlNotebookContentSchema, SqlNotebookHistorySchema, SqlNotebookSummaryArraySchema, SqlQueryResponseSchema, } = require("../schemas.js");
|
|
66
|
+
return commandOutputSchemas = {
|
|
67
|
+
"code-env.list": CodeEnvSummaryArraySchema,
|
|
68
|
+
"code-env.get": CodeEnvDetailsSchema,
|
|
69
|
+
"code-env.list-logs": CodeEnvLogSummaryArraySchema,
|
|
70
|
+
"code-env.get-log": {
|
|
71
|
+
oneOf: [
|
|
72
|
+
{
|
|
73
|
+
type: "object",
|
|
74
|
+
required: ["log", "bytes", "truncated", "tailed", "envLang", "envName", "logName",],
|
|
75
|
+
properties: {
|
|
76
|
+
log: { type: "string", },
|
|
77
|
+
bytes: { type: "integer", minimum: 0, },
|
|
78
|
+
truncated: { type: "boolean", },
|
|
79
|
+
tailed: { type: "boolean", },
|
|
80
|
+
envLang: { enum: ["PYTHON", "R",], },
|
|
81
|
+
envName: { type: "string", },
|
|
82
|
+
logName: { type: "string", },
|
|
83
|
+
},
|
|
84
|
+
},
|
|
85
|
+
{
|
|
86
|
+
type: "object",
|
|
87
|
+
required: ["path", "bytes", "truncated", "tailed", "envLang", "envName", "logName",],
|
|
88
|
+
properties: {
|
|
89
|
+
path: { type: "string", },
|
|
90
|
+
bytes: { type: "integer", minimum: 0, },
|
|
91
|
+
truncated: { type: "boolean", },
|
|
92
|
+
tailed: { type: "boolean", },
|
|
93
|
+
envLang: { enum: ["PYTHON", "R",], },
|
|
94
|
+
envName: { type: "string", },
|
|
95
|
+
logName: { type: "string", },
|
|
96
|
+
},
|
|
97
|
+
},
|
|
98
|
+
],
|
|
99
|
+
},
|
|
100
|
+
"code-env.version": CodeEnvVersionForProjectSchema,
|
|
101
|
+
"code-env.usages": CodeEnvUsageArraySchema,
|
|
102
|
+
"code.run": {
|
|
103
|
+
type: "object",
|
|
104
|
+
additionalProperties: false,
|
|
105
|
+
required: [
|
|
106
|
+
"outcome",
|
|
107
|
+
"success",
|
|
108
|
+
"runId",
|
|
109
|
+
"elapsedMs",
|
|
110
|
+
"pollCount",
|
|
111
|
+
"output",
|
|
112
|
+
"logTruncated",
|
|
113
|
+
"maxLogBytes",
|
|
114
|
+
"cleanup",
|
|
115
|
+
],
|
|
116
|
+
properties: {
|
|
117
|
+
outcome: { type: "string", },
|
|
118
|
+
success: { type: "boolean", },
|
|
119
|
+
runId: { type: "string", },
|
|
120
|
+
elapsedMs: { type: "number", minimum: 0, },
|
|
121
|
+
pollCount: { type: "integer", minimum: 0, },
|
|
122
|
+
output: { type: "string", },
|
|
123
|
+
log: { type: "string", },
|
|
124
|
+
logTruncated: { type: "boolean", },
|
|
125
|
+
maxLogBytes: { type: "integer", minimum: 0, },
|
|
126
|
+
timedOut: { const: true, },
|
|
127
|
+
timeoutMs: { type: "integer", minimum: 0, },
|
|
128
|
+
cleanup: {
|
|
129
|
+
type: "object",
|
|
130
|
+
additionalProperties: false,
|
|
131
|
+
required: ["status",],
|
|
132
|
+
properties: {
|
|
133
|
+
status: { enum: ["deleted", "kept", "failed",], },
|
|
134
|
+
error: { type: "string", },
|
|
135
|
+
},
|
|
136
|
+
},
|
|
137
|
+
},
|
|
138
|
+
},
|
|
139
|
+
"notebook.list-jupyter": JupyterNotebookSummaryArraySchema,
|
|
140
|
+
"notebook.get-jupyter": JupyterNotebookContentSchema,
|
|
141
|
+
"notebook.sessions-jupyter": NotebookSessionArraySchema,
|
|
142
|
+
"notebook.list-sql": SqlNotebookSummaryArraySchema,
|
|
143
|
+
"notebook.get-sql": SqlNotebookContentSchema,
|
|
144
|
+
"notebook.history-sql": SqlNotebookHistorySchema,
|
|
145
|
+
"notebook.save-jupyter": NOTEBOOK_SAVE_OUTPUT_SCHEMA,
|
|
146
|
+
"notebook.save-sql": NOTEBOOK_SAVE_SQL_OUTPUT_SCHEMA,
|
|
147
|
+
"project-library.list": { type: "array", items: PROJECT_LIBRARY_ITEM_OUTPUT_SCHEMA, },
|
|
148
|
+
"project-library.get-bytes": {
|
|
149
|
+
type: "object",
|
|
150
|
+
additionalProperties: false,
|
|
151
|
+
required: ["path", "bytes", "sha256",],
|
|
152
|
+
properties: {
|
|
153
|
+
path: { type: "string", },
|
|
154
|
+
bytes: { type: "integer", minimum: 0, },
|
|
155
|
+
sha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
156
|
+
},
|
|
157
|
+
},
|
|
158
|
+
"project-library.put": {
|
|
159
|
+
type: "object",
|
|
160
|
+
additionalProperties: false,
|
|
161
|
+
required: ["updated", "bytes", "sha256",],
|
|
162
|
+
properties: {
|
|
163
|
+
updated: { type: "string", },
|
|
164
|
+
bytes: { type: "integer", minimum: 0, },
|
|
165
|
+
sha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
166
|
+
beforeSha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
167
|
+
},
|
|
168
|
+
},
|
|
169
|
+
"project-library.diff": {
|
|
170
|
+
type: "object",
|
|
171
|
+
additionalProperties: false,
|
|
172
|
+
required: [
|
|
173
|
+
"path",
|
|
174
|
+
"unchanged",
|
|
175
|
+
"added",
|
|
176
|
+
"removed",
|
|
177
|
+
"diff",
|
|
178
|
+
"diffTruncated",
|
|
179
|
+
"localSha256",
|
|
180
|
+
"localBytes",
|
|
181
|
+
"maxLines",
|
|
182
|
+
],
|
|
183
|
+
properties: {
|
|
184
|
+
path: { type: "string", },
|
|
185
|
+
unchanged: { type: "boolean", },
|
|
186
|
+
added: { type: "integer", minimum: 0, },
|
|
187
|
+
removed: { type: "integer", minimum: 0, },
|
|
188
|
+
diff: { type: "string", },
|
|
189
|
+
diffTruncated: { type: "boolean", },
|
|
190
|
+
binary: { type: "boolean", },
|
|
191
|
+
remoteAbsent: { type: "boolean", },
|
|
192
|
+
remoteSha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
193
|
+
remoteBytes: { type: "integer", minimum: 0, },
|
|
194
|
+
localSha256: { type: "string", pattern: "^[a-f0-9]{64}$", },
|
|
195
|
+
localBytes: { type: "integer", minimum: 0, },
|
|
196
|
+
maxLines: { type: "integer", minimum: 0, },
|
|
197
|
+
},
|
|
198
|
+
},
|
|
199
|
+
"project.list": ProjectSummaryArraySchema,
|
|
200
|
+
"project.get": ProjectDetailsSchema,
|
|
201
|
+
"project.metadata": ProjectMetadataSchema,
|
|
202
|
+
"dataset.list": DatasetSummaryArraySchema,
|
|
203
|
+
"dataset.get": DatasetDetailsSchema,
|
|
204
|
+
"dataset.schema": DatasetSchemaSchema,
|
|
205
|
+
"recipe.list": RecipeSummaryArraySchema,
|
|
206
|
+
"job.list": JobSummaryArraySchema,
|
|
207
|
+
"job.wait": JobWaitResultSchema,
|
|
208
|
+
"job.monitor": JobWaitResultSchema,
|
|
209
|
+
"scenario.list": ScenarioSummaryArraySchema,
|
|
210
|
+
"scenario.get": ScenarioDetailsSchema,
|
|
211
|
+
"scenario.status": ScenarioStatusSchema,
|
|
212
|
+
"flow-zone.list": FlowZoneArraySchema,
|
|
213
|
+
"flow-zone.get": FlowZoneSchema,
|
|
214
|
+
"sql.query": {
|
|
215
|
+
anyOf: [
|
|
216
|
+
SqlQueryResponseSchema,
|
|
217
|
+
{
|
|
218
|
+
type: "object",
|
|
219
|
+
required: ["queryId", "rowCount", "preview",],
|
|
220
|
+
additionalProperties: true,
|
|
221
|
+
properties: {
|
|
222
|
+
queryId: { type: "string", },
|
|
223
|
+
rowCount: { type: "number", },
|
|
224
|
+
preview: { type: "array", items: true, },
|
|
225
|
+
truncated: { type: "boolean", },
|
|
226
|
+
outputPath: { type: "string", },
|
|
227
|
+
written: { type: "string", },
|
|
228
|
+
},
|
|
229
|
+
},
|
|
230
|
+
],
|
|
231
|
+
},
|
|
232
|
+
};
|
|
233
|
+
}
|