@toloka-platform/sdk 0.1.0-preview.7 → 0.1.0-preview.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +3 -4
- package/dist/cli/toloka.js +133 -130
- package/dist/cli/toloka.js.jsc +0 -0
- package/dist/cli/toloka.js.jsc.sha256 +1 -1
- package/dist/cli/toloka.js.sha256 +1 -1
- package/docs/authoring.md +3 -5
- package/docs/cli.md +15 -0
- package/docs/limitations.md +1 -1
- package/examples/face-image-collection/nodes/order-faces/node.ts +2 -4
- package/examples/video-pair-rating/nodes/adjudication_match/node.ts +15 -17
- package/examples/video-pair-rating/nodes/assemble/node.ts +25 -27
- package/examples/video-pair-rating/nodes/cascade/node.ts +24 -26
- package/examples/video-pair-rating/nodes/finalize_overall/node.ts +17 -19
- package/examples/video-pair-rating/nodes/merge_adjudication/node.ts +13 -15
- package/examples/video-pair-rating/nodes/merge_op_adjudication/node.ts +12 -14
- package/examples/video-pair-rating/nodes/op_quorum_resolve/node.ts +13 -15
- package/examples/video-pair-rating/nodes/route_adjudication/node.ts +1 -3
- package/examples/video-pair-rating/nodes/route_op_adjudication/node.ts +5 -7
- package/examples/video-pair-rating/nodes/route_overall/node.ts +1 -3
- package/examples/video-pair-rating/nodes/shuffle_ab/node.ts +11 -13
- package/openapi/gen/models/datasetItem.ts +0 -2
- package/openapi/gen/models/index.ts +0 -1
- package/package.json +1 -1
- package/skills/catalog.json +4 -4
- package/skills/toloka-build/SKILL.md +1 -0
- package/skills/toloka-build/references/node-semantics.md +5 -5
- package/skills/toloka-build/references/workflow-and-cli.md +6 -1
- package/skills/toloka-report-issue/references/report-template.md +1 -1
- package/src/authoring/index.ts +27 -26
- package/src/cli/analytics.ts +8 -19
- package/src/cli/auto-update.ts +367 -0
- package/src/cli/boolean-flag.ts +13 -0
- package/src/cli/credentials.ts +3 -0
- package/src/cli/program.ts +37 -1
- package/src/cli/settings.ts +93 -0
- package/src/importer.ts +12 -16
- package/openapi/gen/models/datasetItemDuplicateOfSubmissionId.ts +0 -2
package/dist/cli/toloka.js.jsc
CHANGED
|
Binary file
|
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
ce1d79afa2e57b085b55b98cf5c4cb90ab84b8a6ba50e586586f208a70f7f088 toloka.js.jsc
|
|
@@ -1 +1 @@
|
|
|
1
|
-
|
|
1
|
+
44e401115084c2e912fe03234c4104090d1698c95e170f16cac9c4505952709a toloka.js
|
package/docs/authoring.md
CHANGED
|
@@ -61,10 +61,8 @@ import { code } from "@toloka-platform/sdk";
|
|
|
61
61
|
export const orderFaces = code("order_faces", {
|
|
62
62
|
about: { name: "order_faces", description: "Sort annotations top-left to bottom-right." },
|
|
63
63
|
envVars: [],
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
output: [{ field: faces, optional: false }],
|
|
67
|
-
},
|
|
64
|
+
input: [{ field: draftFaces, optional: false }],
|
|
65
|
+
output: [{ field: faces, optional: false }],
|
|
68
66
|
code: {
|
|
69
67
|
code: `from main_types import Inputs, Outputs
|
|
70
68
|
|
|
@@ -210,7 +208,7 @@ export const routeDisputed = match("route_disputed", {
|
|
|
210
208
|
predicates: [{ field: verdict, jsonPath: "$", operator: "text_eq", value: "disputed" }],
|
|
211
209
|
},
|
|
212
210
|
],
|
|
213
|
-
|
|
211
|
+
input: [{ field: verdict, optional: false }],
|
|
214
212
|
});
|
|
215
213
|
```
|
|
216
214
|
|
package/docs/cli.md
CHANGED
|
@@ -32,6 +32,8 @@ Credentials and the platform base URL resolve from the same precedence order, hi
|
|
|
32
32
|
|
|
33
33
|
`toloka login` verifies the key against the platform before saving anything. It writes `$XDG_CONFIG_HOME/toloka/credentials.json`, which defaults to `~/.config/toloka/credentials.json`, with file mode `0600` and a parent directory with mode `0700`. The file may also store local project aliases. The legacy `~/.toloka/credentials.json` is still read when the new path has no file. `toloka logout` deletes the stored file but does not revoke the key.
|
|
34
34
|
|
|
35
|
+
Machine settings live next to that credentials file in `settings.json` and default `auto-update` to `true`. A globally installed `toloka` checks `https://registry.npmjs.org/` at most once per day and, when a newer `@toloka-platform/sdk` version has been published for at least 24 hours, installs it with `bun add --global` and re-runs the original command. The check is pinned to that registry; a checkout `.env` cannot retarget it. `config`, `version`, and `help` never trigger a check. Project-local installs, CI, and `TOLOKA_SKIP_AUTO_UPDATE=1` also skip it. `TOLOKA_AUTO_UPDATE=0` turns it off for one command; `TOLOKA_AUTO_UPDATE=1` turns it on even when the setting is false.
|
|
36
|
+
|
|
35
37
|
Ordinary usage events (`sdk_cli_command_completed` / `sdk_cli_command_failed`) go to Toloka's PostHog project so we can see which commands run and which error classes they hit. Their payloads contain only an allowlisted command classifier, duration, exit code, outcome or error code, SDK and Bun versions, actor, and channel. Raw arguments, flag values, environment variables, API keys, secret names, base URLs and hosts, local paths, project ids, error messages, stack traces, and source context are not sent. GeoIP enrichment is disabled. Distinct id starts as a random UUID stored next to the credentials file. After a successful authenticated command, the CLI calls `GET /me` and aliases that UUID onto the platform user id without setting person properties. Set `TOLOKA_TELEMETRY=0` or `DO_NOT_TRACK=1` to disable capture. CI environments are disabled unless `TOLOKA_TELEMETRY=1`. The explicitly approved full-text `report` event is documented in [report](#report).
|
|
36
38
|
|
|
37
39
|
The root program accepts these flags before any command:
|
|
@@ -107,6 +109,19 @@ toloka auth-status [--show-token]
|
|
|
107
109
|
|
|
108
110
|
On success it prints the platform URL with its source, the API key masked unless `--show-token` is passed, the selected project, and `Reachable: N project(s) visible.` The project line is printed even when no key is in effect or the platform is not reachable. It exits `1` when no key is in effect or the platform is not reachable.
|
|
109
111
|
|
|
112
|
+
## config
|
|
113
|
+
|
|
114
|
+
Reads and writes machine CLI settings. The only setting today is `auto-update`, which defaults to `true`.
|
|
115
|
+
|
|
116
|
+
```sh
|
|
117
|
+
toloka config
|
|
118
|
+
toloka config get
|
|
119
|
+
toloka config get auto-update
|
|
120
|
+
toloka config set auto-update false
|
|
121
|
+
```
|
|
122
|
+
|
|
123
|
+
`config` and `config get` with no name print `auto-update=true` or `auto-update=false`. `config get auto-update` prints only the boolean. `config set auto-update <value>` accepts `true` / `false` / `1` / `0` / `on` / `off` / `yes` / `no` and prints `Set auto-update to <boolean>.` Unknown names and values exit `1`.
|
|
124
|
+
|
|
110
125
|
## init
|
|
111
126
|
|
|
112
127
|
Initializes the current project directory and installs the Agent Bootstrap entry points.
|
package/docs/limitations.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
What this Preview SDK does not do yet, stated plainly so you can decide what to depend on.
|
|
4
4
|
|
|
5
|
-
`@toloka-platform/sdk` is `0.1.0-preview.
|
|
5
|
+
`@toloka-platform/sdk` is `0.1.0-preview.8`. This is a Preview: the authoring API, the CLI surface, and the generated source layout can change without a major version bump. Pin an exact version. Do not depend on this in production yet.
|
|
6
6
|
|
|
7
7
|
## Local checks do not certify deployability
|
|
8
8
|
|
|
@@ -9,10 +9,8 @@ export const orderFaces = code("order_faces", {
|
|
|
9
9
|
"Order face annotations from top left to bottom right for deterministic downstream data.",
|
|
10
10
|
},
|
|
11
11
|
envVars: [],
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
output: [{ field: faces, optional: false }],
|
|
15
|
-
},
|
|
12
|
+
input: [{ field: draftFaces, optional: false }],
|
|
13
|
+
output: [{ field: faces, optional: false }],
|
|
16
14
|
code: {
|
|
17
15
|
code: `from main_types import Inputs, Outputs
|
|
18
16
|
|
|
@@ -20,23 +20,21 @@ export const adjudicationMatch = code("adjudication_match", {
|
|
|
20
20
|
"Computes per-rater matches_adjudication booleans for image_text_alignment, image_quality, and overall preference. Only populated when adjudication actually ran; otherwise null.",
|
|
21
21
|
},
|
|
22
22
|
envVars: [],
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
],
|
|
39
|
-
},
|
|
23
|
+
input: [
|
|
24
|
+
{ field: raterItaVotes, optional: false },
|
|
25
|
+
{ field: raterIqVotes, optional: false },
|
|
26
|
+
{ field: raterOpVotes, optional: true },
|
|
27
|
+
{ field: disagreedMetrics, optional: false },
|
|
28
|
+
{ field: finalMetrics, optional: false },
|
|
29
|
+
{ field: needsOverallManual, optional: false },
|
|
30
|
+
{ field: opNeedsAdjudication, optional: true },
|
|
31
|
+
{ field: finalOpValue, optional: true },
|
|
32
|
+
],
|
|
33
|
+
output: [
|
|
34
|
+
{ field: raterItaMatchesAdjudication, optional: false },
|
|
35
|
+
{ field: raterIqMatchesAdjudication, optional: false },
|
|
36
|
+
{ field: raterOpMatchesAdjudication, optional: false },
|
|
37
|
+
],
|
|
40
38
|
code: {
|
|
41
39
|
code: `from main_types import Inputs, Outputs
|
|
42
40
|
|
|
@@ -8,32 +8,30 @@ export const assemble = code("assemble", {
|
|
|
8
8
|
"Assembles the final JSONL delivery record containing all evaluation results: video identifiers, text prompt, per-metric scores, overall preference, annotator metadata, resolution method, rater vote history, and a UTC timestamp.",
|
|
9
9
|
},
|
|
10
10
|
envVars: [],
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
output: [{ field: f.deliveryRecord, optional: false }],
|
|
36
|
-
},
|
|
11
|
+
input: [
|
|
12
|
+
{ field: f.recordId, optional: false },
|
|
13
|
+
{ field: f.prompt, optional: false },
|
|
14
|
+
{ field: f.videoA, optional: false },
|
|
15
|
+
{ field: f.videoB, optional: false },
|
|
16
|
+
{ field: f.finalMetrics, optional: false },
|
|
17
|
+
{ field: f.overallPreference, optional: false },
|
|
18
|
+
{ field: f.overallPreferenceSource, optional: false },
|
|
19
|
+
{ field: f.resolutionMethod, optional: false },
|
|
20
|
+
{ field: f.annotatorIdAdjudicator, optional: false },
|
|
21
|
+
{ field: f.raterItaVotes, optional: false },
|
|
22
|
+
{ field: f.raterMtaVotes, optional: false },
|
|
23
|
+
{ field: f.raterIqVotes, optional: false },
|
|
24
|
+
{ field: f.raterMqVotes, optional: false },
|
|
25
|
+
{ field: f.raterSbcVotes, optional: false },
|
|
26
|
+
{ field: f.raterVrVotes, optional: false },
|
|
27
|
+
{ field: f.raterOpVotes, optional: true },
|
|
28
|
+
{ field: f.raterDurationSeconds, optional: true },
|
|
29
|
+
{ field: f.raterBothPlayed, optional: true },
|
|
30
|
+
{ field: f.raterItaMatchesAdjudication, optional: false },
|
|
31
|
+
{ field: f.raterIqMatchesAdjudication, optional: false },
|
|
32
|
+
{ field: f.raterOpMatchesAdjudication, optional: false },
|
|
33
|
+
],
|
|
34
|
+
output: [{ field: f.deliveryRecord, optional: false }],
|
|
37
35
|
code: {
|
|
38
36
|
code: `from datetime import datetime, timezone
|
|
39
37
|
from main_types import Inputs, Outputs
|
|
@@ -60,7 +58,7 @@ async def main(inputs: Inputs) -> Outputs:
|
|
|
60
58
|
"video_realism": final_metrics.get("video_realism"),
|
|
61
59
|
"video_image_text_alignment": final_metrics.get("image_text_alignment"),
|
|
62
60
|
"video_motion_text_alignment": final_metrics.get("motion_text_alignment"),
|
|
63
|
-
# Platform limitation: rater ids are not surfaced as
|
|
61
|
+
# Platform limitation: rater ids are not surfaced as output fields
|
|
64
62
|
# from the quorum node in this pilot. Stubs preserve schema conformance.
|
|
65
63
|
"annotator_id_1": None,
|
|
66
64
|
"annotator_id_2": None,
|
|
@@ -8,32 +8,30 @@ export const cascade = code("cascade", {
|
|
|
8
8
|
"Un-swaps each rater per-metric vote back to canonical video_a/video_b/tie labels using the swap flag. Computes per-metric majority agreement and flags metrics requiring adjudication. Applies a deterministic Overall Preference cascade: image-text alignment decides first, then image quality, falling back to manual review.",
|
|
9
9
|
},
|
|
10
10
|
envVars: [],
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
],
|
|
36
|
-
},
|
|
11
|
+
input: [
|
|
12
|
+
{ field: f.swap, optional: false },
|
|
13
|
+
{ field: f.imageTextAlignment, optional: false },
|
|
14
|
+
{ field: f.motionTextAlignment, optional: false },
|
|
15
|
+
{ field: f.imageQuality, optional: false },
|
|
16
|
+
{ field: f.motionQuality, optional: false },
|
|
17
|
+
{ field: f.subjectBackgroundConsistency, optional: false },
|
|
18
|
+
{ field: f.videoRealism, optional: false },
|
|
19
|
+
],
|
|
20
|
+
output: [
|
|
21
|
+
{ field: f.perMetricResolved, optional: false },
|
|
22
|
+
{ field: f.needsAdjudication, optional: false },
|
|
23
|
+
{ field: f.disagreedTask, optional: false },
|
|
24
|
+
{ field: f.disagreedMetrics, optional: false },
|
|
25
|
+
{ field: f.opValueOrNull, optional: false },
|
|
26
|
+
{ field: f.opSource, optional: false },
|
|
27
|
+
{ field: f.needsOverallManual, optional: false },
|
|
28
|
+
{ field: f.raterItaVotes, optional: false },
|
|
29
|
+
{ field: f.raterMtaVotes, optional: false },
|
|
30
|
+
{ field: f.raterIqVotes, optional: false },
|
|
31
|
+
{ field: f.raterMqVotes, optional: false },
|
|
32
|
+
{ field: f.raterSbcVotes, optional: false },
|
|
33
|
+
{ field: f.raterVrVotes, optional: false },
|
|
34
|
+
],
|
|
37
35
|
code: {
|
|
38
36
|
code: `from main_types import Inputs, Outputs
|
|
39
37
|
|
|
@@ -8,25 +8,23 @@ export const finalizeOverall = code("finalize_overall", {
|
|
|
8
8
|
"Combines cascade-derived and manually adjudicated Overall Preference into the final output. Re-runs the cascade against post-adjudication metrics and falls back to the manual quorum majority vote when both cascade tiers are tied.",
|
|
9
9
|
},
|
|
10
10
|
envVars: [],
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
],
|
|
29
|
-
},
|
|
11
|
+
input: [
|
|
12
|
+
{ field: f.opValueOrNull, optional: true },
|
|
13
|
+
{ field: f.opSource, optional: false },
|
|
14
|
+
{ field: f.needsOverallManual, optional: false },
|
|
15
|
+
{ field: f.finalMetrics, optional: false },
|
|
16
|
+
{ field: f.finalOpValue, optional: true },
|
|
17
|
+
{ field: f.finalOpResolutionMethod, optional: true },
|
|
18
|
+
{ field: f.finalOpAdjudicatorId, optional: true },
|
|
19
|
+
{ field: f.resolutionMethod, optional: false },
|
|
20
|
+
{ field: f.annotatorIdAdjudicator, optional: true },
|
|
21
|
+
],
|
|
22
|
+
output: [
|
|
23
|
+
{ field: f.overallPreference, optional: false },
|
|
24
|
+
{ field: f.overallPreferenceSource, optional: false },
|
|
25
|
+
{ field: f.resolutionMethod, optional: false },
|
|
26
|
+
{ field: f.annotatorIdAdjudicator, optional: false },
|
|
27
|
+
],
|
|
30
28
|
code: {
|
|
31
29
|
code: `from main_types import Inputs, Outputs
|
|
32
30
|
|
|
@@ -18,21 +18,19 @@ export const mergeAdjudication = code("merge_adjudication", {
|
|
|
18
18
|
"Merges majority-vote per-metric resolutions from the cascade with adjudicator overrides on disagreed metrics into final_metrics. Also stamps annotator_id_adjudicator and resolution_method.",
|
|
19
19
|
},
|
|
20
20
|
envVars: [],
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
],
|
|
35
|
-
},
|
|
21
|
+
input: [
|
|
22
|
+
{ field: perMetricResolved, optional: false },
|
|
23
|
+
{ field: adjudicatedMetrics, optional: true },
|
|
24
|
+
{ field: disagreedMetrics, optional: true },
|
|
25
|
+
],
|
|
26
|
+
output: [
|
|
27
|
+
{ field: finalMetrics, optional: false },
|
|
28
|
+
{ field: resolutionMethod, optional: false },
|
|
29
|
+
{ field: annotatorIdAdjudicator, optional: false },
|
|
30
|
+
{ field: opValueOrNull, optional: false },
|
|
31
|
+
{ field: opSource, optional: false },
|
|
32
|
+
{ field: needsOverallManual, optional: false },
|
|
33
|
+
],
|
|
36
34
|
code: {
|
|
37
35
|
code: `from main_types import Inputs, Outputs
|
|
38
36
|
|
|
@@ -15,18 +15,16 @@ export const mergeOpAdjudication = code("merge_op_adjudication", {
|
|
|
15
15
|
"Consolidates the overall preference branch: if adjudication ran, emit the adjudicator pick and resolution_method=adjudication; otherwise emit the quorum majority and resolution_method=agreement.",
|
|
16
16
|
},
|
|
17
17
|
envVars: [],
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
],
|
|
29
|
-
},
|
|
18
|
+
input: [
|
|
19
|
+
{ field: opNeedsAdjudication, optional: false },
|
|
20
|
+
{ field: opManualValue, optional: true },
|
|
21
|
+
{ field: opAdjudicatedValue, optional: true },
|
|
22
|
+
],
|
|
23
|
+
output: [
|
|
24
|
+
{ field: finalOpValue, optional: false },
|
|
25
|
+
{ field: finalOpResolutionMethod, optional: false },
|
|
26
|
+
{ field: finalOpAdjudicatorId, optional: false },
|
|
27
|
+
],
|
|
30
28
|
code: {
|
|
31
29
|
code: `from main_types import Inputs, Outputs
|
|
32
30
|
|
|
@@ -38,8 +36,8 @@ async def main(inputs: Inputs) -> Outputs:
|
|
|
38
36
|
"final_op_value": inputs["op_adjudicated_value"],
|
|
39
37
|
"final_op_resolution_method": "adjudication",
|
|
40
38
|
# Platform limitation (mirrors annotator_id_1/2 in assemble): the
|
|
41
|
-
# OP adjudicator id is not surfaced as
|
|
42
|
-
#
|
|
39
|
+
# OP adjudicator id is not surfaced as an output field from the
|
|
40
|
+
# generation node in this pilot. Emit null.
|
|
43
41
|
"final_op_adjudicator_id": None,
|
|
44
42
|
}
|
|
45
43
|
return {
|
|
@@ -18,21 +18,19 @@ export const opQuorumResolve = code("op_quorum_resolve", {
|
|
|
18
18
|
"Resolves three overall preference votes from the quorum into a majority. If fewer than two votes agree, flags for adjudication and packages the disagreed task payload.",
|
|
19
19
|
},
|
|
20
20
|
envVars: [],
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
],
|
|
35
|
-
},
|
|
21
|
+
input: [
|
|
22
|
+
{ field: prompt, optional: false },
|
|
23
|
+
{ field: videoA, optional: false },
|
|
24
|
+
{ field: videoB, optional: false },
|
|
25
|
+
{ field: swap, optional: false },
|
|
26
|
+
{ field: overallManualVotes, optional: false },
|
|
27
|
+
],
|
|
28
|
+
output: [
|
|
29
|
+
{ field: opManualValue, optional: true },
|
|
30
|
+
{ field: opNeedsAdjudication, optional: false },
|
|
31
|
+
{ field: disagreedOpTask, optional: false },
|
|
32
|
+
{ field: raterOpVotes, optional: false },
|
|
33
|
+
],
|
|
36
34
|
code: {
|
|
37
35
|
code: `from main_types import Inputs, Outputs
|
|
38
36
|
|
|
@@ -20,11 +20,9 @@ export const routeOpAdjudication = match("route_op_adjudication", {
|
|
|
20
20
|
],
|
|
21
21
|
},
|
|
22
22
|
],
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
],
|
|
29
|
-
},
|
|
23
|
+
input: [
|
|
24
|
+
{ field: opNeedsAdjudication, optional: false },
|
|
25
|
+
{ field: disagreedOpTask, optional: true },
|
|
26
|
+
{ field: opManualValue, optional: true },
|
|
27
|
+
],
|
|
30
28
|
});
|
|
@@ -8,19 +8,17 @@ export const shuffleAb = code("shuffle_ab", {
|
|
|
8
8
|
"Mints a deterministic record_id from a hash of the prompt and video pair. Swaps the left/right video assignment based on the hash parity to prevent position bias in rater interfaces.",
|
|
9
9
|
},
|
|
10
10
|
envVars: [],
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
],
|
|
23
|
-
},
|
|
11
|
+
input: [
|
|
12
|
+
{ field: f.prompt, optional: false },
|
|
13
|
+
{ field: f.videoA, optional: false },
|
|
14
|
+
{ field: f.videoB, optional: false },
|
|
15
|
+
],
|
|
16
|
+
output: [
|
|
17
|
+
{ field: f.recordId, optional: false },
|
|
18
|
+
{ field: f.leftVideo, optional: false },
|
|
19
|
+
{ field: f.rightVideo, optional: false },
|
|
20
|
+
{ field: f.swap, optional: false },
|
|
21
|
+
],
|
|
24
22
|
code: {
|
|
25
23
|
code: `import hashlib
|
|
26
24
|
from main_types import Inputs, Outputs
|
|
@@ -1,11 +1,9 @@
|
|
|
1
1
|
import type { DatasetItemId } from './datasetItemId';
|
|
2
2
|
import type { DatasetItemCurrentSubmissionId } from './datasetItemCurrentSubmissionId';
|
|
3
|
-
import type { DatasetItemDuplicateOfSubmissionId } from './datasetItemDuplicateOfSubmissionId';
|
|
4
3
|
|
|
5
4
|
export interface DatasetItem {
|
|
6
5
|
id: DatasetItemId;
|
|
7
6
|
index: number;
|
|
8
7
|
createdAt: string;
|
|
9
8
|
currentSubmissionId: DatasetItemCurrentSubmissionId;
|
|
10
|
-
duplicateOfSubmissionId: DatasetItemDuplicateOfSubmissionId;
|
|
11
9
|
}
|
|
@@ -134,7 +134,6 @@ export * from './datasetFieldJsonSchemaTypeAnyOfItem';
|
|
|
134
134
|
export * from './datasetId';
|
|
135
135
|
export * from './datasetItem';
|
|
136
136
|
export * from './datasetItemCurrentSubmissionId';
|
|
137
|
-
export * from './datasetItemDuplicateOfSubmissionId';
|
|
138
137
|
export * from './datasetItemId';
|
|
139
138
|
export * from './datasetProfile';
|
|
140
139
|
export * from './datasetProfileStamp';
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@toloka-platform/sdk",
|
|
3
|
-
"version": "0.1.0-preview.
|
|
3
|
+
"version": "0.1.0-preview.8",
|
|
4
4
|
"private": false,
|
|
5
5
|
"description": "Preview. Build, inspect, and operate Toloka pipeline projects as TypeScript source. The public API may change.",
|
|
6
6
|
"license": "Apache-2.0",
|
package/skills/catalog.json
CHANGED
|
@@ -13,8 +13,8 @@
|
|
|
13
13
|
"testing",
|
|
14
14
|
"checks"
|
|
15
15
|
],
|
|
16
|
-
"version": "0.1.0-preview.
|
|
17
|
-
"digest": "
|
|
16
|
+
"version": "0.1.0-preview.8",
|
|
17
|
+
"digest": "87dca7b1cc0cf3d61f54ab27703ea4b60622355ec07e289ce1242bb3c2233482",
|
|
18
18
|
"files": ["SKILL.md", "references/node-semantics.md", "references/workflow-and-cli.md"]
|
|
19
19
|
},
|
|
20
20
|
{
|
|
@@ -22,8 +22,8 @@
|
|
|
22
22
|
"name": "Report a Toloka problem",
|
|
23
23
|
"description": "Report a problem with the Toloka platform, CLI, or SDK as a redacted Markdown issue report sent through the toloka CLI only after explicit approval. Use when the user hits unexpected behavior, an error, or a bug and wants Toloka engineers to investigate.",
|
|
24
24
|
"tags": ["support", "issue", "bug", "report"],
|
|
25
|
-
"version": "0.1.0-preview.
|
|
26
|
-
"digest": "
|
|
25
|
+
"version": "0.1.0-preview.8",
|
|
26
|
+
"digest": "044f8e63faf2a236931a78d3555e6808563199b5f5a06741ca3f2ee9063a71fa",
|
|
27
27
|
"files": ["SKILL.md", "references/report-template.md"]
|
|
28
28
|
}
|
|
29
29
|
]
|
|
@@ -93,6 +93,7 @@ Register code behavior with `test.code(pipeline, node, name, callback)` and exec
|
|
|
93
93
|
that file. `toloka auth-status` reports what is in effect and where it came from; `toloka logout`
|
|
94
94
|
removes it. If a command reports missing credentials, run `toloka login`; never invent a key.
|
|
95
95
|
Anonymous command telemetry is on by default; set `TOLOKA_TELEMETRY=0` or `DO_NOT_TRACK=1` to disable it.
|
|
96
|
+
A globally installed CLI auto-updates by default; turn that off with `toloka config set auto-update false`.
|
|
96
97
|
|
|
97
98
|
## Recover from expected failures
|
|
98
99
|
|
|
@@ -75,7 +75,7 @@ Use a downstream `code` node after `quorum` only when the user asks for a derive
|
|
|
75
75
|
Set `CodeAuthoringSettings`, derived from `CodeNodeSettings`, through `code(name, settings)`:
|
|
76
76
|
|
|
77
77
|
- `about`: node `name`, optional `description`, and optional `projectType`.
|
|
78
|
-
- `
|
|
78
|
+
- `input` and `output`: `{ field, optional }` references to declared `Field` objects.
|
|
79
79
|
- `code.code`: Python source.
|
|
80
80
|
- `envVars`: optional static values or secret bindings:
|
|
81
81
|
- `{ type: 'static', name, value }`
|
|
@@ -137,7 +137,7 @@ Use `about.projectType: 'data_collection'` when experts produce new original con
|
|
|
137
137
|
|
|
138
138
|
#### Entities
|
|
139
139
|
|
|
140
|
-
An authored entity
|
|
140
|
+
An authored entity references a declared `field`. Set `entityType`, `optional`, `jsonPath`, `description`, `relations`, and the discriminant:
|
|
141
141
|
|
|
142
142
|
- `dynamic: false` means one value in one annotation; use a path without `[*]` and omit `maxEntities`.
|
|
143
143
|
- `dynamic: true` means repeated values inside one annotation; use exactly one `[*]` in `jsonPath` and set `maxEntities` from 1 through 50.
|
|
@@ -259,20 +259,20 @@ Use `match` when one item must take one conditional path based on data. Do not u
|
|
|
259
259
|
Set `MatchAuthoringSettings`, derived from `MatchNodeSettings`, through `match(name, settings)`:
|
|
260
260
|
|
|
261
261
|
- `about`: `name`, optional `description`, optional `projectType`.
|
|
262
|
-
- `
|
|
262
|
+
- `input`: every declared `Field` referenced by a predicate, each with `optional`.
|
|
263
263
|
- `conditions`: ordered branches with optional `logic: 'and' | 'or'` and required `predicates`.
|
|
264
264
|
- Each authored predicate uses `field`, `jsonPath`, `operator`, and scalar `value`.
|
|
265
265
|
|
|
266
266
|
The current public type accepts only `operator: 'text_eq' | 'text_neq'` and `value: string | number | boolean | null`. Do not copy broader internal operator lists into source. Use `jsonPath: '$'` for the whole scalar or a restricted path into an object.
|
|
267
267
|
|
|
268
|
-
Ordering is semantic: `conditions[0]` routes to `matched(0)`, the first true condition wins, and an implicit `default` handles no match. Every predicate field must also appear in `
|
|
268
|
+
Ordering is semantic: `conditions[0]` routes to `matched(0)`, the first true condition wins, and an implicit `default` handles no match. Every predicate field must also appear in `input`.
|
|
269
269
|
|
|
270
270
|
### Traps
|
|
271
271
|
|
|
272
272
|
- Do not write a default condition; connect `route.default`.
|
|
273
273
|
- Do not assume all matching branches run.
|
|
274
274
|
- Do not use unsupported comparison, set-membership, range, or emptiness operators. Put that logic in `code`, emit a routing key, then compare it with `text_eq` or `text_neq`.
|
|
275
|
-
- Do not reference a field in a predicate without binding it in `
|
|
275
|
+
- Do not reference a field in a predicate without binding it in `input`.
|
|
276
276
|
|
|
277
277
|
## Public-surface gaps
|
|
278
278
|
|
|
@@ -38,7 +38,7 @@ Select or create the intended platform project before `toloka init` when the che
|
|
|
38
38
|
|
|
39
39
|
`toloka init` checks that `@toloka-platform/sdk` is declared in `dependencies` or `devDependencies`, then maintains one `<toloka>` Agent Bootstrap block in `AGENTS.md` and writes minimal pointers to it: a Claude skill at `.claude/skills/toloka-bootstrap/SKILL.md` when `.claude/` exists (or when no agent configuration exists at all) and an always-applied Cursor rule at `.cursor/rules/toloka-bootstrap.mdc` when `.cursor/rules/` exists. It writes `toloka.json` only from an explicit project flag/environment value or an existing checkout binding.
|
|
40
40
|
|
|
41
|
-
The bootstrap is small: it tells an agent to run `toloka --help`, search the bundled Skill Catalog with `toloka skill search <query>`, install the match with `toloka skill install <skill-id>`, and read `.agents/skills/<skill-id>/SKILL.md`. This skill is that installed copy. Commands warn on stderr when a generated legacy entry point is still present or the recorded bootstrap version differs from the installed SDK. Help, `init`, `report`, `project url`, and version invocations skip this checkout-dependent warning. The warning never blocks a command; re-run `toloka init` to refresh the bootstrap and its pointers.
|
|
41
|
+
The bootstrap is small: it tells an agent to run `toloka --help`, search the bundled Skill Catalog with `toloka skill search <query>`, install the match with `toloka skill install <skill-id>`, and read `.agents/skills/<skill-id>/SKILL.md`. This skill is that installed copy. Commands warn on stderr when a generated legacy entry point is still present or the recorded bootstrap version differs from the installed SDK. Help, `init`, `report`, `project url`, `config`, and version invocations skip this checkout-dependent warning. The warning never blocks a command; re-run `toloka init` to refresh the bootstrap and its pointers.
|
|
42
42
|
|
|
43
43
|
## Store credentials once
|
|
44
44
|
|
|
@@ -64,6 +64,8 @@ toloka logout
|
|
|
64
64
|
unless you pass `--show-token`, and exits non-zero when the platform is unreachable — use it to check a
|
|
65
65
|
setup rather than guessing. `logout` deletes the stored file; it does not revoke the key.
|
|
66
66
|
|
|
67
|
+
A globally installed `toloka` auto-updates by default. It checks npm at most once per day and installs a newer `@toloka-platform/sdk` only after that version has been published for at least 24 hours, then re-runs the original command. `config`, `version`, and `help` never trigger a check. Project-local installs and CI skip it. Turn it off with `toloka config set auto-update false`.
|
|
68
|
+
|
|
67
69
|
## Configure the CLI
|
|
68
70
|
|
|
69
71
|
Provide the API key and base URL on networked workflows through `POTATO_API_KEY` and
|
|
@@ -87,6 +89,9 @@ toloka --version
|
|
|
87
89
|
toloka login
|
|
88
90
|
toloka logout
|
|
89
91
|
toloka auth-status
|
|
92
|
+
toloka config
|
|
93
|
+
toloka config get [name]
|
|
94
|
+
toloka config set <name> <value>
|
|
90
95
|
toloka project list [--limit <n>]
|
|
91
96
|
toloka project add [alias] [projectId]
|
|
92
97
|
toloka project switch <alias>
|
|
@@ -26,7 +26,7 @@ The CLI exits 1 with `Potato API request failed with status 500`. The same comma
|
|
|
26
26
|
|
|
27
27
|
## Environment
|
|
28
28
|
|
|
29
|
-
- toloka CLI 0.1.0-preview.
|
|
29
|
+
- toloka CLI 0.1.0-preview.8 (`toloka --version`), Bun 1.4.0, macOS 15
|
|
30
30
|
- Platform: https://platform.toloka.ai
|
|
31
31
|
|
|
32
32
|
## Redactions
|