bitfab-cli 0.2.486 → 0.2.487
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.js +3 -3
- package/package.json +1 -1
package/dist/index.js
CHANGED
|
@@ -29286,13 +29286,13 @@ var saveTraceAssertions = {
|
|
|
29286
29286
|
approvalState: optionalApprovalState(),
|
|
29287
29287
|
assigneeEmail: external_exports.email().max(MAX_ASSIGNEE_EMAIL_LENGTH).nullish().describe("Email of the organization member who should review this assertion, which routes it into their queue in Bitfab. Assigning decides nothing, so it leaves the approval state alone. The email must belong to a member of this organization, or the call fails naming it. Omit on an edit to preserve the current assignee, and pass null to leave it to nobody.")
|
|
29288
29288
|
})).min(1).max(MAX_ASSERTIONS_PER_REQUEST)).describe(`One entry per assertion (1-${MAX_ASSERTIONS_PER_REQUEST})`),
|
|
29289
|
-
guidedRewrite: external_exports.boolean().optional().describe(
|
|
29289
|
+
guidedRewrite: external_exports.boolean().optional().describe('Check each assertion that carries new text against this trace, and rewrite one that breaks the writing rules (two claims in one, a condition, a value pointed at instead of stated, an id from the recorded run, a word with no clear pass line). Defaults to true. The check runs on Bitfab after the save, usually in under a minute, and this call returns without waiting for it, listing each checked assertion as still being checked. A rewritten assertion replaces what you sent in place. Read it back with get_trace_assertions and approvalStates ["pending"] right after the save, before relying on its wording: that call waits up to 45 seconds for the check, then shows what you sent, what changed, and, when what you sent bundled more than one claim, each other claim as ready-to-send text to save as its own assertion. Pass false only to save text exactly as sent, for example wording a person dictated.')
|
|
29290
29290
|
}
|
|
29291
29291
|
};
|
|
29292
29292
|
var getTraceAssertions = {
|
|
29293
29293
|
name: "get_trace_assertions",
|
|
29294
29294
|
title: "Get Trace Assertions",
|
|
29295
|
-
description: `Read what SHOULD happen when one or more traces are replayed: each trace's assertions, their review state, their pass/fail criteria, people-only human note, who is assigned to review each one, and what part of the evaluated trace each one checks. By default only assertions a person has approved are returned, because only those are checked on a replay; a trace whose assertions are all still awaiting review reads the same as one with none. Pass \`approvalStates\` to read other review states: \`["pending"]\` for the assertions still awaiting review, including the drafts generate_trace_assertions saved, so a batch of them can be reviewed and then decided with save_trace_assertions \`approvalState\`; add \`"rejected"\` to see the ones turned down. Judge a replay only against the approved ones. To find assertions without knowing which traces carry them, such as every draft awaiting review on a function, use list_trace_assertions. Each trace also reports its assertion generation, if one ran: running, completed with how many drafts it saved for review, or failed with the reason. Categorized assertions include category_id and the category's id, title, and description. A human note is returned so it can be read and edited, but it is never assessment evidence: judge only from the assertion, pass/fail criteria, and evaluated trace. Call this before judging a replay so the verdict is measured against what the user actually asked for rather than a guess, and before save_trace_assertions to find the assertion to edit or to avoid creating a duplicate. No span content is loaded, so one call accepts up to ${GET_TRACE_ASSERTIONS_MAX_IDS} ids. Accepts original trace ids and replay trace ids alike: a replay with no assertions of its own reads its original's, and the response says which original they came from. An assertion whose target cannot be found on the trace being evaluated is errored, never passed, so record it with save_agent_verdicts \`skip\` rather than a FAIL.`,
|
|
29295
|
+
description: `Read what SHOULD happen when one or more traces are replayed: each trace's assertions, their review state, their pass/fail criteria, people-only human note, who is assigned to review each one, and what part of the evaluated trace each one checks. By default only assertions a person has approved are returned, because only those are checked on a replay; a trace whose assertions are all still awaiting review reads the same as one with none. Pass \`approvalStates\` to read other review states: \`["pending"]\` for the assertions still awaiting review, including the drafts generate_trace_assertions saved, so a batch of them can be reviewed and then decided with save_trace_assertions \`approvalState\`; add \`"rejected"\` to see the ones turned down. Judge a replay only against the approved ones. To find assertions without knowing which traces carry them, such as every draft awaiting review on a function, use list_trace_assertions. When an assertion it is about to return is still being checked against the writing rules after a save, this call waits up to 45 seconds for the check, then returns the rewritten text and what changed; one still being checked after that is listed as such, so call again for its result. Each trace also reports its assertion generation, if one ran: running, completed with how many drafts it saved for review, or failed with the reason. Categorized assertions include category_id and the category's id, title, and description. A human note is returned so it can be read and edited, but it is never assessment evidence: judge only from the assertion, pass/fail criteria, and evaluated trace. Call this before judging a replay so the verdict is measured against what the user actually asked for rather than a guess, and before save_trace_assertions to find the assertion to edit or to avoid creating a duplicate. No span content is loaded, so one call accepts up to ${GET_TRACE_ASSERTIONS_MAX_IDS} ids. Accepts original trace ids and replay trace ids alike: a replay with no assertions of its own reads its original's, and the response says which original they came from. An assertion whose target cannot be found on the trace being evaluated is errored, never passed, so record it with save_agent_verdicts \`skip\` rather than a FAIL.`,
|
|
29296
29296
|
inputSchema: {
|
|
29297
29297
|
traceIds: external_exports.preprocess(parseJsonString, external_exports.array(external_exports.uuid()).min(1).max(GET_TRACE_ASSERTIONS_MAX_IDS)).describe(`Original trace IDs to read assertions for (1-${GET_TRACE_ASSERTIONS_MAX_IDS})`),
|
|
29298
29298
|
approvalStates: external_exports.preprocess(parseJsonString, external_exports.array(external_exports.enum(["pending", "approved", "rejected"])).min(1).max(3)).optional().describe('Review states to include: "approved" (checked on every replay), "pending" (awaiting review, including generated drafts), "rejected" (turned down, never checked). Defaults to ["approved"], which is what a replay is judged against. Pass ["pending"] to review drafts, or all three to see everything.')
|
|
@@ -29687,7 +29687,7 @@ var generateScenario = {
|
|
|
29687
29687
|
var generateScenarioAssertions = {
|
|
29688
29688
|
name: "generate_scenario_assertions",
|
|
29689
29689
|
title: "Generate Scenario Assertions",
|
|
29690
|
-
description: "Turn a person's label on one scenario, found by its id or by a trace it was built from, into up to four assertions, saved as pending agent assertions for review on the trace the scenario was built from. For a generated scenario, check its trigger, data, and records with get_scenario before calling this. A scenario not generated yet drafts from its trace's top-level input and output instead, and cites the trace's top-level span. The note says what the run got right or wrong; the verdict says whether the run was right overall. Each assertion cites the spans that settle its values, from the scenario's trigger, history, and records the run saw, and is saved on the trace those spans are on; records the run never saw are context only. A note that gives nothing a reviewer could check against a replay yields no assertions. An assertion never concludes that something did not exist from a kind of data the scenario's sources say was not read in full as of the run. A trace with no scenario, a scenario id that does not exist, or a scenario built from a replay is refused. Each drafted assertion then goes through the same check and rewrite as save_trace_assertions, so read the
|
|
29690
|
+
description: "Turn a person's label on one scenario, found by its id or by a trace it was built from, into up to four assertions, saved as pending agent assertions for review on the trace the scenario was built from. For a generated scenario, check its trigger, data, and records with get_scenario before calling this. A scenario not generated yet drafts from its trace's top-level input and output instead, and cites the trace's top-level span. The note says what the run got right or wrong; the verdict says whether the run was right overall. Each assertion cites the spans that settle its values, from the scenario's trigger, history, and records the run saw, and is saved on the trace those spans are on; records the run never saw are context only. A note that gives nothing a reviewer could check against a replay yields no assertions. An assertion never concludes that something did not exist from a kind of data the scenario's sources say was not read in full as of the run. A trace with no scenario, a scenario id that does not exist, or a scenario built from a replay is refused. Each drafted assertion then goes through the same check and rewrite as save_trace_assertions, and this call returns without waiting for it, so read the drafts back with get_trace_assertions for what the rewrite changed, any claim to save as its own assertion, and anything still to fix.",
|
|
29691
29691
|
inputSchema: {
|
|
29692
29692
|
...scenarioTargetSchema,
|
|
29693
29693
|
verdict: external_exports.preprocess(parseJsonString, external_exports.boolean()).optional().describe("true when the person said the run was right, false when wrong. Leave it out when they gave no verdict."),
|
package/package.json
CHANGED