bitfab-cli 0.2.443 → 0.2.445
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/dist/index.js +35 -17
- package/package.json +1 -1
package/dist/index.js
CHANGED
|
@@ -29251,13 +29251,13 @@ var scenarioTargetSchema = {
|
|
|
29251
29251
|
var getScenario = {
|
|
29252
29252
|
name: "get_scenario",
|
|
29253
29253
|
title: "Get Scenario",
|
|
29254
|
-
description: "Read one scenario, found by its id or by a trace it was built from: the situation the trace function's run was in, as the trigger that started the run and the data available when it arrived, with a name and a description. It describes the situation only, never what the run did or should do. Returns the trigger and data summaries, then the name, description, trigger, and data as JSON in exactly the shape save_scenario accepts. Call it before save_scenario so an edit starts from what is saved.",
|
|
29254
|
+
description: "Read one scenario, found by its id or by a trace it was built from: the situation the trace function's run was in, as the trigger that started the run and the data available when it arrived, with a name and a description. It describes the situation only, never what the run did or should do. Returns the trigger and data summaries, then the name, description, trigger, and data as JSON in exactly the shape save_scenario accepts. data.sources, when saved, says where the records came from and, for each kind of data, whether it was read in full, as of the run or as of capture, with columns the capture predates, or not captured at all. Call it before save_scenario so an edit starts from what is saved.",
|
|
29255
29255
|
inputSchema: scenarioTargetSchema
|
|
29256
29256
|
};
|
|
29257
29257
|
var saveScenario = {
|
|
29258
29258
|
name: "save_scenario",
|
|
29259
29259
|
title: "Save Scenario",
|
|
29260
|
-
description: "Create or update one scenario. To create a scenario, or to replace its records, send traceId with records or recordsPath: the records your own system holds that relate to that trace's run, as of the run. Bitfab builds the trigger and data from the trace when the trace has no scenario yet (so far only for traces of Noah's noah-agent-turn function; any other trace with no scenario is refused), then looks for each record's quote, and failing that its text, in the inputs and outputs of the trace's spans. A record found in a span cites that span and can back an assertion; a record found in no span is kept as context the run never saw. A trace that is a replay is refused, because a scenario belongs on the original. records and recordsPath replace the saved records whole, need traceId rather than scenarioId, and cannot be sent with any other
|
|
29260
|
+
description: "Create or update one scenario. To create a scenario, or to replace its records, send traceId with records or recordsPath: the records your own system holds that relate to that trace's run, as of the run. Bitfab builds the trigger and data from the trace when the trace has no scenario yet (so far only for traces of Noah's noah-agent-turn function; any other trace with no scenario is refused), then looks for each record's quote, and failing that its text, in the inputs and outputs of the trace's spans. A record found in a span cites that span and can back an assertion; a record found in no span is kept as context the run never saw. A trace that is a replay is refused, because a scenario belongs on the original. Send sources beside records to say where the records came from and how much of each kind of data was read; recordsPath forwards the file's sources on its own. records and recordsPath replace the saved records and sources whole (records with no sources clear the saved sources), need traceId rather than scenarioId, and cannot be sent with any field other than sources. To edit a scenario that already exists, find it by its id or by a trace it was built from and send name, description, trigger, data, or appendData, for example to add history the run had available, record data it was missing, or correct the trigger. name, description, trigger, and data each replace the saved value whole, and fields left out are kept. To add to the data without resending it, use appendData instead of data: its history entries go after the saved ones and its missing items after the saved missing items, in one step that cannot lose another caller's append made at the same time. Send data or appendData, not both. trigger, data, and appendData entries must keep the shape get_scenario returns. A history entry's source names the span its text came from, on any trace linked to the scenario (a trace that finds it by traceId is one); set it to null or leave it out when the text is not in such a trace, and the entry still counts as data the run had but cannot be cited as evidence when the scenario is turned into assertions. A source naming a span on any other trace is refused. The trigger and data summaries are rebuilt from what is saved. An edit to a trace with no scenario, or to a scenario id that does not exist, is refused. Returns the saved scenario.",
|
|
29261
29261
|
inputSchema: {
|
|
29262
29262
|
...scenarioTargetSchema,
|
|
29263
29263
|
name: external_exports.string().min(1).optional().describe("The scenario's name."),
|
|
@@ -29269,13 +29269,14 @@ var saveScenario = {
|
|
|
29269
29269
|
missing: external_exports.array(external_exports.string()).optional()
|
|
29270
29270
|
})).optional().describe("Data to add to the saved data: history entries (same shape as data.history, source optional, in time order, after the saved ones) and missing items. At least one of the two must be non-empty. Leave data out when sending this."),
|
|
29271
29271
|
records: external_exports.preprocess(parseJsonString, external_exports.array(external_exports.record(external_exports.string(), external_exports.unknown()))).optional().describe(`The records related to this run, each { kind, ref, at, asOf, title, text, quote }. kind is one of "message", "contact", "preference", "memory", "email", "calendar_event", "action_item". ref is your own pointer to the row, such as "Input:234605". at is when the record came to exist, or null. asOf is "run" when the record is as it was at the run, or "now" when no history is kept for it. quote is a short exact string to look for in the run's spans, or null to look for the whole text. Prefer recordsPath, so the records never pass through the conversation. Send records or recordsPath, not both.`),
|
|
29272
|
-
recordsPath: external_exports.string().min(1).optional().describe("Path to a local JSON file holding the records, either as an array or as an object with a records array. The Bitfab plugin reads the file and sends its records, so they never pass through the conversation. Send records or recordsPath, not both.")
|
|
29272
|
+
recordsPath: external_exports.string().min(1).optional().describe("Path to a local JSON file holding the records, either as an array or as an object with a records array. The Bitfab plugin reads the file and sends its records, so they never pass through the conversation. When the object also has frameAt, capturedAt, windows, and coverage, as Noah's scenario builder writes them, the plugin sends those as sources. Send records or recordsPath, not both."),
|
|
29273
|
+
sources: external_exports.preprocess(parseJsonString, external_exports.record(external_exports.string(), external_exports.unknown())).optional().describe(`Where the records came from, as { frameAt, capturedAt, windows, coverage }: when the run started, when its data was captured (or null), how many days each windowed kind reaches (such as { "messagesDays": 3, "calendarDays": 7 }), and coverage, which gives each kind of data its capture status ("rewound-to-frame" for as of the run, "current-at-capture" for as of capture, "not-recorded" or "unavailable" for not captured) plus filledColumns { absentNullable, defaultedNonNullable }, the "table.column" names the capture predates. Only with records, never with recordsPath, which sends the file's own.`)
|
|
29273
29274
|
}
|
|
29274
29275
|
};
|
|
29275
29276
|
var generateScenarioAssertions = {
|
|
29276
29277
|
name: "generate_scenario_assertions",
|
|
29277
29278
|
title: "Generate Scenario Assertions",
|
|
29278
|
-
description: "Turn a person's label on one scenario, found by its id or by a trace it was built from, into up to four assertions, saved as pending agent assertions for review on the trace the scenario was built from. Create the scenario first with save_scenario and check its trigger, data, and records before calling this. The note says what the run got right or wrong; the verdict says whether the run was right overall. Each assertion cites the spans that settle its values, from the scenario's trigger, history, and records the run saw, and is saved on the trace those spans are on; records the run never saw are context only. A note that gives nothing a reviewer could check against a replay yields no assertions. A trace with no scenario, a scenario id that does not exist, or a scenario built from a replay is refused. Returns the saved assertions.",
|
|
29279
|
+
description: "Turn a person's label on one scenario, found by its id or by a trace it was built from, into up to four assertions, saved as pending agent assertions for review on the trace the scenario was built from. Create the scenario first with save_scenario and check its trigger, data, and records before calling this. The note says what the run got right or wrong; the verdict says whether the run was right overall. Each assertion cites the spans that settle its values, from the scenario's trigger, history, and records the run saw, and is saved on the trace those spans are on; records the run never saw are context only. A note that gives nothing a reviewer could check against a replay yields no assertions. An assertion never concludes that something did not exist from a kind of data the scenario's sources say was not read in full as of the run. A trace with no scenario, a scenario id that does not exist, or a scenario built from a replay is refused. Returns the saved assertions.",
|
|
29279
29280
|
inputSchema: {
|
|
29280
29281
|
...scenarioTargetSchema,
|
|
29281
29282
|
verdict: external_exports.preprocess(parseJsonString, external_exports.boolean()).optional().describe("true when the person said the run was right, false when wrong. Leave it out when they gave no verdict."),
|
|
@@ -29735,14 +29736,28 @@ function withLegacyExperimentKeys(args) {
|
|
|
29735
29736
|
...args.experimentIds === void 0 ? {} : { testRunIds: args.experimentIds }
|
|
29736
29737
|
};
|
|
29737
29738
|
}
|
|
29739
|
+
var SOURCE_FIELDS = ["frameAt", "capturedAt", "windows", "coverage"];
|
|
29740
|
+
function isObjectWithRecords(parsed) {
|
|
29741
|
+
return typeof parsed === "object" && parsed !== null && "records" in parsed && Array.isArray(parsed.records);
|
|
29742
|
+
}
|
|
29743
|
+
function sourcesInFile(parsed) {
|
|
29744
|
+
if (!("coverage" in parsed)) {
|
|
29745
|
+
return void 0;
|
|
29746
|
+
}
|
|
29747
|
+
return Object.fromEntries(SOURCE_FIELDS.filter((field) => field in parsed).map((field) => [
|
|
29748
|
+
field,
|
|
29749
|
+
parsed[field]
|
|
29750
|
+
]));
|
|
29751
|
+
}
|
|
29738
29752
|
function recordsInFile(parsed) {
|
|
29739
29753
|
if (Array.isArray(parsed)) {
|
|
29740
|
-
return parsed;
|
|
29754
|
+
return { records: parsed };
|
|
29741
29755
|
}
|
|
29742
|
-
if (
|
|
29743
|
-
return
|
|
29756
|
+
if (!isObjectWithRecords(parsed)) {
|
|
29757
|
+
return null;
|
|
29744
29758
|
}
|
|
29745
|
-
|
|
29759
|
+
const sources = sourcesInFile(parsed);
|
|
29760
|
+
return sources === void 0 ? { records: parsed.records } : { records: parsed.records, sources };
|
|
29746
29761
|
}
|
|
29747
29762
|
async function readRecordsFile(recordsPath) {
|
|
29748
29763
|
const fullPath = resolve(recordsPath);
|
|
@@ -29760,10 +29775,10 @@ async function readRecordsFile(recordsPath) {
|
|
|
29760
29775
|
} catch {
|
|
29761
29776
|
return { error: `recordsPath ${fullPath} is not JSON.` };
|
|
29762
29777
|
}
|
|
29763
|
-
const
|
|
29764
|
-
return
|
|
29778
|
+
const file2 = recordsInFile(parsed);
|
|
29779
|
+
return file2 === null ? {
|
|
29765
29780
|
error: `recordsPath ${fullPath} holds neither an array of records nor an object with a records array.`
|
|
29766
|
-
} :
|
|
29781
|
+
} : file2;
|
|
29767
29782
|
}
|
|
29768
29783
|
function createBitfabToolHandlers(platform2, getConfig2, pluginVersion) {
|
|
29769
29784
|
const proxy = new McpProxy(platform2, pluginVersion);
|
|
@@ -29815,11 +29830,14 @@ function createBitfabToolHandlers(platform2, getConfig2, pluginVersion) {
|
|
|
29815
29830
|
if (rest.records !== void 0) {
|
|
29816
29831
|
return errorResult("Error saving scenario: send records or recordsPath, not both.");
|
|
29817
29832
|
}
|
|
29833
|
+
if (rest.sources !== void 0) {
|
|
29834
|
+
return errorResult("Error saving scenario: recordsPath sends the file's own sources, so send sources only with records.");
|
|
29835
|
+
}
|
|
29818
29836
|
const read = await readRecordsFile(recordsPath);
|
|
29819
29837
|
if ("error" in read) {
|
|
29820
29838
|
return errorResult(`Error saving scenario: ${read.error}`);
|
|
29821
29839
|
}
|
|
29822
|
-
return proxyToolCall("save_scenario", { ...rest,
|
|
29840
|
+
return proxyToolCall("save_scenario", { ...rest, ...read });
|
|
29823
29841
|
}
|
|
29824
29842
|
const specialHandlers = {
|
|
29825
29843
|
[saveAgentLabels.name]: handleSaveAgentLabels,
|
|
@@ -29992,10 +30010,10 @@ var semver3 = __toESM(require_semver2(), 1);
|
|
|
29992
30010
|
|
|
29993
30011
|
// ../bitfab-plugin-lib/dist/bakedSdkVersions.js
|
|
29994
30012
|
var BAKED_SDK_VERSIONS = {
|
|
29995
|
-
typescript: "0.64.
|
|
29996
|
-
python: "0.64.
|
|
29997
|
-
ruby: "0.64.
|
|
29998
|
-
go: "0.64.
|
|
30013
|
+
typescript: "0.64.23",
|
|
30014
|
+
python: "0.64.21",
|
|
30015
|
+
ruby: "0.64.19",
|
|
30016
|
+
go: "0.64.20"
|
|
29999
30017
|
};
|
|
30000
30018
|
|
|
30001
30019
|
// ../bitfab-plugin-lib/dist/installedSdk.js
|
|
@@ -36764,7 +36782,7 @@ State who can start a replay: once GitHub registers the workflow, anyone with wr
|
|
|
36764
36782
|
|
|
36765
36783
|
Edit the workflow directly for anything detection missed, such as a build step, code generation, a monorepo install filter, or a service, and pin any added action to a commit SHA. When the user normally starts replay through a wrapper that sets environment variables, add them to the replay step's env. Every variable read from a secret stays in that env as a GitHub expression reading secrets.<name>, which is how --cloud-secrets and the runner's empty-secret check find them; a secret stored under another name is mapped directly, such as OPENAI_API_KEY reading secrets.EVAL_OPENAI_API_KEY, and a non-secret value reads vars.<name>. When a setup step needs a credential file, write it under $RUNNER_TEMP with umask 077 and export its path through $GITHUB_ENV; do not use google-github-actions/auth or similar actions that write the key into the checked-out workspace. Do not copy deployment steps or secret values into the workflow.
|
|
36766
36784
|
|
|
36767
|
-
For BITFAB_API_KEY, have the user create a dedicated key for the runner on the Bitfab dashboard's API keys page, in the organization that holds the registry's traces, named for what it is (such as "GitHub cloud replay"), and set it by hand. Each replay started with bitfab-replay --cloud
|
|
36785
|
+
For BITFAB_API_KEY, have the user create a dedicated key for the runner on the Bitfab dashboard's API keys page, in the organization that holds the registry's traces, named for what it is (such as "GitHub cloud replay"), and set it by hand. Each replay started with bitfab-replay --cloud runs as whoever started it, through a one-time token in the key_id input that only a run of that workflow, on that replay's branch, in that repository can trade for a short-lived key, which it proves with GitHub's OIDC token for the run. This secret is the key a run falls back to when no token can be passed, such as a dispatch from another workflow. Keep the key_id input and the id-token: write permission --cloud-init writes. When a workflow already exists, run --cloud-update (the same command as --cloud-init) anyway: it adds a missing key_id input and id-token: write permission, removes the BITFAB_REPLAY_BITFAB_API_KEY line older SDKs wrote (it made GitHub hand the runner every secret in the repository), leaves the rest alone, and lists anything it could not place for you to add by hand. Its result also lists in oldKeySecrets any BITFAB_REPLAY_KEY_* secrets an older SDK left in the repository, each a copy of the Bitfab API key of whoever started that replay. Name them to the user and ask once whether to delete them; on yes, run --cloud-update --remove-old-key-secrets, which returns removedOldKeySecrets and lists any it could not delete with the page to delete them on. For the other secrets, when local environment files define them, ask once whether to copy them, then run bitfab-replay --cloud-secrets --env-file <file> --dry-run, then without --dry-run. It pipes values to gh on standard input, creates a missing Environment, and without the GitHub CLI returns createByHand with the page to create them on; relay that list instead of treating it as a failure. Never hand-extract a value with grep, cut, or a shell variable. Warn before copying a value a runner cannot use, such as a localhost URL. Setting secrets needs the same explicit authorization as any other remote write.
|
|
36768
36786
|
|
|
36769
36787
|
Each replay pushes a temporary bitfab-replay/<UUID> branch. For each push-triggered workflow or deployment in reviewPushTriggers, offer to exclude those branches: branches-ignore: [bitfab-replay/**] on a push trigger (or a last !bitfab-replay/** entry when it already lists branches), and git.deploymentEnabled {"bitfab-replay/**": false} in vercel.json. Ask before changing existing CI or deployments.`,
|
|
36770
36788
|
next: "cloud/verify"
|
package/package.json
CHANGED