@hue-run/sdk 0.8.1 → 0.10.0

package/README.md

@@ -35,7 +35,7 @@ import { createHue, HueExportError } from "@hue-run/sdk";
 const hue = createHue({
   apiKey: process.env.HUE_API_KEY!, // a project service key, on the server only
   serviceName: "my-agent",
-  captureContent: false, // required: explicitly choose true or false
+  captureContent: true, // required: true records content; false sends metadata only
   onExportIssue: (issue) => console.error(issue), // sanitized counts, never server bodies
 });
 
@@ -58,6 +58,11 @@ try {
 await hue.shutdown(); // flushes and releases providers owned by this client
 ```
 
+`captureContent` is required. The example sets `true`, the recommended choice: trace inspection,
+evaluations and judges read the prompts, responses and tool inputs/outputs it records. Choose
+`false` for metadata only when a policy forbids sending that content; see
+[Privacy and content](#privacy-and-content).
+
 The default destination is `https://app.hue.run`. Data goes to
 `/api/v1/otlp/v1/traces` and `/api/v1/otlp/v1/logs` with a Bearer project key.
 Set `baseUrl` only for another Hue deployment. It must be an origin without an API path; a trailing slash is accepted.
@@ -287,6 +292,12 @@ outputs and exports through Hue's attached processors keeps the label.
 
 ## Privacy and content
 
+Set `captureContent: true` for full traces: prompts and messages, responses and tool inputs/outputs,
+alongside model, usage, timing and errors. Trace inspection, evaluations and judges read that
+content. Choose `false`, metadata-only mode, when your users decline or an application or data
+policy forbids sending that content to another service. `redact` and the credential filtering
+described below apply in both modes.
+
 `captureContent: false` disables manual input/output/messages/tool content and
 removes recognized GenAI, Vercel, OpenInference and OpenLLMetry content attributes,
 legacy GenAI content events, log bodies, status messages and exception text before
@@ -306,12 +317,18 @@ an OpenAI hosted MCP tool. Before export, and before `redact`, Hue replaces the
 credential-like fields including `authorization`, `authorization_token`, `headers`, `api_key`,
 `access_token`, `x-api-key`, and keys ending in `token`, `secret`, `password`, `apikey` or
 `credential` (case-insensitively, ignoring `-` and `_`) with `"[redacted]"` in recorded tool definitions
-(`gen_ai.tool.definitions`, `ai.prompt.tools`, `llm.tools.*.tool.json_schema`) and in the `tools`
-and `mcp_servers` entries of a raw provider request or response recorded as `input.value`,
-`output.value` or `llm.invocation_parameters`. Parameters named in a JSON Schema `properties`
+(`gen_ai.tool.definitions`, `ai.prompt.tools`, `llm.tools.*.tool.json_schema`) and in the
+top-level `tools` and `mcp_servers` entries of any JSON object recorded as `input.value`,
+`output.value` or `llm.invocation_parameters`, whether or not it is a provider request; nothing
+else in that object changes. Parameters named in a JSON Schema `properties`
 object keep their schemas, so a tool that takes a `headers` argument is still described. A
 definition nested more than 256 levels deep rejects its record. Sensitive `default`, `const`,
-`examples` and `enum` values under credential-named schema parameters are redacted too.
+`examples` and `enum` values under credential-named schema parameters are redacted too. A `url`
+or `server_url` field (named like the credential fields, so `serverUrl` too) that has userinfo, a
+query or a fragment loses its userinfo and fragment, and each query value becomes `[redacted]`
+under its original name; the result is serialized as WHATWG `URL` does (lowercase scheme and
+host, IDN hosts in Punycode, default ports dropped, dot segments resolved). A URL with none of
+them is exported as recorded, and a value `URL` cannot parse becomes `"[redacted]"`.
 
 With `captureContent: false`, export removes tool definitions but keeps a summary of them on the
 same record: `hue.tool.names` lists each definition's `name` (Chat Completions `function.name`,
@@ -324,12 +341,18 @@ rotates. Only definitions another integration recorded can be summarized: `hueTe
 inputs and exports through Hue's attached processors gets the summary.
 Recorded messages can inline files: GenAI `blob` parts in `gen_ai.input.messages` /
 `gen_ai.output.messages` (what the AI SDK 7 adapter records for a file part) and AI SDK 6 `file`
-parts in `ai.prompt.messages`. A span whose messages exceed 256 KiB would be rejected, so before
-export Hue replaces the `content`/`data` of any such part longer than 64 KiB with the file's
-`sha256` (of the decoded bytes for base64 and `data:` URLs, of the UTF-8 text otherwise) and
-`size`, keeping the part's other fields such as `type`, `mime_type` and `mediaType`. Smaller inline
+parts in `ai.prompt.messages`. A span whose messages exceed 256 KiB would be rejected, so when a
+record is queued Hue replaces the `content`/`data` of any such part longer than 64 KiB with the
+file's `sha256` and `size`, both of the file's own bytes whatever its media type (base64 content
+and `;base64` `data:` URLs decoded, other `data:` URLs percent-decoded, anything else as UTF-8 text), keeping the part's other fields such as `type`, `mime_type` and `mediaType`. Smaller inline
 files are exported as recorded. The digest matches `hue.recordFile`'s `hue.file.sha256` for the same
-bytes, so a file can be recognized wherever it appears.
+bytes, so a file can be recognized wherever it appears. The replacement happens before the record is
+charged to the queue budget, so a large file does not drop its span. It is bounded: a message
+attribute longer than 8 MiB of text is left unchanged, and the default 8 MiB `maxQueueBytes` then
+drops the span. Base64 grows a file by a third, so that ceiling is an inline file of about 6 MiB.
+The same message text is hashed once per record, and a record inspects at most 16 MiB of message
+text in all; messages past that are charged as recorded. With `captureContent: false` the messages,
+which export removes, are neither hashed nor charged.
 
 Manual helpers encode JSON values without converting null into absence. Unknown
 outputs and usage remain absent. This SDK does not estimate tokens or cost. A thrown
@@ -361,7 +384,7 @@ import { createHue, createHueTransport } from "@hue-run/sdk";
 const transport = createHueTransport({
   apiKey: process.env.HUE_API_KEY!,
   serviceName: "existing-app",
-  captureContent: false,
+  captureContent: true, // false sends metadata only
 });
 const tracerProvider = new TracerProvider({ spanProcessors: [transport.spanProcessor] });
 const loggerProvider = new LoggerProvider({ processors: [transport.logRecordProcessor] });
@@ -576,9 +599,9 @@ an uncertain preparation. See the
 an issued facade endpoint or the official Gmail service, and do not claim universal provider
 parity.
 
-An unreleased `directTarget` callback extends the same worker to cases without a world: the runner
-verifies the case's pinned input files, hands them to the agent, and uploads the documents it
-returns as verified Hue artifacts for scoring. See
+The `directTarget` callback, shipped in TypeScript `0.5.0`, extends the same worker to cases
+without a world: the runner verifies the case's pinned input files, hands them to the agent, and
+uploads the documents it returns as verified Hue artifacts for scoring. See
 [direct cases and files](EVALUATIONS.md#direct-cases-and-files).
 
 Scorer deferral shipped in TypeScript `0.3.1`. Only built-ins
@@ -591,17 +614,23 @@ their agents with `runLocalAgent()`; setup does not register workers or launch s
 
 ### Command-line evaluation
 
-The unreleased `hue eval` command wraps `runSimulation()` and `runLocalAgent()` for an adapter
-file or a shell command: `hue eval --case "<name>" ./hue-agent.ts` creates a fresh run
-from a published case's immutable pins, runs the agent in one isolated world per case, waits
+The `hue eval` command, shipped in TypeScript `0.5.0` (`--case` since `0.6.0`, `--scenario`
+before), wraps `runSimulation()` and `runLocalAgent()` for an adapter file or a shell command:
+`hue eval --case "<name>" ./hue-agent.ts --content` creates a fresh run from a published case's
+immutable pins, runs the agent in one isolated world per case, waits
 for Hue's outcome checks and prints the run URL and per-case PASS/FAIL verdicts with an exit code;
 `--worker` registers the same adapter for runs launched from Hue. It needs a Read and
-write key in `HUE_API_KEY` (never printed) and keeps content capture off unless `--content`
-is passed. See [Evaluate an agent against a case](CLI.md#evaluate-an-agent-against-a-case).
+write key in `HUE_API_KEY` (never printed). Content capture stays off unless `--content` is
+passed; the example passes it so case spans carry content. Case outputs, error messages and
+explanations are stored in Hue either way (`--no-output` keeps them out of a one-shot run's
+results); a command's stdout is its stored answer, with the credentials `hue eval` handed the case
+replaced by `[redacted]`.
+Instrument the agent itself for its model and tool spans. See
+[Evaluate an agent against a case](CLI.md#evaluate-an-agent-against-a-case).
 Eval sets whose cases pin files instead of a world run as direct cases through `runExperiment()`:
-`hue eval --set <slug> --scorer <slug> --command "…"` hands the agent each case's pinned files in a
-private directory, uploads the documents it writes and waits for Hue's grading executor to score
-them. See [Evaluate a document eval set](CLI.md#evaluate-a-document-eval-set).
+`hue eval --set <slug> --scorer <slug> --command "…" --content` hands the agent each case's
+pinned files in a private directory, uploads the documents it writes and waits for Hue's grading
+executor to score them. See [Evaluate a document eval set](CLI.md#evaluate-a-document-eval-set).
 
 ## Managed targets
 

package/dist/cli/eval-direct.d.ts

@@ -38,10 +38,21 @@ export declare function stageDirectCase(root: string, input: {
     executionId: string;
     files: LocalFile[];
 }): Promise<DirectCaseLayout>;
+/** Test seams: `beforeEntry` runs before each listed entry is examined, `afterListing` between
+ * the listing and the reads. */
+export interface CollectionSeams {
+    beforeEntry?: (name: string) => Promise<void>;
+    afterListing?: () => Promise<void>;
+}
 /**
  * Turns the output folder into the target's result. Every regular file, in subdirectories too,
  * becomes a generated file; an unsupported extension is the agent's error rather than a silently
  * dropped document. `fallbackOutput` (for example the command's stdout) is used when no JSON
  * output or summary file was written.
+ *
+ * Nothing here trusts a path the agent could have changed: the output directory must be a real
+ * directory, every file is opened without following a final symlink or blocking on a FIFO, must
+ * be the regular file the listing saw and within its size limit, and the result carries the bytes
+ * read from that open file. `seams` are for tests only.
  */
-export declare function collectDirectOutputs(outputDirectory: string, fallbackOutput?: JsonValue): Promise<TargetResult>;
+export declare function collectDirectOutputs(outputDirectory: string, fallbackOutput?: JsonValue, seams?: CollectionSeams): Promise<TargetResult>;

package/dist/cli/eval-direct.js

@@ -1,6 +1,7 @@
-import { copyFile, mkdir, readdir, readFile, stat, writeFile } from "node:fs/promises";
+import { constants } from "node:fs";
+import { copyFile, lstat, mkdir, opendir, writeFile } from "node:fs/promises";
 import { basename, extname, join } from "node:path";
-import { outputContentTypes } from "../evals/files.js";
+import { outputContentTypes, outputFileLimits, readAgentFile, safeFilename, } from "../evals/files.js";
 import { withFiles } from "../evals/types.js";
 /**
  * Direct (file) cases for `hue eval`: a case is a task plus pinned input files, and the agent's
@@ -43,6 +44,11 @@ for (const type of Object.values(outputExtensions))
         throw new Error(`outputExtensions maps to a content type Hue does not accept: ${type}`);
 const HELPER_FILES = new Set(["manifest.json", "result.json", "output.json"]);
 const SUMMARY_FILES = ["summary.txt", "summary.md", "final.txt", "resumen.txt"];
+/** The most a helper (`manifest.json`, `result.json`, `summary.txt`, …) may hold: the same bound
+ * as the command's stdout. */
+const HELPER_BYTES = 4 * 1024 * 1024;
+/** Entries a collection visits at most, hidden and skipped ones included. */
+const MAX_ENTRIES = 1024;
 /** Writes the case directory: inputs, descriptor, verified file copies and an empty output folder. */
 export async function stageDirectCase(root, input) {
     const caseDirectory = join(root, "case");
@@ -50,21 +56,25 @@ export async function stageDirectCase(root, input) {
     await mkdir(outputDirectory, { recursive: true, mode: 0o700 });
     const inputsPath = join(caseDirectory, "inputs.json");
     await writeFile(inputsPath, JSON.stringify(input.inputs, null, 2), { mode: 0o600 });
+    // Names that differ only in case or Unicode normalization are the same file on macOS and
+    // Windows filesystems; keep both inputs under distinct names, and never overwrite one.
     const used = new Set();
+    const key = (folder, name) => `${folder}/${name.normalize("NFC").toLowerCase()}`;
     const files = [];
     for (const file of input.files) {
         const folder = join(caseDirectory, "files", file.role);
         await mkdir(folder, { recursive: true, mode: 0o700 });
         // Two inputs may share a filename (two incident reports, say); keep both.
-        const name = basename(file.filename);
+        const name = safeFilename(basename(file.filename));
         const extension = extname(name);
         const stem = name.slice(0, name.length - extension.length);
-        let target = join(folder, name);
-        for (let copy = 2; used.has(target); copy++)
-            target = join(folder, `${stem} (${copy})${extension}`);
-        used.add(target);
-        await copyFile(file.path, target);
-        files.push({ role: file.role, filename: basename(target), path: target });
+        let target = name;
+        for (let copy = 2; used.has(key(folder, target)); copy++)
+            target = `${stem} (${copy})${extension}`;
+        used.add(key(folder, target));
+        const path = join(folder, target);
+        await copyFile(file.path, path, constants.COPYFILE_EXCL);
+        files.push({ role: file.role, filename: target, path });
     }
     await writeFile(join(caseDirectory, "case.json"), JSON.stringify({
         id: input.item.id,
@@ -76,33 +86,119 @@ export async function stageDirectCase(root, input) {
     }, null, 2), { mode: 0o600 });
     return { caseDirectory, inputsPath, outputDirectory, files };
 }
-async function readJsonFile(path) {
+const identityOf = (info) => ({
+    dev: info.dev,
+    ino: info.ino,
+});
+const sameIdentity = (a, b) => a.dev === b.dev && a.ino === b.ino;
+/** A real directory (not a symlink) and its identity, or undefined when `path` is absent. A
+ * parent that is no longer a directory (ENOTDIR) is a change, never an absent directory: a case
+ * directory replaced by a file must not read as "wrote nothing". */
+async function directoryIdentity(path, label) {
+    let info;
     try {
-        return JSON.parse(await readFile(path, "utf8"));
+        info = await lstat(path);
     }
     catch (error) {
         if (error.code === "ENOENT")
             return undefined;
-        throw new Error(`${basename(path)} in the output directory is not valid JSON`);
+        throw unexamined(label, error);
     }
+    if (info.isSymbolicLink() || !info.isDirectory())
+        throw new Error(`${label} is not a directory; a symbolic link or file is never collected`);
+    return identityOf(info);
 }
-/** Regular files anywhere under the output directory, named by their `/`-joined relative path.
- * Hidden and lock entries (dot names, `~$…`) and symlinks are skipped at every level. */
-async function listOutputFiles(directory, prefix = "") {
-    const entries = await readdir(directory, { withFileTypes: true }).catch(() => []);
+/** The error for an entry that vanished or changed kind while it was being collected, or could
+ * not be examined at all. */
+function unexamined(label, error) {
+    const code = error.code;
+    return new Error(code === "ENOENT" || code === "ENOTDIR" || code === "ELOOP"
+        ? `${label} changed while it was collected`
+        : `${label} could not be read`);
+}
+/**
+ * Regular files anywhere under the output directory, named by their `/`-joined relative path,
+ * with the device and inode each had when listed. Hidden and lock entries (dot names, `~$…`),
+ * symlinks and anything that is neither a regular file nor a directory are skipped at every
+ * level, except that a top-level helper name must be a regular file when present. The walk stops
+ * past 32 documents or 1024 entries, and an entry that vanished or a directory that changed while
+ * it was listed is refused.
+ */
+async function listOutputFiles(root, rootIdentity, seams) {
     const found = [];
-    for (const entry of entries) {
-        if (entry.name.startsWith(".") || entry.name.startsWith("~$"))
-            continue;
-        const path = join(directory, entry.name);
-        const name = `${prefix}${entry.name}`;
-        if (entry.isDirectory())
-            found.push(...(await listOutputFiles(path, `${name}/`)));
-        else if (entry.isFile())
-            found.push({ name, path });
+    const directories = [
+        { path: root, label: "output/", identity: rootIdentity },
+    ];
+    let visited = 0;
+    let documents = 0;
+    for (let index = 0; index < directories.length; index++) {
+        const directory = directories[index];
+        const prefix = index === 0 ? "" : `${directory.label.slice("output/".length)}`;
+        let entries;
+        try {
+            entries = await opendir(directory.path);
+        }
+        catch (error) {
+            throw unexamined(directory.label, error);
+        }
+        for await (const entry of entries) {
+            if (++visited > MAX_ENTRIES)
+                throw new Error(`The output directory holds more than ${MAX_ENTRIES} entries`);
+            const name = `${prefix}${entry.name}`;
+            const helper = index === 0 && (HELPER_FILES.has(name) || SUMMARY_FILES.includes(name));
+            if (!helper && (entry.name.startsWith(".") || entry.name.startsWith("~$")))
+                continue;
+            const path = join(directory.path, entry.name);
+            if (helper && !entry.isFile())
+                throw new Error(`output/${name} is not a regular file; the agent must write it itself`);
+            if (entry.isDirectory() || entry.isFile())
+                await seams.beforeEntry?.(name);
+            if (entry.isDirectory()) {
+                const identity = await directoryIdentity(path, `output/${name}`);
+                if (!identity)
+                    throw new Error(`output/${name} changed while it was collected`);
+                directories.push({ path, label: `output/${name}/`, identity });
+            }
+            else if (entry.isFile()) {
+                let info;
+                try {
+                    info = await lstat(path);
+                }
+                catch (error) {
+                    throw unexamined(`output/${name}`, error);
+                }
+                if (!info.isFile())
+                    throw new Error(`output/${name} changed while it was collected`);
+                found.push({ name, path, identity: identityOf(info) });
+                if (!helper && ++documents > outputFileLimits.count)
+                    throw new Error(`The agent wrote more than ${outputFileLimits.count} files`);
+            }
+        }
+    }
+    // Every directory must still be the one listed: none was swapped for a link meanwhile.
+    for (const directory of directories) {
+        const now = await directoryIdentity(directory.path, directory.label);
+        if (!now || !sameIdentity(now, directory.identity))
+            throw new Error(`${directory.label} changed while it was collected`);
     }
     return found;
 }
+async function readHelper(listed) {
+    if (!listed)
+        return undefined;
+    const bytes = await readAgentFile(listed.path, `output/${listed.name}`, HELPER_BYTES, listed.identity);
+    return Buffer.from(bytes).toString("utf8");
+}
+function parseHelper(listed, text) {
+    if (text === undefined)
+        return undefined;
+    try {
+        return JSON.parse(text);
+    }
+    catch {
+        throw new Error(`${listed.name} in the output directory is not valid JSON`);
+    }
+}
 /** Artifact filenames cannot contain path separators or controls. Encode only those forbidden
  * characters plus `%`, then represent `/` as `%2F`; distinct output paths stay distinct. */
 function artifactFilename(relativePath) {
@@ -120,9 +216,18 @@ function artifactFilename(relativePath) {
  * becomes a generated file; an unsupported extension is the agent's error rather than a silently
  * dropped document. `fallbackOutput` (for example the command's stdout) is used when no JSON
  * output or summary file was written.
+ *
+ * Nothing here trusts a path the agent could have changed: the output directory must be a real
+ * directory, every file is opened without following a final symlink or blocking on a FIFO, must
+ * be the regular file the listing saw and within its size limit, and the result carries the bytes
+ * read from that open file. `seams` are for tests only.
  */
-export async function collectDirectOutputs(outputDirectory, fallbackOutput) {
-    const manifest = await readJsonFile(join(outputDirectory, "manifest.json"));
+export async function collectDirectOutputs(outputDirectory, fallbackOutput, seams = {}) {
+    const rootIdentity = await directoryIdentity(outputDirectory, "The output directory");
+    const listed = rootIdentity ? await listOutputFiles(outputDirectory, rootIdentity, seams) : [];
+    await seams.afterListing?.();
+    const top = (name) => listed.find((entry) => entry.name === name);
+    const manifest = parseHelper(top("manifest.json"), await readHelper(top("manifest.json")));
     const declaredPrimary = manifest && typeof manifest === "object" && !Array.isArray(manifest)
         ? manifest.primary
         : undefined;
@@ -130,11 +235,11 @@ export async function collectDirectOutputs(outputDirectory, fallbackOutput) {
         ? manifest.output
         : undefined;
     let output = declaredOutput ??
-        (await readJsonFile(join(outputDirectory, "result.json"))) ??
-        (await readJsonFile(join(outputDirectory, "output.json")));
+        parseHelper(top("result.json"), await readHelper(top("result.json"))) ??
+        parseHelper(top("output.json"), await readHelper(top("output.json")));
     if (output === undefined)
         for (const name of SUMMARY_FILES) {
-            const text = await readFile(join(outputDirectory, name), "utf8").catch(() => undefined);
+            const text = await readHelper(top(name));
             if (text !== undefined) {
                 output = { summary: text };
                 break;
@@ -143,7 +248,7 @@ export async function collectDirectOutputs(outputDirectory, fallbackOutput) {
     if (output === undefined)
         output = fallbackOutput;
     // Helper and summary names are reserved at the top level only; a nested one is a document.
-    const entries = (await listOutputFiles(outputDirectory)).filter((entry) => !HELPER_FILES.has(entry.name) && !SUMMARY_FILES.includes(entry.name));
+    const entries = listed.filter((entry) => !HELPER_FILES.has(entry.name) && !SUMMARY_FILES.includes(entry.name));
     // Directory order differs between filesystems; keep uploads stable.
     entries.sort((a, b) => (a.name < b.name ? -1 : a.name > b.name ? 1 : 0));
     const unsupported = entries.filter((entry) => !outputExtensions[extname(entry.name).toLowerCase()]);
@@ -151,10 +256,11 @@ export async function collectDirectOutputs(outputDirectory, fallbackOutput) {
         throw new Error(`The agent wrote files Hue does not accept as generated documents: ${unsupported.map((entry) => entry.name).join(", ")} (accepted: ${Object.keys(outputExtensions).join(" ")})`);
     const files = [];
     for (const entry of entries) {
-        if ((await stat(entry.path)).size === 0)
+        const bytes = (await readAgentFile(entry.path, `Generated file ${entry.name}`, outputFileLimits.bytes, entry.identity));
+        if (!bytes.byteLength)
             throw new Error(`The agent wrote an empty file: ${entry.name}`);
         files.push({
-            path: entry.path,
+            bytes,
             filename: artifactFilename(entry.name),
             contentType: outputExtensions[extname(entry.name).toLowerCase()],
         });
@@ -167,5 +273,11 @@ export async function collectDirectOutputs(outputDirectory, fallbackOutput) {
     }
     else if (files.length === 1)
         files[0].primary = true;
+    // The directory must still be the one listed when every byte has been read.
+    if (rootIdentity) {
+        const now = await directoryIdentity(outputDirectory, "The output directory");
+        if (!now || !sameIdentity(now, rootIdentity))
+            throw new Error("The output directory changed while it was collected");
+    }
     return withFiles(output, files);
 }

package/dist/cli/eval.d.ts

@@ -1,9 +1,10 @@
 import type { ExperimentCase, LocalFile } from "../evals/types.js";
 import { TargetResult } from "../evals/types.js";
 import { type SimulationTargetContext } from "../evals/simulation.js";
+import { type ExperimentVerdicts } from "../evals/verdicts.js";
 import type { JsonValue } from "../types.js";
 /** Adapter contract: the module's `default` or `runMyAgent` export. */
-export type EvalAdapter = (inputs: JsonValue, context: SimulationTargetContext) => JsonValue | undefined | Promise<JsonValue | undefined>;
+export type EvalAdapter = (inputs: JsonValue, context: SimulationTargetContext) => JsonValue | TargetResult | undefined | Promise<JsonValue | TargetResult | undefined>;
 /**
  * Context of a direct (file) case: pinned input files in, generated documents out. The same
  * adapter module serves both modes; `mode` tells it which context it received.
@@ -20,6 +21,32 @@ export interface DirectTargetContext {
     signal?: AbortSignal;
 }
 export type DirectEvalAdapter = (inputs: JsonValue, context: DirectTargetContext) => JsonValue | TargetResult | undefined | Promise<JsonValue | TargetResult | undefined>;
+export interface Output {
+    /** Human progress lines: stdout normally, stderr with --json. */
+    log(line: string): void;
+    /** Diagnostics; always stderr. */
+    error(line: string): void;
+}
+/**
+ * Spawns the agent command once in its own process group and returns its trimmed stdout. A
+ * timeout or Ctrl+C stops the agent the shell started, not only the shell: a survivor would still
+ * hold a world token and could write after Hue recorded the case as failed. The group is signalled
+ * even once the shell has exited, since a compound command's agent can outlive it, and a stopped
+ * command settles only when nothing in the group is left. A command that exits normally but
+ * leaves processes behind has them stopped the same way before it settles, so none can touch its
+ * outputs while they are collected. Once the group is found empty it is never signalled again,
+ * since its ID could be reused. Windows has no process group to signal, so the child alone is
+ * stopped there.
+ */
+export declare function spawnAgentCommand(command: string, options: {
+    cwd?: string;
+    env: Record<string, string | undefined>;
+    stdin?: string;
+    timeoutSeconds: number;
+    signal?: AbortSignal;
+}): Promise<string>;
+export declare function renderTable(verdicts: ExperimentVerdicts, output: Output): void;
+export declare function explain(error: unknown): string;
 /**
  * `hue eval`: runs a local adapter or command against a Scenario or eval set through
  * `runSimulation`, or registers it as an outbound worker through `runLocalAgent`.