@hue-run/sdk 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CLI.md CHANGED
@@ -42,7 +42,7 @@ project manifest are refused because managers can update ancestor locks; Python
42
42
  The generated `hue.setup.mjs` or `hue_setup.py` always selects `captureContent: false` /
43
43
  `capture_content=False`. For a supported application, setup installs the dependency and adds the
44
44
  managed import and middleware registration to the existing entrypoint; an unreferenced helper is
45
- not a completed integration. TypeScript uses `@hue-run/sdk@0.8.0`, `@opentelemetry/api@1.9.1` and
45
+ not a completed integration. TypeScript uses `@hue-run/sdk@0.9.0`, `@opentelemetry/api@1.9.1` and
46
46
  `@opentelemetry/context-async-hooks@2.11.0`; Python setup uses its separately tested package pin.
47
47
  Content capture requires an ordinary account-managed key and a later explicit application decision.
48
48
 
@@ -157,9 +157,13 @@ project. Run them separately with different keys if you want to revoke either us
157
157
  ```sh
158
158
  hue login # one key for both uses into ./.env.hue
159
159
  hue login --keys coding-agent --gitignore # only HUE_MCP_KEY; add .env.hue to .gitignore
160
- hue login --origin https://staging.hue.run --env-file .env.staging
160
+ hue login --origin https://staging.hue.run --env-path .env.staging
161
161
  ```
162
162
 
163
+ `--env-path <path>` writes another env file, creating it when missing. `--env-file <path>` is
164
+ accepted too, but Node 22 and 24 read that flag from the whole command line and exit with
165
+ `node: <path>: not found` before `hue` runs when the file does not exist yet.
166
+
163
167
  The env file is written with mode `0600` through a temporary file and an atomic rename. Other
164
168
  lines are preserved; a symlink or a non-regular file is refused; an existing different value is
165
169
  replaced only with `--force`. Empty values, whitespace and URLs are refused before any request.
@@ -301,7 +305,7 @@ node packages/sdk-typescript/scripts/verify-package.mjs --artifacts-dir .artifac
301
305
  # Set project to an existing supported fixture; use the same directory on resume.
302
306
  project=/absolute/path/to/supported-fixture
303
307
  node packages/sdk-typescript/scripts/verify-setup-live.mjs \
304
- --archive .artifacts/typescript/hue-run-sdk-0.8.0.tgz \
308
+ --archive .artifacts/typescript/hue-run-sdk-0.9.0.tgz \
305
309
  --origin https://STAGING_ORIGIN \
306
310
  --project "$project" --command setup \
307
311
  --evidence .context/setup-staging-before-claim.json
@@ -312,7 +316,7 @@ the private local handoff and finish the real browser claim, then reconcile the
312
316
 
313
317
  ```sh
314
318
  node packages/sdk-typescript/scripts/verify-setup-live.mjs \
315
- --archive .artifacts/typescript/hue-run-sdk-0.8.0.tgz \
319
+ --archive .artifacts/typescript/hue-run-sdk-0.9.0.tgz \
316
320
  --origin https://STAGING_ORIGIN \
317
321
  --project "$project" --command claim \
318
322
  --evidence .context/setup-staging-after-claim.json
@@ -375,7 +379,8 @@ hue eval --worker ./hue-agent.ts --agent-key support-agent --env-file .env.hue
375
379
 
376
380
  The one-shot mode resolves the selection (`--case` by name, ID or URL; `--set` by name, ID or
377
381
  URL with explicit `--scorer-version` pins; or `--dataset-version` with `--scorer-version`),
378
- creates a fresh run from those immutable pins named `<case> · <agent key> · <revision>`
382
+ creates a fresh run from those immutable pins named `<agent key> @ <revision>` (a commit hash is
383
+ shortened to 7 characters)
379
384
  (`--name` overrides), prints `Run: <url>` and `Experiment: <id>` as soon as the experiment exists,
380
385
  one line per case event (world created, agent started, world sealed), then
381
386
  `Waiting for Hue checks...` and a table with one row per case: boolean metrics as `PASS`/`FAIL`,
@@ -411,7 +416,7 @@ apply; Hue chooses the pinned experiment. The worker exits 0 when it stops norma
411
416
  A Scenario or eval set whose dataset version is not saved cannot back an experiment: the command
412
417
  exits 1 and asks for **Save eval-set version** in Hue or `--save-version`, which freezes that
413
418
  version at its current revision. Connection settings are `HUE_API_KEY` and `HUE_BASE_URL`
414
- (default `https://app.hue.run`), loaded from `--env-file <path>` first when given; `--origin`
419
+ (default `https://app.hue.run`), loaded from `--env-file <path>` (or its alias `--env-path`) first when given; `--origin`
415
420
  overrides the origin. Telemetry content capture stays off unless `--content` is passed; in
416
421
  one-shot mode it also decides whether case outputs, error messages and explanations are persisted
417
422
  to Hue. `--worker` always persists them, because a run launched from Hue is read on its run page:
package/ENVIRONMENTS.md CHANGED
@@ -126,6 +126,18 @@ Hue canonicalizes valid synthetic-principal UUIDs to lowercase, and repository r
126
126
  the same before comparing immutable digests, so casing-only UUID changes reuse the stored
127
127
  version without dropping provider bindings.
128
128
 
129
+ Each Gmail provider instance names its mailbox carrier in `configuration`
130
+ (`GmailMailboxConfiguration`, narrowed by `kind`). `gmail_mailbox/v1` binds `messagesCollection`
131
+ and `draftsCollection` in the simplified shape Hue's `hue.gmail.*` actions read.
132
+ `gmail_mailbox/v2` adds `labelsCollection` and binds all three in the entity shapes the Gmail
133
+ mirrors serve: messages may also carry `cc`, `bcc`, `htmlBody`, `snippet`, attachment metadata,
134
+ `sizeEstimate`, `historyId` and `raw`, each draft names a `DRAFT`-labelled message, and threads
135
+ are derived from each message's `threadId`. Only the simulation gateway serves it: the pinned
136
+ provider-profile preflight above reports `profile_unavailable` for a `gmail_mailbox/v2`
137
+ instance. Its `mailboxAddress` must be the synthetic owner `owner@example.test` (compared
138
+ case-insensitively) or publication is refused, and a definition whose provider instances all use
139
+ it may publish with `actions: []`.
140
+
129
141
  ```ts
130
142
  const definition = {
131
143
  kind: "repository" as const,
@@ -183,7 +195,8 @@ await client.finishRun(run.id, { idempotencyKey: randomUUID(), status: "complete
183
195
  ```
184
196
 
185
197
  Each bound call is an ordinary `hue.tool` span. When the catalog names an MCP server, the span
186
- also carries `mcp.server.name`. Wrap any MCP client the same way, using `serverInfo` from
198
+ also carries `mcp.server.name`, plus `hue.mcp.provider` and `hue.mcp.surface` when the catalog
199
+ entry includes them. Wrap any MCP client the same way, using `serverInfo` from
187
200
  `initialize` — this is not specific to Hue-hosted Gmail or Slack:
188
201
 
189
202
  ```ts
@@ -229,22 +242,38 @@ try {
229
242
  await spawnAgent({ env: child, mcpConfigPath: config.path });
230
243
  } finally {
231
244
  await config.dispose();
232
- await environmentClient.finishRun(run.id, {
245
+ const finished = await environmentClient.finishRun(run.id, {
233
246
  idempotencyKey: `execution:${executionId}:completed`,
234
247
  status: "completed",
235
248
  });
249
+ const graceEnd = Date.parse(finished.completingUntil ?? "") || Date.now();
250
+ await new Promise((resolve) =>
251
+ setTimeout(resolve, Math.min(10_000, Math.max(0, graceEnd - Date.now()))),
252
+ );
253
+ const sealDeadline = Date.now() + 30_000;
254
+ let sealed = false;
255
+ while (Date.now() < sealDeadline) {
256
+ if ((await environmentClient.getRun(run.id)).status !== "open") {
257
+ sealed = true;
258
+ break;
259
+ }
260
+ await new Promise((resolve) => setTimeout(resolve, 250));
261
+ }
262
+ if (!sealed) throw new Error("World was not sealed after its completion grace");
236
263
  }
237
264
  ```
238
265
 
239
266
  `runSimulation`, `runLocalAgent` and `hue eval` do this for you: they create the world with the
240
267
  execution, the stable key, the case span's context and the agent revision, pass the handoff as
241
- `context.world`, and finish before returning so telemetry is flushed and the execution completed
242
- afterwards. For one compatibility release `context.mcp` is the world's first MCP mirror with the
268
+ `context.world`, and finish and wait for the seal before returning so telemetry is flushed and the
269
+ execution completed afterwards. For one compatibility release `context.mcp` is the world's first MCP mirror with the
243
270
  world token, so an adapter that read `HUE_MCP_URL` and `HUE_MCP_TOKEN` keeps working;
244
271
  `agentEnvironment` sets those names too unless `legacyMcpVariables: false`. A gateway world binds
245
272
  no Hue-native `tools` (Hue refuses them); a world created while the gateway is off keeps its tools
246
273
  and the `hue_sim_` capability and emits a one-time `DeprecationWarning`.
247
274
 
275
+ A refusal such as 409 `simulation_gateway_required` exposes its validated server code as `HueEnvironmentError.diagnostic`.
276
+
248
277
  `agentEnvironment` removes Hue control-plane credentials from the child by default: `HUE_API_KEY`,
249
278
  `HUE_MCP_KEY` and any variable whose value is a `hue_sk_`, `hue_mcp_` or `hue_attempt_`
250
279
  credential. Pass `includeHueCredentials: true` only for an agent that must call Hue's own API.
@@ -252,7 +281,9 @@ Nothing in these helpers logs the token; keep it out of your own logs and checkp
252
281
 
253
282
  Finish answers `lifecycle: "completing"` with `sealedAt: null` for a gateway world: the seal
254
283
  follows a 5 s grace so in-flight writes land, and a late finish answers 409, which the helpers
255
- treat as the seal they can no longer change. `getEvidence(runId, { section, bodies })` reads the
284
+ treat as the seal they can no longer change. Completing an execution while the world is open
285
+ returns 409, so direct clients read the run after the finish response's `completingUntil`
286
+ until it is no longer open; the helper waits through transient failures. `getEvidence(runId, { section, bodies })` reads the
256
287
  sealed world's evaluator-only evidence (start and end state, the diff, the call ledger, coverage,
257
288
  fingerprint) with the project key; a world token can never read it. The client honors Hue's
258
289
  `Retry-After` on 429 and 503.
package/README.md CHANGED
@@ -131,22 +131,82 @@ await hue.model(
131
131
  );
132
132
  ```
133
133
 
134
+ Tools the provider runs itself, such as OpenAI hosted MCP, web search, file search and code
135
+ interpreter, or Anthropic's MCP connector and server tools, never pass through `hue.tool()`. Inside
136
+ the same `hue.model()` callback, hand the response to `hue.recordProviderToolCalls`:
137
+
138
+ ```ts
139
+ await hue.model(
140
+ "gpt-5-mini",
141
+ async (span) => {
142
+ const response = await openai.responses.create(request);
143
+ span.setOutput(response.output_text);
144
+ hue.recordProviderToolCalls(response, { request });
145
+ return response;
146
+ },
147
+ { provider: "openai" },
148
+ );
149
+ ```
150
+
151
+ Each OpenAI Responses `mcp_call`, `web_search_call`, `file_search_call` or `code_interpreter_call`
152
+ item, and each Anthropic `mcp_tool_use` or `server_tool_use` block with its result, becomes an
153
+ `execute_tool {name}` child span with `gen_ai.tool.type` `extension`, `gen_ai.tool.call.id` and,
154
+ for MCP calls, `mcp.server.name` (the provider's label). Arguments and results follow
155
+ `captureContent`; a failed call carries `error.type` (`mcp_error`, the provider's status or error
156
+ code) and ERROR status. An `mcp_list_tools` item becomes a `tools/list` child span with that
157
+ server's `gen_ai.tool.definitions`. Pass the request so each server's host is recorded as
158
+ `server.address` (only server URLs are read, never credentials), and `servers` to record a
159
+ server's real `name`, `version`, Hue `provider` and `surface` under its label. `provider` defaults
160
+ to the enclosing `model()` call's provider (`openai` or `anthropic`). The spans have no duration of
161
+ their own: the provider ran the tools inside the model request.
162
+
134
163
  The span is named `{operation} {model}` (`operation` defaults to `chat`) with
135
164
  `gen_ai.operation.name`, `gen_ai.request.model` and `gen_ai.provider.name`. Like `withSpan`, the
136
- options come after the callback and also accept `name`, `sessionId`, `userId`, `input` (recorded as
137
- `gen_ai.input.messages`) and `parentContext`. `setUsage` records
165
+ options come after the callback and also accept `name`, `sessionId`, `userId`, `workspaceId`, `input` (recorded as
166
+ `gen_ai.input.messages`) and `parentContext`. When you send system instructions or tools separately
167
+ from the messages, pass `systemInstructions` (for example `[{ type: "text", content: instructions }]`)
168
+ and `tools` (for example `[{ type: "function", name, description, parameters }]`); they are recorded
169
+ as `gen_ai.system_instructions` and `gen_ai.tool.definitions` under the same `captureContent` rule
170
+ as the messages. `recordMessages` also accepts `systemInstructions` for its log record. `setUsage` records
138
171
  nonnegative integer `gen_ai.usage.input_tokens` / `output_tokens`; other values are omitted and
139
172
  counted as instrumentation failures. Unknown usage stays absent. `hue.tool(name, input, execute)`
140
173
  creates an `execute_tool {name}` span with `gen_ai.tool.name`, arguments and result; an optional
141
174
  fourth argument `{ callId }` records the provider's tool call id as `gen_ai.tool.call.id`. When the
142
175
  tool came from an MCP server, pass `{ mcp: client.getServerVersion() }` (the MCP `initialize`
143
176
  `serverInfo`) to record `mcp.server.name` and `mcp.server.version` so a generic verb such as
144
- `get_thread` is attributed to that server. Content
177
+ `get_thread` is attributed to that server. When the server is a Hue surface, `mcp.provider` and
178
+ `mcp.surface` (for example `google.gmail` and `google.gmail/mcp`) record `hue.mcp.provider` and
179
+ `hue.mcp.surface`. A blank, over-256-character or otherwise invalid label is omitted and counted as
180
+ an instrumentation failure; the tool still runs. Content
145
181
  helpers (`setInput`, `setOutput`, `tool` arguments and results, `recordMessages`,
146
182
  `SpanOptions.input`) accept any value and encode plain JSON data (`JsonValue`) at runtime; a value
147
183
  that is not JSON, such as a `Date` or a class instance, is omitted with an instrumentation failure
148
184
  while the callback result is returned unchanged.
149
185
 
186
+ ## Files
187
+
188
+ Record a file the work read, received or produced with `hue.recordFile`. It adds a `hue.file`
189
+ event to the active span (or the span of an explicit context passed second), keyed by the file's
190
+ SHA-256, so a trace can be linked to the same file elsewhere without exporting its bytes:
191
+
192
+ ```ts
193
+ import { readFile } from "node:fs/promises";
194
+
195
+ await hue.withSpan("review contract", async () => {
196
+ const pdf = await readFile("contract.pdf");
197
+ hue.recordFile({ role: "input", mediaType: "application/pdf", data: pdf, name: "contract.pdf" });
198
+ // ...
199
+ });
200
+ ```
201
+
202
+ `data` up to 25 MiB is hashed and measured locally and never exported; larger data is omitted and
203
+ counted as an instrumentation failure, so pass `sha256` (and `byteSize`) instead when you already
204
+ have them. The event carries `hue.file.sha256`, `hue.file.role` (`input`,
205
+ `attachment` or `output`), `hue.file.media_type` and `hue.file.size`. It is metadata, so it is
206
+ recorded in both capture modes, while `hue.file.name` is recorded only when `captureContent` is
207
+ true. An invalid record, or one without an active span, is omitted and counted as an
208
+ instrumentation failure.
209
+
150
210
  ## Vercel AI SDK 6
151
211
 
152
212
  AI SDK 6 accepts a per-call tracer through `experimental_telemetry`. Pass
@@ -216,6 +276,15 @@ instrumentations can use `hue.tracer` directly or explicitly attach the processo
216
276
  below. Instrumentations that only use a global provider need your application's
217
277
  normal OTel setup; Hue does not silently replace it.
218
278
 
279
+ Provider-executed tools, such as OpenAI hosted MCP (`openai.tools.mcp`), appear as
280
+ `execute_tool mcp.<name>` spans with `gen_ai.tool.type` `extension`. The server is named only
281
+ by `serverLabel` inside the recorded result, so before export the TypeScript SDK copies it to
282
+ `mcp.server.name`, and a result with an MCP `error` sets ERROR status and `error.type`
283
+ `mcp_error`. Metadata-only export keeps these two attributes while stripping arguments and
284
+ results. The label can only be read when AI SDK recorded the result: `hueTelemetry(hue)` with
285
+ `captureContent: false` records none, whereas an application whose AI SDK integration records
286
+ outputs and exports through Hue's attached processors keeps the label.
287
+
219
288
  ## Privacy and content
220
289
 
221
290
  `captureContent: false` disables manual input/output/messages/tool content and
@@ -232,6 +301,36 @@ attributes, resources, event/link attributes and log bodies. Return a string.
232
301
  Invalid/oversized helper content is omitted with an instrumentation failure; the span can still be delivered. Export-time redactor failures reject the affected record and are reported by flush. Shared resources are redacted once per
233
302
  export batch. Do not put user content or secrets in span names or scope names.
234
303
 
304
+ Hosted tools carry credentials in their definitions, such as the `authorization` and `headers` of
305
+ an OpenAI hosted MCP tool. Before export, and before `redact`, Hue replaces the values of
306
+ credential-like fields including `authorization`, `authorization_token`, `headers`, `api_key`,
307
+ `access_token`, `x-api-key`, and keys ending in `token`, `secret`, `password`, `apikey` or
308
+ `credential` (case-insensitively, ignoring `-` and `_`) with `"[redacted]"` in recorded tool definitions
309
+ (`gen_ai.tool.definitions`, `ai.prompt.tools`, `llm.tools.*.tool.json_schema`) and in the `tools`
310
+ and `mcp_servers` entries of a raw provider request or response recorded as `input.value`,
311
+ `output.value` or `llm.invocation_parameters`. Parameters named in a JSON Schema `properties`
312
+ object keep their schemas, so a tool that takes a `headers` argument is still described. A
313
+ definition nested more than 256 levels deep rejects its record. Sensitive `default`, `const`,
314
+ `examples` and `enum` values under credential-named schema parameters are redacted too.
315
+
316
+ With `captureContent: false`, export removes tool definitions but keeps a summary of them on the
317
+ same record: `hue.tool.names` lists each definition's `name` (Chat Completions `function.name`,
318
+ or the `type` of an unnamed built-in tool such as `mcp`) in order, and
319
+ `hue.tool.definitions.sha256` is the lowercase hex SHA-256 of the
320
+ [RFC 8785](https://www.rfc-editor.org/rfc/rfc8785) canonical JSON of the credential-scrubbed
321
+ definition list. The digest is the same in both SDKs and does not change when a credential
322
+ rotates. Only definitions another integration recorded can be summarized: `hueTelemetry(hue)` with
323
+ `captureContent: false` records none, whereas an application whose AI SDK integration records
324
+ inputs and exports through Hue's attached processors gets the summary.
325
+ Recorded messages can inline files: GenAI `blob` parts in `gen_ai.input.messages` /
326
+ `gen_ai.output.messages` (what the AI SDK 7 adapter records for a file part) and AI SDK 6 `file`
327
+ parts in `ai.prompt.messages`. A span whose messages exceed 256 KiB would be rejected, so before
328
+ export Hue replaces the `content`/`data` of any such part longer than 64 KiB with the file's
329
+ `sha256` (of the decoded bytes for base64 and `data:` URLs, of the UTF-8 text otherwise) and
330
+ `size`, keeping the part's other fields such as `type`, `mime_type` and `mediaType`. Smaller inline
331
+ files are exported as recorded. The digest matches `hue.recordFile`'s `hue.file.sha256` for the same
332
+ bytes, so a file can be recognized wherever it appears.
333
+
235
334
  Manual helpers encode JSON values without converting null into absence. Unknown
236
335
  outputs and usage remain absent. This SDK does not estimate tokens or cost. A thrown
237
336
  application error marks the span with `error.type` (the error's `name`), an ERROR status and an
@@ -281,7 +380,9 @@ For external parent context pass `parentContext` to `withSpan`. Across processes
281
380
  speak W3C `traceparent` only and never include the API key or baggage. Hue registers no global
282
381
  propagator, so `propagation.inject()` from `@opentelemetry/api` is a no-op unless your
283
382
  application configured one. `getContext()` exposes the helper's current context for APIs taking
284
- an explicit context. Session/user identifiers are inherited within a client callback and are
383
+ an explicit context. Session, user and workspace identifiers (`sessionId` as
384
+ `gen_ai.conversation.id`, `userId` as `user.id`, and `workspaceId` as `hue.workspace.id` for the
385
+ application workspace or tenant) are inherited within a client callback and are
285
386
  stamped only on spans created through Hue's tracer (helpers and the AI SDK adapters); spans from
286
387
  other instrumentations on a shared provider carry them only if that instrumentation sets them.
287
388
  Separate requests require separate callbacks.
package/dist/ai-sdk.d.ts CHANGED
@@ -2,9 +2,9 @@ import type { TelemetryOptions } from "ai";
2
2
  import type { HueClient } from "./client.js";
3
3
  /**
4
4
  * Per-call telemetry for AI SDK 7: pass as an agent's or generation call's `telemetry` option.
5
- * Spans come from Hue's tracer, so they parent under `withSpan` and inherit session/user
6
- * identifiers; prompt and response recording follow `captureContent`. It does not change global
7
- * AI SDK integrations.
5
+ * Spans come from Hue's tracer, so they parent under `withSpan` and inherit session, user and
6
+ * workspace identifiers; prompt and response recording follow `captureContent`. It does not
7
+ * change global AI SDK integrations.
8
8
  *
9
9
  * @throws TypeError when the client is enabled and the installed `ai` major version is below 7;
10
10
  * AI SDK 6 applications use `hueExperimentalTelemetry` from `@hue-run/sdk` instead.
package/dist/ai-sdk.js CHANGED
@@ -21,9 +21,9 @@ function aiMajor() {
21
21
  }
22
22
  /**
23
23
  * Per-call telemetry for AI SDK 7: pass as an agent's or generation call's `telemetry` option.
24
- * Spans come from Hue's tracer, so they parent under `withSpan` and inherit session/user
25
- * identifiers; prompt and response recording follow `captureContent`. It does not change global
26
- * AI SDK integrations.
24
+ * Spans come from Hue's tracer, so they parent under `withSpan` and inherit session, user and
25
+ * workspace identifiers; prompt and response recording follow `captureContent`. It does not
26
+ * change global AI SDK integrations.
27
27
  *
28
28
  * @throws TypeError when the client is enabled and the installed `ai` major version is below 7;
29
29
  * AI SDK 6 applications use `hueExperimentalTelemetry` from `@hue-run/sdk` instead.
@@ -0,0 +1,22 @@
1
+ /**
2
+ * Node 22 and 24 scan the whole command line for `--env-file`, including arguments meant for the
3
+ * script, and exit with "node: <path>: not found" before the CLI runs when that file does not
4
+ * exist. `--env-path` is the same option under a name Node ignores, so a command such as
5
+ * `hue login` can name a file it is about to create.
6
+ */
7
+ export declare const envFileOptions: {
8
+ readonly "env-file": {
9
+ readonly type: "string";
10
+ };
11
+ readonly "env-path": {
12
+ readonly type: "string";
13
+ };
14
+ };
15
+ /**
16
+ * The env file named by `--env-path` or `--env-file`; throws when both name different files,
17
+ * comparing them as resolved from `cwd`.
18
+ */
19
+ export declare function envFileArgument(values: {
20
+ "env-file"?: string | undefined;
21
+ "env-path"?: string | undefined;
22
+ }, cwd: string): string | undefined;
@@ -0,0 +1,21 @@
1
+ import { resolve } from "node:path";
2
+ /**
3
+ * Node 22 and 24 scan the whole command line for `--env-file`, including arguments meant for the
4
+ * script, and exit with "node: <path>: not found" before the CLI runs when that file does not
5
+ * exist. `--env-path` is the same option under a name Node ignores, so a command such as
6
+ * `hue login` can name a file it is about to create.
7
+ */
8
+ export const envFileOptions = {
9
+ "env-file": { type: "string" },
10
+ "env-path": { type: "string" },
11
+ };
12
+ /**
13
+ * The env file named by `--env-path` or `--env-file`; throws when both name different files,
14
+ * comparing them as resolved from `cwd`.
15
+ */
16
+ export function envFileArgument(values, cwd) {
17
+ const { "env-file": file, "env-path": path } = values;
18
+ if (file !== undefined && path !== undefined && resolve(cwd, file) !== resolve(cwd, path))
19
+ throw new Error("Pass one of --env-file and --env-path, not both.");
20
+ return path ?? file;
21
+ }
package/dist/cli/eval.js CHANGED
@@ -17,6 +17,7 @@ import { matchByName, parseScenarioSelector, resolveEvalSetPins, resolveScenario
17
17
  import { collectDirectOutputs, stageDirectCase } from "./eval-direct.js";
18
18
  import { runSimulation, } from "../evals/simulation.js";
19
19
  import { collectExperimentVerdicts, compareVerdicts, metricPassed, } from "../evals/verdicts.js";
20
+ import { envFileArgument, envFileOptions } from "./env-file.js";
20
21
  const USAGE = `Usage: hue eval [adapter-file] [options]
21
22
 
22
23
  Run a local agent against a published case or eval set, then print Hue's verdicts.
@@ -56,10 +57,11 @@ Modes:
56
57
 
57
58
  Connection:
58
59
  --env-file <path> Load a dotenv file (HUE_API_KEY, HUE_BASE_URL) first
60
+ --env-path <path> Same as --env-file
59
61
  --origin <url> Hue origin (default: HUE_BASE_URL or https://app.hue.run)
60
62
 
61
63
  Output and limits:
62
- --name <run name> Run name (default: <case> · <agent key> · <revision>)
64
+ --name <run name> Run name (default: <agent key> @ <revision>, hashes shortened)
63
65
  --baseline <experiment id|url> Compare verdicts with a previous experiment
64
66
  --json Print one JSON document on stdout; progress goes to stderr
65
67
  --content Capture telemetry content; one-shot also persists
@@ -105,7 +107,7 @@ function parse(argv) {
105
107
  "agent-key": { type: "string" },
106
108
  "agent-name": { type: "string" },
107
109
  revision: { type: "string" },
108
- "env-file": { type: "string" },
110
+ ...envFileOptions,
109
111
  origin: { type: "string" },
110
112
  name: { type: "string" },
111
113
  baseline: { type: "string" },
@@ -172,6 +174,12 @@ function derivedAgentKey(adapterFile, command) {
172
174
  "";
173
175
  return slug(basename(script, extname(script)));
174
176
  }
177
+ function defaultRunName(agent) {
178
+ const revision = /^[0-9a-f]{12,64}$/i.test(agent.revision)
179
+ ? agent.revision.slice(0, 7)
180
+ : agent.revision;
181
+ return `${agent.key} @ ${revision}`;
182
+ }
175
183
  function gitRevision() {
176
184
  try {
177
185
  const value = execFileSync("git", ["rev-parse", "--short=12", "HEAD"], {
@@ -692,7 +700,7 @@ async function runOnce(values, connection, agents, agent, hue, output, signal) {
692
700
  const frozen = await client.freezeDatasetVersion(pins.datasetVersionId, pins.revision);
693
701
  output.log(`Saved "${pins.name}" version ${frozen.version}.`);
694
702
  }
695
- const runName = values.name ?? `${pins.name} · ${agent.key} · ${agent.revision}`;
703
+ const runName = values.name ?? defaultRunName(agent);
696
704
  const project = await client.checkConnection();
697
705
  if (await detectDirect(client, pins, mode))
698
706
  return runDirect({
@@ -976,12 +984,19 @@ export async function runEvalCommand(argv) {
976
984
  throw new UsageError("--worker takes no selection; Hue chooses the run to execute");
977
985
  if (!values.worker && selections !== 1)
978
986
  throw new UsageError("Pass exactly one of --case, --set or --dataset-version");
979
- if (values["env-file"]) {
987
+ let envFile;
988
+ try {
989
+ envFile = envFileArgument(values, process.cwd());
990
+ }
991
+ catch (error) {
992
+ throw new UsageError(error.message);
993
+ }
994
+ if (envFile) {
980
995
  try {
981
- process.loadEnvFile(resolve(values["env-file"]));
996
+ process.loadEnvFile(resolve(envFile));
982
997
  }
983
998
  catch (error) {
984
- throw new UsageError(`Unable to load ${values["env-file"]}: ${error instanceof Error ? error.message : String(error)}`);
999
+ throw new UsageError(`Unable to load ${envFile}: ${error instanceof Error ? error.message : String(error)}`);
985
1000
  }
986
1001
  }
987
1002
  const apiKey = process.env.HUE_API_KEY?.trim();
@@ -14,7 +14,7 @@ export interface LoginCommandIo {
14
14
  fetch?: typeof fetch;
15
15
  openBrowser?: (url: string) => Promise<boolean>;
16
16
  }
17
- export declare const LOGIN_USAGE = "Usage: hue login [--origin URL] [--env-file PATH] [--keys evaluations|coding-agent|both]\n [--no-browser] [--force] [--gitignore]\n\nStore the key you created in Hue in a private env file. By default one \"Read and write\" key serves\nboth evaluations (HUE_API_KEY) and your coding agent (HUE_MCP_KEY); it is validated against Hue\nbefore it is stored, and key values are never printed.\n\nOptions:\n --origin URL Hue origin (default https://app.hue.run)\n --env-file PATH Env file to write (default .env.hue in the current directory)\n --keys KIND evaluations (HUE_API_KEY), coding-agent (HUE_MCP_KEY) or both from one key\n (default both)\n --no-browser Do not open the key settings page in a browser\n --force Replace an existing different value in the env file\n --gitignore Add the env file to .gitignore when a git repository does not ignore it\n -h, --help Show this help";
17
+ export declare const LOGIN_USAGE = "Usage: hue login [--origin URL] [--env-path PATH] [--keys evaluations|coding-agent|both]\n [--no-browser] [--force] [--gitignore]\n\nStore the key you created in Hue in a private env file. By default one \"Read and write\" key serves\nboth evaluations (HUE_API_KEY) and your coding agent (HUE_MCP_KEY); it is validated against Hue\nbefore it is stored, and key values are never printed.\n\nOptions:\n --origin URL Hue origin (default https://app.hue.run)\n --env-path PATH Env file to write (default .env.hue in the current directory); it is\n created when missing. --env-file also works, but Node itself exits\n before this command runs when that file does not exist yet\n --keys KIND evaluations (HUE_API_KEY), coding-agent (HUE_MCP_KEY) or both from one key\n (default both)\n --no-browser Do not open the key settings page in a browser\n --force Replace an existing different value in the env file\n --gitignore Add the env file to .gitignore when a git repository does not ignore it\n -h, --help Show this help";
18
18
  /** Hue MCP endpoint that pairs with an application origin. */
19
19
  export declare function mcpUrlForOrigin(origin: string): string;
20
20
  /** Normalizes a Hue origin: HTTPS, or HTTP for loopback only; no credentials, path, query or hash. */
package/dist/cli/login.js CHANGED
@@ -7,6 +7,7 @@ import { createInterface } from "node:readline";
7
7
  import { Writable } from "node:stream";
8
8
  import { parseArgs } from "node:util";
9
9
  import { isLoopbackHost } from "../config.js";
10
+ import { envFileArgument, envFileOptions } from "./env-file.js";
10
11
  const DEFAULT_ORIGIN = "https://app.hue.run";
11
12
  const DEFAULT_ENV_FILE = ".env.hue";
12
13
  /** Settings section that lists and creates project service keys. */
@@ -27,7 +28,7 @@ const KEY_KINDS = {
27
28
  };
28
29
  /** Settings preset that authorizes both evaluations and the coding agent's MCP reads and writes. */
29
30
  const KEY_PRESET = "Read and write";
30
- export const LOGIN_USAGE = `Usage: hue login [--origin URL] [--env-file PATH] [--keys evaluations|coding-agent|both]
31
+ export const LOGIN_USAGE = `Usage: hue login [--origin URL] [--env-path PATH] [--keys evaluations|coding-agent|both]
31
32
  [--no-browser] [--force] [--gitignore]
32
33
 
33
34
  Store the key you created in Hue in a private env file. By default one "Read and write" key serves
@@ -36,7 +37,9 @@ before it is stored, and key values are never printed.
36
37
 
37
38
  Options:
38
39
  --origin URL Hue origin (default ${DEFAULT_ORIGIN})
39
- --env-file PATH Env file to write (default ${DEFAULT_ENV_FILE} in the current directory)
40
+ --env-path PATH Env file to write (default ${DEFAULT_ENV_FILE} in the current directory); it is
41
+ created when missing. --env-file also works, but Node itself exits
42
+ before this command runs when that file does not exist yet
40
43
  --keys KIND evaluations (HUE_API_KEY), coding-agent (HUE_MCP_KEY) or both from one key
41
44
  (default both)
42
45
  --no-browser Do not open the key settings page in a browser
@@ -558,7 +561,7 @@ function parseLoginArguments(argv) {
558
561
  strict: true,
559
562
  options: {
560
563
  origin: { type: "string" },
561
- "env-file": { type: "string" },
564
+ ...envFileOptions,
562
565
  keys: { type: "string" },
563
566
  "no-browser": { type: "boolean", default: false },
564
567
  force: { type: "boolean", default: false },
@@ -591,8 +594,10 @@ export async function runLoginCommand(argv, io = {}) {
591
594
  return code;
592
595
  };
593
596
  let parsed;
597
+ let envFileOption;
594
598
  try {
595
599
  parsed = parseLoginArguments(argv);
600
+ envFileOption = envFileArgument(parsed.values, cwd);
596
601
  }
597
602
  catch (error) {
598
603
  return fail(`${error.message}\n\n${LOGIN_USAGE}`, 2);
@@ -615,7 +620,7 @@ export async function runLoginCommand(argv, io = {}) {
615
620
  if (!origin)
616
621
  return fail("--origin must be an HTTPS origin such as https://app.hue.run (plain HTTP is accepted for loopback test servers only).", 2);
617
622
  const mcpUrl = mcpUrlForOrigin(origin);
618
- const envPath = resolve(cwd, parsed.values["env-file"] ?? DEFAULT_ENV_FILE);
623
+ const envPath = resolve(cwd, envFileOption ?? DEFAULT_ENV_FILE);
619
624
  const envDisplay = displayPath(cwd, envPath);
620
625
  let envFile;
621
626
  try {
package/dist/client.d.ts CHANGED
@@ -1,6 +1,6 @@
1
1
  import { type Context, type Span, type Tracer } from "@opentelemetry/api";
2
2
  import { HueTransport } from "./transport.js";
3
- import type { ExportReport, FlushableLoggerProvider, FlushableTracerProvider, HueOptions, HueSpan, ModelOptions, ProjectConnection, SpanOptions, ToolOptions, VerifyTraceOptions, TraceVerification, SafeLifecycleOptions, SafeLifecycleResult } from "./types.js";
3
+ import type { ExportReport, FileRecord, FlushableLoggerProvider, FlushableTracerProvider, HueOptions, HueSpan, ModelOptions, ProjectConnection, SpanOptions, ProviderToolCallOptions, ToolOptions, VerifyTraceOptions, TraceVerification, SafeLifecycleOptions, SafeLifecycleResult } from "./types.js";
4
4
  /**
5
5
  * Attach mode: the application owns its OpenTelemetry providers and passes the transport whose
6
6
  * processors it attached to them. The client flushes these providers but never shuts them down.
@@ -75,7 +75,8 @@ export declare class HueClient {
75
75
  * `options.callId` is recorded as `gen_ai.tool.call.id`, like the Python `call_id=` keyword.
76
76
  * `options.mcp` records the MCP `initialize` `serverInfo` as `mcp.server.name` /
77
77
  * `mcp.server.version` so a generic tool name can be attributed to the server that
78
- * handled it. Pass `client.getServerVersion()`.
78
+ * handled it. Pass `client.getServerVersion()`. `mcp.provider` / `mcp.surface` record the Hue
79
+ * provider and surface as `hue.mcp.provider` / `hue.mcp.surface`.
79
80
  */
80
81
  tool<T>(name: string, input: unknown, execute: () => Promise<T> | T, options?: ToolOptions): Promise<T>;
81
82
  /**
@@ -84,7 +85,8 @@ export declare class HueClient {
84
85
  * `gen_ai.request.model` and `gen_ai.provider.name`. The argument order matches `withSpan`. The
85
86
  * handle's `setInput`/`setOutput` record `gen_ai.input.messages` / `gen_ai.output.messages`,
86
87
  * which should use the GenAI semantic-convention message shape; `recordMessages` inside the
87
- * callback inherits the request metadata.
88
+ * callback inherits the request metadata. `options.systemInstructions` and `options.tools` are
89
+ * recorded as `gen_ai.system_instructions` and `gen_ai.tool.definitions`, content like `input`.
88
90
  */
89
91
  model<T>(model: string, callback: (span: HueSpan) => Promise<T> | T, options: ModelOptions): Promise<T>;
90
92
  private setUsage;
@@ -113,6 +115,8 @@ export declare class HueClient {
113
115
  input?: unknown;
114
116
  /** Output messages, ideally in the GenAI semantic-convention shape; any JSON-encodable value. */
115
117
  output?: unknown;
118
+ /** System instructions sent separately from the messages, as `gen_ai.system_instructions`. */
119
+ systemInstructions?: unknown;
116
120
  /** `gen_ai.operation.name` for the record; defaults to the enclosing `model()` span's value. */
117
121
  operation?: string;
118
122
  /** `gen_ai.provider.name` for the record; defaults to the enclosing `model()` span's value. */
@@ -120,6 +124,30 @@ export declare class HueClient {
120
124
  /** `gen_ai.request.model` for the record; defaults to the enclosing `model()` span's value. */
121
125
  model?: string;
122
126
  }, explicitContext?: Context): void;
127
+ /**
128
+ * Records the tools a model provider executed itself while producing `response`, which no
129
+ * `hue.tool()` call saw: OpenAI Responses `mcp_call`, `web_search_call`, `file_search_call` and
130
+ * `code_interpreter_call` items, and Anthropic Messages `mcp_tool_use` / `server_tool_use`
131
+ * blocks with their result blocks. Each becomes an `execute_tool {name}` child span of the active
132
+ * (or given) context with `gen_ai.tool.type` `extension` and `gen_ai.tool.call.id`; MCP calls add
133
+ * `mcp.server.name` (the provider's label, or the `servers` entry for it). Arguments and results
134
+ * follow `captureContent`; a failed call carries `error.type` and ERROR status. An OpenAI
135
+ * `mcp_list_tools` item becomes a `tools/list` child span carrying that server's tools as
136
+ * `gen_ai.tool.definitions`. Call it inside `hue.model()` so the spans nest under the model call
137
+ * and `provider` defaults to its provider; pass `request` to record each server's host as
138
+ * `server.address`. The spans have no duration of their own: the provider ran the tools inside
139
+ * the model request. Unreadable items are skipped and counted; nothing is thrown.
140
+ */
141
+ recordProviderToolCalls(response: unknown, options?: ProviderToolCallOptions): void;
142
+ /**
143
+ * Adds a `hue.file` event to the active (or given) span for a file the work read, received or
144
+ * produced: `hue.file.sha256`, `hue.file.role`, `hue.file.media_type`, `hue.file.size` when known
145
+ * and, when `captureContent` is true, `hue.file.name`. `data` is hashed and measured locally and
146
+ * never exported. The event is metadata, so it is recorded in both capture modes. An invalid
147
+ * record, or one without an active span, is omitted and counted, never thrown; an invalid name
148
+ * alone is omitted and counted while the rest is recorded.
149
+ */
150
+ recordFile(file: FileRecord, explicitContext?: Context): void;
123
151
  private setContent;
124
152
  /**
125
153
  * Confirms the key and origin by reading the current project; a setup and CI diagnostic, not a