@axlsdk/studio 0.20.0 → 0.21.0

package/README.md

@@ -68,7 +68,9 @@ Your application imports from `src/runtime.ts` directly. Studio discovers everyt
 npx @axlsdk/studio --open
 ```
 
-This loads your config, starts the server on `http://localhost:4400`, and opens the browser.
+This loads your config and starts the server on `http://127.0.0.1:4400`. The
+standalone CLI binds only to IPv4 loopback, emits no CORS headers, and rejects
+non-local HTTP hosts and browser origins across both REST and WebSocket routes.
 
 ## CLI Options
 
@@ -77,19 +79,33 @@ axl-studio [options]
 
 Options:
   --port <number>          Server port (default: 4400)
+  --dangerously-bind <ip>  Bind for a loopback-published container/local-forward tunnel
   --config <path>          Path to config file (default: auto-detect)
   --conditions <list>      Comma-separated Node.js import conditions (e.g., development)
+  --read-only              Disable mutating endpoints
   --open                   Auto-open browser
   -h, --help               Show help
 ```
 
+For a container or devcontainer, bind the published host port to loopback and
+make only the container listener broad:
+
+```bash
+docker run -p 127.0.0.1:4400:4400 ... axl-studio --dangerously-bind 0.0.0.0
+```
+
+The Host/Origin guard remains active. Do not publish that port on a public or
+LAN interface; standalone Studio has no user authentication.
+
 The `--conditions` flag is useful in monorepos where workspace packages use the `"development"` export condition to point at source instead of built dist files. Pass `--conditions development` to resolve imports through source paths.
 
 **Note:** `--conditions` only applies to ESM `import()` resolution. Transitive `require()` calls from packages without `"type": "module"` bypass the hook and use default conditions. If a dependency loads via CJS, it will still resolve from `dist/` regardless of the flag.
 
 ## Panels
 
-Eight panels, all live over WebSocket. The screenshots below are placeholders — see [`docs/assets/CAPTURE.md`](../../docs/assets/CAPTURE.md) for how to record the real thing.
+Eight panels, all live over WebSocket. The captures below use representative seeded
+runs; see [`docs/assets/CAPTURE.md`](../../docs/assets/CAPTURE.md) for how to regenerate
+them.
 
 ### Agent Playground
 
@@ -101,7 +117,12 @@ Chat with any registered agent in real time. Streaming tokens, tool calls with e
 
 ### Trace Explorer
 
-Waterfall visualization of execution traces. Filter by type, agent, or tool; see token counts, cost per step, and duration. The Stats view shows the event-type distribution, top tools, and retry stacks.
+Waterfall visualization of execution traces. Filter by type, agent, or tool; see token
+counts, cost per step, and duration. Tool rows distinguish calls that succeeded,
+failed, were denied, were cancelled, or were rejected before they started. Studio also
+keeps older trace history readable and identifies incomplete runs instead of presenting
+missing events as a complete story. The Stats view shows the event-type distribution,
+top tools, and retry stacks.
 
 > **Studio vs `ctx.events`.** Studio consumes the same `AxlEvent` firehose via `runtime.on('trace', …)` — every event from every execution. Inside a workflow handler, `ctx.events` is the in-handler counterpart (per-context, scoped to the current workflow). The two coexist: Studio is for cross-execution observability and replay; `ctx.events` is for in-handler streaming UIs. See [`docs/observability.md`](../../docs/observability.md#observation-paths).
 
@@ -115,7 +136,14 @@ Track spending across agents, models, workflows, and embedders with time-window
 
 ### Eval Runner
 
-Run evaluations from the UI, watch items stream in, and drill into per-item scores, timing, cost, and LLM-judge reasoning. Compare two runs (baseline vs candidate) with bootstrap-CI significance, a score-distribution chart, and an item-level diff table. The History tab groups multi-run results and tracks mean scores across runs. Toggle **Capture traces** to render each item's events inline. Requires `@axlsdk/eval` as an optional peer dependency.
+Run evaluations from the UI, watch items stream in, and drill into per-item scores,
+timing, cost, and LLM-judge reasoning. Compare two runs (baseline vs candidate) with
+bootstrap-CI significance, a score-distribution chart, and an item-level diff table.
+The History tab groups multi-run results and tracks mean scores across runs. Toggle
+**Capture traces** to render each item's events inline. For tool-using workflows, those
+events show whether a call failed, was denied, was cancelled, or never started, giving
+you better evidence for debugging and custom scorers. Studio does not turn those states
+into scores automatically. Requires `@axlsdk/eval` as an optional peer dependency.
 
 Amber banners flag runs you shouldn't fully trust — a scorer that failed on too many items, a subset run, or annotations dropped by the dataset schema — so a thinned or misconfigured run can't quietly look clean.
 
@@ -165,24 +193,32 @@ const studio = createStudioMiddleware({
   runtime,
   basePath: '/studio',
   // WebSocket upgrades bypass Express middleware — always authenticate here.
-  verifyUpgrade: (req) => {
-    const url = new URL(req.url!, `http://${req.headers.host}`);
-    return url.searchParams.get('token') === process.env.MY_SECRET;
-  },
+  verifyUpgrade: (req) => authenticateStudioRequest(req)?.isAdmin === true,
 });
 
 const app = express();
-app.use('/studio', studio.handler);
+const authenticateStudioHttp: express.RequestHandler = (req, res, next) => {
+  if (authenticateStudioRequest(req)?.isAdmin !== true) return res.sendStatus(403);
+  next();
+};
+app.use('/studio', authenticateStudioHttp, studio.handler);
 const server = app.listen(3000);
 studio.upgradeWebSocket(server); // required for live data
 ```
 
-Key options: `readOnly` (disable mutating endpoints for production monitoring), `evals` (lazy-load eval files), `filterTraceEvent` (per-tenant broadcast scoping), `bufferCaps` (WS replay-buffer limits). `basePath` must match your framework's mount path.
+Key options: `readOnly` (disable mutating endpoints for production monitoring), `evals` (lazy-load eval files), `filterTraceEvent` (per-tenant broadcast scoping), `bufferCaps` (WS replay-buffer limits). `basePath` must match your framework's mount path. In production, `upgradeWebSocket()` refuses to attach without `verifyUpgrade`; `dangerouslyAllowUnauthenticatedWebSockets` is the explicit escape hatch only when an outer upgrade gate is independently guaranteed.
 
 **See [`docs/studio-api.md`](../../docs/studio-api.md) for the full reference:** every REST endpoint, the WebSocket protocol, the complete middleware options/return tables, NestJS/Fastify/Hono examples, host body-limit guidance, lazy eval loading, multi-tenant setup, and the internal architecture.
 
 ## Security
 
+- **Treat Studio as an administrative surface, never as a public application
+  API.** Agent routes intentionally expose resolved system prompts, tool
+  descriptions, schemas, and runtime configuration. `trace.redact` protects
+  user/model observability content; it is not authentication and does not hide
+  static agent configuration.
+- Protect the Studio HTTP mount with your framework's authentication and
+  authorization middleware.
 - **Always** provide `verifyUpgrade` — WebSocket upgrades bypass Express/Fastify/Koa middleware, so your auth middleware does **not** protect WebSocket connections.
 - Consider `readOnly: true` for production monitoring — view traces, costs, and schemas without execution capability.
 - CORS is not applied in embedded mode — the host framework owns CORS policy.

package/dist/{chunk-KQ3HBRTC.js → chunk-YGLWYDQ3.js}

@@ -1363,7 +1363,10 @@ function createSessionRoutes(connMgr) {
     const body = await c.req.json();
     const session = runtime.session(id);
     const result = await session.send(body.workflow, body.message);
-    return c.json({ ok: true, data: { result } });
+    return c.json({
+      ok: true,
+      data: { result: redactValue(result, runtime.isRedactEnabled()) }
+    });
   });
   app5.post("/sessions/:id/stream", async (c) => {
     const runtime = c.get("runtime");
@@ -1372,9 +1375,13 @@ function createSessionRoutes(connMgr) {
     const session = runtime.session(id);
     const stream = await session.stream(body.workflow, body.message);
     const executionId = `session-${id}-${Date.now()}`;
+    const redactOn = runtime.isRedactEnabled();
     (async () => {
       for await (const event of stream) {
-        connMgr.broadcastWithWildcard(`execution:${executionId}`, event);
+        connMgr.broadcastWithWildcard(
+          `execution:${executionId}`,
+          redactStreamEvent(event, redactOn)
+        );
       }
     })();
     return c.json({ ok: true, data: { executionId, streaming: true } });
@@ -16080,7 +16087,7 @@ function createServer(options) {
     emptyState: emptyEvalTrendData,
     windows
   });
-  if (options.cors !== false) {
+  if (options.cors === true) {
     app5.use("*", cors());
   }
   app5.onError(errorHandler);
@@ -16239,4 +16246,4 @@ export {
   EvalAggregator,
   createServer
 };
-//# sourceMappingURL=chunk-KQ3HBRTC.js.map
+//# sourceMappingURL=chunk-YGLWYDQ3.js.map