failproofai 1.0.12-beta.2 → 1.0.13-beta.1

package/.next/standalone/SECURITY.md

@@ -56,8 +56,9 @@ The gate above scans the lockfiles it is *given* (the `--lockfile` list in the
 workflow). GitHub's dependency graph reads **every** lockfile in the repository, so
 Dependabot alerts cover trees the gate does not — in particular the TypeScript SDK's
 integration fixtures, `sdk/typescript/integration/fixtures/*/package-lock.json`.
-Those are not in the scanner's list and are not a Dependabot *update* ecosystem in
-[`.github/dependabot.yml`](.github/dependabot.yml) either, so an alert is the only
+Those are not in the scanner's list and get no Dependabot *version* updates either
+(the four floor fixtures have entries in [`.github/dependabot.yml`](.github/dependabot.yml),
+but only to restrain its security updates; see below), so an alert is the only
 thing that will ever mention them. Treat the alert list as authoritative for those
 paths, and see *Triaging a Dependabot alert in an integration fixture* below for how
 to resolve one.
@@ -118,17 +119,32 @@ That splits an alert in one of these lockfiles into two cases:
    the scanner does not read these lockfiles, so an `[[IgnoredVulns]]` entry for one
    would filter nothing and be reported as an unused ignore.
 
+Dependabot's own security updates know neither case. Given a case-2 advisory they
+reach the fix by moving the framework, which is how #838, #931 and #935 happened
+(#935 opened three minutes after #934 put the pins back). So each floor fixture —
+`ai-4`, `langchain-0.3`, `llamaindex-0.11`, `mastra-0` — has a security-updates-only
+entry in [`.github/dependabot.yml`](.github/dependabot.yml)
+(`open-pull-requests-limit: 0`) whose `ignore` ranges hold its framework on its
+line, and zod on 3 where that framework requires it; a fix that stays on the line
+still arrives. A new floor fixture needs its own entry, and
+`__tests__/ci/ts-sdk-fixture-pins.test.ts` fails without one. A Dependabot PR that
+moves a pin anyway fails that same test: close it rather than merge it.
+
 ### Accepted, with no fix available
 
 | Advisory | Package | Fixtures | Why it stays |
 |---|---|---|---|
 | [GHSA-rwvc-j5jr-mgvh](https://github.com/advisories/GHSA-rwvc-j5jr-mgvh) (low, CVSS 3.7) | `ai@4.3.19` | `ai-4`, `mastra-0` | A filetype-whitelist bypass on **file upload**. Fixed in `ai@5.0.52` — the major the `ai-4` fixture exists to stay below, and one `@mastra/core@0.24.9` cannot take either (it requires `ai@^4`). Neither fixture uploads a file or exposes an upload surface; each runs one scripted agent against an in-process model. |
 | [GHSA-866g-f22w-33x8](https://github.com/advisories/GHSA-866g-f22w-33x8) (low, CVSS 4.3) | `@ai-sdk/provider-utils@2.2.8` | `ai-4`, `mastra-0` | Unbounded response-body reads in `createJsonResponseHandler` and its siblings. Fixed in `3.0.28`; `2.2.8` ships *inside* `ai@4.3.19`, which uses the v2 API, so the fix is the same blocked major as the row above. Exploiting it needs a hostile **model-provider HTTP response**; these fixtures never call one. |
+| [GHSA-mrhx-c7xc-m3w5](https://github.com/advisories/GHSA-mrhx-c7xc-m3w5) (high, CVSS 7.5) | `zod@3.25.76` | `ai-4`, `langchain-0.3`, `llamaindex-0.11`; nested in `langchain-dup-core` and `mastra-0` | An array schema keeps every failing element's validation issues, uncapped, so one oversized input can exhaust memory. Fixed only in `4.6.6`: no zod 3 release carries the fix, and `3.25.76` is the newest. Each copy is held to `^3` by the release its fixture pins — `ai@4.3.19` and its `@ai-sdk/*` packages peer on `^3.23.8`, `@langchain/core@0.3.80` (in `langchain-dup-core`, the vendored provider's own copy) and `@langchain/langgraph@0.4.10` depend on `^3.25.32`, `@llamaindex/core@0.6.8` and `@llamaindex/workflow@1.1.5` need `^3.23.8`, and in `mastra-0`, `zod-from-json-schema-v3` (`zod-from-json-schema@0.0.5`, which builds zod 3 schemas for `@mastra/core` and `@mastra/mcp`) depends on `^3.24.2`. Exploiting it needs untrusted input validated against one of those schemas; each fixture validates only its own scripted tool arguments. |
 
-Both rows are bounded by the same facts: these are test fixtures — `"private": true`,
+Every row is bounded by the same facts: these are test fixtures — `"private": true`,
 never published, installed with `--ignore-scripts`, and run only in CI against
 scripted in-process models. Re-check them whenever the supported framework floor
-moves; dropping `ai` 4.x from the matrix is what actually retires them.
+moves. The first two rows retire once `ai` 4.x has left the matrix, which takes
+`mastra-0` as well as `ai-4`, because `@mastra/core@0.24.9` requires `ai@^4`. The
+zod row retires only once `ai` 4.x, LangChain 0.3, LlamaIndex 0.11 and Mastra 0.x
+have all left it.
 
 ## Maintainer setup (one-time)
 

package/.next/standalone/fp-cloud-cli/CHANGELOG.md

@@ -1,9 +1,32 @@
 # Changelog — `fp` CLI
 
-## 0.0.2b0 — 2026-09-22
+## 0.0.3b0 — 2026-10-25
+
+Open for the next release. `0.0.2` published on 2026-10-25 and the `bump` job
+moved the version here automatically; nothing has landed against `0.0.3b0` yet.
+Add entries as changes merge — this section becomes the GitHub Release body when
+it ships.
+
+## 0.0.2 — 2026-10-25
+
+**Stable release.** The `0.0.2b0` line the `bump` job opened after `0.0.1` is cut stable
+here rather than published, because the server endpoints these commands drive are now
+deployed to FailproofAI Cloud and the CLI should not trail them as a pre-release.
+
+Nothing about the command surface changes at the cut: the entries below are the entire
+diff from `0.0.1`, and the `bump` job opens `0.0.3b0` after the upload.
 
 ### Added
 
+- Hosted evaluator authoring for code, LLM judge and Jev: create/deploy, AI draft specs, dry-run, labels, toggles, instances, schema, jobs and backfill.
+- Dashboards and tiles; session overview/export/history/re-evaluate; latency and sentiment; query composition/export; alert recipients/issues and agent contracts.
+- Policy author/backtest/candidacy, assistant artifact tables, fail-closed interactive write approvals, and agent-curated home briefs.
+- Organization creation, permission-set roles, member deletion/defaults, usage windows, offers/trial, model context windows, OTEL and Slack controls.
+- `fp policies backtest` covers every policy kind: a published `jev`/`both` policy, or `--semantic checks.json` (with or without a source file), runs the dashboard's every-kind backtest — a free estimate by default, `--run [--sample N]` to spend Jev allowance. New `--kind --semantic --hours --agent-id --env --run --sample`; a regex file or id still uses the module backtest.
+- `fp policies author --kind regex|jev|both` asks the assistant for Jev checks or a hybrid policy.
+- `fp issues list|count` take the inbox filters (`--agent-id --env --machine --source --audit-id --assignee --archived --search --since`, and `count --alert-id`); new `fp issues facets` lists the values they can take.
+- `fp sessions import FILE...` imports Claude Code transcripts, Codex rollouts or event JSONL (`--format --env --dry-run`), as the dashboard's import dialog does.
+- `fp list trigger_events`, `fp list eval_envs` and `fp list metric_keys`.
 - `fp fleet deploy --target POLICY=INTEGRATION[/agt_ID]` scopes a machine's assignment; repeat the flag to include multiple alternatives. `--all-agents POLICY` explicitly clears its scope. Deploy plans, show, history, rollback and JSON readback retain assignment targets.
 - `fp policies publish <id> --kind regex|jev|both [--source f.mjs] [--semantic f.json]`: a
   FailproofAI Cloud policy can carry Jev checks — `jev` (declarations only, no JavaScript) or
@@ -41,6 +64,9 @@
 
 ### Changed
 
+- New Cloud API families work in key mode; unsupported session-only commands deny before HTTP and unknown-route 404s exit 6 with an availability message.
+- Shared screens use the mint/pink wordmark and grey labels, without boxes/rules or shortened ids; existing JSON contracts are unchanged.
+- The agent skill, command reference, README and Cloud CLI docs include end-to-end evaluator workflows. No version bump or release is performed.
 - `fp policies publish`'s refusal of Jev fields in the JavaScript now says where Jev checks go:
   the policy's own declarations, `--kind both --semantic checks.json`.
 - `fp guardrails summary` shows how many machines run Jev in each mode, reading the summary's
@@ -56,6 +82,13 @@
 
 ### Fixed
 
+- Remove the unused colour-keyed permission rank, document full-id rendering, emit UTC `Z` timestamps for deploy backfills, and report approval-answer failures without a traceback while restoring the terminal (#885).
+- Reject natural-language binary judge steps, distinguish unknown routes from missing records on 404, normalize standalone backfill ranges to UTC `Z`, expose all subcommands in help, and render artifact headers before titles with consistent session-only wording (#885).
+- New Cloud commands use deliberate human summaries/tables and actionable empty states at 80/140 columns without raw response blobs. Evaluator previews display before writes and require explicit `--force` after failures; plan/claim deny key mode, deploy specs can omit the positional key, and older settings-route 404s degrade clearly. Existing JSON payloads stay unchanged (#885).
+- `fp evaluators deploy` keeps the previous version's condition and labels unless `--condition`/`--label` change them, so a new version no longer silently widens to every session (#885).
+- Evaluation backfill covers code, LLM judge and Jev evaluators, as FailproofAI Cloud now does: `fp evaluators deploy --backfill` no longer refuses judge/Jev, and both commands warn that judge/Jev backfills spend model budget instead of saying they are skipped.
+- `fp usage windows` says it needs an API key (the dashboard has no session route for it) instead of reporting the route as unavailable.
+- `fp issues close --help` no longer claims exit 9 for an issue that already ended: it is exit 1, as it always was, and it names `fp audits dismiss` rather than a command that does not exist.
 - `fp fleet history` keeps `gen`, `when` and `jev` visible at 80 columns (rollback needs `gen`): the policy list is shortened, then dropped, and `change` wraps first.
 - `fp fleet show` and `fp fleet history` name an exact-profile target by the profile label the machine reports (`claude/.claude-work`, with the `agt_…` id kept in `show`), falling back to the bare id.
 

package/.next/standalone/fp-cloud-cli/README.md

@@ -211,6 +211,126 @@ These commands are **session-only** (`fp login`). They are absent from the
 versioned API an API key authenticates against, so `--api-key` exits 2 with the
 reason rather than failing at the request.
 
+## Author an evaluator
+
+`fp evaluators` authors hosted definitions; `fp evals` continues to read results.
+Creation enables a definition by default. Check `fp evaluators instances` first:
+Cloud needs an enabled managed instance. Writes need `evaluations:trigger`;
+session dry-runs and `evaluators schema` also need `events:read`.
+
+Write `error_free.py` as a restricted Python expression (no imports or functions):
+
+```python
+EvalResult(score=Score(1.0 if session.count("error") == 0 else 0.0), reasoning="Error-free session")
+```
+
+```bash
+fp --json evaluators create --key error_free --name "Error free" --code error_free.py \
+  --result-kind score --dry-run-on run-001
+fp --json evaluators dry-run --code error_free.py --session run-001
+fp --json evaluators deploy error_free --code error_free.py --version 2 --dry-run-on run-001
+fp --json evaluators list --key error_free --all
+```
+
+LLM judge (Cloud selects the model; judge dry-runs are refused):
+
+```bash
+fp --json evaluators create --key relevance --name Relevance --judge \
+  --criteria "Grade how directly the answer addresses the request" --threshold 0.7
+fp --json evaluators deploy relevance --version 2 --judge \
+  --criteria "Grade relevance and completeness" --threshold 0.8
+fp --json sessions re-evaluate run-001
+fp --json sessions history run-001
+```
+
+Jev binary or graded rubric:
+
+```bash
+fp --json evaluators create --key resolved --name Resolved --jev noul \
+  --instructions "Was the request resolved?" --true "Resolved" --false "Unresolved"
+fp --json evaluators dry-run --jev noul --instructions "Was the request resolved?" \
+  --true "Resolved" --false "Unresolved" --session run-001
+fp --json evaluators deploy resolved --version 2 --jev score \
+  --instructions "Grade resolution quality" \
+  --level "Unresolved" --level "Partly resolved" --level "Fully resolved"
+```
+
+Jev dry-runs spend model budget. Rubrics need 3–5 distinct levels, worst first.
+Thresholds are 0–1; judge/Jev always return a score. `deploy` creates the new
+immutable version before disabling older enabled versions. If disabling fails,
+the CLI names the published id; inspect `list --all` and recover explicitly.
+`--backfill 7d` works for every kind (judge/Jev spend model budget); `evaluators backfill --from T --to T` accepts ≤90 days and `--dry-run` counts the work first.
+
+Human output shows each `--dry-run-on` session's outcome before creation or
+deployment. Any failed run stops the write; after reviewing the failure, an
+explicit `--force` overrides this guard. Existing `--json` response shapes stay
+unchanged.
+
+AI drafting is session-only and writes an API spec that create/deploy accept:
+
+```bash
+fp login
+fp --json evaluators draft "Measure error-free sessions with code" --kind code --out evaluator.json
+# Review the spec, especially eval_key and evaluator_source, before enabling it.
+fp --json evaluators create --file evaluator.json --dry-run-on run-001
+fp --json evaluators dry-run --file evaluator.json
+```
+
+Spec fields: `eval_key`, `display_name`, `eval_version`, `result_kind`, `labels`,
+`condition_source`, `evaluator_source`, `timeout_seconds`. Judge/Jev sources are
+JSON envelope **strings**, not nested objects. `--file` is exclusive with discrete
+spec flags; deploy may override a file's version. Files written by the CLI are
+created exclusively, so existing files are never silently replaced.
+See [the command reference](skill/references/commands.md#author-an-evaluator).
+
+## Dashboard parity
+
+```text
+fp dashboards list|show|create|update|delete
+fp dashboards tiles <id> list|add|update|remove|layout
+fp sessions show|export|history|re-evaluate <id>
+fp events latency
+fp sentiment summary|messages|sessions|session <id>
+fp query compose "question" [--run]                  # session
+fp query export [<id>] [--sql SQL] --format csv|json [--out FILE]
+fp alerts recipients
+fp alerts issues <id> [--create --title ... --summary ...]
+fp audits contracts list|show|create|update
+fp policies author "intent"|backtest <file-or-id>|candidacy --issue <id>  # session
+fp agent brief [--range 24h|7d]                      # session
+fp orgs create --name N [--slug S]                   # session
+fp roles list|show|create|update|delete
+fp users delete <id>
+fp users defaults
+fp usage windows
+fp plan                                           # session
+fp plan claim <offer>                             # session
+fp settings context-windows list|set <model> <tokens>|delete <model>
+fp otel status|enable|disable
+fp slack status|disconnect                          # session
+```
+
+`fp agent ask` runs artifact SQL through the permission-gated query runner and
+renders tables/stats. It prompts before approving a write on a terminal.
+Scripts and `--json` **decline** unless you explicitly pass `--approve`.
+Normal create/update commands keep their existing non-interactive confirmation
+behavior; assistant approvals are a separate safety boundary.
+
+All new commands accept the global `--json` before the command. Session-only
+commands fail before HTTP in API-key mode, with exit 2 and a `fp login` hint.
+Unknown-route 404s exit 6 with “not available on this FailproofAI Cloud yet”.
+Missing-record 404s also exit 6 but report that the record was not found.
+`plan` and `plan claim` require a signed-in person and deny API-key mode before
+HTTP with the standard `fp login` message.
+Human screens use the mint/pink wordmark, grey labels, open tables and full ids;
+existing JSON shapes are unchanged.
+
+Human commands show summaries, tables and wrapped text rather than response
+objects. Use global `--json` for raw API results. For `sessions export` or
+`query export --format json`, use `--out FILE` to save the JSON; without either
+option, the terminal shows a readable session or query view. CSV exports remain
+plain CSV for piping.
+
 ## Configuration
 
 | Setting | Flag | Env var | Default |

package/.next/standalone/fp-cloud-cli/fp_cli/_version.py

@@ -1 +1 @@
-__version__ = "0.0.2b0"
+__version__ = "0.0.3b0"

package/.next/standalone/fp-cloud-cli/fp_cli/app.py

@@ -17,12 +17,15 @@ from . import orgs as orgsmod
 from ._context import AppState
 from ._version import __version__
 from .commands import (
+    account_cmds,
     agent_cmds,
     alerts_cmds,
     audits_cmds,
     auth_cmds,
+    dashboards_cmds,
     errors_cmds,
     evals_cmds,
+    evaluators_cmds,
     events_cmds,
     fleet_cmds,
     guardrails_cmds,
@@ -33,6 +36,7 @@ from .commands import (
     policies_cmds,
     queries_cmds,
     sessions_cmds,
+    sentiment_cmds,
     settings_cmds,
     usage_cmds,
     users_cmds,
@@ -128,6 +132,12 @@ app = typer.Typer(
 # function Typer calls for this; falls back silently if its internals change.
 try:  # pragma: no cover - exercised via real command output, not unit assertions
     import typer.rich_utils as _rich_utils
+    # Keep generated subcommand help on the same shared screen primitives.
+    _rich_utils.Panel = output.Panel
+    _rich_utils.Table = output.Table
+    _rich_utils.STYLE_OPTION = output.theme.PINK
+    _rich_utils.STYLE_COMMANDS_TABLE_FIRST_COLUMN = output.theme.PINK
+    _rich_utils.STYLE_OPTION_ENVVAR = output.theme.LABEL
 
     import re as _re
 
@@ -327,7 +337,7 @@ def main(
             "must be a positive number of seconds.", param_hint="--timeout"
         )
     cfg = cfgmod.load_config()
-    output.configure(no_color=no_color, quiet=quiet, json=json_output)
+    output.configure(no_color=no_color, quiet=quiet, json=json_output, context=ctx.invoked_subcommand or "Cloud")
     # Resolve the credential BEFORE telemetry starts: the distinct-id depends on it.
     # `get_parameter_source` is what separates a flag from its env var — the values
     # alone cannot, and the precedence rules differ between the two (see resolve_auth).
@@ -452,6 +462,9 @@ orgs_cmds.register(app)
 events_cmds.register(app)
 sessions_cmds.register(app)
 evals_cmds.register(app)
+evaluators_cmds.register(app)
+dashboards_cmds.register(app)
+sentiment_cmds.register(app)
 errors_cmds.register(app)
 usage_cmds.register(app)
 list_cmds.register(app)
@@ -466,6 +479,7 @@ agent_cmds.register(app)
 policies_cmds.register(app)
 fleet_cmds.register(app)
 guardrails_cmds.register(app)
+account_cmds.register(app)
 
 
 def _elapsed_ms(start: float) -> int:

package/.next/standalone/fp-cloud-cli/fp_cli/client.py

@@ -31,6 +31,7 @@ from .errors import (
     ApiError,
     AuthError,
     ForbiddenError,
+    FpCliError,
     KeyModeUnsupportedError,
     NetworkError,
     NotFoundError,
@@ -134,6 +135,14 @@ _V1_MECHANICAL_FAMILIES = frozenset(
         "alerts",
         "audits",
         "evaluations",
+        "evaluator-definitions",
+        "evaluator-instances",
+        "evaluator-schema-profile",
+        "evaluation-jobs",
+        "dashboards",
+        "otel",
+        "offers",
+        "team-trial",
         "events",
         "issues",
         "keys",
@@ -152,6 +161,7 @@ _V1_MECHANICAL_FAMILIES = frozenset(
 # family rule, which is why this must stay exact-match.
 _V1_RENAMED = {
     "/api/evaluations/score-keys": "/v1/evaluations/score_keys",
+    "/api/evaluations/metric-keys": "/v1/evaluations/metric_keys",
 }
 
 # Families with no `/v1` route, and why — the message a user actually sees.
@@ -173,6 +183,8 @@ _V1_NO_EQUIVALENT = {
         "cloud-managed policies are an operator surface — the fleet routes are "
         "deliberately absent from /v1, which is internet-facing"
     ),
+    "integrations": "dashboard integration routes need a signed-in session",
+    "orgs": "organization creation needs a signed-in user",
 }
 
 
@@ -375,6 +387,40 @@ def _required_permission(response: httpx.Response) -> Optional[str]:
     return None
 
 
+def _unknown_route_response(response: httpx.Response) -> Optional[bool]:
+    """Use explicit 404 details first; an empty/generic reply is ambiguous."""
+    try:
+        data = response.json()
+    except ValueError:
+        return None
+    if not isinstance(data, dict):
+        return None
+    code = data.get("code") or data.get("error_code") or data.get("error") or ""
+    if isinstance(code, dict):
+        code = code.get("code", "")
+    code = str(code).lower()
+    if code in ("unknown_route", "route_not_found", "unknown_endpoint", "endpoint_not_found"):
+        return True
+    if code in ("record_not_found", "resource_not_found", "entity_not_found", "session_not_found", "query_not_found"):
+        return False
+    message = _extract_error(response)
+    if not message:
+        return None
+    lower = message.lower().strip().rstrip(".")
+    if re.search(
+        r"\b(?:unknown|unrecognized|unsupported|missing|no such)\s+(?:api\s+)?(?:route|endpoint)\b"
+        r"|\b(?:route|endpoint)\b.*\b(?:not found|not available|does not exist|absent|unknown)\b"
+        r"|\bcannot (?:get|post|put|patch|delete)\s+/",
+        lower,
+    ):
+        return True
+    if lower in ("not found", "not_found", "404", "404 not found"):
+        return None
+    # A specific response ("session not found", "offer expired", etc.) came
+    # from a handler, rather than the router. Do not replace its useful detail.
+    return False
+
+
 def _raise_for_status(response: httpx.Response, ctx: ClientContext) -> None:
     key_mode = ctx.auth_mode is AuthMode.API_KEY
     if response.status_code < 400:
@@ -474,7 +520,9 @@ def _raise_for_status(response: httpx.Response, ctx: ClientContext) -> None:
                     request_id=request_id,
                     hint="point --base-url at the server itself, e.g. http://localhost:8080",
                 )
-        raise NotFoundError(message or "Not found.", request_id=request_id)
+        error = NotFoundError(message or "Not found.", request_id=request_id)
+        error.route_unknown = _unknown_route_response(response)
+        raise error
     if response.status_code == 429:
         retry_after = response.headers.get("retry-after")
         wait = f" Retry after {retry_after}s." if retry_after else " Please wait a moment and try again."
@@ -606,6 +654,121 @@ def _delete(ctx: ClientContext, path: str) -> Any:
     return _request_json(ctx, "DELETE", path)
 
 
+def cloud_request(
+    ctx: ClientContext, method: str, path: str, *,
+    body: Any = None, params: Optional[Dict[str, Any]] = None,
+) -> Any:
+    """New Cloud capabilities, with a predictable older-deployment failure.
+
+    Existing callers retain their response and error contracts. Paths still pass
+    through the credential and identifier guards; this is not a second transport.
+    """
+    try:
+        if method == "GET":
+            return _get_json(ctx, path, params)
+        return _request_json(ctx, method, path, json_body=body, params=params)
+    except (NotFoundError, ApiError) as exc:
+        if isinstance(exc, ApiError) and exc.status != 404:
+            raise
+        raise _cloud_not_found(exc, path, body=body, params=params) from exc
+
+
+def _cloud_not_found(exc: FpCliError, path: str, *, body=None, params=None) -> NotFoundError:
+    """Classify ambiguous 404s from the request; never claim a record is a route.
+
+    Top-level collections and named capability endpoints have no record segment.
+    Scoped collections (/dashboards/ID/tiles) and actions on records default to a
+    missing record unless the response explicitly says the route is unknown.
+    """
+    route_unknown = getattr(exc, "route_unknown", None)
+    if isinstance(exc, ApiError):
+        # The transport only raises HTTP 404 ApiError for an HTML front door.
+        route_unknown = True
+    if route_unknown is None:
+        segments = path.removeprefix("/api/").split("/")
+        family = segments[0]
+        capability = len(segments) == 1 or (
+            family == "agent" and len(segments) == 2
+            or family == "otel" and segments == ["otel", "status"]
+            or family == "offers" and segments[1:] in (["me"], ["claim"])
+            or family == "team-trial" and segments == ["team-trial", "me"]
+            or family == "integrations" and segments == ["integrations", "slack"]
+            or family == "events" and segments in (
+                ["events", "summary"], ["events", "latency_aggregate"],
+                ["events", "sentiment"], ["events", "sentiment", "messages"],
+                ["events", "sentiment", "sessions"],
+            )
+            or family == "evaluator-definitions" and segments == ["evaluator-definitions", "dry-run"]
+            or family == "evaluations" and segments == ["evaluations", "backfill"]
+            or family == "queries" and segments[1:] in (["run"], ["schema"])
+            or family == "audits" and segments == ["audits", "contracts"]
+            or family == "alerts" and segments == ["alerts", "recipients"]
+            or family == "users" and segments == ["users", "defaults"]
+            or family == "usage" and segments == ["usage", "windows"]
+            or family == "settings" and segments == ["settings", "model-context-windows"]
+            or family == "settings" and len(segments) == 2
+        )
+        # Some collection operations name their target in a query or body.
+        request_body = body if isinstance(body, dict) else {}
+        record_selected = (
+            (params or {}).get("session_id") or (params or {}).get("model")
+            or request_body.get("query_id") or request_body.get("callId")
+            or request_body.get("session_ids") or request_body.get("offer_key")
+            or request_body.get("conversationId")
+        )
+        route_unknown = capability and not record_selected
+    if route_unknown:
+        return NotFoundError(
+            "not available on this FailproofAI Cloud yet",
+            hint="upgrade your Cloud deployment to use this route",
+            request_id=exc.request_id,
+        )
+    message = exc.message
+    if message.lower().strip().rstrip(".") in (
+        "not found", "not_found", "record_not_found", "resource_not_found",
+        "entity_not_found", "session_not_found", "query_not_found",
+    ):
+        message = f"Record not found at {path}"
+    return NotFoundError(message, request_id=exc.request_id)
+
+
+def import_events_file(
+    ctx: ClientContext, data: bytes, params: Dict[str, Any],
+) -> Any:
+    """POST /api/events/import — one file's bytes (gzip accepted), as the dashboard's
+    import dialog sends it. ``params``: format, path, environment, dry_run, existing_before."""
+    clean = {k: v for k, v in params.items() if v is not None} or None
+    url = _path(ctx, "/api/events/import")
+    try:
+        with _client(ctx) as client:
+            response = client.request(
+                "POST", url, content=data, params=clean,
+                headers={"Content-Type": "application/octet-stream"},
+            )
+    except httpx.RequestError as exc:
+        raise NetworkError(
+            f"Cannot reach FailproofAI Cloud at {ctx.base_url}: {exc}",
+            request_id=_sent_request_id(exc),
+        )
+    _raise_for_status(response, ctx)
+    if not response.content:
+        return {}
+    try:
+        return response.json()
+    except ValueError:
+        raise ApiError("The dashboard returned an invalid import response.")
+
+
+def cloud_stream(ctx: ClientContext, path: str, body: Dict[str, Any]) -> Iterator[Dict[str, Any]]:
+    """Dashboard SSE with the same additive-capability 404 contract."""
+    try:
+        yield from _stream_sse(ctx, path, body)
+    except (NotFoundError, ApiError) as exc:
+        if isinstance(exc, ApiError) and exc.status != 404:
+            raise
+        raise _cloud_not_found(exc, path, body=body) from exc
+
+
 # --- Auth / identity --------------------------------------------------------
 
 
@@ -793,6 +956,10 @@ _FACET_PATHS = {
     # Evaluation score keys (a distinct endpoint, not /api/events) — the source for
     # the sessions-page score-filter dropdown; needs `evaluations:read`.
     "score_filters": "/api/evaluations/score-keys",
+    # Evaluation-side facets: environments with evaluation results, and metric
+    # keys (the server returns {name, unit, count}; `list` shows the names).
+    "eval_environments": "/api/evaluations/environments",
+    "metric_keys": "/api/evaluations/metric-keys",
 }
 FACET_KINDS = tuple(_FACET_PATHS.keys())
 
@@ -800,7 +967,9 @@ FACET_KINDS = tuple(_FACET_PATHS.keys())
 def list_facet(ctx: ClientContext, kind: str) -> List[str]:
     """GET /api/events/<kind> — distinct facet values (a bare JSON array)."""
     data = _get_json(ctx, _FACET_PATHS[kind])
-    return [str(x) for x in data] if isinstance(data, list) else []
+    if not isinstance(data, list):
+        return []
+    return [str(x.get("name", "")) if isinstance(x, dict) else str(x) for x in data]
 
 
 def get_usage(ctx: ClientContext) -> Dict[str, Any]:
@@ -1195,20 +1364,28 @@ def list_incidents(
     state: Optional[str] = None,
     alert_id: Optional[str] = None,
     limit: Optional[int] = None,
+    filters: Optional[Dict[str, Any]] = None,
 ) -> List[Incident]:
     """GET /api/issues (or /api/alerts/{id}/issues when alert_id given). The
     alert-scoped path honours ``state``/``limit`` too — pass them so the filters aren't silently
     dropped on that path (``_get_json`` omits None params)."""
-    if alert_id:
+    extra = {k: v for k, v in (filters or {}).items() if v is not None}
+    if alert_id and not extra:
         data = _get_json(ctx, f"/api/alerts/{alert_id}/issues", {"state": state, "limit": limit})
+    elif alert_id:
+        # The board's own filters live on /issues, which takes alert_id as one of them.
+        data = _get_json(ctx, "/api/issues", {"state": state, "limit": limit, "alert_id": alert_id, **extra})
     else:
-        data = _get_json(ctx, "/api/issues", {"state": state, "limit": limit})
+        data = _get_json(ctx, "/api/issues", {"state": state, "limit": limit, **extra})
     return [Incident.from_dict(i) for i in (data if isinstance(data, list) else [])]
 
 
-def count_incidents(ctx: ClientContext, *, state: Optional[str] = None) -> int:
+def count_incidents(
+    ctx: ClientContext, *, state: Optional[str] = None, filters: Optional[Dict[str, Any]] = None,
+) -> int:
     """GET /api/issues/count — {count}."""
-    data = _get_json(ctx, "/api/issues/count", {"state": state})
+    extra = {k: v for k, v in (filters or {}).items() if v is not None}
+    data = _get_json(ctx, "/api/issues/count", {"state": state, **extra})
     if not isinstance(data, dict):
         return 0
     try:
@@ -1532,6 +1709,8 @@ def agent_chat_oneshot(
     conversation_id: Optional[str] = None,
     page_context: Optional[str] = None,
     model: Optional[str] = None,
+    on_ask_user: Optional[Callable[[Dict[str, Any]], None]] = None,
+    on_artifact: Optional[Callable[[Dict[str, Any]], None]] = None,
 ) -> Dict[str, Any]:
     """POST /api/agent/chat — accumulate the streamed answer.
 
@@ -1551,20 +1730,32 @@ def agent_chat_oneshot(
         body["pageContext"] = page_context
     if model:
         body["model"] = model
+    if on_artifact:
+        body["pageActions"] = [{"action": "open_artifact"}, {"action": "home"}]
     parts: List[str] = []
     tools: List[str] = []
     interrupted = False
     error: Optional[str] = None
-    for ev in _stream_sse(ctx, "/api/agent/chat", body):
+    for ev in cloud_stream(ctx, "/api/agent/chat", body):
         kind = ev.get("type")
         if kind == "text-delta":
             parts.append(str(ev.get("text", "")))
         elif kind == "tool-start":
             tools.append(str(ev.get("tool", "")))
         elif kind == "ask-user":
+            if on_ask_user and ev.get("callId"):
+                on_ask_user(ev)
+                continue
             interrupted = True
             error = str(ev.get("question") or "the assistant needs interactive input")
             break
+        elif kind == "ui-action" and on_artifact:
+            if ev.get("action") == "open_artifact" and isinstance(ev.get("artifact"), dict):
+                on_artifact(ev["artifact"])
+            elif ev.get("action") == "home" and isinstance(ev.get("home"), dict):
+                for block in ev["home"].get("blocks", []):
+                    if isinstance(block, dict):
+                        on_artifact(block)
         elif kind == "error":
             error = str(ev.get("message", "assistant error"))
         elif kind == "done":