failproofai 1.0.1-beta.0 → 1.0.1

package/.next/standalone/docs-old/agenteye/audits.mdx

@@ -0,0 +1,54 @@
+---
+title: "Audits: your automatic reliability analyst"
+description: "Failproof AI Observability goes looking for the failures you never wrote a rule for and hands you a ranked, evidence-backed to-do list of exactly what to fix."
+---
+
+
+Failproof AI Observability goes looking for the failures you never wrote a rule for and hands you a ranked, evidence-backed to-do list of exactly what to fix. It is like having an analyst comb your logs every night, then leaving the short list on your desk by morning.
+
+<div style={{ position: "relative", width: "100%", paddingBottom: "56.25%", height: 0, overflow: "hidden", borderRadius: "12px", margin: "1.5rem 0" }}>
+  <iframe src="https://www.youtube.com/embed/C5nHKGQIqZQ?autoplay=1&mute=1&loop=1&playlist=C5nHKGQIqZQ&rel=0&playsinline=1" title="Watch: Automatically find where your agent is failing (2 min)" allow="autoplay; encrypted-media; picture-in-picture; fullscreen" allowFullScreen style={{ position: "absolute", top: 0, left: 0, width: "100%", height: "100%", border: 0 }}></iframe>
+</div>
+
+*A two-minute tour: from a scheduled run to a fix you can act on.*
+
+![The Audits page: recurring jobs that scan your sessions for failure patterns, each with a schedule and sensitivity](/agenteye/images/audits.png)
+*Each audit is a recurring job that mines your sessions and writes up ranked, evidence-backed recommendations.*
+
+## Stop guessing what to fix next
+
+Alerts catch the problems you already know to watch for. Audits catch the ones you don't. On a schedule you set, an audit reads across all of your agent sessions and hunts for the patterns worth fixing, so you spend your time acting on findings instead of scrolling logs hoping to spot them yourself.
+
+A single run goes after the failure modes that actually break agents in production:
+
+- **Error clusters**: the same failure repeating under a shared root cause.
+- **Drift versus a baseline**: behaviour quietly sliding away from a known-good window.
+- **Goal failure in transcripts**: runs that technically finished but never did the job.
+- **Tool misuse**: the wrong tool, bad arguments, or loops that burn calls.
+- **Quality and cost trade-offs**: where you are overpaying for output you could get cheaper.
+- **Coverage gaps**: behaviour that no eval or alert is watching.
+
+You decide how hard it looks with a single **sensitivity** setting (low, medium, or high), so a noisy staging agent and a locked-down production one can each be tuned to the signal you want.
+
+## Every recommendation comes with receipts
+
+You never have to take a finding on faith. Each recommendation cites the exact sessions it came from and the SQL that surfaced it, so you can open the evidence and confirm the problem in a click instead of reverse-engineering a claim.
+
+When a finding is about a leaked credential, it goes one step further and links the individual events it matched. Click one and you land on that exact moment in the session, already selected — not the top of a long transcript to scroll through. The link names the event; it never copies the detected secret into the finding, so reading a finding is not a second place your credential is written down. If an event is no longer there because the session has passed your retention window, the page says so plainly rather than leaving you wondering whether you clicked the wrong thing.
+
+That is also what keeps audits honest. The server checks that every cited session actually exists and **discards any recommendation whose evidence does not hold up**, so the audit investigates but never invents. What lands on your list is real, reproducible, and ranked by how much it matters, with the biggest wins at the top.
+
+## Turn a fix into a guardrail
+
+Fixing an issue is only half the win. The other half is making sure it cannot quietly come back. Every finding carries a **one-click shortcut that drafts a recurrence alert**, prefilled with a sensible starting trigger you can tune. Close the finding, arm the alert, and the next time that pattern reappears you get paged instead of rediscovering it in a future audit.
+
+## Where to find it
+
+Audits live in the dashboard at **`/<org-slug>/audits`** (sidebar to *analyze* to *audits*). Viewing runs and findings needs **`audits:read`**; creating, editing, and triaging audits needs **`audits:write`**. Set an audit's scope and cadence, then hit **Run now** whenever you want results immediately instead of waiting for the next scheduled pass.
+
+## Related
+
+- [Alerts](/agenteye/alerts): get paged the moment a threshold you already know about is crossed.
+- [Evaluations](/agenteye/evaluations): score every run so quality regressions surface on their own.
+- [Error tracking](/agenteye/error-tracking): group and follow the errors your agents throw.
+- [Incidents](/agenteye/incidents): track an issue an audit turns up through to its fix.

package/.next/standalone/docs-old/agenteye/cli-and-agents.mdx

@@ -0,0 +1,80 @@
+---
+title: "CLI"
+description: "Your entire Failproof AI Observability deployment, one command away."
+---
+
+
+Your entire Failproof AI Observability deployment, one command away. Check production, cut an API key, or ack an incident without leaving your terminal, then script any of it into CI, or let a coding agent do it for you in plain English.
+
+```bash
+pipx install agenteye
+agenteye login --email you@example.com      # a 6-digit code lands in your inbox
+agenteye --json sessions --since 24h         # every agent run from the last day, newest first
+```
+
+*The `agenteye` CLI talks to your dashboard. It is a different tool from the collector, which ships events to the server.*
+
+## Your whole deployment, one command away
+
+Stop tab-hopping to answer a quick question. The `agenteye` CLI reads your data and administers your org from a single binary, so a check that used to mean clicking through the dashboard becomes one line you can rerun, alias, or paste into a runbook. You get four surfaces:
+
+- **Read your data:** `sessions`, `events`, `evals`, and `errors`, filtered by time, agent, and environment.
+- **Manage your org:** `keys`, `users`, `settings`, `alerts`, and `incidents`.
+- **Run analytics:** saved SQL plus an ad-hoc `query` runner over your event data.
+- **Ask the assistant:** `agent ask` reaches the same read-only analyst you chat with in the dashboard.
+
+Install it once with `pipx`, sign in with an emailed 6-digit code, and you are ready. The session lasts about a day; rerun `agenteye login` when it expires. Reach for it to spot-check production, provision a key, or triage a firing incident, all without opening a browser:
+
+```bash
+agenteye errors --since 24h --aggregate      # what is breaking, grouped by error type
+agenteye incidents list --state firing       # what is on fire right now
+agenteye keys create ci --add events:add     # a key that can only push events, secret shown once
+```
+
+One habit to know: global options like `--json` go before the command. `agenteye --json sessions` is right; `agenteye sessions --json` is not.
+
+## Script it, wire it into CI
+
+Every command takes `--json`, and that changes everything. Clean JSON goes to stdout while human status and warnings go to stderr, so a `--json` capture pipes straight into `jq` with no stray line to strip. That is what makes the CLI equally good for you at a prompt and for a coding agent parsing output:
+
+```bash
+agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id'
+```
+
+It is built to run unattended. Confirmation prompts auto-skip when no terminal is attached, so nothing hangs in a pipeline, and every command returns a meaningful exit code: `0` success, `4` not logged in, `5` missing a permission (the message names it, for example `alerts:write`), `3` dashboard unreachable. A script can branch on a `4` to reauthenticate or a `5` to tell you exactly what to ask an admin for, instead of failing blind.
+
+## Let a coding agent drive it in plain English
+
+Better yet, you should not have to remember any of these flags at all. The **CLI skill** is a small Agent Skill folder named `agenteye-cli` that teaches a coding agent such as Claude Code or Codex to drive the CLI from plain-English requests. Ask "is anything broken today?" and the agent picks the command, runs it as you, and answers in prose.
+
+For Claude Code, drop the `agenteye-cli` folder into `~/.claude/skills/` and it is auto-discovered. Failproof AI Observability provides the folder; there is nothing extra to install, because it only drives the CLI you already installed. Log in yourself first: the skill cannot complete the emailed-code login for you.
+
+Because the agent runs the CLI as you, it can do everything your login permits, reads and writes alike: create keys, change settings, resolve incidents. The CLI's "are you sure?" prompt does not fire for an agent, so the skill is written to state the exact command and wait for your OK before any change. You are the confirmation step.
+
+```text
+you    Why did session run-001 fail?
+
+agent  Running: agenteye --json events --session-id run-001 --all
+       The checkout-agent hit a TimeoutError on its third tool call.
+```
+
+Reads stay instant, and every write pauses for you:
+
+```text
+you    Give CI a key that can only push events.
+
+agent  This creates an API key. I'll run:
+         agenteye keys create ci --add events:add
+       OK to proceed?
+
+you    yes
+
+agent  Done. Key "ci" created with events:add only. The secret is shown once, so store it now.
+```
+
+## Related
+
+- [CLI reference](/agenteye/cli): every command, flag, and JSON shape.
+- [CLI recipes for agents](/agenteye/cli-recipes): copy-paste `jq` patterns and exit-code handling.
+- [CLI agent skill](/agenteye/cli-skill): install and run the `agenteye-cli` skill.
+- [AI assistant](/agenteye/assistant): the in-dashboard analyst that `agent ask` talks to.

package/.next/standalone/docs-old/agenteye/cli-recipes.mdx

@@ -0,0 +1,179 @@
+---
+title: "CLI recipes for agents"
+description: "Copy-paste query patterns and jq recipes that turn session, event, and evaluation data into something a script or coding agent can automate."
+---
+
+
+Pull session, event, and evaluation data (and trigger re-evaluations) straight from a script or coding agent, with clean JSON on stdout that pipes directly into `jq`. These recipes turn Failproof AI Observability's data into something a terminal user or an AI coding agent (Claude Code, Cursor) can query and automate, without clicking through the dashboard.
+
+The patterns below are copy-paste ready for the Failproof AI Observability CLI (`agenteye`). For installation, authentication, and the full option list see [CLI](/agenteye/cli); run `agenteye -h` or `agenteye <command> -h` for the built-in help.
+
+## Golden rules
+
+1. **Global options go *before* the command.** `agenteye --json sessions` is correct; `agenteye sessions --json` is not. The globals are `--json`, `--base-url`, `--org`, `--token`, `--insecure`/`--secure`, `--timeout`, `--quiet`, `--no-color`.
+2. **Pass `--json` whenever you parse output.** Data goes to **stdout** as JSON; human status and errors go to **stderr**, so stdout stays clean to pipe into `jq`.
+3. **Branch on the exit code**, not on stderr text: `0` ok · `1` unexpected error · `2` bad arguments · `3` cannot reach the dashboard · `4` not logged in or expired · `5` missing permission · `6` resource not found.
+4. **Discover with `-h`.** Every command documents its filters, value formats, and JSON shape.
+
+## One-time setup
+
+```bash
+export AGENTEYE_DASHBOARD_URL=https://agenteye.example.com   # so you don't repeat --base-url
+agenteye login --email you@example.com                       # paste the emailed code; valid ~24h
+```
+
+## Confirm auth before doing work
+
+`whoami` never errors on a missing or expired session; it reports `logged_in:false` instead, so an agent can probe auth state safely. (It can still exit non-zero if no base URL is set or the dashboard is unreachable.)
+
+```bash
+if [ "$(agenteye --json whoami | jq -r .logged_in)" != "true" ]; then
+  echo "Not authenticated. Run: agenteye login" >&2; exit 1
+fi
+```
+
+## Find failing or low-scoring sessions
+
+```bash
+# sessions in the last 24h whose evaluation errored
+agenteye --json sessions --since 24h --status error | jq -r '.sessions[].session_id'
+
+# evaluations scoring <= 0.5 on helpfulness, for one agent
+agenteye --json evals --agent-id checkout-bot --score helpfulness:..0.5 \
+  | jq '.evaluations[] | {session_id, scores}'
+```
+
+Score filtering lives on **`evals`**, not `sessions`. `--score KEY:MIN..MAX` is repeatable and AND-combined; either bound is optional (`..0.5` means ≤ 0.5, `0.9..` means ≥ 0.9). You can pass up to 20 score filters per request; more returns HTTP 400. `sessions` shares the `--env`, `--status`, `--agent-id`, `--session-id`, and time-range filters with `evals`, but has no `--score`.
+
+## Read one session end-to-end
+
+There is no single `session show` command. Combine the event trail with the session's evaluation:
+
+```bash
+# the session's latest evaluation (status + scores)
+agenteye --json evals --session-id run-001 | jq '.evaluations[0] | {status, scores}'
+
+# every event in the run (raise --limit for a full sweep)
+agenteye --json events --session-id run-001 --all --limit 1000 | jq '.events[] | {ts, event_type}'
+
+# just the tool calls in a session (--full is required to get the raw payload)
+agenteye --json events --full --session-id run-001 --event-type tool_use,tool_result --all \
+  | jq '.events[].payload'
+```
+
+> **Note:** By default, `events` reads a fast, payload-free feed. Each event carries a server-computed one-line `summary` plus flags like `is_error` and token counts, but `payload` comes back as `{}`. To pull the raw payload, add `--full` (or `--fields payload`). The full feed is slower at scale, so keep it bounded: pair `--full` with a single `--session-id`.
+
+## Fetch everything (pagination)
+
+Results are newest-first and cursor-paginated.
+
+```bash
+# one shot: fetch up to 500 rows in 200-row pages
+agenteye --json events --session-id run-001 --limit 500 --all > events.json
+
+# manual paging: feed next_cursor back in
+page=$(agenteye --json events --limit 100)
+cursor=$(echo "$page" | jq -r '.next_cursor // empty')
+[ -n "$cursor" ] && agenteye --json events --limit 100 --cursor "$cursor"
+```
+
+## Slim the output with --fields
+
+Restrict the keys (in both the table and `--json`) to reduce what an agent must read.
+
+```bash
+agenteye --json sessions --since 7d --fields session_id,status,scores | jq -c '.sessions[]'
+agenteye --json events --session-id run-001 --fields ts,event_type --all
+```
+
+Unknown field names are rejected (exit `2`) with the valid list, a cheap way to discover field names.
+
+## Discover valid filter values
+
+```bash
+agenteye --json list envs | jq -r '.values[]'           # values for --env
+agenteye --json list tools | jq -r '.values[]'          # tool names; also agents, models, event_types, …
+agenteye --json list score_filters | jq -r '.values[]'  # valid KEY for --score KEY:MIN..MAX
+```
+
+## Pick your org (multi-tenant)
+
+If you belong to more than one org, choose the active tenant at login (it's saved):
+
+```bash
+agenteye login --org acme --email you@corp.com   # set the tenant in the same step as login
+agenteye --json orgs list | jq -r '.orgs[].org_slug'
+agenteye --org globex --json sessions --since 24h   # override for one command
+```
+
+A multi-org login without `--org` exits non-zero and prints the orgs to choose from.
+
+## Provision an API key for the SDK/collector
+
+```bash
+# the secret is printed ONCE, with --json it's the .key field
+key=$(agenteye --json keys create ci-bot --add events:read.add | jq -r '.key')
+agenteye keys regenerate ci-bot --yes    # rotate; agenteye keys disable ci-bot --yes to revoke
+```
+
+## Run a saved or ad-hoc query
+
+```bash
+agenteye --json query run --sql "select count(*) from analytics.events" | jq '.rows'
+agenteye --json query run errs --arg prod | jq '.rows'   # a saved query + a positional $1
+```
+
+## Triage an incident non-interactively
+
+```bash
+id=$(agenteye --json incidents list --state firing | jq -r '.incidents[0].id')
+agenteye incidents ack "$id"
+agenteye incidents assign "$id" --assignee you@corp.com
+agenteye incidents resolve "$id" --yes
+```
+
+> **Note:** Mutations auto-skip their confirmation prompt under `--json` or when stdin isn't a TTY, so agents never hang; pass `--yes`/`-y` to skip it explicitly elsewhere.
+
+## Exit-code handling in a script
+
+```bash
+out=$(agenteye --json sessions --since 1h) || code=$?
+case "${code:-0}" in
+  0) echo "$out" | jq '.sessions | length' ;;
+  4) echo "Session expired - run 'agenteye login'." >&2 ;;
+  5) echo "Missing permission (ask an admin for evaluations:read)." >&2 ;;
+  3) echo "Dashboard unreachable - check the URL." >&2 ;;
+  *) echo "Unexpected error (exit ${code})." >&2 ;;
+esac
+```
+
+## JSON output shapes
+
+| Command | stdout JSON (with `--json`) |
+|---|---|
+| `whoami` | `{"logged_in": true, "id", "email", "is_instance_admin", "active_org", "permissions": [...], "memberships": [...]}` or `{"logged_in": false}` |
+| `orgs list` | `{"active_org", "orgs": [{"org_slug","org_name","permission_set","permissions"}]}` |
+| `events` | `{"events": [...], "next_cursor": <cursor or null>}` |
+| `evals` | `{"evaluations": [...], "next_cursor": <cursor or null>}` |
+| `sessions` | `{"sessions": [...], "next_cursor": <cursor or null>}` |
+| `errors` | `{"errors": [...], "next_cursor": <cursor or null>}` |
+| `list <kind>` | `{"kind", "values": [...]}` |
+| `keys list` / `keys create` | `{"keys": [...]}` / `{id, name, permissions, created_at, key}` (`key` shown once) |
+| `query run` | `{columns: [{name,type}], rows: [[...]], truncated, elapsed_ms}` |
+| `users list` / `settings list` | `{"users": [...]}` / `{"settings": [...]}` |
+| `alerts list` / `incidents list` | `{"alerts": [...]}` / `{"incidents": [...]}` |
+| create/update/delete (any) | the resource object, or `{"deleted": true, "id"}` for deletes |
+| failure (any, with `--json`) | `{"error": "...", "exit_code": <n>, "status"?: <http>, "hint"?: "..."}` on stdout |
+
+- Each **event** item (`events`): `id, session_id, agent_id, event_type, ts, payload, environment, summary, is_error, error_type, output_tokens, context_window, context_fill`. Note that `payload` is `{}` unless you request the full feed with `--full` (or `--fields payload`).
+- Each **evaluation** item (`evals`): `id, session_id, agent_id, environment, status, scores, reasoning, summary, error, attempt_count, duration_ms, completed_at, created_at`.
+- Each **session** item (`sessions`): `session_id, agent_id, environment, status, scores, event_count, started_at, last_event_at, first_event_id, last_event_id, latest_evaluation`.
+
+Each command's `--fields` accepts exactly its own item's field names. The set differs between `sessions` and `evals`, so a name valid for one may be rejected by the other.
+
+## Next steps
+
+- [CLI](/agenteye/cli): installation, authentication, and the full option reference for every command.
+- [CLI agent skill](/agenteye/cli-skill): package these recipes as a skill your coding agent can load.
+- [API keys](/agenteye/api-keys): create and scope the keys the CLI, SDK, and collector authenticate with.
+- [Python SDK](/agenteye/python-sdk): send events into Failproof AI Observability so there is data for these recipes to query.

package/.next/standalone/docs-old/agenteye/cli-skill.mdx

@@ -0,0 +1,159 @@
+---
+title: "Failproof AI Observability CLI Agent Skill"
+description: "Ask your coding agent \"is anything broken today?\" and let it answer from your live Failproof AI Observability data, with no commands to memorize."
+---
+
+
+Ask your coding agent *"is anything broken today?"* and let it answer from your live Failproof AI Observability data, with no commands to memorize. The **Failproof AI Observability CLI skill** (`agenteye-cli`) is an *Agent Skill*: a small folder of instructions that a coding agent such as Claude Code or Codex loads on demand. It teaches the agent to operate your Observability deployment through the [`agenteye` CLI](/agenteye/cli) from plain-English requests like *"give CI a key that can only push events"* or *"ack the firing incident and assign it to me."*
+
+It is **not** a service or a separate binary; there is nothing to deploy. It rides on top of the CLI you have already installed: the agent shells out to `agenteye --json …`, parses the clean JSON, and answers you in prose. Everything it can do, you could do yourself by typing the same commands.
+
+---
+
+## How it relates to the other Failproof AI Observability interfaces
+
+Failproof AI Observability gives you four ways to reach the same data and controls. They complement each other:
+
+| Interface | What it is | Where it runs | Reach for it when |
+|---|---|---|---|
+| **[CLI](/agenteye/cli)** | The command/flag reference for `agenteye` | Your terminal | You want to run or script a specific command |
+| **[CLI recipes](/agenteye/cli-recipes)** | Copy-paste `jq`/pipeline patterns | Your terminal / scripts | You're wiring the CLI into automation |
+| **CLI skill** (this doc) | A natural-language front door on the CLI | Your coding agent, on your workstation | You want to *just ask* and let the agent pick the command |
+| **[Evaluator skill](/agenteye/evaluator-skill)** | A sibling skill that designs and builds your scoring service | Your coding agent, on your workstation | You want to *produce* eval scores rather than read them |
+| **[Python SDK skill](/agenteye/python-sdk-skill)** | A sibling skill that instruments your agent so it emits telemetry at all | Your coding agent, on your workstation | You want your agent to *produce* the events this skill reads |
+| **[In-dashboard AI assistant](/agenteye/assistant)** | A chat embedded in the dashboard | Server-side (in the dashboard) | You want in-dashboard Q&A over your data |
+
+The skill itself has no privileges of its own; it just turns your words into CLI calls that run as you:
+
+```mermaid
+flowchart TD
+    YOU["you: 'ack the firing incident'"] --> AGENT["coding agent (Claude Code / Codex)<br/>loads the agenteye-cli skill"]
+    AGENT --> CLI["agenteye --json incidents ack ..."]
+    CLI -->|your authenticated CLI session| API["Observability dashboard API"]
+```
+
+### vs. the in-dashboard AI assistant: an important distinction
+
+These are two different tools with very different blast radii:
+
+- The **in-dashboard AI assistant** ([AI assistant](/agenteye/assistant)) is a chat embedded in the dashboard, backed by the agent service. It is **read-only plus approval-gated authoring**: it can draft saved queries and dashboards, but every write pauses for your explicit click-approval, and it never deletes. It is gated by the `agent:use` permission and only ever sees data for the org you're viewing.
+- The **CLI skill** runs on *your* workstation inside *your* coding agent and drives the `agenteye` CLI as **you**. It can perform the CLI's **full surface, including mutations** (create/rotate/disable API keys, change org settings, resolve incidents, delete saved queries), bounded only by the permissions of your CLI login. Treat it exactly as carefully as you would treat running those commands by hand.
+
+---
+
+## Prerequisites
+
+1. The **`agenteye` CLI installed** and on `PATH` (see the [CLI](/agenteye/cli) reference: `pipx install agenteye`).
+2. Your **dashboard URL** set (`AGENTEYE_DASHBOARD_URL`, or the agent passes `--base-url`).
+3. A **logged-in session**: run `agenteye login` yourself first. The skill **cannot** complete the emailed one-time-code login for you; it will tell you to run `agenteye login` if the session is missing or expired (CLI exit code `4`).
+
+---
+
+## Where to get it
+
+The skill is published in Failproof AI's public skills collection:
+
+**[github.com/FailproofAI/skills](https://github.com/FailproofAI/skills)** → [`skills/agenteye-cli/`](https://github.com/FailproofAI/skills/tree/main/skills/agenteye-cli)
+
+Nothing about it is gated — the repository is public and the skill needs no credential of its own, because it only drives the **public** `agenteye` CLI against *your* dashboard, using the session *you* logged in with. You do not need to ask anyone for it.
+
+Note it ships as its own folder and is **not** inside the `pipx install agenteye` package, so don't look for it there.
+
+## Installing the skill
+
+The quickest path is the [`skills`](https://skills.sh) CLI, which fetches the folder and drops it where your agent looks:
+
+```bash
+# Claude Code, this project only
+npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code
+
+# every project (installs to ~/.claude/skills/)
+npx skills add FailproofAI/skills --skill agenteye-cli -a claude-code -g --copy
+
+# Codex instead
+npx skills add FailproofAI/skills --skill agenteye-cli -a codex
+```
+
+Then manage it like any other skill:
+
+```bash
+npx skills list -a claude-code      # what's installed
+npx skills update agenteye-cli      # pull the latest version
+npx skills remove agenteye-cli      # remove it
+```
+
+Prefer to install by hand? An Agent Skill is just a folder containing a `SKILL.md` (plus optional references), so copying it works too:
+
+- **Claude Code**: put the `agenteye-cli/` folder in `~/.claude/skills/` (every project) or `<your-repo>/.claude/skills/` (that repo only). Claude Code auto-discovers it — verify with the `/skills` list, or simply ask a question that matches its description.
+- **Codex (OpenAI)**: Codex reads the same `SKILL.md`. The bundled `agents/openai.yaml` sets `allow_implicit_invocation: true`, so Codex auto-selects the skill when a task matches; otherwise invoke it explicitly as `$agenteye-cli`.
+
+---
+
+## Safety: mutations do NOT prompt when an agent runs the CLI
+
+> **Warning:** Read this before letting an agent make changes.
+
+The `agenteye` CLI normally asks *"are you sure?"* before a destructive action. It **auto-skips that confirmation whenever it is not attached to a terminal (which is exactly how a coding agent runs it), and `--json` skips it too.** So the safety prompt will **not** fire for the agent.
+
+The skill is written to compensate: it is instructed to state the exact command it will run and get your explicit **OK before any state change**. Keep that discipline. When you drive Failproof AI Observability through an agent, *you* are the confirmation step. The state-changing commands to watch for:
+
+- `keys create` / `update` / `disable` / `regenerate`
+- `users create` / `update` / `disable` / `enable`
+- `settings set`
+- `alerts create` / `update` / `delete` / `test`
+- the writing `incidents` subcommands: `ack` / `assign` / `resolve` / `open` / `comment-add` / `comment-delete` / `subscribe` / `unsubscribe`
+- `query create` / `update` / `delete`
+- `agent rename` / `delete`
+- `orgs switch`
+
+Everything under **Observe** (`events`, `sessions`, `evals`, `errors`, `list`, `whoami`, `orgs list/current/perms`) is read-only and changes nothing.
+
+Because the agent acts as **you**, it can only do what your login is permitted to do; permissions are resolved **per org** (see [API keys](/agenteye/api-keys)). A command you lack permission for returns exit code `5` with the exact permission named, so the agent can tell you precisely what to ask an admin for rather than failing opaquely.
+
+---
+
+## What you can ask it
+
+Here's what a real round-trip looks like, a read, then a change that waits for your OK:
+
+```text
+you   ▸ Is anything broken in the last day?
+
+agent ▸ Running: agenteye --json errors --since 24h --aggregate
+        3 error types in the last 24h. The biggest is TimeoutError
+        (18 hits, all on the checkout-agent), then RateLimited (5).
+        Want the failing sessions for TimeoutError?
+
+you   ▸ Give CI a key that can only push events.
+
+agent ▸ This creates an API key. I'll run:
+          agenteye keys create ci --add events:add
+        OK to proceed?
+
+you   ▸ yes
+
+agent ▸ Done. Key "ci" created with events:add only.
+        The secret is shown only once, so store it now. I can't reprint it.
+```
+
+The skill maps each plain-English intent to the right `agenteye` command, discovering valid values first (`list <kind>`, `whoami`) so it doesn't guess, and stating the exact command before any change. More examples:
+
+- *"Is anything broken / failing in the last 24 hours?"* → `errors --since 24h --aggregate`, then a breakdown.
+- *"Why did session `run-001` fail?"* → `events --session-id run-001 --all` + `evals --session-id run-001`.
+- *"How is quality trending this week?"* → `evals --aggregate --since 7d`, then drill into low-scoring runs.
+- *"Give CI a key that can only push events."* → `keys create ci --add events:add` (it states the command, then creates it and captures the one-time secret).
+- *"Who has access? Make Dana read-only."* → `users list` → `users update dana@… --permission-set read-only` (after confirming with you).
+- *"Ack the firing incident and assign it to me."* → `incidents list --state firing` → `incidents ack <id>` / `incidents assign <id> you@…`.
+
+For the exact commands, flags, and JSON shapes behind these, see the [CLI](/agenteye/cli) reference and [CLI recipes for agents](/agenteye/cli-recipes).
+
+---
+
+## Next steps
+
+- **[CLI](/agenteye/cli)**: full command and flag reference for `agenteye`.
+- **[CLI recipes for agents](/agenteye/cli-recipes)**: copy-paste `jq` patterns and exit-code handling.
+- **[Evaluator agent skill](/agenteye/evaluator-skill)**: the sibling skill, for building the evaluator whose scores `agenteye evals` reads.
+- **[Python SDK agent skill](/agenteye/python-sdk-skill)**: the sibling skill, for instrumenting an agent so it emits the telemetry `agenteye` reads.
+- **[AI assistant](/agenteye/assistant)**: the in-dashboard assistant (not to be confused with this terminal skill).
+- **[API keys](/agenteye/api-keys)**: the per-org permission model that bounds what the skill can do.