@dsalazardev/ai-hub-cli 0.0.0-stage → 0.2.0

package/skills/shared/agent-eval/references/runner-and-gate.md

@@ -0,0 +1,183 @@
+# The runner, the gate, and framework equivalents
+
+Depth offloaded from `SKILL.md`. A provider-neutral runner you can drop into any repo, the CI
+wiring, and the same eval expressed in DeepEval and Inspect AI so you can switch tools without
+relearning the craft.
+
+## The five stages, in code
+
+`dataset → runner → scorers → metrics → gate`. Keep them separable so each can be swapped.
+
+### Dataset loader (JSONL, decontaminated, versioned)
+
+```python
+import json
+from pathlib import Path
+
+REQUIRED = {"id", "input", "expected"}
+
+def load_cases(path: str) -> list[dict]:
+    cases = []
+    for i, line in enumerate(Path(path).read_text().splitlines(), 1):
+        line = line.strip()
+        if not line:
+            continue
+        case = json.loads(line)
+        missing = REQUIRED - case.keys()
+        if missing:
+            raise ValueError(f"{path}:{i} missing keys {missing}")
+        cases.append(case)
+    return cases
+```
+
+### Scorer registry
+
+Every scorer satisfies one protocol — deterministic and judge are interchangeable. See
+`judge-design.md` for the judge implementations.
+
+```python
+from dataclasses import dataclass
+from typing import Callable
+
+@dataclass
+class Scorer:
+    name: str
+    fn: Callable[[dict, str], float]   # (case, output) -> 0.0..1.0
+
+def exact_match(case, output):
+    return 1.0 if output.strip() == case["expected"].strip() else 0.0
+
+def latency_under(threshold_s):
+    def _fn(case, output):
+        return 1.0 if case["meta"].get("latency_s", 0) <= threshold_s else 0.0
+    return _fn
+
+REGISTRY = [
+    Scorer("exact_match", exact_match),
+    Scorer("latency_2s", latency_under(2.0)),
+    # Scorer("faithfulness", FaithfulnessJudge(judge_model).score),  # 30% judge
+]
+```
+
+### Runner + bootstrap-CI metrics
+
+The bootstrap CI is what stops judge noise from flapping the gate: only a drop beyond the
+interval counts as a real regression.
+
+```python
+import random, statistics
+
+def run(system, cases, scorers):
+    rows = []
+    for case in cases:
+        output = system(case["input"])               # the system under test
+        row = {"id": case["id"],
+               "failure_mode": case["meta"].get("failure_mode", "none")}
+        for s in scorers:
+            row[s.name] = s.fn(case, output)
+        rows.append(row)
+    return rows
+
+def metrics(rows, scorers, n_boot=1000):
+    out = {}
+    for s in scorers:
+        vals = [r[s.name] for r in rows]
+        mean = statistics.fmean(vals)
+        boots = [statistics.fmean(random.choices(vals, k=len(vals)))
+                 for _ in range(n_boot)]
+        boots.sort()
+        out[s.name] = {"mean": round(mean, 4),
+                       "ci_low": round(boots[int(0.025 * n_boot)], 4),
+                       "ci_high": round(boots[int(0.975 * n_boot)], 4)}
+    return out
+```
+
+### Gate (block on regression vs committed baseline)
+
+```python
+import json, sys
+
+def gate(current: dict, baseline: dict, report_path="eval-report.json") -> int:
+    regressed = []
+    for metric, m in current.items():
+        base = baseline.get(metric, {}).get("mean")
+        # Real regression: the current CI is entirely below the baseline mean.
+        if base is not None and m["ci_high"] < base:
+            regressed.append({"metric": metric, "baseline": base,
+                              "now": m["mean"], "ci_high": m["ci_high"]})
+    report = {"metrics": current, "baseline": baseline,
+              "regressed": regressed, "passed": not regressed}
+    Path(report_path).write_text(json.dumps(report, indent=2))
+    for r in regressed:
+        print(f"REGRESSION {r['metric']}: {r['baseline']} -> {r['now']}",
+              file=sys.stderr)
+    return 1 if regressed else 0
+```
+
+Update the baseline deliberately — commit a new `eval-baseline.json` in the PR that *intends*
+to move the metric, so the move is reviewed, not silent.
+
+## GitHub Actions wiring
+
+```yaml
+name: eval-gate
+on: [pull_request]
+jobs:
+  eval:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with: { python-version: "3.12" }
+      - run: pip install -r evals/requirements.txt
+      - name: Run eval gate
+        env:
+          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
+        run: python evals/run.py          # exits non-zero on regression -> blocks merge
+      - uses: actions/upload-artifact@v4
+        if: always()
+        with: { name: eval-report, path: eval-report.json }
+```
+
+The non-zero exit is the gate. Upload the report `if: always()` so a failed run still shows
+*which* metric and failure mode regressed.
+
+## Same eval in DeepEval (pytest-native)
+
+```python
+from deepeval import assert_test
+from deepeval.test_case import LLMTestCase
+from deepeval.metrics import FaithfulnessMetric
+
+def test_faithfulness():
+    case = LLMTestCase(
+        input="Where is my refund for order 4821?",
+        actual_output=system("Where is my refund for order 4821?"),
+        retrieval_context=["policy: refunds 5-7 business days"],
+    )
+    assert_test(case, [FaithfulnessMetric(threshold=0.8)])
+```
+
+DeepEval v4.0.3 reads like tests and runs under pytest; reach for it when CI is already Python.
+
+## Same eval in Inspect AI (Task / Solver / Scorer)
+
+```python
+from inspect_ai import Task, task
+from inspect_ai.dataset import json_dataset
+from inspect_ai.solver import generate
+from inspect_ai.scorer import model_graded_qa
+
+@task
+def refund_quality():
+    return Task(
+        dataset=json_dataset("evals/cases.jsonl"),
+        solver=generate(),
+        scorer=model_graded_qa(),   # judge with rationale; bootstrap CIs built in
+    )
+# inspect eval refund_quality.py --model openai/gpt-... -> pass/fail + CIs
+```
+
+Inspect AI v0.3.225 (UK AISI) gives first-class tool-use and trajectory logging plus bootstrap
+CIs out of the box; reach for it when you are multi-provider or need real trajectory scoring.
+Both express the identical `dataset → scorer → gate` pipeline — the tool is a detail.

package/skills/shared/agent-eval/scripts/verify.sh

@@ -0,0 +1,161 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# verify.sh — agent-eval skill gate. Run from your PROJECT root.
+#
+# What it does (read-only, idempotent, NEVER calls an LLM or network):
+#   1. Discovers *.jsonl golden sets (skips vendor dirs); checks every line parses
+#      and carries the required keys id, input, expected. Hard fail on a bad line.
+#   2. If an eval-report.json exists, validates its shape (a "passed" boolean and a
+#      "metrics" object). Hard fail only on malformed JSON / missing shape.
+#   3. Optional lint, advisory only: ruff + mypy on example Python, markdownlint on docs.
+#
+# Exit non-zero ONLY on a malformed JSONL line, a missing required key, or a malformed
+# eval-report.json. Every optional lint and every missing tool is a yellow WARN/skip.
+# An empty or clean target exits 0. Stock macOS bash 3.2 (no mapfile / associative arrays).
+
+YELLOW=$'\033[33m'; GREEN=$'\033[32m'; RED=$'\033[31m'; NC=$'\033[0m'
+EXIT=0
+skip() { printf '%s[skip]%s %s\n' "$YELLOW" "$NC" "$*"; }
+note() { printf '%s[warn]%s %s\n' "$YELLOW" "$NC" "$*"; }
+ok()   { printf '%s[ok]%s %s\n'   "$GREEN"  "$NC" "$*"; }
+err()  { printf '%s[fail]%s %s\n' "$RED"    "$NC" "$*"; EXIT=1; }
+
+ROOT="$(pwd)"
+
+# Need a JSON-aware tool for the structural checks. Prefer python3, fall back to jq.
+JSON_TOOL=""
+if command -v python3 >/dev/null 2>&1; then JSON_TOOL="python3"
+elif command -v jq >/dev/null 2>&1; then JSON_TOOL="jq"; fi
+
+# --- 1. golden-set JSONL validation -----------------------------------------
+JSONL_FILES=()
+while IFS= read -r -d '' f; do
+  JSONL_FILES+=("$f")
+done < <(
+  find "$ROOT" \
+    \( -path '*/node_modules/*' -o -path '*/.git/*' -o -path '*/vendor/*' \
+       -o -path '*/.venv/*' -o -path '*/dist/*' \) -prune -o \
+    -type f -name '*.jsonl' -print0 2>/dev/null
+)
+
+if [ "${#JSONL_FILES[@]}" -eq 0 ]; then
+  skip "no *.jsonl golden sets found under $ROOT — nothing to validate"
+elif [ -z "$JSON_TOOL" ]; then
+  skip "neither python3 nor jq found — cannot validate JSONL, skipping"
+else
+  for f in "${JSONL_FILES[@]}"; do
+    if [ "$JSON_TOOL" = "python3" ]; then
+      msg="$(python3 - "$f" <<'PY'
+import json, sys
+path = sys.argv[1]
+req = {"id", "input", "expected"}
+bad = []
+with open(path, encoding="utf-8") as fh:
+    for n, line in enumerate(fh, 1):
+        s = line.strip()
+        if not s:
+            continue
+        try:
+            obj = json.loads(s)
+        except ValueError as e:
+            bad.append(f"line {n}: invalid JSON ({e})"); continue
+        if not isinstance(obj, dict):
+            bad.append(f"line {n}: not a JSON object"); continue
+        missing = req - obj.keys()
+        if missing:
+            bad.append(f"line {n}: missing keys {sorted(missing)}")
+print("\n".join(bad), end="")
+PY
+)" || msg="parser crashed on $f"
+      if [ -n "$msg" ]; then
+        err "$f:"; printf '  %s\n' "$msg"
+      else
+        ok "$f valid (id/input/expected present on every line)"
+      fi
+    else
+      # jq path: each line must be an object with the three keys.
+      if jq -e 'has("id") and has("input") and has("expected")' "$f" >/dev/null 2>&1; then
+        ok "$f valid (jq: required keys present)"
+      else
+        err "$f: a line is not an object or is missing id/input/expected (jq)"
+      fi
+    fi
+  done
+fi
+
+# --- 2. eval-report.json shape ------------------------------------------------
+REPORTS=()
+while IFS= read -r -d '' f; do
+  REPORTS+=("$f")
+done < <(
+  find "$ROOT" \
+    \( -path '*/node_modules/*' -o -path '*/.git/*' -o -path '*/.venv/*' \) -prune -o \
+    -type f -name 'eval-report.json' -print0 2>/dev/null
+)
+
+if [ "${#REPORTS[@]}" -eq 0 ]; then
+  skip "no eval-report.json found — gate output not validated"
+elif [ -z "$JSON_TOOL" ]; then
+  skip "no python3/jq — cannot validate eval-report.json shape"
+else
+  for f in "${REPORTS[@]}"; do
+    if [ "$JSON_TOOL" = "python3" ]; then
+      if python3 - "$f" <<'PY'
+import json, sys
+try:
+    r = json.load(open(sys.argv[1], encoding="utf-8"))
+except ValueError as e:
+    print(e); sys.exit(1)
+ok = isinstance(r, dict) and isinstance(r.get("passed"), bool) \
+     and isinstance(r.get("metrics"), dict)
+sys.exit(0 if ok else 1)
+PY
+      then ok "$f shape ok (passed: bool, metrics: object)"
+      else err "$f: malformed or missing 'passed'(bool)/'metrics'(object)"; fi
+    else
+      if jq -e 'type=="object" and (.passed|type=="boolean") and (.metrics|type=="object")' "$f" >/dev/null 2>&1; then
+        ok "$f shape ok (jq)"
+      else err "$f: malformed or missing passed/metrics (jq)"; fi
+    fi
+  done
+fi
+
+# --- 3. optional lint (advisory only) ----------------------------------------
+PY_FILES=()
+while IFS= read -r -d '' f; do
+  PY_FILES+=("$f")
+done < <(
+  find "$ROOT" \
+    \( -path '*/node_modules/*' -o -path '*/.git/*' -o -path '*/.venv/*' \) -prune -o \
+    -type f -name '*.py' -print0 2>/dev/null
+)
+
+if [ "${#PY_FILES[@]}" -gt 0 ]; then
+  if command -v ruff >/dev/null 2>&1; then
+    if ruff check "${PY_FILES[@]}" >/dev/null 2>&1; then ok "ruff clean"; else note "ruff reported lint findings (advisory)"; fi
+  else skip "ruff not installed — skipping Python lint"; fi
+  if command -v mypy >/dev/null 2>&1; then
+    mypy "${PY_FILES[@]}" >/dev/null 2>&1 && ok "mypy clean" || note "mypy reported type findings (advisory)"
+  else skip "mypy not installed — skipping type check"; fi
+else
+  skip "no *.py files — skipping ruff/mypy"
+fi
+
+MD_FILES=()
+while IFS= read -r -d '' f; do
+  MD_FILES+=("$f")
+done < <(
+  find "$ROOT" \
+    \( -path '*/node_modules/*' -o -path '*/.git/*' \) -prune -o \
+    -type f -name '*.md' -print0 2>/dev/null
+)
+if [ "${#MD_FILES[@]}" -gt 0 ] && command -v markdownlint >/dev/null 2>&1; then
+  markdownlint "${MD_FILES[@]}" >/dev/null 2>&1 && ok "markdownlint clean" || note "markdownlint findings (advisory)"
+else
+  skip "markdownlint not installed or no docs — skipping doc lint"
+fi
+
+printf '\n'
+if [ "$EXIT" -eq 0 ]; then ok "verify.sh passed"; else err "verify.sh found failures"; fi
+exit "$EXIT"

package/skills/shared/agent-safety/SKILL.md

@@ -0,0 +1,169 @@
+---
+name: agent-safety
+description: "Use when bounding an LLM agent that already runs — scoping its task domain, gating tools to least privilege, defending against prompt injection in untrusted web/email/RAG text, requiring human approval on irreversible actions, capping runtime and cost, or triaging what it already did. NOT building the loop, tools, or RAG (that is `building-agents`)."
+tags: [agent-security, guardrails, prompt-injection, least-privilege, owasp-agentic]
+recommends: [building-agents, secure-coding, agent-eval]
+origin: ai-hub
+---
+
+# Agent safety
+
+You are the security review for an agent's **agency**, not for its code. The loop works,
+tools are wired, memory persists — your job is to make that autonomy *bounded*. If you
+want to review ordinary endpoints, auth, or secrets handling, that is
+`../secure-coding/SKILL.md` — this skill is the *Agentic* Top 10, the risks that exist only
+because a model has tools and autonomy. If the loop or tools do not exist yet, that is
+`../building-agents/SKILL.md`. You arrive *after* both.
+
+`references/threat-model.md` carries the OWASP Agentic Top 10 2026 risks mapped to the
+controls below, the pre-ship guardrail checklist, and the incident-triage flow for "the
+agent did X" — open it when you are reviewing before ship or reconstructing an incident.
+
+## The ownership split
+
+Agent security splits into four layers — **Model · Harness · Tools · Environment**. The
+model provider owns only the Model layer (alignment, refusals). Everything else is yours:
+the Harness (loop, memory, context assembly), the Tools (what the agent can *do*), and the
+Environment (creds, network, blast radius). Do not outsource a layer you own to "the model
+is aligned."
+
+Three excesses cause almost every agentic incident. Cut all three:
+
+- **Excessive functionality** — tools the task never needs.
+- **Excessive permissions** — broader scopes/creds than the tool needs.
+- **Excessive autonomy** — acting without checking back when it should.
+
+The operating principle is **least agency**: autonomy is earned per task, not defaulted.
+
+## Scope limits
+
+- **Declare the allowed task domain as a hard boundary in the system prompt.** Why: an
+  undeclared scope is an infinite scope; "you are a refund assistant; you do not touch
+  payroll" is a constraint a reviewer can check.
+- **Deny by default — the agent starts with zero tools.** Each tool earns its place by a
+  task justification. Why: an opt-out tool list grows; an opt-in list stays minimal.
+- **Segregate the instruction channel from the data channel.** System/developer prompt =
+  trusted instructions. Everything the agent reads at runtime = data, never instructions.
+  Why: this single boundary is what stops indirect injection (LLM01).
+
+## Tool gating / least agency
+
+Give every tool a profile: **read / write / exec / send**, the exact resources it may
+touch, and an **allowlist** (never a wildcard). Block destructive flags and secret paths
+at the tool boundary, not in the prompt — the prompt is advisory, the boundary is enforced.
+
+```python
+# Bad: one wildcard tool = unbounded blast radius, runs anything the loop emits
+def run_shell(cmd: str) -> str:
+    return subprocess.run(cmd, shell=True, capture_output=True, text=True).stdout
+```
+
+```python
+# Good: narrow tool, allowlisted root, denied patterns, no shell
+ALLOWED_ROOT = pathlib.Path("/srv/agent/workspace").resolve()
+DENY = ("*.key", "*.pem", "*secret*", "*.env", "id_rsa*")
+
+def read_file(path: str) -> str:
+    p = (ALLOWED_ROOT / path).resolve()
+    if not p.is_relative_to(ALLOWED_ROOT):           # no traversal out of scope
+        raise PermissionError("path outside workspace")
+    if any(p.match(g) for g in DENY):                # never read secrets
+        raise PermissionError("denied pattern")
+    return p.read_text()
+```
+
+- **Issue task-scoped, short-lived tokens — not the session's broad creds.** A credential
+  should be valid only for the specific tool and the duration of one task. Why: a hijacked
+  loop cannot reuse a session-wide token it never held.
+- **Prefer read-only by default; writes/sends/exec are separate, gated tools.** Why: most
+  steps only need to read, so most steps should be unable to mutate anything.
+
+## Injection defense
+
+Treat **all** external data as untrusted: user messages, retrieved documents, API
+responses, emails, web pages, other agents' output. Sanitize and delimit before it enters
+context, and **never let external text reach a privileged tool unmediated.**
+
+| Source                         | Trust level | Required mediation before it can act        |
+| ------------------------------ | ----------- | ------------------------------------------- |
+| System / developer prompt      | Trusted     | none (this is the only instruction channel) |
+| End-user chat message          | Untrusted   | delimit; treat as data, not commands        |
+| Retrieved RAG / KB document    | Untrusted   | delimit; strip instruction-like spans       |
+| Fetched web page / API JSON    | Untrusted   | parse to schema; no raw text → tool args    |
+| Inbound email / ticket body    | Untrusted   | delimit; HITL on any action it requests     |
+| Another agent's message        | Untrusted   | same as external user input                 |
+
+```python
+# Bad: retrieved chunk flows straight into a privileged action
+chunk = retriever.search(q)[0].text          # attacker-controlled doc
+agent.call_tool("send_email", to=extract_to(chunk), body=chunk)
+```
+
+```python
+# Good: external content is quarantined data; the action is schema-validated + gated
+chunk = retriever.search(q)[0].text
+ctx = f"<retrieved untrusted>\n{chunk}\n</retrieved untrusted>"   # delimited, labeled
+proposal = agent.draft("send_email", context=ctx)                # model proposes
+args = SendEmail.model_validate(proposal.args)                   # schema or reject
+if args.to_domain not in ALLOWED_DOMAINS:                        # exfil guard
+    raise PermissionError("recipient outside allowlist")
+require_human_approval("send_email", args)                       # irreversible → HITL
+```
+
+- **Validate every tool-call argument against a strict schema before execution.** Why: a
+  schema rejects the surprise field, encoded payload, or off-allowlist recipient injection
+  produces.
+- **Watch for exfiltration shapes** — unexpected outbound URLs, base64 blobs, recipients
+  outside the allowlist. Why: data theft is the common payload of a successful injection.
+
+## Human-in-the-loop by risk class
+
+Do **not** approve every action — reported ~93% of permission prompts get approved without
+being read, so blanket prompting trains a rubber stamp. Gate by **risk class**, keyed on
+reversibility × blast radius. Bind each approval to the **exact parameters** with a
+short-lived token so the approved action cannot be swapped after the click.
+
+| Action type (examples)                          | Reversible? | Blast radius | Control            |
+| ----------------------------------------------- | ----------- | ------------ | ------------------ |
+| Read file, search KB, fetch page                | n/a         | none         | **auto**           |
+| Write to scratch workspace, internal draft      | yes         | local        | **log-only**       |
+| Mutate prod DB, deploy, change config            | hard        | system       | **approve (HITL)** |
+| Send email/payment to external party, post live | no          | external     | **approve (HITL)** |
+| Delete backups, rotate prod creds, mass-delete  | no          | catastrophic | **block** (or step-up auth) |
+
+- **Step up for the top row** — high-value irreversible actions deserve fresh auth, not the
+  ambient session. Why: a hijacked session should not also hold the keys to the worst action.
+
+## Memory hygiene
+
+- **Validate and sanitize content before it is stored.** Why: memory poisoning persists
+  across sessions (OWASP Agentic T1) — unlike session-scoped injection, a poisoned memory
+  re-attacks every future run until purged.
+- **Isolate memory per user and per session; do not let one user's writes color another's
+  reads.** Why: shared memory is a cross-tenant injection channel.
+- **Expire entries and cap memory size; redact PII (SSN, cards, API keys) before persist.**
+  Why: stale instructions and leaked secrets both age into liabilities.
+
+## Runtime kill-switches
+
+A looping or hijacked agent must hit a wall on its own. Set hard caps, fail closed:
+
+- **Tool-call rate cap** (e.g. ~30 calls/min) — runaway loops trip it before they do damage.
+- **Cost cap per session** (e.g. ~$10) — a wallet attack stops at a known ceiling.
+- **Loop / step cap** — a fixed max iterations kills the infinite plan.
+- **Wall-clock timeout** — a stuck agent is terminated, not left running.
+
+Log every tool call (arguments redacted) and alert on repeated approval-bypass attempts.
+
+## Anti-patterns
+
+| Anti-pattern                                   | Why it bites                                                   | Do instead                                          |
+| ---------------------------------------------- | ------------------------------------------------------------- | --------------------------------------------------- |
+| Approve every action                           | ~93% rubber-stamped; the real risky one slips through         | Gate by risk class; HITL only on irreversible/external |
+| One broad session token shared by all tools    | Hijacked loop reuses it everywhere                            | Task-scoped, short-lived per-tool tokens            |
+| Trust RAG / fetched / email content            | Indirect injection (LLM01) becomes direct tool execution      | Delimit as untrusted data; mediate before any tool  |
+| Wildcard `run_shell(cmd)` tool                 | Unbounded blast radius                                        | Narrow tools, allowlisted resources, denied patterns |
+| Raw external text piped into tool args         | Attacker controls the action's parameters                     | Schema-validate args; allowlist recipients/domains  |
+| Scope/limits stated only in the prompt         | Prompt is advisory; the model can be talked out of it         | Enforce at the tool/harness boundary                |
+| No loop / cost / rate cap                       | A hijacked or looping agent runs until it runs out of money   | Hard fail-closed kill-switches                      |
+| Redact PII only in the UI                       | The secret was already written to memory/logs                 | Redact before persistence, at the source            |

package/skills/shared/agent-safety/evals/README.md

@@ -0,0 +1,12 @@
+# Evals: agent-safety
+
+These cases are the trigger contract and capability rubric for the `agent-safety` skill.
+`should_trigger` and `should_not_trigger` assert that the description routes correctly —
+each negative names the real sibling it should route to instead (building-agents,
+secure-coding, agent-eval, prompt-engineering, cost-tracking). Run them through the repo's
+eval harness if one is wired up, or read them directly as the routing spec when reviewing
+the description. The `capability` block is not a pass/fail script: it is a rubric a human or
+an LLM-as-judge scores a sample guardrail design against — the design must cover every
+`must_include` item (deny-by-default tools, untrusted-content mediation, HITL on
+irreversible actions, memory hygiene, runtime caps) and map them to the OWASP Agentic Top
+10. This is a process/review skill, so there is no `verify.sh`; rigor lives in this eval.

package/skills/shared/agent-safety/evals/cases.yaml

@@ -0,0 +1,46 @@
+skill: agent-safety
+
+should_trigger:
+  - prompt: "Our support agent can call any shell command — lock it down."
+    why: Wildcard exec tool with unbounded blast radius; core tool-gating / least-agency case.
+  - prompt: "An email in the inbox told the agent to forward all invoices to an external address and it did."
+    why: Indirect prompt injection (LLM01). Non-obvious — reads like a bug report, but the cause is untrusted email content reaching a privileged send tool.
+  - prompt: "Before we ship this autonomous agent to prod, review what it's allowed to do."
+    why: Pre-ship review of scope, permissions, and HITL — exactly the bounded-autonomy review this skill is.
+  - prompt: "Posa límits a l'agent perquè no pugui esborrar res sense aprovació."
+    why: Catalan. HITL-by-risk-class on an irreversible delete plus scope limiting.
+  - prompt: "The agent's memory keeps repeating a wrong instruction across sessions."
+    why: Memory poisoning (OWASP Agentic T1). Non-obvious — sounds like a quality bug, is actually contaminated persistent memory.
+  - prompt: "Review our MCP server's tool registry — the scopes look way too broad."
+    why: Over-broad tool scopes / excessive permissions; deny-by-default and per-tool allowlists.
+
+should_not_trigger:
+  - prompt: "Help me build the agent's tool-calling loop."
+    route_to: building-agents
+    why: Construction of the agent (loop/tools), not constraining one that already runs.
+  - prompt: "Is this Express auth endpoint vulnerable to broken access control?"
+    route_to: secure-coding
+    why: Plain web/app OWASP on an endpoint; no model, tools, or autonomy involved.
+  - prompt: "Measure whether my agent answers correctly with a golden set."
+    route_to: agent-eval
+    why: Quality / correctness scoring, not safety constraint.
+  - prompt: "Write a sharper system prompt so the agent is more helpful."
+    route_to: prompt-engineering
+    why: Capability tuning, not a safety boundary.
+  - prompt: "Show me how much each model call costs this month."
+    route_to: cost-tracking
+    why: Reporting spend; agent-safety sets the cost *cap*, it does not report the analytics.
+
+capability:
+  - scenario: >
+      A research agent has web-fetch, send-email, and file-write tools plus persistent
+      memory shared across sessions. Design the guardrails that make its autonomy bounded.
+    must_include:
+      - Deny-by-default tools with a per-tool profile (read web vs send email vs write file); no wildcard tool.
+      - Treats web-fetched content as untrusted; delimits it as data; never pipes external text straight into send-email args.
+      - HITL approval for send-email bound to exact parameters (irreversible, external blast radius); recipient/domain allowlist as exfil guard.
+      - File-write scoped to an allowlisted workspace root; blocks secret patterns (*.key/*.env/*secret*) and path traversal.
+      - Memory validated before store, isolated per session, expiring, PII-redacted before persist.
+      - Runtime kill-switches: tool-call rate cap, per-session cost cap, loop/step cap, wall-clock timeout, all failing closed.
+      - Task-scoped short-lived tokens rather than one broad session credential.
+      - Maps each control to OWASP Agentic Top 10 (indirect injection / tool misuse / memory poisoning / excessive agency).

package/skills/shared/agent-safety/references/threat-model.md

@@ -0,0 +1,51 @@
+# Threat model: OWASP Agentic Top 10 (2026) → controls
+
+Mapping of the named agentic risks to the control in `../SKILL.md` that mitigates each.
+Sources: OWASP Gen AI Security Project, "OWASP Top 10 for Agentic Applications for 2026"
+(genai.owasp.org); OWASP Top 10 for LLM Applications v2025; OWASP AI Agent Security Cheat
+Sheet (cheatsheetseries.owasp.org). All accessed 2026-06-02.
+
+| Agentic risk                       | What it is                                                              | Control in this skill                                   |
+| ---------------------------------- | ----------------------------------------------------------------------- | ------------------------------------------------------- |
+| Memory poisoning (T1)              | Contamination persists across sessions, re-attacking every future run   | Memory hygiene: validate before store, isolate, expire  |
+| Tool misuse & exploitation         | Abuse via unsafe composition / recursion / excessive calls *with* valid perms | Tool gating + runtime kill-switches (rate/loop caps)    |
+| Privilege compromise               | Broad or stolen creds let the loop act beyond its task                  | Task-scoped short-lived tokens; least agency            |
+| Indirect prompt injection (LLM01)  | Untrusted content carries instructions the model executes               | Trust-boundary table; segregate data from instructions  |
+| Excessive agency                   | Acting without check-back on irreversible/external actions              | HITL by risk class; deny-by-default scope               |
+| Data exfiltration                  | Stolen data leaves via tool output / outbound calls                     | Output schema validation; exfil-shape detection         |
+
+The OWASP framing of root causes — excessive **functionality**, **permissions**,
+**autonomy** — maps one-to-one onto the three excesses in the SKILL body. Cutting all three
+is the deny-by-default + least-agency posture.
+
+## Pre-ship guardrail checklist
+
+Before an autonomous loop reaches production, confirm:
+
+- [ ] Allowed task domain is declared in the system prompt as a hard boundary.
+- [ ] Tools are deny-by-default; each enabled tool has a written task justification.
+- [ ] Every tool has a profile (read / write / exec / send) and an allowlist, not a wildcard.
+- [ ] No tool can read `*.key` / `*.pem` / `*secret*` / `*.env` or run with destructive flags.
+- [ ] Creds are task-scoped and short-lived, not one broad session token.
+- [ ] All external content (web, email, RAG, API, other agents) is delimited as untrusted.
+- [ ] Tool-call args are schema-validated; recipients/domains are allowlisted (exfil guard).
+- [ ] HITL is gated by risk class; irreversible/external actions require approval bound to exact params.
+- [ ] Memory is validated before store, isolated per user/session, expiring, PII-redacted.
+- [ ] Runtime caps exist and fail closed: rate, cost, loop/step, wall-clock timeout.
+- [ ] Every tool call is logged (redacted); repeated approval-bypass alerts.
+
+## Incident triage: "the agent did X"
+
+When an agent has already done something it should not have, work the loop in order:
+
+1. **Contain.** Revoke the task tokens / creds the loop is holding; trip the kill-switch
+   (pause the loop). Stop the bleeding before diagnosing.
+2. **Find the trust boundary it crossed.** Which untrusted source reached a privileged
+   tool? Trace the action's parameters back to their origin — usually an email body, RAG
+   chunk, fetched page, or a poisoned memory entry.
+3. **Purge poisoned memory.** If the bad behavior repeats across sessions, the cause is
+   stored, not in-context. Delete the contaminated entries; do not just restart the session.
+4. **Add the missing gate.** Reclassify that action's risk, move it behind HITL or block,
+   tighten the tool allowlist, or add the schema/exfil check that would have caught it.
+5. **Confirm with a replay.** Re-run the triggering input against the new guardrail and
+   verify the action is now refused or escalated.