agentdynamics 0.7.0__tar.gz → 0.9.0__tar.gz

{agentdynamics-0.7.0 → agentdynamics-0.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: agentdynamics
-Version: 0.7.0
+Version: 0.9.0
 Summary: Application performance monitoring for AI agents: flows, cost, reliability and process quality
 License-Expression: Apache-2.0
 Project-URL: Homepage, https://github.com/Aditya31398/agentdynamics
@@ -70,7 +70,9 @@ def handle_ticket(question):
 
 Without `outcome()`, success is inferred from errors, interrupts and follow-ups. The console always shows
 which outcomes were stated and which were guessed; grades can also be posted afterwards by a person or an
-eval pipeline (`POST /api/outcomes`).
+eval pipeline (`POST /api/outcomes`), or by your own systems using your own ids: trace with
+`agentdynamics.trace("support", ticket_id="T-123")` and post `{"key": {"ticket_id": "T-123"}, "outcome": "rework"}`
+when the ticket is reopened.
 
 ### 2. Python, zero code changes
 
@@ -148,10 +150,11 @@ AGENTDYNAMICS_URL=http://127.0.0.1:8790 python examples/governed_agent.py 40
 | **Workflows** | The real execution graph mined from your traces: path variants and their success rates, loops, the critical node, handoffs |
 | **Task snapshots** | Span-tree waterfall for each request: context growth, cost against the baseline, what went wrong, what the user said next |
 | **Baselines & Apdex** | "Normal" cost and latency per workflow, learned automatically, plus an agent Apdex score |
-| **Health rules & alerts** | 28 rules (runaway cost, policy denials, boundary probing, revocations, retry loops, node loops, truncation, rate limits, ping-pong handoffs, unverified code changes, …) sent to Slack, PagerDuty or webhooks, routed by project, rule and severity |
-| **Tools & Models** | Error rate and p95 per tool; per model TTFT, tokens/s, truncation rate, cache hit rate and spend |
-| **SLOs** | Success rate, Apdex, latency and cost objectives with error budgets and burn rates |
+| **Health rules & alerts** | 29 rules (runaway cost, policy denials, boundary probing, tripwires, revocations, retry loops, node loops, truncation, rate limits, ping-pong handoffs, unverified code changes, …) sent to Slack, PagerDuty or webhooks, routed by project, rule and severity |
+| **Tools & Models** | Error rate and p95 per tool; per model TTFT, tokens/s, truncation rate, cache hit rate and spend, and the estimate beside what Anthropic billed (with an Admin API key) |
+| **SLOs** | Success rate, Apdex, good-task rate, latency and cost objectives with error budgets and burn rates |
 | **Governance** | Aegis policy enforcement per task: denials by rule, budget stops, kill-switch revocations, unused grants, policy export, and what a policy change would refuse |
+| **Incidents & trust** | Tripwires, probing, refused calls and revocations about one agent, grouped into one incident with its evidence; revoke the agent or resolve it as real or a false alarm, and alert once per incident. Each agent's trust score (0-100) weighs that evidence by age and by the verdicts given, and is exported to Prometheus. An optional checker has a model review each incident in shadow mode, and keeps score of how often it agreed with you |
 | **Process Review** | Scores how well the agent works (efficiency, focus, reliability, verification, context, autonomy, compliance) and suggests fixes |
 | **Analytics & Compare** | Any metric by any dimension; compare models, prompts or releases side by side |
 
@@ -169,7 +172,10 @@ governance.instrument(kernel, root, watchdog=governance.Watchdog(max_repeated_de
 That call sets up four flows:
 1. **Every Aegis decision appears in its task.** Denials carry the rule id, and the Aegis audit log carries the task's run id so the two logs join.
 2. **Model calls are charged to the Aegis budget before they're sent.** When the budget runs out, the call is refused and never made.
-3. **A watchdog revokes the grant** of an agent that keeps probing a boundary, for example after a prompt injection.
+3. **A watchdog revokes the grant** of an agent that keeps probing a boundary, for example after a prompt injection,
+   and **tripwires** (decoy tools and planted canary values) stop a run before the call that touches one is made.
+   For tools that can't be undone, an opt-in **reviewer** model checks each call against what the user asked
+   before Aegis admits it, and refuses what goes beyond it (`review={"tools": ["payments.refund"]}`).
 4. **`agentdynamics policy export` writes a tighter, least-privilege policy** from observed behaviour. `aegis ratify` and `aegis drift` verify its declaration; `agentdynamics policy check` replays the traffic that actually ran through it and reports how much it would refuse, so a policy change can be gated in CI.
 
 The console's **Governance** page shows denials, budget stops, revocations, unused grants and budget headroom per policy version. See [docs/GOVERNANCE.md](docs/GOVERNANCE.md).
@@ -213,6 +219,8 @@ The store is a SQLite file by default. For several instances behind a load balan
 and operate like any other, use Postgres: `pip install "agentdynamics[postgres]"` and set
 `AGENTDYNAMICS_DB_URL=postgresql://...` (or `deploy/docker-compose.postgres.yml`). One instance is elected to
 write the analysis; the others serve the console and take ingest, and one takes over if the writer goes away.
+An existing install moves with its history: `agentdynamics store copy --to postgresql://...`, then
+`agentdynamics store verify`.
 
 - **Security:** role-based API keys, optionally scoped to projects. A scoped key reads only its projects' tasks,
   runs, steps and events, on every endpoint, and can't write into or over another project's traces.

{agentdynamics-0.7.0 → agentdynamics-0.9.0}/README.md

@@ -45,7 +45,9 @@ def handle_ticket(question):
 
 Without `outcome()`, success is inferred from errors, interrupts and follow-ups. The console always shows
 which outcomes were stated and which were guessed; grades can also be posted afterwards by a person or an
-eval pipeline (`POST /api/outcomes`).
+eval pipeline (`POST /api/outcomes`), or by your own systems using your own ids: trace with
+`agentdynamics.trace("support", ticket_id="T-123")` and post `{"key": {"ticket_id": "T-123"}, "outcome": "rework"}`
+when the ticket is reopened.
 
 ### 2. Python, zero code changes
 
@@ -123,10 +125,11 @@ AGENTDYNAMICS_URL=http://127.0.0.1:8790 python examples/governed_agent.py 40
 | **Workflows** | The real execution graph mined from your traces: path variants and their success rates, loops, the critical node, handoffs |
 | **Task snapshots** | Span-tree waterfall for each request: context growth, cost against the baseline, what went wrong, what the user said next |
 | **Baselines & Apdex** | "Normal" cost and latency per workflow, learned automatically, plus an agent Apdex score |
-| **Health rules & alerts** | 28 rules (runaway cost, policy denials, boundary probing, revocations, retry loops, node loops, truncation, rate limits, ping-pong handoffs, unverified code changes, …) sent to Slack, PagerDuty or webhooks, routed by project, rule and severity |
-| **Tools & Models** | Error rate and p95 per tool; per model TTFT, tokens/s, truncation rate, cache hit rate and spend |
-| **SLOs** | Success rate, Apdex, latency and cost objectives with error budgets and burn rates |
+| **Health rules & alerts** | 29 rules (runaway cost, policy denials, boundary probing, tripwires, revocations, retry loops, node loops, truncation, rate limits, ping-pong handoffs, unverified code changes, …) sent to Slack, PagerDuty or webhooks, routed by project, rule and severity |
+| **Tools & Models** | Error rate and p95 per tool; per model TTFT, tokens/s, truncation rate, cache hit rate and spend, and the estimate beside what Anthropic billed (with an Admin API key) |
+| **SLOs** | Success rate, Apdex, good-task rate, latency and cost objectives with error budgets and burn rates |
 | **Governance** | Aegis policy enforcement per task: denials by rule, budget stops, kill-switch revocations, unused grants, policy export, and what a policy change would refuse |
+| **Incidents & trust** | Tripwires, probing, refused calls and revocations about one agent, grouped into one incident with its evidence; revoke the agent or resolve it as real or a false alarm, and alert once per incident. Each agent's trust score (0-100) weighs that evidence by age and by the verdicts given, and is exported to Prometheus. An optional checker has a model review each incident in shadow mode, and keeps score of how often it agreed with you |
 | **Process Review** | Scores how well the agent works (efficiency, focus, reliability, verification, context, autonomy, compliance) and suggests fixes |
 | **Analytics & Compare** | Any metric by any dimension; compare models, prompts or releases side by side |
 
@@ -144,7 +147,10 @@ governance.instrument(kernel, root, watchdog=governance.Watchdog(max_repeated_de
 That call sets up four flows:
 1. **Every Aegis decision appears in its task.** Denials carry the rule id, and the Aegis audit log carries the task's run id so the two logs join.
 2. **Model calls are charged to the Aegis budget before they're sent.** When the budget runs out, the call is refused and never made.
-3. **A watchdog revokes the grant** of an agent that keeps probing a boundary, for example after a prompt injection.
+3. **A watchdog revokes the grant** of an agent that keeps probing a boundary, for example after a prompt injection,
+   and **tripwires** (decoy tools and planted canary values) stop a run before the call that touches one is made.
+   For tools that can't be undone, an opt-in **reviewer** model checks each call against what the user asked
+   before Aegis admits it, and refuses what goes beyond it (`review={"tools": ["payments.refund"]}`).
 4. **`agentdynamics policy export` writes a tighter, least-privilege policy** from observed behaviour. `aegis ratify` and `aegis drift` verify its declaration; `agentdynamics policy check` replays the traffic that actually ran through it and reports how much it would refuse, so a policy change can be gated in CI.
 
 The console's **Governance** page shows denials, budget stops, revocations, unused grants and budget headroom per policy version. See [docs/GOVERNANCE.md](docs/GOVERNANCE.md).
@@ -188,6 +194,8 @@ The store is a SQLite file by default. For several instances behind a load balan
 and operate like any other, use Postgres: `pip install "agentdynamics[postgres]"` and set
 `AGENTDYNAMICS_DB_URL=postgresql://...` (or `deploy/docker-compose.postgres.yml`). One instance is elected to
 write the analysis; the others serve the console and take ingest, and one takes over if the writer goes away.
+An existing install moves with its history: `agentdynamics store copy --to postgresql://...`, then
+`agentdynamics store verify`.
 
 - **Security:** role-based API keys, optionally scoped to projects. A scoped key reads only its projects' tasks,
   runs, steps and events, on every endpoint, and can't write into or over another project's traces.

{agentdynamics-0.7.0 → agentdynamics-0.9.0}/agentdynamics/__init__.py

@@ -5,7 +5,7 @@
 
 See agentdynamics.autotrace for @trace / @tool / span.
 """
-__version__ = "0.7.0"
+__version__ = "0.9.0"
 
 from .autotrace import flush, init, llm_call, outcome, record_llm, span, tool, trace  # noqa: E402,F401
 

{agentdynamics-0.7.0 → agentdynamics-0.9.0}/agentdynamics/__main__.py

@@ -187,6 +187,74 @@ def cmd_alerts(a, data):
     return 1 if failed else 0
 
 
+def cmd_store(a, data):
+    from . import config as cfgmod, migrate
+    from .pg import redact_url, schema_name
+    cfg = cfgmod.load(data)
+    url = a.to or (cfg.get("store") or {}).get("url")
+    if not url:
+        print("say where to: --to postgresql://... (or set [store] url / AGENTDYNAMICS_DB_URL)")
+        return 2
+    schema = schema_name(a.schema or (cfg.get("store") or {}).get("schema") or "agentdynamics", data)
+    if a.action == "verify":
+        ok = True
+        for what, (lite, pg) in migrate.verify(data, url, schema).items():
+            same = lite == pg
+            ok &= same
+            print(f"  {what:<14} SQLite {lite:>10}  Postgres {pg:>10}  {'ok' if same else 'DIFFERENT'}")
+        return 0 if ok else 1
+    print(f"copying {os.path.join(data, 'agentdynamics.db')} -> {redact_url(url)} (schema {schema})")
+    try:
+        migrate.copy(data, url, schema, force=a.force)
+    except migrate.CopyError as ex:
+        print(f"not copied: {ex}")
+        return 1
+    print("done. Point the servers at it ([store] url or AGENTDYNAMICS_DB_URL); their first refresh rebuilds the "
+          "analysis from what was copied. API keys and pricing.json are configuration: copy them yourself.")
+    return 0
+
+
+def cmd_revoke(a, data):
+    from . import config as cfgmod, store
+    con = store.connect(*store.target(cfgmod.load(data), data))
+    now = time.time()
+    if a.clear:
+        n = store.clear_revocation(con, a.clear, now)
+        print("cleared" if n else f"no active directive {a.clear}")
+        return 0 if n else 1
+    if a.list:
+        rows_ = store.revocations(con, now)
+        for d in rows_:
+            status = "cleared" if d["cleared"] else "expired" if d["expires"] <= now else "active"
+            what = "revoke" if d["kind"] == "revoke" else "restrict: " + ", ".join(
+                (d["spec"].get("tools") or []) + ([f"budget x{d['spec']['budget']:g}"] if "budget" in d["spec"] else []))
+            print(f"{d['id']}  {status:<8} {d['project'] or '(all projects)':<18} {d['agent'] or '(all agents)':<18} "
+                  f"until {time.strftime('%Y-%m-%d %H:%M', time.localtime(d['expires']))}  [{d['source']}] {what}  {d['reason']}")
+        if not rows_:
+            print("no directives")
+        return 0
+    if not a.reason:
+        print("--reason is required: it goes into the kernel's audit log")
+        return 2
+    who = f"{'agent ' + a.agent if a.agent else 'every agent'} in {a.project or 'every project'}"
+    if a.tools or a.budget is not None:
+        from .engine import restrict_spec
+        try:
+            spec = restrict_spec((a.tools or "").split(","), a.budget)
+        except ValueError as ex:
+            print(ex)
+            return 2
+        rid = store.add_revocation(con, a.project, a.agent, a.reason, "operator", now, now + a.minutes * 60,
+                                   kind="restrict", spec=spec)
+        print(f"{rid}: {who} loses {', '.join(spec.get('tools', [])) or 'part of its budget'} for {a.minutes:g} min, "
+              "in processes using agentdynamics.integrations.aegis.instrument(..., revocations=True)")
+        return 0
+    rid = store.add_revocation(con, a.project, a.agent, a.reason, "operator", now, now + a.minutes * 60)
+    print(f"{rid}: {who} is revoked "
+          f"for {a.minutes:g} min, in processes using agentdynamics.integrations.aegis.instrument(..., revocations=True)")
+    return 0
+
+
 def cmd_keys(a, data):
     from .config import keys_path, load_keys, save_keys
     keys = load_keys(data)
@@ -373,6 +441,21 @@ def main(argv=None):
     al.add_argument("action", choices=["status", "test"],
                     help="status: destinations, queue and firing SLO alerts; test: send a test alert to each")
     al.add_argument("--to", metavar="NAME", help="test only this destination (its name, as status shows it)")
+    st = sub.add_parser("store", help="copy a SQLite store into Postgres (copy), or compare the two (verify)")
+    st.add_argument("action", choices=["copy", "verify"])
+    st.add_argument("--to", metavar="URL", help="the Postgres URL (default: [store] url / AGENTDYNAMICS_DB_URL)")
+    st.add_argument("--schema", help="the Postgres schema (default: [store] schema, else agentdynamics)")
+    st.add_argument("--force", action="store_true", help="merge into a schema that already holds another store's data")
+    rv = sub.add_parser("revoke", help="revoke an agent's Aegis grants wherever it runs (applied in-process by "
+                                       "agentdynamics.integrations.aegis), or list/clear directives")
+    rv.add_argument("--agent", help="the agent (grant) name; omit to revoke every grant in the project")
+    rv.add_argument("--project", help="omit for every project")
+    rv.add_argument("--reason", help="why (it goes into the kernel's audit log)")
+    rv.add_argument("--minutes", type=float, default=60, help="how long new grants of that agent are refused too")
+    rv.add_argument("--tools", help="restrict instead of revoke: take these tools away (comma-separated)")
+    rv.add_argument("--budget", type=float, help="restrict: keep only this share (0-1) of what remains of the budget")
+    rv.add_argument("--list", action="store_true", help="list directives")
+    rv.add_argument("--clear", metavar="ID", help="stop a directive (grants it revoked stay revoked)")
     sub.add_parser("ingest", help="scan sources once and rebuild the database")
     rp = sub.add_parser("report", help="print a text summary")
     rp.add_argument("--project")
@@ -397,6 +480,10 @@ def main(argv=None):
         return cmd_keys(a, a.data)
     if cmd == "alerts":
         return cmd_alerts(a, a.data)
+    if cmd == "revoke":
+        return cmd_revoke(a, a.data)
+    if cmd == "store":
+        return cmd_store(a, a.data)
 
     from .engine import Engine
     eng = Engine(a.data, a.claude_root or None)

{agentdynamics-0.7.0 → agentdynamics-0.9.0}/agentdynamics/alerts.py

@@ -10,7 +10,9 @@ Destinations are `[[alerts.webhooks]]` in agentdynamics.toml:
     format = "pagerduty"                 # json (default) | slack | pagerduty
     routing_key_env = "PD_ROUTING_KEY"   # PagerDuty Events API v2 integration key (or routing_key = "...")
     min_severity = "critical"            # info | warning (default) | critical
-    kinds = ["events", "slos"]           # default ["events"]; "slos" adds SLO burn-rate alerts
+    kinds = ["events", "slos"]           # default ["events"]; "slos" adds SLO burn-rate alerts,
+                                         # "revocations" the directives that stop an agent (server-side #8),
+                                         # "incidents" one alert per security incident (incidents.py)
     projects = ["checkout"]              # optional: only these projects
     rules = ["run_failed"]               # optional: only these health rules
 
@@ -18,6 +20,9 @@ What is sent:
 - A health-rule event is a point in time. Slack and JSON get each one (JSON in the original
   `{"source": "agentdynamics", "events": [...]}` shape, a contract). PagerDuty gets one alert per rule,
   project and task type (the dedup key), so a burst of runaway-cost tasks is one incident, not fifty.
+- An incident is several security signals about one agent: a trigger when it opens, another when it
+  escalates (warning to critical), a resolve when a person gives it a verdict. A destination that takes
+  incidents instead of events hears once about an agent tripping five health rules, not five times.
 - An SLO alert has a start and an end: a trigger when a burn-rate policy starts firing and a resolve when
   it stops (slo.alert_conditions). What is firing is kept in the durable `alert_state` table, so a restart
   neither repeats a trigger nor forgets to resolve.
@@ -37,7 +42,7 @@ from collections import OrderedDict
 from urllib.parse import quote
 
 FORMATS = ("json", "slack", "pagerduty")
-KINDS = ("events", "slos")
+KINDS = ("events", "slos", "revocations", "incidents")
 SEV = {"info": 1, "warning": 2, "critical": 3}
 # https://raw.githubusercontent.com/PagerDuty/api-schema/main/reference/events-v2/openapiv3.json
 PAGERDUTY_URL = "https://events.pagerduty.com/v2/enqueue"
@@ -135,6 +140,23 @@ def from_slo(key, c, action, now):
             "project": (c.get("scope") or {}).get("project") or None, "slo": c["slo"], "ts": now, "condition": c}
 
 
+def from_revocation(d):
+    who = f"agent {d['agent']}" if d.get("agent") else "every agent"
+    where = f" in project {d['project']}" if d.get("project") else ""
+    until = time.strftime("%Y-%m-%d %H:%M", time.localtime(d["expires"]))
+    return {"kind": "revocation", "action": "trigger", "key": f"revocation/{d['id']}", "severity": "critical",
+            "summary": f"Revoked {who}{where} until {until}: {d['reason']}", "project": d.get("project"),
+            "ts": d["created"], "revocation": d}
+
+
+def from_incident(inc, summary, action="trigger"):
+    """`action`: trigger (opened), escalate (now more severe) or resolve (a person gave a verdict)."""
+    return {"kind": "incident", "action": action, "key": f"incident/{inc['id']}", "severity": inc["severity"],
+            "summary": summary, "project": inc["project"], "ts": inc["resolved_at"] if action == "resolve" else inc["updated"],
+            "incident": {k: inc[k] for k in ("id", "project", "agent", "workflow", "opened", "updated", "status",
+                                               "severity", "signals", "verdict", "note")}}
+
+
 # ---------------------------------------------------------------- formats
 
 def _dedup_key(key):
@@ -148,6 +170,10 @@ def link(console_url, alert):
     base = console_url.rstrip("/")
     if alert["kind"] == "event":
         return f"{base}/#/task/{quote(alert['event']['task_id'], safe='')}"
+    if alert["kind"] == "revocation":
+        return f"{base}/#/governance"
+    if alert["kind"] == "incident":
+        return f"{base}/#/incident/{quote(alert['incident']['id'], safe='')}"
     return f"{base}/#/slos"
 
 
@@ -172,6 +198,13 @@ def render(dest, alerts, console_url=""):
                  "link": link(console_url, a), **a["condition"]} for a in alerts if a["kind"] == "slo"]
         if slos:
             bodies.append({"source": "agentdynamics", "slo_alerts": slos})
+        revs = [dict(a["revocation"], summary=a["summary"]) for a in alerts if a["kind"] == "revocation"]
+        if revs:
+            bodies.append({"source": "agentdynamics", "revocations": revs})
+        incs = [dict(a["incident"], action=a["action"], summary=a["summary"], link=link(console_url, a))
+                for a in alerts if a["kind"] == "incident"]
+        if incs:
+            bodies.append({"source": "agentdynamics", "incidents": incs})
         return bodies
     # one line (Slack) or one alert (PagerDuty) per dedup key, so a burst of the same violation reads as one
     groups = OrderedDict()
@@ -185,6 +218,14 @@ def render(dest, alerts, console_url=""):
             where = " / ".join(_slack_escape(x) for x in (a.get("project"), a.get("task_type")) if x)
             url = link(console_url, a)
             ref = f" · <{url}|{'latest' if len(g) > 1 else 'open'}>" if url else ""
+            if a["kind"] == "revocation":
+                lines.append(f"*REVOKED* · {_slack_escape(a['summary'])}{ref}")
+                continue
+            if a["kind"] == "incident":
+                head = {"trigger": "INCIDENT", "escalate": "ESCALATED", "resolve": "RESOLVED"}[a["action"]]
+                verdict = f" ({a['incident']['verdict'].replace('_', ' ')})" if a["action"] == "resolve" else ""
+                lines.append(f"*{head}*{verdict} · {_slack_escape(a['summary'])}{ref}")
+                continue
             if a["kind"] == "event":
                 e = a["event"]
                 times = f" ×{len(g)}" if len(g) > 1 else ""
@@ -200,7 +241,13 @@ def render(dest, alerts, console_url=""):
         if a["action"] == "resolve":
             bodies.append({"event_action": "resolve", "dedup_key": _dedup_key(key)})
             continue
-        if a["kind"] == "event":
+        if a["kind"] == "revocation":
+            d = a["revocation"]
+            details, group, klass = dict(d), d.get("agent") or "all agents", "revocation"
+        elif a["kind"] == "incident":
+            i = a["incident"]
+            details, group, klass = dict(i), i.get("agent") or i.get("workflow") or "all agents", "incident"
+        elif a["kind"] == "event":
             e = a["event"]
             details = {"rule": e["rule"], "rule_id": e["rule_id"], "message": e["message"], "value": e.get("value"),
                        "task_id": e["task_id"], "run_id": e.get("run_id"), "project": e["project"],