@sponsio/sdk 0.1.0 → 0.2.0-alpha.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (123) hide show
  1. package/contracts/benchmark/agentdojo.yaml +551 -0
  2. package/contracts/benchmark/odcv_bench.yaml +393 -0
  3. package/contracts/benchmark/redcode_exec.yaml +488 -0
  4. package/contracts/benchmark/swebench.yaml +176 -0
  5. package/contracts/benchmark/tau2_bench.yaml +984 -0
  6. package/contracts/capability/destructive.yaml +213 -0
  7. package/contracts/capability/factual-claims.yaml +79 -0
  8. package/contracts/capability/host-config-integrity.yaml +42 -9
  9. package/contracts/capability/self-modify.yaml +32 -1
  10. package/contracts/capability/shell.yaml +19 -10
  11. package/contracts/incident/claude-code-secret-bypass.yaml +48 -16
  12. package/contracts/incident/cursor-railway-wipe.yaml +60 -4
  13. package/contracts/incident/openclaw.yaml +9 -62
  14. package/contracts/premium/.gitkeep +5 -6
  15. package/dist/cli/cli.js +32 -0
  16. package/dist/cli/cli.js.map +1 -1
  17. package/dist/cli/packs.d.ts.map +1 -1
  18. package/dist/cli/packs.js +14 -8
  19. package/dist/cli/packs.js.map +1 -1
  20. package/dist/cli/patterns.d.ts +5 -4
  21. package/dist/cli/patterns.d.ts.map +1 -1
  22. package/dist/cli/patterns.js +11 -10
  23. package/dist/cli/patterns.js.map +1 -1
  24. package/dist/cli/prompt.js +5 -5
  25. package/dist/cli/prompt.js.map +1 -1
  26. package/dist/cli/scan.js +2 -2
  27. package/dist/cli/skill.d.ts +1 -1
  28. package/dist/cli/skill.js +1 -1
  29. package/dist/cli/validate.js +5 -5
  30. package/dist/cli/validate.js.map +1 -1
  31. package/dist/cloud/client.d.ts +76 -0
  32. package/dist/cloud/client.d.ts.map +1 -0
  33. package/dist/cloud/client.js +198 -0
  34. package/dist/cloud/client.js.map +1 -0
  35. package/dist/cloud/evidence.d.ts +108 -0
  36. package/dist/cloud/evidence.d.ts.map +1 -0
  37. package/dist/cloud/evidence.js +184 -0
  38. package/dist/cloud/evidence.js.map +1 -0
  39. package/dist/cloud/privacy.d.ts +67 -0
  40. package/dist/cloud/privacy.d.ts.map +1 -0
  41. package/dist/cloud/privacy.js +170 -0
  42. package/dist/cloud/privacy.js.map +1 -0
  43. package/dist/cloud/view-model.d.ts +49 -0
  44. package/dist/cloud/view-model.d.ts.map +1 -0
  45. package/dist/cloud/view-model.js +172 -0
  46. package/dist/cloud/view-model.js.map +1 -0
  47. package/dist/core/config-loader.d.ts.map +1 -1
  48. package/dist/core/config-loader.js +81 -9
  49. package/dist/core/config-loader.js.map +1 -1
  50. package/dist/core/evaluator.d.ts +11 -1
  51. package/dist/core/evaluator.d.ts.map +1 -1
  52. package/dist/core/evaluator.js +275 -13
  53. package/dist/core/evaluator.js.map +1 -1
  54. package/dist/core/formula.d.ts +76 -17
  55. package/dist/core/formula.d.ts.map +1 -1
  56. package/dist/core/formula.js +83 -1
  57. package/dist/core/formula.js.map +1 -1
  58. package/dist/core/grounding.d.ts +44 -8
  59. package/dist/core/grounding.d.ts.map +1 -1
  60. package/dist/core/grounding.js +197 -26
  61. package/dist/core/grounding.js.map +1 -1
  62. package/dist/core/nl-parser.d.ts +2 -2
  63. package/dist/core/nl-parser.d.ts.map +1 -1
  64. package/dist/core/nl-parser.js +617 -41
  65. package/dist/core/nl-parser.js.map +1 -1
  66. package/dist/core/pack-loader.d.ts.map +1 -1
  67. package/dist/core/pack-loader.js +14 -2
  68. package/dist/core/pack-loader.js.map +1 -1
  69. package/dist/core/parser.d.ts.map +1 -1
  70. package/dist/core/parser.js +30 -1
  71. package/dist/core/parser.js.map +1 -1
  72. package/dist/core/pattern-factory.d.ts.map +1 -1
  73. package/dist/core/pattern-factory.js +14 -1
  74. package/dist/core/pattern-factory.js.map +1 -1
  75. package/dist/core/patterns.d.ts +118 -1
  76. package/dist/core/patterns.d.ts.map +1 -1
  77. package/dist/core/patterns.js +189 -10
  78. package/dist/core/patterns.js.map +1 -1
  79. package/dist/core/session-log.d.ts.map +1 -1
  80. package/dist/core/session-log.js +20 -1
  81. package/dist/core/session-log.js.map +1 -1
  82. package/dist/core/sto.d.ts +21 -27
  83. package/dist/core/sto.d.ts.map +1 -1
  84. package/dist/core/sto.js +30 -33
  85. package/dist/core/sto.js.map +1 -1
  86. package/dist/core/tool-names.d.ts +59 -0
  87. package/dist/core/tool-names.d.ts.map +1 -0
  88. package/dist/core/tool-names.js +128 -0
  89. package/dist/core/tool-names.js.map +1 -0
  90. package/dist/index.d.ts +181 -32
  91. package/dist/index.d.ts.map +1 -1
  92. package/dist/index.js +415 -67
  93. package/dist/index.js.map +1 -1
  94. package/dist/integrations/claude-agent.d.ts +5 -0
  95. package/dist/integrations/claude-agent.d.ts.map +1 -1
  96. package/dist/integrations/claude-agent.js +42 -13
  97. package/dist/integrations/claude-agent.js.map +1 -1
  98. package/dist/integrations/google-adk.d.ts +5 -3
  99. package/dist/integrations/google-adk.d.ts.map +1 -1
  100. package/dist/integrations/google-adk.js +9 -5
  101. package/dist/integrations/google-adk.js.map +1 -1
  102. package/dist/integrations/langchain.d.ts +7 -0
  103. package/dist/integrations/langchain.d.ts.map +1 -1
  104. package/dist/integrations/langchain.js +21 -3
  105. package/dist/integrations/langchain.js.map +1 -1
  106. package/dist/integrations/openai-agents.d.ts +22 -10
  107. package/dist/integrations/openai-agents.d.ts.map +1 -1
  108. package/dist/integrations/openai-agents.js +55 -15
  109. package/dist/integrations/openai-agents.js.map +1 -1
  110. package/dist/integrations/openai.d.ts +11 -0
  111. package/dist/integrations/openai.d.ts.map +1 -1
  112. package/dist/integrations/openai.js +130 -48
  113. package/dist/integrations/openai.js.map +1 -1
  114. package/dist/integrations/vercel-ai.d.ts +20 -9
  115. package/dist/integrations/vercel-ai.d.ts.map +1 -1
  116. package/dist/integrations/vercel-ai.js +131 -32
  117. package/dist/integrations/vercel-ai.js.map +1 -1
  118. package/package.json +17 -9
  119. package/prompts/onboard.md +26 -17
  120. package/prompts/scan.md +3 -5
  121. package/skills/SKILL.md +13 -83
  122. package/contracts/core/llm_safety.yaml +0 -140
  123. package/prompts/refresh.md +0 -118
package/prompts/scan.md CHANGED
@@ -105,11 +105,9 @@ to enforce and they wedge the agent. For each tool in
105
105
  here — take them seriously. A policy doc saying "no destructive
106
106
  Railway calls without confirmation" should produce a concrete
107
107
  arg_blacklist + irreversible_once pair.
108
- - **`trace_summary.total_events`** > 0: ordering / sequence rules are
109
- more reliable now (you can run `sponsio refresh --emit-traces` to
110
- mine them properly). Without traces, prefer single-event patterns
111
- (`arg_blacklist`, `rate_limit`, `irreversible_once`) over
112
- trace-aware ones.
108
+ - Prefer single-event patterns (`arg_blacklist`, `rate_limit`,
109
+ `irreversible_once`) unless the code clearly establishes a required
110
+ ordering; only then reach for sequence-shaped rules.
113
111
  - **`existing_yaml`**: don't duplicate rules already there. Read its
114
112
  contracts, only emit gaps.
115
113
 
package/skills/SKILL.md CHANGED
@@ -1,11 +1,11 @@
1
1
  ---
2
2
  name: sponsio
3
- description: Install, observe, tune, enforce, and periodically refresh Sponsio — a runtime contract layer for LLM agents that blocks unsafe tool calls and scores output quality against declared rules. Use when the user wants to set up / add / install Sponsio, add guardrails or runtime safety to an LLM agent, generate or refine a sponsio.yaml, audit tool configurations for risks (data leaks, unguarded writes, missing confirmations), explain or review existing contracts, check what Sponsio would have blocked (`sponsio report`), refresh the contract library from recent traces (`sponsio refresh`), move from observe to enforce mode, or debug why a contract is (or isn't) firing. Triggers on phrases like "set up sponsio", "add sponsio", "install sponsio", "add guardrails", "monitor my agent", "harden my agent", "audit my agent", "generate contracts", "explain my sponsio.yaml", "sponsio report", "refresh contracts", "update my sponsio.yaml from traces", "flip to enforce", "false positive", "why is this rule firing".
3
+ description: Install, observe, tune, and enforce Sponsio: a runtime contract layer for LLM agents that blocks unsafe tool calls and scores output quality against declared rules. Use when the user wants to set up / add / install Sponsio, add guardrails or runtime safety to an LLM agent, generate or refine a sponsio.yaml, audit tool configurations for risks (data leaks, unguarded writes, missing confirmations), explain or review existing contracts, check what Sponsio would have blocked (`sponsio report`), move from observe to enforce mode, or debug why a contract is (or isn't) firing. Triggers on phrases like "set up sponsio", "add sponsio", "install sponsio", "add guardrails", "monitor my agent", "harden my agent", "audit my agent", "generate contracts", "explain my sponsio.yaml", "sponsio report", "flip to enforce", "false positive", "why is this rule firing".
4
4
  ---
5
5
 
6
6
  # Sponsio — Agent Safety Lifecycle Companion
7
7
 
8
- Sponsio is a Python/TypeScript runtime safety layer for LLM agents: it evaluates deterministic contracts against each tool call and can block (enforce) or just log (observe) violations. (LLM-judged stochastic contracts — tone, prompt-injection, semantic PII, scope respect — are a Sponsio Cloud feature available via `pip install sponsio[cloud]`; the OSS engine is det-only.) This skill covers the full lifecycle — first-time setup, contract authoring/review, observe-mode tuning, and flipping to enforce — by orchestrating Sponsio's CLI and explaining its output in plain language.
8
+ Sponsio is a Python/TypeScript runtime safety layer for LLM agents: it evaluates deterministic contracts against each tool call and can block (enforce) or just log (observe) violations. The engine is deterministic-only. This skill covers the full lifecycle — first-time setup, contract authoring/review, observe-mode tuning, and flipping to enforce — by orchestrating Sponsio's CLI and explaining its output in plain language.
9
9
 
10
10
  This skill does NOT reimplement Sponsio's logic; it calls the CLI and interprets results.
11
11
 
@@ -18,7 +18,6 @@ Dispatch by what the user is trying to do. Pick ONE workflow and follow it; do n
18
18
  | Setting up Sponsio for the first time in a project ("add sponsio", "install sponsio", "add guardrails") | **W1 — Initial setup** |
19
19
  | Handing you a codebase and asking "what could go wrong?" / wants a fresh contract file from scratch / has a policy doc to encode | **W2 — Audit & refine** |
20
20
  | Has Sponsio running in observe mode and wants to review violations, tune thresholds, silence false positives | **W3 — Tune in observe** |
21
- | Wants to re-mine contracts from accumulated production traces / periodically maintain the library | **W3b — Refresh from traces** |
22
21
  | Ready to ship — wants to move from observe to enforce, needs regression confidence | **W4 — Flip to enforce** |
23
22
  | Sponsio errored, a rule isn't firing when it should, a rule is firing when it shouldn't | **W5 — Troubleshoot** |
24
23
 
@@ -64,7 +63,7 @@ Goal: from "project has no Sponsio" to "agent runs under observe mode with a san
64
63
 
65
64
  | Pack | Auto-included when… | Notes |
66
65
  |---|---|---|
67
- | `sponsio:core/universal` | Always | Empty stub on OSS — kept so existing `include:` lines don't error. The output-quality contracts (injection / jailbreak / toxic / PII / harm) live in `sponsio:core/llm_safety` and require Sponsio Cloud (`pip install sponsio[cloud]`) |
66
+ | `sponsio:core/universal` | Always | Empty stub, kept so existing `include:` lines don't error. |
68
67
  | `sponsio:core/runaway` | Framework runs a multi-step loop (langgraph/crewai/…) | token budget, delegation depth, loop detection; no LLM calls |
69
68
  | `sponsio:capability/shell` | A tool name matches `{bash, shell, exec, execute, execute_command, run_command, run_shell, run_bash, terminal, subprocess}` | Auto-fills `tool_rename:` if the user's tool name isn't the canonical `exec` |
70
69
  | `sponsio:capability/filesystem` | A tool name matches `{read, read_file, open_file, write, write_file, edit, edit_file, apply_patch, patch_file, ...}` | Auto-fills `tool_rename:` and `workspace:` |
@@ -92,17 +91,16 @@ Sponsio contracts come from four sources, mixable in one yaml:
92
91
  | 1 | **Shipped packs** | Pre-built, parameterized rule sets (`sponsio:core/universal`, `sponsio:capability/shell`, …) | Hand-add `include: [sponsio:<spec>]` — or W1's `onboard` does it automatically |
93
92
  | 2 | **Extraction** | AST + optional LLM inference from your code / policy docs / execution traces | `sponsio scan <paths> [--llm] [--policy <doc>] [-t <trace-glob>]` |
94
93
  | 3 | **User input** | An NL sentence or a structured dict the user writes | Hand-edit `sponsio.yaml`; validate a single NL string with `sponsio validate "<NL>"` |
95
- | 4 | **Pattern library** | Deterministic parameterized templates (`rate_limit`, `must_precede`, `arg_blacklist`, …) — full list via `sponsio patterns`. Stochastic templates (`injection_free`, `tone`, `llm_judge`, …) live in Sponsio Cloud | `sponsio patterns` to browse; hand-write the YAML entry |
94
+ | 4 | **Pattern library** | Deterministic parameterized templates (`rate_limit`, `must_precede`, `arg_blacklist`, …) — full list via `sponsio patterns` | `sponsio patterns` to browse; hand-write the YAML entry |
96
95
 
97
96
  Match the user's input to the source(s):
98
97
 
99
98
  - "Explain / review my `sponsio.yaml`" → source 1 and/or others already applied; jump to "Explain contracts" below.
100
99
  - "Scan my agent code" → source 2, code-only.
101
100
  - "We have a security policy document" → source 2, add `--policy <doc> --llm`.
102
- - "We already have session logs / OTLP traces" → source 2, add `-t '<glob>'` (trace mining; no LLM needed).
103
101
  - "I know the pattern I want but not the syntax" → source 4, then show them the yaml entry.
104
102
 
105
- If ambiguous: ask ONE question — "(a) scan your code, (b) extract from a policy document, or (c) mine from traces?"
103
+ If ambiguous: ask ONE question — "(a) scan your code, or (b) extract from a policy document?"
106
104
 
107
105
  ### Run scan (when extraction is needed)
108
106
 
@@ -115,12 +113,9 @@ sponsio scan <PATHS> --agent <AGENT> --llm -o ./sponsio.yaml
115
113
 
116
114
  # + policy doc:
117
115
  sponsio scan <PATHS> --policy <POLICY.md> --llm -o ./sponsio.yaml
118
-
119
- # + trace mining (works on OTLP/JSON, OTLP JSONL, native, Sponsio session logs):
120
- sponsio scan <PATHS> -t '~/.sponsio/sessions/<agent>/*.jsonl' -o ./sponsio.yaml
121
116
  ```
122
117
 
123
- Scan auto-validates before writing; only contracts that parse cleanly are saved. Source-tagged with `source: scan` / `source: policy` / `source: trace`. The `source: trace` subset is the one `sponsio refresh` maintains over time (W3b).
118
+ Scan auto-validates before writing; only contracts that parse cleanly are saved. Source-tagged with `source: scan` / `source: policy`.
124
119
 
125
120
  ### Validate existing yaml (explain-only path)
126
121
 
@@ -128,7 +123,7 @@ Scan auto-validates before writing; only contracts that parse cleanly are saved.
128
123
  sponsio validate --config ./sponsio.yaml --json
129
124
  ```
130
125
 
131
- JSON shape per entry: `{nl, ok, type: "det"|"sto"|"unknown", pattern, formula, agent, section}`. `ok: false` means the NL didn't match any pattern — surface those as "ambiguous — needs refinement".
126
+ JSON shape per entry: `{nl, ok, type: "det"|"unknown", pattern, formula, agent, section}`. `ok: false` means the NL didn't match any pattern — surface those as "ambiguous — needs refinement".
132
127
 
133
128
  ### Explain contracts (use this as the report template)
134
129
 
@@ -138,7 +133,7 @@ JSON shape per entry: `{nl, ok, type: "det"|"sto"|"unknown", pattern, formula, a
138
133
  **Setup**: <N> contracts total, <sources breakdown>. Mode: <observe|enforce>.
139
134
 
140
135
  ### Active packs
141
- - `sponsio:core/universal` (empty stub on OSS — judge-based output safety lives in `sponsio:core/llm_safety` and requires Sponsio Cloud)
136
+ - `sponsio:core/universal` (empty stub, kept so existing `include:` lines don't error)
142
137
  - `sponsio:capability/shell` (11 det) — guards on <tool name after rename>
143
138
  <...>
144
139
 
@@ -211,66 +206,6 @@ With the assumption, the rule stays silent until the integration actually emits
211
206
 
212
207
  ---
213
208
 
214
- ## W3b — Refresh from traces
215
-
216
- Goal: treat `sponsio.yaml` as a **living library**, not a one-shot output. As real session logs accumulate, re-mine them to discover new patterns and retire stale ones, without clobbering anything the user wrote or tuned.
217
-
218
- ### When to run
219
-
220
- - Weekly / sprint boundary — recommended cadence for an agent seeing live traffic.
221
- - After a material behavior change (new tools, new workflow, new integration).
222
- - Before flipping to enforce (W4) — catches late-arriving trace-sourced rules that weren't present in the original scan.
223
-
224
- ### Steps
225
-
226
- 1. Dry-run first. Always.
227
-
228
- ```bash
229
- sponsio refresh --since 7d
230
- ```
231
-
232
- Prints a structured diff per agent:
233
-
234
- ```
235
- Agent: support_bot
236
- + new must_precede(validate_payment, charge_card)
237
- ~ drifted rate_limit(send_email, 5) → args [send_email, 12]
238
- - stale idempotent(list_users) (not re-observed)
239
- = 8 unchanged (source: trace, re-observed)
240
- = 12 preserved (user / scan / policy / overrides — not touched)
241
- ```
242
-
243
- 2. Review with the user. For each bucket:
244
- - `+ new` — "the agent started doing X that your current yaml doesn't cover." User usually wants this.
245
- - `~ drifted` — threshold moved. If new value is bigger, the rule was too tight; if smaller, production tightened up. User decides.
246
- - `- stale` — rule hasn't fired in this window. **Not necessarily dead** — could just be rare. Conservative default is `--mode add-only` (never remove), which we recommend for the first few refreshes until the user trusts the window size.
247
- - `= preserved` — these include every user rule, `source: scan`, `source: policy`, and anything under `overrides:`. The count being non-zero is the load-bearing invariant: refresh **only** ever changes `source: trace` entries.
248
-
249
- 3. Apply.
250
-
251
- ```bash
252
- sponsio refresh --since 7d --apply # default mode: replace-trace
253
- sponsio refresh --since 7d --apply --mode add-only # conservative: never remove
254
- ```
255
-
256
- Writes `sponsio.yaml` and backs the old file up to `sponsio.yaml.sponsio.bak`. **Comments in the YAML are not preserved** — the backup is how users recover any prose annotations they'd inlined. Warn them of this before running.
257
-
258
- ### Mode selection
259
-
260
- | Situation | Mode |
261
- |---|---|
262
- | First time running refresh, or small trace window | `add-only` (never removes) |
263
- | Healthy trace volume, stable agent behavior | `replace-trace` (default — recent traces are authoritative) |
264
- | Window just covers a launch / migration / incident (abnormal traffic) | Skip — artifact traces mislead the miner |
265
-
266
- ### Do NOT
267
-
268
- - Do NOT run refresh on traces from an agent still in early iteration. Wait until the workflow is stable; otherwise every sprint invalidates the library.
269
- - Do NOT use `--apply` without first showing the dry-run diff.
270
- - Do NOT remove `source: scan` / `source: policy` entries in the yaml by hand just because refresh doesn't touch them — they represent knowledge refresh can't reconstruct (code AST, policy docs). If you think one is wrong, edit `overrides:`, don't delete.
271
-
272
- ---
273
-
274
209
  ## W4 — Flip to enforce
275
210
 
276
211
  Goal: move from observe to enforce with regression confidence — no more logging, actual blocking. **This is a production change**; don't skip the checks.
@@ -332,7 +267,6 @@ Check in this order:
332
267
  1. `sponsio validate --config sponsio.yaml --json` — does the contract parse? `ok: false` means it was silently dropped.
333
268
  2. The tool name in the contract matches the actual tool name the agent calls (LangGraph `node_id` vs the `@tool` Python name is a common trap; see `tool_rename:`).
334
269
  3. The rule has an `A:` (assumption) that isn't holding. Look at the trace — is the precondition ever true?
335
- 4. The rule is a `sto` pattern (`injection_free`, `tone_*`, `semantic_pii_free`, …). The OSS engine ships no evaluator for those — `pip install sponsio[cloud]` to activate the judge pipeline. Without Cloud, OSS rejects the contract loudly at config load.
336
270
 
337
271
  ### "A rule is firing that shouldn't."
338
272
 
@@ -393,10 +327,6 @@ agents:
393
327
  runtime:
394
328
  mode: observe # "observe" | "enforce"
395
329
  dashboard: http://localhost:8000 # URL | true | false | null
396
-
397
- judge: # Sponsio Cloud only — required when any include uses sto patterns
398
- provider: openai # openai | anthropic | gemini
399
- model: gpt-4o-mini
400
330
  ```
401
331
 
402
332
  Both `A` and `G` accept: scalar NL string, list (AND), or structured dict `{pattern, args, source?}`.
@@ -407,7 +337,7 @@ Placeholders rewritten at include-time: `<workspace>/` (from agent's `workspace:
407
337
 
408
338
  ## Pattern reference
409
339
 
410
- Full list: `sponsio patterns` (deterministic templates only). LLM-judged stochastic templates are a Sponsio Cloud feature.
340
+ Full list: `sponsio patterns` (deterministic templates only).
411
341
 
412
342
  ### Core Temporal (14 det)
413
343
  | Pattern | Meaning |
@@ -436,8 +366,8 @@ Full list: `sponsio patterns` (deterministic templates only). LLM-judged stochas
436
366
  ### Resource & Delegation (3 det)
437
367
  `token_budget(max_tokens)` · `arg_value_range(tool, field, min, max)` · `delegation_depth_limit(max_depth)`
438
368
 
439
- ### Soft Evaluators
440
- LLM-judged stochastic atoms (`tone_*`, `relevance`, `llm_judge`, `injection_free`, `semantic_pii_free`, `scope_respect`, …) ship in Sponsio Cloud (`pip install sponsio[cloud]`). The OSS engine refuses to load them at config time with a clear pointer. Det response-quality patterns (`no_pii` regex, `max_length`, `no_keywords`) remain available in OSS.
369
+ ### Response-quality patterns (3 det)
370
+ Deterministic response-quality patterns (`no_pii` regex, `max_length`, `no_keywords`) check `llm_said` and need no judge.
441
371
 
442
372
  ---
443
373
 
@@ -492,8 +422,8 @@ If asked for something out of scope (e.g., "also check my DB schema"), say so an
492
422
 
493
423
  This skill only uses these. Internal refactors are safe as long as these stay stable.
494
424
 
495
- 1. **CLI**: `sponsio onboard [--apply]`, `sponsio scan PATHS [--agent N] [--llm] [--policy P] [-t GLOB] [-o FILE] [--append]`, `sponsio refresh [-c FILE] [-a AGENT] [-t GLOB] [--since DUR] [--mode add-only|replace-trace] [--apply]`, `sponsio validate [--config FILE | "NL string"] [--json]`, `sponsio check --trace FILE --config FILE --agent ID`, `sponsio report --agent ID --since DUR`, `sponsio doctor`, `sponsio patterns`, `sponsio packs`, `sponsio skill install [--tool cursor|claude|codex|both|auto]`. Exit 0 on success.
496
- 2. **YAML**: top-level `agents:` as dict; each agent has optional `include:` / `tool_rename:` / `overrides:` / `workspace:` and required `contracts:`; top-level `runtime:` and `judge:`.
425
+ 1. **CLI**: `sponsio onboard [--apply]`, `sponsio scan PATHS [--agent N] [--llm] [--policy P] [-o FILE] [--append]`, `sponsio validate [--config FILE | "NL string"] [--json]`, `sponsio check --trace FILE --config FILE --agent ID`, `sponsio report --agent ID --since DUR`, `sponsio doctor`, `sponsio patterns`, `sponsio packs`, `sponsio skill install [--tool cursor|claude|codex|both|auto]`. Exit 0 on success.
426
+ 2. **YAML**: top-level `agents:` as dict; each agent has optional `include:` / `tool_rename:` / `overrides:` / `workspace:` and required `contracts:`; top-level `runtime:`.
497
427
  3. **Patterns**: names in the table above keep their semantics. Renaming is a breaking change for this skill.
498
428
  4. **`validate --json` shape**: per-contract `ok` / `type` / `pattern` / `formula` / `agent`.
499
429
  5. **`onboard` JSON report shape**: `out_path` / `tools_count` / `contracts_count` / `mode` / `framework` / `provider` / `doctor_results` / `apply_result.diff`.
@@ -1,140 +0,0 @@
1
- # =============================================================================
2
- # sponsio/contracts/core/llm_safety.yaml
3
- #
4
- # !! Sponsio Cloud only !! — this pack is composed entirely of LLM-judged
5
- # stochastic atoms (`injection_free`, `jailbreak_free`, `harmful`,
6
- # `toxic_free`, `semantic_pii_free`). The OSS engine ships no evaluator
7
- # for them and will refuse to load this pack with a clear pointer to
8
- # `pip install sponsio[cloud]`.
9
- #
10
- # It still ships in the OSS package so Cloud installs see the same on-disk
11
- # layout — Cloud's StoEvaluator (auto-discovered via the
12
- # `sponsio.evaluators` entry-point group) is what actually evaluates
13
- # these contracts.
14
- #
15
- # Once Cloud is installed, opt in from your config:
16
- #
17
- # include:
18
- # - sponsio:core/llm_safety
19
- #
20
- # Why opt-in even on Cloud: every contract here triggers a judge LLM call
21
- # on each evaluation. That's the right default for response-quality work
22
- # but the wrong default for tool-call-only agents (no LLM responses to
23
- # score → judge calls add latency without value). Splitting out of
24
- # `core/universal` was the user-driven fix for "why is the SRE tool-call
25
- # demo running prompt-injection scoring".
26
- #
27
- # Source tag convention for this file:
28
- # library:tier0.<category> where category ∈
29
- # adversarial | harm | privacy | scope
30
- # =============================================================================
31
-
32
- version: "1"
33
-
34
- # NOTE: no `tools:` section here because these contracts are response-scoped.
35
- # They evaluate against llm_response events, not tool calls. Agents that
36
- # include this file will add their own `tools:` inventory separately.
37
-
38
- agents:
39
- # Template agent id — replaced at include time by the including config.
40
- # The include mechanism substitutes this section under the real agent id.
41
- "*":
42
- contracts:
43
-
44
- # -- Adversarial resistance ----------------------------------------
45
- #
46
- # `beta` weights the cost of a *missed* violation in the contract
47
- # score. Higher beta = a missed violation hurts the score more,
48
- # so the judge has to be near-certain the response is *clean*
49
- # before allowing it. We ship the values below as defaults
50
- # calibrated for the threat profile (slip-through cost) of each
51
- # category. Override per-agent in your config if your workload
52
- # has different stakes.
53
-
54
- # 0.95 — prompt-injection slip-throughs cause downstream tool
55
- # misuse with full agent privileges. High slip-through cost.
56
- - desc: "Every LLM response must be free of prompt-injection compliance"
57
- G:
58
- pattern: injection_free
59
- context_scope: full_trace
60
- source: "library:tier0.adversarial"
61
- beta: 0.95
62
-
63
- # 0.90 — jailbreak compliance is bad but mostly self-contained
64
- # (response-level harm, not tool-level harm).
65
- - desc: "LLM must not comply with jailbreak attempts (DAN, roleplay, hypothetical bypass)"
66
- G:
67
- pattern: jailbreak_free
68
- source: "library:tier0.adversarial"
69
- beta: 0.9
70
-
71
- # -- Harm avoidance -----------------------------------------------
72
-
73
- # 0.95 — real-world-harm enablement (CSAM, weapons, self-harm
74
- # methods) carries the same high slip-through cost as injection.
75
- - desc: "Response must not encourage or enable real-world harm"
76
- G:
77
- pattern: harmful
78
- source: "library:tier0.harm"
79
- beta: 0.95
80
-
81
- # 0.90 — toxic language is corrosive but not safety-critical;
82
- # match the jailbreak weight rather than the harm weight.
83
- - desc: "Response free of toxic / harassing language"
84
- G:
85
- pattern: toxic_free
86
- source: "library:tier0.harm"
87
- beta: 0.9
88
-
89
- # -- Privacy ------------------------------------------------------
90
-
91
- # 0.95 — PII slip-throughs are regulatory issues (GDPR / CCPA)
92
- # not just trust issues. High slip-through cost.
93
- - desc: "Response free of contextual PII (names tied to conditions, inferable identity)"
94
- G:
95
- pattern: semantic_pii_free
96
- source: "library:tier0.privacy"
97
- beta: 0.95
98
-
99
- # -- Scope --------------------------------------------------------
100
- #
101
- # `scope_respect` is deliberately NOT shipped on by default. It
102
- # needs a concrete, agent-specific scope string to be useful —
103
- # something like "respond only to loan-origination questions for
104
- # the approved product catalogue". A generic default like "do
105
- # not act beyond the task" is irredeemably abstract: the judge
106
- # has nothing to anchor on and will either flag everything or
107
- # pass everything, depending on model temperament. Shipping
108
- # such a rule as default-on produces noise, not signal.
109
- #
110
- # Add it to your own agent config once you have a concrete scope
111
- # statement, e.g.::
112
- #
113
- # agents:
114
- # support_bot:
115
- # contracts:
116
- # - desc: "Stays within customer-support remit"
117
- # E:
118
- # pattern: scope_respect
119
- # args: ["answer questions about order status,
120
- # refunds, and shipping — no legal advice,
121
- # no medical advice, no code execution"]
122
- # beta: 0.85
123
- #
124
- # The β=0.85 default acknowledges that "out of scope" is fuzzier
125
- # than "contains PII"; a higher beta flags too aggressively on
126
- # responses that drift only slightly.
127
-
128
- # -----------------------------------------------------------------------------
129
- # What this file deliberately does NOT include
130
- # -----------------------------------------------------------------------------
131
- #
132
- # - Token / loop budgets — those live in core/budget.yaml and core/runaway.yaml
133
- # respectively, so an agent with a custom budget can include only that.
134
- #
135
- # - Tool-specific protections — any constraint that mentions a specific tool
136
- # name (exec, web_fetch, ...) belongs in capability/*.yaml.
137
- #
138
- # - Hallucination / faithfulness atoms — those require a retrieval source
139
- # argument and are not universally applicable. See docs/sto-atoms.md §1
140
- # for when to add them per-agent.
@@ -1,118 +0,0 @@
1
- # Contract refresh prompt — sponsio refresh from traces
2
-
3
- You are tuning an existing `sponsio.yaml` against accumulated
4
- session traces. The library already exists; you're proposing
5
- deltas — added contracts, retired stale ones, tightened thresholds
6
- — based on what the agent actually did.
7
-
8
- This is the **self-evolve loop**: each refresh round looks at recent
9
- near-misses (would-have-blocked in observe mode) and confirmed
10
- patterns (rules that fire frequently and correctly), and proposes
11
- targeted edits.
12
-
13
- ## Input
14
-
15
- A JSON object from `sponsio refresh --emit-traces`:
16
-
17
- ```json
18
- {
19
- "agent": "...",
20
- "since": "7d",
21
- "existing_contracts": [
22
- {"desc": "...", "pattern": "...", "args": [...], "source": "..."}
23
- ],
24
- "trace_summary": {
25
- "total_events": 1247,
26
- "would_have_blocked": [
27
- {
28
- "tool": "send_email",
29
- "rule_desc": "rate_limit 5",
30
- "fire_count": 12,
31
- "sample_calls": [
32
- {"args": {...}, "ts": "...", "agent_outcome": "succeeded"}
33
- ]
34
- }
35
- ],
36
- "blocked_actual": [...],
37
- "uncovered_patterns": [
38
- {
39
- "tool": "transfer_funds",
40
- "call_count": 3,
41
- "sample_args": [{...}],
42
- "note": "tool was called 3x; no contract covers it"
43
- }
44
- ]
45
- }
46
- }
47
- ```
48
-
49
- ## What you produce
50
-
51
- A YAML diff with three sections:
52
-
53
- ```yaml
54
- proposed_changes:
55
- add:
56
- - desc: "..."
57
- G: {pattern: ..., args: [...]}
58
- source: agent-extracted-from-traces
59
- retire:
60
- - match: { desc: "<exact desc of existing contract>" }
61
- reason: "fired 0 times in 7d window"
62
- tighten:
63
- - match: { desc: "<exact desc>" }
64
- from: {pattern: rate_limit, args: [send_email, 5]}
65
- to: {pattern: rate_limit, args: [send_email, 10]}
66
- reason: "false-positive rate 12/30 over last 7d"
67
- ```
68
-
69
- ## Rules of thumb
70
-
71
- ### Adding contracts
72
-
73
- * `uncovered_patterns` with `call_count` ≥ 3 and clear semantics →
74
- add a contract.
75
- * Don't add for tools that fired 1–2x — too sparse.
76
- * Match the source-tag convention: new agent-derived contracts get
77
- `source: agent-extracted-from-traces`.
78
-
79
- ### Retiring (only `source: trace` or `source: agent-extracted-*`)
80
-
81
- * Existing contract that fired 0 times in the window → candidate for
82
- retirement, but ONLY if it's `source: trace` or
83
- `source: agent-extracted-*`.
84
- * **Never propose retiring** `source: scan`, `source: policy`, or
85
- user-written (no source / `source: user`) contracts.
86
- * If retiring, include a one-line `reason`.
87
-
88
- ### Tightening / loosening
89
-
90
- * High false-positive rate (`would_have_blocked` clusters with
91
- `agent_outcome: succeeded` — the call was actually fine) →
92
- loosen the cap or add `arg_blacklist` carve-outs.
93
- * High true-positive rate (would-have-blocked + agent_outcome
94
- shows the agent was caught doing something problematic) →
95
- tighten.
96
- * Always cite the trace counts in `reason` so the user can verify.
97
-
98
- ## Pattern vocabulary
99
-
100
- Same as plugin scan / onboard — `arg_blacklist`, `rate_limit`,
101
- `loop_detection`, `irreversible_once`, `must_precede`,
102
- `arg_value_range`, `arg_length_limit`.
103
-
104
- ## What you must not do
105
-
106
- * **Don't** retire user-written or pack-derived contracts.
107
- * **Don't** propose changes without trace evidence — every proposal
108
- needs a count + window in `reason`.
109
- * **Don't** merge proposals into the YAML directly — output the
110
- diff for the user to review and apply. Apply via
111
- `sponsio refresh --apply` (which respects the same source-tag
112
- protections automatically) or by hand-editing for ad-hoc changes.
113
-
114
- ## Output format
115
-
116
- ONLY the `proposed_changes:` YAML block above. No prose, no
117
- markdown wrapping. The host driver will show it to the user
118
- verbatim.