@pickled-dev/cli 0.16.1 → 0.17.0

package/README.md

@@ -76,12 +76,14 @@ Run agent scenarios against registered sources.
 | `-o, --output <file>` | Save JSON report to file                                            |
 | `-v, --verbose`       | Show progress while scenarios run                                   |
 | `-t, --threshold <n>` | Minimum score percent needed to pass                                |
-| `--target <name>`     | Run only the named target (overrides matrix)                        |
+| `--target <name>`     | Restrict to the named target. Overrides `matrix.target` for non-matrix scenarios; for matrix scenarios, also acts as `--interface` unless that flag is explicitly set. |
 | `--scenario <name>`   | Run only the named scenario (CI-matrix-friendly)                    |
-| `--interface <name>`  | Matrix cell filter: run only cells with this interface              |
+| `--interface <name>`  | Matrix cell filter: run only cells with this interface. Takes precedence over `--target` for matrix cells. |
 | `--source <name>`     | Matrix cell filter: run only cells with this source id              |
 | `--toolset <name>`    | Matrix cell filter: run only cells with this toolset name           |
 
+`--target` and `--interface` are related but distinct: `--target` is the legacy flag that narrows the top-level `matrix.target` axis (used before per-scenario `scenario.matrix.interfaces` shipped in v0.16.0). When `--target` is the only flag passed, the CLI also applies it as `--interface` so matrix scenarios narrow consistently. Pass `--interface` explicitly to override.
+
 The cell filters work with `scenario.matrix` declarations. Designed for GitHub Actions matrix usage where each CI job runs one cell:
 
 ```yaml
@@ -160,6 +162,40 @@ Codebase loader safety defaults: skips directories (`onlyFiles`), does not follo
 
 URL sources are NOT scanned by the audit's trap cross-reference in v1; they are fetched only during `pickled check`.
 
+## Toolsets
+
+Matrix mode (`scenario.matrix.toolsets`) iterates each scenario across named toolset profiles. v0.16.x ships two:
+
+- **`none`** (the deterministic baseline). Pickled injects the cell's active source content into the agent's prompt. Citation contract applies if `requiredSources` is declared. Same scoring shape as non-matrix scenarios.
+- **`web`** on Claude Code only. Maps to `allowedTools: ["WebSearch", "WebFetch"]` on the cell's Claude Code target. Source is NOT injected; the cell's prompt is rewritten to name the active source as the discovery target ("the canonical source for this question is at ..."). Citation contract is skipped; the cell scores on traps + `expected.includes`/`excludes`.
+
+Declare profiles at the top level of `pickled.yml`:
+
+```yaml
+toolsets:
+  none: {}
+  web:
+    webSearch: true
+    webFetch: true
+```
+
+Then reference them per scenario:
+
+```yaml
+scenarios:
+  - name: "Install"
+    matrix:
+      interfaces: [quick]
+      sources: [llms]
+      toolsets: [none, web]
+    expected:
+      includes: ["bunx pickled"]
+```
+
+That scenario produces 2 cells: `[quick · llms · none]` (injected) and `[quick · llms · web]` (discovered via tools).
+
+Custom toolset names that have no recognized adapter throw a clear "not yet implemented" error per cell. Web toolset on a non-Claude-Code interface throws "implemented only on the claude-code interface" so the misconfiguration is obvious.
+
 ## Targets
 
 Pickled ships three target shapes today. Each target is a distinct surface that exercises the agent differently; results are comparable but not identical.