@plaited/agent-eval-harness 0.6.1 → 0.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +11 -9
- package/package.json +4 -4
package/README.md
CHANGED
|
@@ -25,7 +25,7 @@ export ANTHROPIC_API_KEY=sk-... # For Claude
|
|
|
25
25
|
export GEMINI_API_KEY=... # For Gemini
|
|
26
26
|
```
|
|
27
27
|
|
|
28
|
-
Pre-built schemas are available in `.
|
|
28
|
+
Pre-built schemas are available in `.plaited/skills/headless-adapters/schemas/` for Claude and Gemini.
|
|
29
29
|
|
|
30
30
|
### Core Commands
|
|
31
31
|
|
|
@@ -85,7 +85,7 @@ bunx @plaited/agent-eval-harness compare run1.jsonl run2.jsonl -o comparison.jso
|
|
|
85
85
|
**Install skills** for use with AI coding agents:
|
|
86
86
|
|
|
87
87
|
```bash
|
|
88
|
-
curl -fsSL https://raw.githubusercontent.com/plaited/skills-installer/main/install.sh | bash -s -- --
|
|
88
|
+
curl -fsSL https://raw.githubusercontent.com/plaited/skills-installer/main/install.sh | bash -s -- --agents <agent-name> --project agent-eval-harness
|
|
89
89
|
```
|
|
90
90
|
|
|
91
91
|
Replace `<agent-name>` with your agent: `claude`, `cursor`, `copilot`, `opencode`, `amp`, `goose`, `factory`
|
|
@@ -253,18 +253,20 @@ cat results.jsonl | your-scoring-script.ts
|
|
|
253
253
|
## Development
|
|
254
254
|
|
|
255
255
|
```bash
|
|
256
|
-
bun install
|
|
257
|
-
bun run check
|
|
258
|
-
bun test
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
256
|
+
bun install # Install dependencies
|
|
257
|
+
bun run check # Type check + lint + format
|
|
258
|
+
bun test # Run unit tests
|
|
259
|
+
bun run test:integration # Run integration tests (requires API keys)
|
|
260
|
+
|
|
261
|
+
# Alternative: Run integration tests in Docker
|
|
262
|
+
ANTHROPIC_API_KEY=sk-... GEMINI_API_KEY=... \
|
|
263
|
+
docker compose -f docker-compose.test.yml run --rm test
|
|
262
264
|
```
|
|
263
265
|
|
|
264
266
|
## Requirements
|
|
265
267
|
|
|
266
268
|
- **Runtime:** Bun >= 1.2.9
|
|
267
|
-
- **Schema:** JSON schema describing CLI agent interaction (see `.
|
|
269
|
+
- **Schema:** JSON schema describing CLI agent interaction (see `.plaited/skills/headless-adapters/schemas/`)
|
|
268
270
|
- **API Key:** `ANTHROPIC_API_KEY` for Claude, `GEMINI_API_KEY` for Gemini
|
|
269
271
|
|
|
270
272
|
## License
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@plaited/agent-eval-harness",
|
|
3
|
-
"version": "0.6.
|
|
3
|
+
"version": "0.6.2",
|
|
4
4
|
"description": "CLI tool for capturing agent trajectories from headless CLI agents",
|
|
5
5
|
"license": "ISC",
|
|
6
6
|
"engines": {
|
|
@@ -54,11 +54,11 @@
|
|
|
54
54
|
]
|
|
55
55
|
},
|
|
56
56
|
"dependencies": {
|
|
57
|
-
"
|
|
58
|
-
"
|
|
57
|
+
"@plaited/development-skills": "0.6.5",
|
|
58
|
+
"zod": "^4.3.6"
|
|
59
59
|
},
|
|
60
60
|
"devDependencies": {
|
|
61
|
-
"@biomejs/biome": "2.3.
|
|
61
|
+
"@biomejs/biome": "2.3.12",
|
|
62
62
|
"@types/bun": "1.3.6",
|
|
63
63
|
"format-package": "7.0.0",
|
|
64
64
|
"lint-staged": "16.2.7",
|