@swarmtools/evals 0.2.29 → 0.2.30
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -3
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -19,9 +19,12 @@ This package contains the evaluation framework for the swarm-tools ecosystem. Ex
|
|
|
19
19
|
## What Gets Evaluated
|
|
20
20
|
|
|
21
21
|
- **Coordinator Protocol** - Does the coordinator spawn workers vs doing work itself?
|
|
22
|
+
- **Coordinator Behavior** - LLM behavior after compaction (stays in coordinator role)
|
|
23
|
+
- **Compaction Resumption** - Context injection correctness after compaction
|
|
24
|
+
- **Compaction Prompt Quality** - Quality of continuation prompts generated
|
|
22
25
|
- **Task Decomposition** - Quality of task splitting, file conflict detection
|
|
23
|
-
- **
|
|
24
|
-
- **
|
|
26
|
+
- **Strategy Selection** - Correct strategy choice for task characteristics
|
|
27
|
+
- **Decision Quality** - Strategy selection quality and precedent relevance
|
|
25
28
|
|
|
26
29
|
## Usage
|
|
27
30
|
|
|
@@ -46,7 +49,7 @@ This package is part of the swarm-tools monorepo:
|
|
|
46
49
|
|
|
47
50
|
## Development
|
|
48
51
|
|
|
49
|
-
Evals use real coordinator sessions captured to `~/.config/swarm-tools/sessions/*.jsonl`. See
|
|
52
|
+
Evals use real coordinator sessions captured to `~/.config/swarm-tools/sessions/*.jsonl`. See `docs/README.md` in this package for details on session capture and eval architecture.
|
|
50
53
|
|
|
51
54
|
## License
|
|
52
55
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@swarmtools/evals",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.30",
|
|
4
4
|
"description": "Evaluation suite for swarm-tools multi-agent coordination",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.js",
|
|
@@ -33,7 +33,7 @@
|
|
|
33
33
|
"dependencies": {
|
|
34
34
|
"evalite": "^1.0.0-beta.10",
|
|
35
35
|
"ai": "6.0.0-beta.150",
|
|
36
|
-
"opencode-swarm-plugin": "0.
|
|
36
|
+
"opencode-swarm-plugin": "0.58.4",
|
|
37
37
|
"swarm-mail": "1.10.2"
|
|
38
38
|
},
|
|
39
39
|
"devDependencies": {
|