opencrowd 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +19 -0
- package/bundle/opencrowd.js +20814 -2699
- package/knowledge/INDEX.md +11 -0
- package/knowledge/L0.md +13 -0
- package/knowledge/categories/business-regulatory.md +16 -0
- package/knowledge/categories/files-communications.md +15 -0
- package/knowledge/categories/finance-crypto.md +16 -0
- package/knowledge/categories/free-first.md +15 -0
- package/knowledge/categories/llm-gateways.md +11 -0
- package/knowledge/categories/media-generation.md +15 -0
- package/knowledge/categories/social-news.md +12 -0
- package/knowledge/categories/utilities.md +18 -0
- package/knowledge/categories/weather-travel.md +14 -0
- package/knowledge/categories/web-extraction.md +14 -0
- package/knowledge/categories/web-search.md +15 -0
- package/knowledge/meta.json +38 -0
- package/package.json +5 -3
- package/tasks/attachments/events.json +604 -0
- package/tasks/attachments/note.txt +2 -0
- package/tasks/attachments/sales.csv +61 -0
- package/tasks/usage.json +128 -0
package/README.md
CHANGED
|
@@ -122,8 +122,27 @@ opencrowd config show
|
|
|
122
122
|
opencrowd wallet balance
|
|
123
123
|
opencrowd doctor
|
|
124
124
|
opencrowd evals gaia --tier smoke --harness opencrowd,claude,codex
|
|
125
|
+
opencrowd evals suite --set all --tag baseline --model openai/gpt-5.6-luna --subagent-model off
|
|
126
|
+
opencrowd evals evolve --baseline-report <run>/report.json --generations 2 --candidates 2
|
|
125
127
|
```
|
|
126
128
|
|
|
129
|
+
`evals suite` runs the 40-task suite (15 real-usage tasks derived from
|
|
130
|
+
CrowdCode demand, 15 seeded GAIA level 2/3, 10 seeded AssistantBench) and
|
|
131
|
+
reports accuracy alongside on-chain LLM and service spend, turns, paid calls,
|
|
132
|
+
and unnecessary purchases. `evals evolve` is the meta-harness: it derives a
|
|
133
|
+
service knowledge tree (`L0.md` in the prompt, `INDEX.md` + category pages
|
|
134
|
+
under `artifacts/knowledge/`) from CrowdCode review evidence, mutates it
|
|
135
|
+
against held-in failures, keeps the quality-vs-cost Pareto frontier, and
|
|
136
|
+
never shows the proposer a held-out task. `evals suite` uses the runtime's
|
|
137
|
+
knowledge tree by default (the snapshot bundled with `@opencrowd/agent-runtime`,
|
|
138
|
+
or `OPENCROWD_KNOWLEDGE_DIR`); pass `--knowledge <tree dir>` to evaluate a
|
|
139
|
+
candidate or `--knowledge none` for the static-prompt baseline.
|
|
140
|
+
|
|
141
|
+
The agent loads the same tree in normal runs: `L0.md` becomes the prompt's
|
|
142
|
+
capability index and the category pages land under `artifacts/knowledge/`.
|
|
143
|
+
Set `OPENCROWD_KNOWLEDGE_DIR=none` to disable it or point it at another tree;
|
|
144
|
+
the loaded version is recorded on the session.
|
|
145
|
+
|
|
127
146
|
## Headless runs
|
|
128
147
|
|
|
129
148
|
```sh
|