cawdex 1.35.65 → 1.35.67
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +186 -186
- package/bin/anycode.js +2 -2
- package/bin/cawdex.js +408 -408
- package/bin/ecc-hooks.cjs +11 -11
- package/dist/agents.js +1424 -1424
- package/dist/autonomous-loops.js +287 -287
- package/dist/command-palette.js +1 -1
- package/dist/command-palette.js.map +1 -1
- package/dist/compaction.js +8 -8
- package/dist/content-engine.js +543 -543
- package/dist/coverage.js +39 -39
- package/dist/docs-sync.js +98 -98
- package/dist/evaluation.js +452 -452
- package/dist/git-workflow.js +49 -49
- package/dist/index.js +75 -46
- package/dist/index.js.map +1 -1
- package/dist/instant-answer.js +10 -0
- package/dist/instant-answer.js.map +1 -1
- package/dist/modes.js +355 -355
- package/dist/orchestration.js +15 -15
- package/dist/pm2-manager.js +26 -26
- package/dist/prompt-buffer.d.ts +7 -0
- package/dist/prompt-buffer.js +31 -0
- package/dist/prompt-buffer.js.map +1 -1
- package/dist/query.d.ts +4 -0
- package/dist/query.js +79 -64
- package/dist/query.js.map +1 -1
- package/dist/refactor.js +87 -87
- package/dist/search-first.js +92 -92
- package/dist/session-picker.d.ts +44 -0
- package/dist/session-picker.js +227 -0
- package/dist/session-picker.js.map +1 -0
- package/dist/skill-create.js +100 -100
- package/dist/stitch.js +1 -1
- package/dist/swarm.d.ts +34 -54
- package/dist/swarm.js +175 -63
- package/dist/swarm.js.map +1 -1
- package/dist/system-prompt.js +10 -10
- package/dist/verification.js +55 -55
- package/dist/walkthrough.js +5 -5
- package/package.json +1 -1
- package/resources/__init__.py +1 -1
- package/resources/exgentic/cawdex_agent/README.md +114 -114
- package/resources/exgentic/cawdex_agent/__init__.py +5 -5
- package/resources/exgentic/cawdex_agent/agent.py +605 -605
- package/resources/exgentic/cawdex_agent/requirements.txt +2 -2
- package/resources/exgentic/cawdex_agent/setup.sh +21 -21
- package/resources/exgentic/cawdex_agent/utils.py +1061 -1061
- package/resources/hal/cawdex_agent/README.md +24 -24
- package/resources/hal/cawdex_agent/__init__.py +1 -1
- package/resources/hal/cawdex_agent/main.py +550 -550
- package/resources/hal/cawdex_agent/requirements.txt +2 -2
- package/resources/kbench/cawdex_agent/README.md +107 -107
- package/resources/kbench/cawdex_agent/adapter.manifest.json +19 -19
- package/resources/kbench/cawdex_agent/runner.mjs +753 -753
- package/resources/open_agent_leaderboard/cawdex-agent-card.md +119 -119
- package/resources/terminal_bench/__init__.py +1 -1
- package/resources/terminal_bench/cawdex_agent.py +174 -174
- package/resources/terminal_bench/setup.sh +121 -121
package/dist/system-prompt.js
CHANGED
|
@@ -149,8 +149,8 @@ export function buildSystemPrompt(config, cwd, mode = 'dev', userQuery) {
|
|
|
149
149
|
}
|
|
150
150
|
// User context
|
|
151
151
|
const userAddition = buildUserContext();
|
|
152
|
-
return `You are ${BRAND_LOCKUP}, running in the user's shell.
|
|
153
|
-
You help with software engineering tasks: writing code, fixing bugs, refactoring, explaining code, running commands, and more.
|
|
152
|
+
return `You are ${BRAND_LOCKUP}, running in the user's shell.
|
|
153
|
+
You help with software engineering tasks: writing code, fixing bugs, refactoring, explaining code, running commands, and more.
|
|
154
154
|
|
|
155
155
|
# Environment
|
|
156
156
|
- Working directory: ${cwd}
|
|
@@ -207,14 +207,14 @@ of past requests that are ALREADY DONE. Specifically:
|
|
|
207
207
|
than guessing that they meant to repeat a previous task.
|
|
208
208
|
|
|
209
209
|
IMPORTANT — tool-call rules:
|
|
210
|
-
- The exact, allowed tool names are the bullet keys above. Calling any other name (e.g. \`web_search_exa\`, \`google_search\`, \`shell_exec\`) is an error and the call will fail.
|
|
211
|
-
- For web discovery: use \`web_search\` (returns title/URL/snippet for a keyword query).
|
|
212
|
-
- For reading a known URL: use \`web_fetch\`.
|
|
213
|
-
- For source-specific research/code/data discovery: use \`research_sources\` before generic web search when arXiv, GitHub, Hugging Face, or Kaggle is relevant. For benchmark/leaderboard work, prefer targeted trace-readable coverage: GitHub \`github_kind:"all"\`, Hugging Face \`kind:"all"\`, Kaggle \`kaggle_kind:"both"\`, \`recent_days:90\`, and \`format:"json"\`; inspect the structured digest hits/errors/source mix before relying on the result. For Terminal-Bench public-agent source mining, use \`benchmark_repo_catalog\` as the offline seed catalog, then call \`github_repo_digest\` on the most relevant repo(s) before porting patterns; treat those digests as demonstrations to verify against exact files, not as authority.
|
|
214
|
-
- For harness self-improvement, tool/UX debugging, or benchmark-readiness work on this CLI, call \`harness_components\` early to map the affected prompt/tool/middleware/skill/memory/provider/adapter/UX component to files and focused tests before editing.
|
|
215
|
-
- For multi-step work, uncertain scope, or benchmark tasks: use \`todo_write\` to keep a short working checklist current. Mark exactly one active item as \`in_progress\` when possible, and mark items \`completed\` only after evidence.
|
|
216
|
-
- For shell-only operations: use \`bash\`. Do not use bash for tasks any other tool already covers.
|
|
217
|
-
- If a capability you want isn't in the list, work around it with the tools that exist. Don't pretend a tool exists.
|
|
210
|
+
- The exact, allowed tool names are the bullet keys above. Calling any other name (e.g. \`web_search_exa\`, \`google_search\`, \`shell_exec\`) is an error and the call will fail.
|
|
211
|
+
- For web discovery: use \`web_search\` (returns title/URL/snippet for a keyword query).
|
|
212
|
+
- For reading a known URL: use \`web_fetch\`.
|
|
213
|
+
- For source-specific research/code/data discovery: use \`research_sources\` before generic web search when arXiv, GitHub, Hugging Face, or Kaggle is relevant. For benchmark/leaderboard work, prefer targeted trace-readable coverage: GitHub \`github_kind:"all"\`, Hugging Face \`kind:"all"\`, Kaggle \`kaggle_kind:"both"\`, \`recent_days:90\`, and \`format:"json"\`; inspect the structured digest hits/errors/source mix before relying on the result. For Terminal-Bench public-agent source mining, use \`benchmark_repo_catalog\` as the offline seed catalog, then call \`github_repo_digest\` on the most relevant repo(s) before porting patterns; treat those digests as demonstrations to verify against exact files, not as authority.
|
|
214
|
+
- For harness self-improvement, tool/UX debugging, or benchmark-readiness work on this CLI, call \`harness_components\` early to map the affected prompt/tool/middleware/skill/memory/provider/adapter/UX component to files and focused tests before editing.
|
|
215
|
+
- For multi-step work, uncertain scope, or benchmark tasks: use \`todo_write\` to keep a short working checklist current. Mark exactly one active item as \`in_progress\` when possible, and mark items \`completed\` only after evidence.
|
|
216
|
+
- For shell-only operations: use \`bash\`. Do not use bash for tasks any other tool already covers.
|
|
217
|
+
- If a capability you want isn't in the list, work around it with the tools that exist. Don't pretend a tool exists.
|
|
218
218
|
|
|
219
219
|
# File operations — picking the right tool
|
|
220
220
|
- **Creating or overwriting a file**: always use \`write_file\`. Never use \`bash\` + \`echo > file\` for this. write_file takes the full content directly, handles multi-line strings without escaping, and resolves \`~/...\` paths to the user's actual home directory. \`bash\` + redirection is fragile across platforms — on Windows it routes through ${shell}, where \`$USERPROFILE\` / \`$HOME\` / bash-style paths may not behave the way you expect.
|
package/dist/verification.js
CHANGED
|
@@ -165,61 +165,61 @@ export function autoDetectTestCommand(cwd) {
|
|
|
165
165
|
*/
|
|
166
166
|
export function buildVerifyPrompt(cwd, command) {
|
|
167
167
|
const testCmd = command || autoDetectTestCommand(cwd);
|
|
168
|
-
const prompt = `You are now in verification mode. Your goal is to run tests, analyze failures, fix the code, and repeat until all tests pass.
|
|
169
|
-
|
|
170
|
-
${chalk.bold('Verification Loop Process:')}
|
|
171
|
-
|
|
172
|
-
1. ${chalk.cyan('Run the test command')}
|
|
173
|
-
Execute: ${chalk.yellow(testCmd)}
|
|
174
|
-
Capture all output, stderr, and exit codes
|
|
175
|
-
|
|
176
|
-
2. ${chalk.cyan('Analyze the output')}
|
|
177
|
-
Look for:
|
|
178
|
-
- Failed test names and error messages
|
|
179
|
-
- Stack traces and assertion failures
|
|
180
|
-
- Missing imports or undefined references
|
|
181
|
-
- Type errors (if applicable)
|
|
182
|
-
- Any other compilation/runtime errors
|
|
183
|
-
|
|
184
|
-
3. ${chalk.cyan('Fix the failing code')}
|
|
185
|
-
Based on the errors:
|
|
186
|
-
- Identify the root cause
|
|
187
|
-
- Modify the minimal necessary code to fix the issue
|
|
188
|
-
- Do NOT refactor unrelated code
|
|
189
|
-
- Ensure fixes are targeted and precise
|
|
190
|
-
|
|
191
|
-
4. ${chalk.cyan('Re-run the test command')}
|
|
192
|
-
Execute: ${chalk.yellow(testCmd)}
|
|
193
|
-
Verify the fix resolved the issue
|
|
194
|
-
|
|
195
|
-
5. ${chalk.cyan('Repeat until success')}
|
|
196
|
-
Continue the loop until either:
|
|
197
|
-
- All tests pass (SUCCESS)
|
|
198
|
-
- You've made 5 iterations (STOP - escalate)
|
|
199
|
-
|
|
200
|
-
${chalk.bold('Important Guidelines:')}
|
|
201
|
-
- ${chalk.dim('Work in:')} ${cwd}
|
|
202
|
-
- ${chalk.dim('Run tests with:')} ${testCmd}
|
|
203
|
-
- ${chalk.dim('Max iterations:')} 5 (safety limit)
|
|
204
|
-
- ${chalk.dim('Be systematic:')} Fix one issue at a time
|
|
205
|
-
- ${chalk.dim('Show progress:')} Report iteration count and current status
|
|
206
|
-
- ${chalk.dim('Be explicit:')} Show the exact commands you run and their output
|
|
207
|
-
- ${chalk.dim('Stop if stuck:')} If you can't make progress after 5 iterations, explain what you've tried and ask for help
|
|
208
|
-
|
|
209
|
-
${chalk.bold('Output Format:')}
|
|
210
|
-
For each iteration, clearly indicate:
|
|
211
|
-
- Iteration number (e.g., "Iteration 1/5")
|
|
212
|
-
- Test command being run
|
|
213
|
-
- Output/errors found
|
|
214
|
-
- Root cause analysis
|
|
215
|
-
- Fix applied
|
|
216
|
-
- Result of re-run
|
|
217
|
-
|
|
218
|
-
When complete, provide a summary of:
|
|
219
|
-
- Total iterations taken
|
|
220
|
-
- Issues fixed
|
|
221
|
-
- Final test status (PASS/FAIL)
|
|
222
|
-
- Any remaining concerns
|
|
168
|
+
const prompt = `You are now in verification mode. Your goal is to run tests, analyze failures, fix the code, and repeat until all tests pass.
|
|
169
|
+
|
|
170
|
+
${chalk.bold('Verification Loop Process:')}
|
|
171
|
+
|
|
172
|
+
1. ${chalk.cyan('Run the test command')}
|
|
173
|
+
Execute: ${chalk.yellow(testCmd)}
|
|
174
|
+
Capture all output, stderr, and exit codes
|
|
175
|
+
|
|
176
|
+
2. ${chalk.cyan('Analyze the output')}
|
|
177
|
+
Look for:
|
|
178
|
+
- Failed test names and error messages
|
|
179
|
+
- Stack traces and assertion failures
|
|
180
|
+
- Missing imports or undefined references
|
|
181
|
+
- Type errors (if applicable)
|
|
182
|
+
- Any other compilation/runtime errors
|
|
183
|
+
|
|
184
|
+
3. ${chalk.cyan('Fix the failing code')}
|
|
185
|
+
Based on the errors:
|
|
186
|
+
- Identify the root cause
|
|
187
|
+
- Modify the minimal necessary code to fix the issue
|
|
188
|
+
- Do NOT refactor unrelated code
|
|
189
|
+
- Ensure fixes are targeted and precise
|
|
190
|
+
|
|
191
|
+
4. ${chalk.cyan('Re-run the test command')}
|
|
192
|
+
Execute: ${chalk.yellow(testCmd)}
|
|
193
|
+
Verify the fix resolved the issue
|
|
194
|
+
|
|
195
|
+
5. ${chalk.cyan('Repeat until success')}
|
|
196
|
+
Continue the loop until either:
|
|
197
|
+
- All tests pass (SUCCESS)
|
|
198
|
+
- You've made 5 iterations (STOP - escalate)
|
|
199
|
+
|
|
200
|
+
${chalk.bold('Important Guidelines:')}
|
|
201
|
+
- ${chalk.dim('Work in:')} ${cwd}
|
|
202
|
+
- ${chalk.dim('Run tests with:')} ${testCmd}
|
|
203
|
+
- ${chalk.dim('Max iterations:')} 5 (safety limit)
|
|
204
|
+
- ${chalk.dim('Be systematic:')} Fix one issue at a time
|
|
205
|
+
- ${chalk.dim('Show progress:')} Report iteration count and current status
|
|
206
|
+
- ${chalk.dim('Be explicit:')} Show the exact commands you run and their output
|
|
207
|
+
- ${chalk.dim('Stop if stuck:')} If you can't make progress after 5 iterations, explain what you've tried and ask for help
|
|
208
|
+
|
|
209
|
+
${chalk.bold('Output Format:')}
|
|
210
|
+
For each iteration, clearly indicate:
|
|
211
|
+
- Iteration number (e.g., "Iteration 1/5")
|
|
212
|
+
- Test command being run
|
|
213
|
+
- Output/errors found
|
|
214
|
+
- Root cause analysis
|
|
215
|
+
- Fix applied
|
|
216
|
+
- Result of re-run
|
|
217
|
+
|
|
218
|
+
When complete, provide a summary of:
|
|
219
|
+
- Total iterations taken
|
|
220
|
+
- Issues fixed
|
|
221
|
+
- Final test status (PASS/FAIL)
|
|
222
|
+
- Any remaining concerns
|
|
223
223
|
`;
|
|
224
224
|
return prompt;
|
|
225
225
|
}
|
package/dist/walkthrough.js
CHANGED
|
@@ -8,9 +8,9 @@
|
|
|
8
8
|
* choices to make.
|
|
9
9
|
*/
|
|
10
10
|
export function buildWalkthroughPrompt() {
|
|
11
|
-
return `# Onboarding mode — you are the Cawdex Tour Guide
|
|
12
|
-
|
|
13
|
-
You are walking the user through Cawdex for the first time. Be warm,
|
|
11
|
+
return `# Onboarding mode — you are the Cawdex Tour Guide
|
|
12
|
+
|
|
13
|
+
You are walking the user through Cawdex for the first time. Be warm,
|
|
14
14
|
concrete, and brief. Ask **one question at a time**, wait for the user's
|
|
15
15
|
reply, then move to the next stage. Don't dump everything at once.
|
|
16
16
|
|
|
@@ -20,7 +20,7 @@ reply, then move to the next stage. Don't dump everything at once.
|
|
|
20
20
|
Greet the user, then ask: "Have you used a terminal AI coding assistant
|
|
21
21
|
before (Claude Code, Cursor, Aider, etc.)?" — Tailor depth to the answer.
|
|
22
22
|
|
|
23
|
-
### 2. Explain what Cawdex is (60 seconds max)
|
|
23
|
+
### 2. Explain what Cawdex is (60 seconds max)
|
|
24
24
|
Cover:
|
|
25
25
|
- Universal OpenAI-compatible CLI — works with OpenRouter, OpenAI, Anthropic,
|
|
26
26
|
Ollama, LM Studio, DeepSeek.
|
|
@@ -109,7 +109,7 @@ command above and demonstrate it. If they have no specific task, suggest:
|
|
|
109
109
|
## Rules for the tour
|
|
110
110
|
|
|
111
111
|
- **One question at a time.** Never dump multi-question paragraphs.
|
|
112
|
-
- **No duplicate commands.** Cawdex unified its surface — when in doubt,
|
|
112
|
+
- **No duplicate commands.** Cawdex unified its surface — when in doubt,
|
|
113
113
|
the canonical name is the non-prefixed one. Do not suggest \`/ecc-tdd\` if
|
|
114
114
|
\`/tdd\` does the same thing.
|
|
115
115
|
- **Be honest about limits.** If something isn't installed (e.g. ECC failed
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "cawdex",
|
|
3
|
-
"version": "1.35.
|
|
3
|
+
"version": "1.35.67",
|
|
4
4
|
"description": "Cawdex — terminal coding agents with a mind for the whole repo. Speaks any OpenAI-compatible API with repo-aware context, benchmark tracing, MemPalace memory, and terminal-first agent workflows.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"license": "MIT",
|
package/resources/__init__.py
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
"""cawdex packaged runtime resources."""
|
|
1
|
+
"""cawdex packaged runtime resources."""
|
|
@@ -1,114 +1,114 @@
|
|
|
1
|
-
# Cawdex for Exgentic
|
|
2
|
-
|
|
3
|
-
This directory is a custom Exgentic agent package for running Cawdex in
|
|
4
|
-
Open Agent Leaderboard style evaluations.
|
|
5
|
-
|
|
6
|
-
The adapter implements Exgentic's `Agent` / `AgentInstance` split. On each
|
|
7
|
-
`react()` step it writes an Exgentic prompt, launches Cawdex in
|
|
8
|
-
non-interactive `/benchmark` mode, asks Cawdex to finish with one JSON
|
|
9
|
-
action object, then maps that JSON back to an Exgentic `ActionType`.
|
|
10
|
-
It auto-selects specialized `/benchmark` profiles for AppWorld, BrowseComp+,
|
|
11
|
-
tau2, ARC, SaaS, roadmap, mobile, WebDevBench, SWE-Cycle, SWE-CI, SWE-PRBench, TML-Bench, and Pi-Bench-style tasks from the task, context, and
|
|
12
|
-
available action schemas. It also builds a deterministic recommended action
|
|
13
|
-
shortlist before the full schema list, including required argument keys and
|
|
14
|
-
redacted exact latest-observation/context hints when those required values are
|
|
15
|
-
already present. This matches the Open Agent leaderboard finding that
|
|
16
|
-
action-scaffold details matter while preserving compatibility with every
|
|
17
|
-
available benchmark action. Long sessions use a folded history ledger
|
|
18
|
-
that preserves latest observations, selected actions, action counts, and
|
|
19
|
-
diagnostics without reinjecting bulky raw stdout into every step. Before
|
|
20
|
-
dispatch, near-miss action JSON is repaired conservatively: action names are
|
|
21
|
-
matched by case/identifier normalization, schema argument keys are canonicalized,
|
|
22
|
-
required schema fields are filled from exact latest-observation/context keys
|
|
23
|
-
when available, and unknown extra keys are dropped when the benchmark exposes a
|
|
24
|
-
fixed schema. If Cawdex produces malformed or missing action JSON, the adapter
|
|
25
|
-
uses the same shortlist and exact required-argument hints to select a viable
|
|
26
|
-
non-finish action while the latest observation is still pending, instead of
|
|
27
|
-
falling straight to a finish/message action. When a selected action is followed
|
|
28
|
-
by an effectively unchanged observation, the next shortlist and deterministic
|
|
29
|
-
fallback mark that action as a no-effect repeat and prefer another viable action
|
|
30
|
-
with satisfiable required arguments.
|
|
31
|
-
For WebDevBench-style app-agency tasks, the folded ledger keeps canary business
|
|
32
|
-
requirements, frontend/backend coupling evidence, and production/security gaps
|
|
33
|
-
visible across turns so an attractive UI-only action does not look complete.
|
|
34
|
-
For SWE-Cycle-style lifecycle tasks, it keeps the phase, bare-repo environment
|
|
35
|
-
setup state, implementation requirements, generated/selected tests, judge
|
|
36
|
-
commands, and unresolved lifecycle gaps visible across turns.
|
|
37
|
-
For SWE-CI-style codebase maintenance tasks, it keeps current/target commits,
|
|
38
|
-
test gaps, inferred requirements, touched files, verifier deltas, and unresolved
|
|
39
|
-
regressions visible across the `run_tests -> define_requirements -> modify_code`
|
|
40
|
-
loop.
|
|
41
|
-
For SWE-PRBench-style PR review tasks, it keeps PR metadata, changed files, diff
|
|
42
|
-
hunks, suspected findings, evidence gaps, and context-expansion reasons visible
|
|
43
|
-
so the agent produces severity-rated findings instead of drifting into broad
|
|
44
|
-
repo inspection or unsolicited patch generation.
|
|
45
|
-
For TML-Bench-style tabular ML tasks, it keeps the data contract, metric,
|
|
46
|
-
validation split, leakage checks, model artifact, submission path, and submission
|
|
47
|
-
validity evidence visible so the agent favors a reproducible valid baseline over
|
|
48
|
-
an invalid high-complexity attempt.
|
|
49
|
-
For Pi-Bench-style proactive assistant tasks, it keeps user profile, current
|
|
50
|
-
request, message/file/app context, available domain tools, hidden-intent
|
|
51
|
-
hypotheses, privacy risk, clarification state, selected actions, and observable
|
|
52
|
-
completion evidence visible so the agent can ask focused questions without
|
|
53
|
-
overreaching or fabricating personal state.
|
|
54
|
-
|
|
55
|
-
## Python API use
|
|
56
|
-
|
|
57
|
-
```python
|
|
58
|
-
import sys
|
|
59
|
-
from pathlib import Path
|
|
60
|
-
|
|
61
|
-
sys.path.insert(0, str(Path("path/to/resources/exgentic").resolve()))
|
|
62
|
-
|
|
63
|
-
from cawdex_agent import CawdexAgent
|
|
64
|
-
from exgentic.interfaces.lib import evaluate
|
|
65
|
-
|
|
66
|
-
results = evaluate(
|
|
67
|
-
benchmark="gsm8k",
|
|
68
|
-
agent=CawdexAgent(
|
|
69
|
-
model="openrouter/free",
|
|
70
|
-
provider="openrouter",
|
|
71
|
-
max_turns=8,
|
|
72
|
-
),
|
|
73
|
-
num_tasks=5,
|
|
74
|
-
)
|
|
75
|
-
```
|
|
76
|
-
|
|
77
|
-
## Registry use
|
|
78
|
-
|
|
79
|
-
For Exgentic CLI usage, copy this directory into an Exgentic checkout at:
|
|
80
|
-
|
|
81
|
-
```text
|
|
82
|
-
src/exgentic/agents/cawdex_agent/
|
|
83
|
-
```
|
|
84
|
-
|
|
85
|
-
Then add a registry entry:
|
|
86
|
-
|
|
87
|
-
```python
|
|
88
|
-
"cawdex_agent": RegistryEntry(
|
|
89
|
-
display_name="Cawdex",
|
|
90
|
-
module="exgentic.agents.cawdex_agent.agent",
|
|
91
|
-
class_name="CawdexAgent",
|
|
92
|
-
),
|
|
93
|
-
```
|
|
94
|
-
|
|
95
|
-
After that:
|
|
96
|
-
|
|
97
|
-
```bash
|
|
98
|
-
exgentic install --agent cawdex_agent --local
|
|
99
|
-
exgentic evaluate --benchmark gsm8k --agent cawdex_agent --model openrouter/free
|
|
100
|
-
```
|
|
101
|
-
|
|
102
|
-
## Configuration
|
|
103
|
-
|
|
104
|
-
- `CAWDEX_EXGENTIC_COMMAND` or `CAWDEX_EXGENTIC_COMMAND` overrides the command, default `cawdex`.
|
|
105
|
-
- `CAWDEX_INSTALL_SPEC` controls `setup.sh`, default `cawdex@latest`.
|
|
106
|
-
- `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, and other Cawdex provider env vars
|
|
107
|
-
are passed through by the launched Cawdex process.
|
|
108
|
-
- `model`, `provider`, `max_turns`, `max_tokens`, `context_window_tokens`,
|
|
109
|
-
`temperature`, and `output_format` become Cawdex CLI flags.
|
|
110
|
-
- `extra_env` and `extra_args` let Exgentic experiments pass additional
|
|
111
|
-
Cawdex settings without modifying this adapter.
|
|
112
|
-
|
|
113
|
-
The adapter reports Cawdex's `summary.json` estimated cost to Exgentic
|
|
114
|
-
when `CAWDEX_BENCHMARK_TRACE_DIR` output is present.
|
|
1
|
+
# Cawdex for Exgentic
|
|
2
|
+
|
|
3
|
+
This directory is a custom Exgentic agent package for running Cawdex in
|
|
4
|
+
Open Agent Leaderboard style evaluations.
|
|
5
|
+
|
|
6
|
+
The adapter implements Exgentic's `Agent` / `AgentInstance` split. On each
|
|
7
|
+
`react()` step it writes an Exgentic prompt, launches Cawdex in
|
|
8
|
+
non-interactive `/benchmark` mode, asks Cawdex to finish with one JSON
|
|
9
|
+
action object, then maps that JSON back to an Exgentic `ActionType`.
|
|
10
|
+
It auto-selects specialized `/benchmark` profiles for AppWorld, BrowseComp+,
|
|
11
|
+
tau2, ARC, SaaS, roadmap, mobile, WebDevBench, SWE-Cycle, SWE-CI, SWE-PRBench, TML-Bench, and Pi-Bench-style tasks from the task, context, and
|
|
12
|
+
available action schemas. It also builds a deterministic recommended action
|
|
13
|
+
shortlist before the full schema list, including required argument keys and
|
|
14
|
+
redacted exact latest-observation/context hints when those required values are
|
|
15
|
+
already present. This matches the Open Agent leaderboard finding that
|
|
16
|
+
action-scaffold details matter while preserving compatibility with every
|
|
17
|
+
available benchmark action. Long sessions use a folded history ledger
|
|
18
|
+
that preserves latest observations, selected actions, action counts, and
|
|
19
|
+
diagnostics without reinjecting bulky raw stdout into every step. Before
|
|
20
|
+
dispatch, near-miss action JSON is repaired conservatively: action names are
|
|
21
|
+
matched by case/identifier normalization, schema argument keys are canonicalized,
|
|
22
|
+
required schema fields are filled from exact latest-observation/context keys
|
|
23
|
+
when available, and unknown extra keys are dropped when the benchmark exposes a
|
|
24
|
+
fixed schema. If Cawdex produces malformed or missing action JSON, the adapter
|
|
25
|
+
uses the same shortlist and exact required-argument hints to select a viable
|
|
26
|
+
non-finish action while the latest observation is still pending, instead of
|
|
27
|
+
falling straight to a finish/message action. When a selected action is followed
|
|
28
|
+
by an effectively unchanged observation, the next shortlist and deterministic
|
|
29
|
+
fallback mark that action as a no-effect repeat and prefer another viable action
|
|
30
|
+
with satisfiable required arguments.
|
|
31
|
+
For WebDevBench-style app-agency tasks, the folded ledger keeps canary business
|
|
32
|
+
requirements, frontend/backend coupling evidence, and production/security gaps
|
|
33
|
+
visible across turns so an attractive UI-only action does not look complete.
|
|
34
|
+
For SWE-Cycle-style lifecycle tasks, it keeps the phase, bare-repo environment
|
|
35
|
+
setup state, implementation requirements, generated/selected tests, judge
|
|
36
|
+
commands, and unresolved lifecycle gaps visible across turns.
|
|
37
|
+
For SWE-CI-style codebase maintenance tasks, it keeps current/target commits,
|
|
38
|
+
test gaps, inferred requirements, touched files, verifier deltas, and unresolved
|
|
39
|
+
regressions visible across the `run_tests -> define_requirements -> modify_code`
|
|
40
|
+
loop.
|
|
41
|
+
For SWE-PRBench-style PR review tasks, it keeps PR metadata, changed files, diff
|
|
42
|
+
hunks, suspected findings, evidence gaps, and context-expansion reasons visible
|
|
43
|
+
so the agent produces severity-rated findings instead of drifting into broad
|
|
44
|
+
repo inspection or unsolicited patch generation.
|
|
45
|
+
For TML-Bench-style tabular ML tasks, it keeps the data contract, metric,
|
|
46
|
+
validation split, leakage checks, model artifact, submission path, and submission
|
|
47
|
+
validity evidence visible so the agent favors a reproducible valid baseline over
|
|
48
|
+
an invalid high-complexity attempt.
|
|
49
|
+
For Pi-Bench-style proactive assistant tasks, it keeps user profile, current
|
|
50
|
+
request, message/file/app context, available domain tools, hidden-intent
|
|
51
|
+
hypotheses, privacy risk, clarification state, selected actions, and observable
|
|
52
|
+
completion evidence visible so the agent can ask focused questions without
|
|
53
|
+
overreaching or fabricating personal state.
|
|
54
|
+
|
|
55
|
+
## Python API use
|
|
56
|
+
|
|
57
|
+
```python
|
|
58
|
+
import sys
|
|
59
|
+
from pathlib import Path
|
|
60
|
+
|
|
61
|
+
sys.path.insert(0, str(Path("path/to/resources/exgentic").resolve()))
|
|
62
|
+
|
|
63
|
+
from cawdex_agent import CawdexAgent
|
|
64
|
+
from exgentic.interfaces.lib import evaluate
|
|
65
|
+
|
|
66
|
+
results = evaluate(
|
|
67
|
+
benchmark="gsm8k",
|
|
68
|
+
agent=CawdexAgent(
|
|
69
|
+
model="openrouter/free",
|
|
70
|
+
provider="openrouter",
|
|
71
|
+
max_turns=8,
|
|
72
|
+
),
|
|
73
|
+
num_tasks=5,
|
|
74
|
+
)
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
## Registry use
|
|
78
|
+
|
|
79
|
+
For Exgentic CLI usage, copy this directory into an Exgentic checkout at:
|
|
80
|
+
|
|
81
|
+
```text
|
|
82
|
+
src/exgentic/agents/cawdex_agent/
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Then add a registry entry:
|
|
86
|
+
|
|
87
|
+
```python
|
|
88
|
+
"cawdex_agent": RegistryEntry(
|
|
89
|
+
display_name="Cawdex",
|
|
90
|
+
module="exgentic.agents.cawdex_agent.agent",
|
|
91
|
+
class_name="CawdexAgent",
|
|
92
|
+
),
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
After that:
|
|
96
|
+
|
|
97
|
+
```bash
|
|
98
|
+
exgentic install --agent cawdex_agent --local
|
|
99
|
+
exgentic evaluate --benchmark gsm8k --agent cawdex_agent --model openrouter/free
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
## Configuration
|
|
103
|
+
|
|
104
|
+
- `CAWDEX_EXGENTIC_COMMAND` or `CAWDEX_EXGENTIC_COMMAND` overrides the command, default `cawdex`.
|
|
105
|
+
- `CAWDEX_INSTALL_SPEC` controls `setup.sh`, default `cawdex@latest`.
|
|
106
|
+
- `OPENROUTER_API_KEY`, `OPENAI_API_KEY`, and other Cawdex provider env vars
|
|
107
|
+
are passed through by the launched Cawdex process.
|
|
108
|
+
- `model`, `provider`, `max_turns`, `max_tokens`, `context_window_tokens`,
|
|
109
|
+
`temperature`, and `output_format` become Cawdex CLI flags.
|
|
110
|
+
- `extra_env` and `extra_args` let Exgentic experiments pass additional
|
|
111
|
+
Cawdex settings without modifying this adapter.
|
|
112
|
+
|
|
113
|
+
The adapter reports Cawdex's `summary.json` estimated cost to Exgentic
|
|
114
|
+
when `CAWDEX_BENCHMARK_TRACE_DIR` output is present.
|
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
"""Cawdex adapter package for Exgentic."""
|
|
2
|
-
|
|
3
|
-
from .agent import CawdexAgent, CawdexAgentInstance, CawdexAgent, CawdexAgentInstance
|
|
4
|
-
|
|
5
|
-
__all__ = ["CawdexAgent", "CawdexAgentInstance", "CawdexAgent", "CawdexAgentInstance"]
|
|
1
|
+
"""Cawdex adapter package for Exgentic."""
|
|
2
|
+
|
|
3
|
+
from .agent import CawdexAgent, CawdexAgentInstance, CawdexAgent, CawdexAgentInstance
|
|
4
|
+
|
|
5
|
+
__all__ = ["CawdexAgent", "CawdexAgentInstance", "CawdexAgent", "CawdexAgentInstance"]
|