adversarial-workflow 0.5.0__tar.gz → 0.6.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/PKG-INFO +94 -4
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/README.md +93 -3
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/__init__.py +1 -1
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/cli.py +155 -246
- adversarial_workflow-0.6.1/adversarial_workflow/evaluators/__init__.py +45 -0
- adversarial_workflow-0.6.1/adversarial_workflow/evaluators/builtins.py +36 -0
- adversarial_workflow-0.6.1/adversarial_workflow/evaluators/config.py +49 -0
- adversarial_workflow-0.6.1/adversarial_workflow/evaluators/discovery.py +212 -0
- adversarial_workflow-0.6.1/adversarial_workflow/evaluators/runner.py +313 -0
- adversarial_workflow-0.6.1/adversarial_workflow/utils/__init__.py +17 -0
- adversarial_workflow-0.6.1/adversarial_workflow/utils/colors.py +9 -0
- adversarial_workflow-0.6.1/adversarial_workflow/utils/config.py +44 -0
- adversarial_workflow-0.6.1/adversarial_workflow/utils/file_splitter.py +378 -0
- adversarial_workflow-0.6.1/adversarial_workflow/utils/validation.py +76 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/PKG-INFO +94 -4
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/SOURCES.txt +18 -1
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/pyproject.toml +4 -3
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_cli.py +1 -1
- adversarial_workflow-0.6.1/tests/test_cli_dynamic_commands.py +621 -0
- adversarial_workflow-0.6.1/tests/test_env_loading.py +214 -0
- adversarial_workflow-0.6.1/tests/test_evaluator_config.py +135 -0
- adversarial_workflow-0.6.1/tests/test_evaluator_discovery.py +680 -0
- adversarial_workflow-0.6.1/tests/test_evaluator_runner.py +249 -0
- adversarial_workflow-0.6.1/tests/test_list_evaluators.py +121 -0
- adversarial_workflow-0.6.1/tests/test_utils_validation.py +116 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/LICENSE +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/__main__.py +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/.aider.conf.yml.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/.env.example.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/README.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/AGENT-SYSTEM-GUIDE.md +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/README.md.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/agent-handoffs-minimal.json.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/agent-handoffs.json.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/current-state.json.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/config.yml.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/evaluate_plan.sh.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/example-task.md.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/proofread_content.sh.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/review_implementation.sh.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/validate_tests.sh.template +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/dependency_links.txt +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/entry_points.txt +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/requires.txt +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/top_level.txt +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/setup.cfg +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/setup.py +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_config.py +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_evaluate.py +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_file_splitter.py +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_python_version.py +0 -0
- {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_split_command.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: adversarial-workflow
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.6.1
|
|
4
4
|
Summary: Multi-stage AI code review system preventing phantom work - Author/Evaluator pattern
|
|
5
5
|
Author: Fredrik Matheson
|
|
6
6
|
License: MIT
|
|
@@ -35,6 +35,10 @@ Dynamic: license-file
|
|
|
35
35
|
|
|
36
36
|
# Adversarial Workflow
|
|
37
37
|
|
|
38
|
+
[](https://pypi.org/project/adversarial-workflow/)
|
|
39
|
+
[](https://www.python.org/downloads/)
|
|
40
|
+
[](https://opensource.org/licenses/MIT)
|
|
41
|
+
|
|
38
42
|
**A multi-stage AI code review system that makes your code better**
|
|
39
43
|
|
|
40
44
|
Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to implement but not delivering) through adversarial verification using independent review stages. A battle-tested workflow from the [thematic-cuts](https://github.com/movito/thematic-cuts) project that achieved 96.9% test pass rate improvement.
|
|
@@ -51,6 +55,31 @@ Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to i
|
|
|
51
55
|
- 🎯 **Tool-agnostic**: Use with Claude Code, Cursor, Aider, manual coding, or any workflow
|
|
52
56
|
- ✨ **Interactive onboarding**: Guided setup wizard gets you started in <5 minutes
|
|
53
57
|
|
|
58
|
+
## What's New in v0.6.0
|
|
59
|
+
|
|
60
|
+
🔌 **Plugin Architecture** - Define custom evaluators without modifying the package:
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
# Create a custom evaluator
|
|
64
|
+
mkdir -p .adversarial/evaluators
|
|
65
|
+
cat > .adversarial/evaluators/athena.yml << 'EOF'
|
|
66
|
+
name: athena
|
|
67
|
+
description: Knowledge evaluation using Gemini 2.5 Pro
|
|
68
|
+
model: gemini-2.5-pro
|
|
69
|
+
api_key_env: GEMINI_API_KEY
|
|
70
|
+
prompt: |
|
|
71
|
+
You are Athena, a knowledge evaluation specialist...
|
|
72
|
+
EOF
|
|
73
|
+
|
|
74
|
+
# Use it immediately
|
|
75
|
+
adversarial athena docs/research-plan.md
|
|
76
|
+
|
|
77
|
+
# List all available evaluators
|
|
78
|
+
adversarial list-evaluators
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
See [Custom Evaluators](#custom-evaluators) for full documentation, or check the [CHANGELOG](CHANGELOG.md) for complete release history.
|
|
82
|
+
|
|
54
83
|
## Prerequisites
|
|
55
84
|
|
|
56
85
|
Before installing, ensure you have:
|
|
@@ -382,8 +411,68 @@ adversarial split task.md # Split large files into smaller parts
|
|
|
382
411
|
adversarial split task.md --dry-run # Preview split without creating files
|
|
383
412
|
adversarial review # Phase 3: Review implementation
|
|
384
413
|
adversarial validate "pytest" # Phase 4: Validate with tests
|
|
414
|
+
adversarial list-evaluators # List all available evaluators
|
|
415
|
+
```
|
|
416
|
+
|
|
417
|
+
## Custom Evaluators
|
|
418
|
+
|
|
419
|
+
Starting with v0.6.0, you can define project-specific evaluators without modifying the package.
|
|
420
|
+
|
|
421
|
+
### Creating a Custom Evaluator
|
|
422
|
+
|
|
423
|
+
1. Create the evaluators directory:
|
|
424
|
+
```bash
|
|
425
|
+
mkdir -p .adversarial/evaluators
|
|
426
|
+
```
|
|
427
|
+
|
|
428
|
+
2. Create a YAML definition:
|
|
429
|
+
```yaml
|
|
430
|
+
# .adversarial/evaluators/athena.yml
|
|
431
|
+
name: athena
|
|
432
|
+
description: Knowledge evaluation using Gemini 2.5 Pro
|
|
433
|
+
model: gemini-2.5-pro
|
|
434
|
+
api_key_env: GEMINI_API_KEY
|
|
435
|
+
output_suffix: KNOWLEDGE-EVALUATION
|
|
436
|
+
prompt: |
|
|
437
|
+
You are Athena, a knowledge evaluation specialist...
|
|
438
|
+
|
|
439
|
+
# Optional
|
|
440
|
+
aliases:
|
|
441
|
+
- knowledge
|
|
442
|
+
```
|
|
443
|
+
|
|
444
|
+
3. Use it like any built-in evaluator:
|
|
445
|
+
```bash
|
|
446
|
+
adversarial athena docs/research-plan.md
|
|
447
|
+
```
|
|
448
|
+
|
|
449
|
+
### Evaluator YAML Schema
|
|
450
|
+
|
|
451
|
+
| Field | Required | Description |
|
|
452
|
+
|-------|----------|-------------|
|
|
453
|
+
| `name` | Yes | Command name |
|
|
454
|
+
| `description` | Yes | Help text shown in CLI |
|
|
455
|
+
| `model` | Yes | Model to use (e.g., `gpt-4o`, `gemini-2.5-pro`) |
|
|
456
|
+
| `api_key_env` | Yes | Environment variable for API key |
|
|
457
|
+
| `output_suffix` | Yes | Log file suffix (e.g., `KNOWLEDGE-EVAL`) |
|
|
458
|
+
| `prompt` | Yes | The evaluation prompt |
|
|
459
|
+
| `aliases` | No | Alternative command names |
|
|
460
|
+
| `log_prefix` | No | CLI output prefix |
|
|
461
|
+
| `fallback_model` | No | Fallback model if primary fails |
|
|
462
|
+
| `version` | No | Evaluator version (default: 1.0.0) |
|
|
463
|
+
|
|
464
|
+
### Listing Available Evaluators
|
|
465
|
+
|
|
466
|
+
```bash
|
|
467
|
+
adversarial list-evaluators
|
|
385
468
|
```
|
|
386
469
|
|
|
470
|
+
### Example: Athena Knowledge Evaluator
|
|
471
|
+
|
|
472
|
+
See [docs/examples/athena.yml](docs/examples/athena.yml) for a complete example of a knowledge-focused evaluator using Gemini 2.5 Pro.
|
|
473
|
+
|
|
474
|
+
For full documentation on custom evaluators, see [docs/CUSTOM_EVALUATORS.md](docs/CUSTOM_EVALUATORS.md).
|
|
475
|
+
|
|
387
476
|
## Configuration
|
|
388
477
|
|
|
389
478
|
### Option 1: YAML Config (persistent)
|
|
@@ -796,12 +885,13 @@ From the [thematic-cuts](https://github.com/movito/thematic-cuts) project:
|
|
|
796
885
|
|
|
797
886
|
## Documentation
|
|
798
887
|
|
|
799
|
-
- **
|
|
888
|
+
- **[Custom Evaluators Guide](docs/CUSTOM_EVALUATORS.md)**: Create project-specific evaluators
|
|
889
|
+
- **[Integration Guide](docs/INTEGRATION-GUIDE.md)**: Detailed integration strategies
|
|
890
|
+
- **[CHANGELOG](CHANGELOG.md)**: Release history and version notes
|
|
891
|
+
- **Interaction Patterns**: How Author-Evaluator collaboration works
|
|
800
892
|
- **Token Optimization**: Detailed Aider configuration guide
|
|
801
893
|
- **Workflow Phases**: Step-by-step guide for each phase
|
|
802
894
|
- **Troubleshooting**: Common issues and solutions
|
|
803
|
-
- **Examples**: Real integration scenarios
|
|
804
|
-
- **Terminology**: Official standards for Author/Reviewer concepts
|
|
805
895
|
|
|
806
896
|
See `docs/` directory for comprehensive guides.
|
|
807
897
|
|
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# Adversarial Workflow
|
|
2
2
|
|
|
3
|
+
[](https://pypi.org/project/adversarial-workflow/)
|
|
4
|
+
[](https://www.python.org/downloads/)
|
|
5
|
+
[](https://opensource.org/licenses/MIT)
|
|
6
|
+
|
|
3
7
|
**A multi-stage AI code review system that makes your code better**
|
|
4
8
|
|
|
5
9
|
Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to implement but not delivering) through adversarial verification using independent review stages. A battle-tested workflow from the [thematic-cuts](https://github.com/movito/thematic-cuts) project that achieved 96.9% test pass rate improvement.
|
|
@@ -16,6 +20,31 @@ Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to i
|
|
|
16
20
|
- 🎯 **Tool-agnostic**: Use with Claude Code, Cursor, Aider, manual coding, or any workflow
|
|
17
21
|
- ✨ **Interactive onboarding**: Guided setup wizard gets you started in <5 minutes
|
|
18
22
|
|
|
23
|
+
## What's New in v0.6.0
|
|
24
|
+
|
|
25
|
+
🔌 **Plugin Architecture** - Define custom evaluators without modifying the package:
|
|
26
|
+
|
|
27
|
+
```bash
|
|
28
|
+
# Create a custom evaluator
|
|
29
|
+
mkdir -p .adversarial/evaluators
|
|
30
|
+
cat > .adversarial/evaluators/athena.yml << 'EOF'
|
|
31
|
+
name: athena
|
|
32
|
+
description: Knowledge evaluation using Gemini 2.5 Pro
|
|
33
|
+
model: gemini-2.5-pro
|
|
34
|
+
api_key_env: GEMINI_API_KEY
|
|
35
|
+
prompt: |
|
|
36
|
+
You are Athena, a knowledge evaluation specialist...
|
|
37
|
+
EOF
|
|
38
|
+
|
|
39
|
+
# Use it immediately
|
|
40
|
+
adversarial athena docs/research-plan.md
|
|
41
|
+
|
|
42
|
+
# List all available evaluators
|
|
43
|
+
adversarial list-evaluators
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
See [Custom Evaluators](#custom-evaluators) for full documentation, or check the [CHANGELOG](CHANGELOG.md) for complete release history.
|
|
47
|
+
|
|
19
48
|
## Prerequisites
|
|
20
49
|
|
|
21
50
|
Before installing, ensure you have:
|
|
@@ -347,8 +376,68 @@ adversarial split task.md # Split large files into smaller parts
|
|
|
347
376
|
adversarial split task.md --dry-run # Preview split without creating files
|
|
348
377
|
adversarial review # Phase 3: Review implementation
|
|
349
378
|
adversarial validate "pytest" # Phase 4: Validate with tests
|
|
379
|
+
adversarial list-evaluators # List all available evaluators
|
|
380
|
+
```
|
|
381
|
+
|
|
382
|
+
## Custom Evaluators
|
|
383
|
+
|
|
384
|
+
Starting with v0.6.0, you can define project-specific evaluators without modifying the package.
|
|
385
|
+
|
|
386
|
+
### Creating a Custom Evaluator
|
|
387
|
+
|
|
388
|
+
1. Create the evaluators directory:
|
|
389
|
+
```bash
|
|
390
|
+
mkdir -p .adversarial/evaluators
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
2. Create a YAML definition:
|
|
394
|
+
```yaml
|
|
395
|
+
# .adversarial/evaluators/athena.yml
|
|
396
|
+
name: athena
|
|
397
|
+
description: Knowledge evaluation using Gemini 2.5 Pro
|
|
398
|
+
model: gemini-2.5-pro
|
|
399
|
+
api_key_env: GEMINI_API_KEY
|
|
400
|
+
output_suffix: KNOWLEDGE-EVALUATION
|
|
401
|
+
prompt: |
|
|
402
|
+
You are Athena, a knowledge evaluation specialist...
|
|
403
|
+
|
|
404
|
+
# Optional
|
|
405
|
+
aliases:
|
|
406
|
+
- knowledge
|
|
407
|
+
```
|
|
408
|
+
|
|
409
|
+
3. Use it like any built-in evaluator:
|
|
410
|
+
```bash
|
|
411
|
+
adversarial athena docs/research-plan.md
|
|
412
|
+
```
|
|
413
|
+
|
|
414
|
+
### Evaluator YAML Schema
|
|
415
|
+
|
|
416
|
+
| Field | Required | Description |
|
|
417
|
+
|-------|----------|-------------|
|
|
418
|
+
| `name` | Yes | Command name |
|
|
419
|
+
| `description` | Yes | Help text shown in CLI |
|
|
420
|
+
| `model` | Yes | Model to use (e.g., `gpt-4o`, `gemini-2.5-pro`) |
|
|
421
|
+
| `api_key_env` | Yes | Environment variable for API key |
|
|
422
|
+
| `output_suffix` | Yes | Log file suffix (e.g., `KNOWLEDGE-EVAL`) |
|
|
423
|
+
| `prompt` | Yes | The evaluation prompt |
|
|
424
|
+
| `aliases` | No | Alternative command names |
|
|
425
|
+
| `log_prefix` | No | CLI output prefix |
|
|
426
|
+
| `fallback_model` | No | Fallback model if primary fails |
|
|
427
|
+
| `version` | No | Evaluator version (default: 1.0.0) |
|
|
428
|
+
|
|
429
|
+
### Listing Available Evaluators
|
|
430
|
+
|
|
431
|
+
```bash
|
|
432
|
+
adversarial list-evaluators
|
|
350
433
|
```
|
|
351
434
|
|
|
435
|
+
### Example: Athena Knowledge Evaluator
|
|
436
|
+
|
|
437
|
+
See [docs/examples/athena.yml](docs/examples/athena.yml) for a complete example of a knowledge-focused evaluator using Gemini 2.5 Pro.
|
|
438
|
+
|
|
439
|
+
For full documentation on custom evaluators, see [docs/CUSTOM_EVALUATORS.md](docs/CUSTOM_EVALUATORS.md).
|
|
440
|
+
|
|
352
441
|
## Configuration
|
|
353
442
|
|
|
354
443
|
### Option 1: YAML Config (persistent)
|
|
@@ -761,12 +850,13 @@ From the [thematic-cuts](https://github.com/movito/thematic-cuts) project:
|
|
|
761
850
|
|
|
762
851
|
## Documentation
|
|
763
852
|
|
|
764
|
-
- **
|
|
853
|
+
- **[Custom Evaluators Guide](docs/CUSTOM_EVALUATORS.md)**: Create project-specific evaluators
|
|
854
|
+
- **[Integration Guide](docs/INTEGRATION-GUIDE.md)**: Detailed integration strategies
|
|
855
|
+
- **[CHANGELOG](CHANGELOG.md)**: Release history and version notes
|
|
856
|
+
- **Interaction Patterns**: How Author-Evaluator collaboration works
|
|
765
857
|
- **Token Optimization**: Detailed Aider configuration guide
|
|
766
858
|
- **Workflow Phases**: Step-by-step guide for each phase
|
|
767
859
|
- **Troubleshooting**: Common issues and solutions
|
|
768
|
-
- **Examples**: Real integration scenarios
|
|
769
|
-
- **Terminology**: Official standards for Author/Reviewer concepts
|
|
770
860
|
|
|
771
861
|
See `docs/` directory for comprehensive guides.
|
|
772
862
|
|