adversarial-workflow 0.5.0__tar.gz → 0.6.1__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (52) hide show
  1. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/PKG-INFO +94 -4
  2. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/README.md +93 -3
  3. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/__init__.py +1 -1
  4. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/cli.py +155 -246
  5. adversarial_workflow-0.6.1/adversarial_workflow/evaluators/__init__.py +45 -0
  6. adversarial_workflow-0.6.1/adversarial_workflow/evaluators/builtins.py +36 -0
  7. adversarial_workflow-0.6.1/adversarial_workflow/evaluators/config.py +49 -0
  8. adversarial_workflow-0.6.1/adversarial_workflow/evaluators/discovery.py +212 -0
  9. adversarial_workflow-0.6.1/adversarial_workflow/evaluators/runner.py +313 -0
  10. adversarial_workflow-0.6.1/adversarial_workflow/utils/__init__.py +17 -0
  11. adversarial_workflow-0.6.1/adversarial_workflow/utils/colors.py +9 -0
  12. adversarial_workflow-0.6.1/adversarial_workflow/utils/config.py +44 -0
  13. adversarial_workflow-0.6.1/adversarial_workflow/utils/file_splitter.py +378 -0
  14. adversarial_workflow-0.6.1/adversarial_workflow/utils/validation.py +76 -0
  15. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/PKG-INFO +94 -4
  16. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/SOURCES.txt +18 -1
  17. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/pyproject.toml +4 -3
  18. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_cli.py +1 -1
  19. adversarial_workflow-0.6.1/tests/test_cli_dynamic_commands.py +621 -0
  20. adversarial_workflow-0.6.1/tests/test_env_loading.py +214 -0
  21. adversarial_workflow-0.6.1/tests/test_evaluator_config.py +135 -0
  22. adversarial_workflow-0.6.1/tests/test_evaluator_discovery.py +680 -0
  23. adversarial_workflow-0.6.1/tests/test_evaluator_runner.py +249 -0
  24. adversarial_workflow-0.6.1/tests/test_list_evaluators.py +121 -0
  25. adversarial_workflow-0.6.1/tests/test_utils_validation.py +116 -0
  26. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/LICENSE +0 -0
  27. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/__main__.py +0 -0
  28. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/.aider.conf.yml.template +0 -0
  29. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/.env.example.template +0 -0
  30. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/README.template +0 -0
  31. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/AGENT-SYSTEM-GUIDE.md +0 -0
  32. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/README.md.template +0 -0
  33. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/agent-handoffs-minimal.json.template +0 -0
  34. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/agent-handoffs.json.template +0 -0
  35. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/agent-context/current-state.json.template +0 -0
  36. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/config.yml.template +0 -0
  37. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/evaluate_plan.sh.template +0 -0
  38. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/example-task.md.template +0 -0
  39. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/proofread_content.sh.template +0 -0
  40. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/review_implementation.sh.template +0 -0
  41. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow/templates/validate_tests.sh.template +0 -0
  42. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/dependency_links.txt +0 -0
  43. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/entry_points.txt +0 -0
  44. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/requires.txt +0 -0
  45. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/adversarial_workflow.egg-info/top_level.txt +0 -0
  46. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/setup.cfg +0 -0
  47. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/setup.py +0 -0
  48. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_config.py +0 -0
  49. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_evaluate.py +0 -0
  50. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_file_splitter.py +0 -0
  51. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_python_version.py +0 -0
  52. {adversarial_workflow-0.5.0 → adversarial_workflow-0.6.1}/tests/test_split_command.py +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: adversarial-workflow
3
- Version: 0.5.0
3
+ Version: 0.6.1
4
4
  Summary: Multi-stage AI code review system preventing phantom work - Author/Evaluator pattern
5
5
  Author: Fredrik Matheson
6
6
  License: MIT
@@ -35,6 +35,10 @@ Dynamic: license-file
35
35
 
36
36
  # Adversarial Workflow
37
37
 
38
+ [![PyPI version](https://badge.fury.io/py/adversarial-workflow.svg)](https://pypi.org/project/adversarial-workflow/)
39
+ [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
40
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
41
+
38
42
  **A multi-stage AI code review system that makes your code better**
39
43
 
40
44
  Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to implement but not delivering) through adversarial verification using independent review stages. A battle-tested workflow from the [thematic-cuts](https://github.com/movito/thematic-cuts) project that achieved 96.9% test pass rate improvement.
@@ -51,6 +55,31 @@ Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to i
51
55
  - 🎯 **Tool-agnostic**: Use with Claude Code, Cursor, Aider, manual coding, or any workflow
52
56
  - ✨ **Interactive onboarding**: Guided setup wizard gets you started in <5 minutes
53
57
 
58
+ ## What's New in v0.6.0
59
+
60
+ 🔌 **Plugin Architecture** - Define custom evaluators without modifying the package:
61
+
62
+ ```bash
63
+ # Create a custom evaluator
64
+ mkdir -p .adversarial/evaluators
65
+ cat > .adversarial/evaluators/athena.yml << 'EOF'
66
+ name: athena
67
+ description: Knowledge evaluation using Gemini 2.5 Pro
68
+ model: gemini-2.5-pro
69
+ api_key_env: GEMINI_API_KEY
70
+ prompt: |
71
+ You are Athena, a knowledge evaluation specialist...
72
+ EOF
73
+
74
+ # Use it immediately
75
+ adversarial athena docs/research-plan.md
76
+
77
+ # List all available evaluators
78
+ adversarial list-evaluators
79
+ ```
80
+
81
+ See [Custom Evaluators](#custom-evaluators) for full documentation, or check the [CHANGELOG](CHANGELOG.md) for complete release history.
82
+
54
83
  ## Prerequisites
55
84
 
56
85
  Before installing, ensure you have:
@@ -382,8 +411,68 @@ adversarial split task.md # Split large files into smaller parts
382
411
  adversarial split task.md --dry-run # Preview split without creating files
383
412
  adversarial review # Phase 3: Review implementation
384
413
  adversarial validate "pytest" # Phase 4: Validate with tests
414
+ adversarial list-evaluators # List all available evaluators
415
+ ```
416
+
417
+ ## Custom Evaluators
418
+
419
+ Starting with v0.6.0, you can define project-specific evaluators without modifying the package.
420
+
421
+ ### Creating a Custom Evaluator
422
+
423
+ 1. Create the evaluators directory:
424
+ ```bash
425
+ mkdir -p .adversarial/evaluators
426
+ ```
427
+
428
+ 2. Create a YAML definition:
429
+ ```yaml
430
+ # .adversarial/evaluators/athena.yml
431
+ name: athena
432
+ description: Knowledge evaluation using Gemini 2.5 Pro
433
+ model: gemini-2.5-pro
434
+ api_key_env: GEMINI_API_KEY
435
+ output_suffix: KNOWLEDGE-EVALUATION
436
+ prompt: |
437
+ You are Athena, a knowledge evaluation specialist...
438
+
439
+ # Optional
440
+ aliases:
441
+ - knowledge
442
+ ```
443
+
444
+ 3. Use it like any built-in evaluator:
445
+ ```bash
446
+ adversarial athena docs/research-plan.md
447
+ ```
448
+
449
+ ### Evaluator YAML Schema
450
+
451
+ | Field | Required | Description |
452
+ |-------|----------|-------------|
453
+ | `name` | Yes | Command name |
454
+ | `description` | Yes | Help text shown in CLI |
455
+ | `model` | Yes | Model to use (e.g., `gpt-4o`, `gemini-2.5-pro`) |
456
+ | `api_key_env` | Yes | Environment variable for API key |
457
+ | `output_suffix` | Yes | Log file suffix (e.g., `KNOWLEDGE-EVAL`) |
458
+ | `prompt` | Yes | The evaluation prompt |
459
+ | `aliases` | No | Alternative command names |
460
+ | `log_prefix` | No | CLI output prefix |
461
+ | `fallback_model` | No | Fallback model if primary fails |
462
+ | `version` | No | Evaluator version (default: 1.0.0) |
463
+
464
+ ### Listing Available Evaluators
465
+
466
+ ```bash
467
+ adversarial list-evaluators
385
468
  ```
386
469
 
470
+ ### Example: Athena Knowledge Evaluator
471
+
472
+ See [docs/examples/athena.yml](docs/examples/athena.yml) for a complete example of a knowledge-focused evaluator using Gemini 2.5 Pro.
473
+
474
+ For full documentation on custom evaluators, see [docs/CUSTOM_EVALUATORS.md](docs/CUSTOM_EVALUATORS.md).
475
+
387
476
  ## Configuration
388
477
 
389
478
  ### Option 1: YAML Config (persistent)
@@ -796,12 +885,13 @@ From the [thematic-cuts](https://github.com/movito/thematic-cuts) project:
796
885
 
797
886
  ## Documentation
798
887
 
799
- - **Interaction Patterns**: How Author-Reviewer collaboration works
888
+ - **[Custom Evaluators Guide](docs/CUSTOM_EVALUATORS.md)**: Create project-specific evaluators
889
+ - **[Integration Guide](docs/INTEGRATION-GUIDE.md)**: Detailed integration strategies
890
+ - **[CHANGELOG](CHANGELOG.md)**: Release history and version notes
891
+ - **Interaction Patterns**: How Author-Evaluator collaboration works
800
892
  - **Token Optimization**: Detailed Aider configuration guide
801
893
  - **Workflow Phases**: Step-by-step guide for each phase
802
894
  - **Troubleshooting**: Common issues and solutions
803
- - **Examples**: Real integration scenarios
804
- - **Terminology**: Official standards for Author/Reviewer concepts
805
895
 
806
896
  See `docs/` directory for comprehensive guides.
807
897
 
@@ -1,5 +1,9 @@
1
1
  # Adversarial Workflow
2
2
 
3
+ [![PyPI version](https://badge.fury.io/py/adversarial-workflow.svg)](https://pypi.org/project/adversarial-workflow/)
4
+ [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
5
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
6
+
3
7
  **A multi-stage AI code review system that makes your code better**
4
8
 
5
9
  Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to implement but not delivering) through adversarial verification using independent review stages. A battle-tested workflow from the [thematic-cuts](https://github.com/movito/thematic-cuts) project that achieved 96.9% test pass rate improvement.
@@ -16,6 +20,31 @@ Evaluate proposals, sort out ideas, and prevent "phantom work" (AI claiming to i
16
20
  - 🎯 **Tool-agnostic**: Use with Claude Code, Cursor, Aider, manual coding, or any workflow
17
21
  - ✨ **Interactive onboarding**: Guided setup wizard gets you started in <5 minutes
18
22
 
23
+ ## What's New in v0.6.0
24
+
25
+ 🔌 **Plugin Architecture** - Define custom evaluators without modifying the package:
26
+
27
+ ```bash
28
+ # Create a custom evaluator
29
+ mkdir -p .adversarial/evaluators
30
+ cat > .adversarial/evaluators/athena.yml << 'EOF'
31
+ name: athena
32
+ description: Knowledge evaluation using Gemini 2.5 Pro
33
+ model: gemini-2.5-pro
34
+ api_key_env: GEMINI_API_KEY
35
+ prompt: |
36
+ You are Athena, a knowledge evaluation specialist...
37
+ EOF
38
+
39
+ # Use it immediately
40
+ adversarial athena docs/research-plan.md
41
+
42
+ # List all available evaluators
43
+ adversarial list-evaluators
44
+ ```
45
+
46
+ See [Custom Evaluators](#custom-evaluators) for full documentation, or check the [CHANGELOG](CHANGELOG.md) for complete release history.
47
+
19
48
  ## Prerequisites
20
49
 
21
50
  Before installing, ensure you have:
@@ -347,8 +376,68 @@ adversarial split task.md # Split large files into smaller parts
347
376
  adversarial split task.md --dry-run # Preview split without creating files
348
377
  adversarial review # Phase 3: Review implementation
349
378
  adversarial validate "pytest" # Phase 4: Validate with tests
379
+ adversarial list-evaluators # List all available evaluators
380
+ ```
381
+
382
+ ## Custom Evaluators
383
+
384
+ Starting with v0.6.0, you can define project-specific evaluators without modifying the package.
385
+
386
+ ### Creating a Custom Evaluator
387
+
388
+ 1. Create the evaluators directory:
389
+ ```bash
390
+ mkdir -p .adversarial/evaluators
391
+ ```
392
+
393
+ 2. Create a YAML definition:
394
+ ```yaml
395
+ # .adversarial/evaluators/athena.yml
396
+ name: athena
397
+ description: Knowledge evaluation using Gemini 2.5 Pro
398
+ model: gemini-2.5-pro
399
+ api_key_env: GEMINI_API_KEY
400
+ output_suffix: KNOWLEDGE-EVALUATION
401
+ prompt: |
402
+ You are Athena, a knowledge evaluation specialist...
403
+
404
+ # Optional
405
+ aliases:
406
+ - knowledge
407
+ ```
408
+
409
+ 3. Use it like any built-in evaluator:
410
+ ```bash
411
+ adversarial athena docs/research-plan.md
412
+ ```
413
+
414
+ ### Evaluator YAML Schema
415
+
416
+ | Field | Required | Description |
417
+ |-------|----------|-------------|
418
+ | `name` | Yes | Command name |
419
+ | `description` | Yes | Help text shown in CLI |
420
+ | `model` | Yes | Model to use (e.g., `gpt-4o`, `gemini-2.5-pro`) |
421
+ | `api_key_env` | Yes | Environment variable for API key |
422
+ | `output_suffix` | Yes | Log file suffix (e.g., `KNOWLEDGE-EVAL`) |
423
+ | `prompt` | Yes | The evaluation prompt |
424
+ | `aliases` | No | Alternative command names |
425
+ | `log_prefix` | No | CLI output prefix |
426
+ | `fallback_model` | No | Fallback model if primary fails |
427
+ | `version` | No | Evaluator version (default: 1.0.0) |
428
+
429
+ ### Listing Available Evaluators
430
+
431
+ ```bash
432
+ adversarial list-evaluators
350
433
  ```
351
434
 
435
+ ### Example: Athena Knowledge Evaluator
436
+
437
+ See [docs/examples/athena.yml](docs/examples/athena.yml) for a complete example of a knowledge-focused evaluator using Gemini 2.5 Pro.
438
+
439
+ For full documentation on custom evaluators, see [docs/CUSTOM_EVALUATORS.md](docs/CUSTOM_EVALUATORS.md).
440
+
352
441
  ## Configuration
353
442
 
354
443
  ### Option 1: YAML Config (persistent)
@@ -761,12 +850,13 @@ From the [thematic-cuts](https://github.com/movito/thematic-cuts) project:
761
850
 
762
851
  ## Documentation
763
852
 
764
- - **Interaction Patterns**: How Author-Reviewer collaboration works
853
+ - **[Custom Evaluators Guide](docs/CUSTOM_EVALUATORS.md)**: Create project-specific evaluators
854
+ - **[Integration Guide](docs/INTEGRATION-GUIDE.md)**: Detailed integration strategies
855
+ - **[CHANGELOG](CHANGELOG.md)**: Release history and version notes
856
+ - **Interaction Patterns**: How Author-Evaluator collaboration works
765
857
  - **Token Optimization**: Detailed Aider configuration guide
766
858
  - **Workflow Phases**: Step-by-step guide for each phase
767
859
  - **Troubleshooting**: Common issues and solutions
768
- - **Examples**: Real integration scenarios
769
- - **Terminology**: Official standards for Author/Reviewer concepts
770
860
 
771
861
  See `docs/` directory for comprehensive guides.
772
862
 
@@ -12,7 +12,7 @@ Usage:
12
12
  adversarial validate "pytest"
13
13
  """
14
14
 
15
- __version__ = "0.5.0"
15
+ __version__ = "0.6.1"
16
16
  __author__ = "Fredrik Matheson"
17
17
  __license__ = "MIT"
18
18