myaidev-method 0.3.4 → 0.3.5

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (94) hide show
  1. package/.claude-plugin/plugin.json +0 -1
  2. package/.env.example +5 -4
  3. package/CHANGELOG.md +2 -2
  4. package/CONTENT_CREATION_GUIDE.md +489 -3211
  5. package/DEVELOPER_USE_CASES.md +1 -1
  6. package/MODULAR_INSTALLATION.md +2 -2
  7. package/README.md +39 -33
  8. package/TECHNICAL_ARCHITECTURE.md +1 -1
  9. package/USER_GUIDE.md +242 -190
  10. package/agents/content-editor-agent.md +90 -0
  11. package/agents/content-planner-agent.md +97 -0
  12. package/agents/content-research-agent.md +62 -0
  13. package/agents/content-seo-agent.md +101 -0
  14. package/agents/content-writer-agent.md +69 -0
  15. package/agents/infographic-analyzer-agent.md +63 -0
  16. package/agents/infographic-designer-agent.md +72 -0
  17. package/bin/cli.js +776 -422
  18. package/{content-rules.example.md → content-rules-example.md} +2 -2
  19. package/dist/mcp/health-check.js +82 -68
  20. package/dist/mcp/mcp-config.json +8 -0
  21. package/dist/mcp/openstack-server.js +1746 -1262
  22. package/dist/server/.tsbuildinfo +1 -1
  23. package/extension.json +21 -4
  24. package/package.json +181 -184
  25. package/skills/company-config/SKILL.md +133 -0
  26. package/skills/configure/SKILL.md +1 -1
  27. package/skills/myai-configurator/SKILL.md +77 -0
  28. package/skills/myai-configurator/content-creation-configurator/SKILL.md +516 -0
  29. package/skills/myai-configurator/content-maintenance-configurator/SKILL.md +397 -0
  30. package/skills/myai-content-enrichment/SKILL.md +114 -0
  31. package/skills/myai-content-ideation/SKILL.md +288 -0
  32. package/skills/myai-content-ideation/evals/evals.json +182 -0
  33. package/skills/myai-content-production-coordinator/SKILL.md +946 -0
  34. package/skills/{content-rules-setup → myai-content-rules-setup}/SKILL.md +1 -1
  35. package/skills/{content-verifier → myai-content-verifier}/SKILL.md +1 -1
  36. package/skills/myai-content-writer/SKILL.md +333 -0
  37. package/skills/{infographic → myai-infographic}/SKILL.md +1 -1
  38. package/skills/myai-proprietary-content-verifier/SKILL.md +175 -0
  39. package/skills/myai-proprietary-content-verifier/evals/evals.json +36 -0
  40. package/skills/myai-skill-builder/SKILL.md +699 -0
  41. package/skills/myai-skill-builder/agents/analyzer-agent.md +137 -0
  42. package/skills/myai-skill-builder/agents/comparator-agent.md +77 -0
  43. package/skills/myai-skill-builder/agents/grader-agent.md +103 -0
  44. package/skills/myai-skill-builder/assets/eval_review.html +131 -0
  45. package/skills/myai-skill-builder/references/schemas.md +211 -0
  46. package/skills/myai-skill-builder/scripts/aggregate_benchmark.py +190 -0
  47. package/skills/myai-skill-builder/scripts/generate_review.py +381 -0
  48. package/skills/myai-skill-builder/scripts/package_skill.py +91 -0
  49. package/skills/myai-skill-builder/scripts/run_eval.py +105 -0
  50. package/skills/myai-skill-builder/scripts/run_loop.py +211 -0
  51. package/skills/myai-skill-builder/scripts/utils.py +123 -0
  52. package/skills/myai-visual-generator/SKILL.md +125 -0
  53. package/skills/myai-visual-generator/evals/evals.json +155 -0
  54. package/skills/myai-visual-generator/references/infographic-pipeline.md +73 -0
  55. package/skills/myai-visual-generator/references/research-visuals.md +57 -0
  56. package/skills/myai-visual-generator/references/services.md +89 -0
  57. package/skills/myai-visual-generator/scripts/visual-generation-utils.js +1272 -0
  58. package/skills/myaidev-figma/SKILL.md +212 -0
  59. package/skills/myaidev-figma/capture.js +133 -0
  60. package/skills/myaidev-figma/crawl.js +130 -0
  61. package/skills/myaidev-figma-configure/SKILL.md +130 -0
  62. package/skills/openstack-manager/SKILL.md +1 -1
  63. package/skills/payloadcms-publisher/SKILL.md +141 -77
  64. package/skills/payloadcms-publisher/references/field-mapping.md +142 -0
  65. package/skills/payloadcms-publisher/references/lexical-format.md +97 -0
  66. package/skills/security-auditor/SKILL.md +1 -1
  67. package/src/cli/commands/addon.js +105 -7
  68. package/src/config/workflows.js +172 -228
  69. package/src/lib/ascii-banner.js +197 -182
  70. package/src/lib/{content-coordinator.js → content-production-coordinator.js} +649 -459
  71. package/src/lib/installation-detector.js +93 -59
  72. package/src/lib/payloadcms-utils.js +285 -510
  73. package/src/lib/workflow-installer.js +55 -0
  74. package/src/mcp/health-check.js +82 -68
  75. package/src/mcp/openstack-server.js +1746 -1262
  76. package/src/scripts/configure-visual-apis.js +224 -173
  77. package/src/scripts/configure-wordpress-mcp.js +96 -66
  78. package/src/scripts/init/install.js +109 -85
  79. package/src/scripts/init-project.js +138 -67
  80. package/src/scripts/utils/write-content.js +67 -52
  81. package/src/scripts/wordpress/publish-to-wordpress.js +128 -128
  82. package/src/templates/claude/CLAUDE.md +19 -12
  83. package/hooks/hooks.json +0 -26
  84. package/skills/content-coordinator/SKILL.md +0 -130
  85. package/skills/content-enrichment/SKILL.md +0 -80
  86. package/skills/content-writer/SKILL.md +0 -285
  87. package/skills/skill-builder/SKILL.md +0 -417
  88. package/skills/visual-generator/SKILL.md +0 -140
  89. /package/skills/{content-writer → myai-content-writer}/agents/editor-agent.md +0 -0
  90. /package/skills/{content-writer → myai-content-writer}/agents/planner-agent.md +0 -0
  91. /package/skills/{content-writer → myai-content-writer}/agents/research-agent.md +0 -0
  92. /package/skills/{content-writer → myai-content-writer}/agents/seo-agent.md +0 -0
  93. /package/skills/{content-writer → myai-content-writer}/agents/visual-planner-agent.md +0 -0
  94. /package/skills/{content-writer → myai-content-writer}/agents/writer-agent.md +0 -0
package/USER_GUIDE.md CHANGED
@@ -28,6 +28,8 @@ This guide covers everything you need to know about using, customizing, and exte
28
28
  - [Troubleshooting](#troubleshooting)
29
29
  - [Advanced Usage](#advanced-usage)
30
30
  - [Best Practices](#best-practices)
31
+ - [Skill Builder (Eval-Driven Development)](#-skill-builder-eval-driven-development)
32
+ - [Contributing Skills to the Marketplace](#-contributing-skills-to-the-marketplace)
31
33
 
32
34
  ## 🚀 Quick Start
33
35
 
@@ -97,7 +99,7 @@ Use the interactive configuration command to set up your environment—no manual
97
99
  /configure defaults
98
100
 
99
101
  # Set up content rules for brand voice
100
- /myai-content-rules-setup
102
+ /myai-content-rules-config
101
103
  ```
102
104
 
103
105
  ### Immediate Usage
@@ -184,10 +186,11 @@ Skills are organized into packs for modular installation:
184
186
 
185
187
  | Pack | Skills | Use Case |
186
188
  |------|--------|----------|
187
- | **content** | content-writer, content-verifier, visual-generator, wordpress-publisher | Content creation and publishing |
189
+ | **content** | content-writer, content-verifier, visual-generator, wordpress-publisher, + publishers | Content creation and publishing |
188
190
  | **development** | myaidev-workflow, myaidev-architect, myaidev-coder, myaidev-tester, myaidev-reviewer, myaidev-documenter, myaidev-analyze, myaidev-debug, myaidev-refactor, myaidev-performance, myaidev-migrate | SPARC methodology software development |
189
191
  | **security** | security-tester, security-auditor | Penetration testing and auditing |
190
192
  | **infrastructure** | coolify-deployer, openstack-manager | Deployment and cloud management |
193
+ | **skill-development** | myai-skill-builder, myai-skill-contributor | Skill creation, eval testing, and marketplace submission |
191
194
 
192
195
  ### SKILL.md Format
193
196
 
@@ -219,7 +222,6 @@ Both naming conventions work:
219
222
  | `/content-writer` | `/content-writer` | Create content |
220
223
  | `/configure` | `/configure` | Configuration |
221
224
  | `/wordpress-publisher` | `/wordpress-publisher` | WordPress publishing |
222
- | `/myai-generate-visual` | `/visual-generator` | Visual generation |
223
225
  | `/myai-openstack` | `/openstack-manager` | OpenStack management |
224
226
 
225
227
  ### Installation Management Commands
@@ -261,7 +263,7 @@ MyAIDev Method uses the `/configure` skill for interactive setup—**no manual `
261
263
  | `/configure openstack` | Configure OpenStack VM management |
262
264
  | `/configure defaults` | Set default content settings, visual generation API keys |
263
265
  | `/configure skills` | List, validate, backup, and manage skills |
264
- | `/myai-content-rules-setup` | Interactive brand voice and content rules setup |
266
+ | `/content-rules-config` | Interactive brand voice and content rules setup |
265
267
 
266
268
  ### WordPress Configuration
267
269
 
@@ -413,7 +415,7 @@ $ npx myaidev-method@latest update --claude
413
415
  📦 Updating components...
414
416
 
415
417
  Skills:
416
- ➕ visual-generator/SKILL.md (new)
418
+ myai-visual-generator/SKILL.md (new)
417
419
  ✓ content-writer/SKILL.md (unchanged)
418
420
 
419
421
  📝 wordpress-publisher/SKILL.md has been modified
@@ -518,8 +520,8 @@ Contains all skill definitions as SKILL.md files, organized by category:
518
520
  - `skills/docusaurus-publisher/SKILL.md` - Docusaurus documentation
519
521
  - `skills/mintlify-publisher/SKILL.md` - Mintlify documentation
520
522
  - `skills/astro-publisher/SKILL.md` - Astro site publishing
521
- - `skills/visual-generator/SKILL.md` - Visual generation
522
- - `skills/content-coordinator/SKILL.md` - Content coordination
523
+ - `skills/myai-visual-generator/SKILL.md` - Visual generation
524
+ - `skills/content-production-coordinator/SKILL.md` - Content coordination
523
525
 
524
526
  **Infrastructure:**
525
527
  - `skills/coolify-deployer/SKILL.md` - Coolify deployment automation
@@ -620,7 +622,7 @@ Set up your content creation pipeline with these configuration steps:
620
622
 
621
623
  ```bash
622
624
  # Interactive wizard for brand voice, tone, SEO guidelines, and formatting
623
- /myai-content-rules-setup
625
+ /content-rules-config
624
626
  ```
625
627
 
626
628
  This creates a `content-rules.md` file that customizes all AI-generated content to match your brand. The wizard guides you through:
@@ -665,7 +667,7 @@ Complete workflow from setup to publication:
665
667
 
666
668
  ```bash
667
669
  # 1. First-time setup: Configure brand voice (creates content-rules.md)
668
- /myai-content-rules-setup
670
+ /content-rules-config
669
671
 
670
672
  # 2. Configure publishing destination
671
673
  /configure wordpress
@@ -694,7 +696,7 @@ For publishing multiple pieces of content at once, use the Content Production Co
694
696
 
695
697
  ```bash
696
698
  # Process all content in a queue directory
697
- /content-coordinator ./content-queue/
699
+ /content-production-coordinator ./content-queue/
698
700
  ```
699
701
 
700
702
  **Setting Up a Content Queue:**
@@ -752,7 +754,7 @@ This is the section that gets verified for uniqueness.]
752
754
 
753
755
  3. Run the coordinator:
754
756
  ```bash
755
- /content-coordinator ./content-queue/
757
+ /content-production-coordinator ./content-queue/
756
758
  ```
757
759
 
758
760
  #### Coordinator Features
@@ -760,31 +762,31 @@ This is the section that gets verified for uniqueness.]
760
762
  **Core Operations:**
761
763
  ```bash
762
764
  # Verify only, no publishing
763
- /content-coordinator ./content-queue/ --dry-run
765
+ /content-production-coordinator ./content-queue/ --dry-run
764
766
 
765
767
  # Skip confirmations (for automation)
766
- /content-coordinator ./content-queue/ --force
768
+ /content-production-coordinator ./content-queue/ --force
767
769
 
768
770
  # Detailed progress output
769
- /content-coordinator ./content-queue/ --verbose
771
+ /content-production-coordinator ./content-queue/ --verbose
770
772
 
771
773
  # Custom report directory
772
- /content-coordinator ./content-queue/ --output-dir ./reports/
774
+ /content-production-coordinator ./content-queue/ --output-dir ./reports/
773
775
 
774
776
  # Higher concurrency for large batches
775
- /content-coordinator ./content-queue/ --concurrency 5
777
+ /content-production-coordinator ./content-queue/ --concurrency 5
776
778
  ```
777
779
 
778
780
  **Multi-Platform Publishing:**
779
781
  ```bash
780
782
  # Publish to PayloadCMS instead of WordPress
781
- /content-coordinator ./content-queue/ --platform payloadcms
783
+ /content-production-coordinator ./content-queue/ --platform payloadcms
782
784
 
783
785
  # Generate static markdown files
784
- /content-coordinator ./content-queue/ --platform static --output-dir ./published/
786
+ /content-production-coordinator ./content-queue/ --platform static --output-dir ./published/
785
787
 
786
788
  # Publish to Docusaurus documentation site
787
- /content-coordinator ./docs-queue/ --platform docusaurus
789
+ /content-production-coordinator ./docs-queue/ --platform docusaurus
788
790
  ```
789
791
 
790
792
  Each content item can override the default platform using the `target_platform` front matter field.
@@ -792,30 +794,26 @@ Each content item can override the default platform using the `target_platform`
792
794
  **Content Rules Integration:**
793
795
  ```bash
794
796
  # Use custom content rules file
795
- /content-coordinator ./content-queue/ --content-rules ./brand/content-rules.md
797
+ /content-production-coordinator ./content-queue/ --content-rules ./brand/content-rules.md
796
798
 
797
799
  # Content rules are auto-detected from standard locations
798
- /content-coordinator ./content-queue/ # Finds ./content-rules.md automatically
800
+ /content-production-coordinator ./content-queue/ # Finds ./content-rules.md automatically
799
801
  ```
800
802
 
801
- The coordinator automatically searches for `content-rules.md` in these locations:
802
- 1. Explicit path via `--content-rules`
803
- 2. `./content-rules.md` (current directory)
804
- 3. `./.claude/content-rules.md`
805
- 4. `./docs/content-rules.md`
803
+ The coordinator reads `content-rules.md` from the project root (alongside `brand-config.json`, `product-config.json`, `service-config.json`, and `content-themes.json`).
806
804
 
807
805
  **CI/CD Integration with Webhooks:**
808
806
  ```bash
809
807
  # Send notifications to CI/CD webhook
810
- /content-coordinator ./content-queue/ --webhook-url https://ci.example.com/webhook
808
+ /content-production-coordinator ./content-queue/ --webhook-url https://ci.example.com/webhook
811
809
 
812
810
  # Notify on all events including each published item
813
- /content-coordinator ./content-queue/ \
811
+ /content-production-coordinator ./content-queue/ \
814
812
  --webhook-url https://ci.example.com/webhook \
815
813
  --webhook-events start,complete,error,published
816
814
 
817
815
  # Full CI/CD pipeline integration
818
- /content-coordinator ./content-queue/ --force --webhook-url $CI_WEBHOOK_URL --analytics
816
+ /content-production-coordinator ./content-queue/ --force --webhook-url $CI_WEBHOOK_URL --analytics
819
817
  ```
820
818
 
821
819
  **Available Webhook Events:**
@@ -827,25 +825,25 @@ The coordinator automatically searches for `content-rules.md` in these locations
827
825
  **Queue Management:**
828
826
  ```bash
829
827
  # Add new content to queue without processing
830
- /content-coordinator ./new-articles/ --add-to-queue
828
+ /content-production-coordinator ./new-articles/ --add-to-queue
831
829
 
832
830
  # View queue statistics
833
- /content-coordinator --queue-stats
831
+ /content-production-coordinator --queue-stats
834
832
 
835
833
  # Process items from queue
836
- /content-coordinator --queue .content-queue.json
834
+ /content-production-coordinator --queue .content-queue.json
837
835
 
838
836
  # Clear completed items from queue
839
- /content-coordinator --clear-completed
837
+ /content-production-coordinator --clear-completed
840
838
  ```
841
839
 
842
840
  **Analytics & Monitoring:**
843
841
  ```bash
844
842
  # Enable detailed analytics (default)
845
- /content-coordinator ./content-queue/ --analytics
843
+ /content-production-coordinator ./content-queue/ --analytics
846
844
 
847
845
  # Full production run with all options
848
- /content-coordinator ./content-queue/ \
846
+ /content-production-coordinator ./content-queue/ \
849
847
  --verbose \
850
848
  --output-dir ./reports/ \
851
849
  --concurrency 3 \
@@ -867,25 +865,25 @@ When analytics are enabled, generates `analytics-YYYY-MM-DD-HH-MM-SS.md` with:
867
865
 
868
866
  ```bash
869
867
  # Generate crontab entry with instructions
870
- /content-coordinator --generate-crontab
868
+ /content-production-coordinator --generate-crontab
871
869
 
872
870
  # Run in cron-safe mode (for crontab)
873
- /content-coordinator ./content-queue/ --cron --force
871
+ /content-production-coordinator ./content-queue/ --cron --force
874
872
 
875
873
  # With minimum 6-hour interval between runs
876
- /content-coordinator ./content-queue/ --cron --min-interval 21600
874
+ /content-production-coordinator ./content-queue/ --cron --min-interval 21600
877
875
  ```
878
876
 
879
877
  **Example crontab entries:**
880
878
  ```bash
881
879
  # Every 6 hours
882
- 0 */6 * * * cd /path/to/project && npx myaidev-method coordinate-content ./content-queue --cron --force >> /var/log/content-coordinator.log 2>&1
880
+ 0 */6 * * * cd /path/to/project && npx myaidev-method coordinate-content ./content-queue --cron --force >> /var/log/content-production-coordinator.log 2>&1
883
881
 
884
882
  # Daily at 9 AM
885
- 0 9 * * * cd /path/to/project && npx myaidev-method coordinate-content ./content-queue --cron --force >> /var/log/content-coordinator.log 2>&1
883
+ 0 9 * * * cd /path/to/project && npx myaidev-method coordinate-content ./content-queue --cron --force >> /var/log/content-production-coordinator.log 2>&1
886
884
 
887
885
  # Weekdays at 9 AM and 5 PM
888
- 0 9,17 * * 1-5 cd /path/to/project && npx myaidev-method coordinate-content ./content-queue --cron --force >> /var/log/content-coordinator.log 2>&1
886
+ 0 9,17 * * 1-5 cd /path/to/project && npx myaidev-method coordinate-content ./content-queue --cron --force >> /var/log/content-production-coordinator.log 2>&1
889
887
  ```
890
888
 
891
889
  **WordPress Post Scheduling:**
@@ -894,13 +892,13 @@ Schedule posts for future publication using WordPress's native scheduling:
894
892
 
895
893
  ```bash
896
894
  # Delay all posts by 24 hours
897
- /content-coordinator ./content-queue/ --schedule-delay 24
895
+ /content-production-coordinator ./content-queue/ --schedule-delay 24
898
896
 
899
897
  # Spread posts 6 hours apart
900
- /content-coordinator ./content-queue/ --spread-interval 6
898
+ /content-production-coordinator ./content-queue/ --spread-interval 6
901
899
 
902
900
  # Combined: Start in 24 hours, then every 6 hours
903
- /content-coordinator ./content-queue/ --schedule-delay 24 --spread-interval 6
901
+ /content-production-coordinator ./content-queue/ --schedule-delay 24 --spread-interval 6
904
902
  ```
905
903
 
906
904
  You can also set specific publish dates in content front matter:
@@ -914,7 +912,7 @@ publish_date: "2026-02-15T09:00:00Z" # Schedule for specific date
914
912
  **Combined Automation (Cron + WordPress Scheduling):**
915
913
  ```bash
916
914
  # Run via cron, but schedule posts for optimal times
917
- /content-coordinator ./content-queue/ \
915
+ /content-production-coordinator ./content-queue/ \
918
916
  --cron \
919
917
  --force \
920
918
  --schedule-delay 24 \
@@ -924,7 +922,7 @@ publish_date: "2026-02-15T09:00:00Z" # Schedule for specific date
924
922
 
925
923
  #### Checkpoint/Resume
926
924
 
927
- The coordinator automatically saves progress to `.content-coordinator-state.json`. If interrupted:
925
+ The coordinator automatically saves progress to `.content-production-coordinator-state.json`. If interrupted:
928
926
  - Re-run the same command to resume from the last checkpoint
929
927
  - Use `--fresh` to start over and ignore saved state
930
928
  - Delete the state file manually to reset
@@ -1606,10 +1604,9 @@ cat .claude/skills/content-writer/SKILL.md
1606
1604
 
1607
1605
  ```bash
1608
1606
  # Use interactive setup (recommended)
1609
- /myai-content-rules-setup
1610
-
1607
+ /content-rules-config
1611
1608
  # Or copy the example template
1612
- cp content-rules.example.md content-rules.md
1609
+ cp content-rules-example.md content-rules.md
1613
1610
  ```
1614
1611
 
1615
1612
  **Why use content-rules.md?**
@@ -1953,7 +1950,7 @@ Content rules provide a **non-destructive** way to customize all content generat
1953
1950
  Use the content rules setup command for guided configuration:
1954
1951
 
1955
1952
  ```bash
1956
- /myai-content-rules-setup
1953
+ /content-rules-config
1957
1954
  ```
1958
1955
 
1959
1956
  This interactive wizard will guide you through:
@@ -1968,7 +1965,7 @@ This interactive wizard will guide you through:
1968
1965
 
1969
1966
  ```bash
1970
1967
  # Copy the example template
1971
- cp content-rules.example.md content-rules.md
1968
+ cp content-rules-example.md content-rules.md
1972
1969
 
1973
1970
  # Edit with your preferences
1974
1971
  nano content-rules.md
@@ -2106,7 +2103,7 @@ When you run any content generation command, the system:
2106
2103
 
2107
2104
  ### Best Practices for Content Rules
2108
2105
 
2109
- 1. **Start with the template**: Use `/myai-content-rules-setup` to generate a starting point
2106
+ 1. **Start with the template**: Use `/myai-content-rules-config` to generate a starting point
2110
2107
  2. **Iterate gradually**: Start with basic rules, refine based on output
2111
2108
  3. **Be specific**: Vague rules produce inconsistent results
2112
2109
  4. **Include examples**: Show what good vs. bad looks like
@@ -2708,218 +2705,273 @@ DEBUG=false
2708
2705
  - Test skill behavior
2709
2706
  - Validate output quality
2710
2707
 
2711
- ## 🎁 Contributing Skills to the Marketplace
2708
+ ## 🔬 Skill Builder (Eval-Driven Development)
2712
2709
 
2713
- Share your custom skills with the community as **Tier 1 open source skills** through the MyAIDev marketplace. All contributed skills go through a stringent review process — automated CI validation checks for security vulnerabilities, destructive commands, and malware patterns, followed by manual maintainer review for quality and usefulness. The contribution system uses a PR-based review workflow.
2710
+ The **Skill Builder** (`/myai-skill-builder`) is the recommended tool for creating production-quality skills. It uses eval-driven development — automated testing with parallel A/B comparison, quantitative benchmarking, and iterative refinement to ensure skills meet marketplace standards before submission.
2714
2711
 
2715
- ### Quick Start: Skill Contributor Skill
2712
+ ### Actions
2716
2713
 
2717
- The easiest way to create and submit skills is using the `/skill-contributor` skill inside Claude Code:
2714
+ | Action | Command | Description |
2715
+ |--------|---------|-------------|
2716
+ | **Create** | `/myai-skill-builder create` | Full guided workflow: concept → SKILL.md → evals → testing → publishing |
2717
+ | **Refine** | `/myai-skill-builder refine <path>` | Analyze and improve an existing skill with marketplace quality scoring |
2718
+ | **Test** | `/myai-skill-builder test <path>` | Run automated evals with parallel A/B testing |
2719
+ | **Benchmark** | `/myai-skill-builder benchmark <path>` | Multi-run statistical analysis with variance measurement |
2720
+ | **Validate** | `/myai-skill-builder validate <path>` | Quick structural and security validation |
2721
+ | **Publish** | `/myai-skill-builder publish <path>` | Validate, score, and submit to the marketplace |
2718
2722
 
2719
- ```bash
2720
- # Interactive menu - choose what you want to do
2721
- /skill-contributor
2723
+ ### Create Workflow (11 Phases)
2722
2724
 
2723
- # Create a new skill with guided prompts
2724
- /skill-contributor create
2725
+ The `create` action walks through a complete skill development lifecycle:
2725
2726
 
2726
- # Validate and submit an existing skill via PR
2727
- /skill-contributor submit
2727
+ 1. **Concept Discovery** Interactive questions about purpose, target user, and tools needed
2728
+ 2. **Identity** — Name, description (with "when" clause), and category
2729
+ 3. **Generate SKILL.md** — Creates from a template tier (basic / intermediate / advanced with sub-agents)
2730
+ 4. **Progressive Disclosure** — Moves long reference docs to `references/`, deterministic tasks to `scripts/`, templates to `assets/`
2731
+ 5. **Iterative Refinement** — Self-review against quality checklist, user feedback loop
2732
+ 6. **Validate** — Runs `npx myaidev-method addon validate` to check structure and security
2733
+ 7. **Write Evals** — Generates automated test cases in `evals/evals.json` with assertions
2734
+ 8. **Run & Grade** — Parallel A/B testing (with-skill vs baseline) with grader subagent scoring
2735
+ 9. **Iterate** — Improve skill based on eval results, rerun to verify
2736
+ 10. **Description Optimization** — Train/test split on trigger queries, optimize for activation accuracy
2737
+ 11. **Publish Decision** — Submit to marketplace with eval evidence
2728
2738
 
2729
- # Check the status of your submissions
2730
- /skill-contributor status
2731
- ```
2739
+ ### Eval-Driven Testing
2732
2740
 
2733
- ### Complete Contribution Workflow
2741
+ The skill builder runs automated evaluations to measure whether a skill actually helps:
2734
2742
 
2735
- #### Step 1: Authenticate
2743
+ ```
2744
+ {slug}-workspace/
2745
+ ├── iteration-1/
2746
+ │ ├── eval-basic-create/
2747
+ │ │ ├── with_skill/ # Outputs from skill-assisted run
2748
+ │ │ │ ├── outputs/
2749
+ │ │ │ ├── timing.json # {total_tokens, duration_ms}
2750
+ │ │ │ └── grading.json # Per-assertion PASS/FAIL with evidence
2751
+ │ │ └── without_skill/ # Outputs from vanilla Claude baseline
2752
+ │ │ ├── outputs/
2753
+ │ │ ├── timing.json
2754
+ │ │ └── grading.json
2755
+ │ └── benchmark.json # Aggregated pass rates, token usage, timing
2756
+ ├── iteration-2/
2757
+ │ └── ...
2758
+ ├── benchmark.json # Multi-run statistical results
2759
+ └── feedback.json # User feedback from HTML reviewer
2760
+ ```
2736
2761
 
2737
- Before submitting skills, authenticate with the marketplace:
2762
+ For each eval, **two parallel agents** are spawned: one with the skill loaded and one without (baseline). A **grader subagent** then evaluates each output against assertions and provides PASS/FAIL verdicts with cited evidence.
2763
+
2764
+ The `benchmark` action runs 3+ iterations to measure variance:
2738
2765
 
2739
2766
  ```bash
2740
- npx myaidev-method login
2767
+ /myai-skill-builder benchmark .claude/skills/my-skill
2741
2768
  ```
2742
2769
 
2743
- This opens your browser for OAuth authentication and stores a token locally.
2770
+ This produces statistical analysis (mean, standard deviation) for pass rates, token usage, and timing, plus identifies flaky evals and non-discriminating assertions via an **analyzer subagent**.
2744
2771
 
2745
- #### Step 2: Create Your Skill
2772
+ ### Supporting Files
2746
2773
 
2747
- **Option A: Using the skill-contributor skill (Recommended)**
2774
+ Skills can include supporting directories that travel with the skill through the entire pipeline (submit → store → download → install):
2748
2775
 
2749
- ```bash
2750
- /skill-contributor create
2751
- ```
2776
+ | Directory | Purpose | Examples |
2777
+ |-----------|---------|---------|
2778
+ | `agents/` | Sub-agent definitions for Task tool delegation | `grader-agent.md`, `analyzer-agent.md` |
2779
+ | `scripts/` | Deterministic computation, validation, aggregation | `validate.py`, `aggregate_benchmark.py` |
2780
+ | `references/` | API schemas, long guides, documentation | `schemas.md`, `api-spec.md` |
2781
+ | `assets/` | Templates, HTML files, starter configs | `eval_review.html`, `template.json` |
2782
+ | `evals/` | Automated test definitions | `evals.json` |
2752
2783
 
2753
- You'll be prompted for:
2754
- - **Name**: lowercase with hyphens (e.g., `code-formatter`)
2755
- - **Description**: Include a "when to use" clause
2756
- - **Category**: development, content, publishing, security, deployment, infrastructure, or other
2757
- - **Complexity**: basic, intermediate, or advanced (determines template)
2784
+ When the skill builder detects that agents repeatedly write the same helper code during evals, it extracts the pattern into a bundled script in `scripts/`.
2758
2785
 
2759
- **Option B: Manual creation**
2786
+ ### Marketplace Quality Scoring
2760
2787
 
2761
- Create a directory and SKILL.md file:
2788
+ Skills are scored on 5 categories (0-100 each) both by the skill builder (locally) and by the marketplace's AI analysis (server-side):
2789
+
2790
+ | Category | Weight | What It Measures |
2791
+ |----------|--------|-----------------|
2792
+ | **Quality** | 25% | Instruction clarity, structure, formatting, agent persona specificity |
2793
+ | **Security** | 20% | No credentials, no destructive commands, safe patterns |
2794
+ | **Completeness** | 25% | All sections filled, error handling, edge cases, Quick Start |
2795
+ | **Originality** | 15% | Distinct problem, genuine value, not a trivial wrapper |
2796
+ | **Usefulness** | 15% | Real user need, well-scoped, time savings vs manual approach |
2797
+
2798
+ Preview your score before submitting:
2762
2799
 
2763
2800
  ```bash
2764
- mkdir -p .claude/skills/my-skill
2801
+ /myai-skill-builder refine .claude/skills/my-skill
2765
2802
  ```
2766
2803
 
2767
- Create `.claude/skills/my-skill/SKILL.md`:
2804
+ ### Skill Writing Best Practices
2805
+
2806
+ - **Progressive disclosure**: Keep SKILL.md under 500 lines. Move reference material to `references/`, deterministic tasks to `scripts/`.
2807
+ - **Explain the why**: Don't just say "do X" — explain why X matters so the agent can adapt when circumstances change.
2808
+ - **Avoid heavy-handed MUSTs**: Describe goals and constraints. The agent is intelligent — let it figure out how.
2809
+ - **Include "when" contexts in description**: "Use when building X, debugging Y, or preparing Z." Undertriggering is worse than overtriggering.
2810
+ - **Keep it lean**: After each iteration, remove instructions that don't contribute to passing evals.
2768
2811
 
2769
- ```markdown
2770
- ---
2771
- name: my-skill
2772
- description: "Does X when you need to Y. Use when working with Z."
2773
- argument-hint: "[action] [args]"
2774
- allowed-tools: [Read, Write, Edit, Bash, Glob, Grep, AskUserQuestion]
2775
- context: fork
2776
2812
  ---
2777
2813
 
2778
- # My Skill
2814
+ ## 🎁 Contributing Skills to the Marketplace
2779
2815
 
2780
- You are a **My Skill** agent description of what you do.
2816
+ The MyAIDev Marketplace at `dev.myai1.ai` hosts community and first-party skills. All contributed skills go through automated AI analysis and admin review before being published.
2781
2817
 
2782
- ## Quick Start
2818
+ ### Submission Pipeline
2783
2819
 
2784
2820
  ```
2785
- /my-skill action args
2821
+ Author creates skill
2822
+ → CLI validates locally (structure, security, frontmatter)
2823
+ → CLI bundles SKILL.md + supporting files
2824
+ → API validates bundle (path traversal, size limits, content scanning)
2825
+ → AI analysis scores the skill (quality, security, completeness, originality, usefulness)
2826
+ → Admin reviews and approves/rejects
2827
+ → Published and installable via `addon install`
2786
2828
  ```
2787
2829
 
2788
- ## Arguments
2830
+ ### Quick Start
2789
2831
 
2790
- Parse action from: `$ARGUMENTS`
2832
+ **Option A: Using the Skill Builder (Recommended)**
2791
2833
 
2792
- ### Actions
2793
-
2794
- #### `action1` - Primary Action
2834
+ ```bash
2835
+ # Full guided workflow with eval testing
2836
+ /myai-skill-builder create
2795
2837
 
2796
- **Workflow:**
2797
- 1. Step one
2798
- 2. Step two
2799
- 3. Step three
2838
+ # Or submit an existing skill
2839
+ /myai-skill-builder publish .claude/skills/my-skill
2840
+ ```
2800
2841
 
2801
- ## Error Handling
2842
+ **Option B: Using the Skill Contributor**
2802
2843
 
2803
- - Explain errors and suggest alternatives
2804
- - Never silently skip steps
2844
+ ```bash
2845
+ # Quick scaffolding and submission
2846
+ /skill-contributor create
2847
+ /skill-contributor submit
2805
2848
  ```
2806
2849
 
2807
- #### Step 3: Validate Your Skill
2808
-
2809
- Run validation to check for issues before submitting:
2850
+ **Option C: CLI directly**
2810
2851
 
2811
2852
  ```bash
2812
- # Via CLI
2853
+ npx myaidev-method login
2813
2854
  npx myaidev-method addon validate --dir .claude/skills/my-skill
2855
+ npx myaidev-method addon submit --dir .claude/skills/my-skill
2856
+ npx myaidev-method addon status
2857
+ ```
2814
2858
 
2815
- # Via skill-contributor
2816
- /skill-contributor submit
2817
- # (validates automatically before submission)
2859
+ ### What Gets Submitted
2860
+
2861
+ When you submit, the CLI automatically detects and bundles supporting files alongside the SKILL.md:
2862
+
2863
+ ```
2864
+ my-skill/
2865
+ ├── SKILL.md ← submitted as content
2866
+ ├── agents/grader-agent.md ← bundled as supportingFiles
2867
+ ├── scripts/validate.py ← bundled as supportingFiles
2868
+ ├── references/schemas.md ← bundled as supportingFiles
2869
+ └── evals/evals.json ← bundled as supportingFiles
2818
2870
  ```
2819
2871
 
2820
- **Validation checks:**
2821
- - ✅ Required frontmatter fields (`name`, `description`, `allowed-tools`)
2822
- - ✅ Content structure (H1 heading, sections)
2823
- - ✅ Security patterns (no credential paths, no destructive commands)
2824
- - ✅ File size limits (50KB max per file, 200KB max total)
2825
- - ✅ Name availability (not already taken in marketplace)
2872
+ The CLI skips `__pycache__`, `.pyc`, `.DS_Store`, and `node_modules` when collecting files.
2826
2873
 
2827
- #### Step 4: Submit for Review
2874
+ ### Bundle Validation (Server-Side)
2828
2875
 
2829
- ```bash
2830
- # Via CLI
2831
- npx myaidev-method addon submit --dir .claude/skills/my-skill
2876
+ The server validates all submitted bundles:
2832
2877
 
2833
- # Via skill-contributor
2834
- /skill-contributor submit
2835
- ```
2878
+ - **Path traversal prevention** — No `../` or absolute paths
2879
+ - **Allowed directories only** — Files must be in `agents/`, `scripts/`, `references/`, `assets/`, or `evals/`
2880
+ - **Blocked file types** — No `.exe`, `.dll`, `.sh`, `.bat`, `.env`, `.pem`, `.key`
2881
+ - **Size limits** — Max 500KB total for supporting files, max 50 files
2882
+ - **Content scanning** — All files scanned for dangerous patterns (credential references, destructive commands, remote code execution)
2883
+
2884
+ ### Submission Statuses
2836
2885
 
2837
- This will:
2838
- 1. Fork the [myaidev-marketplace](https://github.com/myaione/myaidev-marketplace) repo
2839
- 2. Create a branch with your skill
2840
- 3. Open a PR for review
2841
- 4. Record your submission in the system
2886
+ | Status | Meaning |
2887
+ |--------|---------|
2888
+ | `pending_review` | Submitted, waiting for processing |
2889
+ | `ai_analyzing` | AI is scoring the skill automatically |
2890
+ | `ready_for_review` | AI analysis complete, waiting for admin review |
2891
+ | `approved` | Published to the marketplace |
2892
+ | `rejected` | Did not meet quality standards |
2893
+ | `changes_requested` | Admin requested modifications |
2842
2894
 
2843
- #### Step 5: Track Your Submission
2895
+ Track your submissions:
2844
2896
 
2845
2897
  ```bash
2846
- # Check all your submissions
2898
+ # All submissions
2847
2899
  npx myaidev-method addon status
2848
2900
 
2849
- # Or via skill-contributor
2850
- /skill-contributor status
2901
+ # Specific submission by ID
2902
+ npx myaidev-method addon status <submission-id>
2851
2903
  ```
2852
2904
 
2853
- ### Skill Quality Guidelines
2905
+ ### Installing Bundled Skills
2854
2906
 
2855
- #### Frontmatter Requirements
2907
+ When a skill has supporting files, the CLI downloads everything as a JSON bundle and recreates the full directory structure:
2856
2908
 
2857
- | Field | Required | Description |
2858
- |-------|----------|-------------|
2859
- | `name` | ✅ | Lowercase, hyphens only, max 64 chars |
2860
- | `description` | ✅ | Must include "when" clause explaining usage |
2861
- | `allowed-tools` | ✅ | Array of tools the skill needs |
2862
- | `argument-hint` | Recommended | Shows usage pattern in help |
2863
- | `context` | Recommended | Usually `fork` for isolation |
2909
+ ```bash
2910
+ npx myaidev-method addon install my-skill
2911
+ ```
2864
2912
 
2865
- #### Content Structure
2913
+ Creates:
2866
2914
 
2867
- - **H1 heading**: Must match skill purpose
2868
- - **Quick Start**: Show basic usage example
2869
- - **Arguments section**: Document all arguments and actions
2870
- - **Workflow steps**: Clear numbered steps for each action
2871
- - **Error handling**: Guidance for edge cases
2915
+ ```
2916
+ .claude/skills/my-skill/
2917
+ ├── SKILL.md
2918
+ ├── agents/
2919
+ │ └── grader-agent.md
2920
+ ├── scripts/
2921
+ │ └── validate.py
2922
+ └── references/
2923
+ └── schemas.md
2924
+ ```
2925
+
2926
+ Skills without supporting files are downloaded as plain markdown for backward compatibility.
2872
2927
 
2873
- #### Security Rules
2928
+ ### Skill Requirements
2874
2929
 
2875
- Your skill must NOT contain:
2876
- - ❌ Credential paths (`~/.ssh`, `~/.aws`, etc.)
2877
- - ❌ Destructive commands (`rm -rf /`, `chmod 777`, etc.)
2878
- - ❌ Dynamic code execution (`eval()`, `exec()`) unless essential
2879
- - ❌ Remote script piping (`curl | bash`)
2930
+ #### Frontmatter
2880
2931
 
2881
- ### Skill Templates
2932
+ | Field | Required | Rules |
2933
+ |-------|----------|-------|
2934
+ | `name` | Yes | Lowercase with hyphens, max 64 chars |
2935
+ | `description` | Yes | Must include "when" clause (e.g., "Use when...") |
2936
+ | `allowed-tools` | Yes | Only tools the skill actually uses |
2937
+ | `argument-hint` | Recommended | Shows expected argument format |
2938
+ | `context` | Recommended | Use `fork` for most skills |
2882
2939
 
2883
- The skill-contributor provides three templates based on complexity:
2940
+ #### Content Structure
2884
2941
 
2885
- | Template | Best For | Features |
2886
- |----------|----------|----------|
2887
- | **Basic** | Simple, single-action skills | Minimal structure, quick to fill |
2888
- | **Intermediate** | Multi-action skills | Multiple actions, quality checks |
2889
- | **Advanced** | Orchestrator skills | Sub-agents via Task tool, configuration |
2942
+ - H1 heading matching skill purpose
2943
+ - Quick Start section with concrete usage example
2944
+ - Arguments section documenting `$ARGUMENTS` parsing
2945
+ - Workflow with numbered steps for each action
2946
+ - Error handling with realistic scenarios
2947
+ - SKILL.md body under 500 lines
2890
2948
 
2891
- ### After Submission
2949
+ #### Security Rules (violations block submission)
2892
2950
 
2893
- 1. **CI Validation**: GitHub Actions validates your SKILL.md automatically
2894
- 2. **Review**: Maintainers review for quality, security, and usefulness
2895
- 3. **Feedback**: You may receive comments requesting changes
2896
- 4. **Merge**: Once approved, your skill is merged and ingested into the marketplace
2897
- 5. **Published**: Users can install via `npx myaidev-method addon install your-skill`
2951
+ - No credential paths (`~/.ssh`, `~/.aws`, `/etc/shadow`)
2952
+ - No destructive commands (`rm -rf /`, `chmod 777`, `mkfs`)
2953
+ - No remote script piping (`curl | bash`, `wget | sh`)
2954
+ - Minimal dynamic code evaluation (`eval()`, `exec()`)
2898
2955
 
2899
- ### Example: Creating a Simple Skill
2956
+ ### Example: End-to-End Skill Creation
2900
2957
 
2901
2958
  ```bash
2902
- # 1. Login
2959
+ # 1. Authenticate
2903
2960
  npx myaidev-method login
2904
2961
 
2905
- # 2. Create skill interactively
2906
- /skill-contributor create
2907
- # Enter: name=code-formatter, description="Formats code files. Use when cleaning up code style.", category=development, complexity=basic
2908
-
2909
- # 3. Edit the generated SKILL.md with your logic
2910
- code .claude/skills/code-formatter/SKILL.md
2911
-
2912
- # 4. Test locally by invoking it
2913
- /code-formatter "src/**/*.js"
2962
+ # 2. Build skill with eval-driven testing
2963
+ /myai-skill-builder create
2964
+ # Concept discovery questions
2965
+ # → Generates SKILL.md + supporting files
2966
+ # Writes evals, runs A/B tests
2967
+ # → Optimizes description for trigger accuracy
2968
+ # → Submits to marketplace with eval evidence
2914
2969
 
2915
- # 5. Validate
2916
- npx myaidev-method addon validate --dir .claude/skills/code-formatter
2917
-
2918
- # 6. Submit
2919
- /skill-contributor submit
2970
+ # 3. Check status
2971
+ npx myaidev-method addon status
2920
2972
 
2921
- # 7. Check status
2922
- /skill-contributor status
2973
+ # 4. After approval, others can install
2974
+ npx myaidev-method addon install your-skill-name
2923
2975
  ```
2924
2976
 
2925
2977
  ## 📚 Additional Resources
@@ -2950,4 +3002,4 @@ If you encounter issues:
2950
3002
 
2951
3003
  **Happy customizing!** 🚀
2952
3004
 
2953
- Remember: The power of MyAIDev Method lies in its customizability. Don't hesitate to experiment with different skill configurations to find what works best for your workflow.
3005
+ Remember: The power of MyAIDev Method lies in its customizability. Don't hesitate to experiment with different skill configurations to find what works best for your workflow.