@exaudeus/workrail 3.27.0 → 3.29.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (160) hide show
  1. package/dist/console/assets/{index-FtTaDku8.js → index-BZ6HkxGf.js} +1 -1
  2. package/dist/console/index.html +1 -1
  3. package/dist/manifest.json +3 -3
  4. package/docs/README.md +57 -0
  5. package/docs/adrs/001-hybrid-storage-backend.md +38 -0
  6. package/docs/adrs/002-four-layer-context-classification.md +38 -0
  7. package/docs/adrs/003-checkpoint-trigger-strategy.md +35 -0
  8. package/docs/adrs/004-opt-in-encryption-strategy.md +36 -0
  9. package/docs/adrs/005-agent-first-workflow-execution-tokens.md +105 -0
  10. package/docs/adrs/006-append-only-session-run-event-log.md +76 -0
  11. package/docs/adrs/007-resume-and-checkpoint-only-sessions.md +51 -0
  12. package/docs/adrs/008-blocked-nodes-architectural-upgrade.md +178 -0
  13. package/docs/adrs/009-bridge-mode-single-instance-mcp.md +195 -0
  14. package/docs/adrs/010-release-pipeline.md +89 -0
  15. package/docs/architecture/README.md +7 -0
  16. package/docs/architecture/refactor-audit.md +364 -0
  17. package/docs/authoring-v2.md +527 -0
  18. package/docs/authoring.md +873 -0
  19. package/docs/changelog-recent.md +201 -0
  20. package/docs/configuration.md +505 -0
  21. package/docs/ctc-mcp-proposal.md +518 -0
  22. package/docs/design/README.md +22 -0
  23. package/docs/design/agent-cascade-protocol.md +96 -0
  24. package/docs/design/autonomous-console-design-candidates.md +253 -0
  25. package/docs/design/autonomous-console-design-review.md +111 -0
  26. package/docs/design/autonomous-platform-mvp-discovery.md +525 -0
  27. package/docs/design/claude-code-source-deep-dive.md +713 -0
  28. package/docs/design/console-cyberpunk-ui-discovery.md +504 -0
  29. package/docs/design/console-execution-trace-candidates-final.md +160 -0
  30. package/docs/design/console-execution-trace-candidates.md +211 -0
  31. package/docs/design/console-execution-trace-design-candidates-v2.md +113 -0
  32. package/docs/design/console-execution-trace-design-review.md +74 -0
  33. package/docs/design/console-execution-trace-discovery.md +394 -0
  34. package/docs/design/console-execution-trace-final-review.md +77 -0
  35. package/docs/design/console-execution-trace-review.md +92 -0
  36. package/docs/design/console-performance-discovery.md +415 -0
  37. package/docs/design/console-ui-backlog.md +280 -0
  38. package/docs/design/daemon-architecture-discovery.md +853 -0
  39. package/docs/design/daemon-design-candidates.md +318 -0
  40. package/docs/design/daemon-design-review-findings.md +119 -0
  41. package/docs/design/daemon-engine-design-candidates.md +210 -0
  42. package/docs/design/daemon-engine-design-review.md +131 -0
  43. package/docs/design/daemon-execution-engine-discovery.md +280 -0
  44. package/docs/design/daemon-gap-analysis.md +554 -0
  45. package/docs/design/daemon-owns-console-plan.md +168 -0
  46. package/docs/design/daemon-owns-console-review.md +91 -0
  47. package/docs/design/daemon-owns-console.md +195 -0
  48. package/docs/design/data-model-erd.md +11 -0
  49. package/docs/design/design-candidates-consolidate-dev-staleness.md +98 -0
  50. package/docs/design/design-candidates-walk-cache-depth-limit.md +80 -0
  51. package/docs/design/design-review-consolidate-dev-staleness.md +54 -0
  52. package/docs/design/design-review-walk-cache-depth-limit.md +48 -0
  53. package/docs/design/implementation-plan-consolidate-dev-staleness.md +142 -0
  54. package/docs/design/implementation-plan-walk-cache-depth-limit.md +141 -0
  55. package/docs/design/layer3b-ghost-nodes-design-candidates.md +229 -0
  56. package/docs/design/layer3b-ghost-nodes-design-review.md +93 -0
  57. package/docs/design/layer3b-ghost-nodes-implementation-plan.md +219 -0
  58. package/docs/design/list-workflows-latency-fix-plan.md +128 -0
  59. package/docs/design/list-workflows-latency-fix-review.md +55 -0
  60. package/docs/design/list-workflows-latency-fix.md +109 -0
  61. package/docs/design/native-context-management-api.md +11 -0
  62. package/docs/design/performance-sweep-2026-04.md +96 -0
  63. package/docs/design/routines-guide.md +219 -0
  64. package/docs/design/sequence-diagrams.md +11 -0
  65. package/docs/design/subagent-design-principles.md +220 -0
  66. package/docs/design/temporal-patterns-design-candidates.md +312 -0
  67. package/docs/design/temporal-patterns-design-review-findings.md +163 -0
  68. package/docs/design/test-isolation-from-config-file.md +335 -0
  69. package/docs/design/v2-core-design-locks.md +2746 -0
  70. package/docs/design/v2-lock-registry.json +734 -0
  71. package/docs/design/workflow-authoring-v2.md +1044 -0
  72. package/docs/design/workflow-docs-spec.md +218 -0
  73. package/docs/design/workflow-extension-points.md +687 -0
  74. package/docs/design/workrail-auto-trigger-system.md +359 -0
  75. package/docs/design/workrail-config-file-discovery.md +513 -0
  76. package/docs/docker.md +110 -0
  77. package/docs/generated/v2-lock-closure-plan.md +26 -0
  78. package/docs/generated/v2-lock-coverage.json +797 -0
  79. package/docs/generated/v2-lock-coverage.md +177 -0
  80. package/docs/ideas/backlog.md +3927 -0
  81. package/docs/ideas/design-candidates-mcp-resilience.md +208 -0
  82. package/docs/ideas/design-review-findings-mcp-resilience.md +119 -0
  83. package/docs/ideas/implementation_plan.md +249 -0
  84. package/docs/ideas/third-party-workflow-setup-design-thinking.md +1948 -0
  85. package/docs/implementation/02-architecture.md +316 -0
  86. package/docs/implementation/04-testing-strategy.md +124 -0
  87. package/docs/implementation/09-simple-workflow-guide.md +835 -0
  88. package/docs/implementation/13-advanced-validation-guide.md +874 -0
  89. package/docs/implementation/README.md +21 -0
  90. package/docs/integrations/claude-code.md +300 -0
  91. package/docs/integrations/firebender.md +315 -0
  92. package/docs/migration/v0.1.0.md +147 -0
  93. package/docs/naming-conventions.md +45 -0
  94. package/docs/planning/README.md +104 -0
  95. package/docs/planning/github-ticketing-playbook.md +195 -0
  96. package/docs/plans/README.md +24 -0
  97. package/docs/plans/agent-managed-ticketing-design.md +605 -0
  98. package/docs/plans/agentic-orchestration-roadmap.md +112 -0
  99. package/docs/plans/assessment-gates-engine-handoff.md +536 -0
  100. package/docs/plans/content-coherence-and-references.md +151 -0
  101. package/docs/plans/library-extraction-plan.md +340 -0
  102. package/docs/plans/mr-review-workflow-redesign.md +1451 -0
  103. package/docs/plans/native-context-management-epic.md +11 -0
  104. package/docs/plans/perf-fixes-design-candidates.md +225 -0
  105. package/docs/plans/perf-fixes-design-review-findings.md +61 -0
  106. package/docs/plans/perf-fixes-new-issues-candidates.md +264 -0
  107. package/docs/plans/perf-fixes-new-issues-review.md +110 -0
  108. package/docs/plans/prompt-fragments.md +53 -0
  109. package/docs/plans/ui-ux-workflow-design-candidates.md +120 -0
  110. package/docs/plans/ui-ux-workflow-discovery.md +100 -0
  111. package/docs/plans/ui-ux-workflow-review.md +48 -0
  112. package/docs/plans/v2-followup-enhancements.md +587 -0
  113. package/docs/plans/workflow-categories-candidates.md +105 -0
  114. package/docs/plans/workflow-categories-discovery.md +110 -0
  115. package/docs/plans/workflow-categories-review.md +51 -0
  116. package/docs/plans/workflow-discovery-model-candidates.md +94 -0
  117. package/docs/plans/workflow-discovery-model-discovery.md +74 -0
  118. package/docs/plans/workflow-discovery-model-review.md +48 -0
  119. package/docs/plans/workflow-source-setup-phase-1.md +245 -0
  120. package/docs/plans/workflow-source-setup-phase-2.md +361 -0
  121. package/docs/plans/workflow-staleness-detection-candidates.md +104 -0
  122. package/docs/plans/workflow-staleness-detection-review.md +58 -0
  123. package/docs/plans/workflow-staleness-detection.md +80 -0
  124. package/docs/plans/workflow-v2-design.md +69 -0
  125. package/docs/plans/workflow-v2-roadmap.md +74 -0
  126. package/docs/plans/workflow-validation-design.md +98 -0
  127. package/docs/plans/workflow-validation-roadmap.md +108 -0
  128. package/docs/plans/workrail-platform-vision.md +420 -0
  129. package/docs/reference/agent-context-cleaner-snippet.md +94 -0
  130. package/docs/reference/agent-context-guidance.md +140 -0
  131. package/docs/reference/context-optimization.md +284 -0
  132. package/docs/reference/example-workflow-repository-template/.github/workflows/validate.yml +125 -0
  133. package/docs/reference/example-workflow-repository-template/README.md +268 -0
  134. package/docs/reference/example-workflow-repository-template/workflows/example-workflow.json +80 -0
  135. package/docs/reference/external-workflow-repositories.md +916 -0
  136. package/docs/reference/feature-flags-architecture.md +472 -0
  137. package/docs/reference/feature-flags.md +349 -0
  138. package/docs/reference/god-tier-workflow-validation.md +272 -0
  139. package/docs/reference/loop-optimization.md +209 -0
  140. package/docs/reference/loop-validation.md +176 -0
  141. package/docs/reference/loops.md +465 -0
  142. package/docs/reference/mcp-platform-constraints.md +59 -0
  143. package/docs/reference/recovery.md +88 -0
  144. package/docs/reference/releases.md +177 -0
  145. package/docs/reference/troubleshooting.md +105 -0
  146. package/docs/reference/workflow-execution-contract.md +998 -0
  147. package/docs/roadmap/README.md +22 -0
  148. package/docs/roadmap/legacy-planning-status.md +103 -0
  149. package/docs/roadmap/now-next-later.md +70 -0
  150. package/docs/roadmap/open-work-inventory.md +389 -0
  151. package/docs/tickets/README.md +39 -0
  152. package/docs/tickets/next-up.md +76 -0
  153. package/docs/workflow-management.md +317 -0
  154. package/docs/workflow-templates.md +423 -0
  155. package/docs/workflow-validation.md +184 -0
  156. package/docs/workflows.md +254 -0
  157. package/package.json +3 -1
  158. package/spec/authoring-spec.json +61 -16
  159. package/workflows/workflow-for-workflows.json +252 -93
  160. package/workflows/workflow-for-workflows.v2.json +188 -77
@@ -0,0 +1,916 @@
1
+ # External Workflow Repositories
2
+
3
+ > **Status**: Infrastructure Complete | Integration Pending
4
+ > **Last Updated**: 2025-01-20
5
+
6
+ ## Table of Contents
7
+
8
+ 1. [Executive Summary](#executive-summary)
9
+ 2. [Architecture](#architecture)
10
+ 3. [Approach Comparison](#approach-comparison)
11
+ 4. [Usage Examples](#usage-examples)
12
+ 5. [Creating a Repository](#creating-a-repository)
13
+ 6. [Implementation Guide](#implementation-guide)
14
+ 7. [Security](#security)
15
+ 8. [Troubleshooting](#troubleshooting)
16
+ 9. [Best Practices](#best-practices)
17
+
18
+ ---
19
+
20
+ ## Executive Summary
21
+
22
+ ### Current Status
23
+
24
+ **Infrastructure: COMPLETE** - All code exists and is production-ready
25
+ **Integration: NOT ENABLED** - Not wired into default configuration
26
+ **What's Missing**: < 1 week of integration work
27
+
28
+ ### Key Finding
29
+
30
+ **WorkRail already has complete, tested infrastructure for external workflow repositories.** The system supports:
31
+
32
+ 1. **Git repositories** (GitHub, GitLab, Bitbucket) - `GitWorkflowStorage`
33
+ 2. **HTTP registries** (npm-style) - `RemoteWorkflowStorage`
34
+ 3. **Plugin packages** (npm) - `PluginWorkflowStorage`
35
+ 4. **Security features** (URL validation, path traversal prevention, file size limits)
36
+ 5. **Graceful degradation** (continues if one source fails)
37
+ 6. **Priority-based merging** (later sources override earlier ones)
38
+
39
+ ### Why This Matters
40
+
41
+ WorkRail supports loading workflows from external sources, enabling teams to:
42
+ - Share workflow collections
43
+ - Consume community workflows
44
+ - Maintain centralized workflow repositories
45
+ - Version control workflows with Git
46
+ - Collaborate using pull requests
47
+
48
+ ### Recommended Approach: Git Repositories
49
+
50
+ | Factor | Git | HTTP Registry | Plugins |
51
+ |--------|-----|--------------|---------|
52
+ | Version Control | Built-in | Not included | Via npm |
53
+ | Infrastructure | GitHub/GitLab free | Need server | npm exists |
54
+ | Familiarity | Developers know Git | Custom API | npm familiar |
55
+ | Offline Support | Local cache | Needs network | node_modules |
56
+ | Pull Request Workflow | Native | Custom | npm publish |
57
+ | Already Implemented | Yes | Yes | Yes |
58
+ | Security | Excellent | Good | Good |
59
+ | Setup Complexity | Low | High | Medium |
60
+
61
+ **Git Workflow Benefits:**
62
+ ```
63
+ Developer → Fork Repo → Add Workflow → PR → Review → Merge → Auto-Sync
64
+
65
+ All users get update
66
+ ```
67
+
68
+ - **No infrastructure**: Use GitHub/GitLab (free)
69
+ - **Familiar workflow**: Developers already know Git/PR process
70
+ - **Built-in review**: PRs provide natural approval process
71
+ - **Version control**: Full history, rollback capability
72
+ - **Free hosting**: GitHub/GitLab provide unlimited public repos
73
+
74
+ ---
75
+
76
+ ## Architecture
77
+
78
+ ### Current Implementation
79
+
80
+ ```
81
+ ┌─────────────────────────────────────────────────────────────────┐
82
+ │ IWorkflowStorage Interface │
83
+ └─────────────────────────────────────────────────────────────────┘
84
+
85
+ ┌───────────────┼───────────────┐
86
+ │ │ │
87
+ ┌───────▼──────┐ ┌─────▼─────┐ ┌──────▼──────┐
88
+ │ FileWorkflow │ │GitWorkflow│ │RemoteWorkflow│
89
+ │ Storage │ │ Storage │ │ Storage │
90
+ └──────────────┘ └───────────┘ └──────────────┘
91
+
92
+ ┌───────▼──────────────────────────────────┐
93
+ │ MultiDirectoryWorkflowStorage (current) │
94
+ │ - Bundled workflows │
95
+ │ - User directory (~/.workrail) │
96
+ │ - Project directory (./workflows) │
97
+ │ - Custom paths (env vars) │
98
+ └──────────────────────────────────────────┘
99
+
100
+ ┌─────────────────────────────────────────────────────────────────┐
101
+ │ Decorator Pattern (Currently Used) │
102
+ │ CachingWorkflowStorage │
103
+ │ → SchemaValidatingWorkflowStorage │
104
+ │ → MultiDirectoryWorkflowStorage (base) │
105
+ └─────────────────────────────────────────────────────────────────┘
106
+ ```
107
+
108
+ ### Design Principles
109
+
110
+ The system follows these principles (from the MCP stateless model):
111
+
112
+ 1. **Stateless** - Storage manages state, not the MCP server
113
+ 2. **Composable** - Multiple storage sources can be combined
114
+ 3. **Graceful Degradation** - Failures in one source don't break others
115
+ 4. **Security-First** - URL validation, path traversal prevention, file size limits
116
+ 5. **Decorator Pattern** - Validation and caching are orthogonal concerns
117
+
118
+ ### Proposed: EnhancedMultiSourceWorkflowStorage
119
+
120
+ ```typescript
121
+ export interface MultiSourceWorkflowConfig {
122
+ // Existing local directories
123
+ includeBundled?: boolean;
124
+ includeUser?: boolean;
125
+ includeProject?: boolean;
126
+ customPaths?: string[];
127
+
128
+ // NEW: Git repositories
129
+ gitRepositories?: GitWorkflowConfig[];
130
+
131
+ // NEW: Remote registries
132
+ remoteRegistries?: RemoteWorkflowRegistryConfig[];
133
+
134
+ // NEW: Plugin directories
135
+ pluginPaths?: string[];
136
+ }
137
+
138
+ export class EnhancedMultiSourceWorkflowStorage implements IWorkflowStorage {
139
+ constructor(config: MultiSourceWorkflowConfig = {}) {
140
+ // Priority order (lowest to highest):
141
+ // 1. Bundled workflows
142
+ // 2. User directory
143
+ // 3. Git repositories
144
+ // 4. Remote registries
145
+ // 5. Project directory (highest priority)
146
+ }
147
+ }
148
+ ```
149
+
150
+ ---
151
+
152
+ ## Approach Comparison
153
+
154
+ ### Option 1: Git-Based Repositories (RECOMMENDED)
155
+
156
+ **Use Case**: Teams want to share workflows via GitHub/GitLab
157
+
158
+ **Pros**:
159
+ - Version control built-in
160
+ - Pull request workflow for contributions
161
+ - Already implemented (`GitWorkflowStorage`)
162
+ - Works offline (local cache)
163
+ - Familiar to developers
164
+ - Free hosting (GitHub/GitLab)
165
+ - Automatic sync with configurable intervals
166
+
167
+ **Cons**:
168
+ - Requires Git installed
169
+ - Clone/pull operations add latency
170
+ - Not suitable for high-frequency updates
171
+
172
+ **Best For**:
173
+ - Team workflow repositories
174
+ - Community workflow collections
175
+ - Organization-wide standard workflows
176
+ - Workflows requiring version control and review
177
+
178
+ ### Option 2: HTTP-Based Registries
179
+
180
+ **Use Case**: npm-style workflow registry with REST API
181
+
182
+ **Pros**:
183
+ - Fast (no clone/pull)
184
+ - Already implemented (`RemoteWorkflowStorage`)
185
+ - Supports authentication (API keys)
186
+ - Good for high-frequency updates
187
+ - Retry logic with exponential backoff
188
+
189
+ **Cons**:
190
+ - Requires running a registry server
191
+ - No built-in version control
192
+ - Requires network for every access (unless cached)
193
+
194
+ **Best For**:
195
+ - Large organizations with internal registries
196
+ - High-frequency workflow updates
197
+ - Centralized workflow management systems
198
+ - Integration with existing artifact management
199
+
200
+ ### Option 3: Plugin-Based (npm packages)
201
+
202
+ **Use Case**: Distribute workflows as npm packages
203
+
204
+ **Pros**:
205
+ - Already implemented (`PluginWorkflowStorage`)
206
+ - Uses npm ecosystem
207
+ - Semantic versioning
208
+ - Dependency management
209
+
210
+ **Cons**:
211
+ - Requires npm/node_modules
212
+ - More complex workflow publishing
213
+ - Version lock-in
214
+
215
+ **Best For**:
216
+ - Public workflow distributions
217
+ - Integration with existing npm packages
218
+ - When you need strict dependency management
219
+
220
+ ### Option 4: Hybrid Approach
221
+
222
+ Combine multiple sources with priority ordering:
223
+
224
+ ```
225
+ Priority (highest to lowest):
226
+ 1. Project directory (team overrides)
227
+ 2. Git repository (shared team workflows)
228
+ 3. User directory (personal workflows)
229
+ 4. Community Git repo (public workflows)
230
+ 5. Bundled workflows (defaults)
231
+ ```
232
+
233
+ **Best For**: Most organizations
234
+
235
+ ---
236
+
237
+ ## Usage Examples
238
+
239
+ ### 1. Simple Team Repository
240
+
241
+ **Scenario**: Small team wants to share workflows via GitHub.
242
+
243
+ #### Configuration
244
+
245
+ ```typescript
246
+ // src/container.ts
247
+ import { createEnhancedMultiSourceWorkflowStorage } from './infrastructure/storage/enhanced-multi-source-workflow-storage';
248
+
249
+ export function createDefaultWorkflowStorage(): CachingWorkflowStorage {
250
+ const baseStorage = createEnhancedMultiSourceWorkflowStorage({
251
+ includeBundled: true,
252
+ includeUser: true,
253
+ includeProject: true,
254
+ gitRepositories: [
255
+ {
256
+ repositoryUrl: 'https://github.com/myteam/workflows.git',
257
+ branch: 'main',
258
+ syncInterval: 60, // Sync every hour
259
+ localPath: path.join(os.homedir(), '.workrail', 'team')
260
+ }
261
+ ]
262
+ });
263
+
264
+ const validatingStorage = new SchemaValidatingWorkflowStorage(baseStorage);
265
+ return new CachingWorkflowStorage(validatingStorage, 300_000);
266
+ }
267
+ ```
268
+
269
+ #### Environment Variables
270
+
271
+ ```bash
272
+ # .env
273
+ WORKFLOW_GIT_REPO_URL=https://github.com/myteam/workflows.git
274
+ WORKFLOW_GIT_REPO_BRANCH=main
275
+ WORKFLOW_GIT_SYNC_INTERVAL=60
276
+ ```
277
+
278
+ #### Usage
279
+
280
+ ```bash
281
+ # Initialize WorkRail (will clone the repository)
282
+ workrail init
283
+
284
+ # List workflows (includes team workflows)
285
+ workrail list
286
+
287
+ # Run a team workflow
288
+ workrail run team-code-review
289
+ ```
290
+
291
+ ### 2. Multi-Repository Setup
292
+
293
+ **Scenario**: Organization wants to combine public community workflows with private team workflows.
294
+
295
+ ```typescript
296
+ export function createDefaultWorkflowStorage(): CachingWorkflowStorage {
297
+ const baseStorage = createEnhancedMultiSourceWorkflowStorage({
298
+ includeBundled: true,
299
+ includeUser: true,
300
+ includeProject: true,
301
+ gitRepositories: [
302
+ // Public community workflows (lower priority)
303
+ {
304
+ repositoryUrl: 'https://github.com/workrail/community-workflows.git',
305
+ branch: 'main',
306
+ syncInterval: 1440, // Daily sync
307
+ localPath: path.join(os.homedir(), '.workrail', 'community')
308
+ },
309
+ // Private team workflows (higher priority)
310
+ {
311
+ repositoryUrl: 'https://github.com/mycompany/team-workflows.git',
312
+ branch: 'production',
313
+ syncInterval: 60, // Hourly sync
314
+ authToken: process.env['GITHUB_TOKEN'],
315
+ localPath: path.join(os.homedir(), '.workrail', 'team')
316
+ }
317
+ ]
318
+ });
319
+
320
+ const validatingStorage = new SchemaValidatingWorkflowStorage(baseStorage);
321
+ return new CachingWorkflowStorage(validatingStorage, 300_000);
322
+ }
323
+ ```
324
+
325
+ **Priority Order**: If the same workflow ID exists in multiple sources, the higher priority source wins:
326
+ 1. Bundled workflows (built-in defaults)
327
+ 2. User workflows (`~/.workrail/workflows`)
328
+ 3. Community workflows (GitHub public repo)
329
+ 4. Team workflows (GitHub private repo)
330
+ 5. Project workflows (`./workflows`) - highest priority
331
+
332
+ ### 3. Private Repository with Authentication
333
+
334
+ **Scenario**: Company uses private GitHub repository with authentication.
335
+
336
+ #### Generate GitHub Personal Access Token
337
+
338
+ 1. Go to GitHub → Settings → Developer settings → Personal access tokens
339
+ 2. Generate new token (classic)
340
+ 3. Select scope: `repo` (for private repositories)
341
+ 4. Copy the token
342
+
343
+ #### Configuration
344
+
345
+ ```typescript
346
+ export function createDefaultWorkflowStorage(): CachingWorkflowStorage {
347
+ const baseStorage = createEnhancedMultiSourceWorkflowStorage({
348
+ gitRepositories: [
349
+ {
350
+ repositoryUrl: 'https://github.com/mycompany/private-workflows.git',
351
+ branch: 'production',
352
+ authToken: process.env['GITHUB_TOKEN'], // Read from environment
353
+ syncInterval: 60,
354
+ maxFileSize: 2 * 1024 * 1024, // 2MB limit
355
+ maxFiles: 100,
356
+ localPath: path.join(os.homedir(), '.workrail', 'private')
357
+ }
358
+ ]
359
+ });
360
+
361
+ const validatingStorage = new SchemaValidatingWorkflowStorage(baseStorage);
362
+ return new CachingWorkflowStorage(validatingStorage, 300_000);
363
+ }
364
+ ```
365
+
366
+ #### Environment Setup
367
+
368
+ ```bash
369
+ # Set your GitHub token
370
+ export GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxxxxxx
371
+
372
+ # Or use a .env file
373
+ echo "GITHUB_TOKEN=ghp_xxxxxxxxxxxxxxxxxxxx" >> .env
374
+ ```
375
+
376
+ **Security Notes**:
377
+ - **Never commit tokens to version control**
378
+ - Use read-only tokens when possible
379
+ - Rotate tokens regularly
380
+ - Consider using GitHub Apps for organization-wide access
381
+
382
+ ### 4. Hybrid Local + Remote
383
+
384
+ **Scenario**: Development uses local workflows, production uses Git repository.
385
+
386
+ ```typescript
387
+ export function createDefaultWorkflowStorage(): CachingWorkflowStorage {
388
+ const isDevelopment = process.env['NODE_ENV'] === 'development';
389
+
390
+ const config = {
391
+ includeBundled: true,
392
+ includeUser: true,
393
+ includeProject: true,
394
+ gitRepositories: isDevelopment ? undefined : [
395
+ {
396
+ repositoryUrl: 'https://github.com/mycompany/workflows.git',
397
+ branch: 'production',
398
+ syncInterval: 60,
399
+ authToken: process.env['GITHUB_TOKEN']
400
+ }
401
+ ]
402
+ };
403
+
404
+ const baseStorage = createEnhancedMultiSourceWorkflowStorage(config);
405
+ const validatingStorage = new SchemaValidatingWorkflowStorage(baseStorage);
406
+ return new CachingWorkflowStorage(validatingStorage, 300_000);
407
+ }
408
+ ```
409
+
410
+ ```bash
411
+ # Development: Use local workflows
412
+ NODE_ENV=development workrail run my-workflow
413
+
414
+ # Production: Use Git workflows
415
+ NODE_ENV=production workrail run my-workflow
416
+ ```
417
+
418
+ ### 5. HTTP Registry
419
+
420
+ **Scenario**: Large organization with internal workflow registry.
421
+
422
+ ```typescript
423
+ export function createDefaultWorkflowStorage(): CachingWorkflowStorage {
424
+ const baseStorage = createEnhancedMultiSourceWorkflowStorage({
425
+ includeBundled: true,
426
+ includeUser: true,
427
+ includeProject: true,
428
+ remoteRegistries: [
429
+ {
430
+ baseUrl: 'https://workflows.mycompany.internal',
431
+ apiKey: process.env['WORKFLOW_REGISTRY_API_KEY'],
432
+ timeout: 10000,
433
+ retryAttempts: 3
434
+ }
435
+ ]
436
+ });
437
+
438
+ const validatingStorage = new SchemaValidatingWorkflowStorage(baseStorage);
439
+ return new CachingWorkflowStorage(validatingStorage, 300_000);
440
+ }
441
+ ```
442
+
443
+ **Registry API Requirements**:
444
+ ```
445
+ GET /workflows → List all workflows
446
+ GET /workflows/:id → Get specific workflow
447
+ GET /workflows/summaries → List workflow summaries
448
+ POST /workflows → Publish workflow (requires auth)
449
+ ```
450
+
451
+ ### 6. Environment Variable Configuration
452
+
453
+ **Git Repository (Simple)**:
454
+ ```bash
455
+ export WORKFLOW_GIT_REPO_URL=https://github.com/myteam/workflows.git
456
+ export WORKFLOW_GIT_REPO_BRANCH=main
457
+ export WORKFLOW_GIT_AUTH_TOKEN=${GITHUB_TOKEN}
458
+ export WORKFLOW_GIT_SYNC_INTERVAL=60
459
+ ```
460
+
461
+ **Git Repository (Advanced - JSON)**:
462
+ ```bash
463
+ export WORKFLOW_GIT_REPOS='[
464
+ {
465
+ "repositoryUrl": "https://github.com/workrail/community-workflows.git",
466
+ "branch": "main",
467
+ "syncInterval": 1440
468
+ },
469
+ {
470
+ "repositoryUrl": "https://github.com/myteam/workflows.git",
471
+ "branch": "production",
472
+ "syncInterval": 60,
473
+ "authToken": "'${GITHUB_TOKEN}'"
474
+ }
475
+ ]'
476
+ ```
477
+
478
+ **Disable Sources**:
479
+ ```bash
480
+ export WORKFLOW_INCLUDE_BUNDLED=false
481
+ export WORKFLOW_INCLUDE_USER=false
482
+ export WORKFLOW_INCLUDE_PROJECT=false
483
+ ```
484
+
485
+ ---
486
+
487
+ ## Creating a Repository
488
+
489
+ ### Repository Structure
490
+
491
+ External repositories should follow this structure:
492
+
493
+ ```
494
+ workflow-repository/
495
+ ├── README.md # Repository documentation
496
+ ├── workflows/ # Workflows directory (required)
497
+ │ ├── bug-investigation.json
498
+ │ ├── code-review.json
499
+ │ └── deployment.json
500
+ ├── .gitignore
501
+ └── .github/
502
+ └── workflows/
503
+ └── validate.yml # CI validation
504
+ ```
505
+
506
+ ### Required Conventions
507
+
508
+ 1. **Directory Name**: Must be named `workflows/` (singular or plural)
509
+ 2. **File Extension**: All workflow files must be `.json`
510
+ 3. **File Naming**: Filename should match workflow ID (e.g., `bug-fix.json` → `"id": "bug-fix"`)
511
+ 4. **Schema Compliance**: All workflows must validate against the WorkRail schema
512
+ 5. **No Subdirectories**: Flat structure (no nested directories)
513
+
514
+ ### Step-by-Step Guide
515
+
516
+ #### 1. Create Repository
517
+
518
+ ```bash
519
+ # Create new repository
520
+ mkdir my-workflows
521
+ cd my-workflows
522
+ git init
523
+
524
+ # Create workflows directory
525
+ mkdir workflows
526
+
527
+ # Create README
528
+ cat > README.md << 'EOF'
529
+ # My Workflows
530
+
531
+ Shared workflow collection for our team.
532
+
533
+ ## Usage
534
+
535
+ ```bash
536
+ export WORKFLOW_GIT_REPO_URL=https://github.com/username/my-workflows.git
537
+ workrail init
538
+ workrail list
539
+ ```
540
+
541
+ ## Contributing
542
+
543
+ 1. Add workflow to `workflows/` directory
544
+ 2. Validate: `workrail validate workflows/your-workflow.json`
545
+ 3. Submit pull request
546
+ EOF
547
+ ```
548
+
549
+ #### 2. Add Workflow
550
+
551
+ ```bash
552
+ cat > workflows/code-review.json << 'EOF'
553
+ {
554
+ "id": "code-review",
555
+ "name": "Code Review Workflow",
556
+ "description": "Systematic code review process",
557
+ "version": "1.0.0",
558
+ "steps": [
559
+ {
560
+ "id": "review-changes",
561
+ "title": "Review Code Changes",
562
+ "prompt": "Review the code changes for correctness, style, and best practices.",
563
+ "guidance": [
564
+ "Check for logic errors",
565
+ "Verify coding standards",
566
+ "Look for security issues"
567
+ ]
568
+ }
569
+ ]
570
+ }
571
+ EOF
572
+ ```
573
+
574
+ #### 3. Add CI Validation
575
+
576
+ ```bash
577
+ mkdir -p .github/workflows
578
+ cat > .github/workflows/validate.yml << 'EOF'
579
+ name: Validate Workflows
580
+
581
+ on:
582
+ push:
583
+ branches: [ main ]
584
+ pull_request:
585
+ branches: [ main ]
586
+
587
+ jobs:
588
+ validate:
589
+ runs-on: ubuntu-latest
590
+ steps:
591
+ - uses: actions/checkout@v3
592
+ - uses: actions/setup-node@v3
593
+ with:
594
+ node-version: '18'
595
+ - name: Install WorkRail
596
+ run: npm install -g workrail
597
+ - name: Validate Workflows
598
+ run: |
599
+ for file in workflows/*.json; do
600
+ echo "Validating $file..."
601
+ workrail validate "$file"
602
+ done
603
+ EOF
604
+ ```
605
+
606
+ #### 4. Add .gitignore
607
+
608
+ ```bash
609
+ cat > .gitignore << 'EOF'
610
+ node_modules/
611
+ .env
612
+ .env.local
613
+ .DS_Store
614
+ .vscode/
615
+ .idea/
616
+ *.swp
617
+ EOF
618
+ ```
619
+
620
+ #### 5. Commit and Push
621
+
622
+ ```bash
623
+ git add .
624
+ git commit -m "Initial workflow repository setup"
625
+
626
+ # Create GitHub repository (via gh CLI)
627
+ gh repo create my-workflows --public --source=. --remote=origin --push
628
+
629
+ # Or manually
630
+ git remote add origin https://github.com/username/my-workflows.git
631
+ git branch -M main
632
+ git push -u origin main
633
+ ```
634
+
635
+ #### 6. Test
636
+
637
+ ```bash
638
+ cd /tmp
639
+ export WORKFLOW_GIT_REPO_URL=https://github.com/username/my-workflows.git
640
+ workrail init
641
+ workrail list
642
+ ```
643
+
644
+ ---
645
+
646
+ ## Implementation Guide
647
+
648
+ ### Implementation Options
649
+
650
+ #### Option A: Minimal Integration (1 day)
651
+
652
+ Just enable `GitWorkflowStorage` for users who want it:
653
+
654
+ ```typescript
655
+ // Add to docs/README.md
656
+ export WORKFLOW_GIT_REPO_URL=https://github.com/myteam/workflows.git
657
+ workrail init
658
+ ```
659
+
660
+ **Pros**: Zero code changes, users can opt-in immediately
661
+ **Cons**: Manual setup, not discoverable
662
+
663
+ #### Option B: Environment Variable Support (3 days)
664
+
665
+ Add env var support to default configuration:
666
+
667
+ ```typescript
668
+ // container.ts - Update createDefaultWorkflowStorage()
669
+ const gitRepoUrl = process.env['WORKFLOW_GIT_REPO_URL'];
670
+ if (gitRepoUrl) {
671
+ config.gitRepositories = [{
672
+ repositoryUrl: gitRepoUrl,
673
+ branch: process.env['WORKFLOW_GIT_REPO_BRANCH'] || 'main',
674
+ authToken: process.env['GITHUB_TOKEN'],
675
+ syncInterval: 60
676
+ }];
677
+ }
678
+ ```
679
+
680
+ **Pros**: Simple, opt-in, no breaking changes
681
+ **Cons**: Limited discoverability
682
+
683
+ #### Option C: Full Integration (1 week) RECOMMENDED
684
+
685
+ Create `EnhancedMultiSourceWorkflowStorage` and make it the default:
686
+
687
+ 1. **Day 1-2**: Implement `EnhancedMultiSourceWorkflowStorage`
688
+ 2. **Day 3**: Add CLI commands (`workrail repo add/remove/list/sync`)
689
+ 3. **Day 4**: Add tests and validation
690
+ 4. **Day 5**: Update documentation
691
+
692
+ **Pros**: Full-featured, discoverable, future-proof
693
+ **Cons**: Most work (but still only 1 week)
694
+
695
+ ### Recommended CLI Commands
696
+
697
+ ```bash
698
+ # Add a Git repository
699
+ workrail repo add github https://github.com/myorg/workflows.git
700
+
701
+ # List configured repositories
702
+ workrail repo list
703
+
704
+ # Sync all repositories
705
+ workrail repo sync
706
+
707
+ # Remove a repository
708
+ workrail repo remove github
709
+
710
+ # Show workflows from specific source
711
+ workrail list --source=github
712
+ ```
713
+
714
+ ### Organization Size Recommendations
715
+
716
+ #### Small Teams (< 10 people)
717
+ **Use**: Git repository approach
718
+ - Single team repository
719
+ - Store in company GitHub/GitLab
720
+ - No additional infrastructure needed
721
+
722
+ #### Medium Organizations (10-100 people)
723
+ **Use**: Multi-repository approach
724
+ - Public community workflows (read-only)
725
+ - Team-specific repositories
726
+ - Optional: Internal registry for high-frequency updates
727
+
728
+ #### Large Enterprises (100+ people)
729
+ **Use**: Hybrid approach
730
+ - Internal HTTP registry for frequent updates
731
+ - Git repositories for team workflows
732
+ - Centralized workflow governance
733
+ - Consider plugin approach for distribution
734
+
735
+ ---
736
+
737
+ ## Security
738
+
739
+ ### GitWorkflowStorage Security Features
740
+
741
+ 1. **URL Validation**: Only whitelisted hosting providers
742
+ - github.com, gitlab.com, bitbucket.org, dev.azure.com, sourceforge.net
743
+ - Must use HTTPS or git:// protocol
744
+
745
+ 2. **Command Injection Prevention**: All shell arguments are escaped
746
+
747
+ 3. **Path Traversal Prevention**: All file operations validated against base directory
748
+
749
+ 4. **Resource Limits**:
750
+ - Max file size: 1MB (configurable)
751
+ - Max files: 100 (configurable)
752
+ - Clone timeout: 60 seconds
753
+ - Pull timeout: 30 seconds
754
+
755
+ 5. **Authentication**: Supports personal access tokens (not username/password)
756
+
757
+ ### Best Practices
758
+
759
+ 1. **Use Read-Only Tokens**: If using authentication, use tokens with read-only access
760
+ 2. **Pin Branches**: Use specific branches or tags instead of 'main' in production
761
+ 3. **Regular Audits**: Review workflow repositories regularly for unauthorized changes
762
+ 4. **Access Control**: Use private repositories for sensitive workflows
763
+ 5. **Sync Intervals**: Balance freshness vs. API rate limits (60+ minutes recommended)
764
+ 6. **Token Security**:
765
+ - Never commit tokens to version control
766
+ - Rotate tokens regularly
767
+ - Use environment variables for token storage
768
+ - Consider GitHub Apps for organization-wide access
769
+
770
+ ---
771
+
772
+ ## Troubleshooting
773
+
774
+ ### Repository Not Found
775
+
776
+ ```bash
777
+ # Check Git URL
778
+ git ls-remote https://github.com/username/workflows.git
779
+
780
+ # Check authentication
781
+ git clone https://github.com/username/workflows.git /tmp/test
782
+ ```
783
+
784
+ ### Authentication Issues
785
+
786
+ ```bash
787
+ # Verify token has access
788
+ curl -H "Authorization: token ${GITHUB_TOKEN}" \
789
+ https://api.github.com/repos/username/workflows
790
+
791
+ # Test with explicit token
792
+ WORKFLOW_GIT_AUTH_TOKEN=ghp_xxx workrail init
793
+ ```
794
+
795
+ ### Sync Issues
796
+
797
+ ```bash
798
+ # Force sync by removing cache
799
+ rm -rf ~/.workrail/team-workflows
800
+ workrail init
801
+
802
+ # Check sync logs
803
+ WORKFLOW_DEBUG=true workrail list
804
+ ```
805
+
806
+ ### Validation Failures
807
+
808
+ ```bash
809
+ # Validate individual workflow
810
+ workrail validate workflows/my-workflow.json
811
+
812
+ # Common issues to check:
813
+ # - Missing required fields (id, name, description, version, steps)
814
+ # - Invalid step structure
815
+ # - Malformed JSON
816
+ ```
817
+
818
+ ---
819
+
820
+ ## Best Practices
821
+
822
+ ### Repository Organization
823
+
824
+ 1. **Use descriptive repository names** (e.g., `myteam-workflows`)
825
+ 2. **Document workflows in README** with usage examples
826
+ 3. **Add CI validation** to catch errors before merge
827
+ 4. **Use semantic versioning** for workflow versions
828
+ 5. **Tag releases** for stable versions
829
+
830
+ ### Security
831
+
832
+ 1. **Never commit authentication tokens**
833
+ 2. **Use read-only tokens** when possible
834
+ 3. **Regularly audit** workflow repositories
835
+ 4. **Use private repositories** for sensitive workflows
836
+ 5. **Review pull requests** carefully before merging
837
+
838
+ ### Performance
839
+
840
+ 1. **Set appropriate sync intervals** (60+ minutes recommended)
841
+ 2. **Use caching** (enabled by default)
842
+ 3. **Limit repository size** (< 100 workflows)
843
+ 4. **Keep workflows under 1MB** each
844
+ 5. **Monitor API rate limits**
845
+
846
+ ### Collaboration
847
+
848
+ 1. **Use pull requests** for all changes
849
+ 2. **Require approvals** for workflow changes
850
+ 3. **Document workflow purposes** and usage
851
+ 4. **Tag releases** for stable versions
852
+ 5. **Maintain a staging repository** for testing
853
+
854
+ ### Testing
855
+
856
+ 1. **Validate workflows in CI/CD** before merge
857
+ 2. **Test workflows** before merging to production branch
858
+ 3. **Use feature branches** for development
859
+ 4. **Maintain a staging repository** for pre-production testing
860
+ 5. **Run end-to-end tests** for critical workflows
861
+
862
+ ---
863
+
864
+ ## Open Questions
865
+
866
+ 1. **Repository Discovery**: Should we provide a workflow marketplace/directory?
867
+ 2. **Workflow Signing**: Do we need GPG signing for security?
868
+ 3. **SSH Keys**: Support SSH authentication in addition to tokens?
869
+ 4. **Monorepo Support**: Load workflows from subdirectories?
870
+ 5. **Webhooks**: Real-time sync instead of polling?
871
+ 6. **Default Repositories**: Should we include a community repo by default?
872
+
873
+ ---
874
+
875
+ ## Conclusion
876
+
877
+ **The infrastructure is complete and production-ready. The only remaining work is integration.**
878
+
879
+ ### Recommended Action
880
+
881
+ Implement **Option C (Full Integration)** over 1 week.
882
+
883
+ This provides:
884
+ 1. Complete feature set
885
+ 2. Good developer experience
886
+ 3. Future extensibility
887
+ 4. Minimal risk
888
+
889
+ ### Why Git-Based Approach?
890
+
891
+ 1. No additional infrastructure needed (use GitHub/GitLab)
892
+ 2. Developers already understand Git workflows
893
+ 3. Perfect for collaboration (PRs, reviews, versioning)
894
+ 4. Already fully implemented and tested
895
+ 5. Free hosting with unlimited public repos
896
+
897
+ ### Implementation Effort
898
+
899
+ - **Minimum viable**: 1 day (Option A)
900
+ - **Environment variables**: 3 days (Option B)
901
+ - **Full feature set**: 1 week (Option C)
902
+
903
+ ### Total Lines of Code Required
904
+
905
+ - `EnhancedMultiSourceWorkflowStorage`: ~100 LOC
906
+ - Configuration support: ~50 LOC
907
+ - CLI commands: ~200 LOC
908
+ - Tests: ~300 LOC
909
+ - **Total**: ~650 LOC
910
+
911
+ ---
912
+
913
+ **Resources**:
914
+ - Implementation: `src/infrastructure/storage/enhanced-multi-source-workflow-storage.ts`
915
+ - Tests: `tests/unit/enhanced-multi-source-workflow-storage.test.ts`
916
+ - Existing implementations: `git-workflow-storage.ts`, `remote-workflow-storage.ts`, `plugin-workflow-storage.ts`