ado-sync 0.1.64 → 0.1.67

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (63) hide show
  1. package/README.md +20 -15
  2. package/dist/__tests__/regressions.test.js +1011 -1
  3. package/dist/__tests__/regressions.test.js.map +1 -1
  4. package/dist/ai/generate-spec.d.ts +1 -1
  5. package/dist/ai/generate-spec.js +23 -0
  6. package/dist/ai/generate-spec.js.map +1 -1
  7. package/dist/ai/summarizer.d.ts +3 -2
  8. package/dist/ai/summarizer.js +50 -1
  9. package/dist/ai/summarizer.js.map +1 -1
  10. package/dist/azure/test-cases.d.ts +11 -1
  11. package/dist/azure/test-cases.js +286 -43
  12. package/dist/azure/test-cases.js.map +1 -1
  13. package/dist/cli.js +91 -14
  14. package/dist/cli.js.map +1 -1
  15. package/dist/config.js +74 -1
  16. package/dist/config.js.map +1 -1
  17. package/dist/id-markers.d.ts +1 -0
  18. package/dist/id-markers.js +13 -0
  19. package/dist/id-markers.js.map +1 -1
  20. package/dist/mcp-server.js +1 -1
  21. package/dist/mcp-server.js.map +1 -1
  22. package/dist/sync/cache.d.ts +2 -0
  23. package/dist/sync/cache.js.map +1 -1
  24. package/dist/sync/engine.d.ts +12 -1
  25. package/dist/sync/engine.js +210 -41
  26. package/dist/sync/engine.js.map +1 -1
  27. package/dist/types.d.ts +56 -4
  28. package/llms.txt +12 -11
  29. package/package.json +8 -1
  30. package/docs/advanced.md +0 -988
  31. package/docs/agent-setup.md +0 -204
  32. package/docs/capability-roadmap.md +0 -280
  33. package/docs/cli.md +0 -609
  34. package/docs/configuration.md +0 -322
  35. package/docs/examples/csharp-mstest-local-llm.yaml +0 -35
  36. package/docs/examples/csharp-mstest.yaml +0 -21
  37. package/docs/examples/csharp-nunit.yaml +0 -21
  38. package/docs/examples/csharp-specflow.yaml +0 -16
  39. package/docs/examples/cypress.yaml +0 -21
  40. package/docs/examples/detox-react-native.yaml +0 -21
  41. package/docs/examples/espresso-android.yaml +0 -21
  42. package/docs/examples/flutter-dart.yaml +0 -21
  43. package/docs/examples/java-junit.yaml +0 -21
  44. package/docs/examples/java-testng.yaml +0 -21
  45. package/docs/examples/js-jasmine-wdio.yaml +0 -21
  46. package/docs/examples/js-jest.yaml +0 -21
  47. package/docs/examples/playwright-js.yaml +0 -21
  48. package/docs/examples/playwright-ts.yaml +0 -21
  49. package/docs/examples/puppeteer.yaml +0 -21
  50. package/docs/examples/python-pytest.yaml +0 -21
  51. package/docs/examples/robot-framework.yaml +0 -19
  52. package/docs/examples/testcafe.yaml +0 -21
  53. package/docs/examples/xcuitest-ios.yaml +0 -21
  54. package/docs/mcp-server.md +0 -312
  55. package/docs/publish-test-results.md +0 -939
  56. package/docs/spec-formats.md +0 -1357
  57. package/docs/troubleshooting.md +0 -101
  58. package/docs/vscode-extension.md +0 -139
  59. package/docs/work-item-links.md +0 -115
  60. package/docs/workflows.md +0 -457
  61. package/mkdocs.yml +0 -40
  62. package/requirements-docs.txt +0 -4
  63. package/scripts/build_site.sh +0 -6
package/docs/cli.md DELETED
@@ -1,609 +0,0 @@
1
- # CLI Reference
2
-
3
- ```
4
- ado-sync [options] [command]
5
-
6
- Options:
7
- -c, --config <path> Path to config file (default: ado-sync.json)
8
- --output <format> Output format: text (default) or json
9
- -V, --version Print version
10
- -h, --help Show help
11
-
12
- Commands:
13
- init Generate a starter config file (interactive wizard)
14
- validate Check config and Azure DevOps connectivity
15
- push Push local specs to Azure DevOps
16
- pull Pull updates from Azure DevOps into local files
17
- status Show diff without making changes
18
- diff Show field-level diff between local and Azure
19
- generate Generate local spec files from ADO User Stories
20
- story-context Show AC, tags, and linked TCs for a User Story
21
- publish-test-results Publish TRX / JUnit / Cucumber JSON results
22
- coverage Spec link rate and story coverage report
23
- stale List (and optionally retire) TCs with no local spec
24
- ac-gate Validate stories have AC + linked TCs (CI gate)
25
- find-tagged Find work items where a tag was added in the last N hours/days
26
- trend Flaky test detection and pass-rate trend report
27
- watch Auto-push on file save
28
- help [command] Help for a specific command
29
- ```
30
-
31
- ---
32
-
33
- ## `init`
34
-
35
- Generate a starter config file. Runs an **interactive wizard** when stdin is a TTY.
36
-
37
- ```bash
38
- ado-sync init # creates ado-sync.json (wizard)
39
- ado-sync init ado-sync.yml # YAML format
40
- ado-sync init --no-interactive # dump template without prompting
41
- ```
42
-
43
- The wizard asks for org URL, project, auth type, token env var, test plan ID, local spec type, and include glob — then writes a minimal valid config.
44
-
45
- ---
46
-
47
- ## `validate`
48
-
49
- Check that the config is valid and Azure DevOps is reachable. Run this before your first `push`.
50
-
51
- ```bash
52
- ado-sync validate
53
- ado-sync validate -c path/to/ado-sync.json
54
- ```
55
-
56
- Output:
57
- ```
58
- ✓ Config loaded — /project/ado-sync.json
59
- ✓ Azure connection — https://dev.azure.com/myorg
60
- ✓ Project "MyProject" found
61
- ✓ Test Plan #42 "Regression Suite" found
62
-
63
- All checks passed.
64
- ```
65
-
66
- ---
67
-
68
- ## `push`
69
-
70
- Push local spec files to Azure DevOps — creates new Test Cases or updates existing ones.
71
-
72
- ```bash
73
- ado-sync push
74
- ado-sync push --dry-run
75
- ado-sync push --tags "@smoke and not @wip"
76
- ado-sync push --config-override testPlan.id=9999
77
-
78
- # AI-generated test steps for code files
79
- ado-sync push --ai-provider heuristic # fast regex, no model needed
80
- ado-sync push --ai-provider local --ai-model ~/.cache/models/qwen2.5-coder-1.5b-instruct-q4_k_m.gguf
81
- ado-sync push --ai-provider ollama --ai-model qwen2.5-coder:7b
82
- ado-sync push --ai-provider openai --ai-key $OPENAI_API_KEY
83
- ado-sync push --ai-provider anthropic --ai-key $ANTHROPIC_API_KEY
84
- ado-sync push --ai-provider huggingface --ai-model mistralai/Mistral-7B-Instruct-v0.3 --ai-key $HF_TOKEN
85
- ado-sync push --ai-provider bedrock --ai-model anthropic.claude-3-haiku-20240307-v1:0 --ai-region us-east-1
86
- ado-sync push --ai-provider azureai --ai-url https://myresource.openai.azure.com --ai-model gpt-4o --ai-key $AZURE_OPENAI_KEY
87
- ado-sync push --ai-provider github --ai-model gpt-4o # uses $GITHUB_TOKEN automatically
88
- ado-sync push --ai-provider azureinference --ai-url https://myendpoint.inference.azure.com --ai-model gpt-4o --ai-key $AZURE_AI_KEY
89
- ado-sync push --ai-provider none # disable AI entirely
90
- ado-sync push --ai-context ./docs/ai-context.md # inject domain context
91
- ```
92
-
93
- | Scenario | Action |
94
- |----------|--------|
95
- | No ID tag | Creates a new Test Case, writes ID back |
96
- | ID tag, no changes | Skipped |
97
- | ID tag, content changed | Updates the existing Test Case |
98
- | Deleted locally, still in Azure | Tagged `ado-sync:removed` in Azure |
99
-
100
- ### AI auto-summary
101
-
102
- For code-based types (`java`, `csharp`, `python`, `javascript`, `playwright`, `cypress`, `testcafe`, `detox`, `espresso`, `xcuitest`, `flutter`), ado-sync reads test function bodies and generates a TC **title**, **description**, and **steps** automatically.
103
-
104
- > No setup required — `push` always works, falling back to fast heuristic analysis.
105
-
106
- | Provider | Quality | Setup |
107
- |---|---|---|
108
- | `local` *(default)* | Good–Excellent | Download a GGUF model (see below) |
109
- | `heuristic` | Basic | None — offline, zero dependencies |
110
- | `ollama` | Good–Excellent | `ollama pull qwen2.5-coder:7b` |
111
- | `openai` | Excellent | `--ai-key $OPENAI_API_KEY` |
112
- | `anthropic` | Excellent | `--ai-key $ANTHROPIC_API_KEY` |
113
- | `huggingface` | Good–Excellent | `--ai-model <model-id> --ai-key $HF_TOKEN` |
114
- | `bedrock` | Excellent | AWS credentials + optional `--ai-region` |
115
- | `azureai` | Excellent | `--ai-url <endpoint> --ai-key $AZURE_OPENAI_KEY` |
116
- | `github` | Excellent | GitHub PAT with `models:read` — `$GITHUB_TOKEN` auto-detected |
117
- | `azureinference` | Excellent | `--ai-url <endpoint> --ai-key $AZURE_AI_KEY` |
118
- | `openai` + `--ai-url` | Excellent | Any OpenAI-compatible proxy (LiteLLM, vLLM) |
119
-
120
- #### Local LLM — model download
121
-
122
- ```bash
123
- # macOS / Linux
124
- mkdir -p ~/.cache/ado-sync/models
125
- curl -L -o ~/.cache/ado-sync/models/qwen2.5-coder-1.5b-instruct-q4_k_m.gguf \
126
- "https://huggingface.co/Qwen/Qwen2.5-Coder-1.5B-Instruct-GGUF/resolve/main/qwen2.5-coder-1.5b-instruct-q4_k_m.gguf"
127
-
128
- ado-sync push --ai-model ~/.cache/ado-sync/models/qwen2.5-coder-1.5b-instruct-q4_k_m.gguf
129
- ```
130
-
131
- | Model | RAM | Quality |
132
- |-------|-----|---------|
133
- | 1.5B Q4_K_M | ~1.1 GB | Good |
134
- | 7B Q4_K_M | ~4.5 GB | Better |
135
- | 14B Q4_K_M | ~8.5 GB | Excellent |
136
-
137
- #### Inject domain context
138
-
139
- ```bash
140
- ado-sync push --ai-context ./docs/ai-context.md
141
- ```
142
-
143
- Or in config:
144
- ```json
145
- { "sync": { "ai": { "provider": "anthropic", "contextFile": "./docs/ai-context.md" } } }
146
- ```
147
-
148
- #### Freeze steps in source files
149
-
150
- Enable `writebackDocComment: true` to write AI-generated steps as JSDoc comments above each `test()` call. On subsequent pushes the parser reads the JSDoc, so AI is not re-invoked.
151
-
152
- ```json
153
- { "sync": { "ai": { "writebackDocComment": true } } }
154
- ```
155
-
156
- ---
157
-
158
- ## `pull`
159
-
160
- Pull Azure DevOps Test Case changes into local spec files.
161
-
162
- ```bash
163
- ado-sync pull
164
- ado-sync pull --dry-run
165
- ado-sync pull --tags "@smoke"
166
-
167
- # AI step generation — same providers as push, used when pull creates new local stubs
168
- ado-sync pull --ai-provider anthropic --ai-key $ANTHROPIC_API_KEY
169
- ado-sync pull --ai-provider openai --ai-key $OPENAI_API_KEY --ai-model gpt-4o
170
- ado-sync pull --ai-provider ollama --ai-model qwen2.5-coder:7b
171
- ado-sync pull --ai-provider huggingface --ai-model mistralai/Mistral-7B-Instruct-v0.3 --ai-key $HF_TOKEN
172
- ado-sync pull --ai-provider bedrock --ai-model anthropic.claude-3-haiku-20240307-v1:0 --ai-region us-east-1
173
- ado-sync pull --ai-provider azureai --ai-url https://myresource.openai.azure.com --ai-model gpt-4o --ai-key $AZURE_OPENAI_KEY
174
- ado-sync pull --ai-provider github --ai-model gpt-4o
175
- ado-sync pull --ai-provider azureinference --ai-url https://myendpoint.inference.azure.com --ai-model gpt-4o --ai-key $AZURE_AI_KEY
176
- ```
177
-
178
- ---
179
-
180
- ## `status`
181
-
182
- Show what would change on the next push without making any modifications.
183
-
184
- ```bash
185
- ado-sync status
186
- ado-sync status --tags "@smoke"
187
- ado-sync status --output json # machine-readable
188
-
189
- # AI step generation — used when computing diff for code-based test types
190
- ado-sync status --ai-provider anthropic --ai-key $ANTHROPIC_API_KEY
191
- ado-sync status --ai-provider openai --ai-key $OPENAI_API_KEY
192
- ado-sync status --ai-provider ollama --ai-model qwen2.5-coder:7b
193
- ado-sync status --ai-provider bedrock --ai-model anthropic.claude-3-haiku-20240307-v1:0 --ai-region us-east-1
194
- ado-sync status --ai-provider azureai --ai-url https://myresource.openai.azure.com --ai-model gpt-4o --ai-key $AZURE_OPENAI_KEY
195
- ado-sync status --ai-provider github --ai-model gpt-4o
196
- ado-sync status --ai-provider azureinference --ai-url https://myendpoint.inference.azure.com --ai-model gpt-4o --ai-key $AZURE_AI_KEY
197
- ```
198
-
199
- ---
200
-
201
- ## `diff`
202
-
203
- Show a field-level diff between local specs and Azure DevOps — richer than `status`.
204
-
205
- ```bash
206
- ado-sync diff
207
- ado-sync diff --tags "@smoke"
208
- ```
209
-
210
- Output example:
211
- ```
212
- ~ specs/login.feature:12 · Login with valid credentials [#1234]
213
- changed fields: title, steps
214
- ```
215
-
216
- ---
217
-
218
- ## `generate`
219
-
220
- Generate local spec files (`.feature` or `.md`) from Azure DevOps User Stories, pulling title, description, and acceptance criteria to scaffold each file.
221
-
222
- ```bash
223
- # By explicit work item IDs
224
- ado-sync generate --story-ids 1234,5678
225
-
226
- # By area path (fetches all User Stories under it)
227
- ado-sync generate --area-path "MyProject\\\\Teams\\\\QA"
228
-
229
- # By WIQL query
230
- ado-sync generate --query "SELECT [System.Id] FROM WorkItems WHERE [System.WorkItemType]='User Story' AND [System.State]='Active'"
231
-
232
- # Options
233
- ado-sync generate --story-ids 1234 --format gherkin # output .feature files
234
- ado-sync generate --story-ids 1234 --format markdown # output .md files (default)
235
- ado-sync generate --story-ids 1234 --output-folder specs/generated
236
- ado-sync generate --story-ids 1234 --force # overwrite existing files
237
- ado-sync generate --story-ids 1234 --dry-run # preview without writing
238
-
239
- # AI-powered generation — fills in real steps from the story description + AC
240
- ado-sync generate --story-ids 1234 --ai-provider anthropic --ai-key $ANTHROPIC_API_KEY
241
- ado-sync generate --story-ids 1234 --ai-provider openai --ai-key $OPENAI_API_KEY --ai-model gpt-4o
242
- ado-sync generate --story-ids 1234 --ai-provider ollama --ai-model qwen2.5-coder:7b
243
- ado-sync generate --story-ids 1234 --ai-provider huggingface --ai-model mistralai/Mistral-7B-Instruct-v0.3 --ai-key $HF_TOKEN
244
- ado-sync generate --story-ids 1234 --ai-provider bedrock --ai-model anthropic.claude-3-haiku-20240307-v1:0 --ai-region us-east-1
245
- ado-sync generate --story-ids 1234 --ai-provider azureai --ai-url https://myresource.openai.azure.com --ai-model gpt-4o --ai-key $AZURE_OPENAI_KEY
246
- ado-sync generate --story-ids 1234 --ai-provider github --ai-model gpt-4o
247
- ado-sync generate --story-ids 1234 --ai-provider azureinference --ai-url https://myendpoint.inference.azure.com --ai-model gpt-4o --ai-key $AZURE_AI_KEY
248
- ado-sync generate --story-ids 1234 --ai-provider local --ai-model ~/.cache/models/qwen2.5-coder-1.5b.gguf
249
- ado-sync generate --story-ids 1234 --ai-provider openai --ai-key $OPENAI_API_KEY \
250
- --ai-context src/orders/** --ai-context tests/orders/**
251
- ```
252
-
253
- ### AI generate flags
254
-
255
- | Flag | Description |
256
- |------|-------------|
257
- | `--ai-provider` | Provider: `local`, `ollama`, `openai`, `anthropic`, `huggingface`, `bedrock`, `azureai`, `github`, `azureinference` |
258
- | `--ai-model` | Model name, path, or deployment ID |
259
- | `--ai-key` | API key (or `$ENV_VAR` reference) |
260
- | `--ai-url` | Base URL override (Ollama, Azure OpenAI full endpoint, OpenAI-compatible) |
261
- | `--ai-region` | AWS region for `bedrock` (default: `AWS_REGION` env or `us-east-1`) |
262
- | `--ai-context` | Additional context source for AI generation. Accepts a file, folder, or glob. Repeatable. |
263
-
264
- ### AI generation context
265
-
266
- `generate` does not scan the entire workspace by default. For better AI-generated specs, pass a small set of relevant files or folders explicitly:
267
-
268
- ```bash
269
- ado-sync generate --story-ids 1234 \
270
- --ai-provider openai --ai-key $OPENAI_API_KEY \
271
- --ai-context src/billing/** \
272
- --ai-context tests/billing/** \
273
- --ai-context docs/billing.md
274
- ```
275
-
276
- Best results come from targeted context: relevant app code, existing automation, and feature docs. The CLI caps the number of files and total context size so prompts stay bounded and predictable.
277
-
278
- Without `--ai-provider`, `generate` uses the template scaffold (no AI required). With it, the AI reads the story's title, description, and acceptance criteria and produces realistic Given/When/Then steps.
279
-
280
- For `bedrock`, AWS credentials are picked up from the standard chain (`AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` env vars, `~/.aws/credentials`, or IAM role). Install the SDK if needed: `npm install @aws-sdk/client-bedrock-runtime`.
281
-
282
- For `github`, create a GitHub PAT with `models:read` scope and set `GITHUB_TOKEN` in your environment — no `--ai-key` flag needed. Available models: `gpt-4o`, `gpt-4o-mini`, `Meta-Llama-3.1-70B-Instruct`, `Mistral-large`, and others from [GitHub Models](https://github.com/marketplace/models).
283
-
284
- For `azureinference`, the endpoint is your Azure AI Inference resource URL (e.g. from Azure AI Foundry). Install the SDK if needed: `npm install @azure-rest/ai-inference @azure/core-auth`.
285
-
286
- AI settings also fall back to `sync.ai` in `ado-sync.json`, so you can configure a provider once and all commands (`push`, `generate`, `publish-test-results`) will use it automatically.
287
-
288
- #### Dry-run with AI preview
289
-
290
- When `--dry-run` and `--ai-provider` are both set, the AI is called but no file is written. The first 20 lines of the generated content are printed to the terminal so you can review quality before committing:
291
-
292
- ```bash
293
- ado-sync generate --story-ids 1234 --dry-run --ai-provider anthropic --ai-key $ANTHROPIC_API_KEY
294
- ```
295
-
296
- #### Automatic `@tc:` tag writeback
297
-
298
- If the ADO story already has a linked Test Case (from a previous `push` run), `generate` automatically injects the `@tc:<id>` tag into the new file. This prevents the file from appearing as an untracked new test on the next `status` run.
299
-
300
- ---
301
-
302
- ## `publish-test-results`
303
-
304
- Publish test results from TRX, JUnit XML, NUnit XML, Cucumber JSON, or Playwright JSON files to Azure DevOps Test Runs.
305
-
306
- ```bash
307
- ado-sync publish-test-results --testResult results/test.trx
308
- ado-sync publish-test-results --testResult results/test.xml --testResultFormat junit --dry-run
309
- ado-sync publish-test-results --testResult results/playwright.json --attachmentsFolder test-results/
310
-
311
- # AI failure analysis — posts root-cause + fix suggestion as a comment on each failed result
312
- ado-sync publish-test-results \
313
- --testResult results/playwright.json \
314
- --analyze-failures \
315
- --ai-provider anthropic \
316
- --ai-model claude-haiku-4-5-20251001 \
317
- --ai-key $ANTHROPIC_API_KEY
318
-
319
- # Or with OpenAI
320
- ado-sync publish-test-results \
321
- --testResult results/test.trx \
322
- --analyze-failures \
323
- --ai-provider openai \
324
- --ai-key $OPENAI_API_KEY
325
-
326
- # Or with a local Ollama server (no cloud cost)
327
- ado-sync publish-test-results \
328
- --testResult results/junit.xml \
329
- --analyze-failures \
330
- --ai-provider ollama \
331
- --ai-model qwen2.5-coder:7b
332
-
333
- # Hugging Face Inference API
334
- ado-sync publish-test-results \
335
- --testResult results/playwright.json \
336
- --analyze-failures \
337
- --ai-provider huggingface \
338
- --ai-model mistralai/Mistral-7B-Instruct-v0.3 \
339
- --ai-key $HF_TOKEN
340
-
341
- # AWS Bedrock (uses default AWS credential chain)
342
- ado-sync publish-test-results \
343
- --testResult results/playwright.json \
344
- --analyze-failures \
345
- --ai-provider bedrock \
346
- --ai-model anthropic.claude-3-haiku-20240307-v1:0 \
347
- --ai-region us-east-1
348
-
349
- # Azure OpenAI Service
350
- ado-sync publish-test-results \
351
- --testResult results/playwright.json \
352
- --analyze-failures \
353
- --ai-provider azureai \
354
- --ai-url https://myresource.openai.azure.com \
355
- --ai-model gpt-4o \
356
- --ai-key $AZURE_OPENAI_KEY
357
-
358
- # GitHub Models (uses $GITHUB_TOKEN automatically)
359
- ado-sync publish-test-results \
360
- --testResult results/playwright.json \
361
- --analyze-failures \
362
- --ai-provider github \
363
- --ai-model gpt-4o
364
-
365
- # Azure AI Inference (Azure AI Foundry endpoint)
366
- ado-sync publish-test-results \
367
- --testResult results/playwright.json \
368
- --analyze-failures \
369
- --ai-provider azureinference \
370
- --ai-url https://myendpoint.inference.azure.com \
371
- --ai-model gpt-4o \
372
- --ai-key $AZURE_AI_KEY
373
- ```
374
-
375
- See [publish-test-results.md](publish-test-results.md) for full reference including config-based setup.
376
-
377
- ---
378
-
379
- ## `find-tagged`
380
-
381
- Find work items where a specific tag was added within the last N hours or days. Uses the Azure DevOps **revisions API** to find the exact date and time the tag first appeared — not just the item's last-changed date.
382
-
383
- ```bash
384
- # Tag added in the last 24 hours
385
- ado-sync find-tagged --tag "regression" --hours 24
386
-
387
- # Tag added in the last 7 days
388
- ado-sync find-tagged --tag "sprint-42" --days 7
389
-
390
- # Different work item type
391
- ado-sync find-tagged --tag "blocker" --hours 48 --work-item-type Bug
392
-
393
- # Machine-readable output
394
- ado-sync find-tagged --tag "regression" --days 3 --output json
395
- ```
396
-
397
- | Flag | Description |
398
- |------|-------------|
399
- | `--tag <name>` | **Required.** Tag to search for |
400
- | `--hours <n>` | Return items where the tag was added in the last N hours |
401
- | `--days <n>` | Return items where the tag was added in the last N days |
402
- | `--work-item-type <type>` | Work item type to search (default: `User Story`) |
403
-
404
- One of `--hours` or `--days` is required.
405
-
406
- Example output:
407
- ```
408
- + [#1234] [Active] User can reset password
409
- Tag added: Mar 18, 2026 14:32:07 by Jane Smith (rev 5)
410
- https://dev.azure.com/myorg/MyProject/_workitems/edit/1234
411
-
412
- 3 items found where "regression" was added in the last 24 hours.
413
- ```
414
-
415
- ---
416
-
417
- ## `--config-override`
418
-
419
- All commands accept `--config-override path=value` (repeatable) to set config values at runtime:
420
-
421
- ```bash
422
- ado-sync push --config-override testPlan.id=9999
423
- ado-sync push --config-override sync.disableLocalChanges=true
424
- ado-sync push --config-override sync.tagPrefix=test --config-override testPlan.id=42
425
- ```
426
-
427
- ---
428
-
429
- ## `--output json`
430
-
431
- All sync commands support `--output json` for machine-readable output:
432
-
433
- ```bash
434
- ado-sync status --output json | jq '.[] | select(.action=="updated")'
435
- ado-sync push --output json > results.json
436
- ```
437
-
438
- Each result object: `{ action, filePath, title, azureId?, detail?, changedFields? }`
439
-
440
- ---
441
-
442
- ## `ac-gate`
443
-
444
- Validate that ADO User Stories have Acceptance Criteria and at least one linked Test Case. Designed as a sprint or merge quality gate — exits with code 1 when any stories fail.
445
-
446
- ```bash
447
- # Validate specific story IDs
448
- ado-sync ac-gate --story-ids 1234,5678
449
-
450
- # Validate all User Stories under an area path
451
- ado-sync ac-gate --area-path "MyProject\\Teams\\QA"
452
-
453
- # Validate via a WIQL query
454
- ado-sync ac-gate --query "SELECT [System.Id] FROM WorkItems WHERE [System.WorkItemType]='User Story' AND [System.State]='Active'"
455
-
456
- # Default: validates all Active/Resolved/Closed stories in the project
457
- ado-sync ac-gate
458
-
459
- # Machine-readable output
460
- ado-sync ac-gate --story-ids 1234,5678 --output json
461
- ```
462
-
463
- Options:
464
-
465
- | Flag | Description |
466
- |------|-------------|
467
- | `--story-ids <ids>` | Comma-separated ADO work item IDs |
468
- | `--area-path <path>` | Validate all User Stories under this area path |
469
- | `--query <wiql>` | WIQL query to select stories |
470
- | `--states <states>` | Story states to include (default: `Active,Resolved,Closed`) |
471
-
472
- **Outcomes per story:**
473
- - `pass` — has AC and at least one linked TC
474
- - `no-ac` — missing Acceptance Criteria
475
- - `no-tc` — has AC but no linked Test Cases (run `ado-sync push` to create them)
476
-
477
- **CI usage:**
478
-
479
- ```yaml
480
- - name: AC gate
481
- run: ado-sync ac-gate --area-path "MyProject\\QA"
482
- env:
483
- AZURE_DEVOPS_TOKEN: ${{ secrets.AZURE_DEVOPS_TOKEN }}
484
- ```
485
-
486
- ---
487
-
488
- ## `stale --retire`
489
-
490
- The `stale` command now supports automatically retiring orphaned Azure Test Cases rather than just listing them.
491
-
492
- ```bash
493
- # List stale TCs (unchanged behaviour)
494
- ado-sync stale
495
-
496
- # Preview what --retire would do
497
- ado-sync stale --retire --dry-run
498
-
499
- # Transition stale TCs to Closed and tag ado-sync:retired
500
- ado-sync stale --retire
501
-
502
- # Use a custom target state
503
- ado-sync stale --retire --retire-state "Inactive"
504
- ```
505
-
506
- Options:
507
-
508
- | Flag | Description |
509
- |------|-------------|
510
- | `--retire` | Transition stale TCs to the target state and add `ado-sync:retired` tag |
511
- | `--retire-state <state>` | Target state (default: `Closed`) |
512
- | `--dry-run` | Preview without making changes |
513
-
514
- ---
515
-
516
- ## `trend`
517
-
518
- Analyse historical Azure DevOps test runs to detect flaky tests and failing patterns. Optionally post a summary to a Slack or Teams webhook.
519
-
520
- ```bash
521
- # Analyse last 30 days (default)
522
- ado-sync trend
523
-
524
- # Narrow the period and number of runs sampled
525
- ado-sync trend --days 14 --max-runs 20
526
-
527
- # Show top 20 failing tests
528
- ado-sync trend --top 20
529
-
530
- # Filter to runs whose name contains "nightly"
531
- ado-sync trend --run-name nightly
532
-
533
- # Post summary to Slack
534
- ado-sync trend --webhook-url https://hooks.slack.com/services/... --webhook-type slack
535
-
536
- # Post to Microsoft Teams
537
- ado-sync trend --webhook-url https://your-org.webhook.office.com/... --webhook-type teams
538
-
539
- # Fail the build when flaky tests are detected
540
- ado-sync trend --fail-on-flaky
541
-
542
- # Fail when overall pass rate drops below 85%
543
- ado-sync trend --fail-below 85
544
-
545
- # Machine-readable output
546
- ado-sync trend --output json
547
- ```
548
-
549
- Options:
550
-
551
- | Flag | Description |
552
- |------|-------------|
553
- | `--days <n>` | Days of history to analyse (default: 30) |
554
- | `--max-runs <n>` | Max test runs to sample (default: 50) |
555
- | `--top <n>` | Top N failing tests to surface (default: 10) |
556
- | `--run-name <filter>` | Only include runs whose name contains this string |
557
- | `--webhook-url <url>` | Webhook URL to post summary to |
558
- | `--webhook-type <type>` | `slack` (default), `teams`, or `generic` |
559
- | `--fail-on-flaky` | Exit 1 when any flaky tests are found |
560
- | `--fail-below <percent>` | Exit 1 when overall pass rate is below threshold |
561
-
562
- **Flaky test definition:** a test that both passed and failed at least once in the sampled period.
563
-
564
- **CI usage:**
565
-
566
- ```yaml
567
- - name: Trend report
568
- run: ado-sync trend --days 30 --fail-on-flaky --webhook-url ${{ secrets.SLACK_WEBHOOK }}
569
- env:
570
- AZURE_DEVOPS_TOKEN: ${{ secrets.AZURE_DEVOPS_TOKEN }}
571
- ```
572
-
573
- ---
574
-
575
- ## GitHub Actions templates
576
-
577
- Two ready-to-use workflow templates ship in `.github/workflows/`:
578
-
579
- ### `ado-sync-pr-check.yml`
580
-
581
- Runs `ado-sync push --dry-run` on every pull request and posts (or updates) a structured comment listing unlinked specs, drift, and conflicts.
582
-
583
- Required secrets: `ADO_PAT`
584
-
585
- ```yaml
586
- # .github/workflows/ado-sync-pr-check.yml (already present in this repo)
587
- on:
588
- pull_request:
589
- types: [opened, synchronize, reopened]
590
- ```
591
-
592
- ### `ado-sync-coverage-gate.yml`
593
-
594
- Runs three checks on push/PR to main:
595
- 1. **Spec link rate gate** — `ado-sync coverage --fail-below N`
596
- 2. **AC gate** — `ado-sync ac-gate` scoped to an area path
597
- 3. **Trend report** — informational, posts to webhook if configured
598
-
599
- Required secrets: `ADO_PAT`
600
-
601
- Key repository variables:
602
-
603
- | Variable | Default | Description |
604
- |----------|---------|-------------|
605
- | `ADO_SYNC_COVERAGE_MIN` | `80` | Minimum spec link rate % |
606
- | `ADO_SYNC_AC_AREA_PATH` | *(all)* | Area path scope for AC gate |
607
- | `ADO_SYNC_TREND_DAYS` | `30` | Days of history for trend report |
608
- | `ADO_SYNC_TREND_WEBHOOK` | | Webhook URL for trend summary |
609
- | `ADO_SYNC_TREND_WEBHOOK_TYPE` | `slack` | `slack`, `teams`, or `generic` |