ado-sync 0.1.23 → 0.1.26

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  Bidirectional sync between local test specs and Azure DevOps Test Cases.
4
4
 
5
- Supports **Cucumber / Gherkin** `.feature` files, **prose Markdown** `.md` spec files (including Playwright test-plan markdown generated by the Playwright MCP agent), and **Azure DevOps tabular exports** in **CSV** and **Excel** (`.xlsx`) format.
5
+ Supports **Cucumber / Gherkin** `.feature` files, **prose Markdown** `.md` spec files (including Playwright test-plan markdown), and **Azure DevOps tabular exports** in **CSV** and **Excel** (`.xlsx`) format.
6
6
  Inspired by [SpecSync](https://docs.specsolutions.eu/specsync/).
7
7
 
8
8
  ---
@@ -17,10 +17,10 @@ Local files ado-sync Azure DevOps
17
17
  .csv files ── push ──► (push-only)
18
18
  .xlsx files ── push ──► (push-only)
19
19
  write ID back
20
- @tc:12345 / <!-- tc: 12345 --> / col A
20
+ @tc:12345 / col A (csv/xlsx)
21
21
  ```
22
22
 
23
- On the **first push** of a scenario, a new Test Case is created in Azure DevOps and its ID is written back into the local file as a tag or comment. Every subsequent push uses that ID to update the existing Test Case. Pulling fetches the latest title and steps from Azure and overwrites the local file.
23
+ On the **first push** of a scenario, a new Test Case is created in Azure DevOps and its ID is written back into the local file as a tag. Every subsequent push uses that ID to update the existing Test Case. Pulling fetches the latest title and steps from Azure and overwrites the local file.
24
24
 
25
25
  ---
26
26
 
@@ -28,7 +28,7 @@ On the **first push** of a scenario, a new Test Case is created in Azure DevOps
28
28
 
29
29
  ```bash
30
30
  npm install -g ado-sync
31
- # or use locally without installing
31
+ # or run without installing
32
32
  npx ado-sync --help
33
33
  ```
34
34
 
@@ -36,51 +36,33 @@ npx ado-sync --help
36
36
 
37
37
  ## Quick start
38
38
 
39
- ### 1. Generate a config file
40
-
41
39
  ```bash
40
+ # 1. Generate a config file
42
41
  ado-sync init # creates ado-sync.json
43
- ado-sync init ado-sync.yml # creates ado-sync.yml (YAML format)
42
+ ado-sync init ado-sync.yml # YAML format
43
+
44
+ # 2. Edit the config with your org, project, plan ID, and token
45
+ export AZURE_DEVOPS_TOKEN=your_personal_access_token
46
+
47
+ # 3. Preview what will be created
48
+ ado-sync push --dry-run
49
+
50
+ # 4. Push to Azure DevOps
51
+ ado-sync push
44
52
  ```
45
53
 
46
- ### 2. Edit the config
54
+ Minimal config:
47
55
 
48
56
  ```json
49
57
  {
50
58
  "orgUrl": "https://dev.azure.com/my-org",
51
59
  "project": "MyProject",
52
- "auth": {
53
- "type": "pat",
54
- "token": "$AZURE_DEVOPS_TOKEN"
55
- },
56
- "testPlan": {
57
- "id": 1234,
58
- "suiteId": 5678
59
- },
60
- "local": {
61
- "type": "gherkin",
62
- "include": "specs/**/*.feature"
63
- },
64
- "sync": {
65
- "tagPrefix": "tc"
66
- }
60
+ "auth": { "type": "pat", "token": "$AZURE_DEVOPS_TOKEN" },
61
+ "testPlan": { "id": 1234 },
62
+ "local": { "type": "gherkin", "include": "specs/**/*.feature" }
67
63
  }
68
64
  ```
69
65
 
70
- ### 3. Set your token
71
-
72
- ```bash
73
- export AZURE_DEVOPS_TOKEN=your_personal_access_token
74
- ```
75
-
76
- ### 4. Push
77
-
78
- ```bash
79
- ado-sync push
80
- ```
81
-
82
- New scenarios are created in Azure DevOps. Their IDs are written back into your local files automatically.
83
-
84
66
  ---
85
67
 
86
68
  ## CLI reference
@@ -94,49 +76,36 @@ Options:
94
76
  -h, --help Show help
95
77
 
96
78
  Commands:
97
- init [output] Generate a starter config file
98
- push [options] Push local specs to Azure DevOps
99
- pull [options] Pull updates from Azure DevOps into local files
100
- status [options] Show what would change without making any modifications
101
- help [command] Help for a specific command
79
+ init [output] Generate a starter config file
80
+ push [options] Push local specs to Azure DevOps
81
+ pull [options] Pull updates from Azure DevOps into local files
82
+ status [options] Show diff without making changes
83
+ publish-test-results [opts] Publish TRX / JUnit / Cucumber JSON results to Azure DevOps
84
+ help [command] Help for a specific command
102
85
  ```
103
86
 
104
87
  ### `init`
105
88
 
106
89
  ```bash
107
90
  ado-sync init # creates ado-sync.json
108
- ado-sync init ado-sync.yml # creates ado-sync.yml
109
- ado-sync init path/to/config.json
91
+ ado-sync init ado-sync.yml # YAML format
110
92
  ```
111
93
 
112
- Creates a starter config file in JSON or YAML format depending on the file extension. Safe — will not overwrite an existing file.
113
-
114
- ---
115
-
116
94
  ### `push`
117
95
 
118
96
  ```bash
119
97
  ado-sync push
120
98
  ado-sync push --dry-run
121
- ado-sync push --tags "@smoke"
122
99
  ado-sync push --tags "@smoke and not @wip"
123
100
  ado-sync push --config-override testPlan.id=9999
124
- ado-sync -c other-config.json push
125
101
  ```
126
102
 
127
- For every scenario / heading in your local spec files:
128
-
129
103
  | Scenario state | Action |
130
104
  |----------------|--------|
131
- | No ID tag yet | Creates a new Test Case, writes ID back to file |
132
- | Has ID tag, no changes | Skipped (uses local cache for speed) |
133
- | Has ID tag, title or steps changed | Updates the existing Test Case |
105
+ | No ID tag | Creates a new Test Case, writes ID back |
106
+ | ID tag, no changes | Skipped |
107
+ | ID tag, content changed | Updates the existing Test Case |
134
108
  | Deleted locally, still in Azure suite | Tagged `ado-sync:removed` in Azure |
135
- | Has ID tag but Test Case deleted in Azure | Reported as error |
136
-
137
- `--dry-run` prints what would happen without modifying anything.
138
-
139
- ---
140
109
 
141
110
  ### `pull`
142
111
 
@@ -144,639 +113,37 @@ For every scenario / heading in your local spec files:
144
113
  ado-sync pull
145
114
  ado-sync pull --dry-run
146
115
  ado-sync pull --tags "@smoke"
147
- ado-sync pull --config-override sync.conflictAction=skip
148
116
  ```
149
117
 
150
- For every locally-linked scenario (has an ID tag):
151
-
152
- | Remote state | Action |
153
- |--------------|--------|
154
- | Title, steps, or description changed in Azure | Updates the local file |
155
- | No changes | Skipped |
156
- | Test Case not found | Reported as error |
157
-
158
- `--dry-run` prints what would change without modifying local files.
159
-
160
- ---
161
-
162
118
  ### `status`
163
119
 
164
120
  ```bash
165
121
  ado-sync status
166
122
  ado-sync status --tags "@smoke"
167
- ado-sync status --config-override testPlan.id=9999
168
123
  ```
169
124
 
170
125
  Compares local specs against Azure DevOps and prints a diff — no changes made.
171
126
 
172
- ---
173
-
174
- ### `--config-override`
175
-
176
- All commands accept one or more `--config-override path=value` options to set config values from the command line without editing the file. Useful in CI pipelines.
127
+ ### `publish-test-results`
177
128
 
178
129
  ```bash
179
- # Override test plan ID
180
- ado-sync push --config-override testPlan.id=9999
181
-
182
- # Override tag prefix
183
- ado-sync push --config-override sync.tagPrefix=azure
184
-
185
- # Multiple overrides (repeat the flag)
186
- ado-sync push \
187
- --config-override testPlan.id=9999 \
188
- --config-override sync.disableLocalChanges=true
189
- ```
190
-
191
- Dot-path notation is used for nested fields. Numbers and booleans are coerced automatically.
192
-
193
- ---
194
-
195
- ## Config file reference
196
-
197
- Config files can be JSON (`.json`) or YAML (`.yml` / `.yaml`). Run `ado-sync init` to generate a template.
198
-
199
- ### Full example (JSON)
200
-
201
- ```json
202
- {
203
- "orgUrl": "https://dev.azure.com/YOUR_ORG",
204
- "project": "YOUR_PROJECT",
205
-
206
- "auth": {
207
- "type": "pat",
208
- "token": "$AZURE_DEVOPS_TOKEN"
209
- },
210
-
211
- "testPlan": {
212
- "id": 1234,
213
- "suiteId": 5678,
214
- "suiteMapping": "flat"
215
- },
216
-
217
- "local": {
218
- "type": "gherkin",
219
- "include": "specs/**/*.feature",
220
- "exclude": []
221
- },
222
-
223
- "sync": {
224
- "tagPrefix": "tc",
225
- "titleField": "System.Title",
226
- "areaPath": "MyProject\\Team A",
227
- "iterationPath": "MyProject\\Sprint 1",
228
- "disableLocalChanges": false,
229
- "conflictAction": "overwrite",
230
- "links": [
231
- { "prefix": "story", "relationship": "System.LinkTypes.Related" },
232
- { "prefix": "bug", "relationship": "System.LinkTypes.Related" }
233
- ]
234
- }
235
- }
236
- ```
237
-
238
- ---
239
-
240
- ### `orgUrl`
241
-
242
- Azure DevOps organisation URL. Format: `https://dev.azure.com/YOUR_ORG`.
243
-
244
- ---
245
-
246
- ### `project`
247
-
248
- Azure DevOps project name.
249
-
250
- ---
251
-
252
- ### `auth`
253
-
254
- | Field | Description |
255
- |-------|-------------|
256
- | `type` | `"pat"` · `"accessToken"` · `"managedIdentity"` |
257
- | `token` | PAT or access token value. Prefix with `$` to read from an environment variable (e.g. `"$AZURE_DEVOPS_TOKEN"`). |
258
- | `applicationIdURI` | Required only when `type` is `"managedIdentity"`. |
259
-
260
- **PAT permissions required:** Test Management (Read & Write), Work Items (Read & Write).
261
-
262
- ---
263
-
264
- ### `testPlan`
265
-
266
- | Field | Default | Description |
267
- |-------|---------|-------------|
268
- | `id` | *(required)* | ID of the Azure DevOps Test Plan. |
269
- | `suiteId` | plan root | ID of the Test Suite within the plan. Defaults to the plan's root suite. |
270
- | `suiteMapping` | `"flat"` | `"flat"` — all Test Cases go into one suite. `"byFolder"` — folder structure is mirrored as nested child suites (e.g. `specs/login/` → suite named `login`). |
271
-
272
- ---
273
-
274
- ### `testPlans` (multi-plan)
275
-
276
- To sync the same repository against multiple Test Plans, use the `testPlans` array instead of (or in addition to) `testPlan`. Each entry overrides `testPlan` and can specify its own file globs.
277
-
278
- ```json
279
- {
280
- "testPlans": [
281
- {
282
- "id": 1001,
283
- "suiteId": 2001,
284
- "include": "specs/smoke/**/*.feature"
285
- },
286
- {
287
- "id": 1002,
288
- "suiteId": 2002,
289
- "include": "specs/regression/**/*.feature",
290
- "suiteMapping": "byFolder"
291
- }
292
- ]
293
- }
294
- ```
295
-
296
- | Field | Description |
297
- |-------|-------------|
298
- | `id` | Test Plan ID |
299
- | `suiteId` | *(Optional)* Target suite ID |
300
- | `suiteMapping` | *(Optional)* `"flat"` or `"byFolder"` |
301
- | `include` | *(Optional)* Override `local.include` for this plan |
302
- | `exclude` | *(Optional)* Override `local.exclude` for this plan |
303
-
304
- ---
305
-
306
- ### `local`
307
-
308
- | Field | Description |
309
- |-------|-------------|
310
- | `type` | `"gherkin"` · `"markdown"` · `"csv"` · `"excel"` — see [Spec file formats](#spec-file-formats) |
311
- | `include` | Glob pattern(s) relative to the config file. String or array. |
312
- | `exclude` | *(Optional)* Glob pattern(s) to exclude. String or array. |
313
-
314
- ---
315
-
316
- ### `sync`
317
-
318
- | Field | Default | Description |
319
- |-------|---------|-------------|
320
- | `tagPrefix` | `"tc"` | Prefix used in ID tags. See [ID tags](#id-tags). |
321
- | `titleField` | `"System.Title"` | Azure DevOps field used as the test case title. |
322
- | `areaPath` | *(none)* | Area path for newly created Test Cases. |
323
- | `iterationPath` | *(none)* | Iteration path for newly created Test Cases. |
324
- | `disableLocalChanges` | `false` | When `true`, no local files are modified — no ID writeback, no pull apply. Use in CI pipelines that should not commit file changes. |
325
- | `conflictAction` | `"overwrite"` | How to handle conflicts (remote changed since last sync AND local also differs). `"overwrite"` — push local version. `"skip"` — emit a conflict result, leave both sides unchanged. `"fail"` — throw an error listing all conflicts. |
326
- | `links` | `[]` | Work item link configurations. See [Work item linking](#work-item-linking). |
327
-
328
- ---
329
-
330
- ## ID tags
331
-
332
- After a scenario is pushed for the first time, ado-sync writes the Azure Test Case ID back into the local file. The format depends on the spec type.
333
-
334
- | Format | Writeback location |
335
- |--------|--------------------|
336
- | Gherkin | `@tc:12345` tag above the `Scenario:` line |
337
- | Markdown | `<!-- tc: 12345 -->` comment after the `### heading` |
338
- | CSV | Numeric ID in column A of the matching title row |
339
- | Excel | Numeric ID in cell A of the matching title row |
340
-
341
- ### Gherkin
342
-
343
- The ID tag is inserted on its own line immediately above the `Scenario:` or `Scenario Outline:` keyword:
344
-
345
- ```gherkin
346
- Feature: Login
347
-
348
- @tc:1042
349
- Scenario: Successful login with valid credentials
350
- Given I am on the login page
351
- When I enter username "standard_user" and password "secret_sauce"
352
- Then I am redirected to the inventory page
353
- ```
354
-
355
- For `Scenario Outline`, a **single** parametrized Test Case is created in Azure with a data table — one `@tc:ID` tag for the whole outline, not one per example row.
356
-
357
- ```gherkin
358
- @tc:1043
359
- Scenario Outline: Login with different roles
360
- Given I am logged in as "<role>"
361
- Then I can see "<dashboard>"
362
-
363
- Examples:
364
- | role | dashboard |
365
- | admin | Admin view |
366
- | tester | Test view |
367
- ```
368
-
369
- ### Markdown
370
-
371
- The ID comment is inserted on the line immediately after the `### heading`:
372
-
373
- ```markdown
374
- ### 1. Login (happy path)
375
- <!-- tc: 1042 -->
376
-
377
- Assumption: Fresh browser session.
378
-
379
- Steps:
380
- 1. Enter username `standard_user` in the Username field.
381
- 2. Enter password `secret_sauce` in the Password field.
382
- 3. Click `Login`.
383
-
384
- Expected results:
385
- - User is redirected to `/inventory.html`.
386
- - Product list is visible.
387
-
388
- ---
389
- ```
390
-
391
- The comment is invisible when the Markdown is rendered.
392
-
393
- ### Custom prefix
394
-
395
- Change the `sync.tagPrefix` in your config to use a different prefix:
396
-
397
- ```json
398
- { "sync": { "tagPrefix": "azure" } }
399
- ```
400
-
401
- Result:
402
- - Gherkin: `@azure:1042`
403
- - Markdown: `<!-- azure: 1042 -->`
404
-
405
- > **Warning:** Changing the prefix on an existing project means all existing ID tags will no longer be recognised. Do a project-wide find-and-replace on the old prefix before changing it.
406
-
407
- ---
408
-
409
- ## Tag filtering
410
-
411
- All commands accept a `--tags` option to limit which scenarios are synced. The syntax is the standard Cucumber tag expression language.
412
-
413
- ```bash
414
- # Only sync scenarios tagged @smoke
415
- ado-sync push --tags "@smoke"
416
-
417
- # Sync everything except @wip
418
- ado-sync push --tags "not @wip"
419
-
420
- # Combine tags with and / or
421
- ado-sync push --tags "@smoke and not @slow"
422
- ado-sync pull --tags "@regression or @critical"
423
- ```
424
-
425
- Tags are evaluated against all tags on a scenario, including:
426
-
427
- - Tags inherited from the Feature block (Gherkin)
428
- - Tags on the Scenario / Scenario Outline block
429
- - Tags on individual Examples tables
430
- - Inline `<!-- tags: -->` comments in Markdown (see below)
431
- - **Path-based auto-tags** — directory segments prefixed with `@` are automatically applied as tags to all scenarios in that directory:
432
-
433
- ```
434
- specs/
435
- @smoke/
436
- login.feature ← all scenarios get tag 'smoke'
437
- @regression/
438
- @slow/
439
- checkout.feature ← all scenarios get tags 'regression' and 'slow'
440
- ```
441
-
442
- ```bash
443
- ado-sync push --tags "@smoke" # only push specs/@smoke/** scenarios
444
- ```
445
-
446
- ### Tags in Markdown
447
-
448
- Add a `<!-- tags: -->` HTML comment anywhere inside a scenario block to assign tags:
449
-
450
- ```markdown
451
- ### Login with valid credentials
452
- <!-- tc: 1042 -->
453
- <!-- tags: @smoke, @regression -->
454
-
455
- Steps:
456
- 1. Navigate to the login page
457
- 2. Enter valid credentials
458
- 3. Click Login
459
-
460
- Expected results:
461
- - Dashboard is shown
462
-
463
- ---
464
- ```
465
-
466
- Path-based auto-tagging also works for Markdown files — placing them inside an `@smoke/` folder automatically adds the `smoke` tag.
467
-
468
- ---
469
-
470
- ## Spec file formats
471
-
472
- ### Gherkin `.feature`
473
-
474
- Standard Gherkin syntax is supported, including:
475
- - `Feature` / `Scenario` / `Scenario Outline`
476
- - `Given` / `When` / `Then` / `And` / `But`
477
- - Feature-level and scenario-level tags
478
- - `Scenario Outline` with `Examples` tables — creates a **single** parametrized Test Case in Azure (data table included), not one TC per row
479
-
480
- ```gherkin
481
- Feature: Checkout
482
-
483
- @smoke
484
- Scenario: Add item and complete checkout
485
- Given I am logged in as "standard_user"
486
- When I add "Sauce Labs Backpack" to the cart
487
- And I proceed through checkout with name "Test User" and zip "12345"
488
- Then I see the order confirmation page
489
-
490
- Scenario Outline: Checkout with different users
491
- Given I am logged in as "<user>"
492
- When I complete a checkout
493
- Then the result is "<result>"
494
-
495
- Examples:
496
- | user | result |
497
- | standard_user | success |
498
- | performance_glitch_user | success |
499
- ```
500
-
501
- ### Markdown `.md`
502
-
503
- Each `### heading` is treated as one test case. The file can contain any number of scenarios separated by `---`.
504
-
505
- ```markdown
506
- # My Feature Test Plan
507
-
508
- ## Test scenarios
509
-
510
- ### Login with valid credentials
511
- <!-- tags: @smoke -->
512
-
513
- Assumption: Fresh browser session.
514
-
515
- Steps:
516
- 1. Navigate to https://example.com/login
517
- 2. Enter username "admin" and password "secret"
518
- 3. Click the Login button
519
-
520
- Expected results:
521
- - The dashboard page is shown
522
- - The username appears in the top navigation
523
-
524
- ---
525
-
526
- ### Login with invalid credentials
527
-
528
- Steps:
529
- 1. Navigate to https://example.com/login
530
- 2. Enter username "wrong" and password "wrong"
531
- 3. Click the Login button
532
-
533
- Expected results:
534
- - An error message "Invalid credentials" is displayed
535
- - The user remains on the login page
536
-
537
- ---
538
- ```
539
-
540
- Sections recognised: `Steps:`, `Expected results:` (case-insensitive). All other prose is captured as the test case description. Heading number prefixes (`### 1. Title` or `### Title`) are both supported.
541
-
542
- #### Playwright test-plan markdown
543
-
544
- Markdown `.md` files generated by the [Playwright MCP agent](https://playwright.dev/docs/test-agents) are fully supported as a local spec source. Set `local.type: "markdown"` and point `local.include` at the generated files to sync them into Azure DevOps as Test Cases.
545
-
546
- ---
547
-
548
- ### CSV `.csv`
549
-
550
- Set `local.type: "csv"` to parse Azure DevOps / SpecSync tabular CSV exports.
551
-
552
- **Expected column layout (9 columns):**
553
-
554
- | Col | Field | Description |
555
- |-----|-------|-------------|
556
- | A (0) | ID | Azure Test Case ID — empty for new cases, filled in after first push |
557
- | B (1) | Work Item Type | Always `Test Case` (ignored) |
558
- | C (2) | Title | `Scenario: My test` or just `My test` — non-empty on the header row, empty on step rows |
559
- | D (3) | Test Step | Step number (1, 2, …) — empty on the header row |
560
- | E (4) | Step Action | Step text, optionally prefixed with a Gherkin keyword (`Given`, `When`, `Then`, …) |
561
- | F (5) | Step Expected | Expected result text (optional) |
562
- | G–I (6–8) | Area Path, Assigned To, State | Preserved but not used during parse |
563
-
564
- **Example CSV:**
565
-
566
- ```csv
567
- ID,Work Item Type,Title,Test Step,Step Action,Step Expected,Area Path,Assigned To,State
568
- "","Test Case","Scenario: Login happy path",,,,...
569
- ,,,,,"1","Given I open the app","",,...
570
- ,,,,,"2","When I enter valid credentials","Then I see the dashboard",...
571
- "","Test Case","Scenario: Login with bad password",,,,...
572
- ,,,,,"1","Given I enter wrong credentials","Then I see an error",...
130
+ ado-sync publish-test-results --testResult results/test.trx
131
+ ado-sync publish-test-results --testResult results/test.xml --testResultFormat junit --dry-run
573
132
  ```
574
133
 
575
- The first row (column headers) is always skipped automatically.
576
-
577
- **ID writeback:** After the first push, the numeric TC ID is written into **column A** of the matching title row.
578
-
579
- > **Note:** Pull (`ado-sync pull`) is **not supported** for CSV files — these files are typically generated by external tools (e.g. the Playwright planner agent) and are not hand-authored. Only push is supported.
580
-
581
- **Config example:**
582
-
583
- ```json
584
- {
585
- "local": {
586
- "type": "csv",
587
- "include": "specs/**/*.csv"
588
- }
589
- }
590
- ```
591
-
592
- ---
593
-
594
- ### Excel `.xlsx`
595
-
596
- Set `local.type: "excel"` to parse Azure DevOps / SpecSync tabular Excel exports.
597
-
598
- The xlsx format uses the same 9-column layout as CSV above. ado-sync reads the raw worksheet XML directly (no heavyweight xlsx library required) and handles both plain-string (`t="str"`) and rich-text inline-string (`t="inlineStr"`) cells produced by Azure DevOps exports.
599
-
600
- **ID writeback:** After the first push, the numeric TC ID is written into **cell A** of the matching title row and the file is repacked in place.
601
-
602
- > **Note:** Pull (`ado-sync pull`) is **not supported** for Excel files for the same reason as CSV — these are externally generated files. Only push is supported.
603
-
604
- **Config example:**
605
-
606
- ```json
607
- {
608
- "local": {
609
- "type": "excel",
610
- "include": "specs/**/*.xlsx"
611
- }
612
- }
613
- ```
614
-
615
- ---
616
-
617
- ## Work item linking
618
-
619
- Tags matching a configured `links` prefix are turned into Azure DevOps work item relations on the Test Case.
620
-
621
- ### Config
622
-
623
- ```json
624
- {
625
- "sync": {
626
- "links": [
627
- { "prefix": "story", "relationship": "System.LinkTypes.Related" },
628
- { "prefix": "bug", "relationship": "System.LinkTypes.Related" },
629
- { "prefix": "req", "relationship": "Microsoft.VSTS.Common.TestedBy-Reverse" }
630
- ]
631
- }
632
- }
633
- ```
634
-
635
- ### Usage in Gherkin
636
-
637
- ```gherkin
638
- @tc:1042 @story:555 @bug:789
639
- Scenario: User can add items to cart
640
- ...
641
- ```
642
-
643
- ### Usage in Markdown
644
-
645
- ```markdown
646
- ### User can add items to cart
647
- <!-- tc: 1042 -->
648
- <!-- tags: @story:555, @bug:789 -->
649
- ```
650
-
651
- On each `push`, relations are synced: new links are added and stale links (whose tag was removed) are deleted. The `relationship` value is the ADO relation type; `"System.LinkTypes.Related"` is the default if omitted.
652
-
653
- ---
654
-
655
- ## Suite hierarchy
656
-
657
- By default, all Test Cases are created in a single flat suite (`suiteMapping: "flat"`). Setting `suiteMapping: "byFolder"` mirrors the folder structure of your spec files as nested child suites in Azure.
658
-
659
- ```json
660
- {
661
- "testPlan": {
662
- "id": 1234,
663
- "suiteId": 5678,
664
- "suiteMapping": "byFolder"
665
- }
666
- }
667
- ```
668
-
669
- ```
670
- specs/
671
- login/
672
- basic.feature → suite "login" → TC "Successful login"
673
- checkout/
674
- happy.feature → suite "checkout" → TC "Add item and checkout"
675
- ```
134
+ See [docs/publish-test-results.md](docs/publish-test-results.md) for full reference.
676
135
 
677
- Child suites are created automatically if they do not exist. The suite hierarchy is re-used across runs.
678
-
679
- ---
680
-
681
- ## Conflict detection
682
-
683
- ado-sync uses a local state cache (`.ado-sync-state.json`) to detect conflicts — cases where **both** the local file and the Azure Test Case were changed since the last sync.
684
-
685
- The `sync.conflictAction` setting controls what happens:
686
-
687
- | Value | Behaviour |
688
- |-------|-----------|
689
- | `"overwrite"` *(default)* | Push the local version to Azure, overwriting the remote change. |
690
- | `"skip"` | Emit a `!` conflict result for the scenario and leave both sides unchanged. |
691
- | `"fail"` | Throw an error listing all conflicting scenarios and abort the sync. |
692
-
693
- ```json
694
- { "sync": { "conflictAction": "skip" } }
695
- ```
696
-
697
- ---
698
-
699
- ## Local state cache
700
-
701
- ado-sync writes a `.ado-sync-state.json` file in the same directory as your config file. This cache stores SHA-256 hashes of the last-synced title, steps, and description for each Test Case, plus the Azure `changedDate` timestamp.
702
-
703
- **Commit this file to version control** so all team members and CI share the same last-synced state, enabling accurate conflict detection across machines.
704
-
705
- The cache also speeds up `push` — if neither the local file nor the Azure `changedDate` has changed since the last sync, the Azure API call is skipped entirely and the scenario is marked `skipped`.
706
-
707
- ---
708
-
709
- ## Build server / CI mode
710
-
711
- Set `sync.disableLocalChanges: true` to prevent ado-sync from writing back to local files. In this mode:
712
-
713
- - `push` — creates and updates Test Cases in Azure, but does **not** write ID tags back to local files.
714
- - `pull` — computes what would change but does **not** modify local files (behaves like `--dry-run`).
715
-
716
- ```json
717
- { "sync": { "disableLocalChanges": true } }
718
- ```
136
+ ### `--config-override`
719
137
 
720
- Or use `--config-override` per run:
138
+ All commands accept `--config-override path=value` to set config values without editing the file:
721
139
 
722
140
  ```bash
141
+ ado-sync push --config-override testPlan.id=9999
723
142
  ado-sync push --config-override sync.disableLocalChanges=true
724
143
  ```
725
144
 
726
145
  ---
727
146
 
728
- ## Removed scenario detection
729
-
730
- When a scenario is deleted from a local spec file but its Test Case still exists in the Azure suite, ado-sync detects this on the next `push` and appends the tag `ado-sync:removed` to the Azure Test Case (without deleting it). A `−` removed line is printed in the output.
731
-
732
- To completely remove the Test Case from Azure, delete it manually in Test Plans after reviewing.
733
-
734
- ---
735
-
736
- ## YAML config example
737
-
738
- ```yaml
739
- orgUrl: https://dev.azure.com/my-org
740
- project: MyProject
741
-
742
- auth:
743
- type: pat
744
- token: $AZURE_DEVOPS_TOKEN
745
-
746
- testPlan:
747
- id: 1234
748
- suiteId: 5678
749
- suiteMapping: byFolder
750
-
751
- local:
752
- type: gherkin
753
- include: specs/**/*.feature
754
- exclude:
755
- - specs/archive/**
756
-
757
- sync:
758
- tagPrefix: tc
759
- areaPath: "MyProject\\QA Team"
760
- conflictAction: skip
761
- disableLocalChanges: false
762
- links:
763
- - prefix: story
764
- relationship: System.LinkTypes.Related
765
- ```
766
-
767
- ---
768
-
769
- ## Environment variables
770
-
771
- | Variable | Description |
772
- |----------|-------------|
773
- | `AZURE_DEVOPS_TOKEN` | PAT or access token. Reference it in config with `"$AZURE_DEVOPS_TOKEN"`. |
774
- | Any name | Any env var can be used — set `auth.token` to `"$MY_VAR_NAME"`. |
775
-
776
- You can also use a `.env` file in the working directory. It is loaded automatically.
777
-
778
- ---
779
-
780
147
  ## Output symbols
781
148
 
782
149
  ```
@@ -791,68 +158,67 @@ You can also use a `.env` file in the working directory. It is loaded automatica
791
158
 
792
159
  ---
793
160
 
794
- ## Workflow examples
161
+ ## Documentation
795
162
 
796
- ### First-time setup
797
-
798
- ```bash
799
- # Generate config
800
- ado-sync init
163
+ | Topic | Link |
164
+ |-------|------|
165
+ | Full configuration reference | [docs/configuration.md](docs/configuration.md) |
166
+ | Spec file formats (Gherkin, Markdown, CSV, Excel) | [docs/spec-formats.md](docs/spec-formats.md) |
167
+ | Advanced features (format, state, fieldUpdates, customizations, attachments, CI mode) | [docs/advanced.md](docs/advanced.md) |
168
+ | Publishing test results | [docs/publish-test-results.md](docs/publish-test-results.md) |
801
169
 
802
- # Edit ado-sync.json with your org, project, plan ID, and token
803
- export AZURE_DEVOPS_TOKEN=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
804
-
805
- # Preview what will be created
806
- ado-sync push --dry-run
170
+ ---
807
171
 
808
- # Create test cases in Azure DevOps
809
- ado-sync push
810
- ```
172
+ ## Workflow examples
811
173
 
812
174
  ### Day-to-day: local changes first
813
175
 
814
176
  ```bash
815
- # Edit your .feature or .md files locally
816
- # Then push changes to Azure DevOps
177
+ # Edit your .feature or .md files, then push
817
178
  ado-sync push
818
179
  ```
819
180
 
820
181
  ### Day-to-day: Azure changes first
821
182
 
822
183
  ```bash
823
- # Someone edited a test case in Azure DevOps Test Plans UI
824
- # Pull the changes into your local files
184
+ # Someone edited a Test Case in the Azure DevOps UI
825
185
  ado-sync pull
826
186
  ```
827
187
 
828
- ### Check for drift before a PR
829
-
830
- ```bash
831
- ado-sync status
832
- ```
833
-
834
- ### CI pipeline (no file writeback)
188
+ ### CI pipeline
835
189
 
836
190
  ```yaml
837
- # GitHub Actions example
191
+ # GitHub Actions
838
192
  - name: Sync test cases to Azure DevOps
839
193
  run: ado-sync push --config-override sync.disableLocalChanges=true
840
194
  env:
841
195
  AZURE_DEVOPS_TOKEN: ${{ secrets.AZURE_DEVOPS_TOKEN }}
196
+
197
+ - name: Publish test results
198
+ run: ado-sync publish-test-results --testResult results/test.trx
199
+ env:
200
+ AZURE_DEVOPS_TOKEN: ${{ secrets.AZURE_DEVOPS_TOKEN }}
842
201
  ```
843
202
 
844
- ### Override plan per environment
203
+ ### Check for drift before a PR
845
204
 
846
205
  ```bash
847
- # Push to staging plan
848
- ado-sync push --config-override testPlan.id=2001
849
-
850
- # Push to production plan
851
- ado-sync push --config-override testPlan.id=3001
206
+ ado-sync status
852
207
  ```
853
208
 
854
209
  ---
855
210
 
211
+ ## Environment variables
212
+
213
+ | Variable | Description |
214
+ |----------|-------------|
215
+ | `AZURE_DEVOPS_TOKEN` | PAT or access token. Reference in config with `"$AZURE_DEVOPS_TOKEN"`. |
216
+ | Any name | Any env var — set `auth.token` to `"$MY_VAR_NAME"`. |
217
+
218
+ A `.env` file in the working directory is loaded automatically.
219
+
220
+ ---
221
+
856
222
  ## Troubleshooting
857
223
 
858
224
  **`No config file found`**
@@ -862,28 +228,25 @@ Run `ado-sync init` or pass `-c path/to/config.json`.
862
228
  Your config references `$X` in `auth.token` but the variable is not exported. Run `export X=...` or add it to a `.env` file.
863
229
 
864
230
  **Test Case created but ID not written back**
865
- Check that the local file is writable, or set `sync.disableLocalChanges: false`. On the next `push` it will try again.
231
+ Check that the file is writable, or that `sync.disableLocalChanges` is not `true`.
866
232
 
867
233
  **`Test case #N not found in Azure DevOps`**
868
234
  The test case was deleted in Azure. Remove the ID tag from the local file to recreate it, or restore the test case in Azure.
869
235
 
870
236
  **`Failed to parse <file>`**
871
- Gherkin syntax error in a `.feature` file. Run `npx cucumber-js --dry-run` to identify the problem line.
237
+ Gherkin syntax error. Run `npx cucumber-js --dry-run` to identify the problem line.
872
238
 
873
239
  **Changes not detected on push**
874
- The comparison uses title + steps + description. If only `areaPath` or `iterationPath` changed, touch any step to trigger an update, or use `--config-override` to set those fields and rely on the next sync.
240
+ The comparison uses title + steps + description. Touch any step to force an update, or reset the cache by deleting `.ado-sync-state.json`.
875
241
 
876
242
  **Conflict detected unexpectedly**
877
- Delete `.ado-sync-state.json` to reset the cache. The next push will re-populate it by fetching current state from Azure.
878
-
879
- **`init` generated JSON instead of YAML**
880
- Pass the filename as a positional argument: `ado-sync init ado-sync.yml` (not `--output`).
243
+ Delete `.ado-sync-state.json` to reset the cache. The next push re-populates it from Azure.
881
244
 
882
245
  **CSV/Excel IDs not written back**
883
- Ensure the file is not open in Excel or another application (which would lock it). Also check that `sync.disableLocalChanges` is not `true`.
246
+ Ensure the file is not open in another application. Check that `sync.disableLocalChanges` is not `true`.
884
247
 
885
248
  **Excel file not parsed / `No worksheet found`**
886
- ado-sync looks for `xl/worksheets/sheet.xml` or `xl/worksheets/sheet1.xml` inside the xlsx ZIP. Files exported directly from Azure DevOps or saved by the Playwright planner agent match this structure. Workbooks re-saved by Excel may use a different sheet name — export again from Azure DevOps to regenerate a compatible file.
249
+ ado-sync looks for `xl/worksheets/sheet.xml` or `xl/worksheets/sheet1.xml` inside the xlsx ZIP. Re-export from Azure DevOps to get a compatible file.
887
250
 
888
251
  **Pull has no effect on CSV/Excel files**
889
- Pull is not supported for CSV and Excel formats these files are managed by external tools. Use `push` to sync local changes to Azure DevOps.
252
+ Pull is not supported for CSV and Excel — only push. These formats are managed by external tools.