ado-sync 0.1.23 → 0.1.26

package/docs/publish-test-results.md

@@ -0,0 +1,119 @@
+# publish-test-results
+
+Parses test result files (TRX, JUnit, Cucumber JSON) and publishes them to an Azure DevOps Test Run, linking results back to Test Cases via the sync tag.
+
+---
+
+## Usage
+
+```bash
+ado-sync publish-test-results \
+  --testResult results/test-results.trx \
+  --runName "CI run #42"
+
+# Multiple result files
+ado-sync publish-test-results \
+  --testResult results/unit.trx \
+  --testResult results/integration.xml \
+  --testResultFormat junit
+
+# Dry run — parse and summarise without publishing
+ado-sync publish-test-results --testResult results/test.trx --dry-run
+
+# Associate with a build
+ado-sync publish-test-results \
+  --testResult results/test.trx \
+  --buildId 12345
+```
+
+### Options
+
+| Option | Description |
+|--------|-------------|
+| `--testResult <path>` | Path to a result file. Repeatable. |
+| `--testResultFormat <format>` | `trx` · `junit` · `cucumberJson`. Auto-detected from file extension/content when omitted. |
+| `--runName <name>` | Name for the Test Run in Azure DevOps. Defaults to `ado-sync <ISO timestamp>`. |
+| `--buildId <id>` | Build ID to associate with the Test Run. |
+| `--dry-run` | Parse results and print summary without creating a run in Azure. |
+| `--config-override` | Override config values (repeatable, same as other commands). |
+
+---
+
+## Supported formats
+
+| Format | Extension | Auto-detected |
+|--------|-----------|---------------|
+| TRX (Visual Studio / .NET) | `.trx` | Yes (XML root `<TestRun>`) |
+| JUnit XML | `.xml` | Yes (XML root `<testsuites>` or `<testsuite>`) |
+| Cucumber JSON | `.json` | Yes (JSON array of feature objects) |
+
+---
+
+## Outcome mapping
+
+| Source outcome | Azure outcome |
+|----------------|---------------|
+| `passed` / `pass` / `success` | `Passed` |
+| `failed` / `fail` / `failure` / `error` | `Failed` |
+| `skipped` / `ignored` / `pending` / `notExecuted` | `NotExecuted` |
+| `inconclusive` | `Inconclusive` (or override with `treatInconclusiveAs`) |
+
+---
+
+## Configuration
+
+Results can also be configured in the config file under `publishTestResults`:
+
+```json
+{
+  "publishTestResults": {
+    "testResult": {
+      "sources": [
+        { "value": "results/unit.trx",        "format": "trx" },
+        { "value": "results/integration.xml", "format": "junit" }
+      ]
+    },
+    "treatInconclusiveAs": "Failed",
+    "testRunSettings": {
+      "name": "My CI Run",
+      "comment": "Automated sync run",
+      "runType": "Automated"
+    },
+    "testResultSettings": {
+      "comment": "Published by ado-sync"
+    },
+    "testConfiguration": {
+      "name": "Default"
+    }
+  }
+}
+```
+
+### `publishTestResults` fields
+
+| Field | Description |
+|-------|-------------|
+| `testResult.sources` | Array of `{ value, format }` objects. `value` is a path relative to config dir. |
+| `treatInconclusiveAs` | Override inconclusive outcome. e.g. `"Failed"` or `"NotExecuted"`. |
+| `flakyTestOutcome` | How to handle flaky tests: `"lastAttemptOutcome"` *(default)* · `"firstAttemptOutcome"` · `"worstOutcome"`. |
+| `testConfiguration.name` | Name of the Azure test configuration to associate. |
+| `testConfiguration.id` | ID of the Azure test configuration. |
+| `testRunSettings.name` | Name for the Test Run. |
+| `testRunSettings.comment` | Comment attached to the Test Run. |
+| `testRunSettings.runType` | `"Automated"` *(default)* · `"Manual"`. |
+| `testResultSettings.comment` | Comment applied to every test result. |
+| `publishAttachmentsForPassingTests` | `"none"` *(default)* · `"files"` · `"all"`. |
+
+---
+
+## Output
+
+```
+ado-sync publish-test-results
+Config: ado-sync.json
+
+Total results: 42
+  38 passed  3 failed  1 other
+Run ID: 9876
+URL: https://dev.azure.com/my-org/MyProject/_testManagement/runs?runId=9876
+```

package/docs/spec-formats.md

@@ -0,0 +1,225 @@
+# Spec file formats
+
+---
+
+## Gherkin `.feature`
+
+Set `local.type: "gherkin"`. Standard Gherkin syntax is supported:
+
+- `Feature` / `Scenario` / `Scenario Outline` / `Background`
+- `Given` / `When` / `Then` / `And` / `But`
+- Feature-level and scenario-level tags
+- `Scenario Outline` with `Examples` tables — creates a **single** parametrized Test Case in Azure, not one TC per row
+- Inline data tables (synced as sub-steps or plain text — see [Format configuration](advanced.md#format-configuration))
+
+```gherkin
+Feature: Checkout
+
+  Background:
+    Given I am on the storefront
+
+  @smoke
+  @tc:1041
+  Scenario: Add item and complete checkout
+    Given I am logged in as "standard_user"
+    When I add "Sauce Labs Backpack" to the cart
+    And I proceed through checkout with name "Test User" and zip "12345"
+    Then I see the order confirmation page
+
+  @tc:1042
+  Scenario Outline: Checkout with different users
+    Given I am logged in as "<user>"
+    When I complete a checkout
+    Then the result is "<result>"
+
+    Examples:
+      | user                    | result  |
+      | standard_user           | success |
+      | performance_glitch_user | success |
+```
+
+---
+
+## Markdown `.md`
+
+Set `local.type: "markdown"`. Each `### heading` is one test case. Scenarios are separated by `---`.
+
+```markdown
+# My Feature Test Plan
+
+## Test scenarios
+
+### Login with valid credentials
+@tc:1042 @smoke
+
+Assumption: Fresh browser session.
+
+Steps:
+1. Navigate to https://example.com/login
+2. Enter username "admin" and password "secret"
+3. Click the Login button
+
+Expected results:
+- The dashboard page is shown
+- The username appears in the top navigation
+
+---
+
+### Login with invalid credentials
+
+Steps:
+1. Navigate to https://example.com/login
+2. Enter username "wrong" and password "wrong"
+3. Click the Login button
+
+Expected results:
+- An error message "Invalid credentials" is displayed
+- The user remains on the login page
+
+---
+```
+
+Sections recognised (case-insensitive): `Steps:`, `Expected results:`. All other prose is captured as the test case description. Heading number prefixes (`### 1. Title` or `### Title`) are both supported.
+
+### Playwright test-plan markdown
+
+Markdown files generated by the Playwright MCP agent are fully supported. Set `local.type: "markdown"` and point `local.include` at the generated files.
+
+---
+
+## CSV `.csv`
+
+Set `local.type: "csv"` to parse Azure DevOps / SpecSync tabular CSV exports.
+
+**Expected column layout (9 columns):**
+
+| Col | Field | Description |
+|-----|-------|-------------|
+| A (0) | ID | Azure Test Case ID — empty for new, filled after first push |
+| B (1) | Work Item Type | Always `Test Case` (ignored) |
+| C (2) | Title | `Scenario: My test` — non-empty on header row, empty on step rows |
+| D (3) | Test Step | Step number — empty on header row |
+| E (4) | Step Action | Step text, optionally prefixed with a Gherkin keyword |
+| F (5) | Step Expected | Expected result text (optional) |
+| G–I (6–8) | Area Path, Assigned To, State | Preserved but not used |
+
+The first row (column headers) is always skipped automatically.
+
+> **Note:** `ado-sync pull` is **not supported** for CSV files. Only push is supported.
+
+---
+
+## Excel `.xlsx`
+
+Set `local.type: "excel"`. Uses the same 9-column layout as CSV. ado-sync reads the raw worksheet XML (no external xlsx library) and handles both `t="str"` and `t="inlineStr"` cells from Azure DevOps exports.
+
+> **Note:** `ado-sync pull` is **not supported** for Excel files. Only push is supported.
+
+---
+
+## ID tags
+
+After a first push, ado-sync writes the Azure TC ID back into the local file.
+
+| Format | Location |
+|--------|----------|
+| Gherkin | `@tc:12345` tag on its own line above the `Scenario:` line |
+| Markdown | `@tc:12345` tag on the line immediately after the `### heading` |
+| CSV | Numeric ID in column A of the matching title row |
+| Excel | Numeric ID in cell A of the matching title row |
+
+### Custom prefix
+
+```json
+{ "sync": { "tagPrefix": "azure" } }
+```
+
+Result: `@azure:12345` in both Gherkin and Markdown files.
+
+> **Warning:** Changing the prefix on an existing project means existing ID tags are no longer recognised. Do a project-wide find-and-replace before changing.
+
+---
+
+## Tag filtering
+
+All commands accept `--tags` to limit which scenarios are processed. Uses the standard Cucumber tag expression language.
+
+```bash
+ado-sync push --tags "@smoke"
+ado-sync push --tags "@smoke and not @wip"
+ado-sync pull --tags "@regression or @critical"
+```
+
+Tags are evaluated against all tags on a scenario, including:
+
+- Tags on the `Feature` block (Gherkin)
+- Tags on the `Scenario` / `Scenario Outline` block
+- Tags on individual `Examples` tables
+- Tags on the Markdown tag line (same `### heading` line or line below it)
+- **Path-based auto-tags** — directory segments starting with `@` are automatically applied as tags:
+
+  ```
+  specs/
+    @smoke/
+      login.feature       ← all scenarios get tag 'smoke'
+    @regression/
+      @slow/
+        checkout.feature  ← all scenarios get tags 'regression' and 'slow'
+  ```
+
+### `local.condition` — permanent filter
+
+Use `local.condition` in config to permanently exclude scenarios without needing `--tags` on every command:
+
+```json
+{ "local": { "condition": "@done and not (@ignored or @planned)" } }
+```
+
+This filter is applied before `--tags`, so it acts as a baseline inclusion gate.
+
+---
+
+## Work item linking
+
+Tags matching a configured `links` prefix are synced as Azure DevOps work item relations on the Test Case.
+
+### Config
+
+```json
+{
+  "sync": {
+    "links": [
+      { "prefix": "story",  "relationship": "System.LinkTypes.Related" },
+      { "prefix": "bug",    "relationship": "System.LinkTypes.Related" },
+      { "prefix": "req",    "relationship": "Microsoft.VSTS.Common.TestedBy-Reverse" }
+    ]
+  }
+}
+```
+
+### Usage in Gherkin
+
+```gherkin
+  @tc:1042 @story:555 @bug:789
+  Scenario: User can add items to cart
+    ...
+```
+
+### Usage in Markdown
+
+```markdown
+### User can add items to cart
+@tc:1042 @story:555 @bug:789
+
+Steps:
+1. Add an item to the cart
+...
+```
+
+On each `push`, relations are synced: new links are added, stale links (whose tag was removed) are deleted. The `relationship` value is the ADO relation type — `"System.LinkTypes.Related"` is the default if omitted.
+
+---
+
+## Attachments
+
+Files can be attached to Azure Test Cases via tags. See [Attachments](advanced.md#attachments).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ado-sync",
-  "version": "0.1.23",
+  "version": "0.1.26",
   "description": "Bidirectional sync between local test specs (Cucumber/Markdown) and Azure DevOps Test Cases",
   "bin": {
     "ado-sync": "./dist/cli.js"