ado-sync 0.1.24 → 0.1.27

package/docs/spec-formats.md

@@ -0,0 +1,595 @@
+# Spec file formats
+
+---
+
+## Gherkin `.feature`
+
+Set `local.type: "gherkin"`. Standard Gherkin syntax is supported:
+
+- `Feature` / `Scenario` / `Scenario Outline` / `Background`
+- `Given` / `When` / `Then` / `And` / `But`
+- Feature-level and scenario-level tags
+- `Scenario Outline` with `Examples` tables — creates a **single** parametrized Test Case in Azure, not one TC per row
+- Inline data tables (synced as sub-steps or plain text — see [Format configuration](advanced.md#format-configuration))
+
+```gherkin
+Feature: Checkout
+
+  Background:
+    Given I am on the storefront
+
+  @smoke
+  @tc:1041
+  Scenario: Add item and complete checkout
+    Given I am logged in as "standard_user"
+    When I add "Sauce Labs Backpack" to the cart
+    And I proceed through checkout with name "Test User" and zip "12345"
+    Then I see the order confirmation page
+
+  @tc:1042
+  Scenario Outline: Checkout with different users
+    Given I am logged in as "<user>"
+    When I complete a checkout
+    Then the result is "<result>"
+
+    Examples:
+      | user                    | result  |
+      | standard_user           | success |
+      | performance_glitch_user | success |
+```
+
+---
+
+## Markdown `.md`
+
+Set `local.type: "markdown"`. Each `### heading` is one test case. Scenarios are separated by `---`.
+
+```markdown
+# My Feature Test Plan
+
+## Test scenarios
+
+### Login with valid credentials
+@tc:1042 @smoke
+
+Assumption: Fresh browser session.
+
+Steps:
+1. Navigate to https://example.com/login
+2. Enter username "admin" and password "secret"
+3. Click the Login button
+
+Expected results:
+- The dashboard page is shown
+- The username appears in the top navigation
+
+---
+
+### Login with invalid credentials
+
+Steps:
+1. Navigate to https://example.com/login
+2. Enter username "wrong" and password "wrong"
+3. Click the Login button
+
+Expected results:
+- An error message "Invalid credentials" is displayed
+- The user remains on the login page
+
+---
+```
+
+Sections recognised (case-insensitive): `Steps:`, `Expected results:`. All other prose is captured as the test case description. Heading number prefixes (`### 1. Title` or `### Title`) are both supported.
+
+### Playwright test-plan markdown
+
+Markdown files generated by the Playwright MCP agent are fully supported. Set `local.type: "markdown"` and point `local.include` at the generated files.
+
+---
+
+## C# MSTest / NUnit `.cs`
+
+Set `local.type: "csharp"`. Both **MSTest** and **NUnit** frameworks are supported. Each test method becomes one Test Case.
+
+### Framework attribute mapping
+
+| Concern | MSTest | NUnit |
+|---------|--------|-------|
+| Test marker | `[TestMethod]` | `[Test]` |
+| Category / tag | `[TestCategory("name")]` | `[Category("name")]` |
+| Custom property | `[TestProperty("key","val")]` | `[Property("key","val")]` |
+| TC ID writeback | `[TestProperty("tc","ID")]` | `[Property("tc","ID")]` |
+
+The framework is auto-detected per method — mixed files (e.g. a shared helper class) are handled correctly.
+
+### Source mapping (both frameworks)
+
+| C# source | Azure TC field |
+|-----------|---------------|
+| XML doc `<summary>` first line | TC **Title** |
+| Numbered lines `N. text` in summary | TC **Steps** (action) |
+| Numbered lines `N. Check: text` | TC **Steps** (expected result, when `useExpectedResult: true`) |
+| `[TestCategory("…")]` / `[Category("…")]` | TC **Tags** (string literals and `const string` constants resolved) |
+| `[TestProperty("tc","ID")]` / `[Property("tc","ID")]` | TC ID (written back after first push) |
+| `Namespace.Class.MethodName` | `AutomatedTestName` (for TRX result linking) |
+
+### MSTest example
+
+```csharp
+/// <summary>
+/// Verify CoCounsel dialog opens on Edge
+/// Test case: 1883058
+/// 1. Sign in to Westlaw Edge
+/// 2. Click CoCounsel link from page header
+/// 3. Check: Verify CoCounsel dialog opens with correct title
+/// 4. Check: Verify Close button is displayed
+/// </summary>
+[TestMethod]
+[TestProperty("tc", "1883058")]   // written back by ado-sync after first push
+[TestCategory("EdgeAalp")]
+[TestCategory("EdgeAalpSmoke")]
+public void EdgeAalpAccessTest()
+{
+    // ...
+}
+```
+
+### NUnit example
+
+```csharp
+/// <summary>
+/// Verify login with valid credentials
+/// 1. Navigate to the login page
+/// 2. Enter username and password
+/// 3. Click the Login button
+/// 4. Check: Dashboard page is shown
+/// 5. Check: Username appears in the top navigation
+/// </summary>
+[Test]
+[Property("tc", "2001")]   // written back by ado-sync after first push
+[Category("Smoke")]
+[Category("Authentication")]
+public void LoginWithValidCredentials()
+{
+    // ...
+}
+```
+
+### Recommended config
+
+```json
+{
+  "local": {
+    "type": "csharp",
+    "include": ["**/RegressionTests/**/*.cs"],
+    "exclude": ["**/*BaseTest.cs", "**/*BaseFixture.cs", "**/*Helper.cs"]
+  },
+  "sync": {
+    "markAutomated": true,
+    "format": { "useExpectedResult": true }
+  }
+}
+```
+
+With `useExpectedResult: true`, `Check:` lines go into the Expected Result column of each TC step. With `markAutomated: true`, the TC's Associated Automation is set to the fully-qualified method name (`Namespace.Class.Method`), enabling `publish-test-results` to link TRX run outcomes back to TCs.
+
+### Notes
+
+- **Constants** — ado-sync resolves `const string` declarations within the same file. Constants defined in a base class are not resolved; use string literals for reliable tagging.
+- **`pull` is not supported** for C# files. Only `push` and `publish-test-results` apply.
+- **Base classes / fixtures** — exclude them from `local.include` to avoid treating non-test methods as test cases.
+- **NUnit `[TestCase]`** — parameterised data rows (`[TestCase(1, "foo")]`) are not expanded into separate TCs. Only the `[Test]` marker is treated as the test definition.
+
+---
+
+## Java JUnit / TestNG `.java`
+
+Set `local.type: "java"`. Supports **JUnit 4**, **JUnit 5**, and **TestNG** (including Selenium-based tests). Each `@Test`-annotated method becomes one Test Case.
+
+### Framework attribute mapping
+
+| Concern | JUnit 4 | JUnit 5 | TestNG |
+|---------|---------|---------|--------|
+| Test marker | `@Test` (`org.junit.Test`) | `@Test` (`org.junit.jupiter.api.Test`) | `@Test` (`org.testng.annotations.Test`) |
+| Tag / category | `@Category(Smoke.class)` | `@Tag("smoke")` | `groups = {"smoke"}` in `@Test` |
+| TC ID writeback | `// @tc:ID` above `@Test` | `@Tag("tc:ID")` above `@Test` | `// @tc:ID` above `@Test` |
+
+The framework is auto-detected per file from import statements — no config required.
+
+### Source mapping (all frameworks)
+
+| Java source | Azure TC field |
+|-------------|---------------|
+| Javadoc `/** ... */` first non-numbered line | TC **Title** |
+| Numbered lines `N. text` in Javadoc | TC **Steps** (action) |
+| Numbered lines `N. Check: text` in Javadoc | TC **Steps** (expected result, when `useExpectedResult: true`) |
+| `@Category`, `@Tag`, `groups` in `@Test` | TC **Tags** |
+| `// @tc:ID` or `@Tag("tc:ID")` | TC ID (written back after first push) |
+| `com.example.MyClass.myMethod` | `AutomatedTestName` (for JUnit XML result linking) |
+
+### JUnit 5 example
+
+```java
+import org.junit.jupiter.api.Tag;
+import org.junit.jupiter.api.Test;
+
+class CheckoutTests {
+
+    /**
+     * Add item and complete checkout
+     * 1. Sign in as standard_user
+     * 2. Add "Sauce Labs Backpack" to cart
+     * 3. Proceed through checkout
+     * 4. Check: Order confirmation page is shown
+     */
+    @Tag("tc:1041")        // written back by ado-sync after first push
+    @Tag("smoke")
+    @Test
+    void addItemAndCompleteCheckout() {
+        // ...
+    }
+}
+```
+
+### JUnit 4 example
+
+```java
+import org.junit.Test;
+import org.junit.experimental.categories.Category;
+
+public class LoginTests {
+
+    /**
+     * Login with valid credentials
+     * 1. Navigate to the login page
+     * 2. Enter username and password
+     * 3. Check: Dashboard page is shown
+     */
+    // @tc:1042               // written back by ado-sync after first push
+    @Test
+    @Category(Smoke.class)
+    public void loginWithValidCredentials() {
+        // ...
+    }
+}
+```
+
+### TestNG example
+
+```java
+import org.testng.annotations.Test;
+
+public class SearchTests {
+
+    /**
+     * Search returns relevant results
+     * 1. Open the search page
+     * 2. Enter "junit 5" in the search box
+     * 3. Check: Results list contains at least one entry
+     */
+    // @tc:1043               // written back by ado-sync after first push
+    @Test(groups = {"regression", "search"})
+    public void searchReturnsRelevantResults() {
+        // ...
+    }
+}
+```
+
+### Recommended config
+
+```json
+{
+  "local": {
+    "type": "java",
+    "include": ["**/src/test/**/*.java"],
+    "exclude": ["**/*BaseTest.java", "**/*Helper.java", "**/*Utils.java"]
+  },
+  "sync": {
+    "markAutomated": true
+  }
+}
+```
+
+### Notes
+
+- **`pull` is not supported** for Java files. Only `push` and `publish-test-results` apply.
+- **Abstract base test classes** — exclude them from `local.include` to avoid treating base methods as test cases.
+- **JUnit 5 `@Tag`** — the `org.junit.jupiter.api.Tag` import must be present or the file must already use Jupiter annotations; ado-sync adds the tag annotation but not the import if it's missing.
+- **TestNG `groups`** — `groups = {"smoke", "regression"}` inside `@Test(...)` are extracted as TC tags.
+- **`@ParameterizedTest` / `@DataProvider`** — parameterised data rows are not expanded into separate TCs; only the method definition is treated as the test case.
+
+---
+
+## Python pytest `.py`
+
+Set `local.type: "python"`. Each `def test_*` function — at module level or inside a class — becomes one Test Case. No extra dependencies required beyond `pytest` itself.
+
+### Source mapping
+
+| Python source | Azure TC field |
+|---------------|---------------|
+| Docstring first non-numbered line | TC **Title** |
+| Numbered lines `N. text` in docstring | TC **Steps** (action) |
+| Numbered lines `N. Check: text` in docstring | TC **Steps** (expected result, when `useExpectedResult: true`) |
+| `@pytest.mark.<tag>` decorators | TC **Tags** (built-in pytest marks excluded) |
+| `@pytest.mark.tc(ID)` | TC ID (written back after first push) |
+| `module.path.ClassName.test_method` | `AutomatedTestName` (for JUnit XML result linking) |
+
+### Example
+
+```python
+import pytest
+
+class TestCheckout:
+
+    @pytest.mark.tc(1041)      # written back by ado-sync after first push
+    @pytest.mark.smoke
+    def test_add_item_and_complete_checkout(self):
+        """
+        Add item and complete checkout
+        1. Sign in as standard_user
+        2. Add Sauce Labs Backpack to cart
+        3. Proceed through checkout
+        4. Check: Order confirmation page is shown
+        """
+        # ...
+
+    @pytest.mark.tc(1042)
+    @pytest.mark.regression
+    def test_checkout_with_invalid_card(self):
+        """
+        Checkout fails with invalid card
+        1. Add an item to the cart
+        2. Enter an invalid credit card number
+        3. Click Place Order
+        4. Check: Error message is displayed
+        """
+        # ...
+```
+
+### Recommended config
+
+```json
+{
+  "local": {
+    "type": "python",
+    "include": ["tests/**/*.py"],
+    "exclude": ["tests/conftest.py", "tests/**/fixtures.py"]
+  },
+  "sync": {
+    "markAutomated": true
+  }
+}
+```
+
+### Notes
+
+- **`pull` is not supported** for Python files. Only `push` and `publish-test-results` apply.
+- **Built-in pytest marks** (`parametrize`, `skip`, `skipif`, `xfail`, `usefixtures`, `filterwarnings`) are not pushed as TC tags.
+- **`@pytest.mark.parametrize`** — parameterised data rows are not expanded into separate TCs; only the function definition is treated as the test case.
+- **Class hierarchy** — only the immediately enclosing class is used for the `automatedTestName`. Nested classes are supported.
+
+---
+
+## JavaScript / TypeScript (Jest, Jasmine, WebdriverIO) `.js` / `.ts`
+
+Set `local.type: "javascript"`. Supports **Jest**, **Jasmine**, and **WebdriverIO** (which uses Jest or Jasmine as its runner). All three share the same `describe()`/`it()`/`test()` API, so a single parser handles all of them.
+
+> **Cucumber `.feature` files** are handled by `local.type: "gherkin"` — not this type.
+
+### Detected test functions
+
+| Function | Description |
+|----------|-------------|
+| `it(title, fn)` | Standard test |
+| `test(title, fn)` | Alias for `it` |
+| `it.only` / `test.only` | Focused test |
+| `it.skip` / `test.skip` | Skipped test (still synced) |
+| `xit` / `xtest` | Jasmine-style skip |
+| `it.concurrent` / `test.concurrent` | Concurrent test |
+
+### Source mapping
+
+| JavaScript/TS source | Azure TC field |
+|----------------------|---------------|
+| JSDoc `/** ... */` first non-numbered line | TC **Title** |
+| Numbered lines `N. text` in JSDoc | TC **Steps** (action) |
+| Numbered lines `N. Check: text` in JSDoc | TC **Steps** (expected result, when `useExpectedResult: true`) |
+| `// @tags: smoke, regression` above `it()` | TC **Tags** (comma-separated list) |
+| `// @smoke` above `it()` (single-word) | TC **Tag** |
+| `// @tc:ID` above `it()` | TC ID (written back after first push) |
+| `{basename} > {describe} > {it title}` | `AutomatedTestName` (Jest report format) |
+
+### Example
+
+```typescript
+describe('Checkout', () => {
+
+  /**
+   * Add item and complete checkout
+   * 1. Sign in as standard_user
+   * 2. Add Sauce Labs Backpack to cart
+   * 3. Proceed through checkout
+   * 4. Check: Order confirmation page is shown
+   */
+  // @tc:1041               // written back by ado-sync after first push
+  // @tags: smoke, regression
+  it('adds item and completes checkout', async () => {
+    // ...
+  });
+
+  // @tc:1042
+  // @smoke
+  it('shows error on invalid card', async () => {
+    // ...
+  });
+
+});
+```
+
+### Recommended config
+
+```json
+{
+  "local": {
+    "type": "javascript",
+    "include": ["src/**/*.spec.ts", "tests/**/*.test.js"],
+    "exclude": ["**/*.helper.ts", "**/*.fixture.ts"]
+  },
+  "sync": {
+    "markAutomated": true
+  }
+}
+```
+
+### Notes
+
+- **`pull` is not supported** for JavaScript/TypeScript files. Only `push` applies.
+- **Dynamic titles** — `it` / `test` calls with template literals or computed values are skipped. Use string literals for reliable syncing.
+- **`describe` nesting** — arbitrarily deep `describe()` nesting is supported. All enclosing describe titles are included in the `automatedTestName`.
+- **Path-based auto-tagging** — directory segments starting with `@` (e.g. `tests/@smoke/`) are automatically applied as tags on all tests inside.
+
+---
+
+## CSV `.csv`
+
+Set `local.type: "csv"` to parse Azure DevOps / SpecSync tabular CSV exports.
+
+**Expected column layout (9 columns):**
+
+| Col | Field | Description |
+|-----|-------|-------------|
+| A (0) | ID | Azure Test Case ID — empty for new, filled after first push |
+| B (1) | Work Item Type | Always `Test Case` (ignored) |
+| C (2) | Title | `Scenario: My test` — non-empty on header row, empty on step rows |
+| D (3) | Test Step | Step number — empty on header row |
+| E (4) | Step Action | Step text, optionally prefixed with a Gherkin keyword |
+| F (5) | Step Expected | Expected result text (optional) |
+| G–I (6–8) | Area Path, Assigned To, State | Preserved but not used |
+
+The first row (column headers) is always skipped automatically.
+
+> **Note:** `ado-sync pull` is **not supported** for CSV files. Only push is supported.
+
+---
+
+## Excel `.xlsx`
+
+Set `local.type: "excel"`. Uses the same 9-column layout as CSV. ado-sync reads the raw worksheet XML (no external xlsx library) and handles both `t="str"` and `t="inlineStr"` cells from Azure DevOps exports.
+
+> **Note:** `ado-sync pull` is **not supported** for Excel files. Only push is supported.
+
+---
+
+## ID tags
+
+After a first push, ado-sync writes the Azure TC ID back into the local file.
+
+| Format | Location |
+|--------|----------|
+| Gherkin | `@tc:12345` tag on its own line above the `Scenario:` line |
+| Markdown | `@tc:12345` tag on the line immediately after the `### heading` |
+| C# MSTest | `[TestProperty("tc", "12345")]` attribute on the test method |
+| C# NUnit | `[Property("tc", "12345")]` attribute on the test method |
+| Java JUnit 5 | `@Tag("tc:12345")` annotation above `@Test` |
+| Java JUnit 4 / TestNG | `// @tc:12345` comment on the line above `@Test` |
+| Python pytest | `@pytest.mark.tc(12345)` decorator above `def test_*` |
+| JavaScript/TS | `// @tc:12345` comment on the line above `it()`/`test()` |
+| CSV | Numeric ID in column A of the matching title row |
+| Excel | Numeric ID in cell A of the matching title row |
+
+### Custom prefix
+
+```json
+{ "sync": { "tagPrefix": "azure" } }
+```
+
+Result: `@azure:12345` in both Gherkin and Markdown files.
+
+> **Warning:** Changing the prefix on an existing project means existing ID tags are no longer recognised. Do a project-wide find-and-replace before changing.
+
+---
+
+## Tag filtering
+
+All commands accept `--tags` to limit which scenarios are processed. Uses the standard Cucumber tag expression language.
+
+```bash
+ado-sync push --tags "@smoke"
+ado-sync push --tags "@smoke and not @wip"
+ado-sync pull --tags "@regression or @critical"
+```
+
+Tags are evaluated against all tags on a scenario, including:
+
+- Tags on the `Feature` block (Gherkin)
+- Tags on the `Scenario` / `Scenario Outline` block
+- Tags on individual `Examples` tables
+- Tags on the Markdown tag line (same `### heading` line or line below it)
+- **Path-based auto-tags** — directory segments starting with `@` are automatically applied as tags:
+
+  ```
+  specs/
+    @smoke/
+      login.feature       ← all scenarios get tag 'smoke'
+    @regression/
+      @slow/
+        checkout.feature  ← all scenarios get tags 'regression' and 'slow'
+  ```
+
+### `local.condition` — permanent filter
+
+Use `local.condition` in config to permanently exclude scenarios without needing `--tags` on every command:
+
+```json
+{ "local": { "condition": "@done and not (@ignored or @planned)" } }
+```
+
+This filter is applied before `--tags`, so it acts as a baseline inclusion gate.
+
+---
+
+## Work item linking
+
+Tags matching a configured `links` prefix are synced as Azure DevOps work item relations on the Test Case.
+
+### Config
+
+```json
+{
+  "sync": {
+    "links": [
+      { "prefix": "story",  "relationship": "System.LinkTypes.Related" },
+      { "prefix": "bug",    "relationship": "System.LinkTypes.Related" },
+      { "prefix": "req",    "relationship": "Microsoft.VSTS.Common.TestedBy-Reverse" }
+    ]
+  }
+}
+```
+
+### Usage in Gherkin
+
+```gherkin
+  @tc:1042 @story:555 @bug:789
+  Scenario: User can add items to cart
+    ...
+```
+
+### Usage in Markdown
+
+```markdown
+### User can add items to cart
+@tc:1042 @story:555 @bug:789
+
+Steps:
+1. Add an item to the cart
+...
+```
+
+On each `push`, relations are synced: new links are added, stale links (whose tag was removed) are deleted. The `relationship` value is the ADO relation type — `"System.LinkTypes.Related"` is the default if omitted.
+
+---
+
+## Attachments
+
+Files can be attached to Azure Test Cases via tags. See [Attachments](advanced.md#attachments).

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ado-sync",
-  "version": "0.1.24",
+  "version": "0.1.27",
   "description": "Bidirectional sync between local test specs (Cucumber/Markdown) and Azure DevOps Test Cases",
   "bin": {
     "ado-sync": "./dist/cli.js"