ado-sync 0.1.31 → 0.1.33

package/docs/advanced.md

@@ -374,7 +374,7 @@ Local source files are **never modified** by the AI summary feature.
 | `local` *(default)* | Good–Excellent | A GGUF model file (see setup below) |
 | `heuristic` | Basic | Nothing — zero dependencies, works offline |
 | `ollama` | Good–Excellent | [Ollama](https://ollama.com) server running locally |
-| `openai` | Excellent | OpenAI API key |
+| `openai` | Excellent | OpenAI API key, or any OpenAI-compatible proxy (LiteLLM, Azure OpenAI, vLLM, etc.) |
 | `anthropic` | Excellent | Anthropic API key |
 
 > **No setup required to try it.** If no `--ai-model` is passed for `local`, it falls back to `heuristic` silently — so `ado-sync push` always works.
@@ -472,6 +472,61 @@ ado-sync push --ai-provider openai    --ai-key $OPENAI_API_KEY
 ado-sync push --ai-provider anthropic --ai-key $ANTHROPIC_API_KEY
 ```
 
+### Using LiteLLM (or any OpenAI-compatible proxy)
+
+[LiteLLM](https://github.com/BerriAI/litellm) is a proxy that exposes an OpenAI-compatible API for 100+ model providers (Azure OpenAI, Bedrock, Gemini, Mistral, Cohere, vLLM, and more). Use the `openai` provider with `--ai-url` pointing at your LiteLLM server:
+
+```bash
+# Start LiteLLM proxy (example)
+litellm --model gpt-4o-mini   # listens on http://localhost:4000 by default
+
+# Push using LiteLLM
+ado-sync push \
+  --ai-provider openai \
+  --ai-url http://localhost:4000 \
+  --ai-key $LITELLM_API_KEY \
+  --ai-model gpt-4o-mini
+```
+
+The same `--ai-url` override works for any other OpenAI-compatible server:
+
+| Service | `--ai-url` |
+|---------|-----------|
+| LiteLLM (local proxy) | `http://localhost:4000` |
+| Azure OpenAI | `https://<resource>.openai.azure.com/openai/deployments/<deployment>` |
+| vLLM | `http://localhost:8000/v1` |
+| LocalAI | `http://localhost:8080/v1` |
+| LM Studio | `http://localhost:1234/v1` |
+
+Config file equivalent — set any `--ai-*` flag in `sync.ai` to avoid repeating it on every push. CLI flags always take precedence over config values:
+
+```json
+{
+  "sync": {
+    "ai": {
+      "provider": "openai",
+      "baseUrl": "http://localhost:4000",
+      "apiKey": "$LITELLM_API_KEY",
+      "model": "gpt-4o-mini"
+    }
+  }
+}
+```
+
+The `sync.ai` block works for any provider:
+
+```json
+{ "sync": { "ai": { "provider": "ollama", "model": "qwen2.5-coder:7b" } } }
+```
+
+```json
+{ "sync": { "ai": { "provider": "anthropic", "apiKey": "$ANTHROPIC_API_KEY" } } }
+```
+
+```json
+{ "sync": { "ai": { "provider": "none" } } }
+```
+
 ### Disabling AI summary
 
 ```bash

package/docs/publish-test-results.md

@@ -48,7 +48,7 @@ ado-sync publish-test-results \
 | NUnit XML (native) | `.xml` | Yes (`<test-run>` root) | Yes — via `[Property("tc","ID")]` | `<output>` + `<attachments>` |
 | JUnit XML | `.xml` | Yes (`<testsuites>` / `<testsuite>` root) | Optional — via `<property name="tc" value="ID"/>` | `<system-out>`, `<system-err>`, `[[ATTACHMENT\|path]]` (Playwright) |
 | Cucumber JSON | `.json` | Yes (JSON array, Cucumber format) | Yes — via `@tc:ID` tag on scenario | `step.embeddings[]` (base64 screenshots/video) |
-| Playwright JSON | `.json` | Yes (JSON object with `suites` key) | Via `@tc:ID` in test title | `test.results[].attachments[]` (screenshots, videos, traces) |
+| Playwright JSON | `.json` | Yes (JSON object with `suites` key) | Yes — via `test.annotations[{ type: 'tc', description: 'ID' }]` (preferred) or `@tc:ID` in test title | `test.results[].attachments[]` (screenshots, videos, traces) |
 
 > **NUnit via TRX**: when NUnit tests are run through the VSTest adapter (`--logger trx`), `[Property]` values are **not** included in the TRX output. Use `--logger "nunit3;LogFileName=results.xml"` to get the native NUnit XML format, which does include property values.
 
@@ -64,7 +64,7 @@ ado-sync publish-test-results \
 
 Results are linked to Azure Test Cases in priority order:
 
-1. **TC ID from file** (preferred) — when the result file contains a TC ID (`[TestProperty]`, `[Property]`, `<property name="tc">`, or `@tc:` tag), the result is posted with `testCase.id` set directly. This is robust to class/method renames.
+1. **TC ID from file** (preferred) — when the result file contains a TC ID (`[TestProperty]`, `[Property]`, `<property name="tc">`, `@tc:` tag, or Playwright `test.annotations[{ type: 'tc' }]`), the result is posted with `testCase.id` set directly. This is robust to class/method renames.
 2. **AutomatedTestName matching** (fallback) — when no TC ID is found, the result is posted with `automatedTestName` = the fully-qualified method name. Azure DevOps links it to a TC whose `AutomatedTestName` field matches. Requires `sync.markAutomated: true` on push.
 
 ---
@@ -303,6 +303,136 @@ ado-sync publish-test-results \
 
 ---
 
+### TestCafe
+
+TestCafe requires the `testcafe-reporter-junit` package to produce JUnit XML:
+
+```bash
+npm install --save-dev testcafe-reporter-junit
+```
+
+```bash
+# Run tests with JUnit reporter
+npx testcafe chrome tests/ --reporter junit:results/junit.xml
+
+# Publish results
+ado-sync publish-test-results --testResult results/junit.xml --testResultFormat junit
+```
+
+TC linking uses `AutomatedTestName` matching — set `sync.markAutomated: true` on push. The `automatedTestName` format stored in the TC is `{fileBasename} > {fixture} > {testTitle}`, which matches the JUnit `suiteName.testName` format produced by `testcafe-reporter-junit`.
+
+> **TC IDs are not embedded in JUnit output**: TestCafe's `test.meta('tc', 'N')` metadata is not written into the JUnit XML. Linking relies on AutomatedTestName matching only.
+
+---
+
+### Cypress
+
+Cypress has a built-in JUnit reporter via Mocha:
+
+```bash
+# Run tests with JUnit reporter (outputs one file per spec by default)
+npx cypress run \
+  --reporter junit \
+  --reporter-options "mochaFile=results/junit-[hash].xml"
+
+# Publish all result files
+ado-sync publish-test-results \
+  --testResult "results/junit-*.xml" \
+  --testResultFormat junit
+```
+
+TC linking uses `AutomatedTestName` matching — set `sync.markAutomated: true` on push.
+
+> **JUnit classname format**: By default Cypress sets `classname` to the spec file path and `name` to the test title. Ensure your `automatedTestName` format matches by setting `reporterOptions: { suiteTitleSeparatedBy: ' > ' }` in `cypress.config.js`.
+
+---
+
+### Detox (React Native)
+
+Detox uses Jest as its runner — use `jest-junit` the same way as Jest:
+
+```bash
+npm install --save-dev jest-junit
+```
+
+```bash
+# Run Detox tests
+JEST_JUNIT_OUTPUT_DIR=results JEST_JUNIT_OUTPUT_NAME=junit.xml \
+  npx detox test --configuration ios.sim.release
+
+# Publish results
+ado-sync publish-test-results --testResult results/junit.xml --testResultFormat junit
+```
+
+TC linking uses `AutomatedTestName` matching — set `sync.markAutomated: true` on push.
+
+---
+
+### XCUITest (iOS / macOS)
+
+Export results from Xcode's `.xcresult` bundle to JUnit XML using `xcresulttool`:
+
+```bash
+# Run tests and save result bundle
+xcodebuild test \
+  -project MyApp.xcodeproj \
+  -scheme MyApp \
+  -destination 'platform=iOS Simulator,name=iPhone 15' \
+  -resultBundlePath TestResults.xcresult
+
+# Export JUnit XML
+xcrun xcresulttool get --path TestResults.xcresult --format junit > results/junit.xml
+
+# Publish results
+ado-sync publish-test-results --testResult results/junit.xml --testResultFormat junit
+```
+
+TC linking uses `AutomatedTestName` matching — set `sync.markAutomated: true` on push. The `automatedTestName` format is `{className}/{funcTestName}` (e.g. `LoginUITests/testValidCredentialsNavigateToInventory`).
+
+> **Attachments**: `xcresulttool` does not embed screenshots in the JUnit export. Use `--attachmentsFolder` to attach screenshot files produced by your test hooks separately.
+
+---
+
+### Espresso (Android)
+
+Gradle's `connectedAndroidTest` task writes JUnit XML to `app/build/outputs/androidTest-results/connected/`:
+
+```bash
+# Run instrumented tests
+./gradlew connectedAndroidTest
+
+# Publish results
+ado-sync publish-test-results \
+  --testResult "app/build/outputs/androidTest-results/connected/TEST-*.xml" \
+  --testResultFormat junit
+```
+
+TC linking uses `AutomatedTestName` matching — set `sync.markAutomated: true` on push. The `automatedTestName` format is `{packageName}.{ClassName}.{methodName}`.
+
+---
+
+### Flutter
+
+Flutter can produce JUnit XML via the `flutter_test_junit` package or by piping `--reporter junit`:
+
+```bash
+# Option A — built-in reporter (Flutter ≥ 3.7)
+flutter test --reporter junit > results/junit.xml
+
+# Option B — flutter_test_junit package
+flutter pub add --dev flutter_test_junit
+dart run flutter_test_junit:main > results/junit.xml
+```
+
+```bash
+# Publish results
+ado-sync publish-test-results --testResult results/junit.xml --testResultFormat junit
+```
+
+TC linking uses `AutomatedTestName` matching — set `sync.markAutomated: true` on push.
+
+---
+
 | Framework | Result format | TC ID in file | Attachments uploaded | Live-tested |
 |-----------|---------------|---------------|----------------------|-------------|
 | C# MSTest | TRX | ✅ `[TestProperty("tc","ID")]` | `<Output><StdOut>` + `<Output><ResultFiles>` files | ✅ |
@@ -314,8 +444,14 @@ ado-sync publish-test-results \
 | Jest | JUnit XML | ⚠️ optional `<property name="tc">` | `<system-out>`, `<system-err>` | ✅ |
 | WebdriverIO / Jasmine | JUnit XML | ⚠️ optional `<property name="tc">` | `<system-out>`, `<system-err>` | ✅ |
 | Cucumber JS | Cucumber JSON | ✅ `@tc:ID` tag | `step.embeddings[]` (base64 screenshots/video) | ✅ |
-| Playwright | Playwright JSON | Via `@tc:ID` in test title | Files from `attachments[].path` (screenshots, videos, traces) | ✅ |
-| Playwright | JUnit XML | Via `@tc:ID` in test title | `[[ATTACHMENT\|path]]` referenced files | ✅ |
+| Playwright | Playwright JSON | ✅ native `annotation: { type: 'tc', description: 'ID' }`; or `@tc:ID` in test title | Files from `attachments[].path` (screenshots, videos, traces) | ✅ |
+| Playwright | JUnit XML | ⚠️ `@tc:ID` in test title only (no annotation in JUnit format) | `[[ATTACHMENT\|path]]` referenced files | ✅ |
+| TestCafe | JUnit XML | ❌ AutomatedTestName matching only | `<system-out>`, `<system-err>` | |
+| Cypress | JUnit XML | ❌ AutomatedTestName matching only | `<system-out>`, `<system-err>` | |
+| Detox | JUnit XML | ❌ AutomatedTestName matching only | `<system-out>`, `<system-err>` | |
+| XCUITest | JUnit XML | ❌ AutomatedTestName matching only | none (use `--attachmentsFolder`) | |
+| Espresso | JUnit XML | ❌ AutomatedTestName matching only | `<system-out>`, `<system-err>` | |
+| Flutter | JUnit XML | ❌ AutomatedTestName matching only | `<system-out>`, `<system-err>` | |
 
 ---
 

package/docs/spec-formats.md

@@ -471,6 +471,36 @@ Set `local.type: "playwright"`. Supports **Playwright Test** (`@playwright/test`
 | `test.describe.parallel` | Parallel describe block |
 | `test.describe.serial` | Serial describe block |
 
+### ID tagging — native annotation (recommended)
+
+Playwright Test has a built-in annotation API. Use it to attach the TC ID directly in the test definition — no comments needed:
+
+```typescript
+// Single annotation (most common)
+test('completes checkout', {
+  annotation: { type: 'tc', description: '1041' },
+  tag: '@smoke',
+}, async ({ page }) => { ... });
+
+// Tag-style ID (alternative)
+test('cart badge shows count', {
+  tag: ['@tc:1043', '@smoke'],
+}, async ({ page }) => { ... });
+
+// Array form — combine TC ID with other annotations
+test.fixme('promo code applies discount', {
+  annotation: [
+    { type: 'tc', description: '1042' },
+    { type: 'issue', description: 'promo-code not yet implemented' },
+  ],
+  tag: '@wip',
+}, async ({ page }) => { ... });
+```
+
+**Writeback** — on the first `push`, ado-sync injects `{ annotation: { type: 'tc', description: 'N' } }` into the test options object. Subsequent pushes update the description in place.
+
+**Comment fallback** — `// @tc:ID` above `test()` is still recognised and written for edge cases where the options object cannot be parsed (e.g., multi-line title spanning several lines).
+
 ### Source mapping
 
 | Playwright source | Azure TC field |
@@ -478,10 +508,16 @@ Set `local.type: "playwright"`. Supports **Playwright Test** (`@playwright/test`
 | JSDoc `/** ... */` first non-numbered line | TC **Title** |
 | Numbered lines `N. text` in JSDoc | TC **Steps** (action) |
 | Numbered lines `N. Check: text` in JSDoc | TC **Steps** (expected result, when `useExpectedResult: true`) |
-| `// @tags: smoke, regression` above `test()` | TC **Tags** |
-| `// @tc:ID` above `test()` | TC ID (written back after first push) |
+| `annotation: { type: 'tc', description: 'N' }` | TC ID (**preferred**) |
+| `annotation: [{ type: 'tc', description: 'N' }, ...]` | TC ID (array form) |
+| `tag: '@tc:N'` or `tag: ['@tc:N', ...]` | TC ID (tag form) |
+| `tag: '@smoke'` etc. | TC **Tags** |
+| `// @tags: smoke, regression` above `test()` | TC **Tags** (comment form) |
+| `// @tc:ID` above `test()` | TC ID (comment fallback) |
 | `{basename} > {describe} > {test title}` | `AutomatedTestName` |
 
+**Priority**: native `annotation` > native `tag` > `// @tc:N` comment.
+
 ### TypeScript example
 
 ```typescript
@@ -496,28 +532,35 @@ test.describe('SauceDemo Checkout', () => {
    * 3. Proceed through checkout and enter name and zip
    * 4. Check: Order confirmation page is displayed
    */
-  // @tc:1041               // written back by ado-sync after first push
-  // @tags: smoke, checkout
-  test('completes checkout with valid details', async ({ page }) => {
+  test('completes checkout with valid details', {
+    annotation: { type: 'tc', description: '1041' },
+    tag: '@smoke',
+  }, async ({ page }) => {
     // ...
   });
 
-  // @tc:1042
-  test.fixme('promo code applies discount', async ({ page }) => {
-    // known issue — still synced to Azure, tagged wip
+  test.fixme('promo code applies discount', {
+    annotation: [
+      { type: 'tc', description: '1042' },
+      { type: 'issue', description: 'promo-code not yet implemented' },
+    ],
+    tag: '@wip',
+  }, async ({ page }) => {
+    // known issue — still synced to Azure
   });
 });
 
 test.describe.parallel('Cart assertions', () => {
 
-  // @tc:1043
-  test('cart badge shows correct count', async ({ page }) => {
+  test('cart badge shows correct count', {
+    annotation: { type: 'tc', description: '1043' },
+  }, async ({ page }) => {
     // ...
   });
 });
 ```
 
-### JavaScript example (CommonJS)
+### JavaScript example (comment fallback style)
 
 ```javascript
 const { test, expect } = require('@playwright/test');
@@ -531,7 +574,7 @@ test.describe('SauceDemo Login', () => {
    * 3. Click the login button
    * 4. Check: URL contains "inventory.html"
    */
-  // @tc:1044
+  // @tc:1044   ← comment style also works; ado-sync upgrades to annotation on next push
   test('valid credentials redirect to inventory', async ({ page }) => {
     // ...
   });
@@ -557,8 +600,9 @@ test.describe('SauceDemo Login', () => {
 
 - **`pull` is not supported** for Playwright files. Only `push` applies.
 - **`test.describe` nesting** — arbitrarily deep nesting is supported. All enclosing describe titles are included in the `automatedTestName`.
-- **`test.fixme` / `test.fail`** — both are parsed and synced as normal test cases. Add `// @tags: wip` above `test.fixme` to mark them in Azure.
+- **`test.fixme` / `test.fail`** — both are parsed and synced as normal test cases. Use `tag: '@wip'` on `test.fixme` to mark them in Azure.
 - **Publishing results** — set `testResultFormat: playwrightJson` when using `publish-test-results`.
+- **Native annotation priority** — `annotation: { type: 'tc', … }` is read before `tag:` which is read before `// @tc:N` comments. The tool writes back using native annotation on every sync.
 
 ---
 
@@ -664,7 +708,45 @@ describe('SauceDemo Login', () => {
 
 Set `local.type: "testcafe"`. Supports **TestCafe** tests using the `fixture` / `test` API.
 
-```typescript
+### ID tagging — native `test.meta()` (recommended)
+
+TestCafe has a built-in `.meta()` method that attaches key-value metadata to a test. Use it to store the TC ID natively — no comments needed:
+
+```javascript
+// Key-value form (recommended — clean and simple)
+test.meta('tc', '1080')('redirects to inventory on valid login', async t => { ... });
+
+// Object form (use when adding multiple metadata fields)
+test.meta({ tc: '1081', priority: 'high' })('locked out user sees error', async t => { ... });
+
+// Combined with test.skip
+test.skip.meta('tc', '1082')('skipped test', async t => { ... });
+```
+
+**Writeback** — on the first `push`, ado-sync injects `.meta('tc', 'N')` between the test function and its title call. For example:
+```
+test('title', fn)  →  test.meta('tc', '12345')('title', fn)
+```
+Subsequent pushes update the existing `.meta()` value in place.
+
+**Comment fallback** — `// @tc:N` above `test()` is still recognised.
+
+### Source mapping
+
+| TestCafe source | Azure TC field |
+|-----------------|---------------|
+| JSDoc `/** ... */` first non-numbered line | TC **Title** |
+| Numbered lines `N. text` in JSDoc | TC **Steps** (action) |
+| Numbered lines `N. Check: text` in JSDoc | TC **Steps** (expected result) |
+| `test.meta('tc', 'N')` | TC ID (**preferred**) |
+| `test.meta({ tc: 'N' })` | TC ID (object form) |
+| `// @tags: smoke, regression` above `test()` | TC **Tags** |
+| `// @tc:N` above `test()` | TC ID (comment fallback) |
+| `{basename} > {fixture} > {test title}` | `AutomatedTestName` |
+
+### Example
+
+```javascript
 fixture('SauceDemo Login')
   .page('https://www.saucedemo.com');
 
@@ -675,16 +757,18 @@ fixture('SauceDemo Login')
  * 3. Click the login button
  * 4. Check: URL contains "inventory"
  */
-// @tc:1080
-// @tags: smoke
-test('redirects to inventory on valid login', async t => {
+// @smoke
+test.meta('tc', '1080')('redirects to inventory on valid login', async t => {
   // ...
 });
 
-// @tc:1081
-test.skip('locked out user sees error', async t => {
-  // skipped — still synced to Azure
+test.meta({ tc: '1081', priority: 'high' })('locked out user sees error', async t => {
+  // ...
 });
+
+// Comment style still works (written by ado-sync before meta support was added)
+// @tc:1082
+test.skip('skipped test — still synced', async t => { ... });
 ```
 
 ### Recommended config
@@ -693,7 +777,7 @@ test.skip('locked out user sees error', async t => {
 {
   "local": {
     "type": "testcafe",
-    "include": ["tests/**/*.ts"],
+    "include": ["tests/**/*.js", "tests/**/*.ts"],
     "exclude": ["tests/helpers/**"]
   },
   "sync": { "markAutomated": true }
@@ -704,9 +788,8 @@ test.skip('locked out user sees error', async t => {
 
 - The `fixture()` title is used as the group / suite name and included in `automatedTestName`.
 - `test.skip()` and `test.only()` are both parsed and synced.
-- ID writeback format: `// @tc:12345` immediately above the `test()` line.
 - `pull` is not supported for TestCafe files — only `push` applies.
-- JSDoc `/** ... */` above a `test()` call is parsed for TC title and numbered steps, same as Playwright and Jest.
+- JSDoc `/** ... */` above a `test.meta(...)` call is parsed for TC title and numbered steps.
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ado-sync",
-  "version": "0.1.31",
+  "version": "0.1.33",
   "description": "Bidirectional sync between local test specs (Cucumber/Markdown) and Azure DevOps Test Cases",
   "bin": {
     "ado-sync": "./dist/cli.js"