@ash-mallick/browserstack-sync 1.0.6 → 1.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,18 +1,10 @@
1
1
  # @ash-mallick/browserstack-sync
2
2
 
3
- Sync **Playwright** and **Cypress** e2e test specs to CSV files and **BrowserStack Test Management**.
3
+ Sync **Playwright** and **Cypress** e2e specs to CSV (TC-001, TC-002, …) and optionally to **BrowserStack Test Management** (one folder per spec, create/update test cases).
4
4
 
5
- **By Ashutosh Mallick**
6
-
7
- ---
8
-
9
- ## What It Does
5
+ **🦙 FREE AI-powered test step extraction** using **Ollama** — runs 100% locally, no data sent to cloud!
10
6
 
11
- - Scans your e2e test files (Playwright or Cypress)
12
- - Extracts test cases and assigns IDs (TC-001, TC-002, ...)
13
- - Generates **one CSV file per spec** with test details
14
- - Optionally syncs to **BrowserStack Test Management** (creates folders and test cases)
15
- - **Idempotent**: existing tests are updated, new ones are created (no duplicates)
7
+ **By Ashutosh Mallick**
16
8
 
17
9
  ---
18
10
 
@@ -22,225 +14,180 @@ Sync **Playwright** and **Cypress** e2e test specs to CSV files and **BrowserSta
22
14
  npm install @ash-mallick/browserstack-sync
23
15
  ```
24
16
 
25
- Or run directly without installing:
26
-
27
- ```bash
28
- npx @ash-mallick/browserstack-sync --csv-only
29
- ```
17
+ Or run without installing: `npx @ash-mallick/browserstack-sync --csv-only`
30
18
 
31
19
  ---
32
20
 
33
- ## Usage
21
+ ## Run
34
22
 
35
- Run from your project root:
23
+ From your project root (where your e2e specs live):
36
24
 
37
25
  ```bash
38
- # Generate CSVs only (no BrowserStack sync)
26
+ # Generate CSVs only
39
27
  npx am-browserstack-sync --csv-only
40
28
 
41
- # Sync to BrowserStack (interactive prompt)
29
+ # Sync to BrowserStack (interactive: choose all or specific specs)
42
30
  npx am-browserstack-sync
43
31
 
44
- # Sync all specs without prompt (for CI/CD)
32
+ # Sync all specs, no prompt (e.g. CI)
45
33
  npx am-browserstack-sync --all
46
34
 
47
- # Sync specific specs only
35
+ # Sync only certain specs
48
36
  npx am-browserstack-sync --spec=login.spec,checkout.spec
37
+
38
+ # Disable AI analysis (use regex extraction only)
39
+ npx am-browserstack-sync --no-ai
40
+
41
+ # Use a specific Ollama model
42
+ npx am-browserstack-sync --model=codellama
49
43
  ```
50
44
 
51
- ### Add to package.json scripts
45
+ **Scripts** in `package.json`:
52
46
 
53
47
  ```json
54
48
  {
55
49
  "scripts": {
56
50
  "sync:e2e": "am-browserstack-sync",
57
- "sync:csv": "am-browserstack-sync --csv-only"
51
+ "sync:e2e-csv": "am-browserstack-sync --csv-only"
58
52
  }
59
53
  }
60
54
  ```
61
55
 
62
56
  ---
63
57
 
64
- ## Supported Frameworks
58
+ ## 🦙 AI-Powered Step Analysis (FREE with Ollama)
65
59
 
66
- | Framework | File Patterns | Auto-Detected Directory |
67
- |-----------|---------------|------------------------|
68
- | Playwright | `*.spec.ts`, `*.spec.js`, `*.test.ts`, `*.test.js` | `playwright/e2e` |
69
- | Cypress | `*.cy.ts`, `*.cy.js` | `cypress/e2e` |
70
- | Generic | Any of the above | `e2e` |
60
+ The tool uses **Ollama** to analyze your test code and generate **human-readable test steps**. Ollama runs **100% locally** on your machine — no data sent to cloud, completely free, no API key needed!
71
61
 
72
- **Auto-detection priority:**
73
- 1. `playwright/e2e` (if exists)
74
- 2. `cypress/e2e` (if exists)
75
- 3. `e2e` (generic fallback)
62
+ **Example transformation:**
76
63
 
77
- No configuration needed for standard project structures!
64
+ ```typescript
65
+ // Your test code:
66
+ test('should log in successfully', async ({ page }) => {
67
+ await page.goto('/login');
68
+ await page.getByLabel(/email/i).fill('user@example.com');
69
+ await page.getByLabel(/password/i).fill('validpassword');
70
+ await page.getByRole('button', { name: /sign in/i }).click();
71
+ await expect(page).toHaveURL(/\/dashboard/);
72
+ });
73
+ ```
78
74
 
79
- ---
75
+ **Generated steps:**
80
76
 
81
- ## Configuration (Optional)
77
+ | Step | Expected Result |
78
+ |------|-----------------|
79
+ | Navigate to /login page | Login page loads successfully |
80
+ | Enter 'user@example.com' in the Email field | Email is entered |
81
+ | Enter 'validpassword' in the Password field | Password is masked and entered |
82
+ | Click the 'Sign In' button | Form is submitted |
83
+ | Verify URL | URL matches /dashboard |
82
84
 
83
- Override defaults via **`.am-browserstack-sync.json`** in project root:
85
+ ### Setup Ollama
84
86
 
85
- ```json
86
- {
87
- "e2eDir": "tests/e2e",
88
- "csvOutputDir": "tests/csv-output"
89
- }
90
- ```
87
+ 1. **Download and install** Ollama from [ollama.ai](https://ollama.ai)
91
88
 
92
- Or in **package.json**:
89
+ 2. **Pull a model** (llama3.2 recommended):
90
+ ```bash
91
+ ollama pull llama3.2
92
+ ```
93
93
 
94
- ```json
95
- {
96
- "amBrowserstackSync": {
97
- "e2eDir": "cypress/e2e",
98
- "csvOutputDir": "cypress/csv"
99
- }
100
- }
101
- ```
94
+ 3. **Start Ollama** (runs automatically on macOS, or run manually):
95
+ ```bash
96
+ ollama serve
97
+ ```
102
98
 
103
- Or via **environment variables**:
99
+ 4. **Run the sync** — AI analysis is automatic when Ollama is running!
100
+ ```bash
101
+ npx am-browserstack-sync --csv-only
102
+ ```
104
103
 
105
- ```bash
106
- PLAYWRIGHT_BROWSERSTACK_E2E_DIR=tests/e2e
107
- PLAYWRIGHT_BROWSERSTACK_CSV_DIR=tests/csv
108
- ```
104
+ ### Options
105
+
106
+ - `--no-ai` — Disable AI, use regex-based extraction instead
107
+ - `--model=codellama` — Use a different Ollama model
108
+ - `OLLAMA_MODEL=llama3.2` — Set default model via env variable
109
+ - `OLLAMA_HOST=http://localhost:11434` — Custom Ollama host
110
+
111
+ ### Recommended Models
112
+
113
+ | Model | Size | Best for |
114
+ |-------|------|----------|
115
+ | `llama3.2` | 2GB | General purpose, fast (default) |
116
+ | `codellama` | 4GB | Better code understanding |
117
+ | `llama3.2:1b` | 1GB | Fastest, lower quality |
118
+ | `mistral` | 4GB | Good balance |
119
+
120
+ **Fallback:** If Ollama is not running, the tool automatically uses regex-based step extraction, which still provides meaningful steps.
109
121
 
110
122
  ---
111
123
 
112
- ## BrowserStack Integration
124
+ ## Config (optional)
113
125
 
114
- ### Setup
126
+ Defaults: e2e dir `playwright/e2e`, CSV dir `playwright/e2e-csv`. For Cypress use e.g. `cypress/e2e`.
115
127
 
116
- 1. Create a **`.env`** file in your project root (don't commit this!):
128
+ Override via **`.am-browserstack-sync.json`** in project root:
117
129
 
118
- ```env
119
- BROWSERSTACK_USERNAME=your_username
120
- BROWSERSTACK_ACCESS_KEY=your_access_key
121
- BROWSERSTACK_PROJECT_ID=PR-XXXX
130
+ ```json
131
+ {
132
+ "e2eDir": "playwright/e2e",
133
+ "csvOutputDir": "playwright/e2e-csv"
134
+ }
122
135
  ```
123
136
 
124
- Or use a single API token:
137
+ Or **package.json**: `"amBrowserstackSync": { "e2eDir": "...", "csvOutputDir": "..." }`
138
+ Or env: `PLAYWRIGHT_BROWSERSTACK_E2E_DIR`, `PLAYWRIGHT_BROWSERSTACK_CSV_DIR`.
125
139
 
126
- ```env
127
- BROWSERSTACK_API_TOKEN=your_token
128
- BROWSERSTACK_PROJECT_ID=PR-XXXX
129
- ```
140
+ ---
130
141
 
131
- 2. Get credentials from [BrowserStack Test Management → Settings → API Keys](https://test-management.browserstack.com/settings/api-keys)
142
+ ## BrowserStack sync
132
143
 
133
- 3. Find your Project ID in the BrowserStack project URL (e.g., `PR-1234`)
144
+ Sync pushes your e2e tests into **BrowserStack Test Management** so you can track test cases, link runs, and keep specs in sync with one source of truth. Under your chosen project it creates **one folder per spec file** (e.g. `login.spec`, `checkout.spec`) and one **test case** per test, with title, description, steps, state (Active), type (Functional), automation status, and tags. Existing test cases are matched by title or TC-id and **updated**; new ones are **created**. No duplicates.
134
145
 
135
- ### What Gets Synced
146
+ **Setup:**
136
147
 
137
- For each spec file, the tool:
138
- - Creates a **folder** in BrowserStack (named after the spec file)
139
- - Creates **test cases** with:
140
- - Title (from test name)
141
- - Description
142
- - Steps and expected results
143
- - State (Active)
144
- - Type (Functional)
145
- - Automation status (automated)
146
- - Tags (framework, spec name, test keywords)
148
+ 1. In project root, create **`.env`** (do not commit):
147
149
 
148
- ### Idempotent Sync
150
+ ```env
151
+ BROWSERSTACK_USERNAME=your_username
152
+ BROWSERSTACK_ACCESS_KEY=your_access_key
153
+ BROWSERSTACK_PROJECT_ID=PR-XXXX
154
+ ```
155
+ Or use a single token: `BROWSERSTACK_API_TOKEN=your_token`
149
156
 
150
- - **New tests**Created in BrowserStack
151
- - **Existing tests** (matched by title or TC-ID tag) → Updated
152
- - **No duplicates** are created
157
+ 2. Get credentials and project ID from [Test Management API keys](https://test-management.browserstack.com/settings/api-keys). The project ID is in the project URL (e.g. `PR-1234`).
153
158
 
154
- ---
159
+ 3. **Install Ollama** for AI-powered step analysis (optional but recommended).
155
160
 
156
- ## CSV Output
161
+ 4. Run **`npx am-browserstack-sync`** (without `--csv-only`). You'll be prompted to sync all specs or pick specific ones (unless you use `--all` or `--spec=...`). After sync, open your project in BrowserStack to see the new folders and test cases.
157
162
 
158
- Each spec file generates a CSV with these columns:
163
+ ---
164
+
165
+ ## What it does
159
166
 
160
- | Column | Description |
161
- |--------|-------------|
162
- | `test_case_id` | TC-001, TC-002, ... |
163
- | `title` | Test name |
164
- | `state` | Active, Draft, etc. |
165
- | `case_type` | Functional, Regression, etc. |
166
- | `steps` | Test steps |
167
- | `expected_results` | Expected outcomes |
168
- | `jira_issues` | Linked Jira tickets |
169
- | `automation_status` | automated / manual |
170
- | `tags` | Framework, functionality tags |
171
- | `description` | Test description |
172
- | `spec_file` | Source file name |
167
+ - Finds **Playwright** (`*.spec.*`, `*.test.*`) and **Cypress** (`*.cy.*`) spec files in your e2e dir.
168
+ - Extracts test titles from `test('...')` / `it('...')`, assigns TC-001, TC-002, …
169
+ - **Analyzes test code** with Ollama AI (local, free) or regex to generate human-readable steps and expected results.
170
+ - Enriches with state (Active), type (Functional), automation (automated), tags (from spec + title).
171
+ - Writes **one CSV per spec** (test_case_id, title, state, case_type, steps, expected_results, jira_issues, automation_status, tags, description, spec_file).
172
+ - Optionally syncs to BrowserStack with description, steps, and tags.
173
173
 
174
174
  ---
175
175
 
176
176
  ## Programmatic API
177
177
 
178
- Use in your own scripts:
179
-
180
- ```javascript
178
+ ```js
181
179
  import { runSync } from '@ash-mallick/browserstack-sync';
182
180
 
183
181
  await runSync({
184
182
  cwd: '/path/to/project',
185
- csvOnly: false,
183
+ csvOnly: true,
186
184
  all: true,
187
- // Or specify specific specs:
188
- // spec: ['login.spec', 'checkout.spec'],
185
+ spec: ['login.spec'],
186
+ useAI: true, // Enable AI analysis (default: true if Ollama is running)
187
+ model: 'llama3.2', // Ollama model to use
189
188
  });
190
189
  ```
191
190
 
192
- ### Options
193
-
194
- | Option | Type | Description |
195
- |--------|------|-------------|
196
- | `cwd` | string | Project root directory |
197
- | `csvOnly` | boolean | Skip BrowserStack sync |
198
- | `all` | boolean | Sync all specs (no prompt) |
199
- | `spec` | string[] | Specific specs to sync |
200
- | `e2eDir` | string | Override e2e directory |
201
- | `csvOutputDir` | string | Override CSV output directory |
202
-
203
- ---
204
-
205
- ## Example Project Structure
206
-
207
- ### Playwright
208
-
209
- ```
210
- my-project/
211
- ├── package.json
212
- ├── .env # BrowserStack credentials
213
- ├── playwright/
214
- │ ├── e2e/ # Your test files
215
- │ │ ├── login.spec.ts
216
- │ │ └── checkout.spec.ts
217
- │ └── e2e-csv/ # Generated CSVs
218
- │ ├── login.spec.csv
219
- │ └── checkout.spec.csv
220
- ```
221
-
222
- ### Cypress
223
-
224
- ```
225
- my-project/
226
- ├── package.json
227
- ├── .env
228
- ├── cypress/
229
- │ ├── e2e/ # Your test files
230
- │ │ ├── login.cy.ts
231
- │ │ └── cart.cy.ts
232
- │ └── e2e-csv/ # Generated CSVs
233
- │ ├── login.cy.csv
234
- │ └── cart.cy.csv
235
- ```
236
-
237
- ---
238
-
239
- ## License
240
-
241
- MIT
242
-
243
191
  ---
244
192
 
245
- **Author:** Ashutosh Mallick
246
- **npm:** [@ash-mallick/browserstack-sync](https://www.npmjs.com/package/@ash-mallick/browserstack-sync)
193
+ **Author:** Ashutosh Mallick
package/bin/cli.js CHANGED
@@ -5,9 +5,23 @@
5
5
  * Uses cwd as project root; config from .env, config file, or package.json.
6
6
  *
7
7
  * Options:
8
- * --csv-only Generate CSVs only, do not sync to BrowserStack
9
- * --all Sync all spec files (no interactive prompt)
10
- * --spec=name Sync only these specs (comma-separated). e.g. --spec=login.spec,checkout.spec
8
+ * --csv-only Generate CSVs only, do not sync to BrowserStack
9
+ * --all Sync all spec files (no interactive prompt)
10
+ * --spec=name Sync only these specs (comma-separated). e.g. --spec=login.spec,checkout.spec
11
+ * --no-ai Disable AI-powered step analysis (use regex extraction only)
12
+ * --model=name Ollama model to use (default: llama3.2). e.g. --model=codellama
13
+ *
14
+ * AI Analysis (FREE, Local with Ollama):
15
+ * Install Ollama from https://ollama.ai, then:
16
+ * ollama pull llama3.2
17
+ * ollama serve
18
+ *
19
+ * AI analysis is automatic when Ollama is running! No API key needed.
20
+ * The AI will analyze your test code and generate human-readable steps like:
21
+ * - "Navigate to /login page"
22
+ * - "Enter 'user@test.com' in the Email field"
23
+ * - "Click the 'Sign In' button"
24
+ * - "Verify URL matches /dashboard"
11
25
  */
12
26
 
13
27
  import { runSync } from '../lib/index.js';
@@ -15,15 +29,31 @@ import { runSync } from '../lib/index.js';
15
29
  const argv = process.argv.slice(2);
16
30
  const csvOnly = argv.includes('--csv-only');
17
31
  const all = argv.includes('--all');
32
+ const noAI = argv.includes('--no-ai');
33
+
34
+ // Parse --spec=value
18
35
  const specArg = argv.find((a) => a.startsWith('--spec='));
19
36
  let spec = [];
20
37
  if (specArg) {
21
38
  const value = specArg.slice('--spec='.length).trim();
22
39
  if (value) spec = value.split(',').map((s) => s.trim()).filter(Boolean);
23
40
  }
41
+
42
+ // Parse --model=value
43
+ const modelArg = argv.find((a) => a.startsWith('--model='));
44
+ const model = modelArg ? modelArg.slice('--model='.length).trim() : undefined;
45
+
46
+ // Parse --cwd value
24
47
  const cwd = argv.includes('--cwd') ? argv[argv.indexOf('--cwd') + 1] : process.cwd();
25
48
 
26
- runSync({ cwd, csvOnly, all, spec }).catch((err) => {
49
+ runSync({
50
+ cwd,
51
+ csvOnly,
52
+ all,
53
+ spec,
54
+ useAI: !noAI,
55
+ model,
56
+ }).catch((err) => {
27
57
  console.error(err);
28
58
  process.exit(1);
29
59
  });
@@ -0,0 +1,267 @@
1
+ /**
2
+ * AI-powered test step analyzer using Ollama (FREE, local).
3
+ *
4
+ * Ollama runs 100% locally on your machine - no data sent to cloud, completely free.
5
+ * Install Ollama: https://ollama.ai
6
+ * Then run: ollama pull llama3.2
7
+ */
8
+
9
+ import { Ollama } from 'ollama';
10
+
11
+ const DEFAULT_MODEL = 'llama3.2';
12
+ const OLLAMA_HOST = process.env.OLLAMA_HOST || 'http://localhost:11434';
13
+
14
+ /**
15
+ * System prompt for the AI to understand what we want.
16
+ */
17
+ const SYSTEM_PROMPT = `You are a test automation expert. Your job is to analyze Playwright or Cypress test code and extract human-readable test steps.
18
+
19
+ For each test, output a JSON array of steps. Each step should have:
20
+ - "step": A clear, human-readable action (e.g., "Navigate to /login page", "Enter 'test@example.com' in the email field")
21
+ - "result": The expected result or assertion for this step (e.g., "Login form is displayed", "Error message 'Invalid credentials' appears")
22
+
23
+ Rules:
24
+ 1. Convert code actions to plain English that a manual tester could follow
25
+ 2. For page.goto(url) -> "Navigate to <url>"
26
+ 3. For page.fill(selector, value) or getByLabel().fill() -> "Enter '<value>' in the <field name> field"
27
+ 4. For page.click() or getByRole().click() -> "Click the '<button/link name>' button/link"
28
+ 5. For expect() assertions -> This becomes the "result" of the previous step
29
+ 6. If there's no explicit assertion, the result should describe the expected state
30
+ 7. Be concise but clear
31
+ 8. Group related actions if they form one logical step
32
+
33
+ Output ONLY valid JSON array, no markdown, no explanation. Example:
34
+ [
35
+ {"step": "Navigate to /login page", "result": "Login page loads successfully"},
36
+ {"step": "Enter 'user@test.com' in the Email field", "result": "Email is entered"},
37
+ {"step": "Enter 'password123' in the Password field", "result": "Password is masked and entered"},
38
+ {"step": "Click the 'Sign In' button", "result": "User is redirected to /dashboard"}
39
+ ]`;
40
+
41
+ /**
42
+ * Build the user prompt for AI analysis.
43
+ */
44
+ function buildUserPrompt(testTitle, testCode) {
45
+ return `Analyze this Playwright/Cypress test and extract the test steps:
46
+
47
+ Test Title: ${testTitle}
48
+
49
+ Test Code:
50
+ \`\`\`
51
+ ${testCode}
52
+ \`\`\`
53
+
54
+ Return a JSON array of steps with "step" and "result" fields.`;
55
+ }
56
+
57
+ /**
58
+ * Parse AI response content to extract steps array.
59
+ */
60
+ function parseAIResponse(content) {
61
+ if (!content) {
62
+ throw new Error('Empty response from AI');
63
+ }
64
+
65
+ // Remove markdown code block wrapper if present: ```json ... ``` or ``` ... ```
66
+ let jsonStr = content.trim();
67
+ const codeBlockMatch = jsonStr.match(/^```(?:json)?\s*\n?([\s\S]*?)\n?```$/);
68
+ if (codeBlockMatch) {
69
+ jsonStr = codeBlockMatch[1].trim();
70
+ }
71
+
72
+ // Also try to extract JSON array from anywhere in the response
73
+ if (!jsonStr.startsWith('[')) {
74
+ const arrayMatch = jsonStr.match(/\[[\s\S]*\]/);
75
+ if (arrayMatch) {
76
+ jsonStr = arrayMatch[0];
77
+ }
78
+ }
79
+
80
+ let steps;
81
+ try {
82
+ steps = JSON.parse(jsonStr);
83
+ } catch (parseError) {
84
+ throw new Error(`Failed to parse AI response as JSON: ${parseError.message}. Response: ${content.slice(0, 200)}`);
85
+ }
86
+
87
+ if (!Array.isArray(steps)) {
88
+ throw new Error('AI response is not an array');
89
+ }
90
+
91
+ return steps.map((s) => ({
92
+ step: String(s.step || ''),
93
+ result: String(s.result || ''),
94
+ }));
95
+ }
96
+
97
+ /**
98
+ * Check if Ollama is running and accessible.
99
+ */
100
+ export async function checkOllamaConnection() {
101
+ try {
102
+ const ollama = new Ollama({ host: OLLAMA_HOST });
103
+ await ollama.list(); // Simple API call to check connection
104
+ return true;
105
+ } catch (error) {
106
+ return false;
107
+ }
108
+ }
109
+
110
+ /**
111
+ * Get list of available Ollama models.
112
+ */
113
+ export async function listOllamaModels() {
114
+ try {
115
+ const ollama = new Ollama({ host: OLLAMA_HOST });
116
+ const response = await ollama.list();
117
+ return response.models?.map((m) => m.name) || [];
118
+ } catch (error) {
119
+ return [];
120
+ }
121
+ }
122
+
123
+ /**
124
+ * Analyze using Ollama (FREE, local).
125
+ * Requires Ollama installed: https://ollama.ai
126
+ * Run: ollama pull llama3.2 (or another model)
127
+ */
128
+ async function analyzeWithOllama(testTitle, testCode, model) {
129
+ const ollama = new Ollama({ host: OLLAMA_HOST });
130
+ const modelName = model || process.env.OLLAMA_MODEL || DEFAULT_MODEL;
131
+
132
+ const response = await ollama.chat({
133
+ model: modelName,
134
+ messages: [
135
+ { role: 'system', content: SYSTEM_PROMPT },
136
+ { role: 'user', content: buildUserPrompt(testTitle, testCode) },
137
+ ],
138
+ options: {
139
+ temperature: 0.2,
140
+ },
141
+ });
142
+
143
+ return response.message?.content;
144
+ }
145
+
146
+ /**
147
+ * Analyze a single test's code and return structured steps using Ollama (FREE, local).
148
+ *
149
+ * @param {string} testTitle - The test title
150
+ * @param {string} testCode - The full test code block
151
+ * @param {Object} options - { model: string }
152
+ * @returns {Promise<Array<{step: string, result: string}>>}
153
+ */
154
+ export async function analyzeTestWithAI(testTitle, testCode, apiKey, options = {}) {
155
+ const model = options.model || process.env.OLLAMA_MODEL || DEFAULT_MODEL;
156
+
157
+ try {
158
+ const content = await analyzeWithOllama(testTitle, testCode, model);
159
+ return parseAIResponse(content);
160
+ } catch (error) {
161
+ // Provide helpful error messages
162
+ if (error.message?.includes('ECONNREFUSED') || error.message?.includes('fetch failed')) {
163
+ console.warn(` ⚠ Ollama not running. Start it with: ollama serve`);
164
+ } else if (error.message?.includes('model') && error.message?.includes('not found')) {
165
+ console.warn(` ⚠ Model "${model}" not found. Pull it with: ollama pull ${model}`);
166
+ } else {
167
+ console.warn(` ⚠ AI analysis failed for "${testTitle}": ${error.message}`);
168
+ }
169
+ return null; // Return null to indicate failure, caller will use fallback
170
+ }
171
+ }
172
+
173
+ /**
174
+ * Fallback: Extract steps using regex patterns (no AI).
175
+ * Less accurate but works without Ollama.
176
+ */
177
+ export function extractStepsWithRegex(testCode, isCypress = false) {
178
+ const steps = [];
179
+
180
+ // Common Playwright patterns
181
+ const patterns = [
182
+ // page.goto or cy.visit
183
+ {
184
+ regex: /(?:page\.goto|cy\.visit)\s*\(\s*['"`]([^'"`]+)['"`]/gi,
185
+ format: (match) => ({ step: `Navigate to ${match[1]}`, result: 'Page loads successfully' }),
186
+ },
187
+ // page.fill or cy.get().type() with getByLabel
188
+ {
189
+ regex: /(?:getByLabel|getByPlaceholder)\s*\([^)]+\)\.fill\s*\(\s*['"`]([^'"`]+)['"`]/gi,
190
+ format: (match) => {
191
+ const labelMatch = match.input.match(/(?:getByLabel|getByPlaceholder)\s*\(\s*['"`/]([^'"`/]+)/i);
192
+ const label = labelMatch ? labelMatch[1].replace(/\\?\/i$/, '') : 'field';
193
+ return { step: `Enter '${match[1]}' in the ${label} field`, result: 'Value is entered' };
194
+ },
195
+ },
196
+ // page.click or getByRole().click()
197
+ {
198
+ regex: /getByRole\s*\(\s*['"`]([^'"`]+)['"`]\s*,\s*\{[^}]*name:\s*['"`/]([^'"`/]+)/gi,
199
+ format: (match) => ({ step: `Click the '${match[2].replace(/\\?\/i$/, '')}' ${match[1]}`, result: `${match[1]} is clicked` }),
200
+ },
201
+ // expect toBeVisible
202
+ {
203
+ regex: /expect\s*\([^)]+\)\.toBeVisible\s*\(\s*\)/gi,
204
+ format: () => ({ step: 'Verify element is visible', result: 'Element is visible on the page' }),
205
+ },
206
+ // expect toHaveURL
207
+ {
208
+ regex: /expect\s*\([^)]+\)\.toHaveURL\s*\(\s*['"`/]([^'"`/]+)/gi,
209
+ format: (match) => ({ step: 'Verify URL', result: `URL matches ${match[1]}` }),
210
+ },
211
+ // expect toHaveText or toContainText
212
+ {
213
+ regex: /expect\s*\([^)]+\)\.(?:toHaveText|toContainText)\s*\(\s*['"`/]([^'"`/]+)/gi,
214
+ format: (match) => ({ step: 'Verify text content', result: `Text '${match[1]}' is displayed` }),
215
+ },
216
+ // cy.get().should()
217
+ {
218
+ regex: /cy\.get\s*\([^)]+\)\.should\s*\(\s*['"`]([^'"`]+)['"`]/gi,
219
+ format: (match) => ({ step: 'Verify element state', result: `Element should ${match[1]}` }),
220
+ },
221
+ ];
222
+
223
+ for (const { regex, format } of patterns) {
224
+ let match;
225
+ regex.lastIndex = 0; // Reset regex state
226
+ while ((match = regex.exec(testCode)) !== null) {
227
+ try {
228
+ const step = format(match);
229
+ if (step && step.step) {
230
+ steps.push(step);
231
+ }
232
+ } catch (_) {
233
+ // Skip malformed matches
234
+ }
235
+ }
236
+ }
237
+
238
+ // If no steps extracted, provide a generic one
239
+ if (steps.length === 0) {
240
+ const fw = isCypress ? 'Cypress' : 'Playwright';
241
+ steps.push({
242
+ step: `Execute ${fw} automated test`,
243
+ result: 'Test passes with all assertions verified',
244
+ });
245
+ }
246
+
247
+ return steps;
248
+ }
249
+
250
+ /**
251
+ * Check if AI analysis is available (Ollama is running).
252
+ */
253
+ export async function isAIAvailable() {
254
+ return await checkOllamaConnection();
255
+ }
256
+
257
+ /**
258
+ * Get AI configuration.
259
+ * @returns {{ provider: string, model: string }}
260
+ */
261
+ export function getAIConfig(env) {
262
+ return {
263
+ provider: 'ollama',
264
+ apiKey: null,
265
+ model: env.OLLAMA_MODEL || DEFAULT_MODEL,
266
+ };
267
+ }
package/lib/config.js CHANGED
@@ -11,37 +11,19 @@ const CONFIG_FILES = [
11
11
  '.config/am-browserstack-sync.json',
12
12
  ];
13
13
 
14
- /**
15
- * Auto-detect e2e directory based on common framework structures.
16
- * Checks for existence in order: playwright/e2e, cypress/e2e, then fallback.
17
- */
18
- function detectE2eDir(resolvedCwd) {
19
- const candidates = [
20
- { e2eDir: 'playwright/e2e', csvOutputDir: 'playwright/e2e-csv' },
21
- { e2eDir: 'cypress/e2e', csvOutputDir: 'cypress/e2e-csv' },
22
- { e2eDir: 'e2e', csvOutputDir: 'e2e-csv' }, // Generic fallback
23
- ];
24
- for (const candidate of candidates) {
25
- const fullPath = path.join(resolvedCwd, candidate.e2eDir);
26
- if (fs.existsSync(fullPath)) {
27
- return candidate;
28
- }
29
- }
30
- // Default to playwright if nothing found (user will see "directory not found" error)
31
- return { e2eDir: 'playwright/e2e', csvOutputDir: 'playwright/e2e-csv' };
32
- }
14
+ const DEFAULTS = {
15
+ e2eDir: 'playwright/e2e',
16
+ csvOutputDir: 'playwright/e2e-csv',
17
+ };
33
18
 
34
19
  /**
35
- * Load config from cwd: env > config file > package.json field > auto-detect.
20
+ * Load config from cwd: env > config file > package.json field > defaults.
36
21
  * Returns { cwd, e2eDir, csvOutputDir } (paths are absolute).
37
22
  */
38
23
  export function loadConfig(cwd) {
39
24
  const resolvedCwd = path.resolve(cwd || process.cwd());
40
-
41
- // Auto-detect based on existing directory structure
42
- const detected = detectE2eDir(resolvedCwd);
43
- let e2eDir = detected.e2eDir;
44
- let csvOutputDir = detected.csvOutputDir;
25
+ let e2eDir = DEFAULTS.e2eDir;
26
+ let csvOutputDir = DEFAULTS.csvOutputDir;
45
27
 
46
28
  // package.json field
47
29
  const pkgPath = path.join(resolvedCwd, 'package.json');
package/lib/enrich.js CHANGED
@@ -1,8 +1,16 @@
1
1
  /**
2
2
  * Enrich test cases with state, type, steps, tags, description, etc.
3
3
  * Supports Playwright and Cypress; infer tags from spec name + title.
4
+ * Supports AI-powered step extraction using Ollama (FREE, local).
4
5
  */
5
6
 
7
+ import {
8
+ analyzeTestWithAI,
9
+ extractStepsWithRegex,
10
+ getAIConfig,
11
+ checkOllamaConnection,
12
+ } from './ai-analyzer.js';
13
+
6
14
  const DEFAULT_STATE = 'Active';
7
15
  const DEFAULT_CASE_TYPE = 'Functional';
8
16
  const DEFAULT_AUTOMATION_STATUS = 'automated';
@@ -34,16 +42,31 @@ function inferTags(specBaseName, title, isCypress) {
34
42
 
35
43
  /**
36
44
  * Enrich a single case with state, case_type, steps, expected_results, jira_issues, automation_status, tags, description.
45
+ * @param {Object} case_ - Test case with id, title, code
46
+ * @param {string} specBaseName - Base name of spec file
47
+ * @param {boolean} isCypress - Whether this is a Cypress test
48
+ * @param {Array} aiSteps - AI-generated steps (if available)
37
49
  */
38
- export function enrichCase(case_, specBaseName, isCypress = false) {
50
+ export function enrichCase(case_, specBaseName, isCypress = false, aiSteps = null) {
39
51
  const title = case_.title || '';
40
52
  const id = case_.id || '';
53
+ const code = case_.code || '';
41
54
  const fw = isCypress ? FRAMEWORK.cypress : FRAMEWORK.playwright;
42
55
  const tags = inferTags(specBaseName, title, isCypress);
43
- const steps = case_.steps && case_.steps.length > 0
44
- ? case_.steps
45
- : [{ step: fw.step, result: DEFAULT_EXPECTED_RESULT }];
46
- const expectedResults = case_.expected_results ?? steps.map((s) => s.result).join(' ');
56
+
57
+ // Determine steps: AI steps > regex-extracted steps > default
58
+ let steps;
59
+ if (aiSteps && aiSteps.length > 0) {
60
+ steps = aiSteps;
61
+ } else if (code) {
62
+ // Try regex-based extraction from code
63
+ steps = extractStepsWithRegex(code, isCypress);
64
+ } else {
65
+ // Fallback to generic step
66
+ steps = [{ step: fw.step, result: DEFAULT_EXPECTED_RESULT }];
67
+ }
68
+
69
+ const expectedResults = case_.expected_results ?? steps.map((s) => s.result).filter(Boolean).join(' | ');
47
70
  const description = case_.description ?? `${fw.desc}. ID: ${id}. ${title}`;
48
71
 
49
72
  return {
@@ -62,12 +85,74 @@ export function enrichCase(case_, specBaseName, isCypress = false) {
62
85
  }
63
86
 
64
87
  /**
65
- * Enrich all cases in specsMap. Mutates cases in place; returns specsMap.
88
+ * Enrich all cases in specsMap (synchronous, no AI).
89
+ * Use this when AI is not available or not desired.
66
90
  */
67
91
  export function enrichSpecsMap(specsMap) {
68
92
  for (const [baseName, data] of specsMap) {
69
93
  const isCypress = data.isCypress === true;
70
- data.cases = data.cases.map((c) => enrichCase(c, baseName, isCypress));
94
+ data.cases = data.cases.map((c) => enrichCase(c, baseName, isCypress, null));
95
+ }
96
+ return specsMap;
97
+ }
98
+
99
+ /**
100
+ * Enrich all cases in specsMap with AI-powered step analysis using Ollama.
101
+ * Falls back to regex extraction if Ollama is not running.
102
+ *
103
+ * Ollama runs 100% locally - no data sent to cloud, completely free.
104
+ * Install: https://ollama.ai
105
+ * Then: ollama pull llama3.2
106
+ *
107
+ * @param {Map} specsMap - Map of spec name -> { specFile, cases, isCypress }
108
+ * @param {Object} options - { useAI: boolean, model: string }
109
+ */
110
+ export async function enrichSpecsMapWithAI(specsMap, options = {}) {
111
+ const wantsAI = options.useAI !== false;
112
+ const aiConfig = getAIConfig(process.env);
113
+ const model = options.model || aiConfig.model;
114
+
115
+ // Check if Ollama is running
116
+ let ollamaAvailable = false;
117
+ if (wantsAI) {
118
+ ollamaAvailable = await checkOllamaConnection();
71
119
  }
120
+
121
+ if (wantsAI && ollamaAvailable) {
122
+ console.log(`\n🦙 Ollama (FREE, local) - analyzing test steps (model: ${model})`);
123
+ } else if (wantsAI && !ollamaAvailable) {
124
+ console.log('\n📝 Using regex-based step extraction');
125
+ console.log('');
126
+ console.log(' 💡 For AI-powered step analysis (FREE, runs locally):');
127
+ console.log(' 1. Download Ollama from https://ollama.ai');
128
+ console.log(' 2. Run: ollama pull llama3.2');
129
+ console.log(' 3. Start: ollama serve');
130
+ console.log('');
131
+ } else {
132
+ console.log('\n📝 Using regex-based step extraction (AI disabled with --no-ai)');
133
+ }
134
+
135
+ for (const [baseName, data] of specsMap) {
136
+ const isCypress = data.isCypress === true;
137
+
138
+ if (wantsAI && ollamaAvailable) {
139
+ // Analyze each test with Ollama
140
+ for (let i = 0; i < data.cases.length; i++) {
141
+ const testCase = data.cases[i];
142
+ let aiSteps = null;
143
+
144
+ if (testCase.code) {
145
+ console.log(` Analyzing: ${testCase.title}...`);
146
+ aiSteps = await analyzeTestWithAI(testCase.title, testCase.code, null, { model });
147
+ }
148
+
149
+ data.cases[i] = enrichCase(testCase, baseName, isCypress, aiSteps);
150
+ }
151
+ } else {
152
+ // Fallback: regex extraction
153
+ data.cases = data.cases.map((c) => enrichCase(c, baseName, isCypress, null));
154
+ }
155
+ }
156
+
72
157
  return specsMap;
73
158
  }
package/lib/index.js CHANGED
@@ -1,7 +1,7 @@
1
1
  import dotenv from 'dotenv';
2
2
  import path from 'path';
3
3
  import { analyzeSpecs } from './parser.js';
4
- import { enrichSpecsMap } from './enrich.js';
4
+ import { enrichSpecsMap, enrichSpecsMapWithAI } from './enrich.js';
5
5
  import { writeCsvFiles } from './csv.js';
6
6
  import { syncToBrowserStack } from './browserstack.js';
7
7
  import { loadConfig, getBrowserStackEnv } from './config.js';
@@ -18,6 +18,8 @@ import { promptSpecSelection } from './prompt.js';
18
18
  * @param {string[]} [options.spec] - Sync only these spec base names (e.g. ['login.spec']). Skips prompt.
19
19
  * @param {boolean} [options.interactive] - If false, skip prompt and sync all (default: true when no --all/--spec)
20
20
  * @param {Object} [options.auth] - { username, accessKey, projectId } (default: from env)
21
+ * @param {boolean} [options.useAI] - If true, use AI to analyze test steps (requires OPENAI_API_KEY)
22
+ * @param {string} [options.model] - OpenAI model to use (default: gpt-4o-mini)
21
23
  */
22
24
  export async function runSync(options = {}) {
23
25
  const config = loadConfig(options.cwd);
@@ -28,13 +30,19 @@ export async function runSync(options = {}) {
28
30
  const e2eDir = options.e2eDir != null ? path.resolve(cwd, options.e2eDir) : config.e2eDir;
29
31
  const csvOutputDir = options.csvOutputDir != null ? path.resolve(cwd, options.csvOutputDir) : config.csvOutputDir;
30
32
  const csvOnly = options.csvOnly === true;
33
+ const useAI = options.useAI !== false; // Default to true (use AI if key available)
31
34
 
32
35
  let specsMap = analyzeSpecs(e2eDir);
33
36
  if (specsMap.size === 0) {
34
37
  console.log('No spec files found in', e2eDir);
35
38
  return;
36
39
  }
37
- enrichSpecsMap(specsMap);
40
+
41
+ // Enrich with AI-powered step analysis (falls back to regex if no API key)
42
+ await enrichSpecsMapWithAI(specsMap, {
43
+ useAI,
44
+ model: options.model,
45
+ });
38
46
 
39
47
  if (!csvOnly) {
40
48
  if (options.all) {
package/lib/parser.js CHANGED
@@ -21,13 +21,83 @@ export function extractTestTitles(content) {
21
21
  return titles;
22
22
  }
23
23
 
24
+ /**
25
+ * Extract test blocks with their full code body.
26
+ * Returns array of { title: string, code: string, startLine: number, endLine: number }
27
+ */
28
+ export function extractTestBlocks(content) {
29
+ const tests = [];
30
+ const lines = content.split('\n');
31
+
32
+ // Regex to match start of a test: test('title', async ({ page }) => {
33
+ // Captures: 1=test/it, 2=title (single/double quote), 3=title (backtick)
34
+ const testStartRegex = /\b(test|it)\s*(?:\.\s*(?:only|skip))?\s*\(\s*(?:['"]([^'"]*)['"]\s*|`([^`]*)`\s*)/;
35
+
36
+ let i = 0;
37
+ while (i < lines.length) {
38
+ const line = lines[i];
39
+ const match = testStartRegex.exec(line);
40
+
41
+ if (match && !line.includes('describe')) {
42
+ const title = (match[2] || match[3] || '').trim();
43
+ if (!title) {
44
+ i++;
45
+ continue;
46
+ }
47
+
48
+ // Find the opening brace of the test function
49
+ let braceCount = 0;
50
+ let foundOpening = false;
51
+ let startLine = i;
52
+ let codeLines = [];
53
+
54
+ // Scan from this line to find the complete test block
55
+ for (let j = i; j < lines.length; j++) {
56
+ const currentLine = lines[j];
57
+ codeLines.push(currentLine);
58
+
59
+ // Count braces
60
+ for (const char of currentLine) {
61
+ if (char === '{') {
62
+ braceCount++;
63
+ foundOpening = true;
64
+ } else if (char === '}') {
65
+ braceCount--;
66
+ }
67
+ }
68
+
69
+ // If we found the opening brace and now braces are balanced, we found the end
70
+ if (foundOpening && braceCount === 0) {
71
+ tests.push({
72
+ title,
73
+ code: codeLines.join('\n'),
74
+ startLine: startLine + 1, // 1-indexed
75
+ endLine: j + 1,
76
+ });
77
+ i = j + 1;
78
+ break;
79
+ }
80
+ }
81
+
82
+ // If we didn't find balanced braces, just move to next line
83
+ if (braceCount !== 0) {
84
+ i++;
85
+ }
86
+ } else {
87
+ i++;
88
+ }
89
+ }
90
+
91
+ return tests;
92
+ }
93
+
24
94
  /** Match Playwright (*.spec, *.test) and Cypress (*.cy) spec files */
25
95
  const SPEC_FILE_PATTERN = /\.(spec|test|cy)\.(ts|js)$/i;
26
96
 
27
97
  /**
28
98
  * For each spec file in e2eDir, extract tests and assign TC-001, TC-002, ...
29
99
  * Supports Playwright (*.spec.ts/js, *.test.ts/js) and Cypress (*.cy.ts/js).
30
- * Returns Map<specBasename, { specFile, cases: Array<{ id, title }> }>
100
+ * Returns Map<specBasename, { specFile, fileContent, cases: Array<{ id, title, code }> }>
31
101
  */
32
102
  export function analyzeSpecs(e2eDir) {
33
103
  if (!fs.existsSync(e2eDir)) {
@@ -38,13 +108,38 @@ export function analyzeSpecs(e2eDir) {
38
108
  for (const file of files) {
39
109
  const filePath = path.join(e2eDir, file);
40
110
  const content = fs.readFileSync(filePath, 'utf-8');
41
- const titles = extractTestTitles(content);
111
+ const isCypress = /\.cy\.(ts|js)$/i.test(file);
112
+
113
+ // Extract full test blocks with code
114
+ const testBlocks = extractTestBlocks(content);
115
+
42
116
  const baseName = path.basename(file, path.extname(file));
43
- const cases = titles.map((title, index) => ({
117
+ const cases = testBlocks.map((block, index) => ({
44
118
  id: `TC-${String(index + 1).padStart(3, '0')}`,
45
- title: title.replace(/^\s*TC-\d+\s*[-–]\s*/i, '').trim() || title,
119
+ title: block.title.replace(/^\s*TC-\d+\s*[-–]\s*/i, '').trim() || block.title,
120
+ code: block.code, // Full test code for AI analysis
121
+ startLine: block.startLine,
122
+ endLine: block.endLine,
46
123
  }));
47
- results.set(baseName, { specFile: file, cases, isCypress: /\.cy\.(ts|js)$/i.test(file) });
124
+
125
+ // Fallback: if extractTestBlocks found nothing, use extractTestTitles
126
+ if (cases.length === 0) {
127
+ const titles = extractTestTitles(content);
128
+ titles.forEach((title, index) => {
129
+ cases.push({
130
+ id: `TC-${String(index + 1).padStart(3, '0')}`,
131
+ title: title.replace(/^\s*TC-\d+\s*[-–]\s*/i, '').trim() || title,
132
+ code: '', // No code available
133
+ });
134
+ });
135
+ }
136
+
137
+ results.set(baseName, {
138
+ specFile: file,
139
+ fileContent: content, // Keep full file content for reference
140
+ cases,
141
+ isCypress,
142
+ });
48
143
  }
49
144
  return results;
50
145
  }
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "@ash-mallick/browserstack-sync",
3
- "version": "1.0.6",
4
- "description": "Sync Playwright & Cypress e2e specs to CSV and BrowserStack Test Management (one folder per spec)",
3
+ "version": "1.1.1",
4
+ "description": "Sync Playwright & Cypress e2e specs to CSV and BrowserStack Test Management with FREE AI-powered test step extraction using Ollama (local)",
5
5
  "author": "Ashutosh Mallick",
6
6
  "type": "module",
7
7
  "main": "lib/index.js",
@@ -28,6 +28,12 @@
28
28
  "e2e",
29
29
  "test-management",
30
30
  "sync",
31
+ "ai",
32
+ "ollama",
33
+ "llama",
34
+ "test-steps",
35
+ "free",
36
+ "local",
31
37
  "ash-mallick"
32
38
  ],
33
39
  "license": "MIT",
@@ -36,13 +42,13 @@
36
42
  "url": ""
37
43
  },
38
44
  "dependencies": {
39
- "dotenv": "^16.4.5"
45
+ "dotenv": "^16.4.5",
46
+ "ollama": "^0.6.3"
40
47
  },
41
48
  "devDependencies": {
42
49
  "@playwright/test": "^1.49.0",
43
50
  "playwright": "^1.49.0"
44
51
  },
45
- "peerDependencies": {},
46
52
  "peerDependenciesMeta": {},
47
53
  "engines": {
48
54
  "node": ">=18"