testeiya 0.3.9 → 0.3.10

package/skills/testomatio/test-automation/automate-manual-test-cases/references/CODECEPTJS_BEST_PRACTICES.md

@@ -0,0 +1,182 @@
+# CodeceptJS Best Practices
+
+High-level CodeceptJS-specific recommendations. They complement general testing and POM best practices.
+Prefer adapting to the existing project over enforcing new patterns.
+
+## Architecture
+
+Think in layers of responsibility, not folders. Each layer has one job — do not mix locators, business logic, and assertions:
+
+- Pages: dumb locator containers with simple UI interactions only. No business logic.
+- Actors (I) / Custom Steps: own business logic; orchestrate Page actions into domain workflows, multi-step interactions, conditional logic.
+- Helpers / Utilities: low-level technical operations — browser automation wrappers (Playwright, WebDriver, Puppeteer), API calls, DB access, file operations, custom locators.
+- Test Data / Constants: shared configuration, constants, enums, mock data, fixtures.
+- Plugins / Extensions: optional enhancements — auto-login, soft assertions, reporting.
+- Tests / Specs: independent, readable, scenario-focused.
+
+Choose the simplest approach that matches the project:
+- If project uses Page Objects => use them.
+- Else => interact directly in tests.
+
+### Register New Page Objects
+
+After creating a new page object file, add it to:
+1. `codecept.conf.ts` — `include` section
+2. `steps.d.ts` — type declarations
+
+## Naming Rules
+
+No abbreviations — always write full words:
+
+| Wrong | Correct |
+|-------|---------|
+| `Nav` | `Navigation` |
+| `Btn` | `Button` |
+| `Pg`  | `Page` |
+
+Locator names must include the element type as a suffix:
+
+| Wrong (no type) | Correct (with type) |
+|-----------------|---------------------|
+| `compareOverview` | `compareOverviewSection` |
+| `finishRun` | `finishRunButton` |
+| `runGroupTitle` | `runGroupTitleHeader` |
+
+## Locator Strategy
+
+Priority:
+1. Semantic text / element roles (`I.seeElement()`, `{ name: 'field' }`).
+2. CSS ID selectors.
+3. `data-testid` (when semantic locators aren't possible).
+4. Custom locators (last resort).
+
+```javascript
+// Good locators
+I.click('Submit')
+I.fillField('Email', 'user@test.com')
+I.click({ testId: 'submit-form-btn' })
+I.click('.btn-primary.submit-form') // only if necessary
+```
+
+### Reusing Locators
+
+- Before creating a new locator, search `src/pages/`, `pages/`, `components/`.
+- If a locator for the same element already exists — reuse it.
+- Avoid duplicates (e.g. `reportButton` vs `combinedReportButton`) — they create confusion and maintenance issues.
+
+### Tab Active State Verification
+
+After clicking a tab, verify it is active. Different implementations use different indicators:
+
+```typescript
+private generateActiveTabLocator(tabName: string) {
+  return locate('[role="tab"][aria-selected="true"]').withText(tabName);
+}
+
+checkThatTabIsActive(tabName: string) {
+  const activeTab = this.generateActiveTabLocator(tabName);
+  I.waitForElement(activeTab, timeouts.THREE);
+  I.seeElement(activeTab);
+}
+```
+
+## Custom Steps (Actor Methods)
+
+- Multi-step logic belongs in Actor methods.
+- Use for login, conditional flows, API+UI flows, or explicit waits.
+
+```javascript
+module.exports = function() {
+  return actor({
+    async loginAs(email, password) {
+      I.amOnPage('/login')
+      I.fillField('Email', email)
+      I.fillField('Password', password)
+      I.click('Sign In')
+      I.waitForUrl('**/dashboard')
+    },
+
+    async ensureLoggedIn(user) {
+      const loggedIn = await I.grabNumberOfVisibleElements('.logout-button') > 0
+      if (!loggedIn) await this.loginAs(user.email, user.password)
+    }
+  })
+}
+```
+
+## Fixtures & Hooks
+
+- Use `Before` hook for setup and authentication.
+- Use `After` hook for cleanup (API cleanup only, no UI).
+- Reuse existing fixtures when present in the project.
+
+```typescript
+Before(async ({ app }) => {
+  const login = await app.login(token);
+  app.setToken(`Bearer ${login.data.jwt}`);
+  loginAs("mainUser");
+});
+
+After(async ({ app }) => {
+  await app.cleanup();
+});
+```
+
+### Test Data Usage
+
+Prefer shared constants, enums, or test data over hardcoded values:
+
+```typescript
+import { timeouts } from "../testData/timeout";
+
+I.waitForElement(locator, timeouts.THREE); // 3s
+```
+
+### API in Preconditions, UI for Test Steps
+
+- Before hook: use API to create test data (users, projects, runs, etc.).
+- Scenario body: use UI interactions — that is what the test is verifying.
+- Exception: simple navigation to a starting URL in Before is acceptable.
+
+### Formatting Rules
+
+- One blank line between every action in Scenario body.
+- No step comments inside Scenario body — method names must be self-documenting.
+- No magic strings in test body — extract to `const` at top of file.
+- No long `projectData.x.y.z.w` chains in test body — extract to named `const`.
+
+## Test Tags & Filtering
+
+- Tagging and filtering are project-specific.
+- If the project already uses tags (e.g., @Smoke, @Regression, @P0), follow existing conventions.
+- Do not introduce new tagging systems unless explicitly required.
+
+## Debugging & Reporting
+
+- Use `--debug` / `--verbose`.
+- Save screenshots: `I.saveScreenshot('name.png')`.
+- Playwright traces can be enabled in helper config.
+- Custom helpers: `DebugHelper` for logs, screenshots, pause.
+
+## Anti-Patterns (Avoid)
+
+- Never hardcode test data — use data files or environment variables.
+- Never share state between tests — each test sets up its own state.
+- Never commit secrets or credentials — use environment variables.
+- Never ignore flaky tests — fix immediately with proper waits.
+- Never skip test cleanup — use After hooks.
+- Prefer semantic locators (`{ name: 'button' }`) over CSS-only (`.btn-primary`).
+
+Code smells from legacy code (do not repeat):
+- Hardcoded waits (like `I.wait(n)`).
+- Inline cleanup inside test scenarios (`// cleanup`).
+- Locators directly in test files instead of page objects.
+- Hardcoded IDs and config values in tests.
+
+## Checklist
+
+- [ ] Keep tests independent — each test sets up its own state
+- [ ] Use test data constants instead of hardcoded values
+- [ ] Follow naming rules (full words, element type suffix)
+- [ ] Reuse existing locators before creating new ones
+- [ ] Add assertions based on expected results from manual steps

package/skills/testomatio/test-automation/automate-manual-test-cases/references/FINAL_SUMMARY_TEMPLATE.md

@@ -0,0 +1,22 @@
+# Final Summary Template
+
+Use this template after completing the conversion process:
+
+```
+**Summary:**
+- Framework: [Playwright/CodeceptJS/Cypress]
+- Tests generated: [N]
+- Passed: [N]
+- Needs review: [N]
+
+**Generated Files:**
+- [path/to/test1.spec.ts] ✅
+- [path/to/test2.spec.ts] ⚠️ ([issue])
+- [path/to/test3.spec.ts] ✅
+
+**Key Issues:**
+[Short description of main problems, if any]
+
+**Next steps:**
+...
+```

package/skills/testomatio/test-automation/automate-manual-test-cases/references/PLAYWRIGHT_BEST_PRACTICES.md

@@ -0,0 +1,90 @@
+# Playwright Best Practices
+
+High-level Playwright-specific recommendations. They complement general testing and POM best practices.
+Prefer adapting to the existing project over enforcing new patterns.
+
+## Architecture & Project Structure
+
+- Do not assume a fixed architecture (e.g., Pages, Modules, Fixtures).
+- Detect and follow the existing project structure.
+- Common patterns may include:
+  - Pages: dumb locator containers with simple element interactions only.
+  - Modules: own all business logic, orchestrate Page actions into domain workflows.
+  - Fixtures: dependency injection — tests never instantiate Pages or Modules directly.
+  - `test.step()`: use for reporting, trace analysis, and debugging.
+
+Choose the simplest approach that matches the project:
+- If project uses Page Objects => use them.
+- Else => interact directly in tests.
+
+## Locator Strategy
+
+- Prefer user-facing and semantic locators.
+- Use built-in Playwright locator strategies when possible.
+- Prefer stability over brevity.
+- Avoid selectors tied to layout or styling.
+
+Priority:
+1. Role-based selectors (buttons, links, alerts).
+2. Labels (form inputs).
+3. Visible text.
+4. Test-specific attributes (data-testid).
+5. CSS selectors.
+
+Stay consistent within the project:
+- If the project relies on semantic locators => follow that pattern.
+- If it uses test IDs => prefer `data-testid` as the primary selector.
+
+```typescript
+// Preferred (semantic)
+await page.getByRole('button', { name: 'Submit' }).click()
+await page.getByLabel('Email').fill('user@test.com')
+
+// Fallback (only if necessary)
+await page.locator('.btn-primary.submit-form').click()
+```
+
+## Fixtures
+
+- Playwright provides built-in fixtures (e.g., `page`, `context`, `request`).
+- Reuse existing custom fixtures for setup, authentication, or shared state.
+- If no fixture structure exists, simple tests can interact directly without introducing abstraction.
+- Introduce new fixtures only when they clearly improve reuse, readability, or setup consistency.
+
+## API Testing
+
+Playwright supports both API and UI interactions via `request`.
+
+- Prefer API for: test setup, teardown, data preparation.
+- Use UI for: behavior validation, end-user flows.
+- Pattern: setup via API, verify via UI.
+
+## Test Tags & Filtering
+
+- Tagging and filtering are project-specific.
+- If the project already uses tags (e.g., @Smoke, @Regression, @P0), follow existing conventions.
+- Do not introduce new tagging systems unless explicitly required.
+
+## Debugging & Reporting
+
+- Use built-in Playwright debugging tools or Playwright MCP when needed: traces, screenshots, logs.
+- Add extra steps or logs only if they improve clarity.
+- Avoid excessive logging or noise in tests.
+
+## Anti-Patterns (Avoid)
+
+- Never embed test data directly in test logic — prefer centralized test data (JSON, factories, or generators).
+- Never share state between tests — each test is isolated.
+- Never ignore flaky tests — fix immediately with proper waits.
+- Never use arbitrary `sleep()` — use explicit waits instead.
+- Never commit secrets or credentials — use environment variables.
+- Never over-abstract simple test flows.
+
+## Checklist
+
+- [ ] Use `test.step()` for reporting and trace analysis
+- [ ] Use API for setup/teardown over UI when possible
+- [ ] Keep tests independent — each test sets up its own state
+- [ ] Use test data instead of hardcode
+- [ ] Run tests in parallel (when safe)
+- [ ] Implement soft assertions for non-critical checks

package/skills/testomatio/test-automation/automate-manual-test-cases/references/POM_BEST_PRACTICES.md

@@ -0,0 +1,53 @@
+# Page Object Model (POM) Best Practices
+
+## Core Principles
+
+### Single Responsibility
+- Each Page Object represents one page or a clearly defined UI component.
+- Each method performs a single, clear action.
+- Keep structure flat and simple.
+- Avoid unnecessary abstractions.
+- Introduce shared components only when reuse is clear.
+
+### Encapsulation
+- Hide implementation details such as selectors and waiting logic.
+- Expose only meaningful, high-level actions to tests.
+- Do not expose raw selectors outside the Page Object.
+
+### Reusability
+- Extract shared logic only when reuse is clear.
+- Extract shared functionality into parent classes.
+- Use composition over inheritance when appropriate.
+
+## Wait Strategies
+
+- Use condition-based waits (element visible, enabled, loaded).
+- Avoid fixed delays or hard waits.
+- Rely on framework auto-waiting where available.
+- Do not duplicate waiting logic unnecessarily.
+
+```typescript
+// Bad: hard wait
+await page.waitForTimeout(5000)
+```
+
+## Error Handling
+
+- Do not override or wrap framework errors for UI interactions.
+- Rely on built-in error handling for element actions.
+- Avoid adding manual visibility or existence checks before actions.
+
+- Add contextual error handling only for:
+  - API interactions
+  - Data processing
+  - Non-UI logic
+
+## Summary Checklist
+
+- [ ] One class per page/component.
+- [ ] Private/protected locators.
+- [ ] Explicit waits over hard waits.
+- [ ] Meaningful method names.
+- [ ] No assertions in Page Objects.
+- [ ] Reusable component structure.
+- [ ] Clear error messages.

package/skills/testomatio/test-automation/automate-manual-test-cases/references/TEST_DATA_MANAGEMENT.md

@@ -0,0 +1,52 @@
+# Test Data Management
+
+## Core Principles
+
+- Test data must support the test scenario, not define it.
+- Keep test data minimal, relevant, and easy to understand.
+- Ensure tests remain independent and do not rely on shared mutable data.
+- Prefer clarity over reusability when in conflict.
+
+## Data Placement Strategy
+
+When deciding where to store test data:
+1. If data is used only within a single test => Keep it simple or inline within that test.
+2. If data is reused across multiple tests => Extract it into a shared test-data.
+3. If data is large, structured, or tabular => Store it in external files (e.g., JSON, CSV).
+  - LOAD data from external file.
+  - ITERATE over dataset in tests.
+4. If data depends on environment (URLs, credentials, configs) => Use environment variables or configuration files.
+  - READ baseUrl from environment.
+  - USE baseUrl in test.
+
+**Priority:** Start local - Extract only when reuse is clear.
+
+## Data Structure Guidelines
+
+- Use clear, descriptive names for all data variables.
+- Keep data structures simple and predictable.
+- Avoid deeply nested or overly complex data unless required.
+- Group related data logically (e.g., user profiles, product sets).
+
+## Test Isolation Rules
+
+- Each test must be fully independent.
+- Tests must not share mutable data or state.
+- Avoid dependencies between tests.
+- Prefer framework-provided isolation over manual resource handling.
+
+## Reusability Guidelines
+
+- Reuse test data only when it reduces duplication and improves clarity.
+- Do not over-centralize data if it makes tests harder to read.
+- Keep frequently used data in shared modules.
+- Keep unique or scenario-specific data close to the test.
+
+## Anti-Patterns to Avoid
+
+- Hardcoding credentials or secrets in test files.
+- Using real user or production data.
+- Sharing state between tests.
+- Creating test data without cleanup.
+- Over-engineering data structures for simple scenarios.
+

package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: debug-fix-failed-flaky-autotests
+description: Diagnose and fix failing automated tests. Analyzes failures, inspects DOM, identifies root causes, and applies targeted fixes using framework tools and MCP/CLI debug modes. Use when tests fail, are flaky, or behave inconsistently (e.g., pass locally but fail in CI).
+license: MIT
+metadata:
+  author: Testomat.io
+  version: 1.0.0
+---
+
+# Debug and Fix Failing Autotests
+
+Diagnose failing automated tests and apply targeted fixes.
+Follow the four steps in order.
+Detailed fixes and code examples: [references/DEBUGGING_QUICK_REFERENCE.md](references/DEBUGGING_QUICK_REFERENCE.md).
+
+## Step 1: Analyze Failure
+
+- Identify the framework. It determines the debug tools:
+  - Playwright: CLI debug, trace viewer, MCP snapshots.
+  - CodeceptJS: `--steps` mode, CDP inspection.
+  - Cypress: devTools, screenshots on failure.
+  - WebdriverIO: Selenium logs.
+- Categorize the error:
+  - Locator (element missing/unreachable).
+  - Timing (timeout/race).
+  - Assertion (expected ≠ actual).
+  - Flow (navigation, preconditions).
+  - Infrastructure (CI/browser/network).
+- End with a hypothesis: root cause + fix category.
+
+## Step 2: Inspect and Diagnose
+
+- DOM inspection:
+  - Playwright: `page.locator('selector').evaluate(el => el.outerHTML)`.
+  - CodeceptJS: `I.executeScript(() => document.querySelector('.selector').outerHTML)`.
+  - MCP snapshot: capture live DOM state for complex elements.
+- Console and network logs:
+  - Console: `page.on('console', msg => console.log(msg.text()))`.
+  - Failed requests: `page.on('requestfailed', r => console.log(r.failure().errorText))`.
+- Trace analysis (Playwright): `npx playwright show-trace trace.zip`.
+  - Inspect timing and order of clicks, navigation, network requests.
+
+## Step 3: Apply Fix
+
+**Fix in priority order: locators → timing → assertions → flow.**
+
+- Locators: use stable selectors (`data-testid` → `aria-label` → `role` → `id` → `text` → CSS/XPath).
+  - Re-fetch stale elements, scroll hidden elements into view, use `.first()` for multiple matches.
+- Timing: use framework-native waits (`waitFor`, `waitForNavigation`, `waitForLoadState('networkidle')`).
+  - **No hard sleeps** like `sleep(5000)` or `wait(2)`.
+- Assertions: use contains/regex for flexible matching.
+  - Verify expected values against the test spec, not assumptions.
+- Flow: ensure preconditions (login, navigation order, test data).
+  - Isolate test state from prior runs.
+
+Before/after examples for each category: [references/DEBUGGING_QUICK_REFERENCE.md](references/DEBUGGING_QUICK_REFERENCE.md).
+
+## Step 4: Verify and Stabilize
+
+Re-run the test:
+
+```bash
+npx playwright test path/to/test.spec.ts
+npx codeceptjs run path/to/test.js
+```
+
+Test is stable when:
+- Passes 2 consecutive runs.
+- Uses resilient locators.
+- Waits properly for dynamic content.
+
+**Max 3 healing attempts total.** After 3 failed attempts: stop, document what was tried, ask the user for guidance.
+
+Document the fix: root cause, applied fix, verification result.
+
+## MCP Debug Tools (Optional)
+
+Use MCP when standard fixes fail, the UI is complex (dropdowns, modals, dynamic content), or live DOM inspection is needed:
+- Playwright MCP: live snapshots, element counts, DOM inspection after each action.
+- CodeceptJS MCP: CDP-based inspection, `--steps` mode for live debugging.
+
+## User Interaction
+
+Ask when unclear:
+
+```
+❓ Which element should I target for this action?
+
+Options:
+1. Submit button (data-testid="submit")
+2. Save button (aria-label="Save")
+3. Other (specify)
+```
+
+Report progress:
+
+```
+🔍 Analyzing failure...
+  - Error: Element not found
+  - Context: Login test, line 42
+
+🔧 Applying fix...
+  - Issue: Selector too broad
+  - Fix: Using getByRole('button', { name: 'Login' })
+
+✅ Test passed! (Run 1/2)
+```

package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/references/DEBUGGING_QUICK_REFERENCE.md

@@ -0,0 +1,113 @@
+# Debugging Quick Reference
+
+Fix in priority order: locators → timing → assertions → flow.
+
+---
+
+## Locator Fixes
+
+| Problem | Avoid | Prefer |
+|---------|-------|--------|
+| Deep nesting | `.parent .child .grandchild` | `[data-testid="target"]` |
+| Index-based | `button:nth-child(2)` | `getByRole('button', { name: 'Submit' })` |
+| Partial text | `button:contains("Save")` | `getByText('Save', { exact: true })` |
+| Dynamic classes | `.btn-primary-123` | `[data-testid="save-btn"]` |
+
+Element state issues:
+
+- Not found => wait for element before action.
+- Not visible => scroll into view.
+- Not clickable => wait for enabled state; check overlays.
+- Stale element => re-fetch element before action.
+- Multiple matches => filter or use `.first()`.
+
+---
+
+## Timing Fixes
+
+Use framework-native waits instead of hard sleeps.
+
+- Avoid hard sleeps (`sleep(5000)`, `wait(2)`) => flaky.
+- Avoid implicit short waits => unreliable.
+- Works locally, fails in CI => CI is slower; increase timeouts and add explicit waits.
+
+**Playwright:**
+```typescript
+await page.getByTestId('loader').waitFor({ state: 'hidden' });
+await page.waitForURL('/dashboard/**');
+await page.waitForResponse(r => r.status() === 200);
+```
+
+**CodeceptJS:**
+```javascript
+I.waitForNavigationVisible();
+I.waitForResponse(response => response.status() === 200);
+```
+
+### Fix Race Conditions
+
+Ensure assertions happen after dynamic content updates.
+Use `expect` with Playwright's built-in waiting, or `locator.waitFor()` only if needed.
+
+```typescript
+// Click triggers dynamic update
+await page.getByRole('button', { name: 'Submit' }).click();
+
+// Wait for result text to appear (optional, only if dynamically loaded)
+await expect(page.getByTestId('result')).toHaveText('Success'); // automatically waits
+```
+
+---
+
+## Assertion Fixes
+
+- Exact match too strict => use `toContain()` or regex.
+- Wrong expected value => verify against test spec.
+- Timing issues => wait before assertion.
+- Multiple elements => use `.first()` or `.nth(0)`.
+
+Examples:
+
+```typescript
+// Before: exact match fails on whitespace
+expect(await page.locator('.title').textContent()).toBe('Hello World');
+
+// After: contains check
+await expect(page.locator('.title')).toContainText('Hello');
+
+// Better: trim if exact needed
+const text = (await page.locator('.title').textContent()).trim();
+expect(text).toBe('Hello World');
+```
+
+---
+
+## Flow Fixes
+
+- Missing preconditions => ensure login, proper navigation, and required test data.
+- Fails on 2nd run => add cleanup hooks.
+- Depends on order => use `beforeEach` for fresh state.
+- Shared state => isolate per test worker.
+
+---
+
+## Framework-Specific Commands
+
+**Playwright:**
+- Show trace: `npx playwright show-trace trace.zip`
+- Run single file: `npx playwright test path/to/test.spec.ts`
+
+**CodeceptJS:**
+- Steps mode: `npx codeceptjs run --steps`
+- Verbose: `npx codeceptjs run --verbose`
+- Single file: `npx codeceptjs run path/to/test.js`
+
+---
+
+## Max 3 Healing Attempts
+
+1. Attempt 1: apply fix based on initial diagnosis.
+2. Attempt 2: if still fails, re-diagnose with new info.
+3. Attempt 3: last try with an alternative approach.
+
+If still failing => STOP, document what was tried, ask the user for guidance.

package/skills/testomatio/test-automation/qa-automation-test-consolidation/SKILL.md

@@ -0,0 +1,29 @@
+---
+name: qa-automation-test-consolidation
+description: Detect redundant tests, duplicated test logic, semantic overlaps, and parameterization opportunities across the entire test suite. Present recommendations and request user approval before refactoring.
+license: MIT
+metadata:
+  author: Testomat.io
+  version: 1.0.0
+---
+
+## Summary
+
+This skill analyzes an entire test suite to identify duplicate tests, semantic overlaps, duplicated test logic, parameterization opportunities, and overlapping business-rule validations. It prioritizes behavioral similarity over code similarity and recommends refactorings that reduce maintenance effort while preserving test coverage and business intent. No changes are applied without explicit user approval.
+
+## How the Loop Works
+
+- Analyze the entire test suite for consolidation opportunities;
+- Group findings by type and confidence level;
+- Present a summary of the proposed changes and request user approval;
+- Apply the approved refactorings;
+- Re-analyze the updated test suite to identify newly exposed consolidation opportunities;
+- Repeat until no additional consolidation opportunities with a confidence level greater than 80% remain.
+
+## Definition of Done
+
+- The entire test suite has been analyzed;
+- All consolidation opportunities have been presented to the user;
+- All approved refactorings have been applied;
+- The test suite has been re-analyzed after each iteration;
+- No duplicate tests, duplicated test logic, parameterization opportunities, or business-rule overlaps with a confidence level greater than 80% remain.