testeiya 0.3.9 → 0.3.14

package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/SKILL.md

@@ -0,0 +1,220 @@
+---
+name: qa-sprint-report-by-testomatio
+description: Generate a QA Sprint Progress Summary Report from Testomat.io TMS data. Use when the user wants to produce a structured end-of-sprint report covering test design coverage, execution results, defect trends, and quality signals. The skill reads data via Testomat.io MCP tools, maps it to the report sections, and outputs an `.html` file by default (or `.md` if explicitly requested).
+license: MIT
+metadata:
+  author: Testomat.io
+  version: 1.0.0
+---
+
+# QA Sprint Report by Testomat.io
+
+## When to Use
+
+- End of sprint — produce a shareable QA report for stakeholders.
+- Sprint review — need a structured view of what was tested, passed, failed, blocked.
+- Mid-sprint status — capture current execution progress.
+- The user mentions "sprint report", "QA summary", "sprint progress", or similar.
+
+## How to Identify the Sprint
+
+The user may identify the sprint in one of these ways (in priority order):
+
+1. **Milestone name or ID** — e.g. "Sprint 42", "Sprint 2024.3" — use `milestones_list(type="Sprint")` to find it.
+2. **Run group title** — e.g. "Sprint 42 Run" — use `rungroups_list` / `rungroups_get`.
+3. **Direct run ID or run title** — use `runs_get` or `runs_list`.
+4. **Plan title or ID** — use `plans_list` / `plans_get`.
+5. **No identifier provided** — ask the user to specify the sprint milestone, run group, or run.
+
+> If the user provides only a sprint name like "Sprint 42", always resolve it to a concrete milestone/rungroup/run first. Do not assume the title directly maps to an ID.
+
+## Rules
+
+- **Do not fabricate data.** If MCP does not return a value, use `[None]` or hide the section. Never invent numbers or statuses.
+- **Resolve sprint first.** Never assume a sprint name maps directly to a run ID without verification.
+- **Hide unavailable sections.** If analytics (Section 8) is not available, omit it entirely rather than leaving placeholder text.
+- **One blank line between actions.** In any generated code or CLI snippets.
+- **Use TQL for filtering.** Always prefer `tql` filters over client-side filtering when calling `runs_list`, `tests_list`, `testruns_list`.
+- **HTML is the default output.** Generate `.html` first; only offer `.md` if the user explicitly asks for it or their initial prompt includes a request to save as `.md`.
+
+---
+
+# MCP Tools Mapping
+
+Each report section maps to specific MCP tools. See the full section structure and metrics in the template:
+
+- [qa-sprint-report.md](./references/qa-sprint-report.md) — section layout with fill-in placeholders
+
+### Quick Commands
+
+| Action | MCP Tool | Key Parameters |
+|--------|----------|----------------|
+| Find sprint milestone | `milestones_list` | `type="Sprint"`, `status="active"` |
+| List runs for sprint | `runs_list` | `tql`: `milestone == '{id}'` |
+| Get run details | `runs_get` | `run_id` |
+| Get test results | `testruns_list` | `run_id`, `filter_status` |
+| Get suite tests | `tests_list` | `suite_id`, `per_page=100` |
+| Check analytics availability | `system_ping` | (Enterprise flag in response) |
+| Get test health analytics | `analytics_tests` | `kind`, `days`, `from`, `to` |
+| Get stats analytics | `analytics_stats` | `kind`, `from`, `to` |
+
+---
+
+# Workflow
+
+### Step 1: Resolve Sprint Identity
+
+Ask the user if no sprint identifier is provided. Try to resolve:
+
+```
+1. milestones_list(type="Sprint") — find milestone matching sprint name
+2. rungroups_list — find rungroup with sprint title
+3. runs_list(tql="title % 'Sprint N'") — find runs by title
+```
+
+### Step 2: Gather Sprint Metadata & Time Range
+
+Collect for the report header:
+
+- `milestones_get(milestone_id)` or `rungroups_get(rungroup_id)` for sprint title/dates.
+- Ask user for QA Lead name if not in TMS.
+
+**Critical: Extract Sprint Time Range**
+
+Resolve the sprint time range in this priority order:
+
+1. **User-provided** — If the user specified a time range in their original prompt, use it.
+2. **Milestone/Rungroup dates** — Extract `start_date` / `due_date` from `milestones_get` or `rungroups_get`.
+3. **Run timestamps** — If milestone/rungroup has no dates, derive from earliest `created_at` and latest `finished_at` across the sprint's runs (`runs_get`).
+4. **Manual input** — If no time range can be determined from TMS, ask the user to specify the sprint period manually.
+
+The time range is required for:
+- `analytics_tests(kind="...", from="YYYY-MM-DD", to="YYYY-MM-DD")`
+- `analytics_stats(kind="...", from="YYYY-MM-DD", to="YYYY-MM-DD")`
+- Filtering runs by sprint period
+
+### Step 3: Collect Run Data
+
+1. `runs_list(tql="milestone == '{id}'")` — all runs for the sprint milestone
+2. For each run, call `runs_get` to get status, counts, environments
+3. `testruns_list(run_id, filter_status)` — get passed/failed/skipped per run
+
+### Step 4: Build Report Sections
+
+Iterate sections 1–9 of the template, populating each with MCP data. Apply rules:
+
+- If a metric is not available → use `[None]` in the cell
+- If an entire section has no data → hide the section (do not show placeholder text)
+- Section 8 (Analytics) → check `system_ping` response for Enterprise capability; if not available, hide
+
+### Step 5: Write Output File
+
+Write the filled report to:
+```
+{user-specified-path}/QA_Sprint_Progress_Report_{SprintName}_{YYYY-MM-DD}.html
+```
+
+If no path specified, save to current working directory.
+
+### Step 6: Offer Markdown Export
+
+After writing the `.html` file, ask the user if they also want to export the report as a `.md` file:
+
+```
+❓ The report has been saved to `{path}`.
+Would you like to also export it as a `.md` file?
+
+1. ✏️ Yes, save as `.md` in the same location
+2. 👍 No, keep the `.html` only
+```
+
+**If user picks option 1:**
+1. Convert the HTML content to Markdown format.
+1. Generate ".md" markdown format report using template from the references - [qa-sprint-report.md](./references/qa-sprint-report.md).
+2. Save as `QA_Sprint_Progress_Report_{SprintName}_{YYYY-MM-DD}.md` in the **same directory** as the HTML file.
+
+**If user picks option 2:**
+- Confirm the `.html` path and end the workflow.
+
+## HTML Template Styling
+
+When exporting to HTML, apply the following styles:
+
+### Color Palette
+
+| Token | Hex | Usage |
+|-------|-----|-------|
+| `--primary-color` | `#6366f1` | Headers, badges, links |
+| `--primary-hover` | `#4f46e5` | Hover states |
+| `--success-color` | `#10b981` | Passed/ready badges |
+| `--danger-color` | `#ef4444` | Failed/blocked badges |
+| `--warning-color` | `#f59e0b` | In-progress badges |
+| `--info-color` | `#3b82f6` | Info badges |
+| `--gray-50` to `--gray-900` | `#f9fafb` → `#111827` | Text hierarchy |
+
+### Typography
+
+- **Font Family:** Inter (Google Fonts) with fallback: `-apple-system, BlinkMacSystemFont, sans-serif`
+- **Font Weights:** 300 (light), 400 (regular), 500 (medium), 600 (semibold), 700 (bold), 800 (extrabold)
+- **Monospace:** `ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas` — for ticket IDs and code
+- **Icons:** Font Awesome 6.4.0 (`fa-solid`)
+
+### Status Badges
+
+| Status | Background | Text Color |
+|--------|------------|------------|
+| `passed`, `ready` | `#d1fae5` | `#065f46` |
+| `failed`, `blocked` | `#fee2e2` | `#991b1b` |
+| `progress` | `#fef3c7` | `#92400e` |
+| `info` | `#dbeafe` | `#1e40af` |
+
+### Key Elements
+
+| Element | Description |
+|---------|-------------|
+| `.main-container` | White glass card (`rgba(255,255,255,0.98)`) with `backdrop-filter: blur(20px)`, max-width 1300px, border-radius 20px |
+| `.header` | Gradient background (`linear-gradient(135deg, #6366f1, #4f46e5)`), white text, flex layout |
+| `.section-card` | White background, 1px border `--gray-200`, border-radius 12px, box-shadow |
+| `.report-table` | Full-width, collapse borders, hover highlight on rows |
+| `.meta-grid` | CSS Grid (`repeat(auto-fit, minmax(220px, 1fr))`), gap 20px |
+| `.instruction-card` | Info box with left 5px border in `--primary-color`, background `#eef2ff` |
+| `.tms-link` | Primary color link with hover underline |
+
+### HTML Structure
+
+```html
+<div class="main-container">
+  <header class="header">
+    <div class="header-title">
+      <h1>QA Sprint Progress Report</h1>
+      <p><i class="fa-solid fa-cubes"></i> [Project Name]</p>
+    </div>
+    <div class="header-badge">
+      <i class="fa-solid fa-calendar-days"></i> Sprint: [Sprint Number]
+    </div>
+  </header>
+
+  <section class="instruction-section">
+    <div class="instruction-card">
+      <h5><i class="fa-solid fa-circle-info"></i> How to Use This Template</h5>
+      <ul>...</ul>
+    </div>
+  </section>
+
+  <main class="content-section">
+    <div class="section-card">
+      <div class="section-title"><i class="fa-solid fa-chart-pie"></i> 1. Sprint Summary Scorecard</div>
+      <div class="meta-grid">...</div>
+    </div>
+    <!-- Repeat section-card for each section -->
+  </main>
+</div>
+```
+
+---
+
+# Report Template Reference
+
+| Section | Description |
+|---------|-------------|
+| [qa-sprint-report.md](./references/qa-sprint-report.md) | Full template with all sections, emoji, and fill-in placeholders |

package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/references/qa-sprint-report.md

@@ -0,0 +1,126 @@
+# QA Sprint Progress Summary Report Example
+
+The **QA Sprint Progress Summary Report** provides a structured, end-of-cycle view of all testing activities carried out during a sprint. It captures test design coverage, execution results, defect trends, and quality signals to keep stakeholders informed on product readiness.
+
+> All sections and metrics in this report are sourced from **Testomat.io TMS** via MCP tools. If a metric or data point is not available in the TMS, the corresponding section is omitted or the value is set to `[None]`. Sections that rely solely on unavailable data are hidden entirely.
+
+## 📋 General Sprint Meta
+* **Project / Stream:** [Project Name / Stream]
+* **QA Manager:** [Your Name]
+* **Sprint Cycle:** [Sprint Number / Name]
+* **Timeline Period:** [DD.MM.YYYY – DD.MM.YYYY]
+
+---
+
+## 📊 1. Sprint Summary Scorecard
+
+| Metric | Value | Notes & Data Source |
+| :--- | :---: | :--- |
+| 🖊️ **New Test Cases Added to TMS** | [N] | Net new TCs authored in Testomat.io during this sprint. |
+| 📋 **Total TCs in Project (End of Sprint)** | [N] | `tests_list` count at sprint close. |
+| ⚙️ **Automation Rate** | [XX%] | Automated TCs ÷ Total TCs — from `analytics_stats(kind="automation-rate-by-date")`. |
+| 📈 **Overall Pass Rate** | [XX%] | From `analytics_stats(kind="success-rate-by-date")` for the sprint window. |
+| 🏁 **Tickets Tested** | [N] | Total number of tickets which were finished during the sprint. |
+| ✅ **Tickets Completed (All Tests Passed)** | [N] | Formally approved and ready for delivery/release -> finished with all passed tests. |
+| 🐛 **Defects/Bugs Identified** | [N] | Newly identified defects during execution. |
+
+---
+
+## 🛠️ 2. Test Design & Coverage Expansion
+
+| Ticket ID | Suite / Feature Area | Test Case Title(s) | Total TCs | TMS State |
+| :--- | :--- | :--- | :---: | :--- |
+| **[PROJ-123]** | Authentication | [TC Title 1]; [TC Title 2] | [N] | [Manual / Automated] |
+| **[PROJ-456]** | Data Export | [TC Title 1] | [N] | [Automated] |
+| **[PROJ-789]** | Data Export | [TC Title 1]; [TC Title 2] | [N] | [Mix: Automated & Manual] |
+| *— No new test cases added —* | | | | |
+
+> **TMS State** column: pull from `tests_list` → `state` field (`Manual` / `Automated` / `Mix: Automated & Manual`).
+
+---
+
+## 🎯 3. Testing Execution Progress by Ticket, Suite
+
+> Report group is **Suite** (the primary TMS grouping). Each row corresponds to a Testomat.io Suite. Use `testruns_list(run_id="...")` to populate Passed / Failed / Skipped counts per suite.
+
+| Ticket Name / Suite / Feature Area | Run Title | Total TCs | Passed | Failed | Skipped | Blocked | Execution % | Suite Release Status |
+| :--- | :--- | :---: | :---: | :---: | :---: | :---: | :---: | :--- |
+| [Suite Name — e.g. Billing] | [Run Title] | [N] | `[N]` | `[N]` | `[N]` | `[N]` | `[XX]%` | 🟡 In Progress |
+| [Suite Name — e.g. Profile] | [Run Title] | [N] | `[N]` | `[N]` | `[N]` | `[N]` | `[XX]%` | ✅ Completed (All Tests Passed) |
+| [Suite Name — e.g. Analytics] | [Run Title] | [N] | `[N]` | `[N]` | `[N]` | `[N]` | `[XX]%` | 🔴 Blocked |
+| **TOTAL** | | **[N]** | **`[N]`** | **`[N]`** | **`[N]`** | **`[N]`** | **`[XX]%`** | **Overall Pass Rate:** `[XX%]` |
+
+> **Status Key:**
+> ✅ **Completed (All Tests Passed):** 100% test execution completed with zero critical defects.
+> 🟡 **In Progress:** Validations are actively running or includes in-progress cases; no major technical hurdles encountered (🟡 >51% pass = In Progress).
+> 🔴 **Blocked:** Discovered failure, issues (🔴<50% pass or blocked).
+
+---
+
+## ✅ 4. Tickets Moved to Sign-Off (already passed)
+*   `[PROJ-XXX]` — `[User Story Name]` | **Verified:** `[DD.MM.YYYY]` | **TMS Run:** `[Link to Testomat.io Run]`
+*   `[PROJ-YYY]` — `[User Story Name]` | **Verified:** `[DD.MM.YYYY]` | **TMS Run:** `[Link to Testomat.io Run]`
+*   *— No tickets signed off during this period —*
+
+---
+
+## 🔁 5. Quality Feedback Loop (Returned to Dev)
+*   `[PROJ-881]` — *`[Feature Name]`*: `[AC #2 failed: App crashes when session expires / Linked to BUG-99]` (**Return Count:** `[1st time / 2nd time]`)
+*   *— No tickets returned to development —*
+
+---
+
+## 🐛 6. Sprint Bug Tracking
+
+| Bug ID | Title / Summary Description | Priority | Linked Ticket |
+| :--- | :--- | :---: | :--- | :--- |
+| `[BUG-001]` | `[Short descriptive bug title]` | 🔴 Critical | `[PROJ-XXX]` |
+| `[BUG-002]` | `[Short descriptive bug title]` | 🟠 Major | `[PROJ-XXX]` |
+| *— No bugs registered this week —* | | |
+
+---
+
+## 📊 8. TMS Test Health Analytics
+
+> Data sourced via MCP `analytics_tests` and `analytics_stats` for the **current sprint period** only in case if you have access to this analytics.
+(If most of the metrics could not be obtained, this section should be removed from the results.)
+
+### 8.1 Test Health Overview
+| Metric | Value | Notes |
+| :--- | :---: | :--- |
+| 🔀 **Flaky Tests** | [N] | `analytics_tests(kind="flaky", days=N)` |
+| ⏭️ **Never Executed** | [N] | `analytics_tests(kind="never-executed", maturity_days=N)` |
+| 🌿 **Evergreen (Stable)** | [N] | `analytics_tests(kind="evergreen", days=N)` |
+| ⏭️ **Skipped Tests** | [N] | `analytics_tests(kind="skipped", days=N)` |
+| 🐛 **Tests with Defects** | [N] | `analytics_tests(kind="defects", days=N)` |
+| 🐌 **Slow Tests (>N ms)** | [N] | `analytics_tests(kind="slow", threshold_ms=N)` |
+
+### 8.2 Success Rate Trend
+> `analytics_stats(kind="success-rate-by-date", from="DD.MM.YYYY", to="DD.MM.YYYY")`
+
+| Date | Success Rate % |
+| :--- | :---: |
+| [DD.MM.YYYY] | [XX%] |
+| [DD.MM.YYYY] | [XX%] |
+| [DD.MM.YYYY] | [XX%] |
+
+### 8.3 Execution Volume Trend
+> `analytics_stats(kind="testruns-by-date", from="DD.MM.YYYY", to="DD.MM.YYYY")`
+
+| Date | Test Runs Executed |
+| :--- | :---: |
+| [DD.MM.YYYY] | [N] |
+| [DD.MM.YYYY] | [N] |
+
+---
+
+## 📊 9. Sprint Health & Project Readiness Assessment
+
+| Assessment Area | Risk Level | Details & Impact Narrative |
+| :--- | :---: | :--- |
+| 🟢 **Sprint Completion Risk** | `[Green / Yellow / Red]` | Narrative based on execution % and sign-off rate. |
+| 📦 **Spillover / Scope Creep Risk** | `[None / Low / High]` | Highlight any items vulnerable to rolling over into the next cycle. |
+| 🖥️ **Environment & CI/CD Stability** | `[Stable / Degraded]` | Log testing framework bottlenecks, env down times, or pipeline blockers. |
+
+**Sprint Summary Overview:**
+[Write a summary 1-5 sentences overview by current sprint based on the provided info from this report.]

package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md

@@ -0,0 +1,174 @@
+---
+name: qa-test-code-coverage
+description: Map test cases — manual (markdown) and automated e2e (Playwright, Cypress, WebdriverIO, CodeceptJS, Puppeteer, Appium) — to source code files and produce a `coverage.tests.yml` mapping consumed by `@testomatio/reporter --filter "coverage:..."`. Use this skill when the user wants to run only the tests affected by a code change, generate a coverage file (manual, e2e, or both), build a traceability matrix between tests and source code, or set up change-aware regression.
+license: MIT
+metadata:
+  author: Testomat.io
+  version: 1.0.0
+---
+
+# Tests to Code Coverage
+
+Analyze the project's tests — manual markdown cases and automated e2e tests — and its source code. Produce `coverage.tests.yml` — a map from source files (or globs) to test identifiers: suite IDs, test IDs, tags. Testomat.io keeps manual and automated tests in one project, so one coverage file serves both. `@testomatio/reporter` consumes it to run only the tests affected by a diff (Step 6).
+
+## Critical Constraints
+
+- **Map the tests that exist — manual markdown cases, automated e2e tests, or both.** No unit tests. Do not suggest creating new tests.
+- You may write to exactly three places, nothing else:
+  - the coverage file — default `coverage.tests.yml`, or the path the user gave;
+  - the `.testeiya/` cache dir — clone automated test repos, pull manual cases from Testomat.io (Step 1);
+  - `.gitignore` — add a `.testeiya/` line if it is missing.
+- **Never modify a source or test file.**
+- Never pull or clone tests into a tracked folder — only into the gitignored `.testeiya/`.
+- **Everything in the coverage file must come from this project:** file keys from its source tree, identifiers from the Step 2 inventory. Never copy a path or ID from this skill's docs — they are placeholders.
+- **No ad-hoc scripts, no parsers, never Python.** File reads and `grep` only; the single exception is the bundled checker (Step 5); beyond that a one-line `node -e '…'` is the limit.
+- **Be proactive — never stop at a representative sample.** A few entries per domain is a failed run. Keep mapping without asking permission to continue until the completion criterion in Step 4 is met.
+- Stop if you cannot write the output file, or no tests are found and the user declines every option in Step 1.
+
+## Workflow
+
+### Step 1: Discover the project
+
+- Run the `scan-automation-project` skill to inventory the project. Use its output as the source of truth — do not duplicate the scan.
+- Capture from its result:
+  - Manual Tests — the `.test.md` files and their suite/test titles.
+  - Automated Tests — the e2e test files and framework(s). See [E2E Frameworks](./references/E2E_FRAMEWORKS.md) if the scan is ambiguous.
+  - Project Overview — languages, frameworks, complexity (drives the subagent split in Step 4).
+- Map whatever exists: both kinds go into one coverage file; a single kind is fine too.
+- ❓ If the scan finds no tests at all (it checks the `.testeiya/` cache too), ask the user:
+  1. Pull manual cases from Testomat.io — have `sync-test-cases-with-tms` pull into the cache: `npx check-tests pull -d .testeiya/manual-tests`, add `.testeiya/` to `.gitignore` if missing, re-run the scan.
+  2. Clone the automated tests repo: `git clone <url> .testeiya/e2e-tests`, add `.testeiya/` to `.gitignore` if missing, re-run the scan.
+  3. Point to a directory the scan missed, then re-run the scan there.
+  4. Stop.
+- ❓ If two automated frameworks are detected (say Jest and Playwright), ask which one is the e2e framework — coverage filtering runs per runner.
+
+### Step 2: Extract test information
+
+Build one inventory of identifiers across both kinds:
+
+- Suite IDs — `@S` + 8 chars.
+- Test IDs — `@T` + 8 chars.
+- Tags — other `@word` markers.
+- Context — titles, steps, page objects, routes hit. This drives the mapping in Step 4.
+
+Where IDs live:
+
+- Manual cases follow the [Classical Tests Markdown Format](../qa-write-test-cases/references/test-case-format.md) — IDs in `<!-- suite ... id: @S... -->` / `<!-- test ... id: @T... -->` blocks, tags in titles and `tags:` lines.
+- Automated tests carry IDs in `describe` / `it` / `test` / `Feature` / `Scenario` titles — see [E2E Frameworks](./references/E2E_FRAMEWORKS.md).
+
+Read the files, or `grep` the directories that hold the tests:
+
+```bash
+grep -rnE 'id:[[:space:]]*@[ST]' <dir>          # manual suite/test IDs
+grep -rnoE '@[ST][0-9a-f]{8}' <dir>             # IDs in automated test titles
+grep -rhoE '@[A-Za-z0-9_-]+' <dir> | sort -u    # every @token, tags included
+```
+
+Missing IDs mean the tests were never synced with Testomat.io — the reporter cannot select them:
+
+- ❓ Manual files without IDs: ask whether to push them first via `sync-test-cases-with-tms`, or skip those files.
+- Automated tests without IDs in most files: stop and instruct the user to run `npx check-tests@latest <Framework> "<glob>" --update-ids` first (per-framework commands: [E2E Frameworks](./references/E2E_FRAMEWORKS.md)).
+
+### Step 3: Plan the coverage map by domain
+
+**Do not crawl the codebase file by file.** Identify the application's domain areas first; Step 4 then maps area by area.
+
+- Derive the areas from both sides:
+  - the suite tree from Step 2 — the suite hierarchy is the QA team's own map of the product;
+  - the source structure — modules, routes, services, entry points.
+- Pair each area with the suites that test it and the source folders that implement it.
+- Order the areas by criticality, most critical first: payments/billing, auth and access control, data integrity, core business flows — then supporting features, then peripheral UI. Suite size is a signal: what QA tests heavily, they consider critical.
+- Show the ordered plan to the user before mapping.
+- Business code only: skip test code, manual test directories, dependency/build/vendor folders, framework configs, lock files.
+- **Templates and views are mappable source** (`.vue`, `.erb`, `.blade.php`, …) — tests check the rendered UI, so map them like code.
+- ❓ If areas or their priority are unclear, ask the user.
+
+### Step 4: Map source files to tests
+
+**The Step 2 inventory is the work list. The map is complete only when every suite in it is either present in the coverage file or recorded as unmapped with a reason.** Work through the areas in Step 3's order — most critical first, so even an interrupted run covers what matters most. Run two passes — always both:
+
+- Code → tests: for each area, map its source files and subtrees to the identifiers of the tests that check them.
+- Tests → code: walk the inventory; for every suite still absent from the map, find the source it exercises and add it — or record why it cannot be mapped (feature has no code here, external system, needs user input).
+
+Scale the passes to the project — codebase size and suite count from Step 1:
+
+- Small codebase and few suites — run both passes in this session.
+- Large codebase or dozens of suites — spawn subagents in parallel: one per domain area for the code→tests pass, then one per batch of still-unmapped suites for the tests→code pass. Give each subagent:
+  - its slice — one area (its suites and source folders), or a batch of suites;
+  - the full test inventory from Step 2 — subagents must not re-extract it;
+  - the mapping rules below and the [Coverage File Format](./references/COVERAGE_FILE_FORMAT.md);
+  - the instruction to return a YAML fragment for its slice only.
+- Merge the fragments into one map, drop duplicate keys and empty entries. Only the main session writes the file (Step 5).
+
+For each candidate source file, pick the identifier that keeps selecting the right tests as the test suite grows:
+
+- Suite (`@S`) — the default. Most tests in the suite relate to the file.
+- Tag (`@word`) — the relevant tests live across several suites.
+- Test (`@T`) — the exception. Use only when just one or two tests in a suite match the file.
+
+Rules:
+
+- **More than a few test IDs from one suite or category → replace them with the suite or tag.** Tests added there later are picked up automatically; a list of test IDs goes stale.
+- Never list a test ID whose suite is already in the same entry.
+- Keys are source file paths or globs, relative to the repo root; use a glob when a whole subtree maps to the same identifiers.
+- Manual and automated identifiers mix freely in one entry.
+- No empty entries.
+- Annotate each identifier with a `#` comment naming its suite/test/tag title.
+
+YAML grammar: [Coverage File Format](./references/COVERAGE_FILE_FORMAT.md).
+
+### Step 5: Save and validate the coverage file
+
+- Write the YAML to the output path, keeping the `#` comments.
+- Keys are paths to source files in this repo — never `.testeiya/...` paths.
+- Check it with the bundled checker. It ships in this skill's `scripts/` directory (next to this SKILL.md), not in the project — resolve its path from the skill location:
+
+```bash
+npx js-yaml coverage.tests.yml | node <path-to-this-skill>/scripts/check-coverage.mjs
+```
+
+- The checker flags keys missing on disk and empty entries, and prints every identifier referenced — cross-check them against the Step 2 inventory; only you know which are real.
+- Reconcile the other direction too: every suite in the inventory must appear in the file or in the unmapped list. Anything missing from both means the tests→code pass is not finished — go back to Step 4.
+
+### Step 6: Report coverage completeness
+
+Show the user:
+
+- Suites mapped vs. total suites in the inventory; same for tests and tags where meaningful.
+- Every unmapped suite with its reason.
+- The produced YAML (or its path and entry count when large).
+
+### Step 7: Show next steps
+
+Tell the user how to use the file with `@testomatio/reporter`:
+
+```bash
+# Automated tests — the project's runner command must be passed in
+npx @testomatio/reporter run "npx playwright test" \
+  --filter "coverage:file=coverage.tests.yml,diff=main"
+
+# Manual tests — creates a pending run with only the affected cases
+npx @testomatio/reporter run --kind manual \
+  --filter "coverage:file=coverage.tests.yml,diff=main"
+```
+
+- Per-framework runner commands and a GitHub Actions example: [Coverage File Format](./references/COVERAGE_FILE_FORMAT.md).
+- To group manual and automated runs of one regression cycle, set the same `TESTOMATIO_RUNGROUP` env var on both commands.
+- Recommend committing the coverage file so CI and teammates use the same mapping.
+- In CI, diff against the base branch — `diff=origin/main` — and checkout with `fetch-depth: 0`.
+- Pulled or cloned tests stay in the gitignored `.testeiya/` cache for re-runs — don't delete it or move it into a tracked folder.
+
+### Step 8: Suggest follow-ups
+
+- Wire the map into CI — runs created per PR and launched on preview/merge: delegate to `setup-change-aware-pr-testing`.
+- Run the affected tests right now from the terminal: delegate to `run-tests-with-testomatio-reporter`.
+- Coverage gaps — source features no test maps to. On approval, propose new cases (delegate to `qa-write-test-cases`).
+- Dead tests — tests whose features no longer exist in source.
+- Answer questions like "do we have tests for X?" from the inventory.
+- Editing pulled manual cases: edit them in `.testeiya/manual-tests/` and push back with `sync-test-cases-with-tms`.
+
+## References
+
+- [Coverage File Format](./references/COVERAGE_FILE_FORMAT.md) — coverage YAML grammar, reporter contract, GitHub Actions example.
+- [E2E Frameworks](./references/E2E_FRAMEWORKS.md) — framework detection signals, where IDs live, per-framework `check-tests` commands.
+- [Manual test markdown format](../qa-write-test-cases/references/test-case-format.md) — canonical, owned by `qa-write-test-cases`.

package/skills/testomatio/test-management/qa-test-code-coverage/references/COVERAGE_FILE_FORMAT.md

@@ -0,0 +1,138 @@
+# Coverage File Format
+
+`coverage.tests.yml` (any path works) maps source files to manual and automated test identifiers — one grammar for both kinds. It is read by `@testomatio/reporter run --filter "coverage:file=<path>,diff=<branch>"`.
+
+Every path and identifier in this document is a placeholder showing the shape. Real file keys come from the project's source tree; real IDs come from its test inventory. Never copy a value from here into a coverage file.
+
+## Top-level shape
+
+A YAML map. Keys are file paths or globs relative to the repository root. Values are lists of identifiers — Suite IDs, Test IDs, or tags.
+
+```yaml
+<file or glob>:
+  - "@S<8 chars>"   # Suite ID
+  - "@T<8 chars>"   # Test ID
+  - "@<tag>"        # Tag (matches anything tagged with that label)
+```
+
+## Identifier types
+
+| Identifier | Format       | Shape         | Meaning                                  |
+| ---------- | ------------ | ------------- | ---------------------------------------- |
+| Suite ID   | `@S` + 8 hex | `@S1a2b3c4d`  | All tests inside the suite are selected. |
+| Test ID    | `@T` + 8 hex | `@T5e6f7a8b`  | Only this specific test is selected.     |
+| Tag        | `@<word>`    | `@smoke`      | All tests tagged with the label.         |
+
+For automated tests, IDs live in describe / it / test names:
+
+```javascript
+describe('<suite title> @S1a2b3c4d', () => {
+  it('<test title> @T5e6f7a8b', () => { ... });
+});
+```
+
+They are populated by `npx check-tests <Framework> "<glob>" --update-ids` after the tests are imported into Testomat.io.
+
+## File keys
+
+Either an exact path or a glob:
+
+```yaml
+# Exact file
+<path/to/file>:
+  - "@S1a2b3c4d"
+
+# Glob — any file under the directory
+<path/to/dir>/**:
+  - "@<tag>"
+
+# Multiple identifiers per file
+<path/to/file>:
+  - "@S1a2b3c4d"
+  - "@T5e6f7a8b"
+```
+
+## Comments
+
+Use `#` to annotate each identifier with its human-readable title or purpose:
+
+```yaml
+<path/to/file>:
+  - "@S1a2b3c4d"  # Suite: <suite title>
+  - "@T5e6f7a8b"  # Test: <test title>
+```
+
+## Reporter usage
+
+For automated tests the runner command **must** be the first positional argument — `--filter` generates a `--grep` that the runner consumes.
+
+```bash
+# Playwright
+npx @testomatio/reporter run "npx playwright test" \
+  --filter "coverage:file=coverage.tests.yml,diff=main"
+
+# Cypress
+npx @testomatio/reporter run "npx cypress run" \
+  --filter "coverage:file=coverage.tests.yml,diff=main"
+
+# WebdriverIO / Mocha / Jest / CodeceptJS — pass the corresponding runner command
+npx @testomatio/reporter run "npx codeceptjs run" \
+  --filter "coverage:file=coverage.tests.yml,diff=main"
+```
+
+For manual tests pass `--kind manual` instead of a runner — the reporter creates a pending run in Testomat.io with only the affected cases:
+
+```bash
+npx @testomatio/reporter run --kind manual \
+  --filter "coverage:file=coverage.tests.yml,diff=main"
+```
+
+The `diff` value must be a stable branch (`main`, `origin/main`) the reporter can run `git diff` against.
+
+## GitHub Actions example
+
+```yaml
+name: Run Tests Based on Changes
+
+on:
+  pull_request:
+    branches: [main]
+
+jobs:
+  e2e-tests:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      # install browsers and prepare app for testing here
+
+      - name: Run affected E2E tests
+        env:
+          TESTOMATIO: ${{ secrets.TESTOMATIO }}
+        run: |
+          npx @testomatio/reporter run "npx playwright test" \
+            --filter "coverage:file=coverage.tests.yml,diff=origin/main"
+```
+
+## Checking the coverage file
+
+One command — no parser of your own. Run it with the project root as working directory (file keys resolve against it), but the checker itself lives in the skill's `scripts/` directory, not in the project:
+
+```bash
+npx js-yaml coverage.tests.yml | node <path-to-this-skill>/scripts/check-coverage.mjs
+```
+
+`npx js-yaml` parses the YAML (it fails loudly on a malformed file, so a broken one never reaches the script). `check-coverage.mjs` (~25 lines, zero deps) reads that parsed map on stdin, flags any key whose path is missing on disk, flags any key with no identifiers, lists every `@S…` / `@T…` / tag the file references, and exits non-zero on a problem.
+
+The script can't tell which identifiers are real — check the listed ones against the set you extracted earlier in the workflow. Don't re-parse the test set, don't write your own YAML parser, and never use `python`.
+
+## Authoring tips
+
+- Prefer Suite IDs when most of a suite relates to a file.
+- Use tags for cross-cutting concerns that span suites.
+- Use Test IDs only when one or two tests in a suite are relevant — a long test-ID list goes stale as tests are added, a suite or tag keeps selecting new ones.
+- Never list a test ID whose suite is already in the entry — the suite ID already selects it.
+- Avoid empty entries.
+- Commit the coverage file to the repo so CI and teammates use the same mapping.