testeiya 0.3.9 → 0.3.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +353 -27
- package/dist/prompt/index.js +6 -2
- package/dist/prompt/index.js.map +1 -1
- package/dist/prompt/print.js +8 -0
- package/dist/prompt/print.js.map +1 -1
- package/dist/prompt/system-prompt.js +14 -4
- package/dist/prompt/system-prompt.js.map +1 -1
- package/dist/prompt/testomatio.js +3 -3
- package/dist/prompt/tools.js +28 -7
- package/dist/prompt/tools.js.map +1 -1
- package/dist/src/args.js +198 -46
- package/dist/src/args.js.map +1 -1
- package/dist/src/cli.js +64 -20
- package/dist/src/cli.js.map +1 -1
- package/dist/src/doctor.js +171 -0
- package/dist/src/doctor.js.map +1 -0
- package/dist/src/env.js +39 -5
- package/dist/src/env.js.map +1 -1
- package/dist/src/mcp.js +3 -3
- package/dist/src/mcp.js.map +1 -1
- package/dist/src/model.js +15 -10
- package/dist/src/model.js.map +1 -1
- package/dist/src/models.js +38 -0
- package/dist/src/models.js.map +1 -0
- package/dist/src/output.js +216 -0
- package/dist/src/output.js.map +1 -0
- package/dist/src/run.js +141 -27
- package/dist/src/run.js.map +1 -1
- package/dist/src/session.js +46 -25
- package/dist/src/session.js.map +1 -1
- package/dist/src/sessions.js +72 -0
- package/dist/src/sessions.js.map +1 -0
- package/dist/src/skills.js +130 -0
- package/dist/src/skills.js.map +1 -0
- package/package.json +7 -3
- package/prompt/index.ts +11 -2
- package/prompt/print.ts +9 -0
- package/prompt/system-prompt.ts +18 -4
- package/prompt/testomatio.ts +3 -3
- package/prompt/tools.ts +30 -7
- package/skills/codeceptjs/ci-fix-tests/SKILL.md +119 -0
- package/skills/codeceptjs/codeceptjs-auth/SKILL.md +141 -0
- package/skills/codeceptjs/codeceptjs-exploration/SKILL.md +100 -0
- package/skills/codeceptjs/codeceptjs-fundamentals/SKILL.md +196 -0
- package/skills/codeceptjs/codeceptjs-run-analysis/SKILL.md +83 -0
- package/skills/codeceptjs/debugging-codeceptjs-tests/SKILL.md +126 -0
- package/skills/codeceptjs/migrate-codeceptjs-4/SKILL.md +109 -0
- package/skills/codeceptjs/migrate-cypress-to-codeceptjs/SKILL.md +295 -0
- package/skills/codeceptjs/migrate-protractor-to-codeceptjs/SKILL.md +339 -0
- package/skills/codeceptjs/migrate-selenium-java-to-codeceptjs/SKILL.md +459 -0
- package/skills/codeceptjs/migrate-testcafe-to-codeceptjs/SKILL.md +338 -0
- package/skills/codeceptjs/refactoring-codeceptjs-tests/SKILL.md +78 -0
- package/skills/codeceptjs/writing-codeceptjs-tests/SKILL.md +102 -0
- package/skills/playwright/playwright-best-practices/SKILL.md +303 -0
- package/skills/playwright/playwright-best-practices/advanced/authentication-flows.md +360 -0
- package/skills/playwright/playwright-best-practices/advanced/authentication.md +871 -0
- package/skills/playwright/playwright-best-practices/advanced/clock-mocking.md +364 -0
- package/skills/playwright/playwright-best-practices/advanced/mobile-testing.md +409 -0
- package/skills/playwright/playwright-best-practices/advanced/multi-context.md +288 -0
- package/skills/playwright/playwright-best-practices/advanced/multi-user.md +393 -0
- package/skills/playwright/playwright-best-practices/advanced/network-advanced.md +452 -0
- package/skills/playwright/playwright-best-practices/advanced/third-party.md +464 -0
- package/skills/playwright/playwright-best-practices/architecture/pom-vs-fixtures.md +363 -0
- package/skills/playwright/playwright-best-practices/architecture/test-architecture.md +369 -0
- package/skills/playwright/playwright-best-practices/architecture/when-to-mock.md +383 -0
- package/skills/playwright/playwright-best-practices/browser-apis/browser-apis.md +391 -0
- package/skills/playwright/playwright-best-practices/browser-apis/iframes.md +403 -0
- package/skills/playwright/playwright-best-practices/browser-apis/service-workers.md +504 -0
- package/skills/playwright/playwright-best-practices/browser-apis/websockets.md +403 -0
- package/skills/playwright/playwright-best-practices/core/annotations.md +424 -0
- package/skills/playwright/playwright-best-practices/core/assertions-waiting.md +361 -0
- package/skills/playwright/playwright-best-practices/core/configuration.md +452 -0
- package/skills/playwright/playwright-best-practices/core/fixtures-hooks.md +417 -0
- package/skills/playwright/playwright-best-practices/core/global-setup.md +434 -0
- package/skills/playwright/playwright-best-practices/core/locators.md +242 -0
- package/skills/playwright/playwright-best-practices/core/page-object-model.md +315 -0
- package/skills/playwright/playwright-best-practices/core/projects-dependencies.md +453 -0
- package/skills/playwright/playwright-best-practices/core/test-data.md +492 -0
- package/skills/playwright/playwright-best-practices/core/test-suite-structure.md +361 -0
- package/skills/playwright/playwright-best-practices/core/test-tags.md +298 -0
- package/skills/playwright/playwright-best-practices/debugging/console-errors.md +420 -0
- package/skills/playwright/playwright-best-practices/debugging/debugging.md +504 -0
- package/skills/playwright/playwright-best-practices/debugging/error-testing.md +360 -0
- package/skills/playwright/playwright-best-practices/debugging/flaky-tests.md +496 -0
- package/skills/playwright/playwright-best-practices/frameworks/angular.md +530 -0
- package/skills/playwright/playwright-best-practices/frameworks/nextjs.md +469 -0
- package/skills/playwright/playwright-best-practices/frameworks/react.md +531 -0
- package/skills/playwright/playwright-best-practices/frameworks/vue.md +574 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/ci-cd.md +468 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/docker.md +283 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/github-actions.md +546 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/gitlab.md +397 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/other-providers.md +521 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/parallel-sharding.md +371 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/performance.md +453 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/reporting.md +424 -0
- package/skills/playwright/playwright-best-practices/infrastructure-ci-cd/test-coverage.md +497 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/accessibility.md +359 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/api-testing.md +719 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/browser-extensions.md +506 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/canvas-webgl.md +493 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/component-testing.md +500 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/drag-drop.md +576 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/electron.md +509 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/file-operations.md +377 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/file-upload-download.md +562 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/forms-validation.md +561 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/graphql-testing.md +331 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/i18n.md +508 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/performance-testing.md +476 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/security-testing.md +430 -0
- package/skills/playwright/playwright-best-practices/testing-patterns/visual-regression.md +634 -0
- package/skills/playwright/playwright-cli/SKILL.md +420 -0
- package/skills/playwright/playwright-cli/references/element-attributes.md +23 -0
- package/skills/playwright/playwright-cli/references/playwright-tests.md +39 -0
- package/skills/playwright/playwright-cli/references/request-mocking.md +87 -0
- package/skills/playwright/playwright-cli/references/running-code.md +241 -0
- package/skills/playwright/playwright-cli/references/session-management.md +225 -0
- package/skills/playwright/playwright-cli/references/storage-state.md +275 -0
- package/skills/playwright/playwright-cli/references/test-generation.md +433 -0
- package/skills/playwright/playwright-cli/references/tracing.md +139 -0
- package/skills/playwright/playwright-cli/references/video-recording.md +143 -0
- package/skills/skills.lock.json +41 -37
- package/skills/skills.yaml +1 -0
- package/skills/testomatio/explorbot/explorbot-fundamentals/SKILL.md +76 -0
- package/skills/testomatio/explorbot/explorbot-fundamentals/references/no-install.md +36 -0
- package/skills/testomatio/explorbot/explorbot-plan/SKILL.md +108 -0
- package/skills/testomatio/explorbot/explorbot-setup/SKILL.md +86 -0
- package/skills/testomatio/explorbot/prima/SKILL.md +87 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/SKILL.md +111 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/answer-shapes.md +116 -0
- package/skills/testomatio/qa-process/qa-explain-behavior/references/examples.md +157 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/SKILL.md +91 -0
- package/skills/testomatio/qa-process/qa-lead-strategy-advisor/references/output-format.md +71 -0
- package/skills/testomatio/qa-process/testing-workflow/SKILL.md +227 -0
- package/skills/testomatio/test-automation/automate-manual-test-cases/CLAUDE.md +56 -0
- package/skills/testomatio/test-automation/automate-manual-test-cases/SKILL.md +242 -0
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/CODECEPTJS_BEST_PRACTICES.md +182 -0
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/FINAL_SUMMARY_TEMPLATE.md +22 -0
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/PLAYWRIGHT_BEST_PRACTICES.md +90 -0
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/POM_BEST_PRACTICES.md +53 -0
- package/skills/testomatio/test-automation/automate-manual-test-cases/references/TEST_DATA_MANAGEMENT.md +52 -0
- package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/SKILL.md +107 -0
- package/skills/testomatio/test-automation/debug-fix-failed-flaky-autotests/references/DEBUGGING_QUICK_REFERENCE.md +113 -0
- package/skills/testomatio/test-automation/qa-automation-test-consolidation/SKILL.md +29 -0
- package/skills/testomatio/test-automation/qa-data-seeder/SKILL.md +83 -0
- package/skills/testomatio/test-automation/run-tests-with-testomatio-reporter/SKILL.md +153 -0
- package/skills/testomatio/test-automation/setup-change-aware-pr-testing/SKILL.md +190 -0
- package/skills/testomatio/test-automation/setup-ci-automation/SKILL.md +76 -0
- package/skills/testomatio/test-automation/testomat-allure-adapter/SKILL.md +289 -0
- package/skills/testomatio/test-management/detect-duplicate-test-cases/SKILL.md +86 -0
- package/skills/testomatio/test-management/detect-duplicate-test-cases/references/DUPLICATE_INSTRUCTIONS.md +54 -0
- package/skills/testomatio/test-management/improve-test-cases/SKILL.md +114 -0
- package/skills/testomatio/test-management/improve-test-cases/references/TESTOMAT_MARKDOWN_EXAMPLE.md +66 -0
- package/skills/testomatio/test-management/pull-request-diff-analyzer/SKILL.md +141 -0
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/SKILL.md +354 -0
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_ARTIFACTS.md +82 -0
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_HTML_REPORT.md +37 -0
- package/skills/testomatio/test-management/qa-e2e-tests-reporting/references/TESTOMATIO_REPORTERS_CONFIG.md +118 -0
- package/skills/testomatio/test-management/qa-pr-requirements-analyzer/SKILL.md +149 -0
- package/skills/testomatio/test-management/qa-pr-requirements-analyzer/references/summary-example.md +54 -0
- package/skills/testomatio/test-management/qa-requirement-reviewer/SKILL.md +126 -0
- package/skills/testomatio/test-management/qa-requirement-reviewer/references/requirements_reviewer_examples.md +128 -0
- package/skills/testomatio/test-management/qa-split-testing-levels-pyramid/SKILL.md +55 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/SKILL.md +220 -0
- package/skills/testomatio/test-management/qa-sprint-report-by-testomatio/references/qa-sprint-report.md +126 -0
- package/skills/testomatio/test-management/qa-test-code-coverage/SKILL.md +174 -0
- package/skills/testomatio/test-management/qa-test-code-coverage/references/COVERAGE_FILE_FORMAT.md +138 -0
- package/skills/testomatio/test-management/qa-test-code-coverage/references/E2E_FRAMEWORKS.md +65 -0
- package/skills/testomatio/test-management/qa-test-code-coverage/scripts/check-coverage.mjs +40 -0
- package/skills/testomatio/test-management/qa-thinking/SKILL.md +36 -0
- package/skills/testomatio/test-management/qa-write-test-cases/SKILL.md +267 -0
- package/skills/testomatio/test-management/qa-write-test-cases/references/test-case-format.md +387 -0
- package/skills/testomatio/test-management/qa-write-test-cases/references/testomat-tms-guide.md +72 -0
- package/skills/testomatio/test-management/qa-write-test-cases/references/writing-rule.md +159 -0
- package/skills/testomatio/test-management/scan-automation-project/SKILL.md +152 -0
- package/skills/testomatio/test-management/sync-test-cases-with-tms/SKILL.md +107 -0
- package/skills/testomatio/test-management/sync-test-cases-with-tms/references/TESTOMATIO_CLI.md +140 -0
- package/skills/testomatio/test-management/testomatio-mcp/SKILL.md +195 -0
- package/skills/testomatio/test-management/testomatio-mcp/references/MCP_SETUP.md +177 -0
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: explorbot-setup
|
|
3
|
+
description: Use to install Explorbot into a project — first-time setup, an `explorbot.config.js` in the repo, connecting an AI provider, teaching it how to log in, or when a command fails because the project has no config. Ends once `explorbot navigate` reaches the app.
|
|
4
|
+
license: MIT
|
|
5
|
+
metadata:
|
|
6
|
+
author: Testomat.io
|
|
7
|
+
version: 1.2.0
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Explorbot Setup
|
|
11
|
+
|
|
12
|
+
Install Explorbot into the user's project, connect a provider, teach it how to reach the app, and prove it with one navigation.
|
|
13
|
+
|
|
14
|
+
The config, the knowledge files, and the generated tests all land in the repo, so they are reviewed and versioned like the rest of the code. That is the point of installing locally.
|
|
15
|
+
|
|
16
|
+
**To run Explorbot without installing it into the project, this is the wrong skill** — [[explorbot-fundamentals]] covers that: `npx`, a provider, no project files.
|
|
17
|
+
|
|
18
|
+
**Scope ends when `npx explorbot navigate <path>` exits `0`** — exploring and testing belong to [[explorbot-fundamentals]].
|
|
19
|
+
|
|
20
|
+
## 1. Requirements
|
|
21
|
+
|
|
22
|
+
- Node ≥ 24 or Bun. If neither exists, stop — do not install a runtime.
|
|
23
|
+
- The generated config uses `import`, so `package.json` must be ESM. `npm init -y` produces CommonJS; if `"type"` is missing or `"commonjs"` in an existing file, ask before changing it, and write `explorbot.config.mjs` (passed as `--config`) if the user declines.
|
|
24
|
+
- Explorbot is for CRUD-heavy apps. Warn before installing if the target is a landing page, blog, CMS, or static site.
|
|
25
|
+
|
|
26
|
+
## 2. Install
|
|
27
|
+
|
|
28
|
+
```bash
|
|
29
|
+
npm pkg set type=module
|
|
30
|
+
npm i explorbot --save
|
|
31
|
+
npx playwright install chromium # ~/.cache/ms-playwright, once per machine
|
|
32
|
+
npx explorbot init # writes explorbot.config.js, .env, output/
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
## 3. Provider
|
|
36
|
+
|
|
37
|
+
**Providers, model ids, and key variable names live in Explorbot, never in this skill.** The generated config ships one provider; `basics/providers.md` in the docs names the package and model ids for the others, and `models.json` holds the current recommendations.
|
|
38
|
+
|
|
39
|
+
- Ask which provider the user has an account with, and swap the import and the three model lines if it is not the one in the template.
|
|
40
|
+
- `npm i` the matching provider package.
|
|
41
|
+
|
|
42
|
+
The key: ask the user to paste it into `.env` in their editor and say "continue", then verify with `grep -q "^<KEY>=." .env`. Confirm `.env` is gitignored. Never echo a key back.
|
|
43
|
+
|
|
44
|
+
`~/.explorbot/.env` is also read on every run, filling in variables the project `.env` does not set.
|
|
45
|
+
|
|
46
|
+
## 4. App URL
|
|
47
|
+
|
|
48
|
+
Ask for it, and write it to `web.url` in the config — host only, no path. Explorbot clicks, fills, and submits on its own, so staging beats production.
|
|
49
|
+
|
|
50
|
+
Do not guess it. Look for evidence in the project — `package.json` scripts, `.env` files, framework ports — and suggest what you find with its source. Otherwise ask in plain text. Never propose a URL from this skill or from memory.
|
|
51
|
+
|
|
52
|
+
## 5. Verify
|
|
53
|
+
|
|
54
|
+
Walk the rungs in order. Stop at the first failure, fix it, resume.
|
|
55
|
+
|
|
56
|
+
1. `curl -sS -o /dev/null -w "%{http_code}\n" <url>` — `000` is DNS/VPN/wrong host, `5xx` is the app itself, `200` skips to rung 3.
|
|
57
|
+
2. `curl -sIL -o /dev/null -w "%{url_effective}\n" <url>` after a `3xx`/`401`/`403`. A login-looking final URL means credentials are needed: do rung 4 first.
|
|
58
|
+
3. `npx explorbot navigate <path> --session` — exit `0` ends setup. `--session` saves cookies to `output/session.json`. On exit `1`, read `output/explorbot.log`: an auth wall sends you to rung 4, anything else is a wrong path or a broken page.
|
|
59
|
+
4. Teach auth, knowledge first, values second:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
npx explorbot knows /login # reuse existing variable names if auth knowledge exists
|
|
63
|
+
npx explorbot learn "/login" 'Sign in with ${env.APP_USER} / ${env.APP_PASSWORD}'
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
Name only the credentials — no selectors or DOM hints; the Navigator finds the fields. Then have the user add the two variables to `.env`, verify with `grep`, and repeat rung 3. Ask for a test account on a non-production environment.
|
|
67
|
+
|
|
68
|
+
Still failing on auth: add one sentence to the knowledge with `learn` (real form URL, SSO, second factor), never a rewrite.
|
|
69
|
+
|
|
70
|
+
`npx explorbot context <path>` prints headings, matched knowledge, and interactive elements with no AI calls — use it to debug a rung for free.
|
|
71
|
+
|
|
72
|
+
## 6. Handoff
|
|
73
|
+
|
|
74
|
+
Report what the project now has: `explorbot.config.js`, `.env` (gitignored), `knowledge/`, `output/` with a saved session. Then hand off to [[explorbot-fundamentals]] with a first command:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
npx explorbot explore /<crud-page> --max-tests 10
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
## Anti-patterns
|
|
81
|
+
|
|
82
|
+
- Installing into the project when the user only wanted to try Explorbot — that is [[explorbot-fundamentals]].
|
|
83
|
+
- Naming providers, model ids, or key variables without reading them from the docs or the generated config.
|
|
84
|
+
- Putting secrets in knowledge files or config — always `${env.NAME}`, values in `.env`.
|
|
85
|
+
- Driving `explorbot start` — it is an interactive TUI; ask the user to run it.
|
|
86
|
+
- Asking upfront for target pages, features, or test counts.
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: prima
|
|
3
|
+
description: Use for any browser work — driving a web app, checking a behaviour, filling a form, reading a page — via npx prima-cli, in preference to npx playwright-cli, which prima runs on top of and falls back to. Covers npx prima-cli check, do, pw, verify, ask, research, go, status, report, browser, config, and session setup. Trigger on "prima", "prima-cli", "npx prima-cli", on a browser task described as behaviour ("confirm the editor saves"), and whenever npx playwright-cli would otherwise be driven step by step.
|
|
4
|
+
license: MIT
|
|
5
|
+
metadata:
|
|
6
|
+
author: Testomat.io
|
|
7
|
+
version: 0.3.0
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# Prima
|
|
11
|
+
|
|
12
|
+
Prima is an AI layer on top of playwright-cli. It drives the browser playwright-cli already has open, taking behaviour described in words instead of locators, and returns a plain-text envelope. Always invoke both through npx: `npx prima-cli ...` and `npx playwright-cli ...`.
|
|
13
|
+
|
|
14
|
+
**Prefer prima to playwright-cli for anything on a web page.** Snapshots stay inside prima instead of landing in your context, and it reuses the research maps and recorded experience it already has for a page — so a scenario costs a fraction of the tokens of the same scenario driven by hand. Use `npx playwright-cli` directly when prima has no command for the job, or when no AI model is configured (`pw` still works then).
|
|
15
|
+
|
|
16
|
+
## Models
|
|
17
|
+
|
|
18
|
+
Prima needs an AI model, taken from the environment — there is no init step:
|
|
19
|
+
|
|
20
|
+
```bash
|
|
21
|
+
export PRIMA_CLI_AI_MODEL=openrouter/openai/gpt-oss-120b # provider comes from the name
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
Screenshot analysis needs its own model, never guessed from the main one: set `PRIMA_CLI_VISION_MODEL`, or pass `--vision-model` for one run. Without it prima still runs, but `check` settles outcomes from the run log rather than the page as seen. `npx prima-cli config` prints what this directory resolves to; `--json` for machine form. Every `PRIMA_CLI_*` variable mirrors the `EXPLORBOT_*` one and wins over it, so an explorbot setup is not disturbed.
|
|
25
|
+
|
|
26
|
+
## Session
|
|
27
|
+
|
|
28
|
+
Attach to a playwright-cli session:
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
npx playwright-cli open http://localhost:3000 # start
|
|
32
|
+
npx prima-cli <command> ... # drive
|
|
33
|
+
npx playwright-cli close # end
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Or let prima own the browser itself:
|
|
37
|
+
|
|
38
|
+
```bash
|
|
39
|
+
npx prima-cli browser start --url http://localhost:3000
|
|
40
|
+
npx prima-cli <command> ...
|
|
41
|
+
npx prima-cli browser stop # --all stops every instance
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
- Attached to playwright-cli, prima never launches or closes the browser — it attaches and disconnects. With a prima-owned browser, `browser start` holds it open until `browser stop`; `browser status` and `browser list` report what is running.
|
|
45
|
+
- `--pw-session <title>` picks the session when several playwright-cli sessions are open; `--instance <name>` separates prima-owned browsers for parallel work.
|
|
46
|
+
- `--url <url>` opens that page first when the session has none; `--endpoint <ep>` attaches to a browser server directly, skipping discovery.
|
|
47
|
+
- Requires Node.js 24+. Playwright browsers come from `npx playwright install chromium`.
|
|
48
|
+
- Every command is logged as it runs; `npx prima-cli report` turns the session into an html and markdown report, browser open or not.
|
|
49
|
+
- `npx explorbot prima <command>` runs the same tool if explorbot is already installed.
|
|
50
|
+
|
|
51
|
+
## Tiers
|
|
52
|
+
|
|
53
|
+
Start at `check`. Come down a rung only when the one above cannot hold the work.
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
npx prima-cli check "a workflow can be created and appears in the list" --expected "the new workflow is listed"
|
|
57
|
+
npx prima-cli do "open the account menu" "choose settings" "switch the theme to dark" "check it took effect"
|
|
58
|
+
npx prima-cli pw "({ page }) => page.click('[data-test=submit]')"
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
- **Pass `do` the whole remaining sequence, never one step per call** — that is the entire cost advantage.
|
|
62
|
+
- `pw` takes executable code only. Never give it a description; never give `check`/`do` a locator.
|
|
63
|
+
- `check` and `do` legitimately run for minutes. Do not kill and retry.
|
|
64
|
+
- `check` runs on the page already open and never reloads it, so an open dialog, a selected tab or a filled form survives it.
|
|
65
|
+
- `--expected` is repeatable for several outcomes; without it the scenario text is the single expected outcome.
|
|
66
|
+
- Below `pw` sits playwright-cli itself.
|
|
67
|
+
|
|
68
|
+
Also: `verify` (alias `assert`), `ask` (`--no-vision` answers from structure only), `research` (`--data` includes extraction, `--deep` expands hidden elements, `--fresh` re-maps past the cache), `go`, `status <hash>`, `report`, `config`, `browser`. Run `npx prima-cli <command> --help` for each.
|
|
69
|
+
|
|
70
|
+
## Reading the envelope
|
|
71
|
+
|
|
72
|
+
- **Trust the verdict in `### Result`.** Re-verifying a PASSED outcome with another command is the waste this tool exists to remove.
|
|
73
|
+
- `ok: true` means the action you asked for landed; nothing is substituted or retried along a different route.
|
|
74
|
+
- `### Steps` marks each line `ok`, `FAIL` or `??`. `??` is an instruction that ran but the run ended without confirming — the actions that ran are listed above it, judge from those. Only `FAIL` and an instruction the page could not carry out fail the command.
|
|
75
|
+
- `not verified` means the run never checked that outcome — not that it is false, and not a failure.
|
|
76
|
+
- **`CONTRADICTION` is a finding, not a verdict to argue with.** The run and the picture disagree: something the assertions matched is not visible on screen, or the reverse. Both sides are quoted under the outcome, and `### Artifacts` names the html, aria and screenshot on disk — read those and judge the page yourself instead of taking the verdict on trust. Treat it as a bug in the app and look at it before anything else.
|
|
77
|
+
- A `### Warning` saying the outcomes came from the run log alone means no screenshot backed them: set `PRIMA_CLI_VISION_MODEL` (or pass `--vision-model`), and until then do not trust a visual claim from `check`.
|
|
78
|
+
- A run that could not complete says so, rather than reporting it as a failure of the app.
|
|
79
|
+
- `npx prima-cli config` shows which model answers for each role.
|
|
80
|
+
|
|
81
|
+
## Visual questions
|
|
82
|
+
|
|
83
|
+
Layout, position, colour, overlap, whether something is cut off — an accessibility tree does not carry any of it, and neither does an assertion that passed.
|
|
84
|
+
|
|
85
|
+
- `check` settles its outcomes against a screenshot of the final page. What a user can see is the proof; the run log only says what was done.
|
|
86
|
+
- `ask "<question>"` — reads a screenshot, answers in prose. Use for anything open-ended about appearance.
|
|
87
|
+
- `verify` proves claims with assertions; each comes back PASSED or FAILED with its playwright form, no overall verdict — read the lines and decide. When no assertion can express a claim it reports "none ran" instead of judging it failed.
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qa-explain-behavior
|
|
3
|
+
description: Answers QA questions about product behavior — what features exist, how user flows work, what business rules apply, what edge cases the system handles, and what is or is not implemented. Use whenever the user asks a question about WHAT the system does (features, flows, rules, conditions, who-can-do-what, when-does-X-happen, is-Y-supported). Trigger even when the user does not say "QA" — phrases like "what happens when…", "can a user…", "is it possible to…", "does the system…", "is X implemented", "what should I test for…", "what are the edge cases of…", or "walk me through the flow for…" all qualify. Do NOT trigger for questions that are clearly about implementation (architecture, database schema, code structure, performance, types, dependencies).
|
|
4
|
+
license: MIT
|
|
5
|
+
metadata:
|
|
6
|
+
author: Testomat.io
|
|
7
|
+
version: 1.0.0
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
# QA Explain Behavior
|
|
11
|
+
|
|
12
|
+
Answer a QA engineer who is testing the product. They care about **product behavior**, not implementation. Read the codebase like a manual and translate it into plain-language descriptions of what the system does.
|
|
13
|
+
|
|
14
|
+
| What | File |
|
|
15
|
+
|------|------|
|
|
16
|
+
| Output shape per question type — feature, flow, yes/no, permissions, edge cases | [references/answer-shapes.md](references/answer-shapes.md) |
|
|
17
|
+
| Worked answers, plus bad-vs-good contrasts | [references/examples.md](references/examples.md) |
|
|
18
|
+
|
|
19
|
+
## The mindset shift
|
|
20
|
+
|
|
21
|
+
The QA engineer is not going to read the code. They are going to click buttons, fill forms, and try to break things. The answer should help them know what to click, what to expect, and where the product might fail. Describe the product like someone who has used it end-to-end.
|
|
22
|
+
|
|
23
|
+
A QA engineer's mental model is built from:
|
|
24
|
+
|
|
25
|
+
- **Actors** — who is doing this: guest, member, admin, owner, integration.
|
|
26
|
+
- **Actions** — what they do: click, submit, upload, invite.
|
|
27
|
+
- **Outcomes** — what they see or what changes: page, message, email, status.
|
|
28
|
+
- **Rules** — when it's allowed, when it's blocked, what limits apply.
|
|
29
|
+
- **Edge cases** — empty state, very large input, duplicate, expired, offline.
|
|
30
|
+
|
|
31
|
+
Frame every answer in those terms.
|
|
32
|
+
|
|
33
|
+
## What to investigate
|
|
34
|
+
|
|
35
|
+
Look at the parts of the codebase that **describe behavior**, not the parts that describe data:
|
|
36
|
+
|
|
37
|
+
- **Service and controller layers** — what user actions exist, and the multi-step flows behind them.
|
|
38
|
+
- **Business logic** — the rules that decide what is allowed, what changes, and what happens next.
|
|
39
|
+
- **Authorization and permissions system** — who can do what, and whether the block is in the UI, on the server, or both.
|
|
40
|
+
- **Feature flags** — what is switched on, off, or limited to certain accounts.
|
|
41
|
+
- **Configuration** — limits, defaults, and behavior that differs per environment or plan.
|
|
42
|
+
- **Models** — only *validations*, *state machines*, *callbacks*, and *scopes with business meaning*. Not columns, associations, or types.
|
|
43
|
+
- **Views, serializers, mailers** — what the user sees and receives.
|
|
44
|
+
- **Background jobs** — what happens after the user-visible action: notifications, syncs, cleanups.
|
|
45
|
+
- **Tests** — they describe expected behavior in plain English; very useful.
|
|
46
|
+
|
|
47
|
+
## What to ignore
|
|
48
|
+
|
|
49
|
+
Unless the user explicitly asks, never mention:
|
|
50
|
+
|
|
51
|
+
- Database tables, columns, indexes, schema.
|
|
52
|
+
- Class names, file paths, line numbers, module structure.
|
|
53
|
+
- Framework terminology and internals — ORM concepts, callbacks, middleware, queues, dependencies.
|
|
54
|
+
- Variable names, method names, parameters, types.
|
|
55
|
+
- SQL, code snippets, regex.
|
|
56
|
+
- Performance, scaling, architecture decisions.
|
|
57
|
+
- Migration history, refactors.
|
|
58
|
+
|
|
59
|
+
If the user wants this, they will ask. The default is product-level only.
|
|
60
|
+
|
|
61
|
+
## How to answer
|
|
62
|
+
|
|
63
|
+
**Be compact.** A QA engineer scans, they don't read paragraphs. Short bullets, short sentences, plain words.
|
|
64
|
+
|
|
65
|
+
**Lead with the answer.** No preamble, no restating the question, no "Let me explain…".
|
|
66
|
+
|
|
67
|
+
**Use product vocabulary, not code vocabulary:**
|
|
68
|
+
|
|
69
|
+
- "the user", "the admin", "the workspace owner" — not `current_user`.
|
|
70
|
+
- "the settings page" — not a controller action name.
|
|
71
|
+
- "an email is sent" — not a mailer class.
|
|
72
|
+
- "it's blocked" — not "validation fails".
|
|
73
|
+
- "in the background" — not the name of the queue.
|
|
74
|
+
|
|
75
|
+
**Structure by what a tester needs.** When describing a feature, default to this shape:
|
|
76
|
+
|
|
77
|
+
```
|
|
78
|
+
**What it does:** one line
|
|
79
|
+
**Who can do it:** roles / permissions
|
|
80
|
+
**How to trigger it:** user action(s)
|
|
81
|
+
**What happens:** outcomes (UI, emails, notifications, status changes)
|
|
82
|
+
**Rules / limits:** validations, conditions
|
|
83
|
+
**Edge cases:** empty, duplicate, large, expired, blocked, etc.
|
|
84
|
+
**Not implemented:** anything the QA might expect that does NOT exist
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
For flow questions, describe the flow as numbered steps from the user's point of view. For "is X supported?" questions, answer **yes** or **no** first, then a one-line reason, then any nuance. Permission questions, edge-case sweeps, failure behavior, comparisons, and how to phrase uncertainty each have their own shape in [references/answer-shapes.md](references/answer-shapes.md).
|
|
88
|
+
|
|
89
|
+
**Surface what is missing.** The most valuable thing a QA engineer can hear is "this is not implemented", "there is no UI for this — only the API supports it", or "this only works for paying accounts". Always look for gaps and call them out: a flag, a TODO, a commented-out path, an action reachable only via API.
|
|
90
|
+
|
|
91
|
+
**Be honest about uncertainty.** If the codebase is ambiguous, say "I see X but I'm not sure whether Y" rather than guessing. A QA testing on a wrong assumption is worse than a QA who knows to verify.
|
|
92
|
+
|
|
93
|
+
## Do not write code
|
|
94
|
+
|
|
95
|
+
Never produce code blocks, schema, configuration snippets, or pseudocode unless the user explicitly asks ("show me the code", "give me the SQL"). The default response is prose and bullets only. When tempted to paste code "to be helpful", describe the behavior instead.
|
|
96
|
+
|
|
97
|
+
When the product **is** a CLI or an API, the commands, flags, and endpoints a user types are product surface, not implementation — name them exactly, the same way you would name a button.
|
|
98
|
+
|
|
99
|
+
[references/examples.md](references/examples.md) has worked answers — a full feature description, a flow walkthrough, an edge-case sweep — and two bad-vs-good contrasts showing what code leaking into an answer looks like.
|
|
100
|
+
|
|
101
|
+
## Next actions
|
|
102
|
+
|
|
103
|
+
Offer after the answer:
|
|
104
|
+
|
|
105
|
+
- Turn the behavior into risk scenarios → `qa-thinking` skill.
|
|
106
|
+
- Turn it into test cases or a checklist → `qa-write-test-cases` skill.
|
|
107
|
+
- Map which tests already cover it → `qa-test-code-coverage` skill.
|
|
108
|
+
|
|
109
|
+
## Final reminder
|
|
110
|
+
|
|
111
|
+
Compact. Plain language. Behavior, not code. Highlight gaps. No code unless asked.
|
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
# Answer shapes
|
|
2
|
+
|
|
3
|
+
Pick the shape that matches the question. Skip any section that doesn't apply —
|
|
4
|
+
never pad with "N/A" or "None". Bullets over paragraphs, short lines, plain
|
|
5
|
+
words.
|
|
6
|
+
|
|
7
|
+
## Feature description — "what does X do", "explain X"
|
|
8
|
+
|
|
9
|
+
The default shape. Most questions land here.
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
**What it does:** one line
|
|
13
|
+
**Who can do it:** roles / permissions
|
|
14
|
+
**How to trigger it:** the clicks, in order
|
|
15
|
+
**What happens:** outcomes — screen, message, email, status change, background work
|
|
16
|
+
**Rules / limits:** validations, conditions, plan gating
|
|
17
|
+
**Edge cases:** empty, duplicate, too large, expired, blocked, concurrent
|
|
18
|
+
**Not implemented:** anything a tester would reasonably expect that doesn't exist
|
|
19
|
+
```
|
|
20
|
+
|
|
21
|
+
## Flow walkthrough — "walk me through…", "how does someone…"
|
|
22
|
+
|
|
23
|
+
Numbered steps from the user's point of view, one screen per step. Say where
|
|
24
|
+
the flow can branch or die.
|
|
25
|
+
|
|
26
|
+
```
|
|
27
|
+
1. User clicks **X** on the … page.
|
|
28
|
+
2. …
|
|
29
|
+
3. …
|
|
30
|
+
|
|
31
|
+
**Branches:**
|
|
32
|
+
- If <condition>, they land on … instead.
|
|
33
|
+
- If <condition>, it stops with "<the actual message>".
|
|
34
|
+
|
|
35
|
+
**After it finishes:** <emails, notifications, anything delayed>
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## Yes / no — "is X supported", "can I…", "is there a way to…"
|
|
39
|
+
|
|
40
|
+
Answer in the first word. Then one line of why. Then nuance, if any.
|
|
41
|
+
|
|
42
|
+
```
|
|
43
|
+
Yes — partially.
|
|
44
|
+
- <what works, and where>
|
|
45
|
+
- <what doesn't>
|
|
46
|
+
- <what exists in the API but has no UI>
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
If it's a flat no, one or two lines is the whole answer. Don't inflate it.
|
|
50
|
+
|
|
51
|
+
## Permissions — "can a *role* do X", "who can…"
|
|
52
|
+
|
|
53
|
+
Say whether the block is in the UI, on the server, or both — a tester needs to
|
|
54
|
+
know if they can get past it by calling the API directly.
|
|
55
|
+
|
|
56
|
+
```
|
|
57
|
+
No. Guests can view … but cannot …
|
|
58
|
+
The button is hidden for guests, and the action is blocked server-side too.
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
For several roles at once, use a small table:
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
| Role | Create | Edit | Delete |
|
|
65
|
+
|---|---|---|---|
|
|
66
|
+
| Owner | ✅ | ✅ | ✅ |
|
|
67
|
+
| Member | ✅ | ✅ | ❌ |
|
|
68
|
+
| Guest | ❌ | ❌ | ❌ |
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
## Edge cases — "what could go wrong", "what should I test"
|
|
72
|
+
|
|
73
|
+
Group by the kind of input or state, so the list is walkable as a test session.
|
|
74
|
+
Each line is a condition plus what actually happens.
|
|
75
|
+
|
|
76
|
+
```
|
|
77
|
+
**Empty / missing:** …
|
|
78
|
+
**Duplicate:** …
|
|
79
|
+
**Too large / too many:** …
|
|
80
|
+
**Wrong format:** …
|
|
81
|
+
**Expired / stale:** …
|
|
82
|
+
**Permissions:** …
|
|
83
|
+
**Concurrent:** two people doing this at once …
|
|
84
|
+
**External failure:** the integration is down / the upload fails …
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
## Failure behaviour — "what happens if it fails"
|
|
88
|
+
|
|
89
|
+
Testers need the observable result, not the exception.
|
|
90
|
+
|
|
91
|
+
```
|
|
92
|
+
**What the user sees:** the actual message or screen
|
|
93
|
+
**What state it's left in:** saved / not saved / half-saved
|
|
94
|
+
**Whether it retries:** automatically, manually, or not at all
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## Comparison — "what's the difference between X and Y"
|
|
98
|
+
|
|
99
|
+
Two columns, same rows. Only rows where they actually differ.
|
|
100
|
+
|
|
101
|
+
## Phrasing uncertainty
|
|
102
|
+
|
|
103
|
+
Never guess silently. Use the shape "I can see X, but I couldn't confirm Y":
|
|
104
|
+
|
|
105
|
+
- "The API accepts this, but I couldn't find a UI that reaches it."
|
|
106
|
+
- "There's a limit of 100 here, but I'm not sure whether it's enforced on
|
|
107
|
+
import as well — worth verifying."
|
|
108
|
+
- "This looks gated by a feature flag. Whether it's on for your account, I
|
|
109
|
+
can't tell from the code."
|
|
110
|
+
|
|
111
|
+
## Length
|
|
112
|
+
|
|
113
|
+
Match the question. A yes/no question gets two lines. A "walk me through the
|
|
114
|
+
whole import flow" gets a full shape. If your answer runs past about a screen,
|
|
115
|
+
either the question was broad or you're explaining implementation — check
|
|
116
|
+
which.
|
|
@@ -0,0 +1,157 @@
|
|
|
1
|
+
# Worked examples
|
|
2
|
+
|
|
3
|
+
> The product below is **invented** — a generic workspace app with members,
|
|
4
|
+
> documents and file imports. These examples show the shape and register of a
|
|
5
|
+
> good answer, not real behaviour. Always answer from what you actually find in
|
|
6
|
+
> the codebase in front of you.
|
|
7
|
+
|
|
8
|
+
## Example 1 — feature description
|
|
9
|
+
|
|
10
|
+
**Q: "What happens when a user is invited to a workspace?"**
|
|
11
|
+
|
|
12
|
+
```
|
|
13
|
+
**What it does:** sends an invitation so someone can join a workspace with a chosen role.
|
|
14
|
+
**Who can do it:** workspace owner and admins.
|
|
15
|
+
**How to trigger:** Workspace Settings → Members → "Invite" → enter email + pick role.
|
|
16
|
+
**What happens:**
|
|
17
|
+
- Invitation email is sent to that address.
|
|
18
|
+
- Pending invite shows in the members list.
|
|
19
|
+
- If the email matches an existing user, they accept by clicking the link and the workspace appears in their sidebar.
|
|
20
|
+
- If not, they're prompted to register first; after register they land on the workspace.
|
|
21
|
+
**Rules:**
|
|
22
|
+
- Email must be valid.
|
|
23
|
+
- Same email can't have two pending invites for the same workspace.
|
|
24
|
+
- Role must be one of: owner, admin, member, guest.
|
|
25
|
+
**Edge cases:**
|
|
26
|
+
- Invite link expires after 7 days.
|
|
27
|
+
- If the invited user is already a member, the invite is rejected with a message.
|
|
28
|
+
- Re-sending an invite resets the expiry.
|
|
29
|
+
**Not implemented:** bulk invite via CSV is not available in the UI.
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
Why it works: every line is something a tester can act on. The last line is the
|
|
33
|
+
most valuable one in the answer.
|
|
34
|
+
|
|
35
|
+
## Example 2 — permission question
|
|
36
|
+
|
|
37
|
+
**Q: "Can a guest delete a document?"**
|
|
38
|
+
|
|
39
|
+
```
|
|
40
|
+
No. Guests can view documents but cannot edit, create, or delete them. The
|
|
41
|
+
delete button is hidden for guests, and the action is also blocked server-side.
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Two lines, and it answers the follow-up question a tester was about to ask —
|
|
45
|
+
whether the block survives a direct API call.
|
|
46
|
+
|
|
47
|
+
## Example 3 — support question with nuance
|
|
48
|
+
|
|
49
|
+
**Q: "Is there a way to export a document?"**
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
Yes — partially.
|
|
53
|
+
- CSV export is available from the document page (top-right menu).
|
|
54
|
+
- PDF export is **not** implemented.
|
|
55
|
+
- The raw JSON can be retrieved via API, but there's no UI button for it.
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
"Yes — partially" in the first line, then exactly where the line falls.
|
|
59
|
+
|
|
60
|
+
## Example 4 — flow walkthrough
|
|
61
|
+
|
|
62
|
+
**Q: "Walk me through importing records from a file."**
|
|
63
|
+
|
|
64
|
+
```
|
|
65
|
+
1. Workspace → Data → "Import" → pick a file (CSV or XLSX).
|
|
66
|
+
2. A preview shows the first rows with a column-mapping step — you map each
|
|
67
|
+
column to a field (name, owner, category, tags).
|
|
68
|
+
3. Confirm. The import runs in the background; the page shows a progress state.
|
|
69
|
+
4. When it finishes you get an in-app notification with a summary: created,
|
|
70
|
+
updated, skipped.
|
|
71
|
+
|
|
72
|
+
**Branches:**
|
|
73
|
+
- If a row has no name, it's skipped and counted in "skipped".
|
|
74
|
+
- If a record with the same name already exists in that category, it's updated
|
|
75
|
+
rather than duplicated.
|
|
76
|
+
- If the whole file can't be parsed, nothing is imported and you get an error
|
|
77
|
+
on the same screen.
|
|
78
|
+
|
|
79
|
+
**Rules / limits:**
|
|
80
|
+
- File must be under 10 MB.
|
|
81
|
+
- Only CSV and XLSX; other extensions are rejected before upload.
|
|
82
|
+
|
|
83
|
+
**Edge cases:**
|
|
84
|
+
- An empty file imports nothing and reports "0 created".
|
|
85
|
+
- Closing the tab doesn't cancel the import — it keeps running.
|
|
86
|
+
|
|
87
|
+
**Not implemented:** there's no way to undo or roll back an import.
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
## Example 5 — edge-case sweep
|
|
91
|
+
|
|
92
|
+
**Q: "What should I test around document deletion?"**
|
|
93
|
+
|
|
94
|
+
```
|
|
95
|
+
**Permissions:** owner and admin can delete; member and guest cannot (hidden and blocked server-side).
|
|
96
|
+
**Empty:** deleting a document with no content works and removes it immediately.
|
|
97
|
+
**In use:** a document open in someone else's tab can be deleted — their next save fails with a generic error, which is worth checking.
|
|
98
|
+
**Linked data:** attached files are removed in the background, so they can outlive the document by a few seconds.
|
|
99
|
+
**Concurrent:** two admins deleting the same document at once — the second gets a "not found" error rather than a friendly message.
|
|
100
|
+
**Undo:** there is none, and there's no confirmation step on the API — only in the UI.
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
## Contrast — the same question answered badly
|
|
104
|
+
|
|
105
|
+
**Q: "What happens when an import finishes?"**
|
|
106
|
+
|
|
107
|
+
❌ **Bad — implementation leaking through:**
|
|
108
|
+
|
|
109
|
+
```
|
|
110
|
+
When the import completes, the imports controller calls the finalizer service,
|
|
111
|
+
which updates the status column on the imports table to "finished" via a
|
|
112
|
+
persistence callback. A post-commit hook enqueues the notification job on the
|
|
113
|
+
background queue, which reads the workspace notification settings and delivers
|
|
114
|
+
through the import mailer.
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
✅ **Good — same facts, tester's language:**
|
|
118
|
+
|
|
119
|
+
```
|
|
120
|
+
**What happens:**
|
|
121
|
+
- The import's status changes to "Finished" and the page updates with the final created/updated/skipped counts.
|
|
122
|
+
- Everyone subscribed to the workspace gets an email summary — sent in the background, so it can arrive a few seconds late.
|
|
123
|
+
- The import appears in the workspace's history.
|
|
124
|
+
**Edge cases:**
|
|
125
|
+
- An import with no valid rows still finishes, showing 0 created.
|
|
126
|
+
- If notifications are turned off for the workspace, no email is sent — the import still finishes normally.
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
The bad version is not wrong, it's unusable: nothing in it can be clicked,
|
|
130
|
+
observed, or verified from the UI.
|
|
131
|
+
|
|
132
|
+
## Contrast — padding vs. answering
|
|
133
|
+
|
|
134
|
+
**Q: "Can I rename a workspace?"**
|
|
135
|
+
|
|
136
|
+
❌ **Bad:**
|
|
137
|
+
|
|
138
|
+
```
|
|
139
|
+
Great question! Let me walk you through workspace renaming. Workspaces in the
|
|
140
|
+
system have a name attribute which can be modified. Here's what I found:
|
|
141
|
+
**What it does:** allows renaming a workspace.
|
|
142
|
+
**Who can do it:** users with permission.
|
|
143
|
+
**Rules / limits:** N/A
|
|
144
|
+
**Edge cases:** N/A
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
✅ **Good:**
|
|
148
|
+
|
|
149
|
+
```
|
|
150
|
+
Yes. Workspace Settings → General → change the name → Save.
|
|
151
|
+
Only the workspace owner can do it; admins see the field disabled.
|
|
152
|
+
The name must be unique inside the account, 3–60 characters.
|
|
153
|
+
The workspace URL doesn't change when you rename — it keeps the original slug.
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
No preamble, no empty sections, and it ends on the detail a tester would
|
|
157
|
+
otherwise discover the hard way.
|
|
@@ -0,0 +1,91 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: qa-lead-strategy-advisor
|
|
3
|
+
description: A strategic advisor for QA, quality, testing, and automation processes. Acts as a senior QA lead. First interviews the user and scans the project to build rich context, then identifies the highest-impact gaps and delivers a prioritized step-by-step roadmap, routing execution to specialized skills. Trigger this skill when the user wants to set up QA from scratch, improve an existing process, set up test automation, decide where to start with testing, or run a QA maturity review — e.g. "set up QA for this project", "improve our QA/testing/automation process", "where should we start with testing", "we have no tests, help", "review our quality process". Also trigger this skill when the user describes problems like "low/poor quality", "missing/a lot of bugs", "no testing".
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# QA Strategy Advisor
|
|
7
|
+
|
|
8
|
+
**Strategic advisor** for QA, quality, testing, and automation processes. I **interview → assess → prioritize → advise**: gather context first, then tell you the most important thing to do next, and route execution to specialized skills only when user asks. I advise on _what to do, why, and in what order_ — I never dump full detail up front; I expand a step only when user asks.
|
|
9
|
+
|
|
10
|
+
## Phase 0. Plan mode
|
|
11
|
+
|
|
12
|
+
**Strictly** recommend user to use/switch to "PLAN" mode in their AI agent.
|
|
13
|
+
|
|
14
|
+
## Phase 1 — Discovery (gather context before advising)
|
|
15
|
+
|
|
16
|
+
1. **Auto-detect** by scanning the codebase and checking existing MCP tools.
|
|
17
|
+
|
|
18
|
+
Run `scan-automation-project` to detect languages, frameworks, and existing manual/automated tests/cases.
|
|
19
|
+
|
|
20
|
+
2. **Interview** the user — interactively, in rounds.
|
|
21
|
+
|
|
22
|
+
**This step is crucial**. Don't rely just on scanned code. You should know the current product, processes, team, problems. They will not be defined in the codebase.
|
|
23
|
+
Don't be shy to ask questions, clarify, argue. QA is a wide and complex area — the more you know about how it is going, the better you can advise.
|
|
24
|
+
|
|
25
|
+
**This is a dialogue, not a questionnaire.** Never dump all interview questions as one plain-text list — that ends the conversation instead of starting it. Instead:
|
|
26
|
+
|
|
27
|
+
- **Use the `AskUserQuestion` tool** (if your agent provides it) to ask questions with selectable options. Ask **3–5 questions per round**, then wait for the answers before asking the next round.
|
|
28
|
+
- If you are sure about **concrete options** – use them, plus let the user type their own answer. Use multi-select where several answers can be true.
|
|
29
|
+
- **Adapt follow-up rounds to previous answers.** If the user says "no CI", drill into how releases are verified; if they name a TMS, ask for access/links. Skip questions the scan already answered.
|
|
30
|
+
- Run **3–4 rounds** total — enough to cover the topics below, short enough not to exhaust the user.
|
|
31
|
+
- If the `AskUserQuestion` tool is not available, ask **at most 3 questions in plain text and stop your turn** to wait for the reply — never continue past unanswered questions.
|
|
32
|
+
|
|
33
|
+
Topics to cover across the rounds:
|
|
34
|
+
|
|
35
|
+
- **Tools/trackers** — ask user which resources/tools/trackers do they use, ask for links, content, etc.
|
|
36
|
+
- **Processes** — ask to describe the current processes they use.
|
|
37
|
+
- **QA/Testing** — which QA and testing activities are performed today, which testing artifacts exist.
|
|
38
|
+
- **Product & risk** — what the app does, who uses it, highest-risk areas, release cadence.
|
|
39
|
+
- **Team & process** — team size, QA roles, how testing happens today, what's owned by whom; what is the role of current user.
|
|
40
|
+
- **Assets** — requirements/docs, existing test cases, TMS in use
|
|
41
|
+
- **CI/CD** — ask user to describe the current CI/CD pipeline, ask for links, content, etc.
|
|
42
|
+
- **Automation** — frameworks, rough coverage, reporting, flakiness/pain (ask if could not detect from codebase).
|
|
43
|
+
- **Problems/issues** — ask to describe the current problems/issues.
|
|
44
|
+
- **Goals** — biggest pain point, what success looks like.
|
|
45
|
+
|
|
46
|
+
Store gathered context to markdown file `.testeiya/TESTING.md` and check if this file exists. If it exists, append to it. If it does not exist, create it.
|
|
47
|
+
|
|
48
|
+
3. **Summarize** and get approval from the user.
|
|
49
|
+
|
|
50
|
+
Show gathered context to user (in a well structured, formatted and readable way). Keep it scannable — short bullets, not paragraphs.
|
|
51
|
+
|
|
52
|
+
Then ask for approval (use `AskUserQuestion` tool if available):
|
|
53
|
+
|
|
54
|
+
> Did I get the picture right?
|
|
55
|
+
>
|
|
56
|
+
> - ✅ **Yes — build the roadmap** (recommended)
|
|
57
|
+
> - ✏️ **Let's adjust** — I'll tell you what to fix
|
|
58
|
+
> - ❌ **No — let's revisit the questions**
|
|
59
|
+
|
|
60
|
+
If the user picks corrections, apply them, show only the changed lines, and re-confirm the same way. Never end the turn with "say go if it looks right" as plain text.
|
|
61
|
+
|
|
62
|
+
## Phase 2 — Assessment & prioritized roadmap
|
|
63
|
+
|
|
64
|
+
Map findings to QA pillars: **quality assurance and testing processes · test design · test coverage · automation, CI · reporting/metrics · TMS integration**. Find the gaps, rank by **impact first, then effort**, and present an ordered list (starting with the most important).
|
|
65
|
+
|
|
66
|
+
**Formatting is critical** — read [references/output-format.md](references/output-format.md) before producing the roadmap and follow it strictly. It contains the item template and a full example. Never output dense numbered paragraphs.
|
|
67
|
+
|
|
68
|
+
**Display rule (critical)**
|
|
69
|
+
|
|
70
|
+
- Print the full roadmap as the **final message of your turn**, do **not** call `AskUserQuestion` in the same turn when you display the roadmap (it causes the roadmap to be hidden).
|
|
71
|
+
- End the roadmap with the 💬 call-to-action line from the output format (e.g. `Type "execute 1", "expand 1", "adjust", or "save"`). That line replaces the interactive menu for this turn.
|
|
72
|
+
|
|
73
|
+
Store roadmap to `.testeiya/TESTING.md`.
|
|
74
|
+
|
|
75
|
+
## Phase 3. Execute
|
|
76
|
+
|
|
77
|
+
Phase 3 starts when the user chooses what to do next.
|
|
78
|
+
|
|
79
|
+
- If the reply is a clear command ("execute 1", "expand 2", "save") — act on it directly.
|
|
80
|
+
- If the reply is ambiguous ("ok", "looks good", a question) — **now** offer the menu via `AskUserQuestion` (the roadmap is already visible above, so it's safe here):
|
|
81
|
+
|
|
82
|
+
> What's next?
|
|
83
|
+
>
|
|
84
|
+
> - ▶️ **Execute item 1** — <one-line restatement of item 1>
|
|
85
|
+
> - 🔎 **Details of item 1** — see the detailed plan first
|
|
86
|
+
> - ✏️ **Adjust the roadmap** — reorder, drop, or add items
|
|
87
|
+
> - 💾 **Save & stop here** — write the roadmap to a file and finish
|
|
88
|
+
|
|
89
|
+
After the user's approval, route execution to the `testing-workflow` or any other relevant skill (or act without a skill if none fits).
|
|
90
|
+
|
|
91
|
+
When an item finishes, confirm the result and offer the next roadmap item the same way — keep the loop going until the user stops.
|