@qatonic_innovations/qaios 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +196 -0
- package/package.json +3 -3
package/README.md
ADDED
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
# QAIOS
|
|
2
|
+
|
|
3
|
+
> **AI QA engineer in your terminal.** Designs, writes, runs, heals, and explores tests for web UI and APIs — with audit-grade traceability.
|
|
4
|
+
|
|
5
|
+
[](https://www.npmjs.com/package/@qatonic_innovations/qaios)
|
|
6
|
+
[](https://github.com/qatonic/qaios/blob/main/LICENSE)
|
|
7
|
+
[](https://nodejs.org)
|
|
8
|
+
|
|
9
|
+
QAIOS acts like a QA engineer on your team. Point it at a feature description or an OpenAPI spec; it produces a test design, generates Playwright code, runs the tests, classifies failures, heals broken locators, files defects, scans for accessibility issues, and keeps a hash-chained audit log of every decision.
|
|
10
|
+
|
|
11
|
+
**Mental model: Claude Code, but for QA work instead of feature coding.**
|
|
12
|
+
|
|
13
|
+
---
|
|
14
|
+
|
|
15
|
+
## Install
|
|
16
|
+
|
|
17
|
+
```bash
|
|
18
|
+
npm install -g @qatonic_innovations/qaios
|
|
19
|
+
# or: pnpm add -g @qatonic_innovations/qaios
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
The published package is `@qatonic_innovations/qaios`; the installed **command is `qaios`**.
|
|
23
|
+
|
|
24
|
+
### Requirements
|
|
25
|
+
|
|
26
|
+
- **Node.js 20 LTS or newer**
|
|
27
|
+
- An **Anthropic API key** in your environment:
|
|
28
|
+
```bash
|
|
29
|
+
export ANTHROPIC_API_KEY=sk-ant-... # macOS/Linux
|
|
30
|
+
setx ANTHROPIC_API_KEY "sk-ant-..." # Windows (new shell after)
|
|
31
|
+
```
|
|
32
|
+
- **Playwright** in your project, for `qaios run` / `snapshot` / `a11y`:
|
|
33
|
+
```bash
|
|
34
|
+
npm i -D @playwright/test && npx playwright install
|
|
35
|
+
```
|
|
36
|
+
- For `qaios a11y`, also: `npm i -D @axe-core/playwright`
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## 60-second quick start
|
|
41
|
+
|
|
42
|
+
```bash
|
|
43
|
+
cd my-app
|
|
44
|
+
qaios init # creates .qaios/, QAIOS.md; detects Playwright
|
|
45
|
+
qaios doctor # verify your environment is ready
|
|
46
|
+
|
|
47
|
+
# Generate tests from plain English, then run them:
|
|
48
|
+
qaios test "user can log in with email and password" --run
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
`qaios test` produces a DesignSpec, a Playwright spec, and a Page Object — then (`--run`) executes them. That's the core loop.
|
|
52
|
+
|
|
53
|
+
```text
|
|
54
|
+
workflow: 01JTZ1MV7Q8X3R2WFP9D5K8YRH
|
|
55
|
+
|
|
56
|
+
▸ requirements.intake ✓ (2 reqs, confidence 0.85)
|
|
57
|
+
▸ risk.score ✓ (overall HIGH — touches PII)
|
|
58
|
+
▸ design.web ✓ (4 scenarios, confidence 0.84)
|
|
59
|
+
▸ write.web ✓ (1 spec, 4 tests, 1 page object)
|
|
60
|
+
|
|
61
|
+
Generated:
|
|
62
|
+
tests/auth/login.spec.ts (4 tests)
|
|
63
|
+
tests/auth/LoginPage.po.ts
|
|
64
|
+
.qaios/specs/01JTZ1…/DesignSpec.json
|
|
65
|
+
|
|
66
|
+
Cost: 11,403 tokens · $0.06 (3 of 15 LLM calls used)
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
> **Gates:** on HIGH/CRITICAL-risk features QAIOS pauses for review. Approve interactively with `qaios review`, or pass `--yes` to auto-approve for a non-interactive run.
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## What each command does
|
|
74
|
+
|
|
75
|
+
| Command | What it does |
|
|
76
|
+
| ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
|
77
|
+
| `qaios init [path]` | Bootstrap a project: create `.qaios/`, `QAIOS.md`, run migrations. |
|
|
78
|
+
| `qaios doctor` | Health-check your environment (Node, API key, Playwright, config). `--json` for CI. |
|
|
79
|
+
| `qaios test <description>` | Design + generate tests from natural language. `--type api` for OpenAPI. `--run` to execute. `--yes` to auto-approve gates. |
|
|
80
|
+
| `qaios run [pattern]` | Execute Playwright tests, classify failures (real / flaky / locator drift), auto-heal locator breaks, file defects. |
|
|
81
|
+
| `qaios fix [target]` | Self-heal a broken locator. `--last-failure` to target the most recent failure, `--auto` to apply, `--dry-run` to preview. |
|
|
82
|
+
| `qaios explore <url>` | Time-boxed exploratory session against a live URL. `--duration <seconds>`, `--focus "<hint>"`. |
|
|
83
|
+
| `qaios a11y <url>` | Deterministic axe-core accessibility scan + AI triage. `--wcag-level A\|AA\|AAA`. Exits 1 on real violations (CI-gateable). |
|
|
84
|
+
| `qaios snapshot capture\|check\|review` | Visual regression: capture baselines, diff, and triage AI-classified diffs. |
|
|
85
|
+
| `qaios review` | Walk pending human-review gates and approve/reject; resumes the workflow. |
|
|
86
|
+
| `qaios history [--verify]` | List workflows, inspect the audit log, and verify the hash chain. |
|
|
87
|
+
| `qaios config get\|set\|show\|edit` | Read/write `.qaios/config.yaml` (schema-validated). |
|
|
88
|
+
| `qaios mcp list\|add\|remove\|enable\|disable\|test` | Manage MCP servers (GitHub, Jira, Slack, custom). |
|
|
89
|
+
|
|
90
|
+
Run `qaios <command> --help` for the full option list of any command.
|
|
91
|
+
|
|
92
|
+
### Common workflows
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
# Generate API tests from an OpenAPI spec
|
|
96
|
+
qaios test --type api --spec ./openapi.yaml "exercise the /orders endpoints"
|
|
97
|
+
|
|
98
|
+
# Run a suite; QAIOS classifies failures and self-heals locator drift
|
|
99
|
+
qaios run
|
|
100
|
+
|
|
101
|
+
# Heal a specific broken test, previewing the patch first
|
|
102
|
+
qaios fix tests/auth/login.spec.ts --last-failure --dry-run
|
|
103
|
+
|
|
104
|
+
# Accessibility scan, gate CI on real WCAG violations
|
|
105
|
+
qaios a11y https://staging.myapp.com --wcag-level AA
|
|
106
|
+
|
|
107
|
+
# Exploratory testing with a focus hint
|
|
108
|
+
qaios explore https://staging.myapp.com --duration 900 --focus "checkout flow"
|
|
109
|
+
|
|
110
|
+
# Visual regression
|
|
111
|
+
qaios snapshot capture # baseline
|
|
112
|
+
# ...change CSS...
|
|
113
|
+
qaios snapshot check # diff
|
|
114
|
+
qaios snapshot review # triage interactively
|
|
115
|
+
```
|
|
116
|
+
|
|
117
|
+
---
|
|
118
|
+
|
|
119
|
+
## Configuration
|
|
120
|
+
|
|
121
|
+
Config lives in `.qaios/config.yaml` (created by `qaios init`):
|
|
122
|
+
|
|
123
|
+
```yaml
|
|
124
|
+
version: 1
|
|
125
|
+
mode: LITE # LITE | FULL | TRUST
|
|
126
|
+
app:
|
|
127
|
+
baseUrl: https://staging.myapp.com
|
|
128
|
+
llm:
|
|
129
|
+
provider: anthropic
|
|
130
|
+
apiKeyEnv: ANTHROPIC_API_KEY # key is read from env, never stored
|
|
131
|
+
maxLlmCallsPerWorkflow: 15
|
|
132
|
+
costAlertThresholdUsdCents: 50
|
|
133
|
+
testing:
|
|
134
|
+
framework: playwright
|
|
135
|
+
testDir: tests
|
|
136
|
+
testIdAttribute: data-testid # e.g. set to 'data-test' for some apps
|
|
137
|
+
locatorPreference: [data-testid, role, text]
|
|
138
|
+
gates:
|
|
139
|
+
autoExpireOnTimeout: abort # abort | auto_approve | auto_reject
|
|
140
|
+
defects:
|
|
141
|
+
target: stdout # stdout | github | jira
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
`qaios config show` prints the resolved config; `qaios config set <key> <value>` validates against the schema before writing. **API keys are never written to config** — they come from environment variables.
|
|
145
|
+
|
|
146
|
+
### Operating modes
|
|
147
|
+
|
|
148
|
+
- **LITE** (default) — HIGH/CRITICAL risk pauses for review; routine work flows through.
|
|
149
|
+
- **FULL** — every gate active, every AI decision reviewable.
|
|
150
|
+
- **TRUST** — auto-approve when confidence > 0.85 and risk ≤ MEDIUM; CRITICAL still notifies.
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
qaios config set mode TRUST
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
---
|
|
157
|
+
|
|
158
|
+
## Cost & privacy
|
|
159
|
+
|
|
160
|
+
- All v0.1 skills use Claude Sonnet. A typical `qaios test` costs ~**$0.04–0.10**. Each workflow is capped (default `min(15 calls, $0.50)`, configurable) and aborts if exceeded.
|
|
161
|
+
- **Local-first.** No telemetry, no phone-home. The only outbound traffic is (a) LLM calls to Anthropic with the prompts QAIOS builds, and (b) any MCP servers you configure. You bring your own API key; QAIOS does not proxy your requests.
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## Documentation
|
|
166
|
+
|
|
167
|
+
- [Getting Started](https://github.com/qatonic/qaios/blob/main/docs/getting-started.md) — first 15 minutes
|
|
168
|
+
- [Commands Reference](https://github.com/qatonic/qaios/blob/main/docs/commands.md) — every CLI command
|
|
169
|
+
- [Skills Reference](https://github.com/qatonic/qaios/blob/main/docs/skills-reference.md) — what each skill does
|
|
170
|
+
- [MCP Servers](https://github.com/qatonic/qaios/blob/main/docs/mcp-servers.md) — adding integrations
|
|
171
|
+
- A runnable end-to-end example lives in [`examples/live-saucedemo`](https://github.com/qatonic/qaios/tree/main/examples/live-saucedemo).
|
|
172
|
+
|
|
173
|
+
---
|
|
174
|
+
|
|
175
|
+
## Status & roadmap
|
|
176
|
+
|
|
177
|
+
**v0.1 (now):** Web UI + API + visual regression + self-healing + exploratory + accessibility, as an open-source CLI. Early — feedback wanted.
|
|
178
|
+
|
|
179
|
+
- **v0.2** — optional cloud sync + web dashboard
|
|
180
|
+
- **v0.5** — mobile testing
|
|
181
|
+
- **v0.7** — performance + security testing
|
|
182
|
+
- **v1.0** — database testing, stable public API
|
|
183
|
+
|
|
184
|
+
The roadmap is gated on real-world validation, not a calendar.
|
|
185
|
+
|
|
186
|
+
---
|
|
187
|
+
|
|
188
|
+
## Links
|
|
189
|
+
|
|
190
|
+
- **Repository:** https://github.com/qatonic/qaios
|
|
191
|
+
- **Report a bug:** https://github.com/qatonic/qaios/issues
|
|
192
|
+
- **Discuss / request a feature:** https://github.com/qatonic/qaios/discussions
|
|
193
|
+
|
|
194
|
+
## License
|
|
195
|
+
|
|
196
|
+
[MIT](https://github.com/qatonic/qaios/blob/main/LICENSE). The CLI will always remain MIT; cloud features (v0.2+) ship under a separate license.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@qatonic_innovations/qaios",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.1",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "AI QA engineer in your terminal — designs, writes, runs, heals, and explores tests for web UI and APIs with audit-grade traceability.",
|
|
6
6
|
"license": "MIT",
|
|
@@ -67,8 +67,8 @@
|
|
|
67
67
|
"@types/react": "^18.3.28",
|
|
68
68
|
"ink-testing-library": "^4.0.0",
|
|
69
69
|
"@qaios/runtime": "0.0.0",
|
|
70
|
-
"@qaios/
|
|
71
|
-
"@qaios/
|
|
70
|
+
"@qaios/shared": "0.0.0",
|
|
71
|
+
"@qaios/skills": "0.0.0"
|
|
72
72
|
},
|
|
73
73
|
"scripts": {
|
|
74
74
|
"build": "tsup && node scripts/copy-templates.mjs && node scripts/copy-runtime-assets.mjs",
|