@democratize-quality/qualitylens-core 0.1.0 → 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +196 -0
- package/package.json +3 -2
package/README.md
ADDED
|
@@ -0,0 +1,196 @@
|
|
|
1
|
+
# qualitylens
|
|
2
|
+
|
|
3
|
+
> **Test coverage map — automated + manual, by functional area.**
|
|
4
|
+
> Know exactly what's tested, what isn't, and where the risk is — before every release.
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## The problem
|
|
9
|
+
|
|
10
|
+
Your CI is green. Your Playwright suite has 600 tests. But does `/billing/refund` have a test?
|
|
11
|
+
Does `/onboarding/import-data`? When was `/auth/sso` last tested manually?
|
|
12
|
+
|
|
13
|
+
Nobody knows. Until production breaks.
|
|
14
|
+
|
|
15
|
+
`qualitylens` answers the question your coverage tool can't:
|
|
16
|
+
**"Which user-facing flows have no test coverage — automated or manual?"**
|
|
17
|
+
|
|
18
|
+
---
|
|
19
|
+
|
|
20
|
+
## Free vs Pro
|
|
21
|
+
|
|
22
|
+
| Feature | Free (`qualitylens`) | Pro (`qualitylens` + `qualitylens-pro`) |
|
|
23
|
+
|---------|-----------------|--------------------------------|
|
|
24
|
+
| Playwright test source | ✅ | ✅ |
|
|
25
|
+
| YAML manual coverage | ✅ | ✅ |
|
|
26
|
+
| Console reporter | ✅ | ✅ |
|
|
27
|
+
| `qualitylens init` / `validate` | ✅ | ✅ |
|
|
28
|
+
| CI `--fail-under` gate | ✅ | ✅ |
|
|
29
|
+
| HTML reporter | — | ✅ |
|
|
30
|
+
| JSON reporter | — | ✅ |
|
|
31
|
+
| Markdown reporter | — | ✅ |
|
|
32
|
+
| Azure DevOps Test Plans | — | ✅ |
|
|
33
|
+
|
|
34
|
+
Both tiers use the single `qualitylens` binary. Pro features activate automatically when `qualitylens-pro` is installed alongside.
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## Installation
|
|
39
|
+
|
|
40
|
+
**Free:**
|
|
41
|
+
```bash
|
|
42
|
+
npm install -g @democratize-quality/qualitylens
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
**Pro:**
|
|
46
|
+
```bash
|
|
47
|
+
npm install -g @democratize-quality/qualitylens @democratize-quality/qualitylens-pro
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
No config change needed — the CLI detects `qualitylens-pro` at runtime.
|
|
51
|
+
|
|
52
|
+
---
|
|
53
|
+
|
|
54
|
+
## Quick start
|
|
55
|
+
|
|
56
|
+
```bash
|
|
57
|
+
npx @democratize-quality/qualitylens init # creates qualitylens.yaml in your project root
|
|
58
|
+
npx @democratize-quality/qualitylens scan # generates console report (+ HTML/JSON/MD with Pro)
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Sample output:
|
|
62
|
+
|
|
63
|
+
```
|
|
64
|
+
qualitylens: 68% total coverage (41% auto, 27% manual, 32% none)
|
|
65
|
+
|
|
66
|
+
Authentication ████████████████████ 92% ✅
|
|
67
|
+
Checkout & Payments ████████████░░░░░░░░ 54% ⚠️
|
|
68
|
+
Onboarding ████████████░░░░░░░░ 60% ⚠️
|
|
69
|
+
Admin & Settings █████░░░░░░░░░░░░░░░ 28% ❌
|
|
70
|
+
|
|
71
|
+
No coverage found (8 routes):
|
|
72
|
+
❌ /checkout/promo-code
|
|
73
|
+
❌ /billing/refund
|
|
74
|
+
❌ /billing/invoice-download
|
|
75
|
+
❌ /admin/billing-settings
|
|
76
|
+
❌ /admin/permissions
|
|
77
|
+
❌ /admin/audit-log
|
|
78
|
+
❌ /onboarding/import-data
|
|
79
|
+
❌ /auth/mfa-setup
|
|
80
|
+
|
|
81
|
+
Stale manual coverage (>30 days, 2 routes):
|
|
82
|
+
⚠️ /onboarding/welcome — last tested 34d ago (priya)
|
|
83
|
+
⚠️ /onboarding/profile-setup — last tested 34d ago (priya)
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
---
|
|
87
|
+
|
|
88
|
+
## How it works
|
|
89
|
+
|
|
90
|
+
`qualitylens` combines test sources against your app routes:
|
|
91
|
+
|
|
92
|
+
| Source | What it reads | How | Tier |
|
|
93
|
+
|--------|--------------|-----|------|
|
|
94
|
+
| Playwright | `*.spec.ts` test titles | AST parsing | Free |
|
|
95
|
+
| YAML | `manualCoverage` entries | YAML file | Free |
|
|
96
|
+
| Azure DevOps | Test Plan test case names | REST API | Pro |
|
|
97
|
+
|
|
98
|
+
It fuzzy-matches test descriptions against your app's routes, groups them by functional area, and reports what's covered, what's manual-only, what's stale, and what has no coverage at all.
|
|
99
|
+
|
|
100
|
+
See [docs/HOW_IT_WORKS.md](docs/HOW_IT_WORKS.md) for full details.
|
|
101
|
+
|
|
102
|
+
---
|
|
103
|
+
|
|
104
|
+
## Configuration
|
|
105
|
+
|
|
106
|
+
Place a `qualitylens.yaml` in your project root:
|
|
107
|
+
|
|
108
|
+
```yaml
|
|
109
|
+
projectName: my-saas-app
|
|
110
|
+
staleThresholdDays: 30
|
|
111
|
+
|
|
112
|
+
routes:
|
|
113
|
+
type: nextjs
|
|
114
|
+
path: ./pages
|
|
115
|
+
|
|
116
|
+
areas:
|
|
117
|
+
- name: Checkout & Payments
|
|
118
|
+
patterns: [/checkout, /billing]
|
|
119
|
+
- name: Authentication
|
|
120
|
+
patterns: [/auth, /login]
|
|
121
|
+
|
|
122
|
+
# Optional: Azure DevOps Test Plans
|
|
123
|
+
# ado:
|
|
124
|
+
# orgUrl: https://dev.azure.com/myorg
|
|
125
|
+
# project: MyProject
|
|
126
|
+
# planId: 42
|
|
127
|
+
# patEnvVar: ADO_PAT
|
|
128
|
+
|
|
129
|
+
# Manual coverage (from Excel, exploratory testing, etc.)
|
|
130
|
+
manualCoverage:
|
|
131
|
+
- route: /auth/sso
|
|
132
|
+
lastTested: 2025-03-01
|
|
133
|
+
tester: james
|
|
134
|
+
|
|
135
|
+
# CI gates
|
|
136
|
+
thresholds:
|
|
137
|
+
- area: Checkout & Payments
|
|
138
|
+
minCoverage: 70
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
Full reference: [docs/CONFIGURATION.md](docs/CONFIGURATION.md)
|
|
142
|
+
|
|
143
|
+
---
|
|
144
|
+
|
|
145
|
+
## CLI reference
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
qualitylens scan [options]
|
|
149
|
+
--config <path> path to qualitylens.yaml (default: ./qualitylens.yaml)
|
|
150
|
+
--output <path> output directory (default: ./)
|
|
151
|
+
--format <formats> console (free) | html,json,markdown (Pro) — comma-separated
|
|
152
|
+
--since <branch> only show routes changed since this git branch
|
|
153
|
+
--fail-under <n> exit 1 if total coverage < n (CI gate)
|
|
154
|
+
|
|
155
|
+
qualitylens init interactive setup — creates qualitylens.yaml
|
|
156
|
+
qualitylens validate checks config and ADO connection
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
> **Pro formats** — if you run `--format html` without `qualitylens-pro` installed, qualitylens prints an install hint and continues. Nothing crashes.
|
|
160
|
+
|
|
161
|
+
---
|
|
162
|
+
|
|
163
|
+
## CI integration
|
|
164
|
+
|
|
165
|
+
**Free:**
|
|
166
|
+
```yaml
|
|
167
|
+
- name: Run qualitylens
|
|
168
|
+
run: npx @democratize-quality/qualitylens scan --format console --fail-under 60
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
**Pro** (HTML + JSON report, ADO source):
|
|
172
|
+
```yaml
|
|
173
|
+
- name: Run qualitylens Pro
|
|
174
|
+
run: npx @democratize-quality/qualitylens scan --format html,json,console --fail-under 60
|
|
175
|
+
env:
|
|
176
|
+
ADO_PAT: ${{ secrets.ADO_PAT }}
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
Full examples: [docs/CI_INTEGRATION.md](docs/CI_INTEGRATION.md)
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
## VS Code extension (Pro)
|
|
184
|
+
|
|
185
|
+
The qualitylens Pro VS Code extension shows a live coverage dashboard in your sidebar.
|
|
186
|
+
The panel updates automatically when `qualitylens.yaml` changes. Requires `qualitylens-pro`.
|
|
187
|
+
|
|
188
|
+
---
|
|
189
|
+
|
|
190
|
+
## Built by
|
|
191
|
+
|
|
192
|
+
**Raj Uppadhyay** — author of *Scalable Test Automation with Playwright*
|
|
193
|
+
|
|
194
|
+
Related tools:
|
|
195
|
+
- [playwright-mcp-yaml](https://npmjs.com/package/playwright-mcp-yaml)
|
|
196
|
+
- [ADO Testing Extension](https://marketplace.visualstudio.com/items?itemName=RajUppadhyay.ado-testingextension)
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@democratize-quality/qualitylens-core",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.2.0",
|
|
4
4
|
"description": "Core library for qualitylens — engine, types, sources, matchers, reporters",
|
|
5
5
|
"author": "Raj Uppadhyay",
|
|
6
6
|
"license": "AGPL-3.0-only",
|
|
@@ -80,7 +80,8 @@
|
|
|
80
80
|
}
|
|
81
81
|
},
|
|
82
82
|
"files": [
|
|
83
|
-
"dist"
|
|
83
|
+
"dist",
|
|
84
|
+
"README.md"
|
|
84
85
|
],
|
|
85
86
|
"scripts": {
|
|
86
87
|
"build": "tsup",
|