quality.md 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +161 -150
- package/bin/qualitymd.js +2 -1
- package/package.json +7 -7
package/README.md
CHANGED
|
@@ -1,160 +1,208 @@
|
|
|
1
1
|
# QUALITY.md
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
3
|
+
**QUALITY.md** is an agent-friendly file format and companion agent skill and
|
|
4
|
+
CLI for continuously improving the quality of coding agent and AI assistant projects/harnesses.
|
|
5
|
+
|
|
6
|
+
## Install
|
|
7
|
+
|
|
8
|
+
1. Install the agent skill:
|
|
9
|
+
|
|
10
|
+
```sh
|
|
11
|
+
npx skills add qualitymd/quality.md
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
2. Install the CLI:
|
|
15
|
+
|
|
16
|
+
```sh
|
|
17
|
+
npm install quality.md -g
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Usage
|
|
21
|
+
|
|
22
|
+
Invoke the `/quality` skill to manage quality for your project:
|
|
23
|
+
|
|
24
|
+
```text
|
|
25
|
+
/quality setup Get started working with QUALITY.md
|
|
26
|
+
/quality wizard Have your AI assistant/agent help you manage quality
|
|
27
|
+
/quality evaluate Evaluate the quality of your project
|
|
28
|
+
/quality evaluate security Evaluate a specific quality factor or characteristic
|
|
29
|
+
/quality evaluate payments-api Evaluate specific target or project component
|
|
30
|
+
/quality evaluate payments-api maintainability Evaluate a target's specific quality
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## The Format
|
|
34
|
+
|
|
35
|
+
A `QUALITY.md` file combines a structured quality model with plain-language
|
|
36
|
+
rationale. The structured model names what is being evaluated, which quality
|
|
37
|
+
factors matter, what requirements define those factors, and how each requirement
|
|
38
|
+
should be assessed. The Markdown body explains the context: what the work is,
|
|
39
|
+
what "good" means, where the boundaries are, and why these standards matter.
|
|
40
|
+
|
|
41
|
+
### Specification
|
|
42
|
+
|
|
43
|
+
The format is specified in [`SPECIFICATION.md`](SPECIFICATION.md).
|
|
44
|
+
|
|
45
|
+
### Example QUALITY.md
|
|
22
46
|
|
|
23
47
|
```markdown
|
|
24
48
|
---
|
|
25
|
-
title:
|
|
49
|
+
title: Support Inbox
|
|
26
50
|
ratingScale:
|
|
27
51
|
- level: outstanding
|
|
28
52
|
title: Outstanding
|
|
29
|
-
|
|
53
|
+
description: The work clearly exceeds the shared quality bar.
|
|
54
|
+
criterion: "Consistently exceeds the requirement with clear margin."
|
|
30
55
|
- level: target
|
|
31
56
|
title: Target
|
|
32
|
-
|
|
57
|
+
description: The work meets the shared quality bar.
|
|
58
|
+
criterion: "Meets the expected quality bar."
|
|
33
59
|
- level: minimum
|
|
34
60
|
title: Minimum
|
|
35
|
-
|
|
61
|
+
description: The work is acceptable, but has gaps worth improving.
|
|
62
|
+
criterion: "Meets the lowest acceptable bar, with visible gaps."
|
|
36
63
|
- level: unacceptable
|
|
37
64
|
title: Unacceptable
|
|
38
|
-
|
|
65
|
+
description: The work is below the shared quality bar.
|
|
66
|
+
criterion: "Falls below the minimum acceptable bar."
|
|
39
67
|
targets:
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
"accepted orders are durable":
|
|
44
|
-
factors: [reliability]
|
|
45
|
-
assessment: >
|
|
46
|
-
A write is acknowledged to the client only after it is committed to
|
|
47
|
-
durable storage. Failures surface as errors, never as false successes.
|
|
68
|
+
triage:
|
|
69
|
+
title: Triage
|
|
70
|
+
source: ./support
|
|
48
71
|
factors:
|
|
49
|
-
|
|
50
|
-
|
|
72
|
+
responsiveness:
|
|
73
|
+
title: Responsiveness
|
|
74
|
+
description: Customers receive timely, useful attention.
|
|
51
75
|
requirements:
|
|
52
|
-
"
|
|
76
|
+
"urgent messages are visible":
|
|
53
77
|
assessment: >
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
78
|
+
New messages are classified so urgent customer-impacting issues
|
|
79
|
+
are separated from routine requests.
|
|
80
|
+
accuracy:
|
|
81
|
+
title: Accuracy
|
|
82
|
+
description: Replies are correct, complete, and grounded in policy.
|
|
58
83
|
requirements:
|
|
59
|
-
"
|
|
84
|
+
"answers cite the current policy":
|
|
60
85
|
assessment: >
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
docs:
|
|
64
|
-
source: ./docs
|
|
65
|
-
factors:
|
|
66
|
-
usability:
|
|
67
|
-
description: Integrators can understand and use the documentation unaided.
|
|
68
|
-
requirements:
|
|
69
|
-
"integration docs describe the order lifecycle":
|
|
70
|
-
assessment: >
|
|
71
|
-
The documentation explains how an order moves from request to durable
|
|
72
|
-
acknowledgement, including failure responses and retry guidance.
|
|
86
|
+
Customer-facing replies use the active support policy and do not
|
|
87
|
+
rely on outdated guidance or unsupported assumptions.
|
|
73
88
|
---
|
|
74
89
|
|
|
75
|
-
# Quality model
|
|
90
|
+
# Quality model: Support Inbox
|
|
76
91
|
|
|
77
92
|
## Overview
|
|
78
93
|
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
third-party payment provider is a dependency, not part of it.
|
|
83
|
-
|
|
84
|
-
## Targets and factors
|
|
85
|
-
|
|
86
|
-
### api
|
|
87
|
-
|
|
88
|
-
The API target covers the HTTP boundary and storage path.
|
|
94
|
+
This model describes the quality bar for daily support triage. Good support
|
|
95
|
+
means urgent issues are easy to see, routine requests still move, and customers
|
|
96
|
+
receive answers grounded in the current policy.
|
|
89
97
|
|
|
90
|
-
|
|
98
|
+
## Scope
|
|
91
99
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
#### Security
|
|
100
|
+
This model covers message triage and written replies in the support workspace.
|
|
101
|
+
It does not cover billing system behavior or product incident response.
|
|
102
|
+
```
|
|
96
103
|
|
|
97
|
-
|
|
104
|
+
An agent that reads this file can evaluate support work against the stated
|
|
105
|
+
requirements, produce findings, and rate the results against the model's scale.
|
|
106
|
+
|
|
107
|
+
## The Specification
|
|
108
|
+
|
|
109
|
+
The full format specification lives at [`SPECIFICATION.md`](SPECIFICATION.md).
|
|
110
|
+
What follows is a condensed reference.
|
|
111
|
+
|
|
112
|
+
### File Structure
|
|
113
|
+
|
|
114
|
+
A `QUALITY.md` file has two layers:
|
|
115
|
+
|
|
116
|
+
1. **YAML frontmatter** - the structured quality model.
|
|
117
|
+
2. **Markdown body** - the context, rationale, scope, needs, risks, and known
|
|
118
|
+
gaps that help people and agents interpret the model.
|
|
119
|
+
|
|
120
|
+
The document begins with the YAML frontmatter. The Markdown body can be empty,
|
|
121
|
+
but it is where the model explains itself.
|
|
122
|
+
|
|
123
|
+
### Model Schema
|
|
124
|
+
|
|
125
|
+
The root model is a target plus a model-wide `ratingScale`.
|
|
126
|
+
|
|
127
|
+
```yaml
|
|
128
|
+
title: <string> # Required
|
|
129
|
+
description: <string> # Optional
|
|
130
|
+
ratingScale: # Required, ordered best to worst
|
|
131
|
+
- level: <level-name> # Required, unique within the scale
|
|
132
|
+
title: <string> # Required
|
|
133
|
+
description: <string> # Recommended
|
|
134
|
+
criterion: <string> # Required
|
|
135
|
+
source: <string> # Optional
|
|
136
|
+
factors: # Optional*
|
|
137
|
+
<factor-name>:
|
|
138
|
+
title: <string> # Required
|
|
139
|
+
description: <string> # Recommended
|
|
140
|
+
factors: # Optional
|
|
141
|
+
<sub-factor-name>: <Factor>
|
|
142
|
+
requirements: # Optional
|
|
143
|
+
<requirement-statement>: <Requirement>
|
|
144
|
+
requirements: # Optional*
|
|
145
|
+
<requirement-statement>:
|
|
146
|
+
assessment: <string> # Required, exactly one
|
|
147
|
+
factors: [<factor-name>] # Required for direct target requirements
|
|
148
|
+
ratings: # Optional per-level criteria
|
|
149
|
+
<level-name>: <criterion>
|
|
150
|
+
targets: # Optional*
|
|
151
|
+
<target-name>: <Target>
|
|
98
152
|
```
|
|
99
153
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
declared), and **requirements** under either a target or a factor. Each
|
|
103
|
-
requirement carries one **`assessment`** and is connected to at least one factor; the
|
|
104
|
-
evaluator turns that assessment into a finding, then rates the finding against
|
|
105
|
-
the scale criteria. A target's **`source`** identifies the
|
|
106
|
-
material assessed, and child `targets:` decompose or narrow the subject. The
|
|
107
|
-
**body** holds the reasoning the frontmatter cannot: what the system is, what
|
|
108
|
-
*good* means for it, and why these are the right requirements.
|
|
154
|
+
At least one of `factors`, `requirements`, or `targets` must be supplied.
|
|
155
|
+
Targets can nest recursively. `ratingScale` exists only on the root model.
|
|
109
156
|
|
|
110
|
-
|
|
111
|
-
API against it — performing each `assessment` against the target source, then
|
|
112
|
-
reporting where the subject falls short. The `qualitymd` CLI owns deterministic
|
|
113
|
-
mechanical steps such as scaffolding, linting, spec grounding, and evaluation
|
|
114
|
-
artifact mechanics; judging is the skill's part.
|
|
157
|
+
### Core Concepts
|
|
115
158
|
|
|
116
|
-
|
|
159
|
+
| Concept | Meaning |
|
|
160
|
+
| ------------ | --------------------------------------------------------------- |
|
|
161
|
+
| Model | The root quality model in a `QUALITY.md` file. |
|
|
162
|
+
| Target | The thing being evaluated. |
|
|
163
|
+
| Source | The material assessed for a target, such as a path or selector. |
|
|
164
|
+
| Factor | A quality dimension that matters for a target. |
|
|
165
|
+
| Requirement | A specific quality expectation. |
|
|
166
|
+
| Assessment | The means of checking a requirement against a target source. |
|
|
167
|
+
| Finding | An observation produced by an assessment. |
|
|
168
|
+
| Rating Scale | The ordered model-wide scale used to rate results. |
|
|
117
169
|
|
|
118
|
-
|
|
170
|
+
### Evaluation Semantics
|
|
119
171
|
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
Then use the skill in your agent:
|
|
172
|
+
Each requirement has exactly one `assessment`. An evaluator performs that
|
|
173
|
+
assessment against the target source, records findings, and rates the
|
|
174
|
+
requirement's findings against the model's `ratingScale`. A finding records what
|
|
175
|
+
was observed; the rating result records how that observation compares with the
|
|
176
|
+
model's scale.
|
|
126
177
|
|
|
127
178
|
```text
|
|
128
|
-
|
|
129
|
-
/quality wizard
|
|
130
|
-
/quality evaluate
|
|
179
|
+
assessment -> findings -> rating result
|
|
131
180
|
```
|
|
132
181
|
|
|
133
|
-
`setup` and `wizard` verify that the `qualitymd` CLI is installed and compatible
|
|
134
|
-
with the skill. Released skill installs use the CLI SemVer range declared by
|
|
135
|
-
that skill release. If the CLI is missing or stale, they stop and help you
|
|
136
|
-
install or upgrade it before continuing. See [`install.md`](install.md) for the
|
|
137
|
-
full bootstrap flow.
|
|
138
|
-
|
|
139
182
|
## The CLI
|
|
140
183
|
|
|
141
184
|
> **The CLI is an early work in progress.** Today the binary ships
|
|
142
|
-
> `qualitymd init`, `qualitymd lint`, `qualitymd spec`, and the
|
|
185
|
+
> `qualitymd init`, `qualitymd lint`, `qualitymd spec`, `qualitymd status`, and the
|
|
143
186
|
> `qualitymd evaluation` run-record surface.
|
|
144
187
|
|
|
145
188
|
`qualitymd` draws one hard line: the **CLI is deterministic and never calls a
|
|
146
|
-
model
|
|
189
|
+
model.** It scaffolds and validates a `QUALITY.md`, resolves target nodes and
|
|
147
190
|
their `source` manifests, records evaluation artifacts, renders reports, and
|
|
148
|
-
gates CI
|
|
149
|
-
**skills
|
|
150
|
-
`assessment` against the model.
|
|
191
|
+
gates CI. The deep, judgment-based evaluation of a subject against its model is
|
|
192
|
+
carried by **skills**, not by any CLI command.
|
|
151
193
|
|
|
152
|
-
The deterministic
|
|
194
|
+
The deterministic surface:
|
|
153
195
|
|
|
154
196
|
- **`qualitymd init`** — scaffold a starter `QUALITY.md` to fill in.
|
|
155
197
|
- **`qualitymd lint`** — validate a file's structure, fast and deterministic,
|
|
156
198
|
exiting non-zero on errors so it drops into CI.
|
|
157
199
|
- **`qualitymd spec`** — emit the bundled `QUALITY.md` format specification.
|
|
200
|
+
- **`qualitymd status`** — emit a deterministic project-state snapshot for
|
|
201
|
+
routing, automation, and agent use.
|
|
202
|
+
- **`qualitymd version`** — show structured CLI and bundled specification
|
|
203
|
+
version metadata.
|
|
204
|
+
- **`qualitymd upgrade --check`** — explicitly check the current install method,
|
|
205
|
+
latest version, and recommended upgrade action.
|
|
158
206
|
- **`qualitymd evaluation create-run`** — create and number an evaluation run
|
|
159
207
|
folder.
|
|
160
208
|
- **`qualitymd evaluation add-record`** — write assessment, analysis, and
|
|
@@ -163,50 +211,9 @@ The deterministic CLI:
|
|
|
163
211
|
assessment and analysis coverage for resume diagnostics.
|
|
164
212
|
- **`qualitymd evaluation show-status`** — inspect whether a run is ready to
|
|
165
213
|
render.
|
|
166
|
-
- **`qualitymd evaluation build-report`** — derive `report.md
|
|
167
|
-
and optionally gate with
|
|
168
|
-
|
|
169
|
-
The deep, judgment-based evaluation of a subject against its model is carried by
|
|
170
|
-
**skills** that orchestrate those resources — not by a CLI command.
|
|
171
|
-
|
|
172
|
-
## Install
|
|
173
|
-
|
|
174
|
-
> **Status.** The format spec is settled — see
|
|
175
|
-
> [`SPECIFICATION.md`](SPECIFICATION.md) — but implementation is in progress.
|
|
176
|
-
> The documented CLI surface includes **`init`**, **`lint`**, **`spec`**, and
|
|
177
|
-
> **`evaluation`**.
|
|
178
|
-
|
|
179
|
-
Install the `/quality` skill with Agent Skills tooling:
|
|
180
|
-
|
|
181
|
-
```sh
|
|
182
|
-
npx skills add qualitymd/quality.md
|
|
183
|
-
```
|
|
184
|
-
|
|
185
|
-
Install the `qualitymd` CLI with a pre-built binary — via npm:
|
|
186
|
-
|
|
187
|
-
```sh
|
|
188
|
-
npm install -g quality.md
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
or Homebrew:
|
|
192
|
-
|
|
193
|
-
```sh
|
|
194
|
-
brew install qualitymd/tap/qualitymd
|
|
195
|
-
```
|
|
196
|
-
|
|
197
|
-
Or build from source with Go 1.26+:
|
|
198
|
-
|
|
199
|
-
```sh
|
|
200
|
-
go install github.com/qualitymd/quality.md/cmd/qualitymd@latest
|
|
201
|
-
```
|
|
202
|
-
|
|
203
|
-
## Specification
|
|
204
|
-
|
|
205
|
-
The `QUALITY.md` format is specified in [`SPECIFICATION.md`](SPECIFICATION.md),
|
|
206
|
-
the source of truth for the format. The `qualitymd` CLI, `/quality` skill, and
|
|
207
|
-
format specification are versioned separately; see
|
|
208
|
-
[`docs/reference/versioning.md`](docs/reference/versioning.md) for the
|
|
209
|
-
compatibility policy.
|
|
214
|
+
- **`qualitymd evaluation build-report`** — derive `report-summary.md`,
|
|
215
|
+
`report.md`, and `report.json`, and optionally gate with
|
|
216
|
+
`--fail-at-or-below`.
|
|
210
217
|
|
|
211
218
|
## Conceptual model
|
|
212
219
|
|
|
@@ -218,6 +225,10 @@ ideas and vocabulary where they help and diverges where they don't (it uses
|
|
|
218
225
|
*Factors* where ISO says *characteristics*), optimizing first for a practical,
|
|
219
226
|
readable format.
|
|
220
227
|
|
|
228
|
+
## Status
|
|
229
|
+
|
|
230
|
+
The `QUALITY.md` format, `qualitymd` CLI, and `/quality` skill are early and under active development. The format is specified in [`SPECIFICATION.md`](SPECIFICATION.md). Expect the format and tooling to change as they mature.
|
|
231
|
+
|
|
221
232
|
## Contributing
|
|
222
233
|
|
|
223
234
|
Contributor setup and local tasks live in [`CONTRIBUTING.md`](CONTRIBUTING.md).
|
package/bin/qualitymd.js
CHANGED
|
@@ -24,7 +24,8 @@ try {
|
|
|
24
24
|
process.exit(1);
|
|
25
25
|
}
|
|
26
26
|
|
|
27
|
-
const
|
|
27
|
+
const env = { ...process.env, QUALITYMD_INSTALL_METHOD: "npm" };
|
|
28
|
+
const result = spawnSync(binary, process.argv.slice(2), { stdio: "inherit", env });
|
|
28
29
|
|
|
29
30
|
if (result.error) {
|
|
30
31
|
throw result.error;
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "quality.md",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "Companion CLI for the QUALITY.md file format and /quality agent skill, used to evaluate and improve AI assistant projects and harnesses.",
|
|
5
5
|
"homepage": "https://getquality.md",
|
|
6
6
|
"keywords": [
|
|
@@ -34,12 +34,12 @@
|
|
|
34
34
|
"bin"
|
|
35
35
|
],
|
|
36
36
|
"optionalDependencies": {
|
|
37
|
-
"@qualitymd/cli-darwin-arm64": "0.
|
|
38
|
-
"@qualitymd/cli-darwin-x64": "0.
|
|
39
|
-
"@qualitymd/cli-linux-arm64": "0.
|
|
40
|
-
"@qualitymd/cli-linux-x64": "0.
|
|
41
|
-
"@qualitymd/cli-win32-arm64": "0.
|
|
42
|
-
"@qualitymd/cli-win32-x64": "0.
|
|
37
|
+
"@qualitymd/cli-darwin-arm64": "0.3.0",
|
|
38
|
+
"@qualitymd/cli-darwin-x64": "0.3.0",
|
|
39
|
+
"@qualitymd/cli-linux-arm64": "0.3.0",
|
|
40
|
+
"@qualitymd/cli-linux-x64": "0.3.0",
|
|
41
|
+
"@qualitymd/cli-win32-arm64": "0.3.0",
|
|
42
|
+
"@qualitymd/cli-win32-x64": "0.3.0"
|
|
43
43
|
},
|
|
44
44
|
"publishConfig": {
|
|
45
45
|
"access": "public"
|