quality.md 0.2.0 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +218 -7
- package/package.json +28 -10
package/README.md
CHANGED
|
@@ -1,14 +1,225 @@
|
|
|
1
1
|
# QUALITY.md
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**QUALITY.md** is an agent-friendly file format and companion agent skill and
|
|
4
|
+
CLI for continuously improving the quality of coding agent and AI assistant projects/harnesses.
|
|
5
|
+
|
|
6
|
+
## Install
|
|
7
|
+
|
|
8
|
+
1. Install the agent skill:
|
|
4
9
|
|
|
5
10
|
```sh
|
|
6
|
-
npx quality.md
|
|
11
|
+
npx skills add qualitymd/quality.md
|
|
7
12
|
```
|
|
8
13
|
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
14
|
+
2. Install the CLI:
|
|
15
|
+
|
|
16
|
+
```sh
|
|
17
|
+
npm install quality.md -g
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## Usage
|
|
21
|
+
|
|
22
|
+
Invoke the `/quality` skill to manage quality for your project:
|
|
23
|
+
|
|
24
|
+
```text
|
|
25
|
+
/quality setup Get started working with QUALITY.md
|
|
26
|
+
/quality wizard Have your AI assistant/agent help you manage quality
|
|
27
|
+
/quality evaluate Evaluate the quality of your project
|
|
28
|
+
/quality evaluate security Evaluate a specific quality factor or characteristic
|
|
29
|
+
/quality evaluate payments-api Evaluate specific target or project component
|
|
30
|
+
/quality evaluate payments-api maintainability Evaluate a target's specific quality
|
|
31
|
+
```
|
|
32
|
+
|
|
33
|
+
## The Format
|
|
34
|
+
|
|
35
|
+
A `QUALITY.md` file combines a structured quality model with plain-language
|
|
36
|
+
rationale. The structured model names what is being evaluated, which quality
|
|
37
|
+
factors matter, what requirements define those factors, and how each requirement
|
|
38
|
+
should be assessed. The Markdown body explains the context: what the work is,
|
|
39
|
+
what "good" means, where the boundaries are, and why these standards matter.
|
|
40
|
+
|
|
41
|
+
### Specification
|
|
42
|
+
|
|
43
|
+
The format is specified in [`SPECIFICATION.md`](SPECIFICATION.md).
|
|
44
|
+
|
|
45
|
+
### Example QUALITY.md
|
|
46
|
+
|
|
47
|
+
```markdown
|
|
48
|
+
---
|
|
49
|
+
title: Support Inbox
|
|
50
|
+
ratingScale:
|
|
51
|
+
- level: outstanding
|
|
52
|
+
title: Outstanding
|
|
53
|
+
description: The work clearly exceeds the shared quality bar.
|
|
54
|
+
criterion: "Consistently exceeds the requirement with clear margin."
|
|
55
|
+
- level: target
|
|
56
|
+
title: Target
|
|
57
|
+
description: The work meets the shared quality bar.
|
|
58
|
+
criterion: "Meets the expected quality bar."
|
|
59
|
+
- level: minimum
|
|
60
|
+
title: Minimum
|
|
61
|
+
description: The work is acceptable, but has gaps worth improving.
|
|
62
|
+
criterion: "Meets the lowest acceptable bar, with visible gaps."
|
|
63
|
+
- level: unacceptable
|
|
64
|
+
title: Unacceptable
|
|
65
|
+
description: The work is below the shared quality bar.
|
|
66
|
+
criterion: "Falls below the minimum acceptable bar."
|
|
67
|
+
targets:
|
|
68
|
+
triage:
|
|
69
|
+
source: ./support
|
|
70
|
+
factors:
|
|
71
|
+
responsiveness:
|
|
72
|
+
description: Customers receive timely, useful attention.
|
|
73
|
+
requirements:
|
|
74
|
+
"urgent messages are visible":
|
|
75
|
+
assessment: >
|
|
76
|
+
New messages are classified so urgent customer-impacting issues
|
|
77
|
+
are separated from routine requests.
|
|
78
|
+
accuracy:
|
|
79
|
+
description: Replies are correct, complete, and grounded in policy.
|
|
80
|
+
requirements:
|
|
81
|
+
"answers cite the current policy":
|
|
82
|
+
assessment: >
|
|
83
|
+
Customer-facing replies use the active support policy and do not
|
|
84
|
+
rely on outdated guidance or unsupported assumptions.
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
# Quality model: Support Inbox
|
|
88
|
+
|
|
89
|
+
## Overview
|
|
90
|
+
|
|
91
|
+
This model describes the quality bar for daily support triage. Good support
|
|
92
|
+
means urgent issues are easy to see, routine requests still move, and customers
|
|
93
|
+
receive answers grounded in the current policy.
|
|
94
|
+
|
|
95
|
+
## Scope
|
|
96
|
+
|
|
97
|
+
This model covers message triage and written replies in the support workspace.
|
|
98
|
+
It does not cover billing system behavior or product incident response.
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
An agent that reads this file can evaluate support work against the stated
|
|
102
|
+
requirements, produce findings, and rate the results against the model's scale.
|
|
103
|
+
|
|
104
|
+
## The Specification
|
|
105
|
+
|
|
106
|
+
The full format specification lives at [`SPECIFICATION.md`](SPECIFICATION.md).
|
|
107
|
+
What follows is a condensed reference.
|
|
108
|
+
|
|
109
|
+
### File Structure
|
|
110
|
+
|
|
111
|
+
A `QUALITY.md` file has two layers:
|
|
112
|
+
|
|
113
|
+
1. **YAML frontmatter** - the structured quality model.
|
|
114
|
+
2. **Markdown body** - the context, rationale, scope, needs, risks, and known
|
|
115
|
+
gaps that help people and agents interpret the model.
|
|
116
|
+
|
|
117
|
+
The document begins with the YAML frontmatter. The Markdown body can be empty,
|
|
118
|
+
but it is where the model explains itself.
|
|
119
|
+
|
|
120
|
+
### Model Schema
|
|
121
|
+
|
|
122
|
+
The root model is a target plus a model-wide `ratingScale`.
|
|
123
|
+
|
|
124
|
+
```yaml
|
|
125
|
+
title: <string> # Recommended
|
|
126
|
+
description: <string> # Optional
|
|
127
|
+
ratingScale: # Required, ordered best to worst
|
|
128
|
+
- level: <level-name> # Required, unique within the scale
|
|
129
|
+
title: <string> # Optional
|
|
130
|
+
description: <string> # Recommended
|
|
131
|
+
criterion: <string> # Required
|
|
132
|
+
source: <string> # Optional
|
|
133
|
+
factors: # Optional*
|
|
134
|
+
<factor-name>:
|
|
135
|
+
description: <string> # Recommended
|
|
136
|
+
factors: # Optional
|
|
137
|
+
<sub-factor-name>: <Factor>
|
|
138
|
+
requirements: # Optional
|
|
139
|
+
<requirement-statement>: <Requirement>
|
|
140
|
+
requirements: # Optional*
|
|
141
|
+
<requirement-statement>:
|
|
142
|
+
assessment: <string> # Required, exactly one
|
|
143
|
+
factors: [<factor-name>] # Required for direct target requirements
|
|
144
|
+
ratings: # Optional per-level criteria
|
|
145
|
+
<level-name>: <criterion>
|
|
146
|
+
targets: # Optional*
|
|
147
|
+
<target-name>: <Target>
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
At least one of `factors`, `requirements`, or `targets` must be supplied.
|
|
151
|
+
Targets can nest recursively. `ratingScale` exists only on the root model.
|
|
152
|
+
|
|
153
|
+
### Core Concepts
|
|
154
|
+
|
|
155
|
+
| Concept | Meaning |
|
|
156
|
+
| ------------ | --------------------------------------------------------------- |
|
|
157
|
+
| Model | The root quality model in a `QUALITY.md` file. |
|
|
158
|
+
| Target | The thing being evaluated. |
|
|
159
|
+
| Source | The material assessed for a target, such as a path or selector. |
|
|
160
|
+
| Factor | A quality dimension that matters for a target. |
|
|
161
|
+
| Requirement | A specific quality expectation. |
|
|
162
|
+
| Assessment | The means of checking a requirement against a target source. |
|
|
163
|
+
| Finding | An observation produced by an assessment. |
|
|
164
|
+
| Rating Scale | The ordered model-wide scale used to rate results. |
|
|
165
|
+
|
|
166
|
+
### Evaluation Semantics
|
|
167
|
+
|
|
168
|
+
Each requirement has exactly one `assessment`. An evaluator performs that
|
|
169
|
+
assessment against the target source, records findings, and rates the
|
|
170
|
+
requirement's findings against the model's `ratingScale`. A finding records what
|
|
171
|
+
was observed; the rating result records how that observation compares with the
|
|
172
|
+
model's scale.
|
|
173
|
+
|
|
174
|
+
```text
|
|
175
|
+
assessment -> findings -> rating result
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## The CLI
|
|
179
|
+
|
|
180
|
+
> **The CLI is an early work in progress.** Today the binary ships
|
|
181
|
+
> `qualitymd init`, `qualitymd lint`, `qualitymd spec`, and the
|
|
182
|
+
> `qualitymd evaluation` run-record surface.
|
|
183
|
+
|
|
184
|
+
`qualitymd` draws one hard line: the **CLI is deterministic and never calls a
|
|
185
|
+
model.** It scaffolds and validates a `QUALITY.md`, resolves target nodes and
|
|
186
|
+
their `source` manifests, records evaluation artifacts, renders reports, and
|
|
187
|
+
gates CI. The deep, judgment-based evaluation of a subject against its model is
|
|
188
|
+
carried by **skills**, not by any CLI command.
|
|
189
|
+
|
|
190
|
+
The deterministic surface:
|
|
191
|
+
|
|
192
|
+
- **`qualitymd init`** — scaffold a starter `QUALITY.md` to fill in.
|
|
193
|
+
- **`qualitymd lint`** — validate a file's structure, fast and deterministic,
|
|
194
|
+
exiting non-zero on errors so it drops into CI.
|
|
195
|
+
- **`qualitymd spec`** — emit the bundled `QUALITY.md` format specification.
|
|
196
|
+
- **`qualitymd evaluation create-run`** — create and number an evaluation run
|
|
197
|
+
folder.
|
|
198
|
+
- **`qualitymd evaluation add-record`** — write assessment, analysis, and
|
|
199
|
+
recommendation records from judgment payloads.
|
|
200
|
+
- **`qualitymd evaluation set-planned-coverage`** — write optional planned
|
|
201
|
+
assessment and analysis coverage for resume diagnostics.
|
|
202
|
+
- **`qualitymd evaluation show-status`** — inspect whether a run is ready to
|
|
203
|
+
render.
|
|
204
|
+
- **`qualitymd evaluation build-report`** — derive `report.md` / `report.json`
|
|
205
|
+
and optionally gate with `--fail-at-or-below`.
|
|
206
|
+
|
|
207
|
+
## Conceptual model
|
|
208
|
+
|
|
209
|
+
The way `QUALITY.md` frames quality is informed by the **ISO/IEC 25000 (SQuaRE)**
|
|
210
|
+
family of software-quality standards — particularly ISO/IEC 25010 — and, for the
|
|
211
|
+
shape of a well-formed requirement, **ISO/IEC/IEEE 29148**. We acknowledge these
|
|
212
|
+
as the conceptual lineage, not a conformance target: `QUALITY.md` borrows their
|
|
213
|
+
ideas and vocabulary where they help and diverges where they don't (it uses
|
|
214
|
+
*Factors* where ISO says *characteristics*), optimizing first for a practical,
|
|
215
|
+
readable format.
|
|
216
|
+
|
|
217
|
+
## Status
|
|
218
|
+
|
|
219
|
+
The `QUALITY.md` format, `qualitymd` CLI, and `/quality` skill are early and under active development. The format is specified in [`SPECIFICATION.md`](SPECIFICATION.md). Expect the format and tooling to change as they mature.
|
|
220
|
+
|
|
221
|
+
## Contributing
|
|
12
222
|
|
|
13
|
-
|
|
14
|
-
|
|
223
|
+
Contributor setup and local tasks live in [`CONTRIBUTING.md`](CONTRIBUTING.md).
|
|
224
|
+
The release runbook lives in
|
|
225
|
+
[`docs/guides/cut-a-release.md`](docs/guides/cut-a-release.md).
|
package/package.json
CHANGED
|
@@ -1,13 +1,31 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "quality.md",
|
|
3
|
-
"version": "0.2.
|
|
4
|
-
"description": "
|
|
5
|
-
"homepage": "https://
|
|
3
|
+
"version": "0.2.2",
|
|
4
|
+
"description": "Companion CLI for the QUALITY.md file format and /quality agent skill, used to evaluate and improve AI assistant projects and harnesses.",
|
|
5
|
+
"homepage": "https://getquality.md",
|
|
6
|
+
"keywords": [
|
|
7
|
+
"quality",
|
|
8
|
+
"qualitymd",
|
|
9
|
+
"quality.md",
|
|
10
|
+
"quality-model",
|
|
11
|
+
"quality-evaluation",
|
|
12
|
+
"quality-improvement",
|
|
13
|
+
"context-engineering",
|
|
14
|
+
"harness-engineering",
|
|
15
|
+
"cli"
|
|
16
|
+
],
|
|
6
17
|
"repository": {
|
|
7
18
|
"type": "git",
|
|
8
|
-
"url": "https://github.com/qualitymd/quality.md.git"
|
|
19
|
+
"url": "https://github.com/qualitymd/quality.md.git",
|
|
20
|
+
"directory": "npm/quality.md"
|
|
21
|
+
},
|
|
22
|
+
"bugs": {
|
|
23
|
+
"url": "https://github.com/qualitymd/quality.md/issues"
|
|
9
24
|
},
|
|
10
25
|
"license": "MIT",
|
|
26
|
+
"scripts": {
|
|
27
|
+
"prepack": "node ../../scripts/sync-npm-readme.mjs"
|
|
28
|
+
},
|
|
11
29
|
"bin": {
|
|
12
30
|
"qualitymd": "bin/qualitymd.js",
|
|
13
31
|
"quality.md": "bin/qualitymd.js"
|
|
@@ -16,12 +34,12 @@
|
|
|
16
34
|
"bin"
|
|
17
35
|
],
|
|
18
36
|
"optionalDependencies": {
|
|
19
|
-
"@qualitymd/cli-darwin-arm64": "0.2.
|
|
20
|
-
"@qualitymd/cli-darwin-x64": "0.2.
|
|
21
|
-
"@qualitymd/cli-linux-arm64": "0.2.
|
|
22
|
-
"@qualitymd/cli-linux-x64": "0.2.
|
|
23
|
-
"@qualitymd/cli-win32-arm64": "0.2.
|
|
24
|
-
"@qualitymd/cli-win32-x64": "0.2.
|
|
37
|
+
"@qualitymd/cli-darwin-arm64": "0.2.2",
|
|
38
|
+
"@qualitymd/cli-darwin-x64": "0.2.2",
|
|
39
|
+
"@qualitymd/cli-linux-arm64": "0.2.2",
|
|
40
|
+
"@qualitymd/cli-linux-x64": "0.2.2",
|
|
41
|
+
"@qualitymd/cli-win32-arm64": "0.2.2",
|
|
42
|
+
"@qualitymd/cli-win32-x64": "0.2.2"
|
|
25
43
|
},
|
|
26
44
|
"publishConfig": {
|
|
27
45
|
"access": "public"
|