docattest 0.1.1 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/dist/agents/SKILL.md +136 -0
- package/dist/agents/agents-section.md +18 -0
- package/dist/core.json +1 -1
- package/package.json +6 -6
package/README.md
CHANGED
|
@@ -20,8 +20,8 @@ FAILED
|
|
|
20
20
|
```
|
|
21
21
|
|
|
22
22
|
This is the npm package. The same tool for Python is the PyPI package `docattest`, and
|
|
23
|
-
for Go a
|
|
24
|
-
([adapters/rust](../rust/README.md)); all run one shared core. The project's front page is the
|
|
23
|
+
for Go a module that `go install` brings ([adapters/go](../go/README.md)), and for Rust the
|
|
24
|
+
crate `docattest` ([adapters/rust](../rust/README.md)); all run one shared core. The project's front page is the
|
|
25
25
|
[repository README](../../README.md), and the full documentation is in
|
|
26
26
|
[docs/](../../docs/index.md).
|
|
27
27
|
|
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: docattest
|
|
3
|
+
description: Use when writing or editing documentation, or when changing, renaming or removing an exported symbol, in a repository that has a docattest.config.json. docattest makes pages carry their own proof (examples that run, facts checked against the code, tables whose rows run) and maps every symbol back to the pages that describe it.
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# docattest: attested documentation
|
|
7
|
+
|
|
8
|
+
In this repository the docs are tests. A page states behaviour and proves it in the same
|
|
9
|
+
section; `docattest check` runs the proofs and fails at the page and line where the
|
|
10
|
+
prose stopped being true. Follow this skill whenever you touch a page or an exported symbol.
|
|
11
|
+
|
|
12
|
+
Run `docattest` the way this project runs its tools: `npx docattest` where it was installed
|
|
13
|
+
from npm, `docattest` from the project's virtualenv where it was installed with pip; when
|
|
14
|
+
the project's `AGENTS.md` has a docattest section, it gives the exact command. The commands
|
|
15
|
+
below are written bare, and work from any directory inside the project.
|
|
16
|
+
|
|
17
|
+
Two commands matter whatever else you do: after editing source code, run
|
|
18
|
+
`docattest impact --file <the file you edited>` and read every line it prints (it prints
|
|
19
|
+
nothing when your change touches no documented export); before you finish, run
|
|
20
|
+
`docattest check`.
|
|
21
|
+
|
|
22
|
+
## The one rule
|
|
23
|
+
|
|
24
|
+
Follow every sentence that states behaviour with its proof, in the same section, within a
|
|
25
|
+
few lines. Never write a number, name, list or shape that comes from the code as plain
|
|
26
|
+
text: write it as a fact.
|
|
27
|
+
|
|
28
|
+
## Choose the proof
|
|
29
|
+
|
|
30
|
+
| The sentence says… | Write… |
|
|
31
|
+
|-------------------------------------------------|---------------------------------------------------|
|
|
32
|
+
| "Calling X with Y does Z" | a fenced block tagged `check` that asserts Z |
|
|
33
|
+
| "X is 3", "defaults to `eu`", "exports A, B, C" | a fact bound to the literal or list |
|
|
34
|
+
| "Rule 1… Rule 2… Rule 3…" over one operation | a `<!-- cases: runner -->` table, one row per rule|
|
|
35
|
+
| "The service accepts these regions" | a fact over `snapshot('name')` |
|
|
36
|
+
| "Needs Node 22.12 or later" (a manifest says so) | a fact over `manifest('package.json')` |
|
|
37
|
+
| "Running `tool x` prints…" (a command's output) | a `console check` block: `$ tool x`, then its output |
|
|
38
|
+
| a sample page the docs show a user writing | an `md check fixture=<dir>` block |
|
|
39
|
+
| a behaviour an existing test already proves | `<!-- test: file > "title" -->` after the sentence |
|
|
40
|
+
| "We chose this because…" | prose under `## Why`, no proof |
|
|
41
|
+
|
|
42
|
+
Directives and commands are the same in every language. What is written in the project's
|
|
43
|
+
language is the code: examples, fact expressions and the setup module. `"adapter"` in
|
|
44
|
+
`docattest.config.json` names the language.
|
|
45
|
+
|
|
46
|
+
| | `"adapter": "typescript"` | `"adapter": "python"` | `"adapter": "go"` | `"adapter": "rust"` |
|
|
47
|
+
|---|---|---|---|---|
|
|
48
|
+
| a block that runs | ```` ```ts check ```` | ```` ```python check ````, or ```` ```pycon check ```` for a `>>>` session | ```` ```go check ````: statements, declarations, or a whole `package main` | ```` ```rust check ````: the body of `main`, or a whole program with `fn main` |
|
|
49
|
+
| a block that is only checked, not run | ```` ```ts compile ```` (type-checked) | ```` ```python compile ```` (syntax, imports, names) | ```` ```go compile ```` (compiled) | ```` ```rust compile ```` (compiled) |
|
|
50
|
+
| an assertion | `assert.equal(a, b)`; `assert` is in scope | `assert a == b`; in a session, the output shown | a last comment `// Output: …` that what it prints must match, or `if a != b { panic(…) }` | `assert_eq!(a, b)`, `assert!(…)`, or an `unwrap()` |
|
|
51
|
+
| a fact expression | `RETRY.max`, `errors.map((e) => e.field)` | `RETRY["max"]`, `[e.field for e in errors]` | `Retry.Max`, `len(Regions)`: the surface's exports by their own names | `RETRY.max`, `REGIONS.len()`: the exports of the surface's module by their own names |
|
|
52
|
+
| how a value is written on the page | as JavaScript prints it: `true`, `3` | as Python prints it: `True`, `3` | as `fmt.Sprint` prints it: `true`, `3`, `[a b]` | as `Display` prints it, else `Debug`: `true`, `3`, `Some(3)` |
|
|
53
|
+
| setup above the reader's view | lines above `// ---cut---` | none: put it in an earlier block and tag the next `continue` | none: a block is a whole program | lines that begin `# `, as in a doc test |
|
|
54
|
+
| examples in source files | `@example` in a doc comment | `>>>` prompts in a docstring | `Example…` functions in a `_test.go` file | fenced blocks in `///` doc comments |
|
|
55
|
+
| a test to cite | `it("title")`, inside `describe` blocks | a `test_…` function, inside classes | a `Test…` function, or a `t.Run("title", …)` inside one | a `#[test]` function, inside its `mod tests` |
|
|
56
|
+
| the setup module | `export const cases`, `export const facts` | `cases = {…}`, `facts = {…}` | a package with `var Cases = map[string]func(map[string]string) error{…}`; facts use its exports | a file with `pub const CASES: &[(&str, fn(&HashMap<String, String>) -> Result<(), String>)]`; facts use its `pub` items |
|
|
57
|
+
|
|
58
|
+
Syntax:
|
|
59
|
+
|
|
60
|
+
- Example: a fenced block in the project's language tagged `check` (it runs) or `compile`
|
|
61
|
+
(it is checked without running). Import what a user would import. One behaviour per
|
|
62
|
+
block, at least one assertion. Setup the reader need not see must never perform the
|
|
63
|
+
behaviour the section describes.
|
|
64
|
+
- Scalar fact: put the comment right after the literal: ``At most `3` tries. <!-- fact: RETRY.max -->``
|
|
65
|
+
It binds the nearest code span or bare number before it on its own source line, not in
|
|
66
|
+
its sentence. A comment alone on a line, or pushed onto the next line when the paragraph
|
|
67
|
+
is wrapped, fails with "has no target": keep the literal and the comment on one line.
|
|
68
|
+
- List fact: end the line before the list with the comment: `The server reads: <!-- fact: ENV_KEYS -->`
|
|
69
|
+
- Table column: `<!-- fact column=Field: <expression giving the column> -->` before the table.
|
|
70
|
+
- Fenced block: the comment ends the line before the block; whitespace is ignored.
|
|
71
|
+
- Named facts: when an expression is awkward inline, add a helper to the setup module's
|
|
72
|
+
`facts` and call it by bare name. The helper reads the surface; the expectation stays on
|
|
73
|
+
the page.
|
|
74
|
+
- Commands: a block tagged `console check` holds lines `$ command`, each followed by what it
|
|
75
|
+
prints (stdout, then stderr). `...` on a line matches any lines; a last line `[exit N]`
|
|
76
|
+
says the command fails with `N`. Commands run with `sh -c` in the project root, or in
|
|
77
|
+
`cwd=<dir>`; `docattest` in one is the docattest checking the page.
|
|
78
|
+
- Sample pages: a block tagged `md check fixture=<dir>` is checked as a page (at
|
|
79
|
+
`docs/sample.md`, or `as=<path>`) in a copy of the project at `<dir>`; every proof on it
|
|
80
|
+
must pass.
|
|
81
|
+
- Cases: `<!-- cases: runnerName -->` before a table. The first column names the case; the
|
|
82
|
+
other columns are what the runner needs. Rows hold the expectations, never the runner.
|
|
83
|
+
|
|
84
|
+
## Workflow when you change code
|
|
85
|
+
|
|
86
|
+
1. Before changing an exported symbol, find the prose it affects and read those pages:
|
|
87
|
+
|
|
88
|
+
```sh
|
|
89
|
+
docattest where <symbol>
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
2. Make the change.
|
|
93
|
+
3. Check each page `where` listed, then everything:
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
docattest check docs/the-page.md
|
|
97
|
+
docattest check --json
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
4. Every failure is a sentence that is now false, with expected versus actual. Fix the
|
|
101
|
+
prose where the behaviour changed on purpose; fix the code where it did not.
|
|
102
|
+
5. If you added a public symbol, `docattest uncovered --surface` lists it. Document it
|
|
103
|
+
with a proof, or say in the PR why it stays undocumented.
|
|
104
|
+
6. After any source edit, including to internals, `docattest impact --file <file>` lists
|
|
105
|
+
the prose your diff touches (`docattest impact` does so without asking about one file).
|
|
106
|
+
Entries marked as mentions are unproven: read them, they go stale first.
|
|
107
|
+
|
|
108
|
+
## Workflow when you write or edit a page
|
|
109
|
+
|
|
110
|
+
1. Write the claim sentence, then its proof right after it.
|
|
111
|
+
2. Run `docattest check <page>` and read every `fail` and `error`.
|
|
112
|
+
3. Fix the prose or the proof until the page passes, then run `docattest check`.
|
|
113
|
+
4. `docattest audit` reports lists, tables, blocks and values in sentences on the pages that
|
|
114
|
+
look copied from the code and carry no fact. Add the suggested directive; where it reports drift, fix the page.
|
|
115
|
+
|
|
116
|
+
## Never make the build green this way
|
|
117
|
+
|
|
118
|
+
These three make the build pass while making the docs wrong:
|
|
119
|
+
|
|
120
|
+
- weakening an assertion or a fact expression so it asserts less than the prose says;
|
|
121
|
+
- deleting an attestation;
|
|
122
|
+
- freezing a page (config `frozen` or frontmatter `docattest: false`).
|
|
123
|
+
|
|
124
|
+
CI may run `docattest check --base <ref>`, which fails when a claim is kept and its proof
|
|
125
|
+
removed, and reports the other two.
|
|
126
|
+
|
|
127
|
+
## Checklist before a page is done (run it literally)
|
|
128
|
+
|
|
129
|
+
1. `docattest check <page> --json` reports no `fail` and no `error`.
|
|
130
|
+
2. Every section after the orientation paragraph has at least one attestation, or is `## Why`.
|
|
131
|
+
3. No bare number, name, list or type from the code appears without a fact.
|
|
132
|
+
4. Every `check` example contains at least one assertion.
|
|
133
|
+
5. No hidden setup (above a `// ---cut---`, or in an earlier block of a session) performs
|
|
134
|
+
the behaviour the section describes.
|
|
135
|
+
6. External facts reference a snapshot, and the prose says it is a snapshot and when it was taken.
|
|
136
|
+
7. The page's coverage is not worse than before the edit.
|
|
@@ -0,0 +1,18 @@
|
|
|
1
|
+
## Documentation that is checked (docattest)
|
|
2
|
+
|
|
3
|
+
The documentation of this project carries its own proof: examples that run, values read
|
|
4
|
+
from the code, tables whose rows run. `{docattest} check` runs every proof and fails at the
|
|
5
|
+
page and line that is no longer true. Whatever agent you are, do these two things, from the
|
|
6
|
+
directory this file is in:
|
|
7
|
+
|
|
8
|
+
- **After you edit source code**, run `{docattest} impact --file <path>` with the path of
|
|
9
|
+
the file you edited. It prints the documented exports your change affects and every page
|
|
10
|
+
and line that describes them, and prints nothing when there are none. Read the lines it
|
|
11
|
+
lists and bring them up to date with the code.
|
|
12
|
+
- **Before you finish**, run `{docattest} check`. If it fails, make the page true again or
|
|
13
|
+
fix the code. Never delete or weaken a claim to make it pass.
|
|
14
|
+
|
|
15
|
+
`{docattest} where <symbol>` lists the pages that describe an export.
|
|
16
|
+
|
|
17
|
+
This section is written by `{docattest} agents install`, which replaces everything between
|
|
18
|
+
the two markers. Write your own notes outside them.
|
package/dist/core.json
CHANGED
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "docattest",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.2",
|
|
4
4
|
"description": "Documentation that CI can fail: pages carry their own proof, and code links back to the pages that describe it.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"documentation",
|
|
@@ -53,11 +53,11 @@
|
|
|
53
53
|
"prepublishOnly": "cd ../.. && node scripts/version.ts check && node scripts/release/check-published.ts && npm run typecheck && npm run lint && npm run format:check && npm run build && npm run build:core && npm test && npm run test:docs"
|
|
54
54
|
},
|
|
55
55
|
"optionalDependencies": {
|
|
56
|
-
"@docattest/core-darwin-arm64": "0.1.
|
|
57
|
-
"@docattest/core-darwin-x64": "0.1.
|
|
58
|
-
"@docattest/core-linux-arm64": "0.1.
|
|
59
|
-
"@docattest/core-linux-x64": "0.1.
|
|
60
|
-
"@docattest/core-windows-x64": "0.1.
|
|
56
|
+
"@docattest/core-darwin-arm64": "0.1.2",
|
|
57
|
+
"@docattest/core-darwin-x64": "0.1.2",
|
|
58
|
+
"@docattest/core-linux-arm64": "0.1.2",
|
|
59
|
+
"@docattest/core-linux-x64": "0.1.2",
|
|
60
|
+
"@docattest/core-windows-x64": "0.1.2"
|
|
61
61
|
},
|
|
62
62
|
"peerDependencies": {
|
|
63
63
|
"typescript": ">=5.4 <7"
|