incitaciones 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +5 -0
- package/content/distilled/documentation/references/implement-documentation/SKILL.md +4 -1
- package/content/distilled/documentation/references/implement-documentation/references/docs-accuracy.md +111 -0
- package/content/distilled/documentation/references/implement-documentation/references/templates.md +29 -0
- package/content/distilled/documentation/references/implement-documentation/references/visual-cross-repo.md +131 -0
- package/content/distilled/documentation/references/implement-documentation/references/worked-examples.md +83 -0
- package/content/distilled/documentation/references/review-documentation/SKILL.md +10 -2
- package/content/distilled/documentation/references/review-documentation/references/cold-reader.md +130 -0
- package/content/distilled/documentation/references/review-documentation/references/drift-audit.md +98 -0
- package/content/manifest.json +1 -1
- package/package.json +1 -1
- package/pi-package/skills/documentation/references/implement-documentation/SKILL.md +4 -1
- package/pi-package/skills/documentation/references/implement-documentation/references/docs-accuracy.md +111 -0
- package/pi-package/skills/documentation/references/implement-documentation/references/templates.md +29 -0
- package/pi-package/skills/documentation/references/implement-documentation/references/visual-cross-repo.md +131 -0
- package/pi-package/skills/documentation/references/implement-documentation/references/worked-examples.md +83 -0
- package/pi-package/skills/documentation/references/review-documentation/SKILL.md +10 -2
- package/pi-package/skills/documentation/references/review-documentation/references/cold-reader.md +130 -0
- package/pi-package/skills/documentation/references/review-documentation/references/drift-audit.md +98 -0
package/README.md
CHANGED
|
@@ -531,3 +531,8 @@ The LLM writes. The human knows what it should say.
|
|
|
531
531
|
There is a distinction, even if looking at the commit history makes it hard to find. The distinction is that a machine can produce correct code without understanding anything, the same way a calculator can solve an integral without knowing what time is. Understanding what that integral is *for*, whether it actually solves the problem, whether the problem was the right problem to begin with — that remains human territory. For now.
|
|
532
532
|
|
|
533
533
|
[Leer en español](https://charly-vibes.github.io/charly-vibes/)
|
|
534
|
+
|
|
535
|
+
## Documentation provenance
|
|
536
|
+
|
|
537
|
+
Rendered docs are generated artifacts; the repository sources are
|
|
538
|
+
authoritative — edit the repo sources, not the rendered output.
|
|
@@ -17,7 +17,10 @@ Write a high-signal documentation topic using the Unified Frameworks for Technic
|
|
|
17
17
|
2. EPPO initialization: every topic must stand alone. Start with a clear task- or concept-focused title and a context block ("This topic covers [X]. You should already understand [Y] and have [Z] installed.").
|
|
18
18
|
3. Information mapping: chunk the body into units under 7 items; label headers by result or content (e.g., "Configuring the API" not "Configuration"); convert complex comparisons or steps into tables and lists.
|
|
19
19
|
4. AI-first polish: ensure headers are descriptive for RAG indexing and provide a 1-2 sentence TL;DR at the top for LLM ingestion.
|
|
20
|
-
5. If
|
|
20
|
+
5. If the topic is a tutorial, quickstart, or introduces user-facing commands, use `references/worked-examples.md`: it defines the worked-example pattern (one evolving example thread, not disconnected snippets), the quickstart rule (every command shows its expected output and exit code; every install path ends with a post-install verification block), and the example coverage matrix (map every user-facing feature to at least one example page).
|
|
21
|
+
6. If setting up a new docs site or choosing tooling, use `references/tooling-comparison.md` for platform selection and `references/versioning.md` for versioning and changelog strategy. For a full Diátaxis rollout, use `references/diataxis-implementation.md`.
|
|
22
|
+
7. If the topic shows user-facing commands or the docs ship generated artifacts, use `references/docs-accuracy.md`: it defines the docs-accuracy gate (CI re-runs documented commands and diffs normalized captured output against the docs), the stamped-artifact rule (`llms.txt`, `release.md`, README version facts are generated per release, with a freshness check that fails on hand-edits), and the live-site smoke (CI curls the deployed URL so a 404 fails in the pipeline, not at the next human visit).
|
|
23
|
+
8. If the topic embeds diagrams or documents a multi-repo ecosystem, use `references/visual-cross-repo.md`: it defines the static-SVG rule (prefer pre-rendered, build-regenerated SVG over runtime mermaid on static-site books), the fleet pattern (one responsibility map per layer linking to each project's docs, a shared setup template, fleet-wide pages living in exactly one hub repo), and the deployed-vs-source smoke (after deploy, verify rendered behavior — JS errors, diagram rendering, search index — on top of the live-site smoke's reachability check).
|
|
21
24
|
|
|
22
25
|
## Rules
|
|
23
26
|
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# Executable Docs: Accuracy Gate, Stamped Artifacts & Live-Site Smoke
|
|
2
|
+
|
|
3
|
+
Methodology for making documentation machine-checked instead of read-checked.
|
|
4
|
+
Mined from the 2026-10 specodelic and ddl docs work; evidence in
|
|
5
|
+
`.wai/resources/reflections/docs-skill-survey-2026-10.org` (row `lf4b.10`/`.11`).
|
|
6
|
+
Canonical theory: `content/research-documentation-frameworks.md`.
|
|
7
|
+
|
|
8
|
+
The 2026-10 evidence: docs that only humans check drift silently. The fixes are
|
|
9
|
+
three CI gates — the docs-accuracy gate re-runs documented commands and diffs
|
|
10
|
+
their real output against the docs; stamped artifacts are regenerated per
|
|
11
|
+
release and lint-checked for hand-edits; a live-site smoke curls the deployed
|
|
12
|
+
URL so a 404'd book fails in CI, not at the next human visit.
|
|
13
|
+
|
|
14
|
+
## 1. The Docs-Accuracy Gate (CI re-runs the documented commands)
|
|
15
|
+
|
|
16
|
+
Specodelic `lf4b.10` built a gate that re-runs the quickstart commands and
|
|
17
|
+
diffs captured output against the documented snippets (in `docs/src`). The
|
|
18
|
+
quickstart rule from `worked-examples.md` makes commands claimable; the gate
|
|
19
|
+
makes those claims executable. A doc that says one thing and outputs another
|
|
20
|
+
fails the build instead of embarrassing the maintainer in an issue thread.
|
|
21
|
+
|
|
22
|
+
Gate shape:
|
|
23
|
+
|
|
24
|
+
1. Inventory the documented commands (the quickstart and how-to pages are
|
|
25
|
+
usually enough; every command in the example coverage matrix qualifies).
|
|
26
|
+
2. In CI, run each command, capture stdout/stderr and the exit code.
|
|
27
|
+
3. Normalize the capture (see capture-normalize below).
|
|
28
|
+
4. Diff the normalized capture against the documented snippet. Any drift —
|
|
29
|
+
including an output the docs no longer show — fails the build.
|
|
30
|
+
|
|
31
|
+
The capture-normalize technique (specodelic commit `a2108b8`): before diffing,
|
|
32
|
+
normalize tool chatter on **both sides with the same rules**. Real output is
|
|
33
|
+
full of volatility that is not part of the contract:
|
|
34
|
+
|
|
35
|
+
- tool verbosity lines (`cargo`, `gh`, compilers: progress, timings, emoji)
|
|
36
|
+
- content hashes, digests, object IDs
|
|
37
|
+
- timestamps, durations, download speeds
|
|
38
|
+
- absolute paths (home dir, temp dirs, machine names)
|
|
39
|
+
|
|
40
|
+
Normalize these to stable placeholders (e.g. `[TIME]`, `[HASH]`, `[PATH]`)
|
|
41
|
+
before the diff. Only normalized output is diffable — raw captures diff-fail
|
|
42
|
+
on every run for reasons the docs never claimed to specify. Document the
|
|
43
|
+
normalization rules in the docs pipeline itself, so a reader comparing their
|
|
44
|
+
terminal to the docs knows what was masked.
|
|
45
|
+
|
|
46
|
+
Rules:
|
|
47
|
+
|
|
48
|
+
- The gate runs on every PR touching docs **or** code that changes
|
|
49
|
+
command output — output drift is a code-change bug as often as a docs bug.
|
|
50
|
+
- The gate checks exit codes too, not just stdout: a command that starts
|
|
51
|
+
erroring is drift even when stdout looks plausible.
|
|
52
|
+
- Keep the gate fast by scoping it to documented commands only; it is a
|
|
53
|
+
contract check, not a full test suite.
|
|
54
|
+
|
|
55
|
+
## 2. Stamped Artifacts (generated per release, never hand-edited)
|
|
56
|
+
|
|
57
|
+
Specodelic `lf4b.11` made `release.md`, `llms.txt`, and similar release-facing
|
|
58
|
+
artifacts *stamped*: regenerated automatically at release time from the
|
|
59
|
+
repo state, never hand-edited. Facts that a generator can compute (version,
|
|
60
|
+
pack/module counts, release date, changelog digest) must not be maintained by
|
|
61
|
+
hand — hand-maintained facts are exactly the ones that drift (see the
|
|
62
|
+
pack-count drift fixes `lf4b.1`, `d8f97ce`, `fd2fbe0`).
|
|
63
|
+
|
|
64
|
+
Stamped artifacts include:
|
|
65
|
+
|
|
66
|
+
- `llms.txt` / `llm.txt` (AI-facing site indexes derived from the docs tree)
|
|
67
|
+
- `release.md` / release notes derived from tags and changelog
|
|
68
|
+
- README facts a generator owns: version numbers, counts, install shas
|
|
69
|
+
|
|
70
|
+
Freshness check: add a lint step that **regenerates (or re-derives) each
|
|
71
|
+
stamped artifact and fails on any diff**. The regeneration is cheap because
|
|
72
|
+
the stamps are derived from data already in the repo; the diff is the point.
|
|
73
|
+
A hand-edit survives exactly one CI run before it is flagged.
|
|
74
|
+
|
|
75
|
+
Rules:
|
|
76
|
+
|
|
77
|
+
- Stamp at release time from the single source the fact lives in — the stamp
|
|
78
|
+
step copies, it never re-types.
|
|
79
|
+
- Treat a hand-edit to a stamped artifact as a bug: fix the generator (or the
|
|
80
|
+
underlying source), not the artifact.
|
|
81
|
+
- If a fact is duplicated across stamped artifacts and hand-written pages, fix
|
|
82
|
+
it with a single source, not a sync script (the cross-surface drift audit is
|
|
83
|
+
a separate concern; the freshness check here covers machine-derived
|
|
84
|
+
artifacts only).
|
|
85
|
+
- The freshness lint runs in CI on every push, not only at release — release
|
|
86
|
+
day is when you can least afford a surprise.
|
|
87
|
+
|
|
88
|
+
## 3. Live-Site Smoke (CI curls the deployed URL)
|
|
89
|
+
|
|
90
|
+
The ddl precedent: CI curls the deployed docs URL (GitHub Pages, Netlify,
|
|
91
|
+
etc.) after deploy, so a 404'd book fails the pipeline immediately — not at
|
|
92
|
+
the next human visit, which might be weeks away. Build checks validate the
|
|
93
|
+
source; the smoke validates what readers actually receive.
|
|
94
|
+
|
|
95
|
+
Smoke shape:
|
|
96
|
+
|
|
97
|
+
1. After the deploy job succeeds, curl the URL a reader lands on: site root,
|
|
98
|
+
quickstart, at minimum the newest release page.
|
|
99
|
+
2. Fail on non-2xx (and on obvious error bodies — some hosts return 200 for
|
|
100
|
+
soft-404 pages; check for the expected content marker).
|
|
101
|
+
3. Run it in the same pipeline as the deploy, so a bad deploy fails before
|
|
102
|
+
anyone moves on.
|
|
103
|
+
|
|
104
|
+
Rules:
|
|
105
|
+
|
|
106
|
+
- Smoke the rendered URL, not the build artifact — the point is the thing a
|
|
107
|
+
reader's browser gets.
|
|
108
|
+
- Include at least one deep page, not only the index: deploy misconfigs often
|
|
109
|
+
break path-based routing while the root still resolves.
|
|
110
|
+
- Keep the assertions minimal (status + one content marker); this is a smoke,
|
|
111
|
+
not a rendering test suite.
|
package/content/distilled/documentation/references/implement-documentation/references/templates.md
CHANGED
|
@@ -6,6 +6,11 @@ One template per quadrant. Match the template to the quadrant identified in Step
|
|
|
6
6
|
|
|
7
7
|
Goal: build competency through a guided, safe experience. Show, do not lecture.
|
|
8
8
|
|
|
9
|
+
Quickstart rule (required for install/getting-started paths): every command shows
|
|
10
|
+
its expected output and exit code, and the install path ends with a post-install
|
|
11
|
+
verification block. If a step can fail realistically, show the failure and its
|
|
12
|
+
recovery inline. See `references/worked-examples.md` for the full pattern.
|
|
13
|
+
|
|
9
14
|
```markdown
|
|
10
15
|
# [Getting Started with X]
|
|
11
16
|
|
|
@@ -21,9 +26,33 @@ By the end you will have [concrete result]. No prior knowledge of [X] is assumed
|
|
|
21
26
|
## Step 1: [Specific Result]
|
|
22
27
|
[Chunked, do-this-now instructions. Each step yields a visible result.]
|
|
23
28
|
|
|
29
|
+
```bash
|
|
30
|
+
$ [command]
|
|
31
|
+
[expected output]
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Exit code 0. If you instead see `[realistic error message]`, the fix is
|
|
35
|
+
`[recovery command/step]`.
|
|
36
|
+
|
|
24
37
|
## Step 2: [Specific Result]
|
|
25
38
|
[Chunked instructions.]
|
|
26
39
|
|
|
40
|
+
```bash
|
|
41
|
+
$ [command]
|
|
42
|
+
[expected output]
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## Verify Your Install
|
|
46
|
+
Never end an install path on "it should work". Close with the tool's own
|
|
47
|
+
self-check so the learner confirms success before moving on.
|
|
48
|
+
|
|
49
|
+
```bash
|
|
50
|
+
$ [tool] --version
|
|
51
|
+
[expected version string]
|
|
52
|
+
$ [tool] [minimal smoke command]
|
|
53
|
+
[expected output]
|
|
54
|
+
```
|
|
55
|
+
|
|
27
56
|
## Recap & Next Steps
|
|
28
57
|
You built [X]. Next, try [lateral link 1] or [lateral link 2].
|
|
29
58
|
```
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
# Visual Docs & Cross-Repo Ecosystems: Diagrams, Fleet Maps & Rendered-Output Smoke
|
|
2
|
+
|
|
3
|
+
Methodology for documentation that covers pictures and ecosystems: diagrams
|
|
4
|
+
that survive static-site deploys, responsibility maps for multi-repo fleets,
|
|
5
|
+
and smoke checks that verify what readers' browsers actually render. Mined
|
|
6
|
+
from the 2026-10 ddl and specodelic docs work; evidence in
|
|
7
|
+
`.wai/resources/reflections/docs-skill-survey-2026-10.org` (row `iio`).
|
|
8
|
+
Canonical theory: `content/research-documentation-frameworks.md`.
|
|
9
|
+
|
|
10
|
+
The 2026-10 evidence: both failure classes are invisible to link checkers.
|
|
11
|
+
Specodelic shipped mermaid graphs that silently rendered as raw text on the
|
|
12
|
+
live site (`graph-views.html`) while passing every source-side check. Ddl
|
|
13
|
+
solved fleet navigation with a single responsibility-map SVG whose blocks
|
|
14
|
+
link into each project's own docs. The fixes are: prefer static pre-rendered
|
|
15
|
+
SVG over runtime mermaid on static sites; give each fleet layer one
|
|
16
|
+
responsibility map and each fleet-wide page exactly one home; after deploy,
|
|
17
|
+
smoke rendered behavior — not just reachability.
|
|
18
|
+
|
|
19
|
+
## 1. Diagrams: Prefer Static SVG Over Runtime Mermaid on Static-Site Books
|
|
20
|
+
|
|
21
|
+
The specodelic precedent: mermaid diagrams embedded as code blocks render
|
|
22
|
+
fine in the GitHub web UI (GitHub runs its own mermaid pipeline) but fail on
|
|
23
|
+
a self-hosted static-site book unless the site generator loads the mermaid
|
|
24
|
+
runtime — and that runtime is exactly the kind of third-party JS that breaks
|
|
25
|
+
quietly (CSP blocks, plugin version drift, JS load failures on cache-warm
|
|
26
|
+
pages). The `graph-views.html` incident shipped graphs as raw `graph TD`
|
|
27
|
+
text to every reader while the build stayed green, because nothing in the
|
|
28
|
+
pipeline executes a browser.
|
|
29
|
+
|
|
30
|
+
Rule: on a static-site book, prefer **pre-rendered SVG committed to the repo**
|
|
31
|
+
over runtime mermaid. SVG needs no JavaScript, renders in every browser and
|
|
32
|
+
PDF pipeline, and fails loudly (missing file → broken image) rather than
|
|
33
|
+
silently. If a diagram must stay in sync with code, close the loop with a
|
|
34
|
+
build step rather than by hoping:
|
|
35
|
+
|
|
36
|
+
- keep the mermaid (or dot, or similar) **source** in the repo next to the
|
|
37
|
+
page that embeds its rendered SVG;
|
|
38
|
+
- regenerate the SVG in the docs build (or a stamp step, in the sense of
|
|
39
|
+
`docs-accuracy.md` §2) and fail the build when the committed SVG is stale;
|
|
40
|
+
- never hand-edit the SVG — it is generated output, exactly like a stamped
|
|
41
|
+
artifact.
|
|
42
|
+
|
|
43
|
+
When runtime rendering is unavoidable (e.g. diagrams are author-written
|
|
44
|
+
content), smoke the rendered page (§3) so a dead mermaid runtime fails CI,
|
|
45
|
+
not a reader.
|
|
46
|
+
|
|
47
|
+
Rules:
|
|
48
|
+
|
|
49
|
+
- Static SVG first; runtime mermaid only with a rendered-output smoke.
|
|
50
|
+
- Every embedded diagram has a source in the repo and a build step that
|
|
51
|
+
regenerates it — a diagram no one can regenerate is a liability.
|
|
52
|
+
- Color-coding and clickable regions (see §2) belong in the SVG itself, so
|
|
53
|
+
they survive without JS.
|
|
54
|
+
- Diagram alt text names what the diagram claims, not what it depicts
|
|
55
|
+
("Layers of the X ecosystem and which repo owns each", not "Diagram").
|
|
56
|
+
|
|
57
|
+
## 2. Cross-Repo (Fleet) Docs: One Responsibility Map, One Home Per Page
|
|
58
|
+
|
|
59
|
+
The ddl precedent (`cbfa300`): a multi-repo ecosystem is unreadable until
|
|
60
|
+
someone draws it. Ddl built an ecosystem **responsibility map** — an SVG
|
|
61
|
+
with color-coded layers, one block per project, each block a clickable link
|
|
62
|
+
into that project's own docs. A reader landing on any repo's docs can answer
|
|
63
|
+
"what is this part, what is it responsible for, and where is the rest?"
|
|
64
|
+
without reading six READMEs.
|
|
65
|
+
|
|
66
|
+
The pattern has three pieces:
|
|
67
|
+
|
|
68
|
+
1. **A responsibility map per layer** (or one map with visible layers):
|
|
69
|
+
every block names a component, its one-line responsibility, and links to
|
|
70
|
+
the owning project's docs. Color-code by layer (runtime, tooling,
|
|
71
|
+
content, infra) so the map is scannable before it is readable. Place the
|
|
72
|
+
map in the hub repo's ecosystem page and mirror nothing — links, not
|
|
73
|
+
copies.
|
|
74
|
+
2. **Per-repo setup pages follow one shared template.** Each project's docs
|
|
75
|
+
cover its own setup with the same section skeleton (install → configure →
|
|
76
|
+
verify), so a reader moving between repos never re-learns the page
|
|
77
|
+
shape. The template lives once, referenced by each repo's docs; per-repo
|
|
78
|
+
pages differ only in content, never in structure.
|
|
79
|
+
3. **Fleet-wide pages live in exactly one repo.** Pages that describe the
|
|
80
|
+
ecosystem as a whole — the umbrella install page, the architecture
|
|
81
|
+
overview, the compatibility matrix — live in the hub repo. Duplicating a
|
|
82
|
+
fleet-wide page into each member repo guarantees staleness in all but
|
|
83
|
+
one copy; member repos link to the hub page instead. The responsibility
|
|
84
|
+
map's links are the navigation layer that makes "one home" viable.
|
|
85
|
+
|
|
86
|
+
Rules:
|
|
87
|
+
|
|
88
|
+
- One responsibility map, in the hub, linking out — never a fleet diagram
|
|
89
|
+
copied per repo.
|
|
90
|
+
- If a fact spans repos, either it lives in a hub fleet-wide page or it
|
|
91
|
+
lives per-repo behind a stable link — never both.
|
|
92
|
+
- Every member repo's docs start by locating themselves: one sentence plus
|
|
93
|
+
a link to the responsibility map.
|
|
94
|
+
- A shared setup template is enforced by review, not copy-paste: when the
|
|
95
|
+
template changes, a drift audit (see `review-documentation`) checks the
|
|
96
|
+
member pages.
|
|
97
|
+
|
|
98
|
+
## 3. Deployed-vs-Source Smoke: Verify Rendered Behavior, Not Just Links
|
|
99
|
+
|
|
100
|
+
The reachability layer is `docs-accuracy.md` §3 (live-site smoke: curl the
|
|
101
|
+
deployed URL, fail on non-2xx and soft-404s). This smoke builds **on** that
|
|
102
|
+
one: reachability is necessary but not sufficient, because the two 2026-10
|
|
103
|
+
incidents were fully reachable pages rendering wrong. Smoke what readers'
|
|
104
|
+
browsers actually receive and execute.
|
|
105
|
+
|
|
106
|
+
Smoke shape (runs after the reachability smoke, in the same pipeline):
|
|
107
|
+
|
|
108
|
+
1. **JS executes without errors.** Fetch the deployed page the way a browser
|
|
109
|
+
would (a headless browser or a smoke script that asserts no console
|
|
110
|
+
errors) on at least one page that loads JavaScript. A site whose search
|
|
111
|
+
widget throws on load is broken in a way `curl -f` cannot see.
|
|
112
|
+
2. **Diagrams and graphs render.** For pages embedding diagrams, assert the
|
|
113
|
+
rendered output, not the source: either the generated SVG is fetched and
|
|
114
|
+
non-empty, or the JS-rendered graph container holds rendered nodes (not
|
|
115
|
+
the raw mermaid text). This is the check the `graph-views.html` incident
|
|
116
|
+
would have failed.
|
|
117
|
+
3. **The search index builds and serves.** Fetch the search index artifact
|
|
118
|
+
the client would fetch, assert non-empty and parseable, and query it for
|
|
119
|
+
a term from the newest page. A stale or empty index ships green through
|
|
120
|
+
every link checker.
|
|
121
|
+
|
|
122
|
+
Rules:
|
|
123
|
+
|
|
124
|
+
- Smoke the rendered URL, not the build artifact — the point is the thing a
|
|
125
|
+
reader's browser gets.
|
|
126
|
+
- Assertions stay minimal per check (one marker per failure mode); this is a
|
|
127
|
+
smoke, not an end-to-end test suite.
|
|
128
|
+
- Run it in the deploy pipeline, after reachability, so a bad deploy fails
|
|
129
|
+
before anyone moves on.
|
|
130
|
+
- Any content class that needs a JS runtime to be correct (diagrams, search,
|
|
131
|
+
widgets) must appear in at least one rendered-behavior assertion.
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
# Worked Examples & Expected Output
|
|
2
|
+
|
|
3
|
+
Methodology for tutorial, quickstart, and example-page content. Mined from the
|
|
4
|
+
2026-10 docs work (specodelic packs: `data.lineage`, `numeric.predicates`,
|
|
5
|
+
`empirical.registry`); evidence in `.wai/resources/reflections/docs-skill-survey-2026-10.org`.
|
|
6
|
+
Canonical theory: `content/research-documentation-frameworks.md`.
|
|
7
|
+
|
|
8
|
+
## 1. The Worked-Example Pattern (one thread, not N snippets)
|
|
9
|
+
|
|
10
|
+
Examples were the #1 documentation gap in the 2026-10 survey. The failure mode
|
|
11
|
+
was not "too few examples" but **disconnected examples**: N one-off snippets that
|
|
12
|
+
each demonstrate a feature in isolation. Learners cannot see how features
|
|
13
|
+
compose, and maintainers cannot tell which examples are stale.
|
|
14
|
+
|
|
15
|
+
The fix is a single **evolving example thread**: one project that grows across
|
|
16
|
+
every example page, each page building on the previous state.
|
|
17
|
+
|
|
18
|
+
Thread shape (adapt per product):
|
|
19
|
+
|
|
20
|
+
1. **Install** — prerequisites, install command, expected output.
|
|
21
|
+
2. **First command** — smallest useful action, expected output, exit code.
|
|
22
|
+
3. **Verification** — post-install verification block (see §2).
|
|
23
|
+
4. **Realistic failure** — a failure a real first-time user hits, shown verbatim, with recovery.
|
|
24
|
+
5. **Recovery + next feature** — fix it, then extend the same project with the next feature.
|
|
25
|
+
|
|
26
|
+
Rules:
|
|
27
|
+
|
|
28
|
+
- **One continuous project.** Page N+1 imports or extends the artifact built on page N. Name it explicitly ("in the `inventory` project from the previous example…").
|
|
29
|
+
- **Each page stands alone in *task*, not in *state*.** A reader can jump in if they follow a one-line "start from" pointer, but never re-explains earlier steps.
|
|
30
|
+
- **Realistic failure over toy failure.** Mine real user conversations or support logs for the failure to show; a fabricated typo error teaches nothing.
|
|
31
|
+
- **Show the failure verbatim** — the actual error text, not a paraphrase — so learners can match it against their own terminal.
|
|
32
|
+
|
|
33
|
+
## 2. The Quickstart Rule (expected output everywhere)
|
|
34
|
+
|
|
35
|
+
Every command shown in a tutorial, quickstart, or how-to must show its expected
|
|
36
|
+
output and exit code. Two components:
|
|
37
|
+
|
|
38
|
+
**a. Expected output per command.**
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
$ spk explain data.lineage
|
|
42
|
+
lineage: tracks how records flow between datasets
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
Exit code 0. If the command can fail realistically, the recovery goes inline
|
|
46
|
+
right after (see the Tutorial template in `references/templates.md`).
|
|
47
|
+
|
|
48
|
+
**b. Post-install verification block.** Every install path ends with the
|
|
49
|
+
tool's own self-check — never "it should now work":
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
$ spk --version
|
|
53
|
+
spk 0.4.1
|
|
54
|
+
$ spk explain --list
|
|
55
|
+
data.lineage numeric.predicates empirical.registry ...
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
Rules:
|
|
59
|
+
|
|
60
|
+
- Output shown must be **actual captured output**, not invented prose. If the docs are executable, the gate re-runs the commands and diffs against what you wrote — invented output fails.
|
|
61
|
+
- Prefer the tool's self-check (`--version`, `explain`, `--list`) over existence checks (`which`, `ls`) — a binary can exist and still be broken.
|
|
62
|
+
- Normalize volatile output (versions, timestamps, hashes) only if the docs pipeline documents the normalization.
|
|
63
|
+
- Exit codes: show them when a command is known to fail in realistic conditions, so learners can grep their terminal against the doc.
|
|
64
|
+
|
|
65
|
+
## 3. The Example Coverage Matrix (checklist, not tooling)
|
|
66
|
+
|
|
67
|
+
After writing example pages, produce a coverage matrix mapping every
|
|
68
|
+
user-facing feature (pack, command, module, endpoint) to at least one example
|
|
69
|
+
page. Keep it a plain table or checklist in the docs or the tracker — do not
|
|
70
|
+
build tooling for it; the audit is the point.
|
|
71
|
+
|
|
72
|
+
| Feature | Example page | Thread position |
|
|
73
|
+
| :--- | :--- | :--- |
|
|
74
|
+
| [feature/command 1] | [path to example] | 1 (install) |
|
|
75
|
+
| [feature/command 2] | [path to example] | 3 (first command) |
|
|
76
|
+
| [feature/command 3] | *(uncovered)* | — |
|
|
77
|
+
|
|
78
|
+
Rules:
|
|
79
|
+
|
|
80
|
+
- **Every row needs ≥1 example page or an explicit "(uncovered)".** Uncovered rows become follow-up tickets immediately — do not let them linger as implicit debt.
|
|
81
|
+
- **One row per user-facing feature**, from the user's vocabulary, not the internal module graph.
|
|
82
|
+
- **Thread position** records where the feature appears in the evolving example, exposing which pages are doing double duty and which features are stranded outside the thread.
|
|
83
|
+
- Re-run the matrix check after each release; new features enter with a row, not an apology.
|
|
@@ -8,7 +8,7 @@ Perform a Rule of 5 review of the documentation using the Unified Frameworks for
|
|
|
8
8
|
|
|
9
9
|
## Procedure
|
|
10
10
|
|
|
11
|
-
Perform 5 iterative passes. Check for convergence after each pass starting with Pass 2: report new CRITICAL issues, total new issues, the change vs. the previous pass, and status (CONVERGED or CONTINUE). Converge when no new CRITICAL issues are found and the new-issue rate drops below 10% versus the previous pass. Stop early if converged.
|
|
11
|
+
Perform 5 iterative passes — 6 when the subject spans multiple surfaces (README, docs index, installation page): Pass 6 is the cross-surface consistency pass — 7 when the subject is user-facing documentation (for README, quickstart, and tool docs that is effectively always): Pass 7 is the cold-reader & positioning pass. Check for convergence after each pass starting with Pass 2: report new CRITICAL issues, total new issues, the change vs. the previous pass, and status (CONVERGED or CONTINUE). Converge when no new CRITICAL issues are found and the new-issue rate drops below 10% versus the previous pass. Stop early if converged.
|
|
12
12
|
|
|
13
13
|
### Pass 1: Diátaxis & Intent
|
|
14
14
|
|
|
@@ -24,12 +24,20 @@ Goal: can the reader scan the page in under 10 seconds and find the key answer?
|
|
|
24
24
|
|
|
25
25
|
### Pass 4: Accuracy, "Whole Game" & Fail-States
|
|
26
26
|
|
|
27
|
-
Goal: is the content technically sound and resilient? Checks: are code snippets verified? Does it show the complete result? Are common errors documented? In tutorials, is the Martini Glass pattern followed (guided context then open exploration)? If the subject is a runnable tool/API: spot-check documented claims against the artifact itself (the method statement goes in the Final Report provenance block). For entry-point pages (README / landing / quickstart): is there a quickstart reaching a first verified success in minutes (time-to-first-success; see `../research-documentation/references/metrics.md`)? Issue prefix: [ACC-001].
|
|
27
|
+
Goal: is the content technically sound and resilient? Checks: are code snippets verified? Does it show the complete result? Are common errors documented? In tutorials, is the Martini Glass pattern followed (guided context then open exploration)? If the subject is a runnable tool/API: spot-check documented claims against the artifact itself (the method statement goes in the Final Report provenance block). For entry-point pages (README / landing / quickstart): is there a quickstart reaching a first verified success in minutes (time-to-first-success; see `../research-documentation/references/metrics.md`)? For repos with executable docs: does CI re-run the documented commands and diff their captured output against the docs (accuracy gate), are stamped artifacts (`llms.txt`, `release.md`, README version facts) generated rather than hand-edited (freshness check), and does CI smoke the live deployed URL? Flag the absence of any of these as an [ACC-] finding — the methodology lives in `../implement-documentation/references/docs-accuracy.md`; cross-reference it, do not duplicate it here. Issue prefix: [ACC-001].
|
|
28
28
|
|
|
29
29
|
### Pass 5: Excellence & AI-Readiness
|
|
30
30
|
|
|
31
31
|
Goal: is it production-ready and optimized for AI agents? Checks: is there a TL;DR for LLMs? Are headers semantically unique for RAG? Is the Answer First (Pyramid Principle) in the TL;DR and first paragraph? Use `references/ai-readiness-criteria.md` for the deep AI-readiness checks (`llms.txt`, RAG chunking, dual-audience, schema sync). Are claims marked by status — documented / interpreted / proposed / unknown — with unverified claims not stated as fact? Issue prefix: [EXCL-001].
|
|
32
32
|
|
|
33
|
+
### Pass 6: Cross-Surface Consistency (multi-surface subjects only)
|
|
34
|
+
|
|
35
|
+
Goal: do all user-facing surfaces tell one story with one set of facts? Applies when the subject spans multiple surfaces (README, docs index, installation page — any fact published in more than one place). Checks: run the drift audit — enumerate the facts repeated across surfaces (pack/module counts, command names, install steps, links, version claims) and verify every copy agrees; flag copies that should be generated/stamped instead of hand-written. Then check topic alignment: README, docs index, and installation page must tell one consistent story (same audience, same promise, same next step), not drift into different framings. The methodology lives in `references/drift-audit.md`; cross-reference it, do not duplicate it here. Issue prefix: [DRIFT-001].
|
|
36
|
+
|
|
37
|
+
### Pass 7: Cold-Reader & Positioning
|
|
38
|
+
|
|
39
|
+
Goal: would a confused first-time human succeed — not just an AI agent scoring structure (`references/ai-readiness-criteria.md` scores ingestion; it never asks the human question)? Applies when the subject is user-facing documentation. Checks: (1) cold-reader question mining — collect real user questions from support conversations, session logs, and issue titles, and verify each has a doc answer; unanswered ones become FAQ entries or tickets; (2) first-run funnel — can a cold reader reach first success in under 10 minutes (time-to-first-success; see `../research-documentation/references/metrics.md`)? (3) positioning — does every tool/format doc carry an explicit "when NOT to use this" section, and is there a decision guide where entry points overlap? The methodology lives in `references/cold-reader.md`; cross-reference it, do not duplicate it here. Issue prefix: [COLD-001].
|
|
40
|
+
|
|
33
41
|
## Final Report
|
|
34
42
|
|
|
35
43
|
- Total Issues by Severity (Critical, High, Medium, Low).
|
package/content/distilled/documentation/references/review-documentation/references/cold-reader.md
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
# Cold-Reader & Positioning Review (Pass 7)
|
|
2
|
+
|
|
3
|
+
Methodology for Pass 7 of the review-documentation skill. Mined from the
|
|
4
|
+
2026-10 evidence: two separate cold-read exercises drove the specodelic docs
|
|
5
|
+
work — one reviewed real user conversations and turned the actual questions
|
|
6
|
+
into FAQ, decision-guide, and glossary pages (commit `2e52854`, beads `7gh`);
|
|
7
|
+
the other produced the "what the format assumes / when NOT to use it"
|
|
8
|
+
positioning page (commit `427bf9a`). Evidence:
|
|
9
|
+
`.wai/resources/reflections/docs-skill-survey-2026-10.org`.
|
|
10
|
+
|
|
11
|
+
The gap Pass 7 closes: `ai-readiness-criteria.md` scores structure for LLM
|
|
12
|
+
ingestion — chunking, unique headers, dual-audience — and never asks the
|
|
13
|
+
human question: *would a confused first-time reader succeed?* Structure that
|
|
14
|
+
scores perfectly for RAG can still strand a human at the front door.
|
|
15
|
+
|
|
16
|
+
## 1. The Question Mine (enumerate real questions, don't invent them)
|
|
17
|
+
|
|
18
|
+
Before reading the docs at all, collect the questions real users actually
|
|
19
|
+
asked. The evidence is already sitting in three places:
|
|
20
|
+
|
|
21
|
+
- **Support conversations** — chat logs, email threads, community channels.
|
|
22
|
+
Copy the user's question *verbatim*; paraphrase erases the exact
|
|
23
|
+
vocabulary gap that made the docs hard to find.
|
|
24
|
+
- **Session logs / agent transcripts** — where a human or agent worked with
|
|
25
|
+
the tool, the points where they stopped to re-read, backtracked, or asked
|
|
26
|
+
a clarifying question are the confusion map.
|
|
27
|
+
- **Issue titles and discussion threads** — a question filed as an issue is
|
|
28
|
+
a question the docs failed to answer in the most discoverable place.
|
|
29
|
+
|
|
30
|
+
For each question, grep the docs for an answer: not the topic, the actual
|
|
31
|
+
answer a cold reader could act on. Classify each question:
|
|
32
|
+
|
|
33
|
+
- **Answered but undiscoverable** — the answer exists on a page the reader
|
|
34
|
+
would never reach from the entry point. Finding: add the missing link or
|
|
35
|
+
FAQ entry pointing at it.
|
|
36
|
+
- **Answered but unclear** — the answer exists and is findable, yet the
|
|
37
|
+
user still asked. Finding: the page's wording failed; cite the exact
|
|
38
|
+
sentence that did not transfer.
|
|
39
|
+
- **Unanswered** — no page answers it. Finding: becomes a FAQ entry or an
|
|
40
|
+
implementation ticket (`implement-documentation`), not a silent gap.
|
|
41
|
+
|
|
42
|
+
Rules:
|
|
43
|
+
|
|
44
|
+
- Mine at least the last 10 real user questions before judging any page —
|
|
45
|
+
a review built on imagined questions flags what authors fear, not what
|
|
46
|
+
readers hit.
|
|
47
|
+
- Every unanswered question produces an artifact: a FAQ entry with the
|
|
48
|
+
user's own wording, or a filed ticket. A mined question that changes
|
|
49
|
+
nothing was decoration.
|
|
50
|
+
- Record each finding with the question's source (conversation, log, issue
|
|
51
|
+
URL) — evidence-mined findings outrank reviewer intuition.
|
|
52
|
+
|
|
53
|
+
## 2. The First-Run Funnel (time-to-first-success under 10 minutes)
|
|
54
|
+
|
|
55
|
+
Walk the entry path exactly as a cold first-time reader would: land on the
|
|
56
|
+
README or quickstart, follow the first action it names, and do not use any
|
|
57
|
+
knowledge the page does not state. Time it end to end.
|
|
58
|
+
|
|
59
|
+
The 2026-10 prompt was a user's own words: *"we have a presentation today,
|
|
60
|
+
it should be clear and easy to onboard"* — onboarding time is a user-visible
|
|
61
|
+
requirement, not an internal quality goal. The metric is
|
|
62
|
+
time-to-first-success: first page view to first successful, verified use.
|
|
63
|
+
|
|
64
|
+
Checks along the funnel walk:
|
|
65
|
+
|
|
66
|
+
- **Zero-to-install**: does the first page state the prerequisites up front,
|
|
67
|
+
or does the reader discover them by failing? Each discovered-late
|
|
68
|
+
prerequisite is a restart.
|
|
69
|
+
- **Decision count**: count the forks where the reader must choose without
|
|
70
|
+
guidance (package managers, install methods, "advanced vs. simple" paths).
|
|
71
|
+
More than a couple of unguided forks before first success is drift between
|
|
72
|
+
the docs and a cold reader's context.
|
|
73
|
+
- **Verified success**: does the quickstart end with an output the reader
|
|
74
|
+
can check ("you should see…") rather than an unverified "it works"? An
|
|
75
|
+
unverifiable success is indistinguishable from a failure.
|
|
76
|
+
- **Cross-reference, don't re-derive**: the metric definitions (TTFS,
|
|
77
|
+
support-ticket deflection, search analytics) live in
|
|
78
|
+
`../../research-documentation/references/metrics.md` — Pass 4 already uses
|
|
79
|
+
that file for entry-point pages. Read it for the measurement playbook;
|
|
80
|
+
this pass only runs the walk and reports where the clock exceeded 10
|
|
81
|
+
minutes or the reader stalled.
|
|
82
|
+
|
|
83
|
+
Rules:
|
|
84
|
+
|
|
85
|
+
- Time the funnel with a stopwatch or by counting steps, and report the
|
|
86
|
+
number — "it felt long" is not a finding; "install-to-first-success took
|
|
87
|
+
23 steps across 4 pages" is.
|
|
88
|
+
- Walk the path as written, including the parts that are wrong; fixing the
|
|
89
|
+
docs in your head during the walk erases exactly the findings the pass
|
|
90
|
+
exists to produce.
|
|
91
|
+
- If the funnel cannot be completed at all, that is the top-severity
|
|
92
|
+
finding, not a pass failure.
|
|
93
|
+
|
|
94
|
+
## 3. Positioning: "When NOT to Use This" and the Decision Guide
|
|
95
|
+
|
|
96
|
+
The `427bf9a` evidence: a format that explained itself well still needed a
|
|
97
|
+
page stating what the format *assumes* and when NOT to use it — because
|
|
98
|
+
readers who pick the wrong tool blame the tool, and no amount of positive
|
|
99
|
+
documentation prevents the wrong first choice. Positioning is the part of
|
|
100
|
+
the docs written for the reader who should walk away.
|
|
101
|
+
|
|
102
|
+
Checks:
|
|
103
|
+
|
|
104
|
+
- **"When NOT to use" section**: every tool or format doc needs an explicit
|
|
105
|
+
section naming the situations where this tool is the wrong choice, with
|
|
106
|
+
the reason and — where one exists — the alternative to use instead. A
|
|
107
|
+
good section is concrete ("do not use X when your data exceeds memory;
|
|
108
|
+
use Y instead"), not hedged ("X may not be suitable for all use cases" —
|
|
109
|
+
a hedge that answers nothing).
|
|
110
|
+
- **Decision guide for overlapping entry points**: where two pages
|
|
111
|
+
(README vs. docs index, tutorial vs. how-to, tool A vs. tool B) could
|
|
112
|
+
both be a reader's first stop, there must be a stated rule for choosing
|
|
113
|
+
— a short comparison table or "start here if / use instead if" list at
|
|
114
|
+
the top of each overlapping page. Overlapping entry points without a
|
|
115
|
+
decision guide fork the reader at the moment they have no context to
|
|
116
|
+
choose.
|
|
117
|
+
- **Assumptions stated**: the "what the format assumes" list belongs near
|
|
118
|
+
the top of the positioning page — readers invalidated by an unstated
|
|
119
|
+
assumption lose the most time.
|
|
120
|
+
|
|
121
|
+
Rules:
|
|
122
|
+
|
|
123
|
+
- Flag every tool/format doc missing the "when NOT to use" section as a
|
|
124
|
+
finding, not a suggestion — the evidence case shipped one after a real
|
|
125
|
+
user picked the format for a job it was never meant for.
|
|
126
|
+
- Test each "when NOT to use" claim against the artifact where possible
|
|
127
|
+
(same stance as the Pass 4 spot-check); a wrong negative sends readers
|
|
128
|
+
away from a tool that would have worked.
|
|
129
|
+
- The decision guide goes where the overlap is — at the entry points, not
|
|
130
|
+
buried on a comparison page nobody reaches before choosing.
|