agent-doc-system 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +188 -1
- package/README.md +1 -1
- package/adapters/cache.mjs +5 -4
- package/adapters/database.mjs +155 -6
- package/adapters/deploy-config.mjs +3 -2
- package/adapters/dockerfile.mjs +3 -2
- package/adapters/generic.mjs +2 -1
- package/adapters/github-actions.mjs +2 -1
- package/adapters/go.mjs +4 -3
- package/adapters/node.mjs +3 -2
- package/adapters/rust.mjs +3 -0
- package/adapters/turborepo.mjs +3 -2
- package/adapters/typescript.mjs +7 -5
- package/adapters/wrangler.mjs +3 -2
- package/bin/agentdoc.mjs +9 -2
- package/bin/usage.txt +2 -0
- package/core/audit.mjs +7 -5
- package/core/codes.mjs +2 -0
- package/core/compile.mjs +12 -6
- package/core/derive.mjs +4 -3
- package/core/descriptors.mjs +52 -2
- package/core/discover.mjs +3 -2
- package/core/fsx.mjs +174 -42
- package/core/graph.mjs +1 -1
- package/core/impact.mjs +84 -14
- package/core/paths.mjs +71 -0
- package/core/query.mjs +58 -23
- package/core/scaffold.mjs +130 -23
- package/core/sourcescan.mjs +4 -1
- package/docs/ADAPTERS.md +49 -5
- package/docs/BENCHMARK.md +83 -21
- package/docs/CI.md +9 -1
- package/docs/EVALS.md +209 -17
- package/docs/MIGRATION.md +22 -4
- package/docs/PROVENANCE.md +6 -0
- package/docs/QUERY.md +50 -7
- package/docs/SPEC.md +89 -2
- package/docs/ci/github-actions.yml +1 -1
- package/evals/baseline-search.mjs +272 -0
- package/evals/facts.mjs +28 -0
- package/evals/harness.mjs +120 -116
- package/package.json +4 -2
- package/schemas/resource.schema.json +9 -1
- package/skills/agent-doc-system/SKILL.md +2 -1
- package/skills/agent-doc-system/references/cli.md +6 -1
- package/skills/agent-doc-system/references/create-migrate.md +8 -2
- package/templates/ci.yml +1 -1
package/docs/EVALS.md
CHANGED
|
@@ -7,30 +7,42 @@ noise than exploring the repository unaided?
|
|
|
7
7
|
|
|
8
8
|
The question is not "does it compile". A catalog can compile perfectly and still
|
|
9
9
|
route an agent into the wrong corner of the repository. The measurement is
|
|
10
|
-
retrieval quality
|
|
10
|
+
retrieval quality on the same scenarios for three retrieval systems.
|
|
11
11
|
|
|
12
|
-
**baseline-targeted** —
|
|
13
|
-
|
|
14
|
-
|
|
12
|
+
**baseline-targeted** — the historical, graph-assisted reference: the touched
|
|
13
|
+
entity, its subtree, directly connected entities, documents, events and checks,
|
|
14
|
+
plus the original repository-wide documentation sweep. It reads compiled
|
|
15
|
+
relations and authority facts. It is not an agent without AgentDoc.
|
|
15
16
|
|
|
16
|
-
**baseline-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
17
|
+
**baseline-search** — a deterministic model of an agent with file reads and
|
|
18
|
+
text search over the working tree, starting at the touched paths. It searches
|
|
19
|
+
for importers and importees to a fixed point, then reads the reached components'
|
|
20
|
+
docs, manifests and checks. It has no compiled graph or compiler-derived
|
|
21
|
+
relations. Its complete heuristics are below.
|
|
20
22
|
|
|
21
23
|
**catalog** — what `agentdoc query` returns for the touched paths, plus the
|
|
22
24
|
context of the components `impact` says are affected.
|
|
23
25
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
+
**baseline-full** is retained only as a size yardstick: all graph entities,
|
|
27
|
+
relations, event subjects, checks and documents. It is a graph inventory, not
|
|
28
|
+
unaided repository exploration.
|
|
29
|
+
|
|
30
|
+
No model is involved. Retrieval sets are compared as sets. All systems and the
|
|
31
|
+
noise closure use `evals/facts.mjs`: resource links use `uses:` / `usedBy:`, API
|
|
32
|
+
links use `provides:` / `consumes:` / `provider:` / `consumer:`, and component
|
|
33
|
+
and placement links use bare entity refs. A relation also identifies its other
|
|
34
|
+
endpoint, as query's neighbour summaries do. Inverse edges respect direction.
|
|
35
|
+
An API subject supplies `provides:<its-ref>`, as query does. Documents use
|
|
36
|
+
`doc:` throughout, including the old targeted sweep (formerly `repodoc:`).
|
|
37
|
+
Compiler relation names are not separate facts. Tests pin this vocabulary.
|
|
26
38
|
|
|
27
39
|
## Metrics
|
|
28
40
|
|
|
29
41
|
| metric | definition |
|
|
30
42
|
|---|---|
|
|
31
|
-
| recall | fraction of the scenario's ground-truth facts
|
|
43
|
+
| recall | fraction of the scenario's ground-truth facts retrieved by each system |
|
|
32
44
|
| critical recall | the same, restricted to critical context: constraints, conflicts, observations, resources, API boundaries, event subjects |
|
|
33
|
-
| verification recall | fraction of the scenario's verification expectations returned; 1 when there are no expectations, for
|
|
45
|
+
| verification recall | fraction of the scenario's verification expectations returned; 1 when there are no expectations, for every system |
|
|
34
46
|
| check selectivity | distinct recommended check IDs / distinct check IDs in the repository graph (`verification` + `gates`); reported only, not gated; lower means fewer checks recommended |
|
|
35
47
|
| precision | `1 - irrelevance` |
|
|
36
48
|
| irrelevance (noise) | fraction of retrieved facts outside the graph's one-hop closure of the touched entities |
|
|
@@ -52,13 +64,125 @@ graph, selectivity is 1 (there is nothing to narrow). Corpus values are scenario
|
|
|
52
64
|
means; the aggregate is scenario-weighted across corpora. Both CLI reports
|
|
53
65
|
include per-scenario values in their `--json` output.
|
|
54
66
|
|
|
55
|
-
|
|
67
|
+
Targeted baseline verification recall is 1 by construction: its verification strategy
|
|
56
68
|
runs every graph verification entry and repository gate, so it cannot miss a
|
|
57
69
|
registered expectation. Empty expectations also score 1 on both sides. Its
|
|
58
|
-
check selectivity is therefore 1.
|
|
70
|
+
check selectivity is therefore 1. Search discovers its own checks; it does not
|
|
71
|
+
inherit this union. Selectivity is the meaningful comparison of
|
|
59
72
|
how much routing narrows the check set, read alongside catalog verification
|
|
60
73
|
recall so that omitting a required check is not mistaken for an improvement.
|
|
61
74
|
|
|
75
|
+
`recall` is a scenario mean. `microRecall` instead divides total fact hits by
|
|
76
|
+
total truth facts; per-scenario hit/total counts make this independently
|
|
77
|
+
recomputable. Aggregate micro-recall uses these raw counts, not corpus means.
|
|
78
|
+
The legacy JSON keys remain: catalog metrics are at the top level, targeted
|
|
79
|
+
metrics are under `baseline`, and search metrics are under `searchBaseline`.
|
|
80
|
+
|
|
81
|
+
Search returns scoped `(componentRef, command)` pairs without graph check IDs.
|
|
82
|
+
Only the scorer matches these pairs to graph verification entries and gates,
|
|
83
|
+
then counts their IDs for selectivity and `verify-id:` expectations. A root
|
|
84
|
+
command matches only a repository-scoped check. Identical commands in different
|
|
85
|
+
components remain distinct. Unregistered commands remain retrieved facts and
|
|
86
|
+
can contribute noise, but do not select a registered ID. Read selectivity with
|
|
87
|
+
verification recall; a low value can mean a missed check.
|
|
88
|
+
|
|
89
|
+
### Search baseline heuristics
|
|
90
|
+
|
|
91
|
+
`evals/baseline-search.mjs` imports only filesystem/path utilities, the existing
|
|
92
|
+
YAML/TOML syntax readers and the shared fact helper. It never invokes adapters,
|
|
93
|
+
routing, a package manager, a model or a shell search. The following rules are
|
|
94
|
+
fixed independently of scenario truth:
|
|
95
|
+
|
|
96
|
+
1. **Working tree.** Walk regular files in sorted name order, including
|
|
97
|
+
uncommitted files. Do not follow symlinks. Skip `.git`, `.agentdoc`,
|
|
98
|
+
`node_modules`, `.venv`, `venv`, `__pycache__`, `target`, `dist`, `build` and
|
|
99
|
+
`.next`, and any `graph.json`. Read JS/TS (including CJS/MJS/JSX/TSX), Go,
|
|
100
|
+
Rust, Python, proto, SQL, Markdown, RST, TXT, JSON, YAML, TOML, INI, CFG,
|
|
101
|
+
MOD, WORK, LOCK and Dockerfile text; skip NUL-containing files. This is a
|
|
102
|
+
literal exclusion list, not `.gitignore` or a tracked-file snapshot.
|
|
103
|
+
2. **Ownership only.** YAML documents with `apiVersion: agentdoc.dev/...`
|
|
104
|
+
are excluded from search. For `Component` documents only, take the metadata
|
|
105
|
+
name/namespace as identity, the descriptor directory as source root, and
|
|
106
|
+
string arrays under `spec.context` as owned document paths. Explicit document
|
|
107
|
+
ownership wins, otherwise the longest component root wins. No placement,
|
|
108
|
+
relations, API/resource declarations, journeys, observations or authority
|
|
109
|
+
rules are used. Other files under `agentdoc/` are excluded too.
|
|
110
|
+
3. **Import names.** Read each component's root `package.json` name, Cargo
|
|
111
|
+
package name and its underscore spelling, `go.mod` module path, Python
|
|
112
|
+
distribution names, and top-level Python packages with `__init__.py` or
|
|
113
|
+
`__main__.py` (under the root or `src/`). These are search terms, not
|
|
114
|
+
compiler-resolved units. A component without an import name can still own
|
|
115
|
+
touched paths and participate through relative imports or endpoint strings.
|
|
116
|
+
4. **Importers and importees.** Search source for JS/TS `from`, `import` and
|
|
117
|
+
`require` quoted specifiers; Go quoted strings; Rust `use`, `extern crate`
|
|
118
|
+
and `name::`; Python line-leading `import` / `from` names. Match exact names
|
|
119
|
+
or slash/dot subpaths. Search manifests for names at word/path boundaries,
|
|
120
|
+
and for local `path =`, `file:` and `link:` dependencies relative to that
|
|
121
|
+
manifest. Relative source specifiers are resolved against their file's
|
|
122
|
+
directory and mapped to an owner. Rust names declared with `mod` in the
|
|
123
|
+
same file are suppressed; Python imports with a matching local `.py` file
|
|
124
|
+
anywhere in the importing component are suppressed. Ambiguous sibling names
|
|
125
|
+
may reach multiple components. There is no symbol resolver, alias resolver
|
|
126
|
+
or full import grammar (for example Python comma imports are incomplete).
|
|
127
|
+
5. **Fixed point.** Import evidence connects components in both directions.
|
|
128
|
+
Start with owners of touched files (or explicitly touched component refs),
|
|
129
|
+
add all importers and importees, and repeat until no component is added.
|
|
130
|
+
API/endpoint text links described next participate in the same closure.
|
|
131
|
+
There is no hop limit or output cap.
|
|
132
|
+
6. **APIs and endpoints.** Read OpenAPI YAML `info.title` and `paths`, and
|
|
133
|
+
protobuf `service` names. A lowercase hyphenated title plus `-api` (without
|
|
134
|
+
duplicating an existing `-api` suffix) names an OpenAPI fact; a hyphenated
|
|
135
|
+
service name names a protobuf fact. CamelCase boundaries are split. These
|
|
136
|
+
labels are guesses, not lookups in the API catalog; namespaces default to
|
|
137
|
+
`default`. Find readers by literal contract file paths in component files.
|
|
138
|
+
Find providers by exact endpoint strings in source, Next-style
|
|
139
|
+
`/app/<path>/route.ts` files, or `Register<Service>Server` text. Contract
|
|
140
|
+
readers/providers are connected. Touching a contract seeds them and emits
|
|
141
|
+
the API ref and `provides:` fact; reached readers/providers emit API roles,
|
|
142
|
+
with a provider excluded from the consumer set. Other API kinds, dynamic
|
|
143
|
+
routes, semantic serving roles and arbitrary catalog names are not inferred.
|
|
144
|
+
Also connect source/TOML files sharing an exact slash-prefixed endpoint or
|
|
145
|
+
HTTP(S) URL pathname; Next `route.js`/`route.ts` paths count. Ignore `/`.
|
|
146
|
+
Hostnames are ignored. Common paths such as `/api/health` can consequently
|
|
147
|
+
connect unrelated services; the measured noise includes this overreach.
|
|
148
|
+
7. **Docs and events.** Read all text belonging to reached components. Markdown,
|
|
149
|
+
RST and TXT files count as `doc:`, except a basename containing `constraint`
|
|
150
|
+
gives `constraint:` and a path containing `runbook` gives `runbook:`.
|
|
151
|
+
In source lines mentioning subject, event, topic, publish or subscribe
|
|
152
|
+
(case insensitive), quoted lowercase-leading dotted strings with at least
|
|
153
|
+
three segments count as events. There are no configured event prefixes or
|
|
154
|
+
compiler event contracts. Imports reach a shared subject's defining file.
|
|
155
|
+
8. **Resources and externals.** PostgreSQL URL schemes, `sql.Open("postgres"`,
|
|
156
|
+
`pg`, `psycopg`, `asyncpg`, `sqlx`, `diesel`, or SQL files under `migrations/`
|
|
157
|
+
suggest `resource:default/<component-name>-postgres`. The word `redis`
|
|
158
|
+
suggests `<component-name>-redis`. TOML-style `bucket =` / `queue_name =`
|
|
159
|
+
string literals supply their slugged names. These guesses emit resource refs
|
|
160
|
+
and `uses:`; no catalog resource names are read. Plain environment variable
|
|
161
|
+
names without one of these signals are insufficient to identify a resource.
|
|
162
|
+
External names come from Node dependency maps, Cargo dependency tables,
|
|
163
|
+
Go requirements, Python project dependencies/extras/dev groups and
|
|
164
|
+
requirements text. Names already identifying local units are excluded.
|
|
165
|
+
Conditional dependencies and comments are not semantically evaluated.
|
|
166
|
+
9. **Checks.** Node scripts whose names mention test/lint/typecheck/check,
|
|
167
|
+
clippy/vet/pytest/ruff/mypy/unittest return their literal commands; root Turbo
|
|
168
|
+
tasks with those names generate `turbo run <task> --filter=<package>` for
|
|
169
|
+
reached packages. A Go manifest suggests `go test ./...` and `go vet ./...`;
|
|
170
|
+
a Cargo package suggests `cargo test -p <name>` and `cargo clippy -p <name>`.
|
|
171
|
+
Python component text mentioning pytest, ruff or mypy suggests that command
|
|
172
|
+
(`ruff check` for ruff); the nearest ancestor uv/Poetry lockfile selects
|
|
173
|
+
`uv run` / `poetry run`. Unittest without pytest suggests the standard-library
|
|
174
|
+
command `python -m unittest discover`. These are conventional suggestions,
|
|
175
|
+
not proof the checks pass. GitHub Actions YAML run steps are split by lines:
|
|
176
|
+
check-like lines select reached components through explicit package names
|
|
177
|
+
or step/job working directories; unscoped root lines are repository checks.
|
|
178
|
+
Filtered commands, `cd` and matrix expressions without a resolved scope are
|
|
179
|
+
skipped. Expressions, shell semantics and workflow matrices are not expanded.
|
|
180
|
+
Checks deduplicate by component and command.
|
|
181
|
+
10. **No hidden knowledge.** There is no scenario-specific branch, truth-set
|
|
182
|
+
lookup or graph membership filter in retrieval. Unsupported constructs may
|
|
183
|
+
miss facts; naming guesses may add facts absent from the graph. Search does
|
|
184
|
+
not infer placement, runtime observations or authority conflicts from code.
|
|
185
|
+
|
|
62
186
|
### Why noise is measured against reachability, not against the truth set
|
|
63
187
|
|
|
64
188
|
The first definition of noise was the complement of truth recall. It is
|
|
@@ -82,6 +206,65 @@ repository; a router that returned *every fact in the repository* then scored as
|
|
|
82
206
|
precise, because everything was technically adjacent to something. The metric
|
|
83
207
|
was measuring its own closure, not the router.
|
|
84
208
|
|
|
209
|
+
## Measured comparison (2026-10-27)
|
|
210
|
+
|
|
211
|
+
Values are scenario means; the aggregate weights corpora by scenario count.
|
|
212
|
+
Recall is higher-is-better; noise and check selectivity are lower-is-better
|
|
213
|
+
when read alongside recall. All three systems have verification recall 1.0 on
|
|
214
|
+
these scenarios.
|
|
215
|
+
|
|
216
|
+
| Corpus (scenarios) | System | Mean recall | Noise | Check selectivity |
|
|
217
|
+
|---|---|---:|---:|---:|
|
|
218
|
+
| Koda (9) | baseline-targeted (corrected) | 100.00% | 41.23% | 100.00% |
|
|
219
|
+
| Koda (9) | baseline-search | 71.90% | 71.55% | 100.00% |
|
|
220
|
+
| Koda (9) | catalog | 100.00% | 4.02% | 28.79% |
|
|
221
|
+
| Node/TS (3) | baseline-targeted (corrected) | 100.00% | 17.86% | 100.00% |
|
|
222
|
+
| Node/TS (3) | baseline-search | 91.67% | 30.64% | 83.33% |
|
|
223
|
+
| Node/TS (3) | catalog | 100.00% | 0.00% | 100.00% |
|
|
224
|
+
| Go (4) | baseline-targeted (corrected) | 93.75% | 28.94% | 100.00% |
|
|
225
|
+
| Go (4) | baseline-search | 83.33% | 26.01% | 100.00% |
|
|
226
|
+
| Go (4) | catalog | 100.00% | 5.88% | 66.67% |
|
|
227
|
+
| Mixed monorepo (4) | baseline-targeted (corrected) | 100.00% | 37.82% | 100.00% |
|
|
228
|
+
| Mixed monorepo (4) | baseline-search | 100.00% | 41.25% | 100.00% |
|
|
229
|
+
| Mixed monorepo (4) | catalog | 100.00% | 5.88% | 54.17% |
|
|
230
|
+
| Rust (4) | baseline-targeted (corrected) | 100.00% | 48.25% | 100.00% |
|
|
231
|
+
| Rust (4) | baseline-search | 100.00% | 32.81% | 83.33% |
|
|
232
|
+
| Rust (4) | catalog | 100.00% | 0.00% | 25.00% |
|
|
233
|
+
| Python (6) | baseline-targeted (corrected) | 100.00% | 46.70% | 100.00% |
|
|
234
|
+
| Python (6) | baseline-search | 100.00% | 23.68% | 41.67% |
|
|
235
|
+
| Python (6) | catalog | 100.00% | 0.00% | 41.67% |
|
|
236
|
+
| Aggregate (30) | baseline-targeted (corrected) | 99.17% | 38.83% | 100.00% |
|
|
237
|
+
| Aggregate (30) | baseline-search | 88.51% | 42.61% | 84.44% |
|
|
238
|
+
| Aggregate (30) | catalog | 100.00% | 2.77% | 46.42% |
|
|
239
|
+
|
|
240
|
+
Corrected targeted micro-recall is **100/101 = 0.9901** (mean recall 0.9917).
|
|
241
|
+
Its only missing truth fact is `component:default/ingest` in b-04. Search
|
|
242
|
+
micro-recall is **85/101 = 0.8416** (mean recall 0.8851); catalog micro- and
|
|
243
|
+
mean recall are both 1.0. These averages answer different questions and should
|
|
244
|
+
not be interchanged.
|
|
245
|
+
|
|
246
|
+
Search matches catalog recall in 21 scenarios: koda-04, koda-07, a-02, a-03,
|
|
247
|
+
b-01, b-03, b-04, c-01, c-02, c-03, c-04, e-01, e-02, e-03, e-04, f-01,
|
|
248
|
+
f-02, f-03, f-04, f-05 and f-06. It exceeds catalog recall in none.
|
|
249
|
+
|
|
250
|
+
The label correction removes the old recall gap against targeted retrieval.
|
|
251
|
+
Search recovers all expected facts in many import-related changes, but misses
|
|
252
|
+
some catalog identity/placement and authority facts. Its broad fixed point
|
|
253
|
+
also reads unrelated context, especially through common endpoint strings in
|
|
254
|
+
Koda. These numbers compare deterministic retrieval rules on small authored
|
|
255
|
+
corpora; they do not establish an agent productivity or token-saving benefit,
|
|
256
|
+
or an upper bound on search. Noise is judged by the catalog's graph closure,
|
|
257
|
+
so it is not independent evidence that every extra source fact is irrelevant.
|
|
258
|
+
|
|
259
|
+
`recallNotWorseThanTargetedBaseline` still **passes**: catalog and corrected
|
|
260
|
+
targeted recall tie in five corpora; catalog is higher in Go. Every other
|
|
261
|
+
applicable pre-registered gate passes. No threshold or scenario changed and
|
|
262
|
+
search has no new gate. Normalizing the vocabulary changes noise and deduplicates
|
|
263
|
+
fact counts, so old noise/size figures are not directly comparable. The largest
|
|
264
|
+
full inventory now has 86 facts (Koda), below the unchanged 100-fact floor.
|
|
265
|
+
Aggregate retrieval reduction is therefore **not measured** (`null`), with no
|
|
266
|
+
corpus included in that aggregate. Per-corpus raw ratios remain reported.
|
|
267
|
+
|
|
85
268
|
## Thresholds
|
|
86
269
|
|
|
87
270
|
Pre-registered in `evals/thresholds.json`, dated, and not moved since:
|
|
@@ -188,9 +371,18 @@ Each was fixed in the engine, not by relaxing a threshold.
|
|
|
188
371
|
- Ground truth is hand-authored per scenario. It is written from the
|
|
189
372
|
repository's own manifests and contracts, not from the system's output, but it
|
|
190
373
|
is still a human judgement about what matters.
|
|
191
|
-
-
|
|
192
|
-
|
|
193
|
-
|
|
374
|
+
- Search is one explicit model of exploration, not a record of an agent
|
|
375
|
+
session or an upper bound on what search can find. Its import and endpoint
|
|
376
|
+
closure deliberately errs toward reading more; different heuristics can
|
|
377
|
+
produce different recall and noise.
|
|
378
|
+
- The targeted baseline's old label mismatch is corrected. It still reads the
|
|
379
|
+
compiled graph, so its near-perfect recall does not measure unaided search.
|
|
380
|
+
The former 100%-versus-58% comparison remains withdrawn; see
|
|
381
|
+
[BENCHMARK.md](BENCHMARK.md#correction-the-fact-recall-comparison-is-withdrawn).
|
|
382
|
+
- Noise is judged by the catalog's one-hop graph closure, including for search.
|
|
383
|
+
Useful source facts absent from that graph can count as noise, and legitimate
|
|
384
|
+
transitive importer context can fall outside one hop. This is not an
|
|
385
|
+
independent ground truth for irrelevant work or an agent time/token metric.
|
|
194
386
|
- `retrievalReduction` is a fact-count proxy, not a token count. It does not
|
|
195
387
|
weight a long constraint file against a short ref.
|
|
196
388
|
- Scenarios touch one or two files. A change spanning twenty files is not
|
package/docs/MIGRATION.md
CHANGED
|
@@ -164,9 +164,27 @@ agentdoc check
|
|
|
164
164
|
```
|
|
165
165
|
|
|
166
166
|
`init` writes `agentdoc/agentdoc.config.yaml`, the four central descriptor
|
|
167
|
-
files, `agentdoc/journeys.yaml`, the documentation hierarchy
|
|
168
|
-
|
|
169
|
-
|
|
167
|
+
files, `agentdoc/journeys.yaml`, the documentation hierarchy, the observations
|
|
168
|
+
README, and a CI workflow. It never overwrites an existing file without
|
|
169
|
+
`--force`, and reports every path it wrote or skipped.
|
|
170
|
+
|
|
171
|
+
The documentation hierarchy goes where the configuration says. When
|
|
172
|
+
`agentdoc/agentdoc.config.yaml` already exists, `init` and `scaffold --write`
|
|
173
|
+
write `PRODUCT.md`, `ARCHITECTURE.md`, `CONSTRAINTS.md` and the ADR README to
|
|
174
|
+
the configured `docs.product`, `docs.architecture`, `docs.constraints` and
|
|
175
|
+
`docs.adrDir`. A role the configuration leaves out is not written anywhere and
|
|
176
|
+
is listed under `notConfigured`. Without a configuration, `init` uses
|
|
177
|
+
`agentdoc/docs/PRODUCT.md`, `agentdoc/docs/ARCHITECTURE.md`,
|
|
178
|
+
`agentdoc/docs/CONSTRAINTS.md` and `agentdoc/docs/adr/`, and records those
|
|
179
|
+
paths in the configuration it writes.
|
|
180
|
+
|
|
181
|
+
Authored AgentDoc material lives in `agentdoc/`; generated output lives in
|
|
182
|
+
`.agentdoc/`. `docs/` stays the product's own user documentation, which is
|
|
183
|
+
often published as-is (MkDocs, Docusaurus, GitHub Pages). To keep the skeleton
|
|
184
|
+
in `docs/`, or anywhere else, declare `docs.product`, `docs.architecture`,
|
|
185
|
+
`docs.constraints` and `docs.adrDir` in `agentdoc/agentdoc.config.yaml` before
|
|
186
|
+
running `init` or `scaffold --write`. A configuration that already declares
|
|
187
|
+
`docs.*` keeps its paths, `docs/...` included, and nothing ever moves an
|
|
170
188
|
existing file.
|
|
171
189
|
|
|
172
190
|
`scaffold` is a dry run unless `--write` is passed.
|
|
@@ -239,7 +257,7 @@ worth discovering before documenting it.
|
|
|
239
257
|
| `PRODUCT.md` | what it is, for whom, what it does not do | describe components |
|
|
240
258
|
| `ARCHITECTURE.md` | why the shape is what it is | enumerate components or contracts |
|
|
241
259
|
| `CONSTRAINTS.md` | repository-wide rules, one rule one reason one check | restate a component's constraints |
|
|
242
|
-
|
|
|
260
|
+
| `<docs.adrDir>/NNNN-*.md` | one decision, its consequences, superseded not edited | be a design document |
|
|
243
261
|
| `<component>/CONSTRAINTS.md` | rules for that component only | duplicate a repository-wide rule |
|
|
244
262
|
| `<component>/README.md` | how to run it, what it is for | restate the catalog |
|
|
245
263
|
| runbooks | operational procedure | contain architectural decisions |
|
package/docs/PROVENANCE.md
CHANGED
|
@@ -89,6 +89,12 @@ record is complete by construction rather than by remembering to register files.
|
|
|
89
89
|
A change to any input changes the hash; `check` refuses a graph whose hash
|
|
90
90
|
disagrees.
|
|
91
91
|
|
|
92
|
+
Paths git ignores are not inputs (SPEC §7.2): the tracked read path, probes and
|
|
93
|
+
listings all treat them as absent, so local build output in an ignored
|
|
94
|
+
directory cannot move the hash. Tracked files always count, even when they
|
|
95
|
+
match an ignore pattern. Outside git the input set is the filesystem minus
|
|
96
|
+
`discovery.generatedDirs`.
|
|
97
|
+
|
|
92
98
|
## Secrets
|
|
93
99
|
|
|
94
100
|
Two independent layers, because derivation can synthesise a secret-shaped string
|
package/docs/QUERY.md
CHANGED
|
@@ -15,12 +15,29 @@ Resolution order:
|
|
|
15
15
|
1. an exact entity ref;
|
|
16
16
|
2. a documentation or constraint file a descriptor links, even outside the
|
|
17
17
|
component's source root;
|
|
18
|
-
3.
|
|
19
|
-
|
|
20
|
-
|
|
18
|
+
3. the component whose source root most specifically owns the path: a
|
|
19
|
+
directory that is exactly a unit root, or else the longest unit root
|
|
20
|
+
containing it;
|
|
21
|
+
4. a canonical contract file, which routes to the API entity that claims it;
|
|
22
|
+
5. a root component (a descriptor at the repository root, `sourcePath: "."`),
|
|
23
|
+
which owns every file no step above claims.
|
|
24
|
+
|
|
25
|
+
Paths are compared in one canonical, repository-relative form, the one git uses
|
|
26
|
+
for pathspecs and `ls-files`: `./vite.config.ts`, `vite.config.ts` and
|
|
27
|
+
`tooling//config.ts` are the same paths as `vite.config.ts` and
|
|
28
|
+
`tooling/config.ts`. Ownership is decided on whole path segments, never on a
|
|
29
|
+
raw string prefix, so a unit at `src` owns `src/x.ts` and never `src2/x.ts`.
|
|
30
|
+
When several unit roots contain a path, the longest one wins, as in a routing
|
|
31
|
+
table or a CODEOWNERS file. The repository root is the shortest possible root,
|
|
32
|
+
so a nested component always takes precedence over a root component. A path
|
|
33
|
+
that is absolute or climbs out of the repository with `..` resolves to nothing.
|
|
34
|
+
A needle shaped like an entity ref (`kind:namespace/name`) that names no entity
|
|
35
|
+
is unresolved; it is never treated as a file a root component owns.
|
|
21
36
|
|
|
22
37
|
If nothing resolves, the command fails with `AGENTDOC_REF_UNRESOLVED` and exits
|
|
23
38
|
non-zero. It never falls back to "the nearest component" or to a global dump.
|
|
39
|
+
Without a root descriptor, a file outside every unit root stays unresolved: the
|
|
40
|
+
repository root is owned only when a descriptor claims it.
|
|
24
41
|
|
|
25
42
|
## Freshness refusal
|
|
26
43
|
|
|
@@ -191,12 +208,38 @@ contracts, documentation, resources, verification and catalog sources; whether
|
|
|
191
208
|
the graph will be invalidated; the recommended verification commands; any open
|
|
192
209
|
conflicts on the affected subjects; and related journeys.
|
|
193
210
|
|
|
211
|
+
Changed paths resolve to components with the same rules as `query`: a `./`
|
|
212
|
+
prefix is ignored, the longest owning unit root wins, and a root component
|
|
213
|
+
receives the files no other component owns. A file nothing owns is listed in
|
|
214
|
+
`unmatchedFiles`.
|
|
215
|
+
|
|
194
216
|
Reachability rules: a contract change reaches its provider and every consumer;
|
|
195
217
|
a Resource change reaches components bound to that exact Resource. A migration
|
|
196
|
-
change
|
|
197
|
-
|
|
198
|
-
`
|
|
199
|
-
|
|
218
|
+
change is a Resource change. Which Resource is decided from the graph, most
|
|
219
|
+
specific directory first, by whole path segment (`drizzle` never matches
|
|
220
|
+
`drizzle-extra/`):
|
|
221
|
+
|
|
222
|
+
1. **Declared.** Under a path a logical-database Resource lists in
|
|
223
|
+
`spec.migrationPaths`: that Resource, whichever component owns the file and
|
|
224
|
+
whatever that component is bound to. An authored declaration beats a
|
|
225
|
+
detected directory.
|
|
226
|
+
2. **Detected.** Under a directory the `database` adapter recorded in a
|
|
227
|
+
component's `derived.migrationDirs` (a `migrations` directory, Alembic
|
|
228
|
+
`alembic/versions`, Drizzle Kit's `out`): the logical databases bound to
|
|
229
|
+
that component through existing `usesResource` / `resourceUsedBy`
|
|
230
|
+
relations. The changed path is listed too.
|
|
231
|
+
3. **By name.** Under a `migrations/`, `migration/` or `alembic/versions/`
|
|
232
|
+
segment the graph does not record (the `database` adapter is off): the same
|
|
233
|
+
as detected, from the enclosing component.
|
|
234
|
+
|
|
235
|
+
Every component bound to a reached Resource is affected, with the reason
|
|
236
|
+
`resource change touches <ref>`. When the directory was declared or found
|
|
237
|
+
through an ORM config, a second reason names the migration:
|
|
238
|
+
`migration drizzle/0001_x.sql changes resource:default/app-db (migration
|
|
239
|
+
directory drizzle, detected from drizzle-config in component:default/web)`.
|
|
240
|
+
Migration authority alone never establishes ownership or invents a Resource
|
|
241
|
+
binding: a detected directory whose component has no binding reaches no
|
|
242
|
+
Resource, and a declaration routes changes without creating a relation.
|
|
200
243
|
|
|
201
244
|
A component change reaches its reverse build/test dependency closure, plus
|
|
202
245
|
direct runtime-call and schedule peers. Dependencies are not affected just
|
package/docs/SPEC.md
CHANGED
|
@@ -77,6 +77,47 @@ authored in exactly one file, and the location is validated.
|
|
|
77
77
|
The namespace comes from the configuration, so two repositories in one graph
|
|
78
78
|
never collide.
|
|
79
79
|
|
|
80
|
+
### Resource migration paths
|
|
81
|
+
|
|
82
|
+
A `logical-database` Resource may list the repository directories that hold its
|
|
83
|
+
schema migrations:
|
|
84
|
+
|
|
85
|
+
```yaml
|
|
86
|
+
apiVersion: agentdoc.dev/v1
|
|
87
|
+
kind: Resource
|
|
88
|
+
metadata:
|
|
89
|
+
name: app-postgres
|
|
90
|
+
description: "Primary Postgres database of the application."
|
|
91
|
+
spec:
|
|
92
|
+
type: logical-database
|
|
93
|
+
system: app
|
|
94
|
+
migrationPaths:
|
|
95
|
+
- frontend/apps/web/drizzle
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Each entry is a repository path matched as a whole-segment prefix (the rules
|
|
99
|
+
of `core/paths.mjs`): `frontend/apps/web/drizzle` covers every file below it,
|
|
100
|
+
including `meta/_journal.json`, and never `frontend/apps/web/drizzle-extra/`.
|
|
101
|
+
`impact` treats a change under it as a change to this Resource, whichever
|
|
102
|
+
component owns the file, and reaches every component bound to the Resource.
|
|
103
|
+
The declaration is AUTHORED: it beats a migration directory the `database`
|
|
104
|
+
adapter detects, and it is pointed at by its own provenance subject
|
|
105
|
+
(`/entities/<i>/entity/spec/migrationPaths`). It never creates a
|
|
106
|
+
`usesResource` relation; which components use the database is still derived.
|
|
107
|
+
|
|
108
|
+
Validation, at load time:
|
|
109
|
+
|
|
110
|
+
| case | code |
|
|
111
|
+
|---|---|
|
|
112
|
+
| absolute, backslash, `.` or `..` segment, whitespace | `AGENTDOC_PATH_ESCAPE` |
|
|
113
|
+
| a path that does not exist (or is git-ignored) | `AGENTDOC_PATH_UNRESOLVED` |
|
|
114
|
+
| a glob (`*`, `?`, `[`, `{`, `!`): declare the directory itself | `AGENTDOC_MIGRATION_PATH` |
|
|
115
|
+
| the field on a Resource whose type is not `logical-database` | `AGENTDOC_MIGRATION_PATH` |
|
|
116
|
+
| the same path declared by two Resources | `AGENTDOC_MIGRATION_PATH` |
|
|
117
|
+
|
|
118
|
+
Nested declarations on different Resources are allowed; the longest prefix
|
|
119
|
+
wins.
|
|
120
|
+
|
|
80
121
|
### Why no further entities
|
|
81
122
|
|
|
82
123
|
A `Team`, `Milestone` or `Tag` entity was considered and rejected: none of them
|
|
@@ -142,6 +183,7 @@ least one provenance id and an evidence class.
|
|
|
142
183
|
```
|
|
143
184
|
schemaVersion, source{commit, dirty, inputHash}, compiler{name, version, adapters}
|
|
144
185
|
entities[] ref, kind, name, entity, source, sourcePaths[], derived?
|
|
186
|
+
(derived.migrationDirs[]: path, via[], provenanceIds[])
|
|
145
187
|
relations[] type, sourceRef, targetRef, evidenceClass, attributes, provenanceIds[]
|
|
146
188
|
interfaces.events[] pattern, producers, consumers, contract, resolution
|
|
147
189
|
externalDependencies[] componentRef, name, role, mechanism, evidenceClass
|
|
@@ -195,6 +237,34 @@ Every byte flows through the read tracker, so the record is complete by
|
|
|
195
237
|
construction. A consumer must refuse a graph whose `inputHash` or `dirty` flag
|
|
196
238
|
disagrees with the current checkout. There is no fallback.
|
|
197
239
|
|
|
240
|
+
**The input set is the checkout as git sees it.** Reads, probes, listings and
|
|
241
|
+
walks — and therefore discovery as well as the hash — never see a path git
|
|
242
|
+
ignores: a local build writing into an ignored directory (`.astro/`, `build/`,
|
|
243
|
+
`*.log`) changes nothing, and a graph compiled after that build is
|
|
244
|
+
byte-identical to the one a clean checkout or CI produces. An ignored path is
|
|
245
|
+
treated as absent: probed as `false`, left out of listings and walks, and a
|
|
246
|
+
tracked read of it fails like a read of a missing file. Git is the oracle
|
|
247
|
+
(`git status --ignored=matching`, one call per compile, shared with the dirty
|
|
248
|
+
check), so its documented semantics apply exactly:
|
|
249
|
+
|
|
250
|
+
- a tracked file stays an input even when it matches an ignore pattern;
|
|
251
|
+
- nested `.gitignore` files, `$GIT_DIR/info/exclude` and the user's
|
|
252
|
+
`core.excludesFile` apply with gitignore(5) precedence, negations included,
|
|
253
|
+
and a file whose parent directory is excluded cannot be re-included;
|
|
254
|
+
- `core.excludesFile` counts on purpose. An ignored untracked file is absent
|
|
255
|
+
from every clean checkout whatever rule ignored it, so excluding it only ever
|
|
256
|
+
brings the local input set closer to CI's — and it keeps the hash in step
|
|
257
|
+
with `git status`, which drives `dirty`;
|
|
258
|
+
- an ignored *ancestor* of the project root (a project nested in a larger
|
|
259
|
+
checkout) is not honoured: the directory the compiler is pointed at is
|
|
260
|
+
compiled whole.
|
|
261
|
+
|
|
262
|
+
`discovery.generatedDirs` (or its built-in list) is excluded in addition, in a
|
|
263
|
+
git checkout or not. Without git — no repository, no git binary — there is no
|
|
264
|
+
oracle: the input set is the filesystem minus those directories, and
|
|
265
|
+
`.gitignore` files are not interpreted. `check` recomputes the hash by running
|
|
266
|
+
the same compile, so compile and check cannot disagree on the input set.
|
|
267
|
+
|
|
198
268
|
The recorded `commit` is **provenance, not a gate**: committing the graph changes
|
|
199
269
|
`HEAD`, so a commit-equality check could never be satisfied by a committed graph.
|
|
200
270
|
What determines whether the graph still describes the repository is the input
|
|
@@ -235,7 +305,9 @@ that references the contract file from a source or build-configuration file.
|
|
|
235
305
|
|
|
236
306
|
The core has no knowledge of folder names. A directory is a unit when it holds
|
|
237
307
|
a component descriptor, when an adapter recognises it from a workspace file, or
|
|
238
|
-
when it is listed in `discovery.supplementalRoots`.
|
|
308
|
+
when it is listed in `discovery.supplementalRoots`. Discovery sees only the
|
|
309
|
+
input set of §7.2: a descriptor or manifest under a git-ignored path is not
|
|
310
|
+
discovered.
|
|
239
311
|
|
|
240
312
|
`init` configures the recursive component descriptor glob `*/**/agentdoc.yaml`,
|
|
241
313
|
so descriptors for adapter-discovered units and units added later are loaded
|
|
@@ -291,7 +363,7 @@ only the codes in `ACCEPTABLE_WARNING_CODES` may be accepted.
|
|
|
291
363
|
| command | contract |
|
|
292
364
|
|---|---|
|
|
293
365
|
| `init [--force]` | write a configuration and documentation skeleton; never clobber unless `--force` |
|
|
294
|
-
| `scaffold [--write]` | generate descriptors from provable evidence; dry run by default |
|
|
366
|
+
| `scaffold [--write]` | generate descriptors from provable evidence; dry run by default; never clobbers |
|
|
295
367
|
| `audit` | classify every fact; runs even when the repository cannot compile |
|
|
296
368
|
| `validate` | sources, schemas, semantic checks; no artifact written |
|
|
297
369
|
| `compile [--require-clean]` | deterministic build, atomic replace |
|
|
@@ -302,6 +374,21 @@ only the codes in `ACCEPTABLE_WARNING_CODES` may be accepted.
|
|
|
302
374
|
| `pin-evidence [--json]` | check the digests pinned by reviewed overrides |
|
|
303
375
|
| `doctor` | environment self-check: node version, schema bundle, git, writability |
|
|
304
376
|
|
|
377
|
+
`init` and `scaffold` write the documentation skeleton to the paths an
|
|
378
|
+
existing configuration declares in `docs.product`, `docs.architecture`,
|
|
379
|
+
`docs.constraints` and `docs.adrDir`, read through the same loader as
|
|
380
|
+
`compile` and `audit`. A role the configuration omits is not written and is
|
|
381
|
+
reported in `notConfigured`; nothing falls back to a default. Only when no
|
|
382
|
+
configuration exists does `init` use the defaults `agentdoc/docs/PRODUCT.md`,
|
|
383
|
+
`agentdoc/docs/ARCHITECTURE.md`, `agentdoc/docs/CONSTRAINTS.md` and
|
|
384
|
+
`agentdoc/docs/adr/`, and record them in the `docs:` block of the configuration
|
|
385
|
+
it writes. Authored AgentDoc material lives in `agentdoc/`; generated output
|
|
386
|
+
lives in `.agentdoc/`. `docs/` is left to the product's own user documentation:
|
|
387
|
+
`init` and `scaffold` write there only when a configuration declares `docs.*`
|
|
388
|
+
paths under it, and they never move an existing file. A target outside the
|
|
389
|
+
repository, including through a symlink, fails with `AGENTDOC_PATH_ESCAPE`
|
|
390
|
+
before anything is written.
|
|
391
|
+
|
|
305
392
|
Exit codes: `0` ok, `1` failure, `2` usage.
|
|
306
393
|
|
|
307
394
|
## 13. Error codes
|
|
@@ -21,7 +21,7 @@ jobs:
|
|
|
21
21
|
with: { node-version: '24' }
|
|
22
22
|
- name: Install the agentdoc CLI
|
|
23
23
|
# Unpinned tracks main. For CI reproducibility pin a tag or commit:
|
|
24
|
-
# github:lbframe/agent-doc-system#v1.
|
|
24
|
+
# github:lbframe/agent-doc-system#v1.4.0
|
|
25
25
|
# A compiler upgrade intentionally invalidates committed graphs.
|
|
26
26
|
run: npm install -g github:lbframe/agent-doc-system
|
|
27
27
|
- name: Validate and compile
|