agent-doc-system 0.0.0-stage → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (117) hide show
  1. package/INSTALL_FOR_AGENTS.md +60 -0
  2. package/LICENSE +21 -0
  3. package/README.md +151 -2
  4. package/adapters/cache.mjs +27 -0
  5. package/adapters/contract-consumers.mjs +56 -0
  6. package/adapters/database.mjs +116 -0
  7. package/adapters/deploy-config.mjs +34 -0
  8. package/adapters/dockerfile.mjs +26 -0
  9. package/adapters/events.mjs +145 -0
  10. package/adapters/generic.mjs +86 -0
  11. package/adapters/github-actions.mjs +106 -0
  12. package/adapters/go.mjs +91 -0
  13. package/adapters/graphql.mjs +28 -0
  14. package/adapters/index.mjs +55 -0
  15. package/adapters/node.mjs +118 -0
  16. package/adapters/object-store.mjs +64 -0
  17. package/adapters/oidc.mjs +56 -0
  18. package/adapters/openapi.mjs +50 -0
  19. package/adapters/pnpm.mjs +42 -0
  20. package/adapters/protobuf.mjs +35 -0
  21. package/adapters/registry.mjs +27 -0
  22. package/adapters/resource-client.mjs +61 -0
  23. package/adapters/turborepo.mjs +135 -0
  24. package/adapters/typescript.mjs +85 -0
  25. package/adapters/wrangler.mjs +206 -0
  26. package/bin/agentdoc.mjs +478 -0
  27. package/bin/usage.txt +26 -0
  28. package/core/acceptances.mjs +69 -0
  29. package/core/audit.mjs +407 -0
  30. package/core/authority/classes.mjs +39 -0
  31. package/core/authority/engine.mjs +243 -0
  32. package/core/authority/facts.mjs +41 -0
  33. package/core/codes.mjs +119 -0
  34. package/core/compile.mjs +881 -0
  35. package/core/contracts.mjs +121 -0
  36. package/core/derive.mjs +102 -0
  37. package/core/descriptors.mjs +486 -0
  38. package/core/determinism.mjs +95 -0
  39. package/core/discover.mjs +147 -0
  40. package/core/facts.mjs +53 -0
  41. package/core/fsx.mjs +342 -0
  42. package/core/graph.mjs +260 -0
  43. package/core/impact.mjs +175 -0
  44. package/core/indexes.mjs +165 -0
  45. package/core/jsonschema.mjs +298 -0
  46. package/core/observations.mjs +129 -0
  47. package/core/provenance.mjs +83 -0
  48. package/core/query.mjs +384 -0
  49. package/core/relations.mjs +286 -0
  50. package/core/scaffold.mjs +322 -0
  51. package/core/secrets.mjs +92 -0
  52. package/core/sourcescan.mjs +88 -0
  53. package/core/toml.mjs +163 -0
  54. package/core/yaml.mjs +475 -0
  55. package/docs/ADAPTERS.md +172 -0
  56. package/docs/AUTHORITY.md +260 -0
  57. package/docs/CI.md +136 -0
  58. package/docs/COMPARISON-KODA.md +164 -0
  59. package/docs/EVALS.md +166 -0
  60. package/docs/MIGRATION.md +193 -0
  61. package/docs/PROVENANCE.md +133 -0
  62. package/docs/QUERY.md +180 -0
  63. package/docs/SPEC.md +339 -0
  64. package/docs/VERIFICATION.md +28 -0
  65. package/docs/ci/generic-ci.sh +28 -0
  66. package/docs/ci/github-actions.yml +35 -0
  67. package/evals/harness.mjs +426 -0
  68. package/evals/scenarios/fixture-a-01-change-api.json +9 -0
  69. package/evals/scenarios/fixture-a-02-change-db.json +9 -0
  70. package/evals/scenarios/fixture-a-03-enforce-constraint.json +9 -0
  71. package/evals/scenarios/fixture-b-01-change-proto.json +9 -0
  72. package/evals/scenarios/fixture-b-02-change-write-path.json +9 -0
  73. package/evals/scenarios/fixture-b-03-change-shared-lib.json +9 -0
  74. package/evals/scenarios/fixture-b-04-journey.json +9 -0
  75. package/evals/scenarios/fixture-c-01-mixed-runtime-edge.json +19 -0
  76. package/evals/scenarios/fixture-c-02-stale-doc-contradiction.json +9 -0
  77. package/evals/scenarios/fixture-c-03-shared-vocabulary.json +9 -0
  78. package/evals/scenarios/fixture-c-04-contract-change.json +9 -0
  79. package/evals/scenarios/koda-01-modify-api-consumer.json +26 -0
  80. package/evals/scenarios/koda-02-change-db-schema.json +20 -0
  81. package/evals/scenarios/koda-03-change-auth.json +21 -0
  82. package/evals/scenarios/koda-04-add-event-producer.json +19 -0
  83. package/evals/scenarios/koda-05-modify-worker-binding.json +25 -0
  84. package/evals/scenarios/koda-06-change-cross-component-endpoint.json +22 -0
  85. package/evals/scenarios/koda-07-change-shared-package.json +16 -0
  86. package/evals/scenarios/koda-08-change-runtime-cron.json +23 -0
  87. package/evals/scenarios/koda-09-debug-production-mismatch.json +26 -0
  88. package/evals/thresholds.json +11 -0
  89. package/package.json +46 -4
  90. package/schemas/api.schema.json +84 -0
  91. package/schemas/common.schema.json +107 -0
  92. package/schemas/component.schema.json +40 -0
  93. package/schemas/config.schema.json +685 -0
  94. package/schemas/domain.schema.json +14 -0
  95. package/schemas/graph.schema.json +1084 -0
  96. package/schemas/journeys.schema.json +47 -0
  97. package/schemas/observation.schema.json +143 -0
  98. package/schemas/resource.schema.json +28 -0
  99. package/schemas/system.schema.json +26 -0
  100. package/skills/agent-doc-system/SKILL.md +113 -0
  101. package/skills/agent-doc-system/references/authority-and-conflicts.md +70 -0
  102. package/skills/agent-doc-system/references/cli.md +89 -0
  103. package/skills/agent-doc-system/references/create-migrate.md +101 -0
  104. package/skills/agent-doc-system/references/maintenance.md +52 -0
  105. package/templates/ADR.md +26 -0
  106. package/templates/ARCHITECTURE.md +19 -0
  107. package/templates/CONSTRAINTS.md +17 -0
  108. package/templates/PRODUCT.md +17 -0
  109. package/templates/api.yaml +10 -0
  110. package/templates/ci.yml +31 -0
  111. package/templates/component.yaml +14 -0
  112. package/templates/domain.yaml +6 -0
  113. package/templates/journey.yaml +7 -0
  114. package/templates/observation.yaml +19 -0
  115. package/templates/resource.yaml +8 -0
  116. package/templates/system.yaml +10 -0
  117. package/templates/warning-acceptance.yaml +14 -0
package/docs/EVALS.md ADDED
@@ -0,0 +1,166 @@
1
+ # EVALS
2
+
3
+ Does routing an agent through this system retrieve the *right* context, and less
4
+ noise than exploring the repository unaided?
5
+
6
+ ## What is measured
7
+
8
+ The question is not "does it compile". A catalog can compile perfectly and still
9
+ route an agent into the wrong corner of the repository. The measurement is
10
+ retrieval quality, on the same scenarios, for two retrieval systems.
11
+
12
+ **baseline-targeted** — what an agent that has no catalog and looks only where
13
+ the change is: the touched component, its subtree, its relations, its documents
14
+ and its checks.
15
+
16
+ **baseline-full** — what an agent with no routing ends up holding: every entity,
17
+ every relation, every event subject, every check, every document in the
18
+ repository. This is the honest cost of "just read the repository", and it is the
19
+ size yardstick for reduction.
20
+
21
+ **catalog** — what `agentdoc query` returns for the touched paths, plus the
22
+ context of the components `impact` says are affected.
23
+
24
+ No model is involved. Retrieval sets are compared as sets, so the measurement
25
+ is deterministic and cannot be moved by phrasing.
26
+
27
+ ## Metrics
28
+
29
+ | metric | definition |
30
+ |---|---|
31
+ | recall | fraction of the scenario's ground-truth facts the catalog retrieved |
32
+ | critical recall | the same, restricted to critical context: constraints, conflicts, observations, resources, API boundaries, event subjects |
33
+ | verification recall | fraction of the checks that must be run, that were returned |
34
+ | precision | `1 - irrelevance` |
35
+ | irrelevance (noise) | fraction of retrieved facts outside the graph's one-hop closure of the touched entities |
36
+ | retrieval reduction | `1 - catalog facts / baseline-full facts` |
37
+
38
+ ### Why noise is measured against reachability, not against the truth set
39
+
40
+ The first definition of noise was the complement of truth recall. It is
41
+ degenerate: a scenario lists the handful of facts that matter, so any
42
+ *additional legitimate* fact — a sibling constraint, a second check, an adjacent
43
+ contract — is scored as noise. The metric rewarded returning almost nothing.
44
+
45
+ It was replaced, before any repository other than the golden corpus was
46
+ measured, with a question that does not depend on the truth set at all:
47
+
48
+ > did routing drag in anything the graph itself says is unrelated?
49
+
50
+ Relevance is the **one-hop** graph closure of the touched entities, computed
51
+ from the compiled graph. The metric therefore cannot be satisfied by inflating a
52
+ truth set, and it is not satisfied by returning less either — that shows up as
53
+ recall. Recall is still measured against the truth sets, unchanged.
54
+
55
+ "One hop" is load-bearing and is enforced. An earlier implementation took the
56
+ transitive closure, which on the golden corpus grows to nearly the whole
57
+ repository; a router that returned *every fact in the repository* then scored as
58
+ precise, because everything was technically adjacent to something. The metric
59
+ was measuring its own closure, not the router.
60
+
61
+ ## Thresholds
62
+
63
+ Pre-registered in `evals/thresholds.json`, dated, and not moved since:
64
+
65
+ ```json
66
+ { "minRecall": 0.9, "minCriticalRecall": 0.95, "minVerificationRecall": 0.8,
67
+ "minPrecision": 0.75, "maxNoise": 0.6, "minRetrievalReduction": 0.4,
68
+ "minRecallOverBaseline": 0.9 }
69
+ ```
70
+
71
+ Two additional gates use the same numbers:
72
+
73
+ - **recall must not be worse than baseline-targeted.** A catalog that retrieves
74
+ less than reading the component would be worthless.
75
+ - **recall must be at least 0.9 × baseline-full recall.** Routing must not be
76
+ meaningfully worse than reading everything.
77
+
78
+ ### Where the gate is applied
79
+
80
+ - **Aggregate** (`node evals/run-all.mjs`) applies every threshold,
81
+ **and fails if any single corpus fails or is missing**. A corpus that is
82
+ missing fails the gate outright: silently scoring three repositories and
83
+ reporting PASS would be a gate that shrinks when the system shrinks. The two
84
+ relative-baseline checks are evaluated per corpus and folded into the
85
+ aggregate's failure list; the aggregate does not re-derive them, so they are
86
+ lost if a corpus never runs — which is why a missing corpus is itself a
87
+ failure. A per-corpus failure is folded into the aggregate exit
88
+ code, so a broken fixture cannot print FAIL and still exit 0. This is the
89
+ system-level claim.
90
+ - **Per repository** (`agentdoc eval routing`) applies every threshold except
91
+ retrieval reduction, which is reported as *not measured* when the whole-repo
92
+ baseline holds fewer than 100 facts. A one-component repository has almost
93
+ nothing to route away from; failing it would be punishing the fixture for
94
+ being small. The floor and the reasoning are in `evals/harness.mjs`.
95
+
96
+ ## Corpora and scenarios
97
+
98
+ | corpus | scenarios | what it proves |
99
+ |---|---|---|
100
+ | `examples/koda` (golden corpus) | 9 | monorepo discovery, TypeScript + Go, `apps`/`packages`/`workers`, OpenAPI, OIDC, runtime calls, resources, events, verification, journeys, warning acceptances, authority conflicts |
101
+ | `fixtures/a-node-ts-postgres` | 3 | single Node/TypeScript service, REST, Postgres, GitHub Actions |
102
+ | `fixtures/b-go-multi-service` | 4 | multi-service Go, protobuf contracts, shared library, resources |
103
+ | `fixtures/c-alt-monorepo` | 4 | a layout that is deliberately not `apps`+`packages`+`workers`, mixed runtimes, inconsistent pre-existing documentation |
104
+
105
+ Scenarios are repository-scoped: a scenario only runs against the corpus it was
106
+ written for, declared by a `repository` tag in a `.agentdoc-fixture.json` at the
107
+ repository root. Expectations from one fixture can never leak into another's
108
+ score.
109
+
110
+ ### Scenario families
111
+
112
+ The golden corpus covers, in order: modify one API consumer; change a database
113
+ schema; change authentication; add an event producer; modify a worker binding;
114
+ change a cross-component endpoint; change a shared package; change a runtime
115
+ cron; debug a production mismatch.
116
+
117
+ The synthetic fixtures add: change an API contract; change a write path; change
118
+ the only-writer rule; break an end-to-end journey; change an edge worker's flush
119
+ schedule; follow stale documentation while debugging; change a subject vocabulary
120
+ shared across runtimes; add an operation to a contract.
121
+
122
+ ## Running it
123
+
124
+ ```bash
125
+ agentdoc eval routing --json # one repository (from a corpus directory)
126
+ node evals/run-all.mjs # aggregate gate — source checkout only
127
+ node evals/run-all.mjs --json # machine-readable
128
+ ```
129
+
130
+ `run-all.mjs` iterates the corpora in `fixtures/` and `examples/`, so it only
131
+ exists in a source checkout; it is not part of the installed CLI's job.
132
+
133
+ ## What the evaluation changed
134
+
135
+ The evaluation is a tool, not a ceremony. Running it against the golden corpus
136
+ exposed real defects that no unit test had:
137
+
138
+ - contract files did not route (changing an endpoint resolved to nothing);
139
+ - a scheduled trigger did not resolve to the component serving the target route,
140
+ so "who does this cron call?" had no answer;
141
+ - event subjects defined in a shared package were not credited to the component
142
+ that published them;
143
+ - a documentation file linked by a descriptor did not route to its component, so
144
+ stale documentation was unreachable — precisely the migration case;
145
+ - the noise metric was degenerate (above), and the `transitiveClosure` variant
146
+ that replaced it briefly was worse;
147
+ - migration directories were matched by a fixed list of framework conventions
148
+ rather than by the stable fact that the directory is called `migrations`;
149
+ - cross-unit resolution depended on adapter execution order;
150
+ - a repository path was resolved against the process working directory, so the
151
+ same commit compiled differently depending on where the compiler ran.
152
+
153
+ Each was fixed in the engine, not by relaxing a threshold.
154
+
155
+ ## Honest limitations
156
+
157
+ - Ground truth is hand-authored per scenario. It is written from the
158
+ repository's own manifests and contracts, not from the system's output, but it
159
+ is still a human judgement about what matters.
160
+ - The baseline is a model of unassisted exploration, not a record of a real
161
+ agent session. It is generous to the baseline by construction (baseline-full
162
+ is everything), which makes the reduction claim conservative.
163
+ - `retrievalReduction` is a fact-count proxy, not a token count. It does not
164
+ weight a long constraint file against a short ref.
165
+ - Scenarios touch one or two files. A change spanning twenty files is not
166
+ measured.
@@ -0,0 +1,193 @@
1
+ # MIGRATION
2
+
3
+ Two workflows. CREATE for a repository with no catalog; MIGRATE for an
4
+ established one. Both start with the same audit; MIGRATE adds a source-of-truth
5
+ mapping and a reviewed-ambiguity list that a human must work through.
6
+
7
+ ## Choosing
8
+
9
+ | situation | mode |
10
+ |---|---|
11
+ | no `agentdoc/` directory, small or new repository | CREATE |
12
+ | no `agentdoc/`, large or established repository | MIGRATE |
13
+ | an existing catalog, documentation that has drifted | MIGRATE |
14
+ | a catalog that is stale but structurally sound | MAINTAIN |
15
+
16
+ The test is not size. It is whether the repository has documentation that
17
+ *claims* things. Claims need verifying; that is migration work regardless of how
18
+ few components there are.
19
+
20
+ ## The audit
21
+
22
+ ```bash
23
+ agentdoc audit
24
+ ```
25
+
26
+ `audit` runs even when the repository cannot compile, and reports compilation
27
+ failures as part of the report. It inventories four sources of truth
28
+ independently:
29
+
30
+ 1. **manifests and code** — what can be derived;
31
+ 2. **machine-readable contracts** — what boundaries are declared;
32
+ 3. **authored documentation** — what someone believed;
33
+ 4. **authoritative external systems** — what is deployed, via ObservationSets.
34
+
35
+ ## The report
36
+
37
+ ```json
38
+ {
39
+ "inventory": { "files", "units", "entities", "components", "apis", "relations", "contracts", "documentationFiles", "observations", "verification" },
40
+ "facts": [ { "fact": "component:default/teacher placement.system", "classification": "DERIVED", "evidence": ["authored"] } ],
41
+ "counts": { "CONFIRMED": 4, "DERIVED": 38 },
42
+ "contradictions": [ ... ],
43
+ "documentationContradictions": [ ... ],
44
+ "reviewedAmbiguity": [ ... ],
45
+ "documentation": { "orphanDocs", "undocumentedComponents", "unverifiedComponents" },
46
+ "contracts": { "inventory", "unclaimed" },
47
+ "observations": [ { "id", "capturedAt", "maxAgeDays", "environment" } ],
48
+ "scaffold": { "files", "summary", "rule" }
49
+ }
50
+ ```
51
+
52
+ ### Classification
53
+
54
+ Every major fact lands in exactly one class:
55
+
56
+ | class | meaning | may be authored? |
57
+ |---|---|---|
58
+ | `CONFIRMED` | two or more independent sources agree | yes |
59
+ | `DERIVED` | deterministically derivable from the repository | yes, once reviewed |
60
+ | `OBSERVED` | read from an authoritative external system | **no** — it belongs in an ObservationSet |
61
+ | `CONFLICT` | sources disagree and no authority rule settles it | **no** — needs a human |
62
+ | `UNRESOLVED` | neither the repository nor an external system answers it | **no** — needs a human |
63
+
64
+ **Uncertainty is never promoted to authored truth.** This is the rule the whole
65
+ report exists to enforce.
66
+
67
+ ### Documentation contradictions
68
+
69
+ Two checks run against claims the graph actually owns, and only those. A document
70
+ saying "we use Kafka" is not checkable until a Resource descriptor says what the
71
+ Kafka thing is; a document saying "there is no edge component" is checkable
72
+ immediately, and is `NEGATED_COMPONENT`. A port claim in a document linked to a
73
+ component is `PORT_MISMATCH` when the source binds a different port, and
74
+ `UNLINKED_PORT_CLAIM` when the document is linked to nothing and therefore cannot
75
+ be checked at all.
76
+
77
+ The rule for extending this: only check vocabulary the graph owns. A check that
78
+ guesses is a check that cries wolf.
79
+
80
+ ### Reviewed ambiguity
81
+
82
+ The list a human must work through:
83
+
84
+ | kind | what a human decides |
85
+ |---|---|
86
+ | `EVENT_WITHOUT_PRODUCER` | is this subject reserved, or is a producer missing? |
87
+ | `COMPONENT_OWNERSHIP` | which system or domain owns this? |
88
+ | `UNCLAIMED_CONTRACT` | which API owns this contract, or is it dead? |
89
+ | `REVIEWED_OVERRIDE` | is this interpretation still true? |
90
+ | `NEGATED_COMPONENT`, `PORT_MISMATCH`, `UNLINKED_PORT_CLAIM` | which source is right? |
91
+
92
+ ## CREATE
93
+
94
+ ```bash
95
+ agentdoc init # configuration + doc skeleton
96
+ $EDITOR agentdoc/agentdoc.config.yaml # layout, namespace, adapters
97
+ agentdoc audit # what is provable
98
+ agentdoc scaffold --write
99
+ $EDITOR '**/agentdoc.yaml' # every generated description
100
+ agentdoc validate
101
+ agentdoc compile
102
+ agentdoc check
103
+ ```
104
+
105
+ `init` writes `agentdoc/agentdoc.config.yaml`, the four central descriptor
106
+ files, `agentdoc/journeys.yaml`, the documentation hierarchy
107
+ (`docs/PRODUCT.md`, `docs/ARCHITECTURE.md`, `docs/CONSTRAINTS.md`,
108
+ `docs/adr/`), the observations README, and a CI workflow. It never overwrites an
109
+ existing file.
110
+
111
+ `scaffold` is a dry run unless `--write` is passed.
112
+
113
+ ### What the scaffolder will and will not do
114
+
115
+ It writes a component descriptor only for a unit it can name from the
116
+ repository — a package name, a module path, or a directory name. It writes a
117
+ description that is a template sentence naming the artifact evidence. It does
118
+ **not** choose a system or a domain, because placement is an ownership decision
119
+ no extractor can make; the generated descriptor carries a commented
120
+ `placementRationale` prompt instead.
121
+
122
+ That is deliberate. A scaffolder that guesses ownership produces a catalog
123
+ nobody trusts.
124
+
125
+ ## MIGRATE
126
+
127
+ ```bash
128
+ agentdoc init
129
+ # set componentDescriptors to the real layout; enable the right adapters
130
+ agentdoc audit > migration-report.json
131
+ ```
132
+
133
+ Then, in order:
134
+
135
+ 1. **Source-of-truth mapping.** For each major claim, record which of the four
136
+ inventories supports it. Anything supported by none goes to
137
+ `reviewedAmbiguity`.
138
+ 2. **Contradiction detection.** Work `contradictions` and
139
+ `documentationContradictions`. Each one needs a human decision: fix the
140
+ documentation, fix the code, or record an authority rule saying which class
141
+ wins for that key.
142
+ 3. **Orphan documentation.** `documentation.orphanDocs`. For each: link it to a
143
+ component via `context.docs`, or delete it. An orphan is a document nothing
144
+ will ever route to.
145
+ 4. **Missing documentation.** `documentation.undocumentedComponents` and
146
+ `documentation.unverifiedComponents`. These are the components an agent will
147
+ get wrong.
148
+ 5. **Stale contracts.** `contracts.unclaimed`, and any contract whose consumers
149
+ changed. Point the API descriptor at the current file; do not duplicate it.
150
+ 6. **Ownership reconstruction.** Placement comes from the code where the code is
151
+ unambiguous (a service that only ever talks to one identity provider), and
152
+ from a human everywhere else.
153
+ 7. **Catalog proposal.** `scaffold` lists what it would write.
154
+ 8. **Migration.** `scaffold --write`, then review every generated file.
155
+ 9. **Validation.** `validate` → `compile` → `check`.
156
+
157
+ ### Do not trust the documentation
158
+
159
+ The single most common migration failure is copying existing prose into
160
+ descriptions. The audit exists to find out how much of that prose is still true.
161
+ A description is a one-sentence present-tense statement of current purpose; if
162
+ you cannot write one, the component's purpose is not understood yet, and that is
163
+ worth discovering before documenting it.
164
+
165
+ ## Documentation hierarchy
166
+
167
+ | kind | belongs there | must not |
168
+ |---|---|---|
169
+ | `PRODUCT.md` | what it is, for whom, what it does not do | describe components |
170
+ | `ARCHITECTURE.md` | why the shape is what it is | enumerate components or contracts |
171
+ | `CONSTRAINTS.md` | repository-wide rules, one rule one reason one check | restate a component's constraints |
172
+ | `docs/adr/NNNN-*.md` | one decision, its consequences, superseded not edited | be a design document |
173
+ | `<component>/CONSTRAINTS.md` | rules for that component only | duplicate a repository-wide rule |
174
+ | `<component>/README.md` | how to run it, what it is for | restate the catalog |
175
+ | runbooks | operational procedure | contain architectural decisions |
176
+ | `agentdoc/observations/` | what a live system said, with evidence | contain credentials |
177
+ | the graph | topology, contracts, events, checks, conflicts | contain prose |
178
+
179
+ **The duplication rule.** A fact lives in exactly one place. Elsewhere, point at
180
+ it. If `ARCHITECTURE.md` lists the components, it will be wrong within a month,
181
+ and `agentdoc query` will be right. The architecture document explains *why*;
182
+ the graph says *what*.
183
+
184
+ ## Finishing
185
+
186
+ ```bash
187
+ agentdoc validate && agentdoc check
188
+ agentdoc eval routing
189
+ git add -A && git commit
190
+ ```
191
+
192
+ Then install CI (`CI.md`) so the next drift is caught by a gate rather than by a
193
+ review.
@@ -0,0 +1,133 @@
1
+ # PROVENANCE
2
+
3
+ Every important fact in the compiled graph is traceable to evidence. This
4
+ document defines what a provenance record is, how it is produced, and the rules
5
+ that keep evidence free of secrets.
6
+
7
+ ## The record
8
+
9
+ ```json
10
+ {
11
+ "id": "p-0075",
12
+ "class": "observed",
13
+ "path": "workers/koda-cron/wrangler.toml",
14
+ "extractor": { "name": "platform-manifest-extractor", "version": "1.0.0" },
15
+ "contentHash": "sha256:9f2c...",
16
+ "location": { "lineStart": 5, "lineEnd": 7, "jsonPointer": "/facts/2" },
17
+ "subjects": ["/entities/9/derived/bindings/0", "/conflicts/2"]
18
+ }
19
+ ```
20
+
21
+ | field | meaning |
22
+ |---|---|
23
+ | `id` | positional over a canonical sort, so ids are stable for a given input set |
24
+ | `class` | how the evidence was obtained (below) |
25
+ | `path` | repository-relative; never absolute, never outside the checkout |
26
+ | `extractor` | which component read it, and at what version |
27
+ | `contentHash` | `sha256:` of the file bytes as read |
28
+ | `location` | optional: line range or JSON pointer, when the evidence is narrower than the file |
29
+ | `subjects` | RFC 6901 pointers into this graph that the record proves |
30
+
31
+ ## Classes
32
+
33
+ | class | meaning | typical extractor |
34
+ |---|---|---|
35
+ | `declared` | the file states the fact outright | `descriptor-reader`, `placement-extractor` |
36
+ | `reviewed` | a human interpretation recorded in configuration, not derivable from a file | `override-extractor` |
37
+ | `observed` | the file shows the fact | `manifest-extractor`, `import-extractor`, `ci-extractor` |
38
+ | `validated` | two independent files agree | `api-contract-extractor`, `oidc-extractor` |
39
+ | `runtime` | read from an external system | `observation:<collector>` |
40
+ | `standard` | the authority is a published standard | `standard-extractor` |
41
+
42
+ `runtime` records are the only ones whose path is not a source file of the
43
+ repository under analysis in the ordinary sense: they point at the committed
44
+ evidence bundle the observation cites.
45
+
46
+ `reviewed` exists so that a human interpretation is not filed as `declared`.
47
+ Filed that way it mapped to `AUTHORED` — the strongest class — so a reviewed
48
+ override corroborated by a source reference was relabelled `AUTHORED` on the
49
+ merged edge, and the graph understated its own highest-authority input. The class
50
+ is not an election either: elections happen per fact in the authority model, and a
51
+ provenance class must not decide one on its own.
52
+
53
+ ## Coverage rules
54
+
55
+ - Every relation, external dependency, capability, verification entry, gate,
56
+ journey, contract, conflict and assertion carries at least one
57
+ `provenanceIds` entry.
58
+ - Every entity carries provenance for its authored fields and, for components,
59
+ for its derived block.
60
+ - A relation, index entry or fact with no provenance is a compiler defect. The
61
+ pipeline fails with `AGENTDOC_PROVENANCE` rather than emitting it.
62
+
63
+ ## Subject resolution
64
+
65
+ Extractors emit symbolic subjects while they run (`rel:pending`,
66
+ `ass:pending`, `evt:<pattern>`, …) and the compiler binds them to concrete
67
+ pointers once the whole graph exists. This is why an extractor's output does not
68
+ depend on how many other extractors ran first.
69
+
70
+ A subject that never binds is an error: `AGENTDOC_PROVENANCE`. The single
71
+ exception is evidence whose candidate edge or fact was *dropped* (an unresolved
72
+ reviewed override, a rejected candidate). Those records are removed and the
73
+ count is reported as `droppedProvenance` in the compile stats, so a regression
74
+ here is visible rather than silent.
75
+
76
+ ## Input hashing
77
+
78
+ `source.inputHash` covers:
79
+
80
+ 1. compiler and graph-schema versions, entity api version, authority-rule count,
81
+ every enabled adapter's name and version, and the content hash of every
82
+ shipped schema;
83
+ 2. every file the compiler read — path and content, length-prefixed;
84
+ 3. every existence probe — path and result;
85
+ 4. every directory listing — directory and sorted file list.
86
+
87
+ Every byte the compiler reads flows through a single tracked read path, so the
88
+ record is complete by construction rather than by remembering to register files.
89
+ A change to any input changes the hash; `check` refuses a graph whose hash
90
+ disagrees.
91
+
92
+ ## Secrets
93
+
94
+ Two independent layers, because derivation can synthesise a secret-shaped string
95
+ that no source scan would have caught.
96
+
97
+ **Layer 1 — sources.** Every authored or observed document is scanned before it
98
+ can influence any output. Refusal is `AGENTDOC_SECRET`, with the JSON pointer.
99
+
100
+ **Layer 2 — outputs.** The serialized graph, every audit report, every query
101
+ payload and every eval report is scanned again before it is written or printed.
102
+ Refusal is `AGENTDOC_SECRET` and the command exits non-zero. Nothing is written
103
+ partially.
104
+
105
+ Detected shapes include private-key headers, cloud access-key ids, provider
106
+ token prefixes, JWT-shaped triples, URLs with embedded credentials, `Bearer`
107
+ tokens, and `secret: value` / `token = value` style assignments.
108
+
109
+ **Commands are redacted, not dropped.** A verification command is a routing
110
+ index, not a credential transport. An inline assignment whose name contains
111
+ `SECRET`, `TOKEN`, `PASSWORD`, `PASSPHRASE`, `API_KEY`, `PRIVATE_KEY`,
112
+ `CREDENTIAL` or `ACCESS_KEY` keeps its name and loses its value:
113
+
114
+ ```
115
+ DATABASE_URL=postgres://u:p@host/db vitest run
116
+ -> DATABASE_URL=[REDACTED] vitest run
117
+ ```
118
+
119
+ The redaction marker itself is not treated as a secret, so a redacted command
120
+ remains runnable and printable.
121
+
122
+ **Observations.** An `evidenceBundle` must be a committed, repository-relative
123
+ file. A temp path, a console URL with a session token, or a screenshot filename
124
+ is not evidence. The environment identifier is constrained to a slug; nothing in
125
+ an observation may be a credential.
126
+
127
+ ## What is deliberately not provenance
128
+
129
+ - **Confidence.** A heuristic derivation is still traceable; `confidence:
130
+ candidate` says it may not be elected. Provenance and confidence answer
131
+ different questions.
132
+ - **Ownership.** Placement is an authored fact with a `declared` record, not a
133
+ derived one. The extractor identity tells you who read it, not who decided it.
package/docs/QUERY.md ADDED
@@ -0,0 +1,180 @@
1
+ # QUERY
2
+
3
+ The context router. Its job is **routing**, not documentation concatenation: for
4
+ a path or an entity, return the smallest set of facts an agent needs to work
5
+ safely on that subject, and refuse to answer from a stale graph.
6
+
7
+ ## Contract
8
+
9
+ ```bash
10
+ agentdoc query <repo-path|entity-ref> [--json] [--md]
11
+ ```
12
+
13
+ Resolution order:
14
+
15
+ 1. an exact entity ref;
16
+ 2. a documentation or constraint file a descriptor links, even outside the
17
+ component's source root;
18
+ 3. a directory that is exactly a unit root;
19
+ 4. the longest unit root that is a prefix of the path;
20
+ 5. a canonical contract file, which routes to the API entity that claims it.
21
+
22
+ If nothing resolves, the command fails with `AGENTDOC_REF_UNRESOLVED` and exits
23
+ non-zero. It never falls back to "the nearest component" or to a global dump.
24
+
25
+ ## Freshness refusal
26
+
27
+ `query` re-compiles in memory, reads the committed graph, and refuses it unless
28
+ `inputHash` and `dirty` agree. (The recorded commit is provenance, not a gate —
29
+ see `CI.md`.) Time-dependent gates run first: a
30
+ stale observation is reported as `AGENTDOC_OBSERVATION_STALE` before anything
31
+ else, because "your observation is 5 years old" and "rebuild the graph" send the
32
+ reader to completely different places.
33
+
34
+ There is no silent fallback. `--allow-stale-observations` exists as an explicit
35
+ override and is visible in the command line.
36
+
37
+ ## What comes back
38
+
39
+ ```json
40
+ {
41
+ "query": { "needle": "...", "resolved": "component:default/teacher" },
42
+ "entity": { "ref", "kind", "name", "description", "spec", "sourcePaths", "derived" },
43
+ "placement": { "systems": [], "domains": [] },
44
+ "relations": { "items": [ { "type", "direction", "other", "evidenceClass", "attributes" } ], "total", "truncated" },
45
+ "neighbours": [ { "ref", "kind", "name", "description" } ],
46
+ "apis": { "provided": [], "consumed": [], "provider", "consumers", "contract" },
47
+ "events": { "items": [], "total", "truncated" },
48
+ "resources": { "used": [], "usedBy": [] },
49
+ "externalDependencies": { "items": [], "total", "truncated" },
50
+ "capabilities": { "items": [], "total", "truncated" },
51
+ "constraints": { "docs": [], "constraints": [], "runbooks": [] },
52
+ "verification": { "items": [ { "id", "tier", "command", "configPaths" } ], "total", "truncated" },
53
+ "journeys": { "items": [], "total", "truncated" },
54
+ "journeyPeers": [ "component:default/..." ],
55
+ "contracts": { "items": [], "total", "truncated" },
56
+ "authority": {
57
+ "conflicts": { "items": [ { "key", "kind", "status", "election", "assertions" } ] },
58
+ "assertions": { "items": [] },
59
+ "observations":[ { "id", "environment", "sourceSystem", "collector", "capturedAt", "facts" } ]
60
+ },
61
+ "provenance": { "items": [] },
62
+ "globalGates": [ { "id", "tier", "command" } ],
63
+ "truncated": [ "relations", "events" ]
64
+ }
65
+ ```
66
+
67
+ ## Rules that matter
68
+
69
+ **Contracts are pointers, not copies.** `contracts[].ref` is the canonical
70
+ contract; `operations` is the inventory an agent needs to find a handler without
71
+ reading the whole specification.
72
+
73
+ **Critical context is separated from the rest.** `authority.conflicts`,
74
+ `authority.observations` and `constraints.constraints` are the things that cause
75
+ the most damage when missed, and the markdown rendering puts them under explicit
76
+ headings: `CONFLICTS (do not assume one is true)`, `RUNTIME OBSERVATIONS`,
77
+ `CONSTRAINTS`.
78
+
79
+ **Both sides of a conflict are printed.** A conflicted assertion shows its
80
+ evidence class, its confidence and its value. The contradicted value is never
81
+ withheld.
82
+
83
+ **A contract subject carries the provider's and consumers' checks.** Changing an
84
+ endpoint is proved by running their suites, so `verification` for an API subject
85
+ is the union over the provider and its consumers.
86
+
87
+ **Journey peers are named.** If a component appears in a journey, the other
88
+ components in that journey are in scope and are listed explicitly rather than
89
+ left for the agent to rediscover.
90
+
91
+ **Budgets are reported, never silent.** Every capped section returns
92
+ `{ items, total, truncated }`, and the top-level `truncated` lists every section
93
+ that was cut — including the nested `authority.conflicts` and
94
+ `authority.assertions`, reported as `authority.conflicts` and so on. A section is
95
+ never quietly shortened.
96
+
97
+ **Review commitments are surfaced.** `reviewCommitments` collects every conflict
98
+ election, accepted warning and reviewed override with the condition under which
99
+ it must be re-examined. A review condition nobody can see is a review condition
100
+ nobody performs. Where no condition was recorded — a conflict settled with no
101
+ governing authority rule — the entry says so explicitly rather than pointing at a
102
+ record that does not exist.
103
+
104
+ **Budgets are adjustable.** The defaults are deliberately generous, because a
105
+ one-hop route returns a handful of relations and a tight cap would truncate
106
+ everything. When an agent is working in a constrained context it can say so:
107
+
108
+ ```bash
109
+ agentdoc query component:default/api --budget relations=10,verification=4
110
+ ```
111
+
112
+ Limits are positive integers. Every capped section then reports
113
+ `{ items, total, truncated }` and the top-level `truncated` names each one, so
114
+ the slice is never mistaken for the whole.
115
+
116
+ ## Default budgets
117
+
118
+ | section | cap |
119
+ |---|---|
120
+ | relations | 40 |
121
+ | events | 20 |
122
+ | externalDependencies | 15 |
123
+ | capabilities | 10 |
124
+ | verification | 12 |
125
+ | journeys | 10 |
126
+ | conflicts | 20 |
127
+ | assertions | 25 |
128
+ | provenance | 20 |
129
+ | contracts | 10 |
130
+
131
+ ## Markdown rendering
132
+
133
+ `--md` (the default) produces a compact block suitable for a prompt or a human:
134
+
135
+ ```
136
+ # component:default/koda-cron (Component)
137
+ Triggers governed endpoints on a platform schedule and owns no business logic.
138
+ - source: workers/koda-cron
139
+ - system: system:default/scheduler
140
+ - CONSTRAINTS: workers/koda-cron/CONSTRAINTS.md
141
+ - verify:
142
+ - [unit] vitest run
143
+ - CONFLICTS (do not assume one is true):
144
+ - schedule.cron [contradiction/resolved]
145
+ - DERIVED/deterministic: "0 3 * * *"
146
+ - OBSERVED_RUNTIME/direct: "0 * * * *,3 0 * * *"
147
+ - RUNTIME OBSERVATIONS:
148
+ - production via platform-scheduler at 2026-09-27T18:40:00Z (evidence …)
149
+ ```
150
+
151
+ ## Change impact
152
+
153
+ ```bash
154
+ agentdoc impact <paths...>
155
+ agentdoc impact --diff <git-ref>
156
+ ```
157
+
158
+ Impact answers a question about the working tree, so it compiles in memory and
159
+ does not require a fresh committed graph. Using it before rebuilding is the
160
+ point.
161
+
162
+ It reports: affected components with the reason each was reached, affected
163
+ contracts, documentation, resources, verification and catalog sources; whether
164
+ the graph will be invalidated; the recommended verification commands; any open
165
+ conflicts on the affected subjects; and related journeys.
166
+
167
+ Reachability rules: a contract change reaches its provider and every consumer; a
168
+ component change reaches its transitive dependents two hops out, and everything
169
+ connected to it by a runtime call or a schedule. Two hops is deliberate — a
170
+ third hop is where the graph stops being a routing aid and becomes a dump.
171
+
172
+ ## Using it in a prompt
173
+
174
+ Ask for `--md` and give the output to the agent as-is. Two rules keep it
175
+ honest:
176
+
177
+ 1. Do not summarise a conflict away. If the context contains a `CONFLICTS`
178
+ block, the agent must be told about it.
179
+ 2. If the context contains `UNRESOLVED` or a conflict with `basis: none`, the
180
+ correct agent behaviour is to say so and stop, not to choose a side.