agent-doc-system 0.0.0-stage → 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (117) hide show
  1. package/INSTALL_FOR_AGENTS.md +60 -0
  2. package/LICENSE +21 -0
  3. package/README.md +151 -2
  4. package/adapters/cache.mjs +27 -0
  5. package/adapters/contract-consumers.mjs +56 -0
  6. package/adapters/database.mjs +116 -0
  7. package/adapters/deploy-config.mjs +34 -0
  8. package/adapters/dockerfile.mjs +26 -0
  9. package/adapters/events.mjs +145 -0
  10. package/adapters/generic.mjs +86 -0
  11. package/adapters/github-actions.mjs +106 -0
  12. package/adapters/go.mjs +91 -0
  13. package/adapters/graphql.mjs +28 -0
  14. package/adapters/index.mjs +55 -0
  15. package/adapters/node.mjs +118 -0
  16. package/adapters/object-store.mjs +64 -0
  17. package/adapters/oidc.mjs +56 -0
  18. package/adapters/openapi.mjs +50 -0
  19. package/adapters/pnpm.mjs +42 -0
  20. package/adapters/protobuf.mjs +35 -0
  21. package/adapters/registry.mjs +27 -0
  22. package/adapters/resource-client.mjs +61 -0
  23. package/adapters/turborepo.mjs +135 -0
  24. package/adapters/typescript.mjs +85 -0
  25. package/adapters/wrangler.mjs +206 -0
  26. package/bin/agentdoc.mjs +478 -0
  27. package/bin/usage.txt +26 -0
  28. package/core/acceptances.mjs +69 -0
  29. package/core/audit.mjs +407 -0
  30. package/core/authority/classes.mjs +39 -0
  31. package/core/authority/engine.mjs +243 -0
  32. package/core/authority/facts.mjs +41 -0
  33. package/core/codes.mjs +119 -0
  34. package/core/compile.mjs +881 -0
  35. package/core/contracts.mjs +121 -0
  36. package/core/derive.mjs +102 -0
  37. package/core/descriptors.mjs +486 -0
  38. package/core/determinism.mjs +95 -0
  39. package/core/discover.mjs +147 -0
  40. package/core/facts.mjs +53 -0
  41. package/core/fsx.mjs +342 -0
  42. package/core/graph.mjs +260 -0
  43. package/core/impact.mjs +175 -0
  44. package/core/indexes.mjs +165 -0
  45. package/core/jsonschema.mjs +298 -0
  46. package/core/observations.mjs +129 -0
  47. package/core/provenance.mjs +83 -0
  48. package/core/query.mjs +384 -0
  49. package/core/relations.mjs +286 -0
  50. package/core/scaffold.mjs +322 -0
  51. package/core/secrets.mjs +92 -0
  52. package/core/sourcescan.mjs +88 -0
  53. package/core/toml.mjs +163 -0
  54. package/core/yaml.mjs +475 -0
  55. package/docs/ADAPTERS.md +172 -0
  56. package/docs/AUTHORITY.md +260 -0
  57. package/docs/CI.md +136 -0
  58. package/docs/COMPARISON-KODA.md +164 -0
  59. package/docs/EVALS.md +166 -0
  60. package/docs/MIGRATION.md +193 -0
  61. package/docs/PROVENANCE.md +133 -0
  62. package/docs/QUERY.md +180 -0
  63. package/docs/SPEC.md +339 -0
  64. package/docs/VERIFICATION.md +28 -0
  65. package/docs/ci/generic-ci.sh +28 -0
  66. package/docs/ci/github-actions.yml +35 -0
  67. package/evals/harness.mjs +426 -0
  68. package/evals/scenarios/fixture-a-01-change-api.json +9 -0
  69. package/evals/scenarios/fixture-a-02-change-db.json +9 -0
  70. package/evals/scenarios/fixture-a-03-enforce-constraint.json +9 -0
  71. package/evals/scenarios/fixture-b-01-change-proto.json +9 -0
  72. package/evals/scenarios/fixture-b-02-change-write-path.json +9 -0
  73. package/evals/scenarios/fixture-b-03-change-shared-lib.json +9 -0
  74. package/evals/scenarios/fixture-b-04-journey.json +9 -0
  75. package/evals/scenarios/fixture-c-01-mixed-runtime-edge.json +19 -0
  76. package/evals/scenarios/fixture-c-02-stale-doc-contradiction.json +9 -0
  77. package/evals/scenarios/fixture-c-03-shared-vocabulary.json +9 -0
  78. package/evals/scenarios/fixture-c-04-contract-change.json +9 -0
  79. package/evals/scenarios/koda-01-modify-api-consumer.json +26 -0
  80. package/evals/scenarios/koda-02-change-db-schema.json +20 -0
  81. package/evals/scenarios/koda-03-change-auth.json +21 -0
  82. package/evals/scenarios/koda-04-add-event-producer.json +19 -0
  83. package/evals/scenarios/koda-05-modify-worker-binding.json +25 -0
  84. package/evals/scenarios/koda-06-change-cross-component-endpoint.json +22 -0
  85. package/evals/scenarios/koda-07-change-shared-package.json +16 -0
  86. package/evals/scenarios/koda-08-change-runtime-cron.json +23 -0
  87. package/evals/scenarios/koda-09-debug-production-mismatch.json +26 -0
  88. package/evals/thresholds.json +11 -0
  89. package/package.json +46 -4
  90. package/schemas/api.schema.json +84 -0
  91. package/schemas/common.schema.json +107 -0
  92. package/schemas/component.schema.json +40 -0
  93. package/schemas/config.schema.json +685 -0
  94. package/schemas/domain.schema.json +14 -0
  95. package/schemas/graph.schema.json +1084 -0
  96. package/schemas/journeys.schema.json +47 -0
  97. package/schemas/observation.schema.json +143 -0
  98. package/schemas/resource.schema.json +28 -0
  99. package/schemas/system.schema.json +26 -0
  100. package/skills/agent-doc-system/SKILL.md +113 -0
  101. package/skills/agent-doc-system/references/authority-and-conflicts.md +70 -0
  102. package/skills/agent-doc-system/references/cli.md +89 -0
  103. package/skills/agent-doc-system/references/create-migrate.md +101 -0
  104. package/skills/agent-doc-system/references/maintenance.md +52 -0
  105. package/templates/ADR.md +26 -0
  106. package/templates/ARCHITECTURE.md +19 -0
  107. package/templates/CONSTRAINTS.md +17 -0
  108. package/templates/PRODUCT.md +17 -0
  109. package/templates/api.yaml +10 -0
  110. package/templates/ci.yml +31 -0
  111. package/templates/component.yaml +14 -0
  112. package/templates/domain.yaml +6 -0
  113. package/templates/journey.yaml +7 -0
  114. package/templates/observation.yaml +19 -0
  115. package/templates/resource.yaml +8 -0
  116. package/templates/system.yaml +10 -0
  117. package/templates/warning-acceptance.yaml +14 -0
@@ -0,0 +1,260 @@
1
+ # AUTHORITY
2
+
3
+ The model that distinguishes what the repository claims, what can be derived
4
+ from it, and what was actually observed running.
5
+
6
+ ## Why this exists
7
+
8
+ A catalog that merges these three produces confident, confident-looking wrong
9
+ answers. Every one of the following is a real failure mode this model is built
10
+ to make impossible to hide:
11
+
12
+ - the presumed production binding source is not the real authority;
13
+ - a binding assumed to be missing is actually four;
14
+ - a schedule described as daily is an hourly trigger with hour gating;
15
+ - an external upload API is assumed to accept a field the deployed API rejects;
16
+ - recovered runtime state intentionally differs from the governed target state.
17
+
18
+ Each of these is, mechanically, the same event: **two evidence classes made
19
+ claims about the same fact, and the catalog quietly picked one.**
20
+
21
+ ## Evidence classes
22
+
23
+ ### `AUTHORED`
24
+
25
+ Explicitly declared and intentionally maintained in this repository: catalog
26
+ descriptors, architecture documents, ADRs, canonical contracts, configuration
27
+ manifests, declared ownership.
28
+
29
+ - **Strengths:** states intent; reviewed; versioned with the decision.
30
+ - **Failure modes:** drifts; states what *should* be, not what *is*; a config
31
+ file is a claim about deployment, not a deployment.
32
+ - **Never sufficient** for a claim about what is currently running.
33
+
34
+ ### `DERIVED`
35
+
36
+ Facts deterministically extracted from committed inputs: package dependencies,
37
+ imports, routes, runtime bindings, CI checks, service calls, events, database
38
+ usage, artifact shape.
39
+
40
+ - **Strengths:** reproducible; anyone can re-derive it; cannot silently rot.
41
+ - **Failure modes:** heuristic; incomplete; a consumer of a contract that goes
42
+ through a generated client may not name the contract file; absence of evidence
43
+ is not evidence of absence.
44
+ - **A derivation whose result is ambiguous must degrade to a diagnostic, not to
45
+ a fact.** An ambiguous bucket literal produces `AGENTDOC_BINDING_AMBIGUOUS`,
46
+ not a guessed resource edge.
47
+
48
+ ### `OBSERVED_RUNTIME`
49
+
50
+ Facts read from an authoritative external or running system: deployed
51
+ configuration, real scheduler state, live bindings, cloud provider capabilities,
52
+ the actual production version, external API behaviour.
53
+
54
+ Every observation carries:
55
+
56
+ ```
57
+ source the authority system, e.g. platform-scheduler
58
+ timestamp when it was read, ISO-8601 UTC
59
+ environment which environment it describes
60
+ retrieval method the command or API used
61
+ evidence a durable, committed, secret-free reference
62
+ freshness maxAgeDays, a promise the observation must keep
63
+ supersedes the observation set it replaces, if any
64
+ ```
65
+
66
+ Observations are committed to the repository. That is deliberate: an observation
67
+ nobody can reproduce is an anecdote, and an anecdote does not belong in a graph
68
+ that claims to describe reality.
69
+
70
+ **An observation can never overwrite an authored fact.** It is added to the same
71
+ `(subject, key)` group, and the authority engine decides what happens. Without a
72
+ rule, the disagreement is an error.
73
+
74
+ ### `EXTERNAL_STANDARD`
75
+
76
+ Standards whose canonical authority is outside the repository: OIDC discovery,
77
+ OpenAPI, protocol specifications, provider API schemas.
78
+
79
+ - The desired wire surface is `AUTHORED` (the contract file) or
80
+ `EXTERNAL_STANDARD` (the specification).
81
+ - What a live deployment accepts is `OBSERVED_RUNTIME`.
82
+ - These routinely disagree, and that disagreement is the point.
83
+
84
+ ### `REVIEWED_OVERRIDE`
85
+
86
+ A human-approved interpretation, used when deterministic discovery cannot
87
+ resolve a fact. It is bounded and must be justified:
88
+
89
+ ```
90
+ fact which kind of fact is being asserted
91
+ subject the entity ref
92
+ target the entity ref, where applicable
93
+ reason why this is true, in one sentence
94
+ evidence file paths under a mapped component's source root, or the contract
95
+ ```
96
+
97
+ Evidence must live under a mapped component's source root or be the declared
98
+ contract. A reviewed override therefore cannot assert something no file in the
99
+ repository can corroborate. Overrides additionally produce a `REVIEWED_OVERRIDE`
100
+ fact, so the interpretation is a first-class citizen of the conflict machinery
101
+ rather than a bypass.
102
+
103
+ ### `UNRESOLVED`
104
+
105
+ Known ambiguity. It must not become truth, must not be silently dropped, and
106
+ must not be resolved by picking a plausible answer.
107
+
108
+ Two sources of unresolved facts:
109
+
110
+ 1. **Contradiction with no governing rule.** An error.
111
+ 2. **Only heuristic candidates.** A warning; the candidates never become elected
112
+ facts. Migration authority alone does not establish database ownership; a
113
+ library that ships migrations but owns no pool gets a warning and a candidate,
114
+ not an invented `usesResource` edge.
115
+
116
+ ## Confidence
117
+
118
+ A separate axis from evidence class, recording *how* the claim was obtained:
119
+
120
+ | confidence | meaning |
121
+ |---|---|
122
+ | `direct` | read straight from the named authority system |
123
+ | `deterministic` | a reproducible function of committed inputs |
124
+ | `declared` | asserted by a human-maintained document |
125
+ | `reviewed` | a reviewer signed off |
126
+ | `candidate` | heuristic; never elected |
127
+
128
+ `candidate` is enforced mechanically, not by convention: an assertion whose only
129
+ evidence is a candidate stays `status: candidate`, and the group is reported as
130
+ an ambiguity.
131
+
132
+ ## Authority rules
133
+
134
+ A rule says which evidence class is authoritative **for one key**, for a
135
+ subject matching a pattern.
136
+
137
+ ```yaml
138
+ authority:
139
+ rules:
140
+ - id: deployed-schedule-is-runtime-truth
141
+ keyPattern: " schedule\\.(cron|enabled|target)$"
142
+ elect: [OBSERVED_RUNTIME]
143
+ rationale: "The committed manifest states intent; only the live scheduler states what fires."
144
+ reviewWhen: "Review when the scheduler configuration changes."
145
+ degradeOnStale: conflict
146
+ ```
147
+
148
+ - `keyPattern` is matched against `"<subject> <key>"`; the most specific match
149
+ (longest pattern) wins.
150
+ - `elect` may list one to three evidence classes. Naming `UNRESOLVED` is
151
+ rejected at load: electing nothing elects nothing.
152
+ - `rationale` is mandatory. A rule nobody can justify is a rule nobody can
153
+ review.
154
+ - `reviewWhen` is mandatory.
155
+
156
+ **There is no default global precedence.** A fact with disagreeing evidence and
157
+ no matching rule is an error. This is the single most important property of the
158
+ model: silence is not a resolution.
159
+
160
+ ## Election
161
+
162
+ For a group of assertions on one `(subject, key)`:
163
+
164
+ 1. **Agreement** — all values equal. Every assertion is `elected`. Agreement
165
+ across evidence classes is corroboration, and it is visible as such. If the
166
+ only evidence is a heuristic candidate, the group stays unresolved.
167
+ 2. **Contradiction** — values differ. The basis is attempted in order:
168
+ - `reviewed-override` — a rule elects `REVIEWED_OVERRIDE` and exactly one
169
+ exists. Status `accepted`. The divergence is still recorded.
170
+ - `authority-rule` — the rule's elect list intersects the present classes in
171
+ exactly one class with exactly one assertion.
172
+ - `superset` — one candidate's value strictly contains every other (for
173
+ example an observed set that is a strict subset of a declared set). The
174
+ superset is elected; the containment is stated.
175
+ - `recency` — only when the rule names `OBSERVED_RUNTIME` and more than one
176
+ dated observation exists. The newest wins, and the rule is cited.
177
+ - otherwise `basis: none`, status `unresolved`, and an **error**.
178
+ 3. Every non-elected assertion is kept with `status: contradicted` (or
179
+ `candidate`). **The contradicted value stays in the graph.** A system that
180
+ discards the repository's own claim cannot explain the divergence to a human.
181
+
182
+ ## Conflict kinds
183
+
184
+ | kind | meaning |
185
+ |---|---|
186
+ | `contradiction` | two evidence classes disagree about the same value |
187
+ | `staleness` | two runtime observations disagree; one is out of date |
188
+ | `supersession` | a newer observation explicitly replaces an older one |
189
+ | `ambiguity` | only heuristic candidates, or an ungoverned disagreement |
190
+ | `reviewed-divergence` | a reviewer approved the divergence and it is recorded |
191
+
192
+ Status is `unresolved`, `resolved` or `accepted`.
193
+
194
+ ## Staleness
195
+
196
+ An observation is stale when `now - capturedAt > maxAgeDays`. Staleness is
197
+ evaluated at gate time, never at compile time, because a compile must stay a
198
+ pure function of the checkout.
199
+
200
+ - `check` and `query` fail closed on a stale observation and name it.
201
+ - `--allow-stale-observations` exists as an explicit, visible override.
202
+ - `degradeOnStale` lets a rule say what a stale election means: keep the
203
+ election but raise a staleness conflict (`conflict`), or drop to unresolved
204
+ (`unresolved`).
205
+
206
+ ## Worked example
207
+
208
+ The committed manifest declares:
209
+
210
+ ```toml
211
+ [triggers]
212
+ crons = ["0 3 * * *"]
213
+ ```
214
+
215
+ The platform reports:
216
+
217
+ ```json
218
+ { "triggers": [ { "cron": "0 * * * *", "gate": { "hourEquals": 3 } },
219
+ { "cron": "3 0 * * *", "enabled": true } ] }
220
+ ```
221
+
222
+ The observation says, of `schedule.cron`:
223
+
224
+ > The scheduler evaluates an hourly trigger plus a daily trigger; the hourly
225
+ > trigger is gated to hour 3, so the effective daily behaviour is not the
226
+ > manifest's single daily expression.
227
+
228
+ The graph then contains:
229
+
230
+ ```
231
+ conflict component:default/worker schedule.cron contradiction resolved
232
+ assertion AUTHORED/DERIVED "0 3 * * *" contradicted
233
+ assertion OBSERVED_RUNTIME "0 * * * *,3 0 * * *" elected
234
+ election authority-rule / deployed-schedule-is-runtime-truth
235
+ ```
236
+
237
+ A query for that component prints both values, marks the election, and points at
238
+ the evidence. An agent debugging "the nightly job did not run" sees immediately
239
+ that the repository's description of the schedule is not what the scheduler
240
+ does — which is the finding. Silently preferring either side would have hidden
241
+ it.
242
+
243
+ ## The one place a label is chosen without an election is a *relation's*
244
+ `evidenceClass`, and it elects nothing: when its provenance records agree there is
245
+ one class and it is reported; when they disagree the edge is reported as
246
+ `REVIEWED_OVERRIDE` if a human interpretation is among them, and `DERIVED`
247
+ otherwise. Picking by strength would be a global precedence order in a helper, and
248
+ it was actively wrong — a reviewed override corroborated by a source reference was
249
+ relabelled `AUTHORED`, so the graph understated its own highest-authority input
250
+ while the corresponding assertion still read `REVIEWED_OVERRIDE`.
251
+
252
+ Anti-patterns this model forbids
253
+
254
+ - A global precedence order (`AUTHORED` always beats `DERIVED`).
255
+ - Overwriting an authored value with an observed one because the observed one is
256
+ newer, with no rule.
257
+ - Treating a configuration file as evidence of deployment.
258
+ - Accepting a warning to silence a conflict.
259
+ - Deleting the contradicted assertion after an election.
260
+ - Promoting a heuristic candidate because it is the only candidate.
package/docs/CI.md ADDED
@@ -0,0 +1,136 @@
1
+ # CI
2
+
3
+ The graph is a build artifact. The gate proves the committed one is identical to
4
+ a fresh compile, so drift fails in CI rather than in a review three weeks later.
5
+
6
+ ## The gate
7
+
8
+ ```yaml
9
+ name: agentdoc
10
+ on:
11
+ pull_request:
12
+ push:
13
+ branches: [main]
14
+ jobs:
15
+ agentdoc:
16
+ runs-on: ubuntu-latest
17
+ timeout-minutes: 5
18
+ steps:
19
+ - uses: actions/checkout@v4
20
+ with: { fetch-depth: 0 }
21
+ - uses: actions/setup-node@v4
22
+ with: { node-version: '24' }
23
+ # Pin a tag for reproducibility, e.g. github:lbframe/agent-doc-system#v1.0.0 —
24
+ # a compiler upgrade intentionally invalidates every committed graph.
25
+ - run: npm install -g github:lbframe/agent-doc-system
26
+ - run: agentdoc validate
27
+ - run: agentdoc compile
28
+ - run: agentdoc check
29
+ - run: agentdoc check --require-clean
30
+ - run: agentdoc eval routing
31
+ ```
32
+
33
+ `agentdoc init` writes this workflow to `.github/workflows/agentdoc.yml` (from
34
+ `templates/ci.yml` in the CLI package). The same file is kept at
35
+ `docs/ci/github-actions.yml` for reference.
36
+
37
+ `fetch-depth: 0` is recommended: `--diff <ref>` needs history, and `dirty` is
38
+ more meaningful against a real `HEAD`.
39
+
40
+ **What freshness actually checks.** The input hash (every byte the compiler
41
+ read, plus every existence probe, directory listing and walk) and the dirty flag
42
+ (whether a compiler input differs from `HEAD`). It deliberately does **not**
43
+ compare the graph's recorded commit with `HEAD`: committing the graph changes
44
+ `HEAD`, so a commit-equality gate could never be satisfied by a committed graph.
45
+ The commit is recorded as provenance — which checkout produced this graph — not
46
+ as a gate.
47
+
48
+ Two things to decide per repository:
49
+
50
+ - **Commit the graph or ignore it.** Both work. Committing it makes the
51
+ architecture reviewable in the pull request, which is valuable in a mature
52
+ repository. Ignoring it keeps diffs small. The gate works either way, because
53
+ the workflow compiles before checking.
54
+ - **Run the routing evaluation or not.** `agentdoc eval routing` skips cleanly
55
+ on a repository with no registered scenarios, so it is safe to run
56
+ everywhere. The aggregate gate over every corpus (`node evals/run-all.mjs`)
57
+ is development tooling — it belongs to this repository's own CI, not to a
58
+ project that merely uses the catalog.
59
+
60
+ ## Failure modes worth knowing
61
+
62
+ The full list is `core/codes.mjs`; these are the ones an operator meets. A
63
+ YAML or schema error names its file and line and needs no table.
64
+
65
+
66
+ | code | what happened | what to do |
67
+ |---|---|---|
68
+ | `AGENTDOC_STALE_GRAPH` | committed bytes differ from a fresh compile | `compile`, review the graph diff, commit |
69
+ | `AGENTDOC_GRAPH_FRESHNESS` | the input hash or the dirty flag disagrees | rebuild; if the input changed, review the change first |
70
+ | `AGENTDOC_COMPILER_MISMATCH` | graph built by another compiler version | `compile` and commit; the diff is the upgrade's real effect |
71
+ | `AGENTDOC_GRAPH_SCHEMA` | graph schema changed | `compile` and commit |
72
+ | `AGENTDOC_GRAPH_MISSING` | no graph at the configured path | `compile` |
73
+ | `AGENTDOC_DIRTY_SOURCE` | a catalog input differs from `HEAD` | commit the input, then recompile |
74
+ | `AGENTDOC_OBSERVATION_STALE` | an observation passed `maxAgeDays` | re-observe, or restate the fact as authored intent if the divergence is intended |
75
+ | `AGENTDOC_CONFLICT_UNRESOLVED` | two classes disagree and no rule governs the key | add an authority rule, or ask a human |
76
+ | `AGENTDOC_WARNING_ACCEPTANCE` | an acceptance no longer matches, or its evidence moved | review the disposition, then update the digests |
77
+ | `AGENTDOC_SECRET` | secret-shaped input or output | remove the credential; never redact-and-continue |
78
+ | `AGENTDOC_NONDETERMINISTIC` | a host path or unsourced timestamp reached the graph | file a defect; this is a compiler bug |
79
+ | `AGENTDOC_CONTRACT_UNCLAIMED` | a contract file no API claims | point an API at it, or delete it |
80
+ | `AGENTDOC_COMPONENT_UNCORROBORATED` | a descriptor no adapter can corroborate | fix the location, or enable the right adapter |
81
+ | `AGENTDOC_BINDING_AMBIGUOUS` | a client matches several Resources of one type | declare which, or review and add an override |
82
+ | `AGENTDOC_CONTRACT_SYNTAX` | a contract file is not valid for its declared type | fix the contract; do not change its type |
83
+ | `AGENTDOC_OIDC_DISCOVERY` | the provider does not expose its discovery path | fix the provider or the descriptor |
84
+ | `AGENTDOC_DUPLICATE_IDENTITY` | two entities claim the same kind and name | rename one |
85
+ | `AGENTDOC_COMPILER_MISMATCH` | the compiler was upgraded | `compile` and commit |
86
+ | `AGENTDOC_OVERRIDE_STALE` | a reviewed override's cited evidence changed | review the interpretation, then `pin-evidence` and update the digest |
87
+ | `AGENTDOC_VERIFICATION_UNTIERED` | a check task has no recognisable tier | name the task, or extend the adapter |
88
+
89
+ ## Ordering inside `check`
90
+
91
+ 1. compilation errors;
92
+ 2. time-dependent gates (observation staleness, election degradation);
93
+ 3. graph freshness (`--require-clean`, input hash, compiler/schema versions);
94
+ 4. byte-for-byte comparison.
95
+
96
+ Step 2 precedes step 3 on purpose. "Your observation is five years old" and
97
+ "rebuild the graph" send the reader to completely different places, and when
98
+ both are true the first is the one that matters.
99
+
100
+ ## Merge discipline
101
+
102
+ - A PR that changes a compiler input must include the recompiled graph. `impact`
103
+ tells you which files are inputs.
104
+ - A PR that changes an authority rule must explain why that class is
105
+ authoritative for that key. `rationale` and `reviewWhen` are mandatory, and a
106
+ reviewer should read them.
107
+ - A PR that adds a warning acceptance is a reviewed decision. The evidence
108
+ digests are pinned; if the evidence later moves, the acceptance stops
109
+ matching and forces a new review. That is the mechanism working, not failing.
110
+ - Never hand-edit `.agentdoc/graph.json`. The gate rejects it, and it would be
111
+ wrong anyway.
112
+
113
+ ## Local parity
114
+
115
+ ```bash
116
+ agentdoc validate
117
+ agentdoc compile
118
+ agentdoc check --require-clean
119
+ ```
120
+
121
+ The same commands, the same codes, the same exit status. There is no CI-only
122
+ behaviour.
123
+
124
+ ## Other CI systems
125
+
126
+ The gate is four commands and an exit code. Any runner works:
127
+
128
+ ```sh
129
+ set -e
130
+ agentdoc validate
131
+ agentdoc compile
132
+ agentdoc check --require-clean
133
+ ```
134
+
135
+ `ci/generic-ci.sh` (next to this document) is exactly that, plus a clear
136
+ failure when the `agentdoc` CLI is not installed on the runner.
@@ -0,0 +1,164 @@
1
+ # Koda comparison
2
+
3
+ Reference: the Koda production repository (private) at `origin/main` =
4
+ `19b2227f17d0116ef5e8d8a6dd1e86ff1fdb913b`.
5
+ The portable system was run against a working copy of that commit. No Koda source,
6
+ configuration or production behaviour was modified.
7
+
8
+ This file is generated by `scripts/compare-koda.py`; every number in the tables
9
+ below is computed from the two graphs. Prose figures that describe a fixed past
10
+ event — how many duplicates an earlier run emitted, how many rows of noise a
11
+ misconfiguration produced — are historical, and are marked as such.
12
+
13
+ **Two bases are used, and they are not interchangeable.** *Triples* are
14
+ `(type, source, target)` — the architecture-level fact. *Relations* are individual
15
+ emitted edges, which outnumber triples wherever one fact is deliberately several
16
+ edges: a scheduler pointing at four routes of one API is four relations and one
17
+ triple. The comparison table is on triples; the relation totals are given
18
+ separately and are stated as relations.
19
+
20
+ **Relation names are normalised before comparison.** The reference prefixes every
21
+ relation with the product namespace (`koda-build-depends-on`); the portable system
22
+ uses neutral names (`buildDependsOn`). Without that normalisation the shared count
23
+ is 44% rather than 84%.
24
+
25
+ ## Entities — preserved exactly
26
+
27
+ | kind | reference | portable | identical |
28
+ |---|---|---|---|
29
+ | Domain | 3 | 3 | yes |
30
+ | System | 5 | 5 | yes |
31
+ | Component | 24 | 24 | yes |
32
+ | API | 10 | 10 | yes |
33
+ | Resource | 11 | 11 | yes |
34
+ | **total** | **53** | **53** | **yes** |
35
+
36
+ Every entity ref, name, kind and placement is reproduced. No entity was lost.
37
+
38
+ ## Relations — compared as triples
39
+
40
+ | type | reference | portable | shared | reference-only | portable-only |
41
+ |---|---|---|---|---|---|
42
+ | `apiConsumedBy` | 25 | 21 | 19 | 6 | 2 |
43
+ | `apiProvidedBy` | 10 | 10 | 10 | 0 | 0 |
44
+ | `buildDependencyOf` | 24 | 27 | 23 | 1 | 4 |
45
+ | `buildDependsOn` | 24 | 27 | 23 | 1 | 4 |
46
+ | `consumesApi` | 25 | 21 | 19 | 6 | 2 |
47
+ | `hasPart` | 23 | 23 | 23 | 0 | 0 |
48
+ | `partOf` | 23 | 23 | 23 | 0 | 0 |
49
+ | `providesApi` | 10 | 10 | 10 | 0 | 0 |
50
+ | `resourceUsedBy` | 26 | 16 | 16 | 10 | 0 |
51
+ | `runtimeCalledBy` | 3 | 2 | 2 | 1 | 0 |
52
+ | `runtimeCalls` | 3 | 2 | 2 | 1 | 0 |
53
+ | `scheduledBy` | 3 | 2 | 2 | 1 | 0 |
54
+ | `schedules` | 3 | 2 | 2 | 1 | 0 |
55
+ | `testDependencyOf` | 3 | 4 | 3 | 0 | 1 |
56
+ | `testDependsOn` | 3 | 4 | 3 | 0 | 1 |
57
+ | `usesResource` | 26 | 16 | 16 | 10 | 0 |
58
+ | **triples** | **234** | **210** | **196 (84%)** | **38** | **14** |
59
+
60
+ Emitted relation count: reference 234, portable 216. The portable graph emits more
61
+ relations than triples because it does not collapse distinct instances — the
62
+ reference emits the same number as triples, collapsing them.
63
+
64
+ ## Other surfaces
65
+
66
+ | surface | reference | portable | note |
67
+ |---|---|---|---|
68
+ | event subjects | 42 | 44 | 20/42 reference subjects covered; 22 need Koda-specific extraction |
69
+ | external dependencies | 24 | 31 | classification declared in configuration |
70
+ | capabilities | 1 | 1 | preserved, via a declared detector |
71
+ | verification entries | 78 | 95 | plus 2 repository-scoped gates; no unrunnable command is emitted |
72
+ | journeys | 1 | 1 | identical |
73
+ | provenance records | 961 | 709 | every relation, fact, contract, conflict and index entry corroborated in both |
74
+ | diagnostics | 3 | 22 | the portable system surfaces more, because it refuses to guess |
75
+
76
+ ## New surfaces with no reference equivalent
77
+
78
+ | surface | count |
79
+ |---|---|
80
+ | fact assertions | 110 |
81
+ | conflicts | 7 |
82
+ | unresolved conflicts | 7 |
83
+ | contract pointers | 10 |
84
+ | repository gates | 2 |
85
+
86
+ ## What was not reproduced, and why
87
+
88
+ The 38 reference triples absent from the portable graph, grouped by cause:
89
+
90
+ 1. **10 resource edges** (`usesResource`/`resourceUsedBy` and their inverses). The
91
+ reference resolves a component to a logical dataset from a bucket literal plus a
92
+ name token. The portable system reports `AGENTDOC_BINDING_AMBIGUOUS` instead,
93
+ because several authored `logical-dataset` Resources match one object-store client
94
+ and no deterministic rule separates them. The information is surfaced for review
95
+ rather than guessed. This is a deliberate difference, not a regression: the
96
+ reference's heuristic is exactly the kind of inference the authority model forbids
97
+ from becoming a fact, and the ambiguity is what produces the 7 unresolved
98
+ `resource.ownership` conflicts above.
99
+ 2. **6 API-consumer edges** (`consumesApi`/`apiConsumedBy`). Some are declared as
100
+ `reviewedOverrides` in the reference configuration that were not ported for this
101
+ comparison; the rest were derived by a reference-specific rule. The split is not
102
+ stated as a count because the reference graph records no marker distinguishing a
103
+ reviewed override from a derived edge, so the number is not recoverable from the
104
+ artefact and would be a guess presented as a measurement.
105
+ 3. **1 schedule edge and 1 runtime call**: one unreviewed override, and one
106
+ route-owner tie that ranked resolution declines to break.
107
+ 4. **1 build edge** derived by a reference-specific devDependency rule.
108
+
109
+ The portable graph additionally contains 14 triples the reference does not,
110
+ principally TypeScript workspace edges derived from `package.json` dependencies,
111
+ which the reference does not record.
112
+
113
+ ## Defects found by running against the real repository
114
+
115
+ Each of these was found by running against Koda rather than against a fixture, and
116
+ fixed in the engine rather than in a threshold.
117
+
118
+ 1. **Duplicate relations.** The first run emitted 14 relations twice — the same fact
119
+ once per adapter that noticed it — because the derivation tag `via` was part of a
120
+ relation's identity. A `consumesApi` edge asserted by both a reviewed override and
121
+ a source reference appeared twice, with the override's provenance hidden behind a
122
+ derived edge with identical endpoints. `via` is now excluded from identity and the
123
+ derivation routes merge into a sorted list.
124
+ Two follow-on defects of the same kind were found while fixing this: the merge
125
+ *relabelled* a corroborated override as `AUTHORED`, so the graph understated its own
126
+ highest-authority input; and a mechanism route and a declared-client route to one
127
+ Resource still shipped as two edges. Both are fixed.
128
+ 2. A flow-mapping scalar containing a comma — `{ description: Body is bad, invalid, or wrong. }` —
129
+ was rejected. Real OpenAPI documents contain these and the ecosystem parses them.
130
+ The flow parser now matches that behaviour deterministically.
131
+ 3. A Go workspace member depended on only by import, with no `require` line, was
132
+ invisible: the import pattern required three dots and Go module paths have two.
133
+ 4. A repository path was resolved against the process working directory, so the same
134
+ commit compiled differently depending on where the compiler ran.
135
+ 5. A task-graph command was emitted for a component that does not declare the task,
136
+ putting an unrunnable command in front of an agent as if it were the answer.
137
+ 6. Every third-party package was recorded as an external dependency — 602 rows of
138
+ noise. External-service classification is now declared, and an unclassified count
139
+ is reported for a human to triage.
140
+ 7. A command carrying a credential as a separate argument (`curl -u user:pass`)
141
+ reached the graph verbatim. Redaction and the output scanner now cover it.
142
+
143
+ ## Requirements the portable system adds
144
+
145
+ - Every `Resource` must state its placement (`system` or `placementRationale`). The
146
+ reference allowed a resource with no owner statement; the portable system treats
147
+ that as the ambiguity it is. Cost: one line per resource.
148
+ - Capability detectors and external-service classification are declared in
149
+ configuration rather than hard-coded in an extractor.
150
+ - Reviewed overrides must state the condition under which they are re-examined, and
151
+ the digests of the evidence they cite are re-verified on every compile.
152
+ - Observations, facts, conflicts and contract pointers are new surfaces with no
153
+ reference equivalent.
154
+
155
+ ## Verdict
156
+
157
+ All 53 entities, all 46 placement edges, all 10 API provider edges and the journey
158
+ are preserved exactly. 196 of 234 relation triples (84%) are identical. Every missing
159
+ edge is accounted for above, and in each case the portable system declines to guess
160
+ rather than assert something the repository does not state. No information was
161
+ silently lost: unresolved bindings surface as diagnostics carrying the evidence that
162
+ would settle them.
163
+
164
+ **No blocking regression.** The seven defects above were fixed in the engine.