@typedstandards/civic-typed-harness 0.4.0 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,225 @@
3
3
  Factual record of what changed per published version. Section references are
4
4
  to the Typed Standards specification unless noted otherwise.
5
5
 
6
+ ## 0.5.0 — 2026-09-18
7
+
8
+ The list is derived from
9
+ `git log 33a3aff..<this release> -- packages/civic-typed-harness/` — `33a3aff`
10
+ is the 0.4.1 release bump — so it is the commit range rather than a
11
+ recollection.
12
+ Three commits landed after that bump and had no entry; this phase's two
13
+ changes join them.
14
+
15
+ **Together, the two changes below make a record name the servers the run
16
+ actually had, and say nothing where it does not know.** They are the harness
17
+ half of
18
+ [civic-ai-tools#205](https://github.com/npstorey/civic-ai-tools/issues/205)
19
+ and
20
+ [civic-ai-tools-website#449](https://github.com/npstorey/civic-ai-tools-website/issues/449);
21
+ the caller half (a registry built from the instance's own configuration, and
22
+ the packager passing the list) lands in the reference application.
23
+
24
+ ### The environment extension can name every server, not only the skill's
25
+
26
+ - **`buildDatHereEnvironment` accepts a list of MCP servers**, additively. Its
27
+ second parameter is now `string | readonly DatHereMcpServer[] | undefined`:
28
+ the bare string is the pre-0.5.0 shape — the trace's skill-fetch URL, the one
29
+ server the builder could name — and a list is emitted in the order the caller
30
+ gives, each entry carrying `name` only when the caller supplied one. Spec
31
+ §8.7.1 requirement 3 always specified `mcpServers` as "an array of objects
32
+ with `url` and optional `name`"; until now the harness could only ever put
33
+ one object in it.
34
+ - **The single-URL shape is byte-identical to before**, and a test pins those
35
+ exact bytes rather than deep-equality, because an already-signed datHere
36
+ package reproduces through this function. A one-entry list with no `name`
37
+ produces the same bytes as the string, which is the same statement from the
38
+ other side.
39
+ - **`DatHerePolicyInput` gains `mcpServers`**, consulted in preference to
40
+ `skillMcpServerUrl` and falling back to it, so `deriveDatHereEnvelopeFields`
41
+ carries the list through without any existing caller changing.
42
+ - **New export:** the `DatHereMcpServer` interface. An entry whose `url` is
43
+ empty contributes nothing — the same rule the single-URL shape always
44
+ applied: a server with no address is stated by absence, never by an empty
45
+ string.
46
+
47
+ ### A source agent names the configured address, or names no address at all
48
+
49
+ - **`CivicSourceInfo.serverUrl` is optional.** The address a run reached is the
50
+ operator's configuration, not a fact the harness knows. A caller passes a
51
+ registry whose entries carry the addresses that instance is pointed at.
52
+ - **`buildProvenanceGraph` omits `civic:serverUrl` when it has no address.**
53
+ Two shapes reach that: a registry entry that carries no `serverUrl`, and a
54
+ source id the registry does not carry at all. Both now emit an agent with no
55
+ `civic:serverUrl` key — the key's absence, not an empty string — so the
56
+ signed bytes say the address is unknown instead of naming a server the
57
+ analysis may never have contacted. When an address IS known the key keeps its
58
+ position in the emitted key order, which is the byte contract on the legacy
59
+ chain.
60
+ - **The source-id fallback is dropped.** An agent for a source outside the
61
+ registry used to carry `civic:serverUrl` equal to its own source id — a
62
+ string that was never an address. That is the asserted-default class
63
+ ([civic-ai-tools#129](https://github.com/npstorey/civic-ai-tools/issues/129)'s
64
+ family): a stand-in standing in a signed record for a fact the run had and
65
+ did not write.
66
+ - **`CIVIC_SOURCE_REGISTRY` keeps its three `serverUrl` values**, and the
67
+ comment now says what they are for: the reference deployment's own default
68
+ addresses for display, and — on the two aggregate sources — the companions of
69
+ `aggregatePortalUrl`. They are not what a record asserts about a run.
70
+ - **Type-level consequence, stated rather than glossed.** Widening
71
+ `serverUrl` to optional is additive for a caller that BUILDS a registry and
72
+ narrowing for one that READS `info.serverUrl` and expects a `string`. A
73
+ TypeScript consumer doing the latter has to handle `undefined`.
74
+
75
+ **Golden bytes.** Two of the three fixtures are untouched;
76
+ `__fixtures__/website-golden.json` and `__fixtures__/reference-golden.json` are
77
+ frozen reference captures and no byte of either is edited. The 2026-08-01
78
+ website capture carries one MCP agent for a source its registry does not know,
79
+ and that agent's `civic:serverUrl` is the dropped fallback, so the suites lift
80
+ the capture over exactly that one key — derived from the registry, with the
81
+ number of lifted nodes asserted, the same device already used for the two
82
+ Appendix J literals and for the synthetic prompt hash. The lifted graph is
83
+ inside the hashed bytes of the ninth golden-reproduction case, so
84
+ `__fixtures__/span-carrying-golden.json`'s two recorded hashes are regenerated,
85
+ intentionally and by that one dropped key: `envelopeHash` `eaa41755…b9b87d` →
86
+ `2a57c911…c0af89`, `contentHashSha256` `cd036569…0e0202` → `e14adecf…a6af5f`.
87
+ The fixture records the reason beside them. **A package signed before this
88
+ change whose graph carried the source-id fallback no longer reproduces at the
89
+ defaults** — that is the intended consequence of dropping an asserted default,
90
+ and it is named here so nobody reads a moved hash as an accident.
91
+
92
+ ### Already on `main` since 0.4.1, and unreleased
93
+
94
+ - **The test files type-check**
95
+ ([civic-ai-tools#197](https://github.com/npstorey/civic-ai-tools/issues/197),
96
+ `0a861b1`). `tsconfig.test.json` extends the build config, emits nothing, and
97
+ adds the Node types the suite needs; the package's `typecheck` script runs
98
+ both configs, and `src/the-suite-type-checks.test.ts` guards that every test
99
+ file is inside the checking config and none is inside the emitting one. The
100
+ build config now sets `types: []`, so shipped source still cannot reach a
101
+ Node global and type-check. No runtime export changes.
102
+ - **The `civic:` vocabulary is named, and a rejected call states its elapsed**
103
+ ([civic-ai-tools#199](https://github.com/npstorey/civic-ai-tools/issues/199)
104
+ §2, `e4e32e7`). **Fourteen new `CIVIC_TERM_*` exports** —
105
+ `CIVIC_TERM_CONTENT_HASH`, `CIVIC_TERM_SERVER_URL`, `CIVIC_TERM_SOURCE_ID`,
106
+ `CIVIC_TERM_URL`, `CIVIC_TERM_PROMPT_TOKENS`, `CIVIC_TERM_COMPLETION_TOKENS`,
107
+ `CIVIC_TERM_TOOL_NAME`, `CIVIC_TERM_OPERATION_TYPE`, `CIVIC_TERM_DATASET_ID`,
108
+ `CIVIC_TERM_PORTAL_DOMAIN`, `CIVIC_TERM_DATASET_URL`,
109
+ `CIVIC_TERM_CROISSANT_METADATA_URL`, `CIVIC_TERM_RESPONSE_ROWS`,
110
+ `CIVIC_TERM_DURATION_MS` — replacing inline literals at twenty sites in
111
+ `capture/provenance.ts`, with no byte moving (a computed key inserts in the
112
+ same position with the same value). `purity.test.ts`'s guard derives its
113
+ universe rather than listing literals. The same commit corrected two comments
114
+ that said a rejected span carries no `tool.duration_ms`: the reference
115
+ producer records it, so a rejected call's activity emits `civic:durationMs`
116
+ beside `civic:failed`, the duration in its 0.3.1 position and the two markers
117
+ appended after it.
118
+ - **A golden case that carries `mcp_tool_call` spans**
119
+ ([civic-ai-tools#199](https://github.com/npstorey/civic-ai-tools/issues/199)
120
+ §1, `92efbae`). `__fixtures__/span-carrying-golden.json` — a ninth
121
+ golden-reproduction case, in its own file so the verbatim upstream copy stays
122
+ a copy. The other eight carry zero spans of any kind, so until it existed the
123
+ byte-compat suite was green for everything the tool-span loop does. Test-only;
124
+ no export changes.
125
+
126
+ **Correction to the 0.4.0 entry.** That entry's type-level-gate bullet says
127
+ "`tsconfig.json` excludes `src/**/*.test.ts`, so the suite type-checks nothing
128
+ — a test can drive a field the type does not have and `npm run typecheck` stays
129
+ green." **That has been false since `0a861b1`.** The suite is type-checked by
130
+ `tsconfig.test.json`, `npm run typecheck` runs it, and a type error in a test
131
+ file fails that gate. The sentence is true of 0.4.0 as published and is left in
132
+ place as the record of it; this is the forward correction.
133
+
134
+ ## 0.4.1 — 2026-09-12
135
+
136
+ Patch: one existing export changes its output for one input shape, and no
137
+ export is added, removed or retyped. That is measured across the release
138
+ boundary rather than asserted — the published `0.4.0` and this build each
139
+ expose 49 runtime exports from the package root, none added, none removed,
140
+ none changing runtime type, and no declared top-level name in `dist/*.d.ts`
141
+ differing between them. The tarball carries the same 28 files, and 22 of the
142
+ 24 under `dist/` are byte-identical to `0.4.0`; the two that are not are
143
+ `dist/rubric/adversarial-eval-core.js` and its `.d.ts`, the latter differing
144
+ by a doc comment alone.
145
+
146
+ **What this release claims about golden bytes, and what it does not.** Every
147
+ golden byte is unchanged. This time that sentence is not the evidence, and
148
+ saying so is the point: **no golden fixture reaches the export that changed.**
149
+ `buildEvaluationPrompt` is named in two test files, both under `src/rubric/`,
150
+ and in neither golden suite. `__fixtures__/website-golden.json` carries no
151
+ `queries` array at all; `__fixtures__/reference-golden.json` carries eight and
152
+ **zero** occurrences of the `failed` key that is this release's only new
153
+ branch. So the golden suites are green for this change the way they are green
154
+ for a change to a file they never import, and their green is not evidence
155
+ about it. This is the same narrowing
156
+ [civic-ai-tools#199](https://github.com/npstorey/civic-ai-tools/issues/199)
157
+ records for the graph builder, met from the other side.
158
+
159
+ What **is** evidence is a direct measurement of the changed export either side
160
+ of the release boundary: `buildEvaluationPrompt` imported from the published
161
+ `0.4.0` and from this build, run over the same package.
162
+
163
+ - A package in which **no entry records `failed: true`** renders
164
+ **byte-identical** on both — 492 bytes, `sha256 ed77c84e…` — over five entry
165
+ shapes: `resultRows: 12`, `resultRows: 0`, no `resultRows` at all,
166
+ `failed: false`, and a bare `failureKind` carrying no `failed`.
167
+ - The same package plus one entry recorded `failed: true` renders
168
+ **differently** (539 bytes against 607). That case is the reason the first
169
+ one is worth reporting: an instrument that answers "identical" whatever it is
170
+ fed has not measured anything, and this one is shown answering "different"
171
+ on the shape it must flag.
172
+
173
+ **The evaluator reads a rejected call as rejected**
174
+ ([civic-ai-tools#203](https://github.com/npstorey/civic-ai-tools/issues/203),
175
+ Wave N11 P-H1). Behaviour change, confined to one input shape.
176
+
177
+ - **The line that changes.** A `queries[]` entry the producer recorded as
178
+ rejected rendered `→ ? rows` — the same line an entry whose row count was
179
+ simply never recorded produces. A rejection and an unrecorded count were one
180
+ line to the evaluator, and the rubric asks that evaluator to cross-check the
181
+ answer's figures against the data the tool calls returned, so the conflation
182
+ invited it to read data that was never returned as merely uncounted. Its
183
+ score becomes a signed attestation. The entry now reads `→ REJECTED by the
184
+ source — no data was returned, and no row count is claimed`.
185
+ - **`failed: true` is the whole assertion.** `failureKind` is declared on the
186
+ shape and **deliberately not rendered**: it is producer-controlled text, and
187
+ the turn built here is read by a model whose score is signed, so the
188
+ rejection is stated in this module's own fixed words, interpolating nothing
189
+ the producer wrote and claiming no row count.
190
+ - **Absence stays absence.** An entry that records no outcome renders exactly
191
+ as it did before the fields existed. `failed: false` is not a rejection, and
192
+ a `failureKind` with no `failed` is not a rejection — the posture
193
+ `ToolCallSummary.failed` already takes in `capture/data-sources.ts`.
194
+ - **No dependency moves.** The two keys are read off a locally-declared shape
195
+ rather than imported, following that same file: produce-core `0.3.0`, which
196
+ the lockfile resolves, names neither key. No manifest change, no lockfile
197
+ bump, no new dependency.
198
+
199
+ **`RUBRIC_VERSION_SHA256` does not move, and the claim around it is corrected
200
+ rather than left standing**
201
+ ([civic-ai-tools#207](https://github.com/npstorey/civic-ai-tools/issues/207)).
202
+ The hash is `sha256Hex(EVALUATION_RUBRIC)` — the rubric text alone. It does not
203
+ cover `buildEvaluationPrompt`, the template that assembles that rubric with the
204
+ package's tool calls, data sources and model into the turn the evaluator
205
+ actually reads. So this release changes what an evaluator is shown without
206
+ changing the value an attestation pins as `methodology.promptSetVersion`: two
207
+ evaluations either side of it carry the same value, measured here as
208
+ `b62b193c992fd430cc82dff9a80a057bbe0b2b656fbdace450ca1a5d99fffa02` on both the
209
+ published `0.4.0` and this build.
210
+
211
+ - **The overclaim is retired in the same change.** ADR-0015 §3, Q26's
212
+ resolution note and the two module comments said the value pinned the whole
213
+ prompt set that produced the scores. They now say it pins the rubric
214
+ *wording*, and that the prompt template is outside the hash. The phrase they
215
+ used for the wider claim appears nowhere in the tree at this release.
216
+ - **The gap is filed, not resolved.** #207 records what the hash covers, what
217
+ it does not, and the two spec-level options — widen the hash, or add a second
218
+ field — with no recommendation. Which one is right is a specification
219
+ decision, not a package's.
220
+
221
+ A consumer sees none of this until its own lockfile moves: `^0.4.0` admits
222
+ `0.4.1`, so the manifest range need not change and the refresh is the whole of
223
+ the upgrade.
224
+
6
225
  ## 0.4.0 — 2026-09-04
7
226
 
8
227
  Minor: two optional input fields are added and no existing export is removed
@@ -31,7 +31,10 @@ export interface ProvenanceInput {
31
31
  export interface ProvenanceConfig {
32
32
  /** Platform-agent identity — the deployment publishing the record. */
33
33
  platformAgent: PlatformAgentConfig;
34
- /** Source registry supplying agent titles and MCP server URLs. */
34
+ /** Source registry supplying agent titles and MCP server URLs. Caller-
35
+ * supplied configuration: an instance passes the addresses it is pointed
36
+ * at, and an entry with no `serverUrl` makes its agent omit
37
+ * `civic:serverUrl` (civic-ai-tools#205). */
35
38
  sourceRegistry: CivicSourceRegistry;
36
39
  /** Source id untagged tool spans fall back to. Default `socrata`
37
40
  * (pre-source-tagging captures were Socrata-only). */
@@ -17,7 +17,7 @@
17
17
  // order is preserved throughout — the legacy hash chain's byte contract).
18
18
  import { makeProvGraph, makeEntityNode, makeActivityNode, makeAgentNode, provRef, provUsed, provWasGeneratedBy, provWasDerivedFrom, provWasAssociatedWith, xsdDateTime, } from '@typedstandards/produce-core';
19
19
  import { hash } from "./trace.js";
20
- import { CIVIC_VOCABULARY, CIVICAITOOLS_PLATFORM_AGENT, CIVIC_TERM_FAILED, CIVIC_TERM_FAILURE_KIND, } from "../format/vocabulary.js";
20
+ import { CIVIC_VOCABULARY, CIVICAITOOLS_PLATFORM_AGENT, CIVIC_TERM_COMPLETION_TOKENS, CIVIC_TERM_CONTENT_HASH, CIVIC_TERM_CROISSANT_METADATA_URL, CIVIC_TERM_DATASET_ID, CIVIC_TERM_DATASET_URL, CIVIC_TERM_DURATION_MS, CIVIC_TERM_FAILED, CIVIC_TERM_FAILURE_KIND, CIVIC_TERM_OPERATION_TYPE, CIVIC_TERM_PORTAL_DOMAIN, CIVIC_TERM_PROMPT_TOKENS, CIVIC_TERM_RESPONSE_ROWS, CIVIC_TERM_SERVER_URL, CIVIC_TERM_SOURCE_ID, CIVIC_TERM_TOOL_NAME, CIVIC_TERM_URL, } from "../format/vocabulary.js";
21
21
  import { CIVIC_SOURCE_REGISTRY, FALLBACK_SOURCE_ID, isDatasetKeyedSource, } from "../format/sources.js";
22
22
  /** The civicaitools.org reference deployment's values. Passed explicitly by
23
23
  * the reference app — never applied as a default, and never spread into
@@ -88,7 +88,7 @@ export function buildProvenanceGraph(trace, input, config) {
88
88
  // --- Entities ---
89
89
  // User prompt
90
90
  graph.push(makeEntityNode(vocab.urn(packageId, 'prompt', promptHash), {
91
- 'civic:contentHash': `sha256:${promptHash}`,
91
+ [CIVIC_TERM_CONTENT_HASH]: `sha256:${promptHash}`,
92
92
  'dcterms:description': 'User query prompt',
93
93
  ...(promptText ? { 'prov:value': promptText } : {}),
94
94
  }));
@@ -101,13 +101,13 @@ export function buildProvenanceGraph(trace, input, config) {
101
101
  const skillServerUrl = skillSpan ? getAttr(skillSpan.attributes, 'skill.mcp_server_url') : undefined;
102
102
  if (skillHash) {
103
103
  graph.push(makeEntityNode(vocab.urn(packageId, 'skill', skillHash), {
104
- 'civic:contentHash': `sha256:${skillHash}`,
104
+ [CIVIC_TERM_CONTENT_HASH]: `sha256:${skillHash}`,
105
105
  'dcterms:description': 'Composed MCP skill guidance (system prompt)',
106
106
  }, ['prov:Plan']));
107
107
  }
108
108
  // Final output
109
109
  graph.push(makeEntityNode(vocab.urn(packageId, 'output', outputHash), {
110
- 'civic:contentHash': `sha256:${outputHash}`,
110
+ [CIVIC_TERM_CONTENT_HASH]: `sha256:${outputHash}`,
111
111
  'dcterms:description': 'AI-generated analysis output',
112
112
  }));
113
113
  // --- Agents ---
@@ -127,6 +127,13 @@ export function buildProvenanceGraph(trace, input, config) {
127
127
  // configured fallback source (socrata — the only source at the time) for
128
128
  // backwards compatibility. Agent coordinates come from the source registry;
129
129
  // the skill-fetch span URL overrides the skill source's server URL.
130
+ //
131
+ // THE ADDRESS IS THE OPERATOR'S, OR IT IS ABSENT (civic-ai-tools#205, 0.5.0).
132
+ // `serverUrl` is optional on a registry entry and there is no stand-in for a
133
+ // source whose address this run does not know: a source the registry does
134
+ // not carry, and a registry entry with no address, both emit an agent with
135
+ // no `civic:serverUrl` key at all. The signed bytes then say the address is
136
+ // unknown instead of naming a server the analysis may never have contacted.
130
137
  const toolSpans = spans.filter(s => s.name === 'mcp_tool_call');
131
138
  const sourceAgentMap = {};
132
139
  for (const [id, info] of Object.entries(registry)) {
@@ -145,12 +152,13 @@ export function buildProvenanceGraph(trace, input, config) {
145
152
  const meta = sourceAgentMap[sourceId] ?? {
146
153
  urn: vocab.sourceAgentUrn(encodeURIComponent(sourceId)),
147
154
  title: `${sourceId} MCP Server`,
148
- serverUrl: sourceId,
149
155
  };
150
156
  graph.push(makeAgentNode(meta.urn, {
151
157
  'dcterms:title': meta.title,
152
- 'civic:serverUrl': meta.serverUrl,
153
- 'civic:sourceId': sourceId,
158
+ // Conditional spread in place, so a known address keeps its
159
+ // position in the key order the legacy chain hashes.
160
+ ...(meta.serverUrl ? { [CIVIC_TERM_SERVER_URL]: meta.serverUrl } : {}),
161
+ [CIVIC_TERM_SOURCE_ID]: sourceId,
154
162
  }, ['prov:SoftwareAgent']));
155
163
  }
156
164
  // Resolve a source id to its agent URN, falling back to the configured
@@ -165,7 +173,7 @@ export function buildProvenanceGraph(trace, input, config) {
165
173
  // Platform
166
174
  graph.push(makeAgentNode(vocab.platformUrn(config.platformAgent.id), {
167
175
  'dcterms:title': config.platformAgent.title,
168
- 'civic:url': config.platformAgent.url,
176
+ [CIVIC_TERM_URL]: config.platformAgent.url,
169
177
  }, ['prov:SoftwareAgent']));
170
178
  // --- Activities ---
171
179
  // LLM inference spans
@@ -190,8 +198,8 @@ export function buildProvenanceGraph(trace, input, config) {
190
198
  ...(span.endTimeUnixNano
191
199
  ? { 'prov:endedAtTime': xsdDateTime(nanoToIso(span.endTimeUnixNano)) }
192
200
  : {}),
193
- ...(promptTokens ? { 'civic:promptTokens': Number(promptTokens) } : {}),
194
- ...(completionTokens ? { 'civic:completionTokens': Number(completionTokens) } : {}),
201
+ ...(promptTokens ? { [CIVIC_TERM_PROMPT_TOKENS]: Number(promptTokens) } : {}),
202
+ ...(completionTokens ? { [CIVIC_TERM_COMPLETION_TOKENS]: Number(completionTokens) } : {}),
195
203
  }));
196
204
  }
197
205
  // MCP tool call spans.
@@ -233,10 +241,10 @@ export function buildProvenanceGraph(trace, input, config) {
233
241
  .sort((a, b) => Number(b.endTimeUnixNano || '0') - Number(a.endTimeUnixNano || '0'))[0];
234
242
  const queryUrn = vocab.urn(packageId, 'query', queryHash);
235
243
  graph.push(makeEntityNode(queryUrn, {
236
- 'civic:contentHash': `sha256:${queryHash}`,
244
+ [CIVIC_TERM_CONTENT_HASH]: `sha256:${queryHash}`,
237
245
  // Omitted — not placeholdered — when the span named no tool.
238
- ...(toolName ? { 'civic:toolName': toolName } : {}),
239
- 'civic:operationType': opType,
246
+ ...(toolName ? { [CIVIC_TERM_TOOL_NAME]: toolName } : {}),
247
+ [CIVIC_TERM_OPERATION_TYPE]: opType,
240
248
  'dcterms:description': `MCP tool arguments (${opType})`,
241
249
  ...(precedingInference
242
250
  ? provWasGeneratedBy(vocab.urn(packageId, 'inference', precedingInference.spanId))
@@ -264,9 +272,9 @@ export function buildProvenanceGraph(trace, input, config) {
264
272
  : `Data response from ${sourceAgentMap[toolSource]?.title || toolSource}`;
265
273
  const responseRows = getAttr(span.attributes, 'tool.response_rows');
266
274
  graph.push(makeEntityNode(dataUrn, {
267
- 'civic:contentHash': `sha256:${responseHash}`,
275
+ [CIVIC_TERM_CONTENT_HASH]: `sha256:${responseHash}`,
268
276
  'dcterms:description': description,
269
- 'civic:sourceId': toolSource,
277
+ [CIVIC_TERM_SOURCE_ID]: toolSource,
270
278
  ...provWasGeneratedBy(toolCallUrn),
271
279
  // Croissant 1.1 placeholder — only meaningful for dataset-keyed
272
280
  // sources today. The dataset id is stated whenever the span carried
@@ -274,17 +282,17 @@ export function buildProvenanceGraph(trace, input, config) {
274
282
  // the portal as well. Key order is the byte contract.
275
283
  ...(toolSourceDatasetKeyed && datasetId
276
284
  ? {
277
- 'civic:datasetId': datasetId,
285
+ [CIVIC_TERM_DATASET_ID]: datasetId,
278
286
  ...(portalDomain
279
287
  ? {
280
- 'civic:portalDomain': portalDomain,
281
- 'civic:datasetUrl': `https://${portalDomain}/d/${datasetId}`,
288
+ [CIVIC_TERM_PORTAL_DOMAIN]: portalDomain,
289
+ [CIVIC_TERM_DATASET_URL]: `https://${portalDomain}/d/${datasetId}`,
282
290
  }
283
291
  : {}),
284
- 'civic:croissantMetadataUrl': null, // hook for future Croissant integration
292
+ [CIVIC_TERM_CROISSANT_METADATA_URL]: null, // hook for future Croissant integration
285
293
  }
286
294
  : {}),
287
- ...(responseRows ? { 'civic:responseRows': Number(responseRows) } : {}),
295
+ ...(responseRows ? { [CIVIC_TERM_RESPONSE_ROWS]: Number(responseRows) } : {}),
288
296
  }));
289
297
  }
290
298
  // Tool call activity — associated with the MCP source agent that handled
@@ -317,7 +325,7 @@ export function buildProvenanceGraph(trace, input, config) {
317
325
  'dcterms:description': toolName
318
326
  ? `MCP tool call: ${toolName} (${opType})`
319
327
  : `MCP tool call (${opType})`,
320
- 'civic:sourceId': toolSource,
328
+ [CIVIC_TERM_SOURCE_ID]: toolSource,
321
329
  ...provUsed([queryUrn]),
322
330
  ...provWasAssociatedWith(toolAgentUrn),
323
331
  ...(span.startTimeUnixNano
@@ -326,16 +334,25 @@ export function buildProvenanceGraph(trace, input, config) {
326
334
  ...(span.endTimeUnixNano
327
335
  ? { 'prov:endedAtTime': xsdDateTime(nanoToIso(span.endTimeUnixNano)) }
328
336
  : {}),
329
- ...(durationMs ? { 'civic:durationMs': Number(durationMs) } : {}),
337
+ ...(durationMs ? { [CIVIC_TERM_DURATION_MS]: Number(durationMs) } : {}),
330
338
  // Appended LAST and spread conditionally, exactly as `civic:durationMs`
331
339
  // above is: a span that recorded no rejection yields the key list it
332
340
  // yielded at 0.3.1, in the same order, and property insertion order is
333
341
  // the legacy chain's byte contract. `false` is never emitted — a
334
342
  // producer that stated "not failed" and one that stated nothing must
335
343
  // read the same, which is what makes a marker that IS present mean
336
- // something. A rejected span carries no `tool.duration_ms` from the
337
- // reference producer today (civic-ai-tools-website#413), so
338
- // `civic:durationMs` simply does not fire beside these two.
344
+ // something.
345
+ //
346
+ // A REJECTED SPAN NOW CARRIES A DURATION. Until
347
+ // civic-ai-tools-website#413 shipped (Wave N11 P4, merged `ec7f173`)
348
+ // the reference producer measured a rejected call's elapsed and
349
+ // discarded it, so this comment said `civic:durationMs` "simply does
350
+ // not fire beside these two". It fires: the producer writes
351
+ // `tool.duration_ms` on the rejection path, and the conditional above
352
+ // states it — BEFORE the two markers, in the position it has held
353
+ // since 0.3.1, because that conditional is unchanged and this one is
354
+ // still appended after it. `capture/provenance.test.ts` drives that
355
+ // combination and pins the whole key list for it.
339
356
  ...(failed === true ? { [CIVIC_TERM_FAILED]: true } : {}),
340
357
  ...(failureKind ? { [CIVIC_TERM_FAILURE_KIND]: failureKind } : {}),
341
358
  }));
@@ -39,18 +39,37 @@ export declare function selectContentCanonicalization(contentProfile?: string):
39
39
  * value to pass as the envelope's `summary` field, or `undefined` to omit.
40
40
  */
41
41
  export declare function deriveSummaryEmission(contentProfile?: string, summary?: string): string | undefined;
42
+ /** One MCP server the run had available, as the environment extension carries
43
+ * it — spec §8.7.1 requirement 3: an object with `url` and optional `name`.
44
+ * The CALLER supplies the order and the names; the harness neither sorts nor
45
+ * invents them. */
46
+ export interface DatHereMcpServer {
47
+ /** The server's configured address. */
48
+ url: string;
49
+ /** Optional label for the server (the reference caller passes the source
50
+ * id). Omitted from the emitted object when absent, so a nameless entry
51
+ * stays byte-identical to the pre-0.5.0 single-server shape. */
52
+ name?: string;
53
+ }
42
54
  /**
43
55
  * Build the `org.civicaitools.environment` extension content for a datHere
44
56
  * package. Per OES §9.1.1 requirement 3 the extension MUST carry
45
57
  * `modelVersion`, `temperature`, `mcpServers`, `toolDefinitions`, `host`.
46
58
  *
59
+ * `mcpServers` accepts EITHER shape (additive since 0.5.0,
60
+ * civic-ai-tools-website#449): a single URL string — the trace's skill-fetch
61
+ * span URL, the only server the pre-0.5.0 builder could name — or the list of
62
+ * servers the run actually had, in the caller's order, each with an optional
63
+ * `name`. The single-URL shape produces byte-identical output to before, so
64
+ * an already-signed package still reproduces.
65
+ *
47
66
  * Prototype limitations carried over from the reference implementation
48
67
  * (known gaps, tightened in follow-up work): `temperature` placeholder `0`,
49
- * `toolDefinitions` placeholder `[]`; `mcpServers` derives from the trace's
50
- * skill-fetch span URL (the analysis's primary MCP server). Fields captured
51
- * honestly: `modelVersion` and `host`.
68
+ * `toolDefinitions` placeholder `[]`. Fields captured honestly:
69
+ * `modelVersion`, `host`, and — when the caller passes the list — the servers
70
+ * the run had.
52
71
  */
53
- export declare function buildDatHereEnvironment(model: string, skillMcpServerUrl: string | undefined, config: DatHereEnvironmentConfig): Record<string, unknown>;
72
+ export declare function buildDatHereEnvironment(model: string, mcpServers: string | readonly DatHereMcpServer[] | undefined, config: DatHereEnvironmentConfig): Record<string, unknown>;
54
73
  /** Input to the composite datHere-policy derivation. */
55
74
  export interface DatHerePolicyInput {
56
75
  /** Model identifier (becomes `environment.modelVersion`). */
@@ -62,8 +81,15 @@ export interface DatHerePolicyInput {
62
81
  /** Candidate summary text (emitted only under the datHere profile). */
63
82
  summary?: string;
64
83
  /** MCP server URL from the trace's skill-fetch span (capture-side
65
- * extraction — see src/capture/skill-metadata.ts). */
84
+ * extraction — see src/capture/skill-metadata.ts). Consulted only when
85
+ * `mcpServers` is absent, so the pre-0.5.0 call shape keeps producing
86
+ * byte-identical envelopes. */
66
87
  skillMcpServerUrl?: string;
88
+ /** Every MCP server the run had available, in the order the caller wants
89
+ * them emitted, each with an optional `name`
90
+ * (civic-ai-tools-website#449). When supplied this REPLACES
91
+ * `skillMcpServerUrl` as the source of `environment.mcpServers`. */
92
+ mcpServers?: readonly DatHereMcpServer[];
67
93
  /** Caller-supplied extensions to layer the environment extension onto. */
68
94
  extensions?: Record<string, unknown>;
69
95
  }
@@ -69,26 +69,50 @@ export function deriveSummaryEmission(contentProfile, summary) {
69
69
  ? summary
70
70
  : undefined;
71
71
  }
72
+ /** Normalize the two accepted `mcpServers` inputs to the emitted array.
73
+ *
74
+ * A bare string is the pre-0.5.0 shape — the trace's skill-fetch URL — and
75
+ * yields at most the one `{ url }` entry, exactly as before. A list is
76
+ * emitted in the order given, each entry carrying `name` only when the
77
+ * caller supplied one. An entry with an empty `url` contributes nothing,
78
+ * which is the same rule the single-URL shape always applied: a server with
79
+ * no address is stated by absence, never by an empty string. */
80
+ function normalizeMcpServers(servers) {
81
+ if (servers === undefined)
82
+ return [];
83
+ if (typeof servers === 'string') {
84
+ return servers ? [{ url: servers }] : [];
85
+ }
86
+ return servers
87
+ .filter((server) => Boolean(server.url))
88
+ .map((server) => ({
89
+ url: server.url,
90
+ ...(server.name ? { name: server.name } : {}),
91
+ }));
92
+ }
72
93
  /**
73
94
  * Build the `org.civicaitools.environment` extension content for a datHere
74
95
  * package. Per OES §9.1.1 requirement 3 the extension MUST carry
75
96
  * `modelVersion`, `temperature`, `mcpServers`, `toolDefinitions`, `host`.
76
97
  *
98
+ * `mcpServers` accepts EITHER shape (additive since 0.5.0,
99
+ * civic-ai-tools-website#449): a single URL string — the trace's skill-fetch
100
+ * span URL, the only server the pre-0.5.0 builder could name — or the list of
101
+ * servers the run actually had, in the caller's order, each with an optional
102
+ * `name`. The single-URL shape produces byte-identical output to before, so
103
+ * an already-signed package still reproduces.
104
+ *
77
105
  * Prototype limitations carried over from the reference implementation
78
106
  * (known gaps, tightened in follow-up work): `temperature` placeholder `0`,
79
- * `toolDefinitions` placeholder `[]`; `mcpServers` derives from the trace's
80
- * skill-fetch span URL (the analysis's primary MCP server). Fields captured
81
- * honestly: `modelVersion` and `host`.
107
+ * `toolDefinitions` placeholder `[]`. Fields captured honestly:
108
+ * `modelVersion`, `host`, and — when the caller passes the list — the servers
109
+ * the run had.
82
110
  */
83
- export function buildDatHereEnvironment(model, skillMcpServerUrl, config) {
84
- const mcpServers = [];
85
- if (skillMcpServerUrl) {
86
- mcpServers.push({ url: skillMcpServerUrl });
87
- }
111
+ export function buildDatHereEnvironment(model, mcpServers, config) {
88
112
  return {
89
113
  modelVersion: model,
90
114
  temperature: 0,
91
- mcpServers,
115
+ mcpServers: normalizeMcpServers(mcpServers),
92
116
  toolDefinitions: [],
93
117
  host: config.host,
94
118
  };
@@ -102,7 +126,10 @@ export function buildDatHereEnvironment(model, skillMcpServerUrl, config) {
102
126
  export function deriveDatHereEnvelopeFields(input, config) {
103
127
  const extensions = { ...(input.extensions ?? {}) };
104
128
  if (input.contentProfile === DATHERE_CONTENT_PROFILE) {
105
- extensions[ENVIRONMENT_EXTENSION_KEY] = buildDatHereEnvironment(input.model, input.skillMcpServerUrl, config);
129
+ extensions[ENVIRONMENT_EXTENSION_KEY] = buildDatHereEnvironment(input.model,
130
+ // The list when the caller has one, the single skill-fetch URL
131
+ // otherwise — an input carrying neither still emits `mcpServers: []`.
132
+ input.mcpServers ?? input.skillMcpServerUrl, config);
106
133
  }
107
134
  return {
108
135
  producerProfile: deriveProducerProfile(input.producerProfile, input.contentProfile),
@@ -5,8 +5,18 @@ export interface CivicSourceInfo {
5
5
  displayName: string;
6
6
  /** PROV agent `dcterms:title`, e.g. "Socrata MCP Server". */
7
7
  agentTitle: string;
8
- /** MCP endpoint URL — the PROV agent's `civic:serverUrl`. */
9
- serverUrl: string;
8
+ /**
9
+ * MCP endpoint URL — the PROV agent's `civic:serverUrl`, and this source's
10
+ * display address.
11
+ *
12
+ * OPTIONAL since 0.5.0 (civic-ai-tools#205). The address a run reached is
13
+ * the OPERATOR's configuration, not a fact the harness knows, so a caller
14
+ * passes a registry whose entries carry the addresses that instance is
15
+ * pointed at. An entry that carries no address makes the provenance graph
16
+ * OMIT `civic:serverUrl` from that source's agent entirely, rather than
17
+ * assert a stand-in: honest absence over an asserted default.
18
+ */
19
+ serverUrl?: string;
10
20
  /** `dataSources[].catalogType` value for entries from this source. */
11
21
  catalogType: string;
12
22
  /**
@@ -22,7 +32,14 @@ export interface CivicSourceInfo {
22
32
  * order for aggregate `dataSources` entries. */
23
33
  export type CivicSourceRegistry = Record<string, CivicSourceInfo>;
24
34
  /** The civic reference registry — the three demo sources. Passed explicitly
25
- * by the reference app — never applied as a default. */
35
+ * by the reference app — never applied as a default.
36
+ *
37
+ * THE `serverUrl` VALUES BELOW ARE THE REFERENCE DEPLOYMENT'S OWN, and since
38
+ * 0.5.0 (civic-ai-tools#205) that is the whole of their role: a default
39
+ * display address, and — on the two aggregate sources — the companion of
40
+ * `aggregatePortalUrl`. They are not what a record asserts about a run. An
41
+ * instance pointed at other addresses passes a registry carrying those; an
42
+ * entry with no address emits no `civic:serverUrl` at all. */
26
43
  export declare const CIVIC_SOURCE_REGISTRY: CivicSourceRegistry;
27
44
  /** The source id untagged/pre-M9.1 captures fall back to. Socrata was the
28
45
  * only source when those packages were written. */
@@ -13,7 +13,14 @@
13
13
  // demo map is exported as `CIVIC_SOURCE_REGISTRY` for explicit use, never
14
14
  // applied as a default.
15
15
  /** The civic reference registry — the three demo sources. Passed explicitly
16
- * by the reference app — never applied as a default. */
16
+ * by the reference app — never applied as a default.
17
+ *
18
+ * THE `serverUrl` VALUES BELOW ARE THE REFERENCE DEPLOYMENT'S OWN, and since
19
+ * 0.5.0 (civic-ai-tools#205) that is the whole of their role: a default
20
+ * display address, and — on the two aggregate sources — the companion of
21
+ * `aggregatePortalUrl`. They are not what a record asserts about a run. An
22
+ * instance pointed at other addresses passes a registry carrying those; an
23
+ * entry with no address emits no `civic:serverUrl` at all. */
17
24
  export const CIVIC_SOURCE_REGISTRY = {
18
25
  socrata: {
19
26
  displayName: 'Socrata',
@@ -35,6 +35,83 @@ export declare const CIVIC_TERM_FAILED = "civic:failed";
35
35
  * wider vocabulary widens this field rather than putting prose in it.
36
36
  */
37
37
  export declare const CIVIC_TERM_FAILURE_KIND = "civic:failureKind";
38
+ /**
39
+ * `civic:contentHash` — the SHA-256 of the bytes an entity stands for, as
40
+ * `sha256:<hex>`. Emitted on the prompt, skill, output, tool-argument and
41
+ * data-response entities: a reader hashes the content it holds and compares.
42
+ */
43
+ export declare const CIVIC_TERM_CONTENT_HASH = "civic:contentHash";
44
+ /**
45
+ * `civic:serverUrl` — the MCP server URL an source agent answered from, as
46
+ * the deployment's source registry states it (or, for the skill source, as
47
+ * the trace's skill-fetch span carried it).
48
+ */
49
+ export declare const CIVIC_TERM_SERVER_URL = "civic:serverUrl";
50
+ /**
51
+ * `civic:sourceId` — the stable id of the source that answered a call, on
52
+ * both the tool-call activity and the data-response entity. It is the
53
+ * registry key, not a display name: a reader resolves it, and the graph never
54
+ * spells the reader-facing name.
55
+ */
56
+ export declare const CIVIC_TERM_SOURCE_ID = "civic:sourceId";
57
+ /**
58
+ * `civic:url` — the public URL of the platform agent that published the
59
+ * record (the deployment's own address, a typed config input).
60
+ */
61
+ export declare const CIVIC_TERM_URL = "civic:url";
62
+ /** `civic:promptTokens` — prompt tokens the span reported for one inference.
63
+ * Emitted only when the span carried the count; never zero-filled. */
64
+ export declare const CIVIC_TERM_PROMPT_TOKENS = "civic:promptTokens";
65
+ /** `civic:completionTokens` — completion tokens the span reported for one
66
+ * inference. Emitted only when the span carried the count. */
67
+ export declare const CIVIC_TERM_COMPLETION_TOKENS = "civic:completionTokens";
68
+ /**
69
+ * `civic:toolName` — the tool a call invoked, verbatim from the span.
70
+ * Omitted, never placeholdered, when the span named no tool: the graph states
71
+ * absence as absence.
72
+ */
73
+ export declare const CIVIC_TERM_TOOL_NAME = "civic:toolName";
74
+ /**
75
+ * `civic:operationType` — the producer's classification of what a call did
76
+ * (`query`, `metadata`, `unknown`, …). An open string the graph states
77
+ * verbatim and never re-derives.
78
+ */
79
+ export declare const CIVIC_TERM_OPERATION_TYPE = "civic:operationType";
80
+ /**
81
+ * `civic:datasetId` — the dataset a data response came from, stated whenever
82
+ * the span carried one and the source is dataset-keyed.
83
+ */
84
+ export declare const CIVIC_TERM_DATASET_ID = "civic:datasetId";
85
+ /**
86
+ * `civic:portalDomain` — the portal host a tool span carried. Never the run's
87
+ * selected portal: a call that addressed no portal yields a response
88
+ * attributed to none.
89
+ */
90
+ export declare const CIVIC_TERM_PORTAL_DOMAIN = "civic:portalDomain";
91
+ /**
92
+ * `civic:datasetUrl` — the canonical dataset URL, minted only when the span
93
+ * carried BOTH the portal host and the dataset id (a URL needs a host the
94
+ * span actually stated).
95
+ */
96
+ export declare const CIVIC_TERM_DATASET_URL = "civic:datasetUrl";
97
+ /**
98
+ * `civic:croissantMetadataUrl` — the Croissant 1.1 metadata URL for a
99
+ * dataset-keyed data response. Emitted as an explicit `null` placeholder for
100
+ * a future integration; the key is inside signed bytes, so it is stated here
101
+ * rather than left as an undocumented literal.
102
+ */
103
+ export declare const CIVIC_TERM_CROISSANT_METADATA_URL = "civic:croissantMetadataUrl";
104
+ /** `civic:responseRows` — the row count a tool response carried, when the
105
+ * span reported one. */
106
+ export declare const CIVIC_TERM_RESPONSE_ROWS = "civic:responseRows";
107
+ /**
108
+ * `civic:durationMs` — the elapsed the producer measured for one tool call.
109
+ * Emitted whenever the span carried it, on a call that answered and on one
110
+ * the source REFUSED alike: since civic-ai-tools-website#413 the reference
111
+ * producer records the elapsed on its rejection path too, so this term and
112
+ * {@link CIVIC_TERM_FAILED} appear together on the same activity.
113
+ */
114
+ export declare const CIVIC_TERM_DURATION_MS = "civic:durationMs";
38
115
  /**
39
116
  * One era of the civic vocabulary: the two literals plus the emitters bound
40
117
  * to them. The vocabulary is not deployment configuration (a deployment does
@@ -37,16 +37,25 @@ export const PRIOR_ERA_CIVIC_URN_PREFIX = 'urn:civic-evidence';
37
37
  // A `civic:` property name is vocabulary as much as the namespace it hangs
38
38
  // under: it is the word a reader of a signed graph interprets, so it is
39
39
  // declared here and imported by the builder that emits it, never spelled as a
40
- // literal inside capture/ (`purity.test.ts` enforces that for the terms below).
41
- // Both terms are era-independent — the settlement moved the NAMESPACE, not the
42
- // property names — so they carry no era qualifier and are minted for the first
43
- // time after it.
40
+ // literal inside capture/. Every term is era-independent — the 2026-08-19
41
+ // settlement moved the NAMESPACE, not the property names — so none carries an
42
+ // era qualifier.
44
43
  //
45
- // The settlement-era property names that predate Wave N10 (`civic:sourceId`,
46
- // `civic:durationMs`, `civic:datasetId` and their siblings) are still inline
47
- // literals in the capture-side builder. That is a real inconsistency, recorded
48
- // rather than fixed in passing: moving them is a byte-sensitive change across
49
- // every golden fixture and belongs to a phase of its own.
44
+ // COMPLETE, AND GUARDED AS COMPLETE (civic-ai-tools#199 §2). Until this wave
45
+ // two terms were declared here and the other fourteen were spelled inline in
46
+ // capture/provenance.ts and declared nowhere, which the note that stood in
47
+ // this place recorded as "a real inconsistency ... a phase of its own". This
48
+ // is that phase. `purity.test.ts` no longer holds a list of the terms it
49
+ // knows about — it derives the universe from both ends: no string literal
50
+ // anywhere in capture/ may be a `civic:` term, and every `civic:` key a
51
+ // DRIVEN graph emits must be the value of a constant declared here. A term
52
+ // invented inline tomorrow fails both legs without anyone editing a list.
53
+ //
54
+ // NO BYTE MOVES. Naming a key changes no emitted byte: a computed key
55
+ // (`{ [CIVIC_TERM_SOURCE_ID]: v }`) inserts in the same position with the same
56
+ // value as the literal it replaces, and property insertion order is the legacy
57
+ // chain's byte contract. Both golden suites and the span-carrying golden case
58
+ // added in the same phase measure that rather than assume it.
50
59
  /**
51
60
  * `civic:failed` — a tool-call activity whose call the SOURCE REFUSED.
52
61
  *
@@ -71,6 +80,83 @@ export const CIVIC_TERM_FAILED = 'civic:failed';
71
80
  * wider vocabulary widens this field rather than putting prose in it.
72
81
  */
73
82
  export const CIVIC_TERM_FAILURE_KIND = 'civic:failureKind';
83
+ /**
84
+ * `civic:contentHash` — the SHA-256 of the bytes an entity stands for, as
85
+ * `sha256:<hex>`. Emitted on the prompt, skill, output, tool-argument and
86
+ * data-response entities: a reader hashes the content it holds and compares.
87
+ */
88
+ export const CIVIC_TERM_CONTENT_HASH = 'civic:contentHash';
89
+ /**
90
+ * `civic:serverUrl` — the MCP server URL an source agent answered from, as
91
+ * the deployment's source registry states it (or, for the skill source, as
92
+ * the trace's skill-fetch span carried it).
93
+ */
94
+ export const CIVIC_TERM_SERVER_URL = 'civic:serverUrl';
95
+ /**
96
+ * `civic:sourceId` — the stable id of the source that answered a call, on
97
+ * both the tool-call activity and the data-response entity. It is the
98
+ * registry key, not a display name: a reader resolves it, and the graph never
99
+ * spells the reader-facing name.
100
+ */
101
+ export const CIVIC_TERM_SOURCE_ID = 'civic:sourceId';
102
+ /**
103
+ * `civic:url` — the public URL of the platform agent that published the
104
+ * record (the deployment's own address, a typed config input).
105
+ */
106
+ export const CIVIC_TERM_URL = 'civic:url';
107
+ /** `civic:promptTokens` — prompt tokens the span reported for one inference.
108
+ * Emitted only when the span carried the count; never zero-filled. */
109
+ export const CIVIC_TERM_PROMPT_TOKENS = 'civic:promptTokens';
110
+ /** `civic:completionTokens` — completion tokens the span reported for one
111
+ * inference. Emitted only when the span carried the count. */
112
+ export const CIVIC_TERM_COMPLETION_TOKENS = 'civic:completionTokens';
113
+ /**
114
+ * `civic:toolName` — the tool a call invoked, verbatim from the span.
115
+ * Omitted, never placeholdered, when the span named no tool: the graph states
116
+ * absence as absence.
117
+ */
118
+ export const CIVIC_TERM_TOOL_NAME = 'civic:toolName';
119
+ /**
120
+ * `civic:operationType` — the producer's classification of what a call did
121
+ * (`query`, `metadata`, `unknown`, …). An open string the graph states
122
+ * verbatim and never re-derives.
123
+ */
124
+ export const CIVIC_TERM_OPERATION_TYPE = 'civic:operationType';
125
+ /**
126
+ * `civic:datasetId` — the dataset a data response came from, stated whenever
127
+ * the span carried one and the source is dataset-keyed.
128
+ */
129
+ export const CIVIC_TERM_DATASET_ID = 'civic:datasetId';
130
+ /**
131
+ * `civic:portalDomain` — the portal host a tool span carried. Never the run's
132
+ * selected portal: a call that addressed no portal yields a response
133
+ * attributed to none.
134
+ */
135
+ export const CIVIC_TERM_PORTAL_DOMAIN = 'civic:portalDomain';
136
+ /**
137
+ * `civic:datasetUrl` — the canonical dataset URL, minted only when the span
138
+ * carried BOTH the portal host and the dataset id (a URL needs a host the
139
+ * span actually stated).
140
+ */
141
+ export const CIVIC_TERM_DATASET_URL = 'civic:datasetUrl';
142
+ /**
143
+ * `civic:croissantMetadataUrl` — the Croissant 1.1 metadata URL for a
144
+ * dataset-keyed data response. Emitted as an explicit `null` placeholder for
145
+ * a future integration; the key is inside signed bytes, so it is stated here
146
+ * rather than left as an undocumented literal.
147
+ */
148
+ export const CIVIC_TERM_CROISSANT_METADATA_URL = 'civic:croissantMetadataUrl';
149
+ /** `civic:responseRows` — the row count a tool response carried, when the
150
+ * span reported one. */
151
+ export const CIVIC_TERM_RESPONSE_ROWS = 'civic:responseRows';
152
+ /**
153
+ * `civic:durationMs` — the elapsed the producer measured for one tool call.
154
+ * Emitted whenever the span carried it, on a call that answered and on one
155
+ * the source REFUSED alike: since civic-ai-tools-website#413 the reference
156
+ * producer records the elapsed on its rejection path too, so this term and
157
+ * {@link CIVIC_TERM_FAILED} appear together on the same activity.
158
+ */
159
+ export const CIVIC_TERM_DURATION_MS = 'civic:durationMs';
74
160
  /** Bind the emitters to one era's two literals. */
75
161
  export function makeCivicVocabulary(ns, urnPrefix) {
76
162
  return {
@@ -6,7 +6,10 @@ export declare const EVALUATION_CRITERIA: readonly ["dataSourceIdentification",
6
6
  export declare const EVALUATION_RUBRIC = "You are an independent evaluator assessing an AI-generated civic data analysis.\n\nYou will receive:\n1. The original prompt (question asked)\n2. The tool calls made (MCP queries to Socrata open data portals)\n3. The AI's final output/analysis\n\nEvaluate the analysis against these 6 criteria, scoring each 1-10:\n\na) **Data Source Identification** (1-10): Does the output correctly identify the data source(s) and time period? Are dataset IDs, portal domains, and date ranges accurate?\n\nb) **Quantitative Claim Support** (1-10): Are the quantitative claims (numbers, percentages, rankings) supported by the data returned in the tool calls? Cross-check key figures against the raw data.\n\nc) **Confounders and Bias** (1-10): Does the analysis acknowledge obvious confounders, selection biases, or framing issues? Are there lurking variables or cherry-picked timeframes?\n\nd) **Geographic Scope** (1-10): Is the geographic scope appropriate for the question? Does the analysis avoid over-generalizing from one jurisdiction?\n\ne) **Limitations Noted** (1-10): Are limitations and caveats noted? Does the analysis flag data quality issues, missing fields, or incomplete coverage?\n\nf) **Contradictory Conclusion** (1-10): Could the same data reasonably support a contradictory conclusion? Does the analysis consider alternative interpretations?\n\nRespond in this exact JSON format (no markdown fences, just raw JSON):\n{\n \"dataSourceIdentification\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n \"quantitativeClaimSupport\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n \"confoundersAndBias\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n \"geographicScope\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n \"limitationsNoted\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n \"contradictoryConclusion\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n \"overallScore\": <average of all 6 scores, one decimal>,\n \"assessment\": \"<2-4 sentence overall assessment>\"\n}";
7
7
  /** The methodology's `promptSetVersion`: SHA-256 of the rubric text, computed
8
8
  * once at module load. Any wording change to the rubric produces a new
9
- * version, so an attestation pins the exact prompt set that scored it. */
9
+ * version, so an attestation pins the rubric WORDING that scored it — and
10
+ * that wording only. The prompt template around it (`buildEvaluationPrompt`)
11
+ * is outside this hash, so one value does not identify one set of evaluator
12
+ * inputs (civic-ai-tools#207). */
10
13
  export declare const RUBRIC_VERSION_SHA256: string;
11
14
  /** Build the user-turn evaluation content from a package's signed fields.
12
15
  * Takes `RecordPackage` — produce-core 0.3.0's settlement-era name for the
@@ -5,8 +5,14 @@
5
5
  // brief §1. The model runner and DB/blob emission (`adversarial-eval.ts`)
6
6
  // stay app-side — this module never performs I/O.
7
7
  //
8
- // ⚠ RUBRIC_VERSION_SHA256 is a Q26-pinned version hash: attestations pin the
9
- // exact prompt set that scored them by this value. The rubric text below is
8
+ // ⚠ RUBRIC_VERSION_SHA256 is a Q26-pinned version hash over THE RUBRIC TEXT
9
+ // ALONE (`sha256Hex(EVALUATION_RUBRIC)`). It does NOT cover
10
+ // `buildEvaluationPrompt` below — the template that assembles the rubric with
11
+ // the package's tool calls, data sources and model into the turn the evaluator
12
+ // actually reads. So two evaluations either side of a change to that template
13
+ // carry the SAME `methodology.promptSetVersion` while the evaluator saw
14
+ // different inputs; civic-ai-tools#207 files that gap for the specification to
15
+ // settle, and this module does not resolve it. The rubric text below is
10
16
  // byte-exact with the reference implementation (the relocation changes no
11
17
  // hashes), and the test suite asserts the literal digest. Any wording change
12
18
  // to EVALUATION_RUBRIC is a NEW rubric version — never edit the text without
@@ -57,8 +63,24 @@ Respond in this exact JSON format (no markdown fences, just raw JSON):
57
63
  }`;
58
64
  /** The methodology's `promptSetVersion`: SHA-256 of the rubric text, computed
59
65
  * once at module load. Any wording change to the rubric produces a new
60
- * version, so an attestation pins the exact prompt set that scored it. */
66
+ * version, so an attestation pins the rubric WORDING that scored it — and
67
+ * that wording only. The prompt template around it (`buildEvaluationPrompt`)
68
+ * is outside this hash, so one value does not identify one set of evaluator
69
+ * inputs (civic-ai-tools#207). */
61
70
  export const RUBRIC_VERSION_SHA256 = sha256Hex(EVALUATION_RUBRIC);
71
+ /** Did the producer record this entry as REJECTED by the source? Only a
72
+ * literal `failed: true` is a rejection (see `RecordedCallOutcome`). */
73
+ function wasRecordedRejected(query) {
74
+ return query.failed === true;
75
+ }
76
+ /** What the evaluator is told about a call the source rejected. A FIXED
77
+ * string: it interpolates nothing the producer wrote, and it claims no row
78
+ * count — there were no rows to count, which is neither "zero rows" nor "an
79
+ * unknown number of rows". The rubric asks the evaluator to cross-check
80
+ * figures against the data returned in the tool calls, so a rejected call
81
+ * reading `→ ? rows` invited it to treat absent data as merely unrecorded
82
+ * (civic-ai-tools#203). */
83
+ const REJECTED_CALL_RENDERING = 'REJECTED by the source — no data was returned, and no row count is claimed';
62
84
  /** Build the user-turn evaluation content from a package's signed fields.
63
85
  * Takes `RecordPackage` — produce-core 0.3.0's settlement-era name for the
64
86
  * same object (spec Appendix J, migration class alias-and-deprecate). The
@@ -70,7 +92,10 @@ export function buildEvaluationPrompt(pkg) {
70
92
  const argStr = Object.entries(q.arguments)
71
93
  .map(([k, v]) => `${k}=${JSON.stringify(v)}`)
72
94
  .join(', ');
73
- return ` ${i + 1}. ${q.tool}(${argStr}) → ${q.resultRows ?? '?'} rows`;
95
+ const outcome = wasRecordedRejected(q)
96
+ ? REJECTED_CALL_RENDERING
97
+ : `${q.resultRows ?? '?'} rows`;
98
+ return ` ${i + 1}. ${q.tool}(${argStr}) → ${outcome}`;
74
99
  })
75
100
  .join('\n');
76
101
  const dataSources = pkg.dataSources
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@typedstandards/civic-typed-harness",
3
- "version": "0.4.0",
3
+ "version": "0.5.0",
4
4
  "description": "Civic-domain harness for Typed Standards evidence packages — the DOMAIN layer of the ADR-0021 format/domain line: civic PROV-O vocabulary and provenance builder, datHere envelope-policy derivations, and the capture machinery (trace builder, skill-metadata extraction, data-source population, adversarial rubric core). Derives; @typedstandards/produce-core assembles.",
5
5
  "license": "MIT",
6
6
  "author": "Nathan Storey",
@@ -26,7 +26,7 @@
26
26
  "scripts": {
27
27
  "build": "tsc -p tsconfig.json",
28
28
  "clean": "rm -rf dist tsconfig.tsbuildinfo",
29
- "typecheck": "tsc -p tsconfig.json --noEmit",
29
+ "typecheck": "tsc -p tsconfig.json --noEmit && tsc -p tsconfig.test.json",
30
30
  "lint": "eslint .",
31
31
  "test": "node --test --experimental-strip-types \"src/**/*.test.ts\"",
32
32
  "prepare": "npm run build",
@@ -37,6 +37,7 @@
37
37
  },
38
38
  "devDependencies": {
39
39
  "@typedstandards/verify-core": "^0.9.0",
40
+ "@types/node": "^22.20.2",
40
41
  "@typescript-eslint/parser": "^8",
41
42
  "eslint": "^9",
42
43
  "typescript": "^5.9.3"