@typedstandards/civic-typed-harness 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -3,6 +3,206 @@
3
3
  Factual record of what changed per published version. Section references are
4
4
  to the Typed Standards specification unless noted otherwise.
5
5
 
6
+ ## 0.4.0 — 2026-09-04
7
+
8
+ Minor: two optional input fields are added and no existing export is removed
9
+ or retyped. `fallbackPortal` and `ProvenanceInput.portal`, inert since 0.3.1,
10
+ are widened and deprecated rather than dropped — removing either is breaking
11
+ and waits for a major.
12
+
13
+ **What this release claims about golden bytes, and what it does not.** Every
14
+ golden byte is unchanged — both vocabulary eras, all eight
15
+ golden-reproduction cases, and `__fixtures__/website-golden.json`. That
16
+ sentence has appeared in every release note since 0.3.0, and it is narrower
17
+ than it reads: **`__fixtures__/reference-golden.json` carries no spans at
18
+ all** — `spanId` and `scopeSpans` each occur zero times across its eight
19
+ cases, whose traces are empty `resourceSpans` arrays or BlobRefs. So those
20
+ eight cases are green for anything the tool-span loop does, including the
21
+ rejection marker this release adds, and their green is not evidence about it.
22
+
23
+ `website-golden.json` is the only golden fixture that exercises span-derived
24
+ output: five tool-call activities and four data-response descriptions. It is
25
+ the fixture that proves this release's byte-stability claim, and the one the
26
+ conditional spread of `civic:failed` was measured against — driven red by
27
+ making that spread unconditional, which moves it. Read
28
+ [civic-ai-tools#199](https://github.com/npstorey/civic-ai-tools/issues/199)
29
+ before citing a green golden suite as evidence about the graph builder.
30
+
31
+ **The reader-facing source name is measured, not changed**
32
+ ([civic-ai-tools#194](https://github.com/npstorey/civic-ai-tools/issues/194),
33
+ Wave N10 P-H3). Tests only: no exported behaviour changes, and every golden
34
+ byte — both vocabulary eras, all eight golden-reproduction cases — is
35
+ unchanged.
36
+
37
+ - **The change #194 asks for moves frozen golden bytes.** A data response the
38
+ builder cannot describe by a portal is described by its source's registry
39
+ `agentTitle` (`Data response from Socrata MCP Server`) — the agent's title,
40
+ implementation language on a reader-facing surface. Reading the registry's
41
+ `displayName` instead — the reader-facing name, in the registry beside
42
+ `agentTitle` since the package's first release — is a one-expression change,
43
+ and it rewrites two lines inside `__fixtures__/website-golden.json`'s
44
+ captured graph: the data-commons and boston-opencontext data responses, both
45
+ on the agent-title branch. Three whole-graph byte-parity assertions read
46
+ those two lines. A golden fixture is never edited to match new behaviour, so
47
+ the change waits on a decision about whether those bytes move.
48
+ - **The vocabulary era cannot carry it.** `CivicVocabulary` binds exactly two
49
+ literals — the `civic:` namespace and the id-scheme prefix
50
+ (`makeCivicVocabulary(ns, urnPrefix)`) — and both era suites assert the two
51
+ eras differ by those and by nothing else. A description's wording is neither
52
+ of them and is not era-scoped: the same prefix is emitted under both eras, so
53
+ gating it on the era would make prior-era reproduction and settlement-era
54
+ emission disagree about a reader's wording rather than about an identifier.
55
+ - **Two instruments, both able to fail.** `provenance.test.ts` now pins the
56
+ agent node's `dcterms:title` to the registry `agentTitle` — those are signed
57
+ bytes in both golden fixtures and the one thing that does not move under
58
+ either decision — and names the two golden descriptions at stake, asserting
59
+ each reads by agent title today and *not* by display name. The sources they
60
+ drive all carry two distinct registry names, which is what makes the
61
+ assertions able to fail; a builder changed under either one turns it red.
62
+ - **What holds still either way.** The portal branch
63
+ (`Data response from data.cityofnewyork.us`) and the unknown-source fallback
64
+ (`Data response from euro stat`, the raw source id) move under neither
65
+ decision.
66
+
67
+ **The PROV-O activity for a rejected call says it was rejected**
68
+ ([civic-ai-tools#193](https://github.com/npstorey/civic-ai-tools/issues/193),
69
+ Wave N10 P-H2). Additive: no existing export is removed or retyped, and every
70
+ golden byte — both vocabulary eras, all eight golden-reproduction cases — is
71
+ unchanged.
72
+
73
+ - **Two new activity terms.** `CIVIC_TERM_FAILED` (`civic:failed`) and
74
+ `CIVIC_TERM_FAILURE_KIND` (`civic:failureKind`) are declared in
75
+ `src/format/vocabulary.ts` and exported from the package root. A `civic:`
76
+ property name is vocabulary as much as the namespace it hangs under, so the
77
+ capture-side builder imports them rather than spelling them; `purity.test.ts`
78
+ now lists both, which is what makes that a claim able to fail.
79
+ - **`buildProvenanceGraph` reads the span's failure.** A tool span the producer
80
+ ended with the boolean `error: true` yields a tool-call activity carrying
81
+ `civic:failed: true`, and `civic:failureKind` with the span's `error.kind`
82
+ verbatim when it carried one. Previously the builder read nine `tool.*` /
83
+ `mcp.*` attributes and never asked about the outcome, so a call the source
84
+ REFUSED and one that answered were the same node with the same description —
85
+ and, a rejected call having no response hash, the absent data-response entity
86
+ was the only trace of the rejection in the graph. `error.kind` is the one
87
+ attribute name across this package and the reference producer.
88
+ - **`error` is the assertion, `error.kind` only a label on one.** A span
89
+ carrying a kind and no assertion is not a rejection and yields neither key —
90
+ the posture `ToolCallSummary.failed` / `failureKind` already takes.
91
+ - **A new boolean attribute reader.** The module's string attribute reader
92
+ returns `stringValue ?? intValue` and cannot see `boolValue` at all, so it
93
+ returned `undefined` for a span that really did record a rejection.
94
+ `getBoolAttr` is a separate, strictly-typed reader rather than a widening of
95
+ the string one: the nine attributes that reader serves are strings by
96
+ contract, and a truthiness test would read the string `"false"` as an
97
+ assertion of failure. Only the boolean `true` marks an activity.
98
+ - **The description states no cause.** `dcterms:description` is byte-unchanged
99
+ — it states what the call WAS, and the marker states how it ended. The
100
+ classified kind is the only cause the graph will ever carry: the reference
101
+ producer stopped writing a rejection's raw text onto the span in this same
102
+ wave, and the builder does not read it back in one layer up.
103
+ - **Byte consequence.** Both terms are spread conditionally and appended after
104
+ every key the activity already carried, exactly as `civic:durationMs` is. A
105
+ span that recorded no rejection yields the 0.3.1 key list in the 0.3.1 order,
106
+ and `civic:failed: false` is never emitted — a producer that stated "not
107
+ failed" and one that stated nothing must read the same, which is what makes a
108
+ marker that IS present mean something. A rejected span carries no
109
+ `tool.duration_ms` from the reference producer today
110
+ ([civic-ai-tools-website#413](https://github.com/npstorey/civic-ai-tools-website/issues/413)),
111
+ so `civic:durationMs` does not fire beside the two new keys.
112
+
113
+ **A call the record states as failed asserts no access**
114
+ ([civic-ai-tools#192](https://github.com/npstorey/civic-ai-tools/issues/192),
115
+ Wave N10 P-H1). Additive: no existing export is removed, no existing caller
116
+ changes, and every golden byte — both vocabulary eras, all eight
117
+ golden-reproduction cases — is unchanged.
118
+
119
+ - **`ToolCallSummary` can see the rejection.** Two new OPTIONAL fields:
120
+ `failed?: boolean`, the producer's assertion that the source rejected this
121
+ call, and `failureKind?: string`, the producer's own open label for why.
122
+ The harness never interprets the label, and `buildDataSources` never reads
123
+ it — `failed` is the assertion, `failureKind` only a label on one. A
124
+ summary carrying neither is exactly the 0.3.1 shape, and absence means
125
+ "not recorded as failed", never "succeeded".
126
+ - **`buildDataSources` mints nothing from a rejected call.** A call whose
127
+ summary carries `failed: true` contributes no dataset-keyed entry for the
128
+ dataset it never read, and marks no aggregate source accessed. Previously
129
+ the population could not see the failure at all, so a rejected call minted
130
+ its dataset's entry and, on an aggregate source, marked that source
131
+ accessed at a timestamp — inside the bytes a publisher signs. The rejected
132
+ call keeps its POSITION in the walk (calls pair to spans by index), and it
133
+ remains on the PROV-O graph's tool-call activities and in the caller's own
134
+ `queries[]`. A source with any non-rejected call is still accessed; a
135
+ dataset a successful call also read still gets its entry.
136
+ - **The two inert inputs are marked deprecated, not removed.**
137
+ `buildDataSources`'s `fallbackPortal` — accepted and not consulted since
138
+ 0.3.1 — now also accepts `undefined`, so a caller that has stopped
139
+ consulting it can stop supplying a value; it stays third of five positional
140
+ parameters, because dropping a positional parameter is breaking.
141
+ `ProvenanceInput.portal` becomes optional and carries `@deprecated`;
142
+ removing it outright would make the reference app's object literal an
143
+ excess-property error. Both removals wait for a major.
144
+ - **Byte consequence.** A package built from a record that states no failure
145
+ is byte-identical to one built by 0.3.1 — the walk reaches the new branch
146
+ only when a summary carries `failed: true`. A package built from a record
147
+ that DOES state a failure loses the `dataSources` entries that failure
148
+ never earned; that is the defect being fixed, and the wave re-emits
149
+ nothing.
150
+ - **Type-level gate.** `src/capture/data-sources.assert.ts` pins the shape at
151
+ compile time. `tsconfig.json` excludes `src/**/*.test.ts`, so the suite
152
+ type-checks nothing — a test can drive a field the type does not have and
153
+ `npm run typecheck` stays green.
154
+
155
+ ## 0.3.1 — 2026-09-02
156
+
157
+ **The graph states what the span carried, and states absence as absence**
158
+ ([civic-ai-tools-website#384](https://github.com/npstorey/civic-ai-tools-website/issues/384),
159
+ Wave N9 P-H1). Patch, not breaking: no existing export is removed or
160
+ retyped, and every golden byte — both vocabulary eras, all eight
161
+ golden-reproduction cases — is unchanged.
162
+
163
+ - **`buildProvenanceGraph` no longer invents a tool name.** An
164
+ `mcp_tool_call` span with no `tool.name` yields a query entity with no
165
+ `civic:toolName` key (omitted, not placeholdered) and a tool-call activity
166
+ described as `MCP tool call (<operation type>)`. Previously the builder
167
+ substituted `get_data`.
168
+ - **`buildProvenanceGraph` no longer attributes a data response to the run's
169
+ portal.** `civic:portalDomain` and `civic:datasetUrl` are emitted only when
170
+ the span carried both `tool.portal_domain` and `tool.dataset_id`. A
171
+ dataset-keyed span with no portal is described by its source agent's
172
+ registry title (`Data response from Socrata MCP Server`), the form
173
+ aggregate and unknown sources already took; a span with a dataset id and
174
+ no portal states `civic:datasetId` and mints no URL. Previously the builder
175
+ substituted `ProvenanceInput.portal` — the run's selected portal — which
176
+ attributed every `search` and `fetch` response to a portal the call never
177
+ addressed (the Socrata server's `search` and `fetch` take no portal and
178
+ answer from the portal that server is configured for). The graph does not
179
+ parse tool arguments: a portal embedded in a `fetch` id is not a portal the
180
+ span carried.
181
+ - **`buildDataSources` no longer mints an entry on `fallbackPortal`.** A
182
+ dataset-keyed call whose arguments carry `dataset_id` but no `portal`
183
+ contributes no `dataSources` entry — `DataSourceEntry.portalUrl` is a
184
+ required string in produce-core, so an entry with no portal is not a shape
185
+ this package can emit — and the call remains on the graph's tool-call
186
+ activities. Calls that carry a portal (every `get_data` call from the
187
+ reference producer, whose loop injects the run portal before the record is
188
+ built) are unchanged.
189
+ - **Unchanged signatures, two inputs now unused.** `ProvenanceInput.portal`
190
+ and the `fallbackPortal` parameter of `buildDataSources` keep their exact
191
+ types and positions, are accepted, and are not consulted; both say so at
192
+ the declaration.
193
+ - **Byte consequence.** A package produced by this version from a trace
194
+ whose tool spans all carry `tool.name`, and carry `tool.portal_domain`
195
+ wherever they carry a response hash, is byte-identical to one produced by
196
+ 0.3.0. Where a span carried less, the 0.3.0 output asserted a value the
197
+ span did not, and this version's output differs from it by exactly that
198
+ assertion. Packages already signed under 0.3.0 are untouched and remain
199
+ verifiable exactly as published.
200
+ - **Dependency range widened, nothing resolved differently.**
201
+ `@typedstandards/produce-core` is accepted at `^0.3.0 || ^0.4.0` so a
202
+ consumer that takes produce-core 0.4.0 (an additive minor) resolves one
203
+ copy rather than nesting a second under this package. The lockfile here
204
+ still resolves 0.3.0.
205
+
6
206
  ## 0.3.0 — 2026-08-20
7
207
 
8
208
  **Settlement-era civic vocabulary + produce-core 0.3.0**
@@ -0,0 +1 @@
1
+ export {};
@@ -0,0 +1,37 @@
1
+ // Type-level acceptance gate for the 0.4.0 additive change (civic-ai-tools#192):
2
+ // `ToolCallSummary` can see a rejection the producer recorded, and BOTH new
3
+ // fields are OPTIONAL — a caller that records no outcome passes neither and
4
+ // compiles exactly as it did at 0.3.1.
5
+ //
6
+ // Why a file rather than a test: `tsconfig.json` EXCLUDES `src/**/*.test.ts`,
7
+ // so the suite type-checks nothing. A test can drive a field the type does not
8
+ // have and `npm run typecheck` stays green — which is exactly what happened
9
+ // while this phase's red instrument was the only thing pinning the shape. The
10
+ // type is pinned here instead, mirroring `src/required-config.assert.ts`.
11
+ //
12
+ // Compile-time only: nothing here is exported, and the guard is never invoked.
13
+ import { buildDataSources } from "./data-sources.js";
14
+ function pinsToolCallSummaryShape() {
15
+ // The 0.3.1 shape is still a ToolCallSummary — both new fields are optional.
16
+ const unrecorded = { name: 'get_data', args: {} };
17
+ // A recorded outcome is expressible, with or without a producer label.
18
+ const rejected = { name: 'get_data', args: {}, failed: true };
19
+ const succeeded = { name: 'get_data', args: {}, failed: false };
20
+ const labelled = {
21
+ name: 'get_data',
22
+ args: {},
23
+ failed: true,
24
+ failureKind: 'unavailable',
25
+ };
26
+ // @ts-expect-error #192 — `failed` is the producer's boolean assertion, never a string
27
+ const wrongFailedType = { name: 'get_data', args: {}, failed: 'yes' };
28
+ // @ts-expect-error #192 — `failureKind` is an open producer label (string), never a flag
29
+ const wrongKindType = { name: 'get_data', args: {}, failureKind: true };
30
+ // `fallbackPortal` stays positional third of five for callers that still
31
+ // pass it, and since 0.4.0 accepts `undefined` from callers that do not.
32
+ buildDataSources([unrecorded, rejected, succeeded, labelled], trace, 'portal.example', 'now');
33
+ buildDataSources([unrecorded], trace, undefined, 'now');
34
+ void wrongFailedType;
35
+ void wrongKindType;
36
+ }
37
+ void pinsToolCallSummaryShape;
@@ -6,6 +6,22 @@ export type { DataSourceEntry };
6
6
  export interface ToolCallSummary {
7
7
  name: string;
8
8
  args: Record<string, unknown>;
9
+ /** Did the producer record this call as REJECTED by the source? A call
10
+ * recorded as failed asserts no access: it contributes no dataset-keyed
11
+ * entry and marks no aggregate source accessed (see `buildDataSources`).
12
+ *
13
+ * Optional, and absent is absent: a producer that records no outcome
14
+ * passes neither this nor `failureKind`, and gets exactly the entries it
15
+ * got before the fields existed. Absence means "not recorded as failed",
16
+ * never "succeeded". Added 0.4.0. */
17
+ failed?: boolean;
18
+ /** The producer's own label for why the call was rejected, carried so a
19
+ * caller can hand its record through unchanged. An open string with no
20
+ * normative vocabulary: the harness never interprets it, and this module
21
+ * never reads it. `failed` is the assertion and `failureKind` only a label
22
+ * on one — a summary carrying a kind but no `failed` is not treated as a
23
+ * rejection. Added 0.4.0. */
24
+ failureKind?: string;
9
25
  }
10
26
  interface TraceSpan {
11
27
  name: string;
@@ -47,12 +63,29 @@ export declare function resolveToolSource(toolCall: ToolCallSummary, span: Trace
47
63
  * Build the per-source evidence-package `dataSources` array.
48
64
  *
49
65
  * Dataset-keyed sources (Socrata) contribute one entry per unique
50
- * `dataset_id` observed across tool calls. Aggregate sources (Data Commons,
66
+ * `dataset_id` observed across tool calls that also carried a `portal`
67
+ * argument; a dataset-keyed call that carried no portal contributes NO
68
+ * entry (the loop body says why). Aggregate sources (Data Commons,
51
69
  * Boston OpenContext — registry entries carrying `aggregatePortalUrl`)
52
70
  * contribute a single entry when any of their tool calls was made. Unknown
53
71
  * source ids contribute no entry. Each entry is tagged with `sourceId` so
54
72
  * downstream consumers can distinguish provenance. Emission order: the
55
73
  * dataset-keyed entries (first-seen order), then aggregate sources in
56
74
  * registry insertion order — matching the reference implementation.
75
+ *
76
+ * A call the producer recorded as FAILED (`ToolCallSummary.failed`) asserts
77
+ * no access and contributes nothing on either branch: no dataset-keyed entry
78
+ * for a dataset it never read, and no accessed-marking of its aggregate
79
+ * source. It keeps its POSITION in the walk, because calls are paired to
80
+ * spans by index. The call is still on the PROV-O graph's tool-call
81
+ * activities and in the caller's own `queries[]` — what it is not is an
82
+ * assertion, inside signed bytes, that a source was reached at a timestamp.
83
+ *
84
+ * @param fallbackPortal DEPRECATED, and inert since 0.3.1: an entry states
85
+ * the portal the call carried, never the run's. It stays third of five
86
+ * positional parameters so existing callers keep compiling, and since 0.4.0
87
+ * it also accepts `undefined`, which is what a caller that has stopped
88
+ * consulting it should pass. Dropping it is a breaking change and waits for
89
+ * a major.
57
90
  */
58
- export declare function buildDataSources(toolCalls: ToolCallSummary[], trace: Record<string, unknown>, fallbackPortal: string, now: string, options?: DataSourceOptions): DataSourceEntry[];
91
+ export declare function buildDataSources(toolCalls: ToolCallSummary[], trace: Record<string, unknown>, fallbackPortal: string | undefined, now: string, options?: DataSourceOptions): DataSourceEntry[];
@@ -51,13 +51,30 @@ export function resolveToolSource(toolCall, span, resolver = civicToolSourceReso
51
51
  * Build the per-source evidence-package `dataSources` array.
52
52
  *
53
53
  * Dataset-keyed sources (Socrata) contribute one entry per unique
54
- * `dataset_id` observed across tool calls. Aggregate sources (Data Commons,
54
+ * `dataset_id` observed across tool calls that also carried a `portal`
55
+ * argument; a dataset-keyed call that carried no portal contributes NO
56
+ * entry (the loop body says why). Aggregate sources (Data Commons,
55
57
  * Boston OpenContext — registry entries carrying `aggregatePortalUrl`)
56
58
  * contribute a single entry when any of their tool calls was made. Unknown
57
59
  * source ids contribute no entry. Each entry is tagged with `sourceId` so
58
60
  * downstream consumers can distinguish provenance. Emission order: the
59
61
  * dataset-keyed entries (first-seen order), then aggregate sources in
60
62
  * registry insertion order — matching the reference implementation.
63
+ *
64
+ * A call the producer recorded as FAILED (`ToolCallSummary.failed`) asserts
65
+ * no access and contributes nothing on either branch: no dataset-keyed entry
66
+ * for a dataset it never read, and no accessed-marking of its aggregate
67
+ * source. It keeps its POSITION in the walk, because calls are paired to
68
+ * spans by index. The call is still on the PROV-O graph's tool-call
69
+ * activities and in the caller's own `queries[]` — what it is not is an
70
+ * assertion, inside signed bytes, that a source was reached at a timestamp.
71
+ *
72
+ * @param fallbackPortal DEPRECATED, and inert since 0.3.1: an entry states
73
+ * the portal the call carried, never the run's. It stays third of five
74
+ * positional parameters so existing callers keep compiling, and since 0.4.0
75
+ * it also accepts `undefined`, which is what a caller that has stopped
76
+ * consulting it should pass. Dropping it is a breaking change and waits for
77
+ * a major.
61
78
  */
62
79
  export function buildDataSources(toolCalls, trace, fallbackPortal, now, options = {}) {
63
80
  const resolver = options.resolver ?? civicToolSourceResolver;
@@ -68,11 +85,29 @@ export function buildDataSources(toolCalls, trace, fallbackPortal, now, options
68
85
  const aggregateAccessed = new Set();
69
86
  for (let i = 0; i < toolCalls.length; i++) {
70
87
  const tc = toolCalls[i];
88
+ // A call the producer recorded as rejected reached no data, so it mints
89
+ // no entry on either branch below. The walk keeps its index rather than
90
+ // filtering the list: `resolveToolSource` pairs a call to `toolSpans[i]`,
91
+ // and a filtered list would shift every later call onto the wrong span.
92
+ if (tc.failed)
93
+ continue;
71
94
  const source = resolveToolSource(tc, toolSpans[i], resolver, fallbackSourceId);
72
95
  if (isDatasetKeyedSource(source, registry)) {
73
96
  const datasetId = tc.args.dataset_id;
74
- const portal = tc.args.portal || fallbackPortal;
75
- if (datasetId) {
97
+ const portal = tc.args.portal;
98
+ // An entry is minted only from what the call carried: a dataset id AND
99
+ // a portal. A dataset-keyed call with a dataset id and no portal
100
+ // contributes no entry. `DataSourceEntry.portalUrl` is a required
101
+ // string (produce-core), so "an entry with no portal" is not a shape
102
+ // this package can emit; and substituting the run's portal
103
+ // (`fallbackPortal`, before 0.3.1) attributed the call to a portal it
104
+ // never addressed. Omission is the honest shape — the call is still on
105
+ // the PROV-O graph's tool-call activities, stated without a portal.
106
+ // The branch is latent for the reference producer, whose loop injects
107
+ // the run portal into `get_data` arguments before the record is built
108
+ // (run-tool-loop.ts:799 at the time of writing); any caller whose
109
+ // summary carries `dataset_id` without `portal` reaches it.
110
+ if (datasetId && portal) {
76
111
  let byDataset = datasetKeyed.get(source);
77
112
  if (!byDataset) {
78
113
  byDataset = new Map();
@@ -15,7 +15,17 @@ export interface ProvenanceInput {
15
15
  * itself is exactly this value by construction. */
16
16
  outputHash?: string;
17
17
  model: string;
18
- portal: string;
18
+ /** The run's selected portal. Accepted and NOT consulted by the graph
19
+ * builder since 0.3.1: the graph states the portal a tool span carried
20
+ * (`tool.portal_domain`) and states absence as absence — it never
21
+ * substitutes the run's portal for one the call did not address.
22
+ *
23
+ * @deprecated Inert since 0.3.1 and optional since 0.4.0 — a caller that
24
+ * has stopped consulting it should stop passing it. The field stays in
25
+ * the type so callers that do pass it (the reference app does, as an
26
+ * object literal) keep compiling; removing it outright would make that
27
+ * literal an excess-property error, so removal waits for a major. */
28
+ portal?: string;
19
29
  }
20
30
  /** Instance configuration for the graph build (config-not-constants). */
21
31
  export interface ProvenanceConfig {
@@ -45,7 +55,9 @@ export interface ProvenanceConfig {
45
55
  vocabulary?: CivicVocabulary;
46
56
  }
47
57
  /** The civicaitools.org reference deployment's values. Passed explicitly by
48
- * the reference app — never applied as a default. */
58
+ * the reference app — never applied as a default, and never spread into
59
+ * another instance's config, which would assert infrastructure that
60
+ * instance doesn't run. */
49
61
  export declare const CIVICAITOOLS_PROVENANCE_CONFIG: ProvenanceConfig;
50
62
  /**
51
63
  * Build a W3C PROV-O JSON-LD graph from an OTel trace and package metadata.
@@ -17,10 +17,12 @@
17
17
  // order is preserved throughout — the legacy hash chain's byte contract).
18
18
  import { makeProvGraph, makeEntityNode, makeActivityNode, makeAgentNode, provRef, provUsed, provWasGeneratedBy, provWasDerivedFrom, provWasAssociatedWith, xsdDateTime, } from '@typedstandards/produce-core';
19
19
  import { hash } from "./trace.js";
20
- import { CIVIC_VOCABULARY, CIVICAITOOLS_PLATFORM_AGENT, } from "../format/vocabulary.js";
20
+ import { CIVIC_VOCABULARY, CIVICAITOOLS_PLATFORM_AGENT, CIVIC_TERM_FAILED, CIVIC_TERM_FAILURE_KIND, } from "../format/vocabulary.js";
21
21
  import { CIVIC_SOURCE_REGISTRY, FALLBACK_SOURCE_ID, isDatasetKeyedSource, } from "../format/sources.js";
22
22
  /** The civicaitools.org reference deployment's values. Passed explicitly by
23
- * the reference app — never applied as a default. */
23
+ * the reference app — never applied as a default, and never spread into
24
+ * another instance's config, which would assert infrastructure that
25
+ * instance doesn't run. */
24
26
  export const CIVICAITOOLS_PROVENANCE_CONFIG = {
25
27
  platformAgent: CIVICAITOOLS_PLATFORM_AGENT,
26
28
  sourceRegistry: CIVIC_SOURCE_REGISTRY,
@@ -31,6 +33,30 @@ function getAttr(attrs, key) {
31
33
  const attr = attrs.find(a => a.key === key);
32
34
  return attr?.value?.stringValue ?? attr?.value?.intValue ?? undefined;
33
35
  }
36
+ /**
37
+ * Read an attribute the producer wrote as a BOOLEAN.
38
+ *
39
+ * `getAttr` above returns `stringValue ?? intValue` and cannot see the third
40
+ * OTel value shape at all — `TraceBuilder` encodes a boolean as
41
+ * `{ boolValue }` (trace.ts), so `getAttr(attrs, 'error')` returns `undefined`
42
+ * for a span the producer really did end with `error: true`. This is a
43
+ * separate reader rather than a widening of `getAttr` on purpose: the nine
44
+ * attributes `getAttr` serves are strings by contract and feed hashes,
45
+ * descriptions and `Number()`, and teaching it a fourth return shape would
46
+ * change what all nine yield for value shapes nobody has measured.
47
+ *
48
+ * The comparison is strict, and only `true` counts. A truthiness test over
49
+ * `stringValue` would read the string `"false"` as an assertion of failure —
50
+ * marking a call that ANSWERED as refused, inside bytes a publisher signs.
51
+ * The cost of the strictness is that a producer encoding this attribute as a
52
+ * string gets no marker; the reference producer does not, and under-reading a
53
+ * foreign encoding leaves the graph silent, where over-reading one would make
54
+ * it lie.
55
+ */
56
+ function getBoolAttr(attrs, key) {
57
+ const attr = attrs.find(a => a.key === key);
58
+ return typeof attr?.value?.boolValue === 'boolean' ? attr.value.boolValue : undefined;
59
+ }
34
60
  function nanoToIso(nano) {
35
61
  const ms = Math.floor(Number(nano) / 1_000_000);
36
62
  return new Date(ms).toISOString();
@@ -48,7 +74,8 @@ function nanoToIso(nano) {
48
74
  export function buildProvenanceGraph(trace, input, config) {
49
75
  const otel = trace;
50
76
  const spans = otel?.resourceSpans?.[0]?.scopeSpans?.[0]?.spans || [];
51
- const { packageId, promptHash, promptText, outputText, model, portal } = input;
77
+ // `input.portal` is deliberately not read — see `ProvenanceInput.portal`.
78
+ const { packageId, promptHash, promptText, outputText, model } = input;
52
79
  const outputHash = input.outputHash ?? hash(outputText ?? '');
53
80
  const registry = config.sourceRegistry;
54
81
  const fallbackSourceId = config.fallbackSourceId ?? FALLBACK_SOURCE_ID;
@@ -167,17 +194,34 @@ export function buildProvenanceGraph(trace, input, config) {
167
194
  ...(completionTokens ? { 'civic:completionTokens': Number(completionTokens) } : {}),
168
195
  }));
169
196
  }
170
- // MCP tool call spans
197
+ // MCP tool call spans.
198
+ //
199
+ // Every node derived from a span states what the span carried, and states
200
+ // absence as absence: a span with no `tool.name` yields nodes that name no
201
+ // tool, and a span with no `tool.portal_domain` yields a data response
202
+ // attributed to no portal. Nothing here substitutes the run's selected
203
+ // portal (`input.portal`, accepted and unused since 0.3.1) or a default
204
+ // tool name for a value the producer did not write. The reference
205
+ // producer's loop always writes `tool.name`, and writes
206
+ // `tool.portal_domain` only when the call's arguments carried a portal —
207
+ // which its portal injection does for `get_data` and for no other tool.
208
+ // The Socrata server's `search` (one argument, `query`) and `fetch` (one
209
+ // argument, `id`) address the portal that server is configured for, which
210
+ // the producer does not know. A `fetch` id may embed a portal, but that
211
+ // grammar belongs to the server: the graph does not parse arguments, so an
212
+ // id is never a portal the span carried.
171
213
  const dataResponseUrns = [];
172
214
  for (const span of toolSpans) {
173
215
  const toolCallUrn = vocab.urn(packageId, 'tool-call', span.spanId);
174
216
  const argsStr = getAttr(span.attributes, 'tool.arguments') || '{}';
175
217
  const queryHash = hash(argsStr);
176
218
  const responseHash = getAttr(span.attributes, 'tool.response_hash');
177
- const toolName = getAttr(span.attributes, 'tool.name') || 'get_data';
219
+ // Absent when the span carried none — never defaulted.
220
+ const toolName = getAttr(span.attributes, 'tool.name');
178
221
  const opType = getAttr(span.attributes, 'tool.operation_type') || 'unknown';
179
222
  const datasetId = getAttr(span.attributes, 'tool.dataset_id');
180
- const portalDomain = getAttr(span.attributes, 'tool.portal_domain') || portal;
223
+ // Absent when the span carried none — never the run's portal.
224
+ const portalDomain = getAttr(span.attributes, 'tool.portal_domain');
181
225
  const toolSource = getAttr(span.attributes, 'mcp.source') || fallbackSourceId;
182
226
  const toolAgentUrn = agentUrnForSource(toolSource);
183
227
  const toolSourceDatasetKeyed = isDatasetKeyedSource(toolSource, registry);
@@ -190,7 +234,8 @@ export function buildProvenanceGraph(trace, input, config) {
190
234
  const queryUrn = vocab.urn(packageId, 'query', queryHash);
191
235
  graph.push(makeEntityNode(queryUrn, {
192
236
  'civic:contentHash': `sha256:${queryHash}`,
193
- 'civic:toolName': toolName,
237
+ // Omitted — not placeholdered — when the span named no tool.
238
+ ...(toolName ? { 'civic:toolName': toolName } : {}),
194
239
  'civic:operationType': opType,
195
240
  'dcterms:description': `MCP tool arguments (${opType})`,
196
241
  ...(precedingInference
@@ -201,10 +246,20 @@ export function buildProvenanceGraph(trace, input, config) {
201
246
  if (responseHash) {
202
247
  const dataUrn = vocab.urn(packageId, 'data', responseHash);
203
248
  dataResponseUrns.push(dataUrn);
204
- // Description varies by source — dataset-keyed sources (Socrata) have
205
- // a portal domain; aggregate/unknown sources are described by their
206
- // agent title.
207
- const description = toolSourceDatasetKeyed
249
+ // Description: the portal the span carried, when it carried one;
250
+ // otherwise the agent that answered the call, by its registry title —
251
+ // the form aggregate and unknown sources already take. A dataset-keyed
252
+ // source whose span carried a dataset id but no portal takes the
253
+ // agent-title form too, with the dataset id stated below as
254
+ // `civic:datasetId` and no URL minted (a dataset URL needs a host the
255
+ // span did not carry). That branch is latent by construction for the
256
+ // reference producer — its loop injects the run portal into `get_data`
257
+ // arguments before the span opens (run-tool-loop.ts:799 at the time of
258
+ // writing), and `get_data` is the only tool whose arguments carry a
259
+ // dataset id — but any producer that writes `tool.dataset_id` without
260
+ // `tool.portal_domain` reaches it, so it is stated honestly rather
261
+ // than left dead.
262
+ const description = toolSourceDatasetKeyed && portalDomain
208
263
  ? `Data response from ${portalDomain}`
209
264
  : `Data response from ${sourceAgentMap[toolSource]?.title || toolSource}`;
210
265
  const responseRows = getAttr(span.attributes, 'tool.response_rows');
@@ -214,12 +269,18 @@ export function buildProvenanceGraph(trace, input, config) {
214
269
  'civic:sourceId': toolSource,
215
270
  ...provWasGeneratedBy(toolCallUrn),
216
271
  // Croissant 1.1 placeholder — only meaningful for dataset-keyed
217
- // sources today.
272
+ // sources today. The dataset id is stated whenever the span carried
273
+ // one; the portal and the dataset URL only when the span carried
274
+ // the portal as well. Key order is the byte contract.
218
275
  ...(toolSourceDatasetKeyed && datasetId
219
276
  ? {
220
277
  'civic:datasetId': datasetId,
221
- 'civic:portalDomain': portalDomain,
222
- 'civic:datasetUrl': `https://${portalDomain}/d/${datasetId}`,
278
+ ...(portalDomain
279
+ ? {
280
+ 'civic:portalDomain': portalDomain,
281
+ 'civic:datasetUrl': `https://${portalDomain}/d/${datasetId}`,
282
+ }
283
+ : {}),
223
284
  'civic:croissantMetadataUrl': null, // hook for future Croissant integration
224
285
  }
225
286
  : {}),
@@ -230,8 +291,32 @@ export function buildProvenanceGraph(trace, input, config) {
230
291
  // the call. In multi-source analyses each call may target a different
231
292
  // agent (e.g. socrata for `get_data`, data-commons for `get_observations`).
232
293
  const durationMs = getAttr(span.attributes, 'tool.duration_ms');
294
+ // Did the SOURCE REFUSE this call? Before 0.4.0 the graph never asked, so
295
+ // a refused call and one that answered were the same node with the same
296
+ // description — the absence of a data-response entity was the only trace
297
+ // of the rejection, and absence stated as nothing is not absence stated.
298
+ //
299
+ // `error` is the ASSERTION and `error.kind` only a LABEL on one. A span
300
+ // carrying a kind and no assertion is not a rejection, so the kind is read
301
+ // and stated only inside the failure branch — the posture
302
+ // `ToolCallSummary.failed`/`failureKind` already takes on the other input.
303
+ // `error.kind` is the one attribute name across this package and the
304
+ // reference producer (Wave N10 D5); neither side invents a second.
305
+ //
306
+ // The kind is the ONLY cause this graph will ever state. The producer
307
+ // stopped writing a rejection's raw text onto the span in the same wave —
308
+ // that text is authored by the source, can name a host, a port or a stack
309
+ // frame, and the trace travels inline inside the bytes an instance signs.
310
+ // Reading it here would put it back one layer up; the classified vocabulary
311
+ // widens instead. Nor does the description change: it states what the call
312
+ // WAS, and the marker below states how it ended.
313
+ const failed = getBoolAttr(span.attributes, 'error');
314
+ const failureKind = failed === true ? getAttr(span.attributes, 'error.kind') : undefined;
233
315
  graph.push(makeActivityNode(toolCallUrn, {
234
- 'dcterms:description': `MCP tool call: ${toolName} (${opType})`,
316
+ // Names the tool only when the span did.
317
+ 'dcterms:description': toolName
318
+ ? `MCP tool call: ${toolName} (${opType})`
319
+ : `MCP tool call (${opType})`,
235
320
  'civic:sourceId': toolSource,
236
321
  ...provUsed([queryUrn]),
237
322
  ...provWasAssociatedWith(toolAgentUrn),
@@ -242,6 +327,17 @@ export function buildProvenanceGraph(trace, input, config) {
242
327
  ? { 'prov:endedAtTime': xsdDateTime(nanoToIso(span.endTimeUnixNano)) }
243
328
  : {}),
244
329
  ...(durationMs ? { 'civic:durationMs': Number(durationMs) } : {}),
330
+ // Appended LAST and spread conditionally, exactly as `civic:durationMs`
331
+ // above is: a span that recorded no rejection yields the key list it
332
+ // yielded at 0.3.1, in the same order, and property insertion order is
333
+ // the legacy chain's byte contract. `false` is never emitted — a
334
+ // producer that stated "not failed" and one that stated nothing must
335
+ // read the same, which is what makes a marker that IS present mean
336
+ // something. A rejected span carries no `tool.duration_ms` from the
337
+ // reference producer today (civic-ai-tools-website#413), so
338
+ // `civic:durationMs` simply does not fire beside these two.
339
+ ...(failed === true ? { [CIVIC_TERM_FAILED]: true } : {}),
340
+ ...(failureKind ? { [CIVIC_TERM_FAILURE_KIND]: failureKind } : {}),
245
341
  }));
246
342
  }
247
343
  // --- Final output relationships ---
@@ -11,6 +11,30 @@ export declare const PRIOR_ERA_CIVIC_NS = "https://civicaitools.org/ns/evidence/
11
11
  /** The prior-era id-scheme root, frozen inside records signed before the
12
12
  * 2026-08-19 settlement. Never emitted for new packages. */
13
13
  export declare const PRIOR_ERA_CIVIC_URN_PREFIX = "urn:civic-evidence";
14
+ /**
15
+ * `civic:failed` — a tool-call activity whose call the SOURCE REFUSED.
16
+ *
17
+ * Emitted with the boolean `true` and only when the producer stated the
18
+ * rejection; never emitted as `false`. A literal `false` would assert an
19
+ * outcome, and "recorded as not-failed" must stay indistinguishable from
20
+ * "nothing recorded" — the same absent-is-absent posture `ToolCallSummary`
21
+ * takes for `failed`. A reader that finds no key learns that the record
22
+ * states nothing, which is the truth about every package minted before 0.4.0.
23
+ */
24
+ export declare const CIVIC_TERM_FAILED = "civic:failed";
25
+ /**
26
+ * `civic:failureKind` — the producer's own classified label for WHY a call
27
+ * carrying {@link CIVIC_TERM_FAILED} was refused.
28
+ *
29
+ * An open string with no normative vocabulary here: the harness states the
30
+ * label verbatim and never interprets or re-derives it. It is a label on an
31
+ * assertion, not the assertion — a record that carries a kind and no failure
32
+ * is not a rejection, and the term is emitted only alongside
33
+ * {@link CIVIC_TERM_FAILED}. The reference producer's four values are
34
+ * `timeout`, `unavailable`, `not_configured` and `unknown`; a producer with a
35
+ * wider vocabulary widens this field rather than putting prose in it.
36
+ */
37
+ export declare const CIVIC_TERM_FAILURE_KIND = "civic:failureKind";
14
38
  /**
15
39
  * One era of the civic vocabulary: the two literals plus the emitters bound
16
40
  * to them. The vocabulary is not deployment configuration (a deployment does
@@ -32,6 +32,45 @@ export const PRIOR_ERA_CIVIC_NS = 'https://civicaitools.org/ns/evidence/';
32
32
  /** The prior-era id-scheme root, frozen inside records signed before the
33
33
  * 2026-08-19 settlement. Never emitted for new packages. */
34
34
  export const PRIOR_ERA_CIVIC_URN_PREFIX = 'urn:civic-evidence';
35
+ // --- Term names ---
36
+ //
37
+ // A `civic:` property name is vocabulary as much as the namespace it hangs
38
+ // under: it is the word a reader of a signed graph interprets, so it is
39
+ // declared here and imported by the builder that emits it, never spelled as a
40
+ // literal inside capture/ (`purity.test.ts` enforces that for the terms below).
41
+ // Both terms are era-independent — the settlement moved the NAMESPACE, not the
42
+ // property names — so they carry no era qualifier and are minted for the first
43
+ // time after it.
44
+ //
45
+ // The settlement-era property names that predate Wave N10 (`civic:sourceId`,
46
+ // `civic:durationMs`, `civic:datasetId` and their siblings) are still inline
47
+ // literals in the capture-side builder. That is a real inconsistency, recorded
48
+ // rather than fixed in passing: moving them is a byte-sensitive change across
49
+ // every golden fixture and belongs to a phase of its own.
50
+ /**
51
+ * `civic:failed` — a tool-call activity whose call the SOURCE REFUSED.
52
+ *
53
+ * Emitted with the boolean `true` and only when the producer stated the
54
+ * rejection; never emitted as `false`. A literal `false` would assert an
55
+ * outcome, and "recorded as not-failed" must stay indistinguishable from
56
+ * "nothing recorded" — the same absent-is-absent posture `ToolCallSummary`
57
+ * takes for `failed`. A reader that finds no key learns that the record
58
+ * states nothing, which is the truth about every package minted before 0.4.0.
59
+ */
60
+ export const CIVIC_TERM_FAILED = 'civic:failed';
61
+ /**
62
+ * `civic:failureKind` — the producer's own classified label for WHY a call
63
+ * carrying {@link CIVIC_TERM_FAILED} was refused.
64
+ *
65
+ * An open string with no normative vocabulary here: the harness states the
66
+ * label verbatim and never interprets or re-derives it. It is a label on an
67
+ * assertion, not the assertion — a record that carries a kind and no failure
68
+ * is not a rejection, and the term is emitted only alongside
69
+ * {@link CIVIC_TERM_FAILED}. The reference producer's four values are
70
+ * `timeout`, `unavailable`, `not_configured` and `unknown`; a producer with a
71
+ * wider vocabulary widens this field rather than putting prose in it.
72
+ */
73
+ export const CIVIC_TERM_FAILURE_KIND = 'civic:failureKind';
35
74
  /** Bind the emitters to one era's two literals. */
36
75
  export function makeCivicVocabulary(ns, urnPrefix) {
37
76
  return {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@typedstandards/civic-typed-harness",
3
- "version": "0.3.0",
3
+ "version": "0.4.0",
4
4
  "description": "Civic-domain harness for Typed Standards evidence packages — the DOMAIN layer of the ADR-0021 format/domain line: civic PROV-O vocabulary and provenance builder, datHere envelope-policy derivations, and the capture machinery (trace builder, skill-metadata extraction, data-source population, adversarial rubric core). Derives; @typedstandards/produce-core assembles.",
5
5
  "license": "MIT",
6
6
  "author": "Nathan Storey",
@@ -33,7 +33,7 @@
33
33
  "prepublishOnly": "npm run clean && npm run build"
34
34
  },
35
35
  "dependencies": {
36
- "@typedstandards/produce-core": "^0.3.0"
36
+ "@typedstandards/produce-core": "^0.3.0 || ^0.4.0"
37
37
  },
38
38
  "devDependencies": {
39
39
  "@typedstandards/verify-core": "^0.9.0",