@typedstandards/civic-typed-harness 0.3.1 → 0.4.1

package/CHANGELOG.md

@@ -3,6 +3,246 @@
 Factual record of what changed per published version. Section references are
 to the Typed Standards specification unless noted otherwise.
 
+## 0.4.1 — 2026-09-12
+
+Patch: one existing export changes its output for one input shape, and no
+export is added, removed or retyped. That is measured across the release
+boundary rather than asserted — the published `0.4.0` and this build each
+expose 49 runtime exports from the package root, none added, none removed,
+none changing runtime type, and no declared top-level name in `dist/*.d.ts`
+differing between them. The tarball carries the same 28 files, and 22 of the
+24 under `dist/` are byte-identical to `0.4.0`; the two that are not are
+`dist/rubric/adversarial-eval-core.js` and its `.d.ts`, the latter differing
+by a doc comment alone.
+
+**What this release claims about golden bytes, and what it does not.** Every
+golden byte is unchanged. This time that sentence is not the evidence, and
+saying so is the point: **no golden fixture reaches the export that changed.**
+`buildEvaluationPrompt` is named in two test files, both under `src/rubric/`,
+and in neither golden suite. `__fixtures__/website-golden.json` carries no
+`queries` array at all; `__fixtures__/reference-golden.json` carries eight and
+**zero** occurrences of the `failed` key that is this release's only new
+branch. So the golden suites are green for this change the way they are green
+for a change to a file they never import, and their green is not evidence
+about it. This is the same narrowing
+[civic-ai-tools#199](https://github.com/npstorey/civic-ai-tools/issues/199)
+records for the graph builder, met from the other side.
+
+What **is** evidence is a direct measurement of the changed export either side
+of the release boundary: `buildEvaluationPrompt` imported from the published
+`0.4.0` and from this build, run over the same package.
+
+- A package in which **no entry records `failed: true`** renders
+  **byte-identical** on both — 492 bytes, `sha256 ed77c84e…` — over five entry
+  shapes: `resultRows: 12`, `resultRows: 0`, no `resultRows` at all,
+  `failed: false`, and a bare `failureKind` carrying no `failed`.
+- The same package plus one entry recorded `failed: true` renders
+  **differently** (539 bytes against 607). That case is the reason the first
+  one is worth reporting: an instrument that answers "identical" whatever it is
+  fed has not measured anything, and this one is shown answering "different"
+  on the shape it must flag.
+
+**The evaluator reads a rejected call as rejected**
+([civic-ai-tools#203](https://github.com/npstorey/civic-ai-tools/issues/203),
+Wave N11 P-H1). Behaviour change, confined to one input shape.
+
+- **The line that changes.** A `queries[]` entry the producer recorded as
+  rejected rendered `→ ? rows` — the same line an entry whose row count was
+  simply never recorded produces. A rejection and an unrecorded count were one
+  line to the evaluator, and the rubric asks that evaluator to cross-check the
+  answer's figures against the data the tool calls returned, so the conflation
+  invited it to read data that was never returned as merely uncounted. Its
+  score becomes a signed attestation. The entry now reads `→ REJECTED by the
+  source — no data was returned, and no row count is claimed`.
+- **`failed: true` is the whole assertion.** `failureKind` is declared on the
+  shape and **deliberately not rendered**: it is producer-controlled text, and
+  the turn built here is read by a model whose score is signed, so the
+  rejection is stated in this module's own fixed words, interpolating nothing
+  the producer wrote and claiming no row count.
+- **Absence stays absence.** An entry that records no outcome renders exactly
+  as it did before the fields existed. `failed: false` is not a rejection, and
+  a `failureKind` with no `failed` is not a rejection — the posture
+  `ToolCallSummary.failed` already takes in `capture/data-sources.ts`.
+- **No dependency moves.** The two keys are read off a locally-declared shape
+  rather than imported, following that same file: produce-core `0.3.0`, which
+  the lockfile resolves, names neither key. No manifest change, no lockfile
+  bump, no new dependency.
+
+**`RUBRIC_VERSION_SHA256` does not move, and the claim around it is corrected
+rather than left standing**
+([civic-ai-tools#207](https://github.com/npstorey/civic-ai-tools/issues/207)).
+The hash is `sha256Hex(EVALUATION_RUBRIC)` — the rubric text alone. It does not
+cover `buildEvaluationPrompt`, the template that assembles that rubric with the
+package's tool calls, data sources and model into the turn the evaluator
+actually reads. So this release changes what an evaluator is shown without
+changing the value an attestation pins as `methodology.promptSetVersion`: two
+evaluations either side of it carry the same value, measured here as
+`b62b193c992fd430cc82dff9a80a057bbe0b2b656fbdace450ca1a5d99fffa02` on both the
+published `0.4.0` and this build.
+
+- **The overclaim is retired in the same change.** ADR-0015 §3, Q26's
+  resolution note and the two module comments said the value pinned the whole
+  prompt set that produced the scores. They now say it pins the rubric
+  *wording*, and that the prompt template is outside the hash. The phrase they
+  used for the wider claim appears nowhere in the tree at this release.
+- **The gap is filed, not resolved.** #207 records what the hash covers, what
+  it does not, and the two spec-level options — widen the hash, or add a second
+  field — with no recommendation. Which one is right is a specification
+  decision, not a package's.
+
+A consumer sees none of this until its own lockfile moves: `^0.4.0` admits
+`0.4.1`, so the manifest range need not change and the refresh is the whole of
+the upgrade.
+
+## 0.4.0 — 2026-09-04
+
+Minor: two optional input fields are added and no existing export is removed
+or retyped. `fallbackPortal` and `ProvenanceInput.portal`, inert since 0.3.1,
+are widened and deprecated rather than dropped — removing either is breaking
+and waits for a major.
+
+**What this release claims about golden bytes, and what it does not.** Every
+golden byte is unchanged — both vocabulary eras, all eight
+golden-reproduction cases, and `__fixtures__/website-golden.json`. That
+sentence has appeared in every release note since 0.3.0, and it is narrower
+than it reads: **`__fixtures__/reference-golden.json` carries no spans at
+all** — `spanId` and `scopeSpans` each occur zero times across its eight
+cases, whose traces are empty `resourceSpans` arrays or BlobRefs. So those
+eight cases are green for anything the tool-span loop does, including the
+rejection marker this release adds, and their green is not evidence about it.
+
+`website-golden.json` is the only golden fixture that exercises span-derived
+output: five tool-call activities and four data-response descriptions. It is
+the fixture that proves this release's byte-stability claim, and the one the
+conditional spread of `civic:failed` was measured against — driven red by
+making that spread unconditional, which moves it. Read
+[civic-ai-tools#199](https://github.com/npstorey/civic-ai-tools/issues/199)
+before citing a green golden suite as evidence about the graph builder.
+
+**The reader-facing source name is measured, not changed**
+([civic-ai-tools#194](https://github.com/npstorey/civic-ai-tools/issues/194),
+Wave N10 P-H3). Tests only: no exported behaviour changes, and every golden
+byte — both vocabulary eras, all eight golden-reproduction cases — is
+unchanged.
+
+- **The change #194 asks for moves frozen golden bytes.** A data response the
+  builder cannot describe by a portal is described by its source's registry
+  `agentTitle` (`Data response from Socrata MCP Server`) — the agent's title,
+  implementation language on a reader-facing surface. Reading the registry's
+  `displayName` instead — the reader-facing name, in the registry beside
+  `agentTitle` since the package's first release — is a one-expression change,
+  and it rewrites two lines inside `__fixtures__/website-golden.json`'s
+  captured graph: the data-commons and boston-opencontext data responses, both
+  on the agent-title branch. Three whole-graph byte-parity assertions read
+  those two lines. A golden fixture is never edited to match new behaviour, so
+  the change waits on a decision about whether those bytes move.
+- **The vocabulary era cannot carry it.** `CivicVocabulary` binds exactly two
+  literals — the `civic:` namespace and the id-scheme prefix
+  (`makeCivicVocabulary(ns, urnPrefix)`) — and both era suites assert the two
+  eras differ by those and by nothing else. A description's wording is neither
+  of them and is not era-scoped: the same prefix is emitted under both eras, so
+  gating it on the era would make prior-era reproduction and settlement-era
+  emission disagree about a reader's wording rather than about an identifier.
+- **Two instruments, both able to fail.** `provenance.test.ts` now pins the
+  agent node's `dcterms:title` to the registry `agentTitle` — those are signed
+  bytes in both golden fixtures and the one thing that does not move under
+  either decision — and names the two golden descriptions at stake, asserting
+  each reads by agent title today and *not* by display name. The sources they
+  drive all carry two distinct registry names, which is what makes the
+  assertions able to fail; a builder changed under either one turns it red.
+- **What holds still either way.** The portal branch
+  (`Data response from data.cityofnewyork.us`) and the unknown-source fallback
+  (`Data response from euro stat`, the raw source id) move under neither
+  decision.
+
+**The PROV-O activity for a rejected call says it was rejected**
+([civic-ai-tools#193](https://github.com/npstorey/civic-ai-tools/issues/193),
+Wave N10 P-H2). Additive: no existing export is removed or retyped, and every
+golden byte — both vocabulary eras, all eight golden-reproduction cases — is
+unchanged.
+
+- **Two new activity terms.** `CIVIC_TERM_FAILED` (`civic:failed`) and
+  `CIVIC_TERM_FAILURE_KIND` (`civic:failureKind`) are declared in
+  `src/format/vocabulary.ts` and exported from the package root. A `civic:`
+  property name is vocabulary as much as the namespace it hangs under, so the
+  capture-side builder imports them rather than spelling them; `purity.test.ts`
+  now lists both, which is what makes that a claim able to fail.
+- **`buildProvenanceGraph` reads the span's failure.** A tool span the producer
+  ended with the boolean `error: true` yields a tool-call activity carrying
+  `civic:failed: true`, and `civic:failureKind` with the span's `error.kind`
+  verbatim when it carried one. Previously the builder read nine `tool.*` /
+  `mcp.*` attributes and never asked about the outcome, so a call the source
+  REFUSED and one that answered were the same node with the same description —
+  and, a rejected call having no response hash, the absent data-response entity
+  was the only trace of the rejection in the graph. `error.kind` is the one
+  attribute name across this package and the reference producer.
+- **`error` is the assertion, `error.kind` only a label on one.** A span
+  carrying a kind and no assertion is not a rejection and yields neither key —
+  the posture `ToolCallSummary.failed` / `failureKind` already takes.
+- **A new boolean attribute reader.** The module's string attribute reader
+  returns `stringValue ?? intValue` and cannot see `boolValue` at all, so it
+  returned `undefined` for a span that really did record a rejection.
+  `getBoolAttr` is a separate, strictly-typed reader rather than a widening of
+  the string one: the nine attributes that reader serves are strings by
+  contract, and a truthiness test would read the string `"false"` as an
+  assertion of failure. Only the boolean `true` marks an activity.
+- **The description states no cause.** `dcterms:description` is byte-unchanged
+  — it states what the call WAS, and the marker states how it ended. The
+  classified kind is the only cause the graph will ever carry: the reference
+  producer stopped writing a rejection's raw text onto the span in this same
+  wave, and the builder does not read it back in one layer up.
+- **Byte consequence.** Both terms are spread conditionally and appended after
+  every key the activity already carried, exactly as `civic:durationMs` is. A
+  span that recorded no rejection yields the 0.3.1 key list in the 0.3.1 order,
+  and `civic:failed: false` is never emitted — a producer that stated "not
+  failed" and one that stated nothing must read the same, which is what makes a
+  marker that IS present mean something. A rejected span carries no
+  `tool.duration_ms` from the reference producer today
+  ([civic-ai-tools-website#413](https://github.com/npstorey/civic-ai-tools-website/issues/413)),
+  so `civic:durationMs` does not fire beside the two new keys.
+
+**A call the record states as failed asserts no access**
+([civic-ai-tools#192](https://github.com/npstorey/civic-ai-tools/issues/192),
+Wave N10 P-H1). Additive: no existing export is removed, no existing caller
+changes, and every golden byte — both vocabulary eras, all eight
+golden-reproduction cases — is unchanged.
+
+- **`ToolCallSummary` can see the rejection.** Two new OPTIONAL fields:
+  `failed?: boolean`, the producer's assertion that the source rejected this
+  call, and `failureKind?: string`, the producer's own open label for why.
+  The harness never interprets the label, and `buildDataSources` never reads
+  it — `failed` is the assertion, `failureKind` only a label on one. A
+  summary carrying neither is exactly the 0.3.1 shape, and absence means
+  "not recorded as failed", never "succeeded".
+- **`buildDataSources` mints nothing from a rejected call.** A call whose
+  summary carries `failed: true` contributes no dataset-keyed entry for the
+  dataset it never read, and marks no aggregate source accessed. Previously
+  the population could not see the failure at all, so a rejected call minted
+  its dataset's entry and, on an aggregate source, marked that source
+  accessed at a timestamp — inside the bytes a publisher signs. The rejected
+  call keeps its POSITION in the walk (calls pair to spans by index), and it
+  remains on the PROV-O graph's tool-call activities and in the caller's own
+  `queries[]`. A source with any non-rejected call is still accessed; a
+  dataset a successful call also read still gets its entry.
+- **The two inert inputs are marked deprecated, not removed.**
+  `buildDataSources`'s `fallbackPortal` — accepted and not consulted since
+  0.3.1 — now also accepts `undefined`, so a caller that has stopped
+  consulting it can stop supplying a value; it stays third of five positional
+  parameters, because dropping a positional parameter is breaking.
+  `ProvenanceInput.portal` becomes optional and carries `@deprecated`;
+  removing it outright would make the reference app's object literal an
+  excess-property error. Both removals wait for a major.
+- **Byte consequence.** A package built from a record that states no failure
+  is byte-identical to one built by 0.3.1 — the walk reaches the new branch
+  only when a summary carries `failed: true`. A package built from a record
+  that DOES state a failure loses the `dataSources` entries that failure
+  never earned; that is the defect being fixed, and the wave re-emits
+  nothing.
+- **Type-level gate.** `src/capture/data-sources.assert.ts` pins the shape at
+  compile time. `tsconfig.json` excludes `src/**/*.test.ts`, so the suite
+  type-checks nothing — a test can drive a field the type does not have and
+  `npm run typecheck` stays green.
+
 ## 0.3.1 — 2026-09-02
 
 **The graph states what the span carried, and states absence as absence**

package/dist/capture/data-sources.assert.d.ts

@@ -0,0 +1 @@
+export {};

package/dist/capture/data-sources.assert.js

@@ -0,0 +1,37 @@
+// Type-level acceptance gate for the 0.4.0 additive change (civic-ai-tools#192):
+// `ToolCallSummary` can see a rejection the producer recorded, and BOTH new
+// fields are OPTIONAL — a caller that records no outcome passes neither and
+// compiles exactly as it did at 0.3.1.
+//
+// Why a file rather than a test: `tsconfig.json` EXCLUDES `src/**/*.test.ts`,
+// so the suite type-checks nothing. A test can drive a field the type does not
+// have and `npm run typecheck` stays green — which is exactly what happened
+// while this phase's red instrument was the only thing pinning the shape. The
+// type is pinned here instead, mirroring `src/required-config.assert.ts`.
+//
+// Compile-time only: nothing here is exported, and the guard is never invoked.
+import { buildDataSources } from "./data-sources.js";
+function pinsToolCallSummaryShape() {
+    // The 0.3.1 shape is still a ToolCallSummary — both new fields are optional.
+    const unrecorded = { name: 'get_data', args: {} };
+    // A recorded outcome is expressible, with or without a producer label.
+    const rejected = { name: 'get_data', args: {}, failed: true };
+    const succeeded = { name: 'get_data', args: {}, failed: false };
+    const labelled = {
+        name: 'get_data',
+        args: {},
+        failed: true,
+        failureKind: 'unavailable',
+    };
+    // @ts-expect-error #192 — `failed` is the producer's boolean assertion, never a string
+    const wrongFailedType = { name: 'get_data', args: {}, failed: 'yes' };
+    // @ts-expect-error #192 — `failureKind` is an open producer label (string), never a flag
+    const wrongKindType = { name: 'get_data', args: {}, failureKind: true };
+    // `fallbackPortal` stays positional third of five for callers that still
+    // pass it, and since 0.4.0 accepts `undefined` from callers that do not.
+    buildDataSources([unrecorded, rejected, succeeded, labelled], trace, 'portal.example', 'now');
+    buildDataSources([unrecorded], trace, undefined, 'now');
+    void wrongFailedType;
+    void wrongKindType;
+}
+void pinsToolCallSummaryShape;

package/dist/capture/data-sources.d.ts

@@ -6,6 +6,22 @@ export type { DataSourceEntry };
 export interface ToolCallSummary {
     name: string;
     args: Record<string, unknown>;
+    /** Did the producer record this call as REJECTED by the source? A call
+     *  recorded as failed asserts no access: it contributes no dataset-keyed
+     *  entry and marks no aggregate source accessed (see `buildDataSources`).
+     *
+     *  Optional, and absent is absent: a producer that records no outcome
+     *  passes neither this nor `failureKind`, and gets exactly the entries it
+     *  got before the fields existed. Absence means "not recorded as failed",
+     *  never "succeeded". Added 0.4.0. */
+    failed?: boolean;
+    /** The producer's own label for why the call was rejected, carried so a
+     *  caller can hand its record through unchanged. An open string with no
+     *  normative vocabulary: the harness never interprets it, and this module
+     *  never reads it. `failed` is the assertion and `failureKind` only a label
+     *  on one — a summary carrying a kind but no `failed` is not treated as a
+     *  rejection. Added 0.4.0. */
+    failureKind?: string;
 }
 interface TraceSpan {
     name: string;
@@ -57,8 +73,19 @@ export declare function resolveToolSource(toolCall: ToolCallSummary, span: Trace
  * dataset-keyed entries (first-seen order), then aggregate sources in
  * registry insertion order — matching the reference implementation.
  *
- * `fallbackPortal` is accepted and NOT consulted since 0.3.1: an entry
- * states the portal the call carried, never the run's. The parameter stays
- * in the signature so existing callers keep compiling.
+ * A call the producer recorded as FAILED (`ToolCallSummary.failed`) asserts
+ * no access and contributes nothing on either branch: no dataset-keyed entry
+ * for a dataset it never read, and no accessed-marking of its aggregate
+ * source. It keeps its POSITION in the walk, because calls are paired to
+ * spans by index. The call is still on the PROV-O graph's tool-call
+ * activities and in the caller's own `queries[]` — what it is not is an
+ * assertion, inside signed bytes, that a source was reached at a timestamp.
+ *
+ * @param fallbackPortal DEPRECATED, and inert since 0.3.1: an entry states
+ * the portal the call carried, never the run's. It stays third of five
+ * positional parameters so existing callers keep compiling, and since 0.4.0
+ * it also accepts `undefined`, which is what a caller that has stopped
+ * consulting it should pass. Dropping it is a breaking change and waits for
+ * a major.
  */
-export declare function buildDataSources(toolCalls: ToolCallSummary[], trace: Record<string, unknown>, fallbackPortal: string, now: string, options?: DataSourceOptions): DataSourceEntry[];
+export declare function buildDataSources(toolCalls: ToolCallSummary[], trace: Record<string, unknown>, fallbackPortal: string | undefined, now: string, options?: DataSourceOptions): DataSourceEntry[];

package/dist/capture/data-sources.js

@@ -61,9 +61,20 @@ export function resolveToolSource(toolCall, span, resolver = civicToolSourceReso
  * dataset-keyed entries (first-seen order), then aggregate sources in
  * registry insertion order — matching the reference implementation.
  *
- * `fallbackPortal` is accepted and NOT consulted since 0.3.1: an entry
- * states the portal the call carried, never the run's. The parameter stays
- * in the signature so existing callers keep compiling.
+ * A call the producer recorded as FAILED (`ToolCallSummary.failed`) asserts
+ * no access and contributes nothing on either branch: no dataset-keyed entry
+ * for a dataset it never read, and no accessed-marking of its aggregate
+ * source. It keeps its POSITION in the walk, because calls are paired to
+ * spans by index. The call is still on the PROV-O graph's tool-call
+ * activities and in the caller's own `queries[]` — what it is not is an
+ * assertion, inside signed bytes, that a source was reached at a timestamp.
+ *
+ * @param fallbackPortal DEPRECATED, and inert since 0.3.1: an entry states
+ * the portal the call carried, never the run's. It stays third of five
+ * positional parameters so existing callers keep compiling, and since 0.4.0
+ * it also accepts `undefined`, which is what a caller that has stopped
+ * consulting it should pass. Dropping it is a breaking change and waits for
+ * a major.
  */
 export function buildDataSources(toolCalls, trace, fallbackPortal, now, options = {}) {
     const resolver = options.resolver ?? civicToolSourceResolver;
@@ -74,6 +85,12 @@ export function buildDataSources(toolCalls, trace, fallbackPortal, now, options
     const aggregateAccessed = new Set();
     for (let i = 0; i < toolCalls.length; i++) {
         const tc = toolCalls[i];
+        // A call the producer recorded as rejected reached no data, so it mints
+        // no entry on either branch below. The walk keeps its index rather than
+        // filtering the list: `resolveToolSource` pairs a call to `toolSpans[i]`,
+        // and a filtered list would shift every later call onto the wrong span.
+        if (tc.failed)
+            continue;
         const source = resolveToolSource(tc, toolSpans[i], resolver, fallbackSourceId);
         if (isDatasetKeyedSource(source, registry)) {
             const datasetId = tc.args.dataset_id;

package/dist/capture/provenance.d.ts

@@ -18,10 +18,14 @@ export interface ProvenanceInput {
     /** The run's selected portal. Accepted and NOT consulted by the graph
      *  builder since 0.3.1: the graph states the portal a tool span carried
      *  (`tool.portal_domain`) and states absence as absence — it never
-     *  substitutes the run's portal for one the call did not address. The
-     *  field stays in the type so callers that pass it (the reference app
-     *  does, as an object literal) keep compiling. */
-    portal: string;
+     *  substitutes the run's portal for one the call did not address.
+     *
+     *  @deprecated Inert since 0.3.1 and optional since 0.4.0 — a caller that
+     *  has stopped consulting it should stop passing it. The field stays in
+     *  the type so callers that do pass it (the reference app does, as an
+     *  object literal) keep compiling; removing it outright would make that
+     *  literal an excess-property error, so removal waits for a major. */
+    portal?: string;
 }
 /** Instance configuration for the graph build (config-not-constants). */
 export interface ProvenanceConfig {

package/dist/capture/provenance.js

@@ -17,7 +17,7 @@
 // order is preserved throughout — the legacy hash chain's byte contract).
 import { makeProvGraph, makeEntityNode, makeActivityNode, makeAgentNode, provRef, provUsed, provWasGeneratedBy, provWasDerivedFrom, provWasAssociatedWith, xsdDateTime, } from '@typedstandards/produce-core';
 import { hash } from "./trace.js";
-import { CIVIC_VOCABULARY, CIVICAITOOLS_PLATFORM_AGENT, } from "../format/vocabulary.js";
+import { CIVIC_VOCABULARY, CIVICAITOOLS_PLATFORM_AGENT, CIVIC_TERM_FAILED, CIVIC_TERM_FAILURE_KIND, } from "../format/vocabulary.js";
 import { CIVIC_SOURCE_REGISTRY, FALLBACK_SOURCE_ID, isDatasetKeyedSource, } from "../format/sources.js";
 /** The civicaitools.org reference deployment's values. Passed explicitly by
  *  the reference app — never applied as a default, and never spread into
@@ -33,6 +33,30 @@ function getAttr(attrs, key) {
     const attr = attrs.find(a => a.key === key);
     return attr?.value?.stringValue ?? attr?.value?.intValue ?? undefined;
 }
+/**
+ * Read an attribute the producer wrote as a BOOLEAN.
+ *
+ * `getAttr` above returns `stringValue ?? intValue` and cannot see the third
+ * OTel value shape at all — `TraceBuilder` encodes a boolean as
+ * `{ boolValue }` (trace.ts), so `getAttr(attrs, 'error')` returns `undefined`
+ * for a span the producer really did end with `error: true`. This is a
+ * separate reader rather than a widening of `getAttr` on purpose: the nine
+ * attributes `getAttr` serves are strings by contract and feed hashes,
+ * descriptions and `Number()`, and teaching it a fourth return shape would
+ * change what all nine yield for value shapes nobody has measured.
+ *
+ * The comparison is strict, and only `true` counts. A truthiness test over
+ * `stringValue` would read the string `"false"` as an assertion of failure —
+ * marking a call that ANSWERED as refused, inside bytes a publisher signs.
+ * The cost of the strictness is that a producer encoding this attribute as a
+ * string gets no marker; the reference producer does not, and under-reading a
+ * foreign encoding leaves the graph silent, where over-reading one would make
+ * it lie.
+ */
+function getBoolAttr(attrs, key) {
+    const attr = attrs.find(a => a.key === key);
+    return typeof attr?.value?.boolValue === 'boolean' ? attr.value.boolValue : undefined;
+}
 function nanoToIso(nano) {
     const ms = Math.floor(Number(nano) / 1_000_000);
     return new Date(ms).toISOString();
@@ -267,6 +291,27 @@ export function buildProvenanceGraph(trace, input, config) {
         // the call. In multi-source analyses each call may target a different
         // agent (e.g. socrata for `get_data`, data-commons for `get_observations`).
         const durationMs = getAttr(span.attributes, 'tool.duration_ms');
+        // Did the SOURCE REFUSE this call? Before 0.4.0 the graph never asked, so
+        // a refused call and one that answered were the same node with the same
+        // description — the absence of a data-response entity was the only trace
+        // of the rejection, and absence stated as nothing is not absence stated.
+        //
+        // `error` is the ASSERTION and `error.kind` only a LABEL on one. A span
+        // carrying a kind and no assertion is not a rejection, so the kind is read
+        // and stated only inside the failure branch — the posture
+        // `ToolCallSummary.failed`/`failureKind` already takes on the other input.
+        // `error.kind` is the one attribute name across this package and the
+        // reference producer (Wave N10 D5); neither side invents a second.
+        //
+        // The kind is the ONLY cause this graph will ever state. The producer
+        // stopped writing a rejection's raw text onto the span in the same wave —
+        // that text is authored by the source, can name a host, a port or a stack
+        // frame, and the trace travels inline inside the bytes an instance signs.
+        // Reading it here would put it back one layer up; the classified vocabulary
+        // widens instead. Nor does the description change: it states what the call
+        // WAS, and the marker below states how it ended.
+        const failed = getBoolAttr(span.attributes, 'error');
+        const failureKind = failed === true ? getAttr(span.attributes, 'error.kind') : undefined;
         graph.push(makeActivityNode(toolCallUrn, {
             // Names the tool only when the span did.
             'dcterms:description': toolName
@@ -282,6 +327,17 @@ export function buildProvenanceGraph(trace, input, config) {
                 ? { 'prov:endedAtTime': xsdDateTime(nanoToIso(span.endTimeUnixNano)) }
                 : {}),
             ...(durationMs ? { 'civic:durationMs': Number(durationMs) } : {}),
+            // Appended LAST and spread conditionally, exactly as `civic:durationMs`
+            // above is: a span that recorded no rejection yields the key list it
+            // yielded at 0.3.1, in the same order, and property insertion order is
+            // the legacy chain's byte contract. `false` is never emitted — a
+            // producer that stated "not failed" and one that stated nothing must
+            // read the same, which is what makes a marker that IS present mean
+            // something. A rejected span carries no `tool.duration_ms` from the
+            // reference producer today (civic-ai-tools-website#413), so
+            // `civic:durationMs` simply does not fire beside these two.
+            ...(failed === true ? { [CIVIC_TERM_FAILED]: true } : {}),
+            ...(failureKind ? { [CIVIC_TERM_FAILURE_KIND]: failureKind } : {}),
         }));
     }
     // --- Final output relationships ---

package/dist/format/vocabulary.d.ts

@@ -11,6 +11,30 @@ export declare const PRIOR_ERA_CIVIC_NS = "https://civicaitools.org/ns/evidence/
 /** The prior-era id-scheme root, frozen inside records signed before the
  *  2026-08-19 settlement. Never emitted for new packages. */
 export declare const PRIOR_ERA_CIVIC_URN_PREFIX = "urn:civic-evidence";
+/**
+ * `civic:failed` — a tool-call activity whose call the SOURCE REFUSED.
+ *
+ * Emitted with the boolean `true` and only when the producer stated the
+ * rejection; never emitted as `false`. A literal `false` would assert an
+ * outcome, and "recorded as not-failed" must stay indistinguishable from
+ * "nothing recorded" — the same absent-is-absent posture `ToolCallSummary`
+ * takes for `failed`. A reader that finds no key learns that the record
+ * states nothing, which is the truth about every package minted before 0.4.0.
+ */
+export declare const CIVIC_TERM_FAILED = "civic:failed";
+/**
+ * `civic:failureKind` — the producer's own classified label for WHY a call
+ * carrying {@link CIVIC_TERM_FAILED} was refused.
+ *
+ * An open string with no normative vocabulary here: the harness states the
+ * label verbatim and never interprets or re-derives it. It is a label on an
+ * assertion, not the assertion — a record that carries a kind and no failure
+ * is not a rejection, and the term is emitted only alongside
+ * {@link CIVIC_TERM_FAILED}. The reference producer's four values are
+ * `timeout`, `unavailable`, `not_configured` and `unknown`; a producer with a
+ * wider vocabulary widens this field rather than putting prose in it.
+ */
+export declare const CIVIC_TERM_FAILURE_KIND = "civic:failureKind";
 /**
  * One era of the civic vocabulary: the two literals plus the emitters bound
  * to them. The vocabulary is not deployment configuration (a deployment does

package/dist/format/vocabulary.js

@@ -32,6 +32,45 @@ export const PRIOR_ERA_CIVIC_NS = 'https://civicaitools.org/ns/evidence/';
 /** The prior-era id-scheme root, frozen inside records signed before the
  *  2026-08-19 settlement. Never emitted for new packages. */
 export const PRIOR_ERA_CIVIC_URN_PREFIX = 'urn:civic-evidence';
+// --- Term names ---
+//
+// A `civic:` property name is vocabulary as much as the namespace it hangs
+// under: it is the word a reader of a signed graph interprets, so it is
+// declared here and imported by the builder that emits it, never spelled as a
+// literal inside capture/ (`purity.test.ts` enforces that for the terms below).
+// Both terms are era-independent — the settlement moved the NAMESPACE, not the
+// property names — so they carry no era qualifier and are minted for the first
+// time after it.
+//
+// The settlement-era property names that predate Wave N10 (`civic:sourceId`,
+// `civic:durationMs`, `civic:datasetId` and their siblings) are still inline
+// literals in the capture-side builder. That is a real inconsistency, recorded
+// rather than fixed in passing: moving them is a byte-sensitive change across
+// every golden fixture and belongs to a phase of its own.
+/**
+ * `civic:failed` — a tool-call activity whose call the SOURCE REFUSED.
+ *
+ * Emitted with the boolean `true` and only when the producer stated the
+ * rejection; never emitted as `false`. A literal `false` would assert an
+ * outcome, and "recorded as not-failed" must stay indistinguishable from
+ * "nothing recorded" — the same absent-is-absent posture `ToolCallSummary`
+ * takes for `failed`. A reader that finds no key learns that the record
+ * states nothing, which is the truth about every package minted before 0.4.0.
+ */
+export const CIVIC_TERM_FAILED = 'civic:failed';
+/**
+ * `civic:failureKind` — the producer's own classified label for WHY a call
+ * carrying {@link CIVIC_TERM_FAILED} was refused.
+ *
+ * An open string with no normative vocabulary here: the harness states the
+ * label verbatim and never interprets or re-derives it. It is a label on an
+ * assertion, not the assertion — a record that carries a kind and no failure
+ * is not a rejection, and the term is emitted only alongside
+ * {@link CIVIC_TERM_FAILED}. The reference producer's four values are
+ * `timeout`, `unavailable`, `not_configured` and `unknown`; a producer with a
+ * wider vocabulary widens this field rather than putting prose in it.
+ */
+export const CIVIC_TERM_FAILURE_KIND = 'civic:failureKind';
 /** Bind the emitters to one era's two literals. */
 export function makeCivicVocabulary(ns, urnPrefix) {
     return {

package/dist/rubric/adversarial-eval-core.d.ts

@@ -6,7 +6,10 @@ export declare const EVALUATION_CRITERIA: readonly ["dataSourceIdentification",
 export declare const EVALUATION_RUBRIC = "You are an independent evaluator assessing an AI-generated civic data analysis.\n\nYou will receive:\n1. The original prompt (question asked)\n2. The tool calls made (MCP queries to Socrata open data portals)\n3. The AI's final output/analysis\n\nEvaluate the analysis against these 6 criteria, scoring each 1-10:\n\na) **Data Source Identification** (1-10): Does the output correctly identify the data source(s) and time period? Are dataset IDs, portal domains, and date ranges accurate?\n\nb) **Quantitative Claim Support** (1-10): Are the quantitative claims (numbers, percentages, rankings) supported by the data returned in the tool calls? Cross-check key figures against the raw data.\n\nc) **Confounders and Bias** (1-10): Does the analysis acknowledge obvious confounders, selection biases, or framing issues? Are there lurking variables or cherry-picked timeframes?\n\nd) **Geographic Scope** (1-10): Is the geographic scope appropriate for the question? Does the analysis avoid over-generalizing from one jurisdiction?\n\ne) **Limitations Noted** (1-10): Are limitations and caveats noted? Does the analysis flag data quality issues, missing fields, or incomplete coverage?\n\nf) **Contradictory Conclusion** (1-10): Could the same data reasonably support a contradictory conclusion? Does the analysis consider alternative interpretations?\n\nRespond in this exact JSON format (no markdown fences, just raw JSON):\n{\n  \"dataSourceIdentification\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n  \"quantitativeClaimSupport\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n  \"confoundersAndBias\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n  \"geographicScope\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n  \"limitationsNoted\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n  \"contradictoryConclusion\": { \"score\": <1-10>, \"comment\": \"<1-2 sentences>\" },\n  \"overallScore\": <average of all 6 scores, one decimal>,\n  \"assessment\": \"<2-4 sentence overall assessment>\"\n}";
 /** The methodology's `promptSetVersion`: SHA-256 of the rubric text, computed
  *  once at module load. Any wording change to the rubric produces a new
- *  version, so an attestation pins the exact prompt set that scored it. */
+ *  version, so an attestation pins the rubric WORDING that scored it — and
+ *  that wording only. The prompt template around it (`buildEvaluationPrompt`)
+ *  is outside this hash, so one value does not identify one set of evaluator
+ *  inputs (civic-ai-tools#207). */
 export declare const RUBRIC_VERSION_SHA256: string;
 /** Build the user-turn evaluation content from a package's signed fields.
  *  Takes `RecordPackage` — produce-core 0.3.0's settlement-era name for the

package/dist/rubric/adversarial-eval-core.js

@@ -5,8 +5,14 @@
 // brief §1. The model runner and DB/blob emission (`adversarial-eval.ts`)
 // stay app-side — this module never performs I/O.
 //
-// ⚠ RUBRIC_VERSION_SHA256 is a Q26-pinned version hash: attestations pin the
-// exact prompt set that scored them by this value. The rubric text below is
+// ⚠ RUBRIC_VERSION_SHA256 is a Q26-pinned version hash over THE RUBRIC TEXT
+// ALONE (`sha256Hex(EVALUATION_RUBRIC)`). It does NOT cover
+// `buildEvaluationPrompt` below — the template that assembles the rubric with
+// the package's tool calls, data sources and model into the turn the evaluator
+// actually reads. So two evaluations either side of a change to that template
+// carry the SAME `methodology.promptSetVersion` while the evaluator saw
+// different inputs; civic-ai-tools#207 files that gap for the specification to
+// settle, and this module does not resolve it. The rubric text below is
 // byte-exact with the reference implementation (the relocation changes no
 // hashes), and the test suite asserts the literal digest. Any wording change
 // to EVALUATION_RUBRIC is a NEW rubric version — never edit the text without
@@ -57,8 +63,24 @@ Respond in this exact JSON format (no markdown fences, just raw JSON):
 }`;
 /** The methodology's `promptSetVersion`: SHA-256 of the rubric text, computed
  *  once at module load. Any wording change to the rubric produces a new
- *  version, so an attestation pins the exact prompt set that scored it. */
+ *  version, so an attestation pins the rubric WORDING that scored it — and
+ *  that wording only. The prompt template around it (`buildEvaluationPrompt`)
+ *  is outside this hash, so one value does not identify one set of evaluator
+ *  inputs (civic-ai-tools#207). */
 export const RUBRIC_VERSION_SHA256 = sha256Hex(EVALUATION_RUBRIC);
+/** Did the producer record this entry as REJECTED by the source? Only a
+ *  literal `failed: true` is a rejection (see `RecordedCallOutcome`). */
+function wasRecordedRejected(query) {
+    return query.failed === true;
+}
+/** What the evaluator is told about a call the source rejected. A FIXED
+ *  string: it interpolates nothing the producer wrote, and it claims no row
+ *  count — there were no rows to count, which is neither "zero rows" nor "an
+ *  unknown number of rows". The rubric asks the evaluator to cross-check
+ *  figures against the data returned in the tool calls, so a rejected call
+ *  reading `→ ? rows` invited it to treat absent data as merely unrecorded
+ *  (civic-ai-tools#203). */
+const REJECTED_CALL_RENDERING = 'REJECTED by the source — no data was returned, and no row count is claimed';
 /** Build the user-turn evaluation content from a package's signed fields.
  *  Takes `RecordPackage` — produce-core 0.3.0's settlement-era name for the
  *  same object (spec Appendix J, migration class alias-and-deprecate). The
@@ -70,7 +92,10 @@ export function buildEvaluationPrompt(pkg) {
         const argStr = Object.entries(q.arguments)
             .map(([k, v]) => `${k}=${JSON.stringify(v)}`)
             .join(', ');
-        return `  ${i + 1}. ${q.tool}(${argStr}) → ${q.resultRows ?? '?'} rows`;
+        const outcome = wasRecordedRejected(q)
+            ? REJECTED_CALL_RENDERING
+            : `${q.resultRows ?? '?'} rows`;
+        return `  ${i + 1}. ${q.tool}(${argStr}) → ${outcome}`;
     })
         .join('\n');
     const dataSources = pkg.dataSources

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@typedstandards/civic-typed-harness",
-  "version": "0.3.1",
+  "version": "0.4.1",
   "description": "Civic-domain harness for Typed Standards evidence packages — the DOMAIN layer of the ADR-0021 format/domain line: civic PROV-O vocabulary and provenance builder, datHere envelope-policy derivations, and the capture machinery (trace builder, skill-metadata extraction, data-source population, adversarial rubric core). Derives; @typedstandards/produce-core assembles.",
   "license": "MIT",
   "author": "Nathan Storey",