@typedstandards/civic-typed-harness 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,57 @@
1
+ # Changelog — @typedstandards/civic-typed-harness
2
+
3
+ Factual record of what changed per published version. Section references are
4
+ to the Typed Standards specification unless noted otherwise.
5
+
6
+ ## 0.1.0 — 2026-08-02
7
+
8
+ Initial release: the civic **domain harness** for Typed Standards evidence
9
+ packages — the DOMAIN side of the format/domain line drawn in
10
+ [ADR-0021](../../docs/adr/0021-produce-core-extraction.md), relocated whole
11
+ from the reference application and packaged per
12
+ [ADR-0022](../../docs/adr/0022-civic-typed-harness-packaging.md). Operating
13
+ rule: **the harness derives, the core assembles.**
14
+
15
+ - **Published as `@typedstandards/civic-typed-harness`.** The package landed
16
+ in-repo under the working name `civic-typed-harness`, `private: true`, with
17
+ the final npm name deferred to the project's naming taxonomy (ADR-0022 §B).
18
+ The decision selected the standard's scope with the full layer-3 basename;
19
+ rationale, the alternatives scored, and the named reversal condition are in
20
+ the planning-side naming memo (`naming-research-harness-scope.md`), recorded
21
+ as a dated addendum to ADR-0022.
22
+ - **Format-extension group** (`src/format/`) — the `civic:` JSON-LD namespace
23
+ and `urn:civic-evidence:` id scheme, the civic source registry, the datHere
24
+ envelope-policy derivations (producerProfile, content-canonicalization rule,
25
+ summary emission, the `org.civicaitools.environment` extension), and the
26
+ captureMethod vocabulary surface (verify-core's Q32 fallback-table entry,
27
+ single-sourced — flagged, not solved, here).
28
+ - **Capture group** (`src/capture/`) — the OTel-compatible `TraceBuilder`,
29
+ skill-metadata extraction, data-source population, and the civic provenance
30
+ graph builder. Clock and RNG are permitted here only, and are injectable.
31
+ - **Rubric group** (`src/rubric/`) — the civic six-criterion adversarial
32
+ evaluation core: rubric text, the Q26-pinned `RUBRIC_VERSION_SHA256`, prompt
33
+ builder, and response parser. The model runner and result emission stay
34
+ implementation-side; this module performs no I/O.
35
+ - **Config, not constants** (ADR-0020 posture at the domain layer) — every
36
+ value naming a deployment (platform-agent identity/URL, MCP server URLs, the
37
+ environment-extension `host`, trace `service.name` and scope identity) is a
38
+ typed config input with the demo values exported as overridable defaults.
39
+ - **Single declared runtime dependency: `@typedstandards/produce-core`**
40
+ (`^0.2.0`). The verify-core primitives the harness consumes (`sha256Hex`,
41
+ `isBlobRef`, the canonicalization-rule URIs, the Q32 vocabulary table and
42
+ the `CaptureMethod` type) are reached through produce-core 0.2.0's
43
+ re-exports rather than importing `@typedstandards/verify-core` directly —
44
+ no phantom dependency, and producer, harness, and verifier still share one
45
+ hashing implementation by construction (typedstandards#35 class of bug,
46
+ flagged in civic-ai-tools#116 P1).
47
+ - **Purity contract (harness-grade)** — no I/O, no network, no environment
48
+ reads, no Node built-ins, browser-safe throughout; enforced twice, by
49
+ `eslint.config.mjs` and by `src/purity.test.ts`, which also enforces the
50
+ internal format/capture module boundary that reserves the future package
51
+ split (ADR-0022 §C).
52
+ - **Byte-compatibility proof** — golden fixtures captured from the reference
53
+ implementation: provenance graphs and dataSources arrays reproduce
54
+ byte-for-byte under the demo default config, `RUBRIC_VERSION_SHA256` is
55
+ asserted against the exact reference digest, and a produce→verify
56
+ composition round-trip runs the harness output through produce-core's
57
+ envelope assembly and verify-core's §9.2 verification sequence.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2025 Nathan Storey
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,49 @@
1
+ # @typedstandards/civic-typed-harness
2
+
3
+ The civic **domain harness** for [Typed Standards](../../docs/architecture/typed-standards-specification.md) evidence packages — the DOMAIN side of the format/domain line drawn in [ADR-0021](../../docs/adr/0021-produce-core-extraction.md), relocated from the reference app as an installable package ([ADR-0022](../../docs/adr/0022-civic-typed-harness-packaging.md)). The operating rule: **the harness derives, the core assembles** — everything here produces domain-derived values (civic vocabulary terms, datHere policy fields, provenance graphs, capture artifacts) that feed [`@typedstandards/produce-core`](https://www.npmjs.com/package/@typedstandards/produce-core)'s neutral envelope assembly.
4
+
5
+ > **Name.** The package landed in-repo under the working name `civic-typed-harness` with the final npm name deferred (ADR-0022 §B). It publishes as **`@typedstandards/civic-typed-harness`** — the standard's scope with the full layer-3 basename; the rationale, the alternatives scored, and the named reversal condition are recorded in the planning-side naming memo and summarized in the [ADR-0022 addendum](../../docs/adr/0022-civic-typed-harness-packaging.md#addendum-2026-08-02--final-npm-name).
6
+
7
+ ## What's inside
8
+
9
+ | Module group | Modules | What it is |
10
+ |---|---|---|
11
+ | **`src/format/`** — the civic format-extension | `vocabulary.ts`, `sources.ts`, `dathere.ts`, `profiles.ts` | What extends the *standard*: the `civic:` JSON-LD namespace and `urn:civic-evidence:` id scheme, the civic source registry, the datHere envelope-policy derivations (producerProfile, canonicalization rule, summary emission, environment extension), and the captureMethod vocabulary surface. |
12
+ | **`src/capture/`** — the capture machinery | `trace.ts`, `skill-metadata.ts`, `data-sources.ts`, `provenance.ts` | What *produces* under it: the OTel `TraceBuilder`, skill-metadata extraction, data-source population, and the provenance **builder** — which imports its vocabulary from the format group and walks the trace. |
13
+ | **`src/rubric/`** — the adversarial rubric | `adversarial-eval-core.ts` | The civic six-criterion evaluation rubric (pure core: rubric text, pinned version hash, prompt builder, response parser). The model runner and result emission stay implementation-side. |
14
+
15
+ **The internal boundary is load-bearing:** no capture module defines vocabulary, and no format-extension module walks a trace — this sentence reserves a future split of the format-extension group into its own package (a civic extension *of Typed Standards*, distinct from the civic *harness* proper) as a mechanical move rather than a re-partition. The boundary is enforced by `src/purity.test.ts`.
16
+
17
+ ## Config, not constants
18
+
19
+ Every value that names a deployment is a **typed config input**, with the civicaitools.org demo values exported as overridable defaults:
20
+
21
+ | Value | Config type | Demo default export |
22
+ |---|---|---|
23
+ | Platform-agent identity/URL | `PlatformAgentConfig` | `CIVICAITOOLS_PLATFORM_AGENT` |
24
+ | MCP source registry (server URLs, catalog types, display names) | `CivicSourceRegistry` | `CIVIC_SOURCE_REGISTRY` |
25
+ | Tool-name → source-id resolver | `ToolSourceResolver` | `civicToolSourceResolver` |
26
+ | Environment-extension `host` | `DatHereEnvironmentConfig` | `CIVICAITOOLS_ENVIRONMENT_CONFIG` |
27
+ | Trace `service.name` / scope identity | `TraceBuilderConfig` | `CIVICAITOOLS_TRACE_CONFIG` |
28
+ | Provenance build (all of the above + model-agent description) | `ProvenanceConfig` | `CIVICAITOOLS_PROVENANCE_CONFIG` |
29
+
30
+ ## Purity contract (harness-grade)
31
+
32
+ No I/O, no network, no environment reads, no Node built-ins, browser-safe — anywhere in the package. Clock and RNG are permitted **in capture modules only** (span timestamps and ids are what capture *is*) and are injectable for deterministic tests. Enforced mechanically twice: `eslint.config.mjs` (`no-restricted-imports` / `-globals` / `-properties` / `-syntax`) and `src/purity.test.ts` (browser-safety + determinism + module-boundary checks under `node --test`).
33
+
34
+ Runtime dependency: **`@typedstandards/produce-core` (`^0.2.0`) only.** The verify-core primitives the harness consumes (`sha256Hex`, `isBlobRef`, the canonicalization-rule URIs, the Q32 vocabulary table, the `CaptureMethod` type) are reached through produce-core's 0.2.0 re-exports rather than imported from `@typedstandards/verify-core` directly — one declared dependency, no phantom, and still one hashing implementation across producer, harness, and verifier by construction (produce-core re-exports the same bindings verify-core defines, verified by reference identity).
35
+
36
+ > **Q32 adjacency** (open-questions registry): the captureMethod vocabulary re-exported from `src/format/profiles.ts` is verify-core's hardcoded fallback table, pending versioned, content-addressed guidance bundles. Flagged, not solved, here.
37
+
38
+ ## Byte-compatibility
39
+
40
+ The port is tested for **byte parity** against golden outputs captured from the reference implementation (`src/__fixtures__/website-golden.json`): with the demo default config, `buildProvenanceGraph` and `buildDataSources` reproduce the app's output byte-for-byte, and `RUBRIC_VERSION_SHA256` is asserted against the exact reference digest (moving the rubric changes no hashes). Fixture-level byte-compat against produce-core's `reference-golden.json` (`src/golden-reproduction.test.ts`) and the produce→verify composition round-trip (`src/composition-roundtrip.test.ts`) landed in S2 P2 and run with the suite.
41
+
42
+ ## Develop
43
+
44
+ ```bash
45
+ npm install # from the repo root (npm workspaces)
46
+ npm test --workspace @typedstandards/civic-typed-harness
47
+ npm run lint --workspace @typedstandards/civic-typed-harness
48
+ npm run build --workspace @typedstandards/civic-typed-harness
49
+ ```
@@ -0,0 +1,58 @@
1
+ import type { DataSourceEntry } from '@typedstandards/produce-core';
2
+ import { type CivicSourceRegistry, type ToolSourceResolver } from '../format/sources.ts';
3
+ export type { DataSourceEntry };
4
+ /** The caller-supplied per-tool-call summary the population walks alongside
5
+ * the trace. */
6
+ export interface ToolCallSummary {
7
+ name: string;
8
+ args: Record<string, unknown>;
9
+ }
10
+ interface TraceSpan {
11
+ name: string;
12
+ attributes?: Array<{
13
+ key: string;
14
+ value?: {
15
+ stringValue?: string;
16
+ intValue?: string;
17
+ boolValue?: boolean;
18
+ };
19
+ }>;
20
+ }
21
+ /** Optional knobs for `resolveToolSource` / `buildDataSources`. Defaults are
22
+ * the civic demo values. */
23
+ export interface DataSourceOptions {
24
+ /** Tool-name → source-id resolver (fallback when a span carries no
25
+ * `mcp.source` attribute). Default: the civic map. */
26
+ resolver?: ToolSourceResolver;
27
+ /** Source registry driving catalog types, endpoints, and the
28
+ * dataset-keyed vs aggregate split. Default: the civic registry. */
29
+ registry?: CivicSourceRegistry;
30
+ /** Source id for calls neither the trace nor the resolver can identify.
31
+ * Default `socrata` (pre-M9.1 packages predate source tagging). */
32
+ fallbackSourceId?: string;
33
+ }
34
+ /**
35
+ * Resolve the MCP source for a tool call. Prefers the `mcp.source` attribute
36
+ * recorded on the matching `mcp_tool_call` span (the trace is the source of
37
+ * truth); falls back to the resolver's static mapping for packages written
38
+ * before source tagging or callers that ship an empty trace.
39
+ *
40
+ * Tool calls are paired to spans by index — the reference capture emits one
41
+ * span per call in order, so positional matching is exact in the normal
42
+ * flow. When the counts diverge, the static resolver still identifies the
43
+ * source.
44
+ */
45
+ export declare function resolveToolSource(toolCall: ToolCallSummary, span: TraceSpan | undefined, resolver?: ToolSourceResolver, fallbackSourceId?: string): string;
46
+ /**
47
+ * Build the per-source evidence-package `dataSources` array.
48
+ *
49
+ * Dataset-keyed sources (Socrata) contribute one entry per unique
50
+ * `dataset_id` observed across tool calls. Aggregate sources (Data Commons,
51
+ * Boston OpenContext — registry entries carrying `aggregatePortalUrl`)
52
+ * contribute a single entry when any of their tool calls was made. Unknown
53
+ * source ids contribute no entry. Each entry is tagged with `sourceId` so
54
+ * downstream consumers can distinguish provenance. Emission order: the
55
+ * dataset-keyed entries (first-seen order), then aggregate sources in
56
+ * registry insertion order — matching the reference implementation.
57
+ */
58
+ export declare function buildDataSources(toolCalls: ToolCallSummary[], trace: Record<string, unknown>, fallbackPortal: string, now: string, options?: DataSourceOptions): DataSourceEntry[];
@@ -0,0 +1,119 @@
1
+ // Data-source population (CAPTURE group) — walks the trace's `mcp_tool_call`
2
+ // spans plus the caller-supplied tool-call summary to produce one
3
+ // `dataSources` entry per (source, datasetId) tuple. Relocated from
4
+ // civic-ai-tools-website `src/lib/evidence/data-sources.ts:78–185` per the S2
5
+ // brief §1, with two changes:
6
+ // - The tool-name → source-id resolver is a CALLER-SUPPLIED input (the
7
+ // app's MCP registry stays app-side); `civicToolSourceResolver` is the
8
+ // exported civic default.
9
+ // - The per-source coordinates (catalog types, endpoints) come from the
10
+ // format-extension group's source registry instead of module constants —
11
+ // which sources are dataset-keyed vs aggregate is registry-driven.
12
+ //
13
+ // `DataSourceEntry` is produce-core's envelope input shape — the harness
14
+ // populates it, never redefines it.
15
+ import { CIVIC_SOURCE_REGISTRY, FALLBACK_SOURCE_ID, civicToolSourceResolver, isDatasetKeyedSource, } from "../format/sources.js";
16
+ function getToolSpans(trace) {
17
+ try {
18
+ // Untyped walk over the caller's trace shape.
19
+ const spans = trace?.resourceSpans?.[0]?.scopeSpans?.[0]?.spans;
20
+ if (!Array.isArray(spans))
21
+ return [];
22
+ return spans.filter((s) => s.name === 'mcp_tool_call');
23
+ }
24
+ catch {
25
+ return [];
26
+ }
27
+ }
28
+ function spanAttr(span, key) {
29
+ if (!span)
30
+ return undefined;
31
+ const attr = span.attributes?.find((a) => a.key === key);
32
+ return attr?.value?.stringValue ?? attr?.value?.intValue ?? undefined;
33
+ }
34
+ /**
35
+ * Resolve the MCP source for a tool call. Prefers the `mcp.source` attribute
36
+ * recorded on the matching `mcp_tool_call` span (the trace is the source of
37
+ * truth); falls back to the resolver's static mapping for packages written
38
+ * before source tagging or callers that ship an empty trace.
39
+ *
40
+ * Tool calls are paired to spans by index — the reference capture emits one
41
+ * span per call in order, so positional matching is exact in the normal
42
+ * flow. When the counts diverge, the static resolver still identifies the
43
+ * source.
44
+ */
45
+ export function resolveToolSource(toolCall, span, resolver = civicToolSourceResolver, fallbackSourceId = FALLBACK_SOURCE_ID) {
46
+ return spanAttr(span, 'mcp.source')
47
+ ?? resolver(toolCall.name)
48
+ ?? fallbackSourceId;
49
+ }
50
+ /**
51
+ * Build the per-source evidence-package `dataSources` array.
52
+ *
53
+ * Dataset-keyed sources (Socrata) contribute one entry per unique
54
+ * `dataset_id` observed across tool calls. Aggregate sources (Data Commons,
55
+ * Boston OpenContext — registry entries carrying `aggregatePortalUrl`)
56
+ * contribute a single entry when any of their tool calls was made. Unknown
57
+ * source ids contribute no entry. Each entry is tagged with `sourceId` so
58
+ * downstream consumers can distinguish provenance. Emission order: the
59
+ * dataset-keyed entries (first-seen order), then aggregate sources in
60
+ * registry insertion order — matching the reference implementation.
61
+ */
62
+ export function buildDataSources(toolCalls, trace, fallbackPortal, now, options = {}) {
63
+ const resolver = options.resolver ?? civicToolSourceResolver;
64
+ const registry = options.registry ?? CIVIC_SOURCE_REGISTRY;
65
+ const fallbackSourceId = options.fallbackSourceId ?? FALLBACK_SOURCE_ID;
66
+ const toolSpans = getToolSpans(trace);
67
+ const datasetKeyed = new Map();
68
+ const aggregateAccessed = new Set();
69
+ for (let i = 0; i < toolCalls.length; i++) {
70
+ const tc = toolCalls[i];
71
+ const source = resolveToolSource(tc, toolSpans[i], resolver, fallbackSourceId);
72
+ if (isDatasetKeyedSource(source, registry)) {
73
+ const datasetId = tc.args.dataset_id;
74
+ const portal = tc.args.portal || fallbackPortal;
75
+ if (datasetId) {
76
+ let byDataset = datasetKeyed.get(source);
77
+ if (!byDataset) {
78
+ byDataset = new Map();
79
+ datasetKeyed.set(source, byDataset);
80
+ }
81
+ if (!byDataset.has(datasetId)) {
82
+ byDataset.set(datasetId, { portalUrl: `https://${portal}`, datasetId });
83
+ }
84
+ }
85
+ }
86
+ else if (registry[source]?.aggregatePortalUrl !== undefined) {
87
+ aggregateAccessed.add(source);
88
+ }
89
+ // Unknown sources contribute no dataSources entry (their provenance is
90
+ // still visible on the PROV-O graph's tool-call activities).
91
+ }
92
+ const entries = [];
93
+ for (const [sourceId, info] of Object.entries(registry)) {
94
+ if (info.aggregatePortalUrl === undefined) {
95
+ const byDataset = datasetKeyed.get(sourceId);
96
+ if (!byDataset)
97
+ continue;
98
+ for (const { portalUrl, datasetId } of byDataset.values()) {
99
+ entries.push({
100
+ sourceId,
101
+ catalogType: info.catalogType,
102
+ portalUrl,
103
+ datasetId,
104
+ datasetUrl: `${portalUrl}/d/${datasetId}`,
105
+ accessTimestamp: now,
106
+ });
107
+ }
108
+ }
109
+ else if (aggregateAccessed.has(sourceId)) {
110
+ entries.push({
111
+ sourceId,
112
+ catalogType: info.catalogType,
113
+ portalUrl: info.aggregatePortalUrl,
114
+ accessTimestamp: now,
115
+ });
116
+ }
117
+ }
118
+ return entries;
119
+ }
@@ -0,0 +1,48 @@
1
+ import { type ProvGraph, type ProvNode } from '@typedstandards/produce-core';
2
+ import { type PlatformAgentConfig } from '../format/vocabulary.ts';
3
+ import { type CivicSourceRegistry } from '../format/sources.ts';
4
+ export type { ProvGraph, ProvNode };
5
+ /** Package-level inputs to the graph build. */
6
+ export interface ProvenanceInput {
7
+ packageId: string;
8
+ promptHash: string;
9
+ promptText?: string;
10
+ /** Inline output text. Used to derive the output hash unless
11
+ * `outputHash` is supplied explicitly. */
12
+ outputText?: string;
13
+ /** Pre-computed SHA-256 hex of the output. Supplied when the output is
14
+ * stored as a BlobRef and inline text isn't available; the ref hash
15
+ * itself is exactly this value by construction. */
16
+ outputHash?: string;
17
+ model: string;
18
+ portal: string;
19
+ }
20
+ /** Instance configuration for the graph build (config-not-constants). */
21
+ export interface ProvenanceConfig {
22
+ /** Platform-agent identity — the deployment publishing the evidence. */
23
+ platformAgent: PlatformAgentConfig;
24
+ /** Source registry supplying agent titles and MCP server URLs. */
25
+ sourceRegistry: CivicSourceRegistry;
26
+ /** Source id untagged tool spans fall back to. Default `socrata`
27
+ * (pre-source-tagging captures were Socrata-only). */
28
+ fallbackSourceId?: string;
29
+ /** Source whose agent `civic:serverUrl` the trace's skill-fetch span URL
30
+ * overrides when present (the skill is fetched from that source's MCP
31
+ * server). Default `socrata`. */
32
+ skillSourceId?: string;
33
+ /** `dcterms:description` of the model agent. Default names the demo
34
+ * deployment's model gateway. */
35
+ modelAgentDescription?: string;
36
+ }
37
+ /** Demo default: the civicaitools.org reference deployment. */
38
+ export declare const CIVICAITOOLS_PROVENANCE_CONFIG: ProvenanceConfig;
39
+ /**
40
+ * Build a W3C PROV-O JSON-LD graph from an OTel trace and package metadata.
41
+ *
42
+ * Walks the trace spans and maps them to PROV concepts:
43
+ * - LLM inference spans → prov:Activity
44
+ * - MCP tool call spans → prov:Activity
45
+ * - Prompt, skill guidance, data responses, output → prov:Entity
46
+ * - LLM model, MCP server, platform → prov:Agent
47
+ */
48
+ export declare function buildProvenanceGraph(trace: Record<string, unknown>, input: ProvenanceInput, config?: ProvenanceConfig): ProvGraph;
@@ -0,0 +1,290 @@
1
+ // Civic provenance builder (CAPTURE group) — walks an OTel trace and maps the
2
+ // analysis pipeline (LLM inference, MCP tool calls, data responses) to a
3
+ // W3C PROV-O JSON-LD graph. Relocated whole from civic-ai-tools-website
4
+ // `src/lib/evidence/provenance.ts:65–392` per the S2 brief §1, rebuilt on
5
+ // @typedstandards/produce-core's generic ProvGraph/ProvNode types and
6
+ // node/edge helpers. Every `civic:` term, urn scheme, and agent coordinate is
7
+ // imported from the format-extension group — this module WALKS, it does not
8
+ // define vocabulary (the package's internal module boundary).
9
+ //
10
+ // Config-not-constants: the platform-agent identity, the source-agent
11
+ // registry (server URLs), and the model-agent description are typed config
12
+ // inputs; `CIVICAITOOLS_PROVENANCE_CONFIG` is the demo default and
13
+ // reproduces the reference implementation's output byte-for-byte (property
14
+ // insertion order is preserved throughout — the legacy hash chain's byte
15
+ // contract).
16
+ import { makeProvGraph, makeEntityNode, makeActivityNode, makeAgentNode, provRef, provUsed, provWasGeneratedBy, provWasDerivedFrom, provWasAssociatedWith, xsdDateTime, } from '@typedstandards/produce-core';
17
+ import { hash } from "./trace.js";
18
+ import { makeCivicProvContext, civicUrn, civicModelUrn, civicSourceAgentUrn, civicPlatformUrn, CIVICAITOOLS_PLATFORM_AGENT, } from "../format/vocabulary.js";
19
+ import { CIVIC_SOURCE_REGISTRY, FALLBACK_SOURCE_ID, isDatasetKeyedSource, } from "../format/sources.js";
20
+ /** Demo default: the civicaitools.org reference deployment. */
21
+ export const CIVICAITOOLS_PROVENANCE_CONFIG = {
22
+ platformAgent: CIVICAITOOLS_PLATFORM_AGENT,
23
+ sourceRegistry: CIVIC_SOURCE_REGISTRY,
24
+ };
25
+ const DEFAULT_MODEL_AGENT_DESCRIPTION = 'Large language model via OpenRouter';
26
+ // --- Helpers ---
27
+ function getAttr(attrs, key) {
28
+ const attr = attrs.find(a => a.key === key);
29
+ return attr?.value?.stringValue ?? attr?.value?.intValue ?? undefined;
30
+ }
31
+ function nanoToIso(nano) {
32
+ const ms = Math.floor(Number(nano) / 1_000_000);
33
+ return new Date(ms).toISOString();
34
+ }
35
+ // --- Builder ---
36
+ /**
37
+ * Build a W3C PROV-O JSON-LD graph from an OTel trace and package metadata.
38
+ *
39
+ * Walks the trace spans and maps them to PROV concepts:
40
+ * - LLM inference spans → prov:Activity
41
+ * - MCP tool call spans → prov:Activity
42
+ * - Prompt, skill guidance, data responses, output → prov:Entity
43
+ * - LLM model, MCP server, platform → prov:Agent
44
+ */
45
+ export function buildProvenanceGraph(trace, input, config = CIVICAITOOLS_PROVENANCE_CONFIG) {
46
+ const otel = trace;
47
+ const spans = otel?.resourceSpans?.[0]?.scopeSpans?.[0]?.spans || [];
48
+ const { packageId, promptHash, promptText, outputText, model, portal } = input;
49
+ const outputHash = input.outputHash ?? hash(outputText ?? '');
50
+ const registry = config.sourceRegistry;
51
+ const fallbackSourceId = config.fallbackSourceId ?? FALLBACK_SOURCE_ID;
52
+ const skillSourceId = config.skillSourceId ?? FALLBACK_SOURCE_ID;
53
+ const modelDescription = config.modelAgentDescription ?? DEFAULT_MODEL_AGENT_DESCRIPTION;
54
+ const graph = [];
55
+ // --- Entities ---
56
+ // User prompt
57
+ graph.push(makeEntityNode(civicUrn(packageId, 'prompt', promptHash), {
58
+ 'civic:contentHash': `sha256:${promptHash}`,
59
+ 'dcterms:description': 'User query prompt',
60
+ ...(promptText ? { 'prov:value': promptText } : {}),
61
+ }));
62
+ // Skill guidance (from skill_fetch span). The skill is a composition of
63
+ // per-source guidance, so the description is source-neutral. Per-source
64
+ // tool agents are emitted from the tool spans below based on which sources
65
+ // were actually invoked.
66
+ const skillSpan = spans.find(s => s.name === 'skill_fetch');
67
+ const skillHash = skillSpan ? getAttr(skillSpan.attributes, 'skill.text_hash') : undefined;
68
+ const skillServerUrl = skillSpan ? getAttr(skillSpan.attributes, 'skill.mcp_server_url') : undefined;
69
+ if (skillHash) {
70
+ graph.push(makeEntityNode(civicUrn(packageId, 'skill', skillHash), {
71
+ 'civic:contentHash': `sha256:${skillHash}`,
72
+ 'dcterms:description': 'Composed MCP skill guidance (system prompt)',
73
+ }, ['prov:Plan']));
74
+ }
75
+ // Final output
76
+ graph.push(makeEntityNode(civicUrn(packageId, 'output', outputHash), {
77
+ 'civic:contentHash': `sha256:${outputHash}`,
78
+ 'dcterms:description': 'AI-generated analysis output',
79
+ }));
80
+ // --- Agents ---
81
+ // LLM model
82
+ const modelUrn = civicModelUrn(model);
83
+ graph.push(makeAgentNode(modelUrn, {
84
+ 'dcterms:title': model,
85
+ 'dcterms:description': modelDescription,
86
+ }, ['prov:SoftwareAgent']));
87
+ // MCP server agents — one per distinct data source that appears in the
88
+ // trace. Tool spans carry `mcp.source`; evidence records captured before
89
+ // source tagging have no source attribute, so they fall back to the
90
+ // configured fallback source (socrata — the only source at the time) for
91
+ // backwards compatibility. Agent coordinates come from the source registry;
92
+ // the skill-fetch span URL overrides the skill source's server URL.
93
+ const toolSpans = spans.filter(s => s.name === 'mcp_tool_call');
94
+ const sourceAgentMap = {};
95
+ for (const [id, info] of Object.entries(registry)) {
96
+ sourceAgentMap[id] = {
97
+ urn: civicSourceAgentUrn(id),
98
+ title: info.agentTitle,
99
+ serverUrl: id === skillSourceId ? skillServerUrl || info.serverUrl : info.serverUrl,
100
+ };
101
+ }
102
+ const sourcesInTrace = new Set();
103
+ for (const span of toolSpans) {
104
+ const source = getAttr(span.attributes, 'mcp.source') || fallbackSourceId;
105
+ sourcesInTrace.add(source);
106
+ }
107
+ for (const sourceId of sourcesInTrace) {
108
+ const meta = sourceAgentMap[sourceId] ?? {
109
+ urn: civicSourceAgentUrn(encodeURIComponent(sourceId)),
110
+ title: `${sourceId} MCP Server`,
111
+ serverUrl: sourceId,
112
+ };
113
+ graph.push(makeAgentNode(meta.urn, {
114
+ 'dcterms:title': meta.title,
115
+ 'civic:serverUrl': meta.serverUrl,
116
+ 'civic:sourceId': sourceId,
117
+ }, ['prov:SoftwareAgent']));
118
+ }
119
+ // Resolve a source id to its agent URN, falling back to the configured
120
+ // fallback source for records that never tagged the span. (Reference
121
+ // behavior preserved: unknown ids resolve to the fallback source's agent.)
122
+ function agentUrnForSource(sourceId) {
123
+ const id = sourceId || fallbackSourceId;
124
+ return (sourceAgentMap[id]?.urn
125
+ ?? sourceAgentMap[fallbackSourceId]?.urn
126
+ ?? civicSourceAgentUrn(encodeURIComponent(id)));
127
+ }
128
+ // Platform
129
+ graph.push(makeAgentNode(civicPlatformUrn(config.platformAgent.id), {
130
+ 'dcterms:title': config.platformAgent.title,
131
+ 'civic:url': config.platformAgent.url,
132
+ }, ['prov:SoftwareAgent']));
133
+ // --- Activities ---
134
+ // LLM inference spans
135
+ const inferenceSpans = spans.filter(s => s.name === 'llm_inference');
136
+ const inferenceUrns = [];
137
+ for (const span of inferenceSpans) {
138
+ const spanUrn = civicUrn(packageId, 'inference', span.spanId);
139
+ inferenceUrns.push(spanUrn);
140
+ const used = [provRef(civicUrn(packageId, 'prompt', promptHash))];
141
+ if (skillHash) {
142
+ used.push(provRef(civicUrn(packageId, 'skill', skillHash)));
143
+ }
144
+ const promptTokens = getAttr(span.attributes, 'gen_ai.response.prompt_tokens');
145
+ const completionTokens = getAttr(span.attributes, 'gen_ai.response.completion_tokens');
146
+ graph.push(makeActivityNode(spanUrn, {
147
+ 'dcterms:description': `LLM inference call (iteration ${getAttr(span.attributes, 'gen_ai.inference_index') || '0'})`,
148
+ ...provWasAssociatedWith(modelUrn),
149
+ 'prov:used': used,
150
+ ...(span.startTimeUnixNano
151
+ ? { 'prov:startedAtTime': xsdDateTime(nanoToIso(span.startTimeUnixNano)) }
152
+ : {}),
153
+ ...(span.endTimeUnixNano
154
+ ? { 'prov:endedAtTime': xsdDateTime(nanoToIso(span.endTimeUnixNano)) }
155
+ : {}),
156
+ ...(promptTokens ? { 'civic:promptTokens': Number(promptTokens) } : {}),
157
+ ...(completionTokens ? { 'civic:completionTokens': Number(completionTokens) } : {}),
158
+ }));
159
+ }
160
+ // MCP tool call spans
161
+ const dataResponseUrns = [];
162
+ for (const span of toolSpans) {
163
+ const toolCallUrn = civicUrn(packageId, 'tool-call', span.spanId);
164
+ const argsStr = getAttr(span.attributes, 'tool.arguments') || '{}';
165
+ const queryHash = hash(argsStr);
166
+ const responseHash = getAttr(span.attributes, 'tool.response_hash');
167
+ const toolName = getAttr(span.attributes, 'tool.name') || 'get_data';
168
+ const opType = getAttr(span.attributes, 'tool.operation_type') || 'unknown';
169
+ const datasetId = getAttr(span.attributes, 'tool.dataset_id');
170
+ const portalDomain = getAttr(span.attributes, 'tool.portal_domain') || portal;
171
+ const toolSource = getAttr(span.attributes, 'mcp.source') || fallbackSourceId;
172
+ const toolAgentUrn = agentUrnForSource(toolSource);
173
+ const toolSourceDatasetKeyed = isDatasetKeyedSource(toolSource, registry);
174
+ // Query entity (tool arguments). Find the most recent inference span
175
+ // before this tool call to link as generator.
176
+ const toolStart = Number(span.startTimeUnixNano);
177
+ const precedingInference = inferenceSpans
178
+ .filter(is => Number(is.endTimeUnixNano || '0') <= toolStart)
179
+ .sort((a, b) => Number(b.endTimeUnixNano || '0') - Number(a.endTimeUnixNano || '0'))[0];
180
+ const queryUrn = civicUrn(packageId, 'query', queryHash);
181
+ graph.push(makeEntityNode(queryUrn, {
182
+ 'civic:contentHash': `sha256:${queryHash}`,
183
+ 'civic:toolName': toolName,
184
+ 'civic:operationType': opType,
185
+ 'dcterms:description': `MCP tool arguments (${opType})`,
186
+ ...(precedingInference
187
+ ? provWasGeneratedBy(civicUrn(packageId, 'inference', precedingInference.spanId))
188
+ : {}),
189
+ }));
190
+ // Data response entity
191
+ if (responseHash) {
192
+ const dataUrn = civicUrn(packageId, 'data', responseHash);
193
+ dataResponseUrns.push(dataUrn);
194
+ // Description varies by source — dataset-keyed sources (Socrata) have
195
+ // a portal domain; aggregate/unknown sources are described by their
196
+ // agent title.
197
+ const description = toolSourceDatasetKeyed
198
+ ? `Data response from ${portalDomain}`
199
+ : `Data response from ${sourceAgentMap[toolSource]?.title || toolSource}`;
200
+ const responseRows = getAttr(span.attributes, 'tool.response_rows');
201
+ graph.push(makeEntityNode(dataUrn, {
202
+ 'civic:contentHash': `sha256:${responseHash}`,
203
+ 'dcterms:description': description,
204
+ 'civic:sourceId': toolSource,
205
+ ...provWasGeneratedBy(toolCallUrn),
206
+ // Croissant 1.1 placeholder — only meaningful for dataset-keyed
207
+ // sources today.
208
+ ...(toolSourceDatasetKeyed && datasetId
209
+ ? {
210
+ 'civic:datasetId': datasetId,
211
+ 'civic:portalDomain': portalDomain,
212
+ 'civic:datasetUrl': `https://${portalDomain}/d/${datasetId}`,
213
+ 'civic:croissantMetadataUrl': null, // hook for future Croissant integration
214
+ }
215
+ : {}),
216
+ ...(responseRows ? { 'civic:responseRows': Number(responseRows) } : {}),
217
+ }));
218
+ }
219
+ // Tool call activity — associated with the MCP source agent that handled
220
+ // the call. In multi-source analyses each call may target a different
221
+ // agent (e.g. socrata for `get_data`, data-commons for `get_observations`).
222
+ const durationMs = getAttr(span.attributes, 'tool.duration_ms');
223
+ graph.push(makeActivityNode(toolCallUrn, {
224
+ 'dcterms:description': `MCP tool call: ${toolName} (${opType})`,
225
+ 'civic:sourceId': toolSource,
226
+ ...provUsed([queryUrn]),
227
+ ...provWasAssociatedWith(toolAgentUrn),
228
+ ...(span.startTimeUnixNano
229
+ ? { 'prov:startedAtTime': xsdDateTime(nanoToIso(span.startTimeUnixNano)) }
230
+ : {}),
231
+ ...(span.endTimeUnixNano
232
+ ? { 'prov:endedAtTime': xsdDateTime(nanoToIso(span.endTimeUnixNano)) }
233
+ : {}),
234
+ ...(durationMs ? { 'civic:durationMs': Number(durationMs) } : {}),
235
+ }));
236
+ }
237
+ // --- Final output relationships ---
238
+ // Output wasGeneratedBy the last inference or synthesis span
239
+ const synthesisSpan = spans.find(s => s.name === 'synthesis');
240
+ const lastInference = inferenceSpans.length > 0
241
+ ? inferenceSpans[inferenceSpans.length - 1]
242
+ : undefined;
243
+ const generatorSpan = synthesisSpan || lastInference;
244
+ if (generatorSpan) {
245
+ const generatorUrn = synthesisSpan
246
+ ? civicUrn(packageId, 'synthesis', synthesisSpan.spanId)
247
+ : civicUrn(packageId, 'inference', generatorSpan.spanId);
248
+ // Add synthesis activity if it exists
249
+ if (synthesisSpan) {
250
+ graph.push(makeActivityNode(generatorUrn, {
251
+ 'dcterms:description': 'Output synthesis',
252
+ ...provWasAssociatedWith(modelUrn),
253
+ ...(synthesisSpan.startTimeUnixNano
254
+ ? { 'prov:startedAtTime': xsdDateTime(nanoToIso(synthesisSpan.startTimeUnixNano)) }
255
+ : {}),
256
+ ...(synthesisSpan.endTimeUnixNano
257
+ ? { 'prov:endedAtTime': xsdDateTime(nanoToIso(synthesisSpan.endTimeUnixNano)) }
258
+ : {}),
259
+ }));
260
+ }
261
+ // Output wasGeneratedBy
262
+ graph.push(makeEntityNode(civicUrn(packageId, 'output', outputHash), {
263
+ ...provWasGeneratedBy(generatorUrn),
264
+ ...(dataResponseUrns.length > 0
265
+ ? provWasDerivedFrom(dataResponseUrns)
266
+ : {}),
267
+ }));
268
+ }
269
+ // Add data responses to inference used list (inference used data to synthesize)
270
+ if (dataResponseUrns.length > 0 && inferenceUrns.length > 0) {
271
+ const lastInferenceUrn = inferenceUrns[inferenceUrns.length - 1];
272
+ const existing = graph.find(n => n['@id'] === lastInferenceUrn);
273
+ if (existing && Array.isArray(existing['prov:used'])) {
274
+ for (const dUrn of dataResponseUrns) {
275
+ existing['prov:used'].push(provRef(dUrn));
276
+ }
277
+ }
278
+ }
279
+ // --- Skill hadPlan relationship ---
280
+ if (skillHash && inferenceUrns.length > 0) {
281
+ graph.push(makeActivityNode(inferenceUrns[0], {
282
+ 'prov:qualifiedAssociation': {
283
+ '@type': 'prov:Association',
284
+ 'prov:agent': provRef(modelUrn),
285
+ 'prov:hadPlan': provRef(civicUrn(packageId, 'skill', skillHash)),
286
+ },
287
+ }));
288
+ }
289
+ return makeProvGraph(makeCivicProvContext(), graph);
290
+ }