@kontourai/lookout 0.3.1 → 0.3.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +87 -1
- package/dist/src/drift-emission.js +2 -0
- package/dist/src/index.d.ts +2 -0
- package/dist/src/index.js +1 -0
- package/dist/src/proposal-diff.d.ts +4 -0
- package/dist/src/proposal-diff.js +49 -4
- package/dist/src/semantic-review-projection.d.ts +13 -0
- package/dist/src/semantic-review-projection.js +68 -0
- package/dist/src/semantic-review-types.d.ts +95 -0
- package/dist/src/semantic-review-types.js +1 -0
- package/dist/src/semantic-review-work.d.ts +5 -0
- package/dist/src/semantic-review-work.js +36 -0
- package/package.json +7 -5
package/README.md
CHANGED
|
@@ -10,6 +10,52 @@ package. It does **not** implement acquisition or extraction, author trust-layer
|
|
|
10
10
|
records, project to Surface, review claims, notify, crawl, or schedule.
|
|
11
11
|
Operational failures are returned as typed results.
|
|
12
12
|
|
|
13
|
+
## Quick start
|
|
14
|
+
|
|
15
|
+
Register one source in a JSON file (defaults to `<cwd>/lookout.sources.json`;
|
|
16
|
+
see [Registry](#registry) for every field):
|
|
17
|
+
|
|
18
|
+
```json
|
|
19
|
+
{
|
|
20
|
+
"version": 1,
|
|
21
|
+
"sources": [
|
|
22
|
+
{
|
|
23
|
+
"id": "example-home",
|
|
24
|
+
"kind": "web-page",
|
|
25
|
+
"url": "https://example.com/",
|
|
26
|
+
"targetSchema": [{ "path": "title", "type": "string", "required": true }],
|
|
27
|
+
"cadenceHint": "daily",
|
|
28
|
+
"renderPolicy": "never"
|
|
29
|
+
}
|
|
30
|
+
]
|
|
31
|
+
}
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
Then check it with the CLI:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
lookout check example-home --registry ./lookout.sources.json
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
The first run has nothing to compare against, so it persists a baseline
|
|
41
|
+
snapshot and reports `changed`/`initial`:
|
|
42
|
+
|
|
43
|
+
```json
|
|
44
|
+
{"sourceId":"example-home","sourceUrl":"https://example.com/","checkedAt":"2026-07-20T14:12:48.248Z","warnings":[],"kind":"changed","priorSnapshotRef":null,"currentSnapshotRef":"forage-snapshot:example-home?...","changeBasis":"initial"}
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
Run the same command again and, if the source hasn't changed, Forage's
|
|
48
|
+
conditional request returns `304` with zero body transfer — no re-download,
|
|
49
|
+
just confirmation:
|
|
50
|
+
|
|
51
|
+
```json
|
|
52
|
+
{"sourceId":"example-home","sourceUrl":"https://example.com/","checkedAt":"2026-07-20T14:12:53.089Z","warnings":[],"kind":"unchanged-304","snapshotRef":"forage-snapshot:example-home?..."}
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Exit code is `0` for both runs — a per-source result, even a fetch failure, is
|
|
56
|
+
not a CLI failure. See [CLI](#cli) for the full exit-code contract and
|
|
57
|
+
[Check results](#check-results) for what each `kind` means.
|
|
58
|
+
|
|
13
59
|
## Why it's different
|
|
14
60
|
|
|
15
61
|
The naive way to answer "did it change?" is to re-crawl the source and diff the
|
|
@@ -22,7 +68,7 @@ as "no change"). Lookout is the opposite on all three:
|
|
|
22
68
|
| Cost | full re-download every check | **conditional `304`** — often no download at all |
|
|
23
69
|
| Signal | raw bytes (ads/timestamps = "changed") | **proposal-identity diff** — "a *new entity appeared*" vs "a byte moved" |
|
|
24
70
|
| Honesty | a crash or false-`304` silently reads as "unchanged" | **typed results, never throws**; Forage scopes validators to the exact prior resource and Lookout verifies the returned capture identity |
|
|
25
|
-
| Output | your problem to shape | **neutral, typed drift** — events already Hachure-evidence-shaped, ready for a consumer to lift into a trust bundle |
|
|
71
|
+
| Output | your problem to shape | **neutral, typed drift** — events already [Hachure](https://github.com/hachure-org/spec)-evidence-shaped (the open, product-neutral trust-record spec Surface's TrustBundle implements), ready for a consumer to lift into a trust bundle |
|
|
26
72
|
|
|
27
73
|
So the point isn't "diffing" — it's *cheap + honest + semantic + review-ready*
|
|
28
74
|
change detection, so a periodic re-check surfaces **only the real delta** (this
|
|
@@ -288,6 +334,46 @@ The existing `createCheckRunner` and `createDriftEmitter` entrypoints remain
|
|
|
288
334
|
available. Use `createDriftEmitter` when the caller wants deterministic
|
|
289
335
|
proposal-diff events with its own identity callbacks.
|
|
290
336
|
|
|
337
|
+
## Route semantic transitions to review
|
|
338
|
+
|
|
339
|
+
`buildSemanticReviewWork` turns one genuine prior/current proposal transition
|
|
340
|
+
into structurally Survey-compatible `ReviewItem` resources without adding a
|
|
341
|
+
runtime dependency on a review product. The caller supplies entity/field
|
|
342
|
+
identity callbacks and claim meaning; Lookout supplies deterministic transition
|
|
343
|
+
and item identities.
|
|
344
|
+
|
|
345
|
+
```ts
|
|
346
|
+
const work = buildSemanticReviewWork({
|
|
347
|
+
prior,
|
|
348
|
+
current,
|
|
349
|
+
observationIdentity: { prior: priorId, current: currentId },
|
|
350
|
+
schema: source.targetSchema,
|
|
351
|
+
selectEntities,
|
|
352
|
+
entityIdentity,
|
|
353
|
+
proposalsFor,
|
|
354
|
+
fieldIdentity,
|
|
355
|
+
claimTarget(change) {
|
|
356
|
+
return describeClaim(change);
|
|
357
|
+
},
|
|
358
|
+
});
|
|
359
|
+
```
|
|
360
|
+
|
|
361
|
+
Added, removed, moved, provenance-changed, and value-changed proposals become distinct work items.
|
|
362
|
+
New coverage or exact-provenance gaps are also reviewable. Each available side
|
|
363
|
+
retains its exact snapshot reference, observation time, locator, excerpt, and
|
|
364
|
+
extractor. An absent side is explicit and anchored to the corresponding
|
|
365
|
+
observation snapshot, so a removal remains reviewable even when the new
|
|
366
|
+
extraction emits no value. Identical proposal sets produce no semantic work;
|
|
367
|
+
replaying the same pair of observation identities produces byte-identical
|
|
368
|
+
resources.
|
|
369
|
+
|
|
370
|
+
Review resolution, escalation, persistence, and authority policy stay with the
|
|
371
|
+
consumer. Evidence excerpts and caller-supplied claim targets can contain
|
|
372
|
+
sensitive data; apply retention, redaction, and access controls before storing
|
|
373
|
+
or forwarding review work. Only compact typed identities cross the producer
|
|
374
|
+
metadata boundary; no provider messages, native diagnostics, or raw responses
|
|
375
|
+
are copied.
|
|
376
|
+
|
|
291
377
|
## Non-goals
|
|
292
378
|
|
|
293
379
|
Acquisition and extraction implementations, Surface projection, notifications,
|
|
@@ -16,6 +16,8 @@ function normalizeDiff(value) {
|
|
|
16
16
|
removedProposalOccurrences: sorted(value.facts.removedProposalOccurrences),
|
|
17
17
|
provenanceChanges: sorted(value.facts.provenanceChanges),
|
|
18
18
|
removedEntities: [...value.facts.removedEntities].sort(),
|
|
19
|
+
addedProposalEvidence: sorted(value.facts.addedProposalEvidence ?? []),
|
|
20
|
+
removedProposalEvidence: sorted(value.facts.removedProposalEvidence ?? []),
|
|
19
21
|
},
|
|
20
22
|
};
|
|
21
23
|
}
|
package/dist/src/index.d.ts
CHANGED
|
@@ -24,3 +24,5 @@ export { checkSchemaCoverage } from "./coverage.js";
|
|
|
24
24
|
export type { SchemaCoverageGap, SchemaCoverageResult } from "./coverage.js";
|
|
25
25
|
export { createObserveExtractDiff } from "./observe-extract-diff.js";
|
|
26
26
|
export type { ObserveExtractAcquisition, ObserveExtractAttempt, ObserveExtractDiff, ObserveExtractDiffOptions, ObserveExtractError, ObserveExtractErrorKind, ObserveExtractExtraction, ObserveExtractExtractionInput, ObserveExtractObservation, ObserveExtractObservationIdentity, ObserveExtractOutcome, ObserveExtractProviderFailure, ObserveExtractRecorder, ObserveExtractResult, ObserveExtractSource, ObserveExtractSourceSnapshot, } from "./observe-extract-diff.js";
|
|
27
|
+
export { buildSemanticReviewWork, semanticReviewApiVersion } from "./semantic-review-work.js";
|
|
28
|
+
export type { BuildSemanticReviewWorkInput, SemanticClaimTarget, SemanticObservationIdentity, SemanticReviewCandidate, SemanticReviewChange, SemanticReviewItem, SemanticReviewKind, SemanticReviewWork, } from "./semantic-review-work.js";
|
package/dist/src/index.js
CHANGED
|
@@ -10,3 +10,4 @@ export { createObservationStore } from "./observation-store.js";
|
|
|
10
10
|
export { createDriftEmitter } from "./drift-emission.js";
|
|
11
11
|
export { checkSchemaCoverage } from "./coverage.js";
|
|
12
12
|
export { createObserveExtractDiff } from "./observe-extract-diff.js";
|
|
13
|
+
export { buildSemanticReviewWork, semanticReviewApiVersion } from "./semantic-review-work.js";
|
|
@@ -62,6 +62,10 @@ export interface ProposalSetFacts {
|
|
|
62
62
|
readonly removedProposalOccurrences: readonly ExtractionProposal[];
|
|
63
63
|
readonly provenanceChanges: readonly ProvenanceChangeFact[];
|
|
64
64
|
readonly removedEntities: readonly string[];
|
|
65
|
+
/** Exact observation-anchored evidence for every added proposal occurrence. */
|
|
66
|
+
readonly addedProposalEvidence?: readonly ProposalEvidence[];
|
|
67
|
+
/** Exact observation-anchored evidence for every removed proposal occurrence. */
|
|
68
|
+
readonly removedProposalEvidence?: readonly ProposalEvidence[];
|
|
65
69
|
}
|
|
66
70
|
export interface ProposalSetDiff {
|
|
67
71
|
readonly events: readonly ProposalDiffEvent[];
|
|
@@ -73,6 +73,26 @@ function changeKind(prior, current) {
|
|
|
73
73
|
const sameCategory = (prior === null ? "null" : typeof prior) === (current === null ? "null" : typeof current);
|
|
74
74
|
return { ok: true, value: sameCategory ? "value-updated" : "value-replaced" };
|
|
75
75
|
}
|
|
76
|
+
function semanticFieldOrder(items, fieldIdentity) {
|
|
77
|
+
const groups = new Map();
|
|
78
|
+
for (const [index, item] of items.entries()) {
|
|
79
|
+
const fieldKey = identity("fieldIdentity", () => fieldIdentity(item));
|
|
80
|
+
if (!fieldKey.ok)
|
|
81
|
+
return fieldKey;
|
|
82
|
+
let encoded;
|
|
83
|
+
try {
|
|
84
|
+
encoded = canonicalValueKey({ value: item.proposal.candidateValue, provenance: item.proposal.provenance });
|
|
85
|
+
}
|
|
86
|
+
catch (cause) {
|
|
87
|
+
return { ok: false, error: { kind: "unsupported-value", message: "Proposal semantic content could not be inspected", path: "$", cause } };
|
|
88
|
+
}
|
|
89
|
+
if (!encoded.ok)
|
|
90
|
+
return encoded;
|
|
91
|
+
groups.set(fieldKey.value, [...(groups.get(fieldKey.value) ?? []), { item, key: encoded.key, index }]);
|
|
92
|
+
}
|
|
93
|
+
const ordered = [...groups.values()].flatMap((group) => group.sort((left, right) => left.key.localeCompare(right.key) || left.index - right.index));
|
|
94
|
+
return { ok: true, value: ordered.map(({ item }) => item) };
|
|
95
|
+
}
|
|
76
96
|
export function diffProposalSets(input) {
|
|
77
97
|
if (input.prior.sourceId !== input.current.sourceId) {
|
|
78
98
|
return {
|
|
@@ -97,6 +117,8 @@ export function diffProposalSets(input) {
|
|
|
97
117
|
const removedProposalOccurrences = [];
|
|
98
118
|
const provenanceChanges = [];
|
|
99
119
|
const removedEntities = [];
|
|
120
|
+
const addedProposalEvidence = [];
|
|
121
|
+
const removedProposalEvidence = [];
|
|
100
122
|
for (const pair of entities.value.retained) {
|
|
101
123
|
const entityKeyResult = identity("entityIdentity", () => input.entityIdentity(pair.prior));
|
|
102
124
|
if (!entityKeyResult.ok)
|
|
@@ -116,9 +138,25 @@ export function diffProposalSets(input) {
|
|
|
116
138
|
retainedProposalOccurrences.push(...occurrences.value.retained);
|
|
117
139
|
addedProposalOccurrences.push(...occurrences.value.additions);
|
|
118
140
|
removedProposalOccurrences.push(...occurrences.value.removals);
|
|
119
|
-
const
|
|
120
|
-
|
|
121
|
-
|
|
141
|
+
for (const proposal of occurrences.value.additions) {
|
|
142
|
+
const fieldKey = identity("fieldIdentity", () => input.fieldIdentity(pair.current, proposal));
|
|
143
|
+
if (!fieldKey.ok)
|
|
144
|
+
return fieldKey;
|
|
145
|
+
addedProposalEvidence.push(evidence(input.current, entityKey, fieldKey.value, proposal));
|
|
146
|
+
}
|
|
147
|
+
for (const proposal of occurrences.value.removals) {
|
|
148
|
+
const fieldKey = identity("fieldIdentity", () => input.fieldIdentity(pair.prior, proposal));
|
|
149
|
+
if (!fieldKey.ok)
|
|
150
|
+
return fieldKey;
|
|
151
|
+
removedProposalEvidence.push(evidence(input.prior, entityKey, fieldKey.value, proposal));
|
|
152
|
+
}
|
|
153
|
+
const priorFields = semanticFieldOrder(priorProposals.value.map((proposal) => ({ entity: pair.prior, proposal })), ({ entity, proposal }) => input.fieldIdentity(entity, proposal));
|
|
154
|
+
if (!priorFields.ok)
|
|
155
|
+
return priorFields;
|
|
156
|
+
const currentFields = semanticFieldOrder(currentProposals.value.map((proposal) => ({ entity: pair.current, proposal })), ({ entity, proposal }) => input.fieldIdentity(entity, proposal));
|
|
157
|
+
if (!currentFields.ok)
|
|
158
|
+
return currentFields;
|
|
159
|
+
const fields = diffKeyedMultiset(priorFields.value, currentFields.value, {
|
|
122
160
|
identity: ({ entity, proposal }) => input.fieldIdentity(entity, proposal),
|
|
123
161
|
});
|
|
124
162
|
if (!fields.ok)
|
|
@@ -184,6 +222,7 @@ export function diffProposalSets(input) {
|
|
|
184
222
|
if (!fieldKey.ok)
|
|
185
223
|
return fieldKey;
|
|
186
224
|
current.push(evidence(input.current, entityKeyResult.value, fieldKey.value, proposal));
|
|
225
|
+
addedProposalEvidence.push(evidence(input.current, entityKeyResult.value, fieldKey.value, proposal));
|
|
187
226
|
addedProposalOccurrences.push(proposal);
|
|
188
227
|
}
|
|
189
228
|
events.push({ kind: "new-entity-appeared", entityKey: entityKeyResult.value, current });
|
|
@@ -197,12 +236,18 @@ export function diffProposalSets(input) {
|
|
|
197
236
|
if (!proposals.ok)
|
|
198
237
|
return proposals;
|
|
199
238
|
removedProposalOccurrences.push(...proposals.value);
|
|
239
|
+
for (const proposal of proposals.value) {
|
|
240
|
+
const fieldKey = identity("fieldIdentity", () => input.fieldIdentity(entity, proposal));
|
|
241
|
+
if (!fieldKey.ok)
|
|
242
|
+
return fieldKey;
|
|
243
|
+
removedProposalEvidence.push(evidence(input.prior, entityKeyResult.value, fieldKey.value, proposal));
|
|
244
|
+
}
|
|
200
245
|
}
|
|
201
246
|
return {
|
|
202
247
|
ok: true,
|
|
203
248
|
value: {
|
|
204
249
|
events,
|
|
205
|
-
facts: { retainedProposalOccurrences, addedProposalOccurrences, removedProposalOccurrences, provenanceChanges, removedEntities },
|
|
250
|
+
facts: { retainedProposalOccurrences, addedProposalOccurrences, removedProposalOccurrences, provenanceChanges, removedEntities, addedProposalEvidence, removedProposalEvidence },
|
|
206
251
|
},
|
|
207
252
|
};
|
|
208
253
|
}
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
import type { TargetFieldSchema } from "@kontourai/traverse";
|
|
2
|
+
import { type DiffResult } from "./canonical-value.js";
|
|
3
|
+
import type { ProposalSetDiff, ProposalSetObservation } from "./proposal-diff.js";
|
|
4
|
+
import { type SemanticClaimTarget, type SemanticObservationIdentity, type SemanticReviewChange, type SemanticReviewItem } from "./semantic-review-types.js";
|
|
5
|
+
export declare function reviewKey(value: unknown): DiffResult<string>;
|
|
6
|
+
export declare function sanitizeError<T>(result: DiffResult<T>): DiffResult<T>;
|
|
7
|
+
export declare function validateObservationIdentity(identity: SemanticObservationIdentity): DiffResult<SemanticObservationIdentity>;
|
|
8
|
+
export declare function collectSemanticChanges(input: {
|
|
9
|
+
prior: ProposalSetObservation;
|
|
10
|
+
current: ProposalSetObservation;
|
|
11
|
+
schema?: readonly TargetFieldSchema[];
|
|
12
|
+
}, diff: ProposalSetDiff): SemanticReviewChange[];
|
|
13
|
+
export declare function projectReviewItem(change: SemanticReviewChange, occurrence: number, transitionId: string, identities: SemanticObservationIdentity, prior: ProposalSetObservation, current: ProposalSetObservation, target: SemanticClaimTarget): DiffResult<SemanticReviewItem>;
|
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
import { createHash } from "node:crypto";
|
|
2
|
+
import { canonicalValueKey } from "./canonical-value.js";
|
|
3
|
+
import { checkSchemaCoverage } from "./coverage.js";
|
|
4
|
+
import { semanticReviewApiVersion } from "./semantic-review-types.js";
|
|
5
|
+
export function reviewKey(value) {
|
|
6
|
+
const canonical = canonicalValueKey(value);
|
|
7
|
+
return canonical.ok ? { ok: true, value: createHash("sha256").update(canonical.key).digest("hex").slice(0, 24) } : sanitizeError(canonical);
|
|
8
|
+
}
|
|
9
|
+
export function sanitizeError(result) {
|
|
10
|
+
if (result.ok)
|
|
11
|
+
return result;
|
|
12
|
+
return { ok: false, error: { kind: result.error.kind, message: "Semantic review derivation failed" } };
|
|
13
|
+
}
|
|
14
|
+
export function validateObservationIdentity(identity) {
|
|
15
|
+
for (const [name, value] of Object.entries(identity))
|
|
16
|
+
if (typeof value !== "string" || value.length === 0 || /authorization\s*[:=]|bearer\s+|(?:token|secret|password|api[-_]?key)=/i.test(value)) {
|
|
17
|
+
return { ok: false, error: { kind: "unsupported-value", message: `${name} observation identity is invalid`, path: `$.observationIdentity.${name}` } };
|
|
18
|
+
}
|
|
19
|
+
return { ok: true, value: identity };
|
|
20
|
+
}
|
|
21
|
+
const incomplete = (evidence) => evidence.provenance.locator.length === 0 || evidence.provenance.excerpt.length === 0;
|
|
22
|
+
const sameEvidence = (left, right) => left.entityKey === right.entityKey && left.fieldKey === right.fieldKey && left.snapshotRef === right.snapshotRef && left.provenance.locator === right.provenance.locator && left.provenance.excerpt === right.provenance.excerpt;
|
|
23
|
+
const addedKind = (current) => incomplete(current) ? "provenance-gap" : "proposal-added";
|
|
24
|
+
export function collectSemanticChanges(input, diff) {
|
|
25
|
+
const changes = [];
|
|
26
|
+
for (const event of diff.events) {
|
|
27
|
+
if (event.kind === "new-entity-appeared")
|
|
28
|
+
for (const current of event.current)
|
|
29
|
+
changes.push({ kind: addedKind(current), fieldPath: current.fieldPath, entityKey: current.entityKey, current });
|
|
30
|
+
else if (event.prior && event.current)
|
|
31
|
+
changes.push({ kind: "proposal-value-changed", fieldPath: event.current.fieldPath, entityKey: event.entityKey, prior: event.prior, current: event.current });
|
|
32
|
+
else if (event.current)
|
|
33
|
+
changes.push({ kind: addedKind(event.current), fieldPath: event.current.fieldPath, entityKey: event.entityKey, current: event.current });
|
|
34
|
+
else if (event.prior)
|
|
35
|
+
changes.push({ kind: "proposal-removed", fieldPath: event.prior.fieldPath, entityKey: event.entityKey, prior: event.prior });
|
|
36
|
+
}
|
|
37
|
+
const representedPrior = changes.flatMap((change) => change.prior ? [change.prior] : []);
|
|
38
|
+
const representedCurrent = changes.flatMap((change) => change.current ? [change.current] : []);
|
|
39
|
+
for (const prior of diff.facts.removedProposalEvidence ?? [])
|
|
40
|
+
if (!diff.facts.provenanceChanges.some((item) => sameEvidence(item.prior, prior)) && !representedPrior.some((item) => sameEvidence(item, prior)))
|
|
41
|
+
changes.push({ kind: "proposal-removed", fieldPath: prior.fieldPath, entityKey: prior.entityKey, prior });
|
|
42
|
+
for (const current of diff.facts.addedProposalEvidence ?? [])
|
|
43
|
+
if (!diff.facts.provenanceChanges.some((item) => sameEvidence(item.current, current)) && !representedCurrent.some((item) => sameEvidence(item, current)))
|
|
44
|
+
changes.push({ kind: addedKind(current), fieldPath: current.fieldPath, entityKey: current.entityKey, current });
|
|
45
|
+
for (const change of diff.facts.provenanceChanges) {
|
|
46
|
+
const kind = incomplete(change.current) && !incomplete(change.prior) ? "provenance-gap" : change.prior.provenance.locator !== change.current.provenance.locator ? "proposal-moved" : "proposal-provenance-changed";
|
|
47
|
+
changes.push({ kind, fieldPath: change.current.fieldPath, entityKey: change.entityKey, prior: change.prior, current: change.current });
|
|
48
|
+
}
|
|
49
|
+
if (input.schema) {
|
|
50
|
+
const priorGaps = new Set(checkSchemaCoverage(input.schema, input.prior.proposals).gaps.map((gap) => gap.fieldPath));
|
|
51
|
+
for (const gap of checkSchemaCoverage(input.schema, input.current.proposals).gaps)
|
|
52
|
+
if (!priorGaps.has(gap.fieldPath))
|
|
53
|
+
changes.push({ kind: "coverage-gap", fieldPath: gap.fieldPath, entityKey: gap.fieldPath, gap });
|
|
54
|
+
}
|
|
55
|
+
return changes;
|
|
56
|
+
}
|
|
57
|
+
function candidate(id, role, observationId, evidence, fallback, target, fieldPath) {
|
|
58
|
+
const present = evidence !== undefined;
|
|
59
|
+
return { id: `${id}.${role}`, role, value: present ? evidence.value : null, ...(present ? { confidence: evidence.confidence } : {}), source: { sourceRef: evidence?.snapshotRef ?? fallback.snapshotRef, sourceId: evidence?.sourceId ?? fallback.sourceId, observedAt: evidence?.observedAt ?? fallback.observedAt, locatorScheme: "text-span" }, ...(present && !incomplete(evidence) ? { locator: { scheme: "text-span", locator: evidence.provenance.locator, excerpt: evidence.provenance.excerpt } } : {}), extraction: { extractionId: `${id}.${role}.extraction`, target: fieldPath, ...(present ? { confidence: evidence.confidence, extractor: evidence.extractor } : {}), extractedAt: evidence?.observedAt ?? fallback.observedAt }, claimTarget: target, producer: { "lookout.kontourai.io/semantic-transition": { observationId, evidenceState: present ? "present" : "absent" } } };
|
|
60
|
+
}
|
|
61
|
+
export function projectReviewItem(change, occurrence, transitionId, identities, prior, current, target) {
|
|
62
|
+
const itemIdentity = reviewKey({ transitionId, change, occurrence });
|
|
63
|
+
if (!itemIdentity.ok)
|
|
64
|
+
return itemIdentity;
|
|
65
|
+
const name = `lookout-semantic.${itemIdentity.value}`;
|
|
66
|
+
const candidates = [candidate(name, "current", identities.prior, change.prior, prior, target, change.fieldPath), candidate(name, "proposed", identities.current, change.current, current, target, change.fieldPath)];
|
|
67
|
+
return { ok: true, value: { apiVersion: semanticReviewApiVersion, kind: "ReviewItem", metadata: { name, producer: { "lookout.kontourai.io/semantic-transition": { semanticKind: change.kind, transitionId, priorObservationId: identities.prior, currentObservationId: identities.current } } }, spec: { target: change.fieldPath, candidates, candidateSetStatus: "needs-review", editable: false }, status: { observedCandidateCount: candidates.length } } };
|
|
68
|
+
}
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
import type { TargetFieldSchema } from "@kontourai/traverse";
|
|
2
|
+
import type { SchemaCoverageGap } from "./coverage.js";
|
|
3
|
+
import type { ProposalEvidence, ProposalSetDiffInput, ProposalSetObservation } from "./proposal-diff.js";
|
|
4
|
+
export declare const semanticReviewApiVersion = "survey.kontourai.io/v1alpha1";
|
|
5
|
+
export type SemanticReviewKind = "proposal-added" | "proposal-removed" | "proposal-moved" | "proposal-provenance-changed" | "proposal-value-changed" | "coverage-gap" | "provenance-gap";
|
|
6
|
+
export interface SemanticObservationIdentity {
|
|
7
|
+
readonly prior: string;
|
|
8
|
+
readonly current: string;
|
|
9
|
+
}
|
|
10
|
+
export interface SemanticClaimTarget {
|
|
11
|
+
readonly subjectType: string;
|
|
12
|
+
readonly subjectId: string;
|
|
13
|
+
readonly facet: string;
|
|
14
|
+
readonly claimType: string;
|
|
15
|
+
readonly fieldOrBehavior: string;
|
|
16
|
+
readonly impactLevel: "low" | "medium" | "high" | "critical";
|
|
17
|
+
readonly [key: string]: unknown;
|
|
18
|
+
}
|
|
19
|
+
export interface SemanticReviewCandidate {
|
|
20
|
+
readonly id: string;
|
|
21
|
+
readonly role: "current" | "proposed" | "source-version";
|
|
22
|
+
readonly value: unknown;
|
|
23
|
+
readonly confidence?: number;
|
|
24
|
+
readonly source: {
|
|
25
|
+
readonly sourceRef: string;
|
|
26
|
+
readonly sourceId: string;
|
|
27
|
+
readonly observedAt: string;
|
|
28
|
+
readonly locatorScheme: "text-span";
|
|
29
|
+
};
|
|
30
|
+
readonly locator?: {
|
|
31
|
+
readonly scheme: "text-span";
|
|
32
|
+
readonly locator: string;
|
|
33
|
+
readonly excerpt: string;
|
|
34
|
+
};
|
|
35
|
+
readonly extraction: {
|
|
36
|
+
readonly extractionId: string;
|
|
37
|
+
readonly target: string;
|
|
38
|
+
readonly confidence?: number;
|
|
39
|
+
readonly extractor?: string;
|
|
40
|
+
readonly extractedAt: string;
|
|
41
|
+
};
|
|
42
|
+
readonly claimTarget: SemanticClaimTarget;
|
|
43
|
+
readonly producer: {
|
|
44
|
+
readonly "lookout.kontourai.io/semantic-transition": {
|
|
45
|
+
readonly observationId: string;
|
|
46
|
+
readonly evidenceState: "present" | "absent";
|
|
47
|
+
};
|
|
48
|
+
};
|
|
49
|
+
}
|
|
50
|
+
/** Structural match for Survey ReviewItem; Lookout deliberately imports no review package. */
|
|
51
|
+
export interface SemanticReviewItem {
|
|
52
|
+
readonly apiVersion: typeof semanticReviewApiVersion;
|
|
53
|
+
readonly kind: "ReviewItem";
|
|
54
|
+
readonly metadata: {
|
|
55
|
+
readonly name: string;
|
|
56
|
+
readonly producer: {
|
|
57
|
+
readonly "lookout.kontourai.io/semantic-transition": {
|
|
58
|
+
readonly semanticKind: SemanticReviewKind;
|
|
59
|
+
readonly transitionId: string;
|
|
60
|
+
readonly priorObservationId: string;
|
|
61
|
+
readonly currentObservationId: string;
|
|
62
|
+
};
|
|
63
|
+
};
|
|
64
|
+
};
|
|
65
|
+
readonly spec: {
|
|
66
|
+
readonly target: string;
|
|
67
|
+
readonly candidates: SemanticReviewCandidate[];
|
|
68
|
+
readonly candidateSetStatus: "needs-review";
|
|
69
|
+
readonly editable: false;
|
|
70
|
+
};
|
|
71
|
+
readonly status: {
|
|
72
|
+
readonly observedCandidateCount: number;
|
|
73
|
+
};
|
|
74
|
+
}
|
|
75
|
+
export interface SemanticReviewChange {
|
|
76
|
+
readonly kind: SemanticReviewKind;
|
|
77
|
+
readonly fieldPath: string;
|
|
78
|
+
readonly entityKey: string;
|
|
79
|
+
readonly prior?: ProposalEvidence;
|
|
80
|
+
readonly current?: ProposalEvidence;
|
|
81
|
+
readonly gap?: SchemaCoverageGap;
|
|
82
|
+
}
|
|
83
|
+
export interface SemanticReviewWork {
|
|
84
|
+
readonly transitionId: string;
|
|
85
|
+
readonly priorObservationId: string;
|
|
86
|
+
readonly currentObservationId: string;
|
|
87
|
+
readonly items: readonly SemanticReviewItem[];
|
|
88
|
+
}
|
|
89
|
+
export interface BuildSemanticReviewWorkInput<E> extends Omit<ProposalSetDiffInput<E>, "prior" | "current"> {
|
|
90
|
+
readonly prior: ProposalSetObservation;
|
|
91
|
+
readonly current: ProposalSetObservation;
|
|
92
|
+
readonly observationIdentity: SemanticObservationIdentity;
|
|
93
|
+
readonly schema?: readonly TargetFieldSchema[];
|
|
94
|
+
readonly claimTarget: (change: SemanticReviewChange) => SemanticClaimTarget;
|
|
95
|
+
}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export const semanticReviewApiVersion = "survey.kontourai.io/v1alpha1";
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
import type { BuildSemanticReviewWorkInput, SemanticReviewWork } from "./semantic-review-types.js";
|
|
2
|
+
export { semanticReviewApiVersion } from "./semantic-review-types.js";
|
|
3
|
+
export type { BuildSemanticReviewWorkInput, SemanticClaimTarget, SemanticObservationIdentity, SemanticReviewCandidate, SemanticReviewChange, SemanticReviewItem, SemanticReviewKind, SemanticReviewWork } from "./semantic-review-types.js";
|
|
4
|
+
/** Project one genuine proposal-observation transition into deterministic review work. */
|
|
5
|
+
export declare function buildSemanticReviewWork<E>(input: BuildSemanticReviewWorkInput<E>): import("./canonical-value.js").DiffResult<SemanticReviewWork>;
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
import { diffProposalSets } from "./proposal-diff.js";
|
|
2
|
+
import { collectSemanticChanges, projectReviewItem, reviewKey, sanitizeError, validateObservationIdentity } from "./semantic-review-projection.js";
|
|
3
|
+
export { semanticReviewApiVersion } from "./semantic-review-types.js";
|
|
4
|
+
/** Project one genuine proposal-observation transition into deterministic review work. */
|
|
5
|
+
export function buildSemanticReviewWork(input) {
|
|
6
|
+
const identities = validateObservationIdentity(input.observationIdentity);
|
|
7
|
+
if (!identities.ok)
|
|
8
|
+
return identities;
|
|
9
|
+
const diff = diffProposalSets(input);
|
|
10
|
+
if (!diff.ok)
|
|
11
|
+
return sanitizeError(diff);
|
|
12
|
+
const transition = reviewKey({ sourceId: input.current.sourceId, priorObservationId: identities.value.prior, currentObservationId: identities.value.current });
|
|
13
|
+
if (!transition.ok)
|
|
14
|
+
return transition;
|
|
15
|
+
const items = [];
|
|
16
|
+
const occurrences = new Map();
|
|
17
|
+
for (const change of collectSemanticChanges(input, diff.value)) {
|
|
18
|
+
let target;
|
|
19
|
+
try {
|
|
20
|
+
target = input.claimTarget(change);
|
|
21
|
+
}
|
|
22
|
+
catch {
|
|
23
|
+
return { ok: false, error: { kind: "callback-threw", message: "claimTarget callback threw", path: "$.claimTarget" } };
|
|
24
|
+
}
|
|
25
|
+
const base = reviewKey(change);
|
|
26
|
+
if (!base.ok)
|
|
27
|
+
return base;
|
|
28
|
+
const occurrence = occurrences.get(base.value) ?? 0;
|
|
29
|
+
occurrences.set(base.value, occurrence + 1);
|
|
30
|
+
const item = projectReviewItem(change, occurrence, transition.value, identities.value, input.prior, input.current, target);
|
|
31
|
+
if (!item.ok)
|
|
32
|
+
return item;
|
|
33
|
+
items.push(item.value);
|
|
34
|
+
}
|
|
35
|
+
return { ok: true, value: { transitionId: transition.value, priorObservationId: identities.value.prior, currentObservationId: identities.value.current, items } };
|
|
36
|
+
}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@kontourai/lookout",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.3",
|
|
4
4
|
"description": "A small source registry and drift-check runner built on Forage snapshots.",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"type": "module",
|
|
@@ -36,18 +36,20 @@
|
|
|
36
36
|
"prepare": "npm run build",
|
|
37
37
|
"typecheck": "tsc --noEmit",
|
|
38
38
|
"test": "npm run build && node --test dist/tests/*.test.js",
|
|
39
|
-
"verify": "npm run check:content-boundary && npm run check:decisions && npm run typecheck && npm test && npm run check:pack",
|
|
39
|
+
"verify": "npm run check:content-boundary && npm run check:decisions && npm run typecheck && npm test && npm run check:pack && npm run check:consumer",
|
|
40
40
|
"check:content-boundary": "node scripts/check-content-boundary.cjs",
|
|
41
41
|
"check:decisions": "node scripts/check-decisions.cjs check",
|
|
42
42
|
"gen:decisions-index": "node scripts/check-decisions.cjs gen-index",
|
|
43
|
-
"check:pack": "node scripts/check-package-contents.mjs"
|
|
43
|
+
"check:pack": "node scripts/check-package-contents.mjs",
|
|
44
|
+
"check:consumer": "node scripts/check-installed-consumer.mjs"
|
|
44
45
|
},
|
|
45
46
|
"dependencies": {
|
|
46
|
-
"@kontourai/datum": "0.
|
|
47
|
+
"@kontourai/datum": "0.7.0",
|
|
47
48
|
"@kontourai/forage": "0.4.1",
|
|
48
|
-
"@kontourai/traverse": "0.
|
|
49
|
+
"@kontourai/traverse": "0.22.0"
|
|
49
50
|
},
|
|
50
51
|
"devDependencies": {
|
|
52
|
+
"@kontourai/survey": "2.1.0",
|
|
51
53
|
"@types/node": "^25.6.0",
|
|
52
54
|
"typescript": "^5.8.0"
|
|
53
55
|
},
|