@happyvertical/smrt-ingestion 0.55.5 → 0.55.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (85) hide show
  1. package/AGENTS.md +13 -3
  2. package/FEEDBACK.md +117 -0
  3. package/PROPOSALS.md +173 -0
  4. package/README.md +14 -0
  5. package/REVIEW.md +136 -0
  6. package/agents/execution.md +132 -0
  7. package/dist/chunks/execution-CYjtj9_l.js +8031 -0
  8. package/dist/chunks/execution-CYjtj9_l.js.map +1 -0
  9. package/dist/chunks/execution-internal-DWteDHM2.js +7 -0
  10. package/dist/chunks/execution-internal-DWteDHM2.js.map +1 -0
  11. package/dist/chunks/feedback-B_NILilH.js +577 -0
  12. package/dist/chunks/feedback-B_NILilH.js.map +1 -0
  13. package/dist/chunks/proposal-errors-z24VKWnQ.js +301 -0
  14. package/dist/chunks/proposal-errors-z24VKWnQ.js.map +1 -0
  15. package/dist/dto.d.ts +3 -0
  16. package/dist/dto.d.ts.map +1 -1
  17. package/dist/execution-contracts.d.ts +3 -0
  18. package/dist/execution-contracts.d.ts.map +1 -1
  19. package/dist/execution-internal.d.ts +9 -0
  20. package/dist/execution-internal.d.ts.map +1 -0
  21. package/dist/execution.d.ts +23 -1
  22. package/dist/execution.d.ts.map +1 -1
  23. package/dist/execution.js +1 -7614
  24. package/dist/extraction.d.ts.map +1 -1
  25. package/dist/feedback-contracts.d.ts +40 -0
  26. package/dist/feedback-contracts.d.ts.map +1 -0
  27. package/dist/feedback-dto.d.ts +100 -0
  28. package/dist/feedback-dto.d.ts.map +1 -0
  29. package/dist/feedback-internal.d.ts +19 -0
  30. package/dist/feedback-internal.d.ts.map +1 -0
  31. package/dist/feedback.d.ts +53 -0
  32. package/dist/feedback.d.ts.map +1 -0
  33. package/dist/manifest.json +13 -1
  34. package/dist/models.js +1 -1
  35. package/dist/models.js.map +1 -1
  36. package/dist/proposal-contracts.d.ts +89 -0
  37. package/dist/proposal-contracts.d.ts.map +1 -0
  38. package/dist/proposal-dto.d.ts +124 -0
  39. package/dist/proposal-dto.d.ts.map +1 -0
  40. package/dist/proposal-errors.d.ts +5 -0
  41. package/dist/proposal-errors.d.ts.map +1 -0
  42. package/dist/proposal-sdk.d.ts +12 -0
  43. package/dist/proposal-sdk.d.ts.map +1 -0
  44. package/dist/proposal-validation.d.ts +188 -0
  45. package/dist/proposal-validation.d.ts.map +1 -0
  46. package/dist/proposals.d.ts +44 -0
  47. package/dist/proposals.d.ts.map +1 -0
  48. package/dist/proposals.js +964 -0
  49. package/dist/proposals.js.map +1 -0
  50. package/dist/review-dto.d.ts +115 -0
  51. package/dist/review-dto.d.ts.map +1 -0
  52. package/dist/server.d.ts +56 -0
  53. package/dist/server.d.ts.map +1 -1
  54. package/dist/server.js +263 -341
  55. package/dist/server.js.map +1 -1
  56. package/dist/smrt-knowledge.json +87 -12
  57. package/dist/svelte/components/ActionReview.svelte +232 -0
  58. package/dist/svelte/components/ActionReview.svelte.d.ts +15 -0
  59. package/dist/svelte/components/ActionReview.svelte.d.ts.map +1 -0
  60. package/dist/svelte/components/EvidenceViewer.svelte +31 -0
  61. package/dist/svelte/components/EvidenceViewer.svelte.d.ts +8 -0
  62. package/dist/svelte/components/EvidenceViewer.svelte.d.ts.map +1 -0
  63. package/dist/svelte/components/InboxView.svelte +119 -0
  64. package/dist/svelte/components/InboxView.svelte.d.ts +9 -0
  65. package/dist/svelte/components/InboxView.svelte.d.ts.map +1 -0
  66. package/dist/svelte/components/IntakeInbox.svelte +11 -0
  67. package/dist/svelte/components/IntakeInbox.svelte.d.ts +10 -0
  68. package/dist/svelte/components/IntakeInbox.svelte.d.ts.map +1 -0
  69. package/dist/svelte/components/IntakeReview.svelte +11 -0
  70. package/dist/svelte/components/IntakeReview.svelte.d.ts +10 -0
  71. package/dist/svelte/components/IntakeReview.svelte.d.ts.map +1 -0
  72. package/dist/svelte/components/ReviewView.svelte +243 -0
  73. package/dist/svelte/components/ReviewView.svelte.d.ts +9 -0
  74. package/dist/svelte/components/ReviewView.svelte.d.ts.map +1 -0
  75. package/dist/svelte/i18n.d.ts +72 -0
  76. package/dist/svelte/i18n.d.ts.map +1 -0
  77. package/dist/svelte/i18n.js +72 -0
  78. package/dist/svelte/index.d.ts +5 -0
  79. package/dist/svelte/index.d.ts.map +1 -0
  80. package/dist/svelte/index.js +3 -0
  81. package/dist/svelte/view-utils.d.ts +13 -0
  82. package/dist/svelte/view-utils.d.ts.map +1 -0
  83. package/dist/svelte/view-utils.js +96 -0
  84. package/package.json +53 -22
  85. package/dist/execution.js.map +0 -1
package/AGENTS.md CHANGED
@@ -11,6 +11,8 @@ belongs under `/server`. Generated REST/CLI/MCP model mutation is disabled.
11
11
  See [README.md](README.md) for host integration and recovery/deletion operation.
12
12
  See [EXTRACTION.md](EXTRACTION.md) for extraction contracts, provider isolation
13
13
  requirements and corpus validation.
14
+ See [PROPOSALS.md](PROPOSALS.md) for the optional server-only proposal stage,
15
+ current catalog/candidate authority and immutable source pinning.
14
16
 
15
17
  All records require tenant and confidential scope. Hosts create service scopes
16
18
  from authenticated sessions; source metadata is never authority. Every read and
@@ -25,9 +27,17 @@ submission never run inside ingestion database transactions.
25
27
 
26
28
  Validation: `pnpm test`, `pnpm test:postgres`, `pnpm test:providers`, `pnpm typecheck`, `pnpm build`.
27
29
  The PostgreSQL command provisions a real database and fails if unavailable.
30
+ Fixture teardown uses `src/test-support/postgres-cleanup.ts`: only owned
31
+ exec_/ing_/ext_/sources_ names, normal DROP, bounded 55006-only retry. Never
32
+ force-terminate backends, grant signal privileges, or ignore leaked sessions.
28
33
  Build includes browser import validation. The provider lane exercises local SDK
29
34
  adapters and process isolation, without claiming live remote recognition quality.
30
- The component/e2e/evaluation
31
- commands are reserved by ADR 0004 and fail explicitly until their owning children
32
- supply real suites.
35
+ `src/svelte` owns transport-neutral review components and browser-safe host callbacks.
36
+ See [FEEDBACK.md](FEEDBACK.md) for revision-bound explicit feedback, scoped example retrieval,
37
+ provider-boundary revalidation and authorized routing policy adoption.
38
+ See [REVIEW.md](REVIEW.md) for authenticated hosts, bounded saved-review reloads,
39
+ assignment ownership, logical split re-extraction and the maintained browser proof.
40
+ UI validation additionally requires `pnpm test:components` and `pnpm test:e2e`;
41
+ `typecheck` includes Svelte and `build` includes the types-first `/svelte` entry.
42
+ The evaluation command remains reserved for #3677 and fails explicitly.
33
43
  See ../../docs/test-matrix/3670-ingestion-foundation.md for behavior coverage.
package/FEEDBACK.md ADDED
@@ -0,0 +1,117 @@
1
+ # Explicit ingestion feedback
2
+
3
+ Configure `IngestionOptions.feedback` to opt in. The existing `intake_feedback`
4
+ ledger owns these records; no new schema, embedding service or provider is added.
5
+ `FeedbackConfiguration` sets a version, maximum examples, scan count, serialized
6
+ bytes and lexical similarity threshold, plus a live scoped authorization callback.
7
+ Hard ceilings are 20 examples, 1,000 scanned records and 1 MB. Choose substantially
8
+ smaller application limits. The proposal's complete serialized request, including
9
+ examples, must also fit the current minimum application/source/provider budget.
10
+
11
+ `recordFeedback` accepts the exact displayed action revision, review version and
12
+ binding hash, an explicit correct/incorrect judgment, optional reason text and a
13
+ request ID. The server resolves reviewer, tenant/confidential scope, retained
14
+ source evidence, extraction/generation versions and current handler arguments.
15
+ A replay returns the same immutable receipt; changed payloads conflict.
16
+ `supersedesId` appends a replacement for the specified prior interpretation
17
+ judgment. A second successor conflicts. Revised actions make old bindings
18
+ ineligible. Approval and successful execution never imply that extracted fields
19
+ are correct.
20
+
21
+ Decision and downstream execution facts are captured inside their owning
22
+ transaction, keyed by immutable event ID and signal, when the configured capture
23
+ authority allows the optional memory projection and it fits the configured byte
24
+ cap. Denial or an oversized automatic projection leaves the underlying
25
+ authoritative decision/outcome intact, including failure and reconciliation.
26
+ Explicit `recordFeedback` and `observeFeedback` requests still reject invalid or
27
+ oversized writes; actual database failures still propagate and roll back. Failed/rolled-back domain
28
+ work and unknown external outcomes remain distinct. `observeFeedback` can replay
29
+ an authorized existing event; it cannot supply a correctness label or fabricate
30
+ an execution result. An initial denial does not mutate retention. Once authorized,
31
+ a narrowed privacy ceiling survives business rollback and later revocation.
32
+
33
+ ## Retrieval and provider boundaries
34
+
35
+ `retrieveFeedback({ itemId, query })` derives the current offered catalog,
36
+ candidates and model/prompt/configuration versions on the server. Callers cannot
37
+ provide candidate authority. The deterministic matcher uses lowercase Unicode
38
+ word-set Jaccard similarity (minimum 0.2), then immutable record ID for ties.
39
+ Scanning, recursive provenance checks and returned examples are bounded; a limited scan does not promise an
40
+ exhaustive history search. Both positive and negative explicit examples are kept.
41
+
42
+ Every selected source must still be readable in the same tenant and confidential
43
+ scope, with its current action/analysis binding, target grants, handler version,
44
+ model/prompt/configuration compatibility and retention. Each source's current
45
+ provider policy must allow the generation recipient and any configured decision
46
+ recipient; the source owner rechecks that policy on both initial and final live
47
+ contexts. Receiving checks cover every offered handler, before and after source
48
+ checks, using each live grant, actual provider recipients and the narrowest byte
49
+ ceiling. Both retrieval and reference revalidation budget the full reconstructed
50
+ examples. A receiving item's provider grant cannot substitute for a source grant.
51
+ Superseded, revoked, deleted or incompatible sources are excluded. Current explicit human corrections
52
+ take precedence: a generation carrying one does not retrieve learned examples.
53
+
54
+ Full examples exist only in the authorized provider request. The SDK proposal
55
+ adapter includes that typed selection in its serialized user message when
56
+ present; disabled feedback leaves the original request shape unchanged. Saved generation
57
+ provenance contains the receiving query, selection configuration digest and
58
+ immutable feedback IDs/digests, not copied source example content. Selected
59
+ references are rechecked before generation, capability probing, decision calls,
60
+ publication and completed-result reads. A newly added example does not invalidate
61
+ an older selection; a revoked or superseded selected example does. Feedback
62
+ lineage is bounded and fails closed. Saved-action reads and pending decisions,
63
+ automatic authorization, preview publication/replay and execution also revalidate
64
+ selected lineage under the
65
+ owning transaction. Revoking, deleting or superseding a selected example blocks
66
+ pending actions; regenerate their proposals and obtain fresh approval to recover.
67
+ A preview replay checks the exact persisted proposal it returns, including its
68
+ analysis attempt; a newer action revision cannot supply replacement lineage.
69
+ Plan replay and final sealed plan-step publication use the same gate.
70
+ For already successful actions, current-authorized durable outcomes remain
71
+ readable/replayable, but invalid learned lineage suppresses saved proposal
72
+ display, arguments and plan payload. An empty selection carries no learned source
73
+ and does not turn optional feedback-capture denial into action denial. Examples
74
+ never expand the offered catalog, candidate IDs, operation permissions or
75
+ automation eligibility.
76
+
77
+ All durable feedback belongs to the source item's existing retention/redaction
78
+ lifecycle. There is no separate memory store or delayed indexing writer to clean
79
+ up. Hosts using other derived stores still implement the existing `purgeDerived`
80
+ contract. Core `LearningMemory` is intentionally not used here: its success/failure
81
+ reinforcement and default ancestor-scope fallback do not represent separate
82
+ correctness/decision/outcome signals and strict confidential-scope eligibility.
83
+
84
+ ## Routing suggestions and adoption
85
+
86
+ `suggestRoutingRule` requires repeated current explicit positive judgments with
87
+ from distinct source items with matching handler/version and arguments, and returns supporting immutable IDs,
88
+ shared query terms and the policy owner's preview/version. Suggestions cannot
89
+ execute or modify policy. `getRoutingRule` rechecks the displayed binding and all
90
+ supporting examples. Arguments are routing preferences, never executable grants.
91
+
92
+ `adoptRoutingRule` requires the exact suggestion digest and expected policy
93
+ version plus current reviewer and adoption authority. The configured policy owner
94
+ must perform its version compare-and-swap and audit on the supplied database
95
+ executor. The ingestion transaction records the exact returned version/audit ID.
96
+ Replays do not adopt twice. Revocation or permission/policy widening rolls back
97
+ both changes. The host callback may store routing preferences only: it must not
98
+ change grants, provider permissions or automation controls. Adoption always
99
+ returns `automaticActionEligible: false`; subsequent proposals retain normal
100
+ current policy and explicit review requirements.
101
+
102
+ ## Review UI and evidence
103
+
104
+ The optional `IntakeReviewHost.feedback` callback carries the exact displayed
105
+ revision and binding. Correct/incorrect controls are explicit and independent of
106
+ Approve/Apply. Unsaved argument edits disable correctness controls until the
107
+ displayed binding is current again. Hosts must authenticate that callback and connect it to
108
+ `recordFeedback`; client-supplied reviewer/scope fields are not accepted. The
109
+ maintained browser fixture demonstrates this transport with CSRF protection and
110
+ rejects substituted revisions.
111
+
112
+ See [the behavior matrix](../../docs/test-matrix/3676-ingestion-feedback.md) for
113
+ validation and the frozen learning protocol. The small authored deterministic
114
+ cohort demonstrates contract-level influence, not provider quality or measured
115
+ production improvement. Real-provider evaluation belongs to #3677. There are no
116
+ sampled automatic outcomes in this feature; reporting that stratum as unavailable
117
+ avoids inventing correctness labels from successful jobs.
package/PROPOSALS.md ADDED
@@ -0,0 +1,173 @@
1
+ # Application-aware proposal generation
2
+
3
+ `@happyvertical/smrt-ingestion/proposals` is server-only. Configure
4
+ `IngestionOptions.proposals` and the existing `execution.handlers` catalog.
5
+ Handlers opt into discovery with `discovery.mediaTypes`, `references`, and an
6
+ optional `candidates` callback. There is one catalog: the same operation or
7
+ ordered playbook adapter is discovered, validated, previewed, and later applied
8
+ by the execution service. Generation never approves or applies.
9
+
10
+ ## Discovery and authority
11
+
12
+ `listHandlers(itemId)` returns a bounded detached catalog. The owning execution
13
+ service checks the current principal, captured permissions/handler/operation
14
+ ceiling, policy, confidential scope, public registered operation facts, and
15
+ ordered playbook definition. Every configured provider must occur in the current
16
+ policy's `providers` allowlist. Missing grants remove handlers before model input.
17
+ An authorized catalog larger than the configured bound is rejected; the host must
18
+ narrow it explicitly instead of relying on silent truncation.
19
+
20
+ `findCandidates({itemId,handlerId,handlerVersion,query,limit})` returns
21
+ `{items,truncated}`. Its trusted handler callback receives `HandlerContext` with
22
+ the authorized transaction executor. Query by tenant and confidential scope
23
+ **before** selecting, ranking, or building labels. Use a bounded query (often
24
+ `limit + 1`) and report `hasMore`. The service projects candidate identity, model,
25
+ revision and label, then checks every target with the owning execution callback.
26
+ There are no hidden counts or pagination tokens. Truncated discovery forces
27
+ `needs_review` and cannot enter generated preview.
28
+
29
+ Completed generation reload checks process authority, the current attempt, and
30
+ owner/candidate visibility in the same final transaction. Database-backed grant
31
+ callbacks must use the supplied executor. The owning retention lifecycle encloses
32
+ that transaction so a denied read cannot roll back an authorized shorter privacy
33
+ ceiling; restoration starts only after the outer transaction has rolled back.
34
+
35
+ Candidate callbacks perform bounded database reads only: no provider I/O, network,
36
+ or domain mutations. Target locks and existing retention enforcement remain
37
+ owned by execution. Hosts configure their database's statement deadlines; an
38
+ arbitrary trusted callback is not a sandbox. Each existing-record argument must
39
+ be a declared top-level `references` field of kind `candidate` with its qualified
40
+ model, or `evidence`. Closed JSON argument schemas are required. The host owns
41
+ that declaration and its target authorization semantics.
42
+
43
+ ## Durable stages
44
+
45
+ 1. Complete extraction through the existing analysis ledger.
46
+ 2. Call `prepareGeneration(itemId, extractionAttemptId)`. It reads the current
47
+ completed/partial immutable extraction and returns configuration with
48
+ `stage: 'interpret'`, pinned source attempt/revision/input/output/evidence
49
+ digests, prompt/configuration/catalog/provider versions, limits and thresholds.
50
+ 3. Pass that configuration to `analyze(itemId, configuration, requestKey)` and
51
+ claim its new revision with `claimAnalysis`.
52
+ 4. The host's existing job callback routes on the frozen `configuration.stage`:
53
+ extraction invokes `extractAnalysis`; interpretation invokes
54
+ `generateProposals(lease)`.
55
+ 5. Reload the current completed generation using `getCompletedAnalysis(itemId)`,
56
+ or pass an explicit attempt ID to additionally reject stale client state.
57
+ Queued/running revisions never return an older completed attempt.
58
+
59
+ A generation revision does not rewrite terminal extraction. Reprocessing starts
60
+ with a new extraction revision, followed by interpretation, while the host keeps
61
+ the same business intention keys. An interrupted still-current generation lease
62
+ can be retried; terminal generation output remains immutable. `getGenerationInput`
63
+ reads historical extraction only through the current generation lease and its
64
+ frozen source pin. It verifies same-item scope, immutable hashes, durable evidence,
65
+ and current process permission before returning detached data. The source must be
66
+ the immediately preceding revision, so an intervening reprocess invalidates a
67
+ prepared generation. Completed generation reads additionally recheck current
68
+ catalog and target visibility before disclosing persisted candidate labels.
69
+ Expired, cancelled, revoked, retried or superseded workers cannot read or publish late output. The
70
+ current lease and each offered handler/candidate are rechecked immediately before
71
+ each outbound generative/decision boundary and before publication.
72
+
73
+ ## Providers and validation
74
+
75
+ `createSDKProposalGenerator(client, identity, {maxTokens,timeoutMs})` uses an
76
+ existing AI SDK `chat` client. It sends a fixed system instruction and separate
77
+ JSON evidence/catalog data, requests JSON, supplies cancellation and bounds, uses
78
+ no tools or continuation, and rejects known truncation or tool calls,
79
+ unexpected models, malformed JSON and oversized output. Configure SDK clients
80
+ without automatic retry that would bypass application boundary checks. The
81
+ adapter's public identity is exactly provider/model/version; credentials and raw
82
+ provider errors are never provenance.
83
+
84
+ A trusted custom `ProposalGenerator` must likewise honor cancellation, bound its
85
+ response, and explicitly declare validated `complete` or `unknown` completion.
86
+ The SDK currently normalizes missing/unknown finish reasons to `stop`
87
+ ([owning SDK follow-up](https://github.com/happyvertical/sdk/issues/1381)), so its
88
+ adapter conservatively declares `unknown`, persisted as
89
+ `generation_completion_unknown`. Otherwise valid suggestions remain available
90
+ for human preview; automatic eligibility always remains false. The service
91
+ deadline aborts requests and prevents late publication; it does not claim to terminate computation at a remote
92
+ provider or isolate native allocations. Extractor process limits remain separate.
93
+
94
+ Model output can contain only offered handler versions, schema-valid arguments,
95
+ retained evidence locations and offered alternative handler/candidate keys.
96
+ Candidate IDs must occur in the offered set, and evidence IDs in retained source
97
+ results. Unknown enums, invented IDs, extra authority/tool/approval fields and
98
+ malformed structures fail closed. Missing required top-level fields are retained
99
+ as `missingFields`, with `needs_review`; values are never invented to satisfy a
100
+ schema. Observed page/time/source granularity is preserved. Logical scan splits
101
+ use `proposeDocumentSplits`; originals and hashes remain intact.
102
+
103
+ `createSDKProposalDecisionClient(client)` adapts an existing SDK client’s optional
104
+ capability without changing its errors or protocol. Optional `decision` uses public core `executeDecision` and the existing SDK
105
+ predicate, choice, score and batch contracts. Supported-effect predicates,
106
+ handler classification, candidate entity matching, and action ambiguity/risk
107
+ are distinct questions. Their validated probabilities and provenance are stored
108
+ separately from generative provenance and unknown OCR certainty. The complete
109
+ serialized decision request, including suggestions and questions, must fit the
110
+ minimum configured/current policy byte ceiling before capability probing and
111
+ again before deciding. A policy change between these boundaries fails closed.
112
+ Ties, support probability or selected-handler confidence below
113
+ `minimumDecisionProbability`, and actually incomplete extraction require review.
114
+ Unknown provider
115
+ completion metadata is preserved as `source_completion_unknown`; it does not
116
+ block otherwise schema-valid **human** preview or imply completeness. Automatic
117
+ action eligibility remains false. An unconfigured decision path
118
+ is recorded explicitly. Configured errors produce a durable safe failure outcome;
119
+ they never silently switch to a generative decision fallback.
120
+
121
+ Output records zero, one or multiple suggestions, alternatives, missing fields,
122
+ source links, proposed splits, versions, allowlisted usage, warnings and explicit
123
+ `unknown`, `ambiguous`, `no_action`, `needs_review` or `provider_error` outcomes.
124
+ `automaticActionEligible` is always false. Provider input/output and the complete
125
+ serialized persisted envelope are bounded by configuration, live policy and the
126
+ receipt ceiling. Oversized output becomes a bounded review/limit result; when
127
+ no safe envelope fits, the owning service records fixed `failAnalysis` metadata
128
+ outside the provider payload budget without retaining provider output.
129
+
130
+ ## Explicit preview and stable intentions
131
+
132
+ The host chooses persisted suggestion indexes and calls:
133
+
134
+ ```ts
135
+ await service.previewGeneratedProposals({
136
+ itemId,
137
+ attemptId: generationAttemptId,
138
+ selections: [{
139
+ index: 0,
140
+ intentionKey: 'document:meeting-2026-10-09',
141
+ expectedRevision: 0,
142
+ requestId: 'review-request-1',
143
+ }],
144
+ });
145
+ ```
146
+
147
+ `intentionKey` is the host's stable business-intention identity across generation
148
+ revisions. Never derive a fresh key from model output, suggestion order or a retry.
149
+ Selections must reference current persisted validated `ready_for_review` output.
150
+ Each batch requires distinct suggestion indexes and intention keys; duplicate
151
+ indexes are rejected before any operation or plan preview is written.
152
+ The service rechecks current catalog/schema/target/source provenance, then calls
153
+ the existing operation or plan preview. Each result still requires the ordinary
154
+ human review and execution APIs. Reprocessing creates no new business identity;
155
+ successful-action replay protection stays with execution.
156
+
157
+ ## Evidence and limits
158
+
159
+ The test matrix is `docs/test-matrix/3673-ingestion-proposals.md`. SQLite and real
160
+ PostgreSQL suites exercise authority, revisions, abstention, hostile output,
161
+ compound proposals, split provenance and reviewed domain effects. The upload
162
+ reference composes authenticated multipart delivery of a generated embedded-text
163
+ PDF through the actual packaged PDF child, durable interpretation, and the existing
164
+ ordered ContentDocument draft/retained-asset plan. It checks duplicate delivery,
165
+ unauthenticated and foreign-tenant denial, explicit approval of both steps,
166
+ idempotent execution replay, page provenance, and unchanged original bytes.
167
+ The compound-email case uses a declared messages-owner snapshot fixture.
168
+
169
+ Local HTTP fixtures execute the actual AI SDK transport and cancellation contract.
170
+ Generative responses and the separate injected PDF split boundaries establish
171
+ deterministic orchestration, not model accuracy or OCR quality. No held-out routing
172
+ evaluation or automatic approval is claimed. Missing/unknown AI completion remains
173
+ explicit under [SDK #1381](https://github.com/happyvertical/sdk/issues/1381).
package/README.md CHANGED
@@ -146,3 +146,17 @@ raw-body reading. The callback must verify signature, delivery time/replay windo
146
146
  and account ownership, then return trusted binding and durable delivery identity.
147
147
  No vendor implementation or hardware driver is selected. All adapters invoke
148
148
  only receipt preservation, never domain writes, AI tools or external actions.
149
+
150
+ See [PROPOSALS.md](PROPOSALS.md) for bounded application catalog discovery, durable
151
+ interpretation revisions, optional typed decisions and explicit generated preview.
152
+
153
+ ## Human review
154
+
155
+ Mount the `/svelte` inbox and review components through an authenticated host using
156
+ the browser-safe `IntakeReviewHost` DTO contract. See [REVIEW.md](REVIEW.md) for
157
+ exact revision-bound callbacks, live evidence access, logical split corrections,
158
+ assignment ownership, and the maintained upload-to-domain-result browser fixture.
159
+
160
+ Optional explicit feedback and scoped example retrieval are documented in
161
+ [FEEDBACK.md](FEEDBACK.md). Approval and execution outcomes remain separate from
162
+ interpretation correctness; routing policy adoption is explicit and versioned.
package/REVIEW.md ADDED
@@ -0,0 +1,136 @@
1
+ # Authenticated review surfaces
2
+
3
+ `@happyvertical/smrt-ingestion/svelte` exports `IntakeInbox`, `IntakeReview` and
4
+ `EvidenceViewer`. The browser-safe DTO entry exports their `IntakeReviewHost`
5
+ callback contract. Import styling/theme configuration from smrt-ui in the host.
6
+ Components use smrt-ui primitives and the `ingestion.*` i18n catalog.
7
+
8
+ The host authenticates every request, resolves tenant/actor/confidential scope
9
+ from its session, enforces CSRF on mutations, and constructs `IngestionService`.
10
+ Never accept scope, reviewer, permissions or a principal from callback JSON.
11
+ Only supply authorized evidence URLs and escaped text; original URLs are
12
+ same-origin authenticated handlers that recheck `readEvidence` on every read.
13
+ Candidate discovery uses `findCandidates` with current grants. Presentation
14
+ state and disabled buttons provide no server authority.
15
+
16
+ `listReviews(itemId, {cursor?,limit?})` reloads saved action identities and their
17
+ current plan membership without browser storage. It returns at most 50 actions,
18
+ with an opaque action-ID cursor. Every payload requires current item, handler,
19
+ parent plan and target access under the owning transaction. Denied reads fail
20
+ closed. Target reads check current authorization separately from historical
21
+ revision freshness: unfinished drift returns state-only stale reviews without
22
+ removing unrelated entries; succeeded results remain readable after authorized
23
+ target changes. Apply still enforces exact bound target revisions. Expired entries
24
+ contain only review identity/state. Stale entries never disclose old args or target
25
+ labels; an authorized parent plan additionally exposes `stalePlan` identity, key,
26
+ current plan revision and handler identity. Hosts must collect fresh explicit args
27
+ from the reviewer/current authorized evidence and call `previewPlan` with that
28
+ exact plan CAS and the current completed analysis attempt. Action revision is not
29
+ plan revision. Re-expansion preserves action identity and requires fresh approval.
30
+ Success results are rechecked through the durable result reader.
31
+ Plan membership contains the exact parent args/version so edits call
32
+ `previewPlan`, not per-step correction. An item may have more review pages; the
33
+ host must aggregate state across pages when projecting a production inbox state.
34
+
35
+ The host's `load` callback supplies a current `ItemReviewView`, using
36
+ `getCompletedAnalysis` with no historical fallback. Generated suggestions bind
37
+ the **current generation attempt**, not `generation.source.attemptId` (the prior
38
+ extraction). `previewGeneratedProposals` publishes exact reviews and continues to reject stale
39
+ generation candidate pins. When `listReviews` reports `generationStale`, the host
40
+ omits obsolete generation output and renders authorized saved reviews/results.
41
+ Completed saved actions do not hide healthy unprepared suggestions. The saved-binding hint is not
42
+ an authoritative generation-availability check: a fresh human preview can bind a
43
+ current target while the immutable generation still pins its former revision.
44
+ `getCompletedAnalysis` remains strict and throws server-only
45
+ `GenerationSnapshotStaleError` only after every current catalog/provider/candidate
46
+ read is authorized and an immutable candidate revision differs. The maintained
47
+ host catches only that class, reacquires `listReviews` under current owning
48
+ authority, and returns that new page without generation output. It never reuses
49
+ the earlier page after an error. Denials and unknown failures propagate; healthy
50
+ generation still exposes unprepared sibling suggestions. Stale entries expose handler
51
+ and attempt identity only when that attempt is still the current completed/partial
52
+ analysis. An empty-args editor submits fresh explicit human arguments through
53
+ `previewProposal` or `previewPlan` with the saved CAS; it never reuses old generated
54
+ arguments, rewrites the generation snapshot or transfers approval. Analysis-stale
55
+ entries without an attempt require normal reprocessing. Approve,
56
+ reject, defer and correct call `submitDecision`; corrected args create a new
57
+ revision which requires a separate human approval. After an approve/defer/reject
58
+ decision, optional `editAction` calls `previewProposal` with the current action
59
+ revision to publish a new review; completed/executing/unknown actions cannot be
60
+ revised. Local dirty args disable Apply but confer no authority. Plan edits re-expand through
61
+ `previewPlan` with the current **plan** revision. `applyAction` accepts only an
62
+ action ID. Unknown external outcomes require server reconciliation; the UI
63
+ never offers a blind resend. Stable result IDs survive reload and retry.
64
+
65
+ Replace `contextKey` whenever authenticated tenant/session context changes.
66
+ Changing it or itemId destroys the old view and fences late callback results.
67
+ Denied refresh clears prior originals, candidates, arguments and results.
68
+ Missing confidence is unavailable; page/time links exist only for observed
69
+ locations. Provider errors and unresolved/partial results remain review states.
70
+
71
+ ## Assignment and logical splits
72
+
73
+ Assignment is explicitly host-owned, durable and versioned. `assign` receives
74
+ an expected assignment version and request ID; the host must authorize the
75
+ manager, constrain eligible assignees, CAS the durable record and reject stale
76
+ writes. Assignment does not grant read, review or execution rights. The maintained
77
+ fixture provisions its own SQLite assignment table before serving requests; no
78
+ production ingestion table or migration is introduced for assignment.
79
+
80
+ `reviseLogicalSplit` authorizes the current generation through the same-executor
81
+ catalog/candidate gate and the existing reviewer grant and policy reviewer list.
82
+ Process access alone cannot mint or replay a human correction; reviewer authority
83
+ is checked before mutation and again before publication. A generation without a
84
+ reviewable catalogue cannot be corrected through this reviewer API.
85
+ It requires all observed pages exactly once, an exact
86
+ current attempt/revision and a request ID. It records the authenticated actor
87
+ and predecessor as a **human correction** in a new immutable extraction
88
+ configuration, queues re-extraction and immediately makes previous suggestions
89
+ and approvals stale. It never changes original bytes/evidence or labels a human
90
+ edit as model output. The host processes the new extraction, then calls
91
+ `prepareGeneration` and the normal fenced interpretation stage. The supported
92
+ immediate-predecessor extraction pin is unchanged. The generator receives a
93
+ human correction only after its digest matches the authoritative source. Ordinary
94
+ `analyze` calls cannot mint an extraction correction; the owning split API is the
95
+ authenticated origin, and forged interpretation corrections fail before provider
96
+ disclosure. If its proposed grouping differs from the correction, suggestions are withheld
97
+ with `human_split_not_respected`. New proposals require new explicit review.
98
+ The reference host runs these stages synchronously for browser determinism;
99
+ production hosts dispatch the existing durable analysis jobs.
100
+
101
+ `feedback` is an optional explicit correctness callback, separate from approval
102
+ choice and execution success. No learning, feedback storage or rule promotion is
103
+ implemented by these components; #3676 owns those services. Do not pass feedback
104
+ or a feedback callback to readers without the relevant host grant.
105
+
106
+ ## Maintained browser proof
107
+
108
+ `pnpm test:components` runs real Svelte component tests; `pnpm test:e2e` runs the
109
+ package-local authenticated Vite application under Chromium. The fixture uses
110
+ HttpOnly same-site sessions, CSRF, real SQLite/assets/ingestion/content APIs,
111
+ production unpdf child extraction and a deterministic proposal generator. Its
112
+ hardcoded test login is fixture-only and is never a production auth provider.
113
+ Image/audio originals remain inspectable when no OCR/speech provider is
114
+ configured; the suite makes no recognition-quality claim and spends no provider
115
+ budget. `reference/review-host.ts` is a maintained development fixture, not a
116
+ published server entry. Browser tests show actual approved args and persisted
117
+ ContentDocument results, stale tabs, reload, retries and negative access.
118
+
119
+ Run full package `test`, `test:postgres`, `test:providers`, `build`, `typecheck`,
120
+ `test:components`, `test:e2e`; use the root Node/pnpm versions. Set a short
121
+ `CI_TEST_TMPDIR`/`TMPDIR`, and optionally
122
+ `PLAYWRIGHT_CHROMIUM_EXECUTABLE_PATH` to the supported installed Chromium.
123
+ The evaluation lane remains explicitly unavailable until #3677.
124
+
125
+ The existing CI browser job includes ingestion changes and shared UI primitive
126
+ changes in affected mode, and runs the ingestion browser suite in full mode. Its
127
+ normal dependency build precedes the suite and failed runs upload browser evidence.
128
+
129
+ The maintained host derives inbox state by traversing bounded public review pages,
130
+ authorizing every action/parent/target before classification. It retains only
131
+ scalar totals and the requested page; an off-page denial denies the whole view.
132
+ The component's Load more and subsequent refresh/mutation reload every displayed
133
+ page using fresh cursors, then publish the combined authorized set together. Old
134
+ page payloads are cleared while loading; any unavailable page clears all payloads.
135
+ This keeps plan controls and earlier steps visible across page boundaries without
136
+ reusing a prior authorization decision.
@@ -0,0 +1,132 @@
1
+ # Review and execution integration
2
+
3
+ `/execution` exports `IntakeExecutionService`, handler/host types, policy
4
+ resolution, binding digests and `continueIntakeReview`. `IngestionService` exposes
5
+ these operations when its optional `execution` configuration is present. Its
6
+ loader is lazy: importing `/server` alone does not import the principal stack.
7
+ `/dto` exposes execution messages using type-only exports. No generated REST,
8
+ CLI or MCP mutation endpoint is enabled; applications authenticate their own
9
+ service surface and create `IngestionScope` from the session.
10
+
11
+ ## Catalog and authority
12
+
13
+ Applications supply a trusted `IntakeExecutionOptions.handlers` catalog.
14
+ `OperationHandler` has qualified ID, immutable version, description, object JSON
15
+ schemas for arguments/results, allowlisted `resultModels`, `validate`, `preview`,
16
+ a registered `{model, action, version}` operation and explicit effect/idempotency/
17
+ open-world capability declaration. Preview returns display, exact normalized
18
+ arguments and target revision preconditions. There is no arbitrary tool execution
19
+ or expression evaluation. Reviewer display wraps application fields under `preview`
20
+ and symbolic metadata under `dependencies`; plan steps additionally wrap that
21
+ object under `step` alongside the plan preview. Application display keys cannot
22
+ overwrite framework metadata or be silently overwritten by it. A version change
23
+ requires a new preview and approval.
24
+ `PlanHandler` references a real registered playbook key and resolved definition
25
+ hash; expansion contains only ordered operation handlers and typed earlier-step
26
+ result references. Discovery/proposal generation consumes this catalog rather
27
+ than creating another registry. Semantic model output is never host configuration.
28
+
29
+ Authenticated receipt context captures `capturedCeiling.execution` with principal
30
+ ID and permission/handler/operation ceilings. The execution host's `authorize`
31
+ resolves current grants and application/tenant/source/(optional request) policy
32
+ on the supplied executor, checking the active requester and confidential scope.
33
+ Lists intersect, budgets minimize, review requirements OR, and disabled policy
34
+ wins. Policy versions must change when policy semantics change. Normal operation
35
+ requires human review; machine authorization requires explicit evaluated opt-ins
36
+ at all three authority layers and remains separate provenance from human review.
37
+ The reference handlers always require review; no evaluation claim is made here.
38
+
39
+ `mutationBoundary: 'serialized'` is an explicit host guarantee, not an inferred
40
+ property of a permissions array: grant revocation must serialize with the supplied
41
+ transaction, or the owning mutation must revalidate under its own lock. The host
42
+ must implement that guarantee before enabling handlers. `assertTarget` similarly
43
+ locks current tenant/confidential ownership and revision against concurrent
44
+ writers using the supplied executor. The reference host demonstrates a scoped
45
+ row lock before public `Contents` reads. A current `PrincipalRun` intersects live
46
+ permissions with the captured ceiling and checks the exact registered collection
47
+ operation at every effect. No `runOnce` atomicity is assumed.
48
+
49
+ ## Durable review and effects
50
+
51
+ Preview appends a proposal bound to exact analysis attempt/evidence, handler,
52
+ arguments, policy and target preconditions. Decision requests use unique durable
53
+ request IDs; reusing an ID with changed content is rejected. First valid review
54
+ wins under the item lock. Corrections append a new revision; reject/defer remain
55
+ non-executable. Plan corrections use re-expansion. Human decisions expire and
56
+ live authority is checked again before each step. Raw foundation proposals cannot
57
+ bypass execution's schema, preview, evidence and approval verification.
58
+
59
+ Database handlers must use `context.db` for every domain effect and must perform
60
+ no external I/O. Domain result and successful execution commit together. A domain savepoint rolls failed effects
61
+ back while preserving the item lock and atomically consuming the retry attempt. Completed
62
+ actions survive job cleanup and evidence retention; expired originals leave a
63
+ minimal result digest tombstone, preventing a second effect. Active result reads
64
+ recheck host visibility and result target ownership.
65
+
66
+ External handlers reserve a stable action idempotency key before submission.
67
+ Timeout, process death or ambiguous outcome requires reconciliation; an expired
68
+ lease never means permission to resend. `reconcile` must return authoritative
69
+ `succeeded`, `not_applied` or `unknown`. `not_applied` is safe only when the provider
70
+ can exclude a late acceptance of the original send; uncertainty stays `unknown`.
71
+ The same key is reused after a conclusive non-application. Cancellation prevents
72
+ pending work but cannot retract a send already issued. Applications must arrange
73
+ provider reconciliation before privacy retention removes the binding; a redacted
74
+ unknown action fails closed and cannot automatically execute again.
75
+
76
+ Each plan step has a stable action identity. Completed steps are immutable and
77
+ return their original result. Symbolic record IDs bind to exact earlier proposal
78
+ revision, successful result digest/model and target revision; they cannot point
79
+ across items or form cycles. Failed dependent steps never repeat predecessors.
80
+ The plan preview and target preconditions are part of its digest and visible in
81
+ each step review; changed preview/preconditions require a fresh review.
82
+ Abort/continue follows the pinned public playbook definition, with fresh authority
83
+ at every step.
84
+
85
+ `continueIntakeReview` uses the existing jobs continuation context. Applications
86
+ create tasks with owner, tenant and exact `{recordId, revision, inputKey}` binding
87
+ and configure `authorizeMcpTask` to revalidate it. A continuation answer only wakes
88
+ the task; the service reloads the authoritative decision. Waiting persists without
89
+ an occupied worker and survives runner restart.
90
+
91
+ ## Deployment and evidence
92
+
93
+ Apply normal manifest migrations before runtime. Execution imports the public
94
+ agents/users principal stack, which registers profiles including `ProfileAsset`;
95
+ execution-enabled hosts must migrate their dependency graph (including profile
96
+ asset joins used by asset deletion). `/server` receipt-only imports register none
97
+ of these additional models. Content is not a runtime dependency of ingestion.
98
+ `reference/handlers.ts` is an application/test fixture, excluded from published
99
+ entries and declarations; importing it deliberately adds Content and its asset
100
+ joins, whose migrations the application then owns. No runtime DDL is performed.
101
+
102
+ Real SQLite and PostgreSQL execution suites exercise the public Content draft and
103
+ attachment APIs, transaction rollback, actor/context denials, simultaneous applies
104
+ and reviewers, retention replay, external process death and actual TaskRunner
105
+ restart. See the issue 3674 test matrix for the complete release evidence mapping.
106
+
107
+ ## Layered retention and request automation
108
+
109
+ Every automation-bearing policy layer, including an optional request layer, must
110
+ match the evaluated version. An absent request automation block adds no evaluator
111
+ restriction; it cannot replace the three required application/tenant/source opt-ins.
112
+
113
+ Authorized execution-policy access monotonically narrows the existing intake
114
+ expiry to `min(existing expires_at, item.created_at + resolved retentionMs)`.
115
+ The anchor is receipt creation, never review, retry or replay time. A longer later
116
+ policy never extends a recorded deadline. This hard maximum applies to the whole
117
+ intake item: the existing `sweepRetention` protocol removes retained originals and
118
+ derivatives, preserving replay identities. No second expiry column or hidden clock
119
+ is introduced, and no blob I/O occurs in an execution transaction.
120
+
121
+ At an observed expired deadline, execution reads/completions erase proposal,
122
+ decision, execution, plan, action and feedback payloads under the item lock. Review
123
+ reads return existing identity/hash fields with `state: 'expired'` and empty
124
+ `display`; completed action replay returns its digest-only tombstone. Failed writes, including ordinary validation/CAS/policy rejection and domain
125
+ savepoint rollback, cannot roll back an already-authorized retention ceiling or
126
+ redaction. Restoring a recorded authorized maximum needs no new content-read grant;
127
+ later revocation cannot undo it. Initial denied authorization records no intent.
128
+ Completed replay checks the deadline again after target authorization
129
+ callbacks before returning cached content. Proposal publication and automatic
130
+ authorization also recheck after preview/evaluator callbacks, before publishing
131
+ retained proposal or decision payloads. Unknown
132
+ external outcomes remain unknown/reconcile-only, never eligible for blind resend.