@happyvertical/smrt-ingestion 0.57.20 → 0.57.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (115) hide show
  1. package/FEEDBACK.md +137 -0
  2. package/PROPOSALS.md +146 -5
  3. package/README.md +168 -2
  4. package/REVIEW.md +611 -7
  5. package/agents/execution.md +87 -0
  6. package/dist/chunks/{execution-NoDh3pjs.js → execution-BzVQax0n.js} +528 -42
  7. package/dist/chunks/execution-BzVQax0n.js.map +1 -0
  8. package/dist/chunks/{feedback-CJB900iG.js → feedback-D3UTF8fm.js} +175 -36
  9. package/dist/chunks/feedback-D3UTF8fm.js.map +1 -0
  10. package/dist/chunks/{proposal-errors-VxcTU6Db.js → proposal-contracts-77Uo1B1j.js} +38 -44
  11. package/dist/chunks/proposal-contracts-77Uo1B1j.js.map +1 -0
  12. package/dist/chunks/reviewer-instruction-CEc3AOuU.js +33 -0
  13. package/dist/chunks/reviewer-instruction-CEc3AOuU.js.map +1 -0
  14. package/dist/dto.d.ts +75 -0
  15. package/dist/dto.d.ts.map +1 -1
  16. package/dist/execution-contracts.d.ts +4 -1
  17. package/dist/execution-contracts.d.ts.map +1 -1
  18. package/dist/execution-dto.d.ts +16 -0
  19. package/dist/execution-dto.d.ts.map +1 -1
  20. package/dist/execution-internal.d.ts +2 -0
  21. package/dist/execution-internal.d.ts.map +1 -1
  22. package/dist/execution.d.ts +101 -4
  23. package/dist/execution.d.ts.map +1 -1
  24. package/dist/execution.js +2 -2
  25. package/dist/feedback-dto.d.ts +18 -4
  26. package/dist/feedback-dto.d.ts.map +1 -1
  27. package/dist/feedback-internal.d.ts +9 -0
  28. package/dist/feedback-internal.d.ts.map +1 -1
  29. package/dist/feedback.d.ts +10 -0
  30. package/dist/feedback.d.ts.map +1 -1
  31. package/dist/index.js +2 -2
  32. package/dist/manifest.json +339 -3
  33. package/dist/models.d.ts +15 -0
  34. package/dist/models.d.ts.map +1 -1
  35. package/dist/models.js +82 -2
  36. package/dist/models.js.map +1 -1
  37. package/dist/policy.d.ts +3 -0
  38. package/dist/policy.d.ts.map +1 -1
  39. package/dist/presented-filename.d.ts +11 -0
  40. package/dist/presented-filename.d.ts.map +1 -0
  41. package/dist/proposal-contracts.d.ts +31 -6
  42. package/dist/proposal-contracts.d.ts.map +1 -1
  43. package/dist/proposal-dto.d.ts +17 -0
  44. package/dist/proposal-dto.d.ts.map +1 -1
  45. package/dist/proposal-errors.d.ts +20 -0
  46. package/dist/proposal-errors.d.ts.map +1 -1
  47. package/dist/proposal-validation.d.ts +26 -0
  48. package/dist/proposal-validation.d.ts.map +1 -1
  49. package/dist/proposals.d.ts +6 -1
  50. package/dist/proposals.d.ts.map +1 -1
  51. package/dist/proposals.js +268 -71
  52. package/dist/proposals.js.map +1 -1
  53. package/dist/redaction.d.ts +9 -0
  54. package/dist/redaction.d.ts.map +1 -0
  55. package/dist/review-dto.d.ts +108 -5
  56. package/dist/review-dto.d.ts.map +1 -1
  57. package/dist/review-host.d.ts +52 -3
  58. package/dist/review-host.d.ts.map +1 -1
  59. package/dist/review-instruction-view.d.ts +12 -0
  60. package/dist/review-instruction-view.d.ts.map +1 -0
  61. package/dist/review-summary.d.ts.map +1 -1
  62. package/dist/reviewer-instruction.d.ts +20 -0
  63. package/dist/reviewer-instruction.d.ts.map +1 -0
  64. package/dist/server.d.ts +252 -13
  65. package/dist/server.d.ts.map +1 -1
  66. package/dist/server.js +1190 -85
  67. package/dist/server.js.map +1 -1
  68. package/dist/smrt-knowledge.json +137 -23
  69. package/dist/sources/common.d.ts +11 -0
  70. package/dist/sources/common.d.ts.map +1 -1
  71. package/dist/sources/email.d.ts.map +1 -1
  72. package/dist/sources/http.d.ts.map +1 -1
  73. package/dist/sources/watch-folder.d.ts.map +1 -1
  74. package/dist/svelte/components/ActionReview.svelte +54 -55
  75. package/dist/svelte/components/ActionReview.svelte.d.ts +8 -1
  76. package/dist/svelte/components/ActionReview.svelte.d.ts.map +1 -1
  77. package/dist/svelte/components/CandidatePicker.svelte +72 -0
  78. package/dist/svelte/components/CandidatePicker.svelte.d.ts +15 -0
  79. package/dist/svelte/components/CandidatePicker.svelte.d.ts.map +1 -0
  80. package/dist/svelte/components/EvidenceViewer.svelte +60 -4
  81. package/dist/svelte/components/EvidenceViewer.svelte.d.ts +12 -1
  82. package/dist/svelte/components/EvidenceViewer.svelte.d.ts.map +1 -1
  83. package/dist/svelte/components/InboxView.svelte +172 -19
  84. package/dist/svelte/components/InboxView.svelte.d.ts +36 -2
  85. package/dist/svelte/components/InboxView.svelte.d.ts.map +1 -1
  86. package/dist/svelte/components/IntakeInbox.svelte +6 -3
  87. package/dist/svelte/components/IntakeInbox.svelte.d.ts +4 -1
  88. package/dist/svelte/components/IntakeInbox.svelte.d.ts.map +1 -1
  89. package/dist/svelte/components/IntakeReview.svelte +4 -2
  90. package/dist/svelte/components/IntakeReview.svelte.d.ts +2 -0
  91. package/dist/svelte/components/IntakeReview.svelte.d.ts.map +1 -1
  92. package/dist/svelte/components/IntakeWorkspace.svelte +27 -0
  93. package/dist/svelte/components/IntakeWorkspace.svelte.d.ts +19 -0
  94. package/dist/svelte/components/IntakeWorkspace.svelte.d.ts.map +1 -0
  95. package/dist/svelte/components/ReviewView.svelte +653 -71
  96. package/dist/svelte/components/ReviewView.svelte.d.ts +20 -1
  97. package/dist/svelte/components/ReviewView.svelte.d.ts.map +1 -1
  98. package/dist/svelte/components/TriageView.svelte +669 -0
  99. package/dist/svelte/components/TriageView.svelte.d.ts +16 -0
  100. package/dist/svelte/components/TriageView.svelte.d.ts.map +1 -0
  101. package/dist/svelte/i18n.d.ts +56 -0
  102. package/dist/svelte/i18n.d.ts.map +1 -1
  103. package/dist/svelte/i18n.js +56 -0
  104. package/dist/svelte/index.d.ts +2 -1
  105. package/dist/svelte/index.d.ts.map +1 -1
  106. package/dist/svelte/index.js +1 -0
  107. package/dist/svelte/view-utils.d.ts +67 -0
  108. package/dist/svelte/view-utils.d.ts.map +1 -1
  109. package/dist/svelte/view-utils.js +157 -0
  110. package/package.json +13 -13
  111. package/dist/chunks/execution-NoDh3pjs.js.map +0 -1
  112. package/dist/chunks/execution-internal-DWteDHM2.js +0 -7
  113. package/dist/chunks/execution-internal-DWteDHM2.js.map +0 -1
  114. package/dist/chunks/feedback-CJB900iG.js.map +0 -1
  115. package/dist/chunks/proposal-errors-VxcTU6Db.js.map +0 -1
package/FEEDBACK.md CHANGED
@@ -30,6 +30,110 @@ an authorized existing event; it cannot supply a correctness label or fabricate
30
30
  an execution result. An initial denial does not mutate retention. Once authorized,
31
31
  a narrowed privacy ceiling survives business rollback and later revocation.
32
32
 
33
+ Observed `action_decision` events carry the decision's `decisionMode`
34
+ (`review` or `review_own`) and `selfDecision` flag from the durable decision
35
+ record; `selfDecision` is true whenever the decider submitted the item, in
36
+ either mode. A submitter confirming their own proposal (#4085) is observed as a
37
+ decision by that actor, never as reviewer correctness evidence; consumers that
38
+ derive reviewer agreement must exclude `selfDecision: true` events, and nothing
39
+ in this package turns them into automation eligibility.
40
+
41
+ ## Reviewer routing choices
42
+
43
+ A `reviewer_routing` record is appended inside the owning transaction when
44
+ feedback is configured and `capture` is authorized, at the moment the choice is
45
+ final: `resolveItem` (REVIEW.md) records `choice: 'none'` keyed
46
+ `routing:<requestId>`, and `submitDecision` records `choice: 'handler'` keyed
47
+ `routing:<decision requestId>` when a reviewer **approves** a reviewer-selected
48
+ action. Handler-choice capture requires `mode: 'review'`: a `review_own`
49
+ submitter approval authorizes normally but appends no reviewer-routing example or
50
+ rule support. A reviewer using the reviewer path may still record an explicit
51
+ routing choice for their own receipt; it does not judge any model suggestion.
52
+ Starting an action records nothing; rejecting or deferring it never produces an
53
+ example. The record needs no model suggestion: it binds the current
54
+ completed/partial attempt (`attemptId`, output digest, analysis revision, frozen
55
+ evidence hashes and the extracted text as `query`), the acting reviewer
56
+ (`reviewer`, also `approvedBy` for a handler choice), the actor who started the
57
+ action (`startedBy`), the start reason, and for a handler choice the
58
+ `handlerId@version`, exact approved arguments and the approved action binding
59
+ (revision and binding hash). Denied or oversized capture, an unavailable source
60
+ pin or malformed evidence skip the projection and leave the authoritative
61
+ decision or disposition intact; database write failures still roll everything
62
+ back. There is no
63
+ supersession: a correction after approval (`previewProposal` on the current
64
+ revision) changes the revision, which retires the old record from retrieval and
65
+ support until the new revision is approved and captured again; a correction
66
+ before approval records nothing until that revision is approved.
67
+
68
+ Settledness is a selection-time condition. Once a generation used an example,
69
+ or a rule suggestion stored its support, the lineage re-checks (reading the
70
+ generation, reviewing or deciding its actions, reading or adopting the rule,
71
+ and replaying `suggestRoutingRule`, which returns the stored suggestion) verify:
72
+
73
+ - every record: its digest, that it is not superseded, retention (no
74
+ redaction or deletion), the unchanged source attempt digest and frozen
75
+ evidence hashes, and live `retrieve` authority in the same tenant and
76
+ confidential scope with every generation recipient allowed by the source's
77
+ provider policy;
78
+ - an `interpretation` record (unchanged from #3676): its action's current
79
+ revision, binding hash, attempt, handler@version and arguments; the action's
80
+ target preconditions at their recorded revisions unless the action
81
+ `succeeded`; the prompt, configuration and model versions; and that the action
82
+ is not reviewer-selected;
83
+ - a `reviewer_routing` record with `choice: 'handler'`: its action's current
84
+ revision, binding hash, attempt, handler@version and arguments, that the
85
+ action is still reviewer-selected, and that its targets are still accessible
86
+ (not their revisions: the record means the handler choice, not target state);
87
+ - a `reviewer_routing` record with `choice: 'none'`: it has no action, so the
88
+ source item is re-checked under the first currently offered receiving handler
89
+ that still authorizes it.
90
+
91
+ Rule support (`suggestRoutingRule`, and on read, adoption and replay of a stored
92
+ suggestion) additionally requires: at least two records from distinct source
93
+ items; `judgment: 'correct'` with matching prompt/configuration/model versions
94
+ for an interpretation record; `choice: 'handler'` for a routing record; each
95
+ record's handler@version equal to the receiving binding's; and each source's
96
+ live routing projection (or full arguments when no projection is configured)
97
+ equal to the receiving action's. Settledness of a routing source is checked only
98
+ when the support is first selected.
99
+
100
+ A later disposition, approval lapse, outcome change or target write of a routing
101
+ source never revokes it.
102
+
103
+ Retrieval scans `interpretation` and `reviewer_routing` records together under
104
+ the same bounds and lexical matcher. A routing example is returned only when its
105
+ digest verifies, its attempt and evidence hashes are unchanged, the source item
106
+ is still readable in the same tenant and confidential scope, every generation
107
+ recipient is allowed by the source's live provider policy, and (for a chosen
108
+ handler) the action is still reviewer-selected and settled — applied
109
+ (`executing`, `succeeded`) or `authorized` with an unexpired approval, on an item
110
+ without a disposition — its binding is still the current revision, and the handler with its candidate
111
+ references is currently offered to the receiver. A `none` choice is
112
+ revalidated under the first currently offered receiving handler the source item
113
+ authorizes. Routing examples carry `kind: 'reviewer_routing'`,
114
+ `judgment: 'correct'`, `routing: { choice, reason }`, `handlerId: 'none'` with an
115
+ empty action for closed items, and no model/prompt/configuration versions, since
116
+ no suggestion was judged. They are included in provider input and selection
117
+ provenance exactly like interpretation examples and are rechecked by the same
118
+ lineage gates.
119
+
120
+ `suggestRoutingRule` accepts `reviewer_routing` support only for `choice: 'handler'`
121
+ records whose action is settled in the same sense when the support is chosen
122
+ (a lapsed, unapplied approval or a disposition excludes it) and whose handler, version and
123
+ full arguments match the receiving binding and each other, from distinct source
124
+ items, under the live routing projection; `none`, rejected, deferred and
125
+ re-corrected records never support a rule. Suggestions and adoptions keep every existing #3676
126
+ rule: no catalog, candidate, permission or automation expansion, and
127
+ `automaticActionEligible: false`. Reviewer-selected actions themselves are refused
128
+ by `authorizeAutomatic`. They are routing choices, never labels for a model
129
+ suggestion: `recordFeedback` refuses an interpretation judgment on one (even
130
+ when the item also carries a generation, and whatever its decision), retrieval
131
+ and rule support ignore interpretation records bound to one, and `ActionReview`
132
+ hides the correctness buttons for them. A disposition says nothing about
133
+ whether a suggestion was right, so interpretation judgments on a resolved item
134
+ still count. Only
135
+ `startReviewerAction` mints that origin; `createAction` refuses it.
136
+
33
137
  ## Retrieval and provider boundaries
34
138
 
35
139
  `retrieveFeedback({ itemId, query })` derives the current offered catalog,
@@ -51,6 +155,39 @@ examples. A receiving item's provider grant cannot substitute for a source grant
51
155
  Superseded, revoked, deleted or incompatible sources are excluded. Current explicit human corrections
52
156
  take precedence: a generation carrying one does not retrieve learned examples.
53
157
 
158
+ A judgment recorded on an action whose generation ran under a reviewer
159
+ instruction (REVIEW.md) stores only a reference, `instruction: {id, digest}`,
160
+ after verifying the generation's provenance against the authenticated
161
+ extraction record. **Write side:** the reference is ~100 bytes, so instruction
162
+ text never counts against the judgment's write budget (`feedback.maxBytes` /
163
+ `policy.maxBytes`) and cannot make `recordFeedback` fail with `Feedback
164
+ limit`. `retrieve` re-verifies the source record and joins the text, returning
165
+ `{id, digest, text}`; a stored reference that no longer matches the source's
166
+ record excludes the example. Because one long instruction joined at read time
167
+ could otherwise push every selection that ranks it over budget and deny
168
+ generation scope-wide, `recordFeedback` also builds that single joined example
169
+ exactly as `retrieve` would and refuses with `Feedback instruction exceeds
170
+ retrieval budget` (nothing is written) when it alone exceeds
171
+ `min(feedback.maxBytes, handler policy.maxBytes)`. An example that fits exactly
172
+ records and retrieves. **This guard covers only the single-example case** on the
173
+ judged handler's own budget: retrieval still sums up to `maxExamples` examples
174
+ and takes the minimum across *every* offered handler of the receiving item, so a
175
+ multi-example selection, or a receiving item with a narrower handler budget, can
176
+ still exceed the budget and fails closed as below. It is returned as part of the example so comparable later items see the guidance
177
+ that produced the judged suggestion. It is example content only: it does not
178
+ transfer to the receiving generation's configuration or prompt section, and it
179
+ expands no catalog, candidate, permission or automation eligibility. **Read-side budget
180
+ impact:** joined instruction text (up to 4,000 characters, more bytes for
181
+ non-ASCII) counts toward the exact serialized selection budget — the narrowest
182
+ of `feedback.maxBytes` and each offered handler's `policy.maxBytes` — like any
183
+ other example field. The existing fail-closed rule is unchanged: an over-budget
184
+ selection rejects with `Feedback limit` and generation records
185
+ `provider_error` without provider I/O; examples are not silently trimmed. Hosts
186
+ that enable reviewer instructions together with feedback should size
187
+ `feedback.maxBytes` (and the policy ceilings) for at least
188
+ `maxExamples × (4,000 characters of instruction + the example's own fields)`,
189
+ or lower `maxExamples`.
190
+
54
191
  Full examples exist only in the authorized provider request. The SDK proposal
55
192
  adapter includes that typed selection in its serialized user message when
56
193
  present; disabled feedback leaves the original request shape unchanged. Saved generation
package/PROPOSALS.md CHANGED
@@ -32,7 +32,8 @@ the authorized transaction executor. Query by tenant and confidential scope
32
32
  `limit + 1`) and report `hasMore`. The service projects candidate identity, model,
33
33
  revision and label, then checks every target with the owning execution callback.
34
34
  There are no hidden counts or pagination tokens. Truncated discovery forces
35
- `needs_review` and cannot enter generated preview.
35
+ `needs_review`; a person may still preview it explicitly, and automation never
36
+ becomes eligible.
36
37
 
37
38
  Completed generation reload checks process authority, the current attempt, and
38
39
  owner/candidate visibility in the same final transaction. Database-backed grant
@@ -64,6 +65,12 @@ that declaration and its target authorization semantics.
64
65
  or pass an explicit attempt ID to additionally reject stale client state.
65
66
  Queued/running revisions never return an older completed attempt.
66
67
 
68
+ `analyze` refuses an interpretation configuration whose pinned source is not
69
+ the item's latest extraction, checked under the item lock before anything is
70
+ superseded (`GenerationSourceStaleError`, nothing appended). A preparation
71
+ superseded by a later extraction therefore fails at append time rather than
72
+ after a revision was created.
73
+
67
74
  A generation revision does not rewrite terminal extraction. Reprocessing starts
68
75
  with a new extraction revision, followed by interpretation, while the host keeps
69
76
  the same business intention keys. An interrupted still-current generation lease
@@ -78,6 +85,25 @@ Expired, cancelled, revoked, retried or superseded workers cannot read or publis
78
85
  current lease and each offered handler/candidate are rechecked immediately before
79
86
  each outbound generative/decision boundary and before publication.
80
87
 
88
+ ### Inline interpretation
89
+
90
+ Receipts created with `dispatch: 'inline'` (text/structured parts only) run the
91
+ same stages in-process through `interpretInline(itemId, { requestId, extraction
92
+ })`: an immutable extraction revision claimed under a fenced lease and extracted
93
+ with the built-in UTF-8 extractor, then `prepareGeneration` and a fenced
94
+ interpretation revision generated by `generateProposals`. The analysis lease,
95
+ catalog digest and live catalog/candidate checks, source pin, provenance and
96
+ `StageFailure` categories are identical to the job path; no dispatch intent is
97
+ ever deliverable and nothing is enqueued. A retry with the same `requestId`
98
+ replays the recorded revisions rather than re-preparing against a superseded
99
+ source, also under concurrency. The interpretation append rechecks, under the
100
+ item lock, that its pinned extraction is still the latest revision and refuses
101
+ a superseded pin: first through the general latest-extraction check that
102
+ every interpretation append runs (`GenerationSourceStaleError`), then by the
103
+ inline check that the pinned attempt is still current (`Generation source changed`). Generated suggestions still require explicit preview and a human
104
+ decision, either through the reviewer surface or the submitter-scoped surface
105
+ described in [REVIEW.md](REVIEW.md).
106
+
81
107
  ## Providers and validation
82
108
 
83
109
  `createSDKProposalGenerator(client, identity, {maxTokens,timeoutMs})` uses an
@@ -99,6 +125,33 @@ for human preview; automatic eligibility always remains false. The service
99
125
  deadline aborts requests and prevents late publication; it does not claim to terminate computation at a remote
100
126
  provider or isolate native allocations. Extractor process limits remain separate.
101
127
 
128
+ When the pinned extraction carries an authenticated reviewer instruction
129
+ (REVIEW.md), `prepareGeneration` copies it into the interpretation configuration
130
+ and `generateProposals` verifies that copy against the extraction record and its
131
+ own digest before provider disclosure (`integrity` otherwise). The prompt then
132
+ carries it in its own **trusted reviewer guidance** section after the fixed
133
+ system instruction and handler contract: the guidance is named as authenticated
134
+ human input, limited to the offered catalog, and the evidence remains untrusted
135
+ data. `GenerativeProposalInput.reviewerInstruction` exposes `{id, text}` to custom
136
+ generators; `provenance.reviewerInstruction` records `{id, digest}`. The
137
+ instruction never changes `offered`, `catalogDigest`, argument schemas,
138
+ permissions or `automaticActionEligible`; a reply that follows guidance outside
139
+ the catalog fails validation like any other hostile output. **Budget cost:** the instruction counts once against the input budget
140
+ (`maxInputBytes`, each offered handler's `maxBytes` and the live policy
141
+ ceiling), as its JSON-escaped text inside the prompt's guidance section — up
142
+ to 4,000 characters plus escaping and the fixed guidance sentence. The
143
+ `reviewerInstruction` field duplicated for custom generators is not counted
144
+ again. The decision stage also carries it once: its request state holds the
145
+ evidence-side input with the un-guided base instructions under `evidence`, and
146
+ the guidance only under a separate `reviewerGuidance` key, never under
147
+ `evidence`; the decision budget measures that serialized request as sent. Size `maxInputBytes` with that headroom. Learned examples are
148
+ still retrieved for an instructed generation; only a logical split correction
149
+ suppresses them. Every instruction costs a fresh extraction revision before the
150
+ new interpretation: the predecessor-pin contract above (source = immediately
151
+ preceding revision, current-revision reads) is deliberately left intact rather
152
+ than reused for a same-source second interpretation, so OCR/speech hosts budget
153
+ one extraction per instruction, as for split corrections.
154
+
102
155
  Model output can contain only offered handler versions, schema-valid arguments,
103
156
  retained evidence locations and offered alternative handler/candidate keys.
104
157
  Candidate IDs must occur in the offered set, and evidence IDs in retained source
@@ -108,6 +161,26 @@ as `missingFields`, with `needs_review`; values are never invented to satisfy a
108
161
  schema. Observed page/time/source granularity is preserved. Logical scan splits
109
162
  use `proposeDocumentSplits`; originals and hashes remain intact.
110
163
 
164
+ A rejected reply ends the attempt as `provider_error` with the `malformed_output`
165
+ warning and a bounded, value-free `provenance.failure` `{category, reason}` (#4207).
166
+ The reason lists schema-owned paths, keywords and schema constants, for example
167
+ `/outcome enum (allowedValues=proposals|unknown|...)` and
168
+ `/suggestions/0 required (missingProperty=evidence)`, or the fixed internal check
169
+ and its path, such as `Unoffered handler: /suggestions/0`. Path segments are
170
+ kept only when they are keys declared under the schema's `properties` or
171
+ indexes into a value that actually is an array in the rejected reply; any other
172
+ segment, numeric or not (for example a key under an open
173
+ `additionalProperties` argument sub-schema, or a numeric key of an object),
174
+ becomes `?`, and an extra property's name is never echoed.
175
+ It never contains document text, argument values, evidence content or
176
+ model-chosen keys, is cut at 1000 characters,
177
+ and is dropped before the output-limit fallback. Hosts read it from the
178
+ completed/partial analysis at `analysis.result.output.proposals.provenance.failure`.
179
+ The SDK adapter constrains replies with a `json_schema` response format built
180
+ from the validator's schema and the fixed instructions describe the shape; a
181
+ custom `ProposalGenerator` receives no schema, so its raw reply meets the
182
+ validator directly and the same reason is recorded.
183
+
111
184
  `createSDKProposalDecisionClient(client)` adapts an existing SDK client’s optional
112
185
  capability without changing its errors or protocol. Optional `decision` uses public core `executeDecision` and the existing SDK
113
186
  predicate, choice, score and batch contracts. Supported-effect predicates,
@@ -117,15 +190,76 @@ separately from generative provenance and unknown OCR certainty. The complete
117
190
  serialized decision request, including suggestions and questions, must fit the
118
191
  minimum configured/current policy byte ceiling before capability probing and
119
192
  again before deciding. A policy change between these boundaries fails closed.
120
- Ties, support probability or selected-handler confidence below
121
- `minimumDecisionProbability`, and actually incomplete extraction require review.
122
- Unknown provider
193
+ Actually incomplete extraction always requires review. Unknown provider
123
194
  completion metadata is preserved as `source_completion_unknown`; it does not
124
195
  block otherwise schema-valid **human** preview or imply completeness. Automatic
125
196
  action eligibility remains false. An unconfigured decision path
126
197
  is recorded explicitly. Configured errors produce a durable safe failure outcome;
127
198
  they never silently switch to a generative decision fallback.
128
199
 
200
+ ### Decision readiness gate (#4209)
201
+
202
+ For every suggestion `i` the request asks `supported_i` (predicate),
203
+ `route_i` (choice over the offered `handlerId@version` keys plus `none`),
204
+ `risk_i` (ordered score over `clear`, `uncertain`, `ambiguous`) and one
205
+ `entity_i_<field>` choice per candidate-reference argument. The validated
206
+ answers are always persisted under `provenance.decision.answers`. With the
207
+ default `decisionReadiness: 'gate'` they are read as follows, and any failing
208
+ check sets the suggestion to `needs_review` (a tie also sets the outcome to
209
+ `ambiguous`):
210
+
211
+ - `supported.probability >= minimumDecisionProbability` (default `0.8`).
212
+ - `route.choice` equals the suggestion's own `handlerId@handlerVersion`,
213
+ `route.confidence >= minimumDecisionProbability`, and the gap between the two
214
+ largest route probabilities is greater than `tieMargin` (default `0.1`).
215
+ - `risk.score < 1`: only the `clear` level passes; `uncertain` and `ambiguous`
216
+ require review.
217
+ - Each entity match chooses the suggested candidate's key with confidence at or
218
+ above the threshold and a gap above `tieMargin`.
219
+
220
+ These values are provider outputs read as if they were calibrated
221
+ probabilities; nothing in this package calibrates them for this use. The
222
+ teamworks-os measurement in #4209 (self-hosted Laya `typed-decisions`) found no
223
+ separation between right and wrong suggestions: a correctly read vendor invoice
224
+ scored `supported` 0.37, garbage input 0.45–0.50, a wrongly proposed purchase
225
+ order 0.51, route confidence 0.003–0.15 and `risk` ≥ 0.94, so the gate sent
226
+ every suggestion to review. Until a labelled sample shows separation, treat the
227
+ answers as diagnostics and configure:
228
+
229
+ - `decisionMaxLen`: positive integer forwarded unchanged as the per-request
230
+ `maxLen` of the decision client (the SDK Laya provider's token window; the
231
+ checkpoint default is 1024 and the serialized evidence plus suggestions is
232
+ otherwise truncated). Other clients ignore it.
233
+ - `decisionReadiness: 'record'`: the decision still runs, is bounded and
234
+ authorized exactly as before and its answers, usage, `readiness` and `maxLen`
235
+ are recorded, but the gate never changes `outcome` or `disposition`; readiness
236
+ comes from the validator alone (`missingFields`, outcome, source completeness).
237
+ A client error or timeout is recorded as the `decision_unavailable`
238
+ warning. A reply that comes back and then fails the core `executeDecision`
239
+ identity or answer-shape check is recorded as `decision_malformed` with the
240
+ reply's usage; this is the path for injected `DecisionClient`s. SDK clients,
241
+ including the Laya provider behind `createSDKProposalDecisionClient`,
242
+ validate the reply's shape inside `client.decide` and throw, so a malformed
243
+ SDK reply is recorded as `decision_unavailable` with no usage. Every case
244
+ records the `provenance.decision` identity. The generated suggestions stay
245
+ as they were. The serialized request exceeding the byte budget is a
246
+ configuration failure, not a provider answer: it ends the attempt as `limit`
247
+ in both modes, and in `gate` mode every failure above still ends the attempt
248
+ as `provider_error`. Lease, catalog, policy and provider-allowlist revocation
249
+ fail closed in both modes.
250
+
251
+ Successful output also records `provenance.decision.readiness` (`'gate'` or
252
+ `'record'`) and `maxLen` when set; configuration digests are unchanged, but
253
+ persisted output is therefore not byte-identical to earlier releases.
254
+
255
+ Both settings are public configuration: they are part of
256
+ `ProposalReadConfiguration`, the frozen `GenerationStageConfiguration.proposals`
257
+ (present only when set, so existing digests are unchanged) and the completed
258
+ snapshot authority check. Invalid values fail at proposal service construction.
259
+ Neither mode affects automation: `automaticActionEligible` stays `false`, and
260
+ `authorizeAutomatic` reads only the host's `evaluateAutomatic` callback and
261
+ policy, never these answers.
262
+
129
263
  Output records zero, one or multiple suggestions, alternatives, missing fields,
130
264
  source links, proposed splits, versions, allowlisted usage, warnings and explicit
131
265
  `unknown`, `ambiguous`, `no_action`, `needs_review` or `provider_error` outcomes.
@@ -154,7 +288,14 @@ await service.previewGeneratedProposals({
154
288
 
155
289
  `intentionKey` is the host's stable business-intention identity across generation
156
290
  revisions. Never derive a fresh key from model output, suggestion order or a retry.
157
- Selections must reference current persisted validated `ready_for_review` output.
291
+ Selections must reference current persisted validated `ready_for_review` or
292
+ `needs_review` output (#4208); a `provider_error` attempt, an unoffered handler or
293
+ a non-interpretation stage is rejected. A `needs_review` suggestion whose
294
+ `missingFields` are still absent fails the handler schema inside the ordinary
295
+ preview (`Invalid handler payload`) and publishes nothing; the reviewer supplies
296
+ the missing values through the explicit human path (`previewProposal` /
297
+ `previewPlan`, as the published review host does for `preview({args})`), and the
298
+ handler's `validate` and `preview` run again on the completed arguments.
158
299
  Each batch requires distinct suggestion indexes and intention keys; duplicate
159
300
  indexes are rejected before any operation or plan preview is written.
160
301
  The service rechecks current catalog/schema/target/source provenance, then calls
package/README.md CHANGED
@@ -10,7 +10,25 @@ methods through their authenticated application transport.
10
10
 
11
11
  Before starting a service, deploy the package manifest through the public s-m-r-t
12
12
  migration APIs or `smrt db:migrate`. `assertReady()` only checks existing tables.
13
- SQLite and PostgreSQL are supported; other engines fail construction.
13
+ SQLite and PostgreSQL are supported; other engines fail construction. Index
14
+ additions ship the same way: `intake_items_received_idx` (newest-first inbox
15
+ paging on `tenant_id, confidential_scope_id, created_at, id`) is declared on the model and reaches existing deployments through the
16
+ next manifest migration; there are no hand-written migration files in this
17
+ package, and `smrt doctor --db` / `db:status --parity` report the index until
18
+ it is applied.
19
+
20
+ `listItemsPage({ limit?, cursor?, state?, itemIds?, scanLimit? })` returns
21
+ `{ items: [{ item, state, cursor }], scanned, nextCursor? }`: one authorized
22
+ inbox page newest receipt first with each row's durable inbox state and
23
+ continuation cursor, and the number of rows the call examined. `listItems()`
24
+ remains as a deprecated array-returning method that walks the private page
25
+ scan to the end. Without the optional `cursorKey` service option every cursor
26
+ names the last row the actor can read, and a page that finds no readable row
27
+ within its scan limit fails with `IntakeScanLimitError`; with a server secret
28
+ of at least 32 bytes, cursors are sealed and such a page resumes past what it
29
+ skipped. `scanned` is for host
30
+ accounting only. See [REVIEW.md](REVIEW.md) for the paging and authorization
31
+ contract.
14
32
 
15
33
  A host supplies a database, AssetRuntime, authenticated tenant/actor/confidential
16
34
  scope, a live authorization callback, a registered jobs target, and an idempotent
@@ -20,7 +38,29 @@ transaction executor; it must intersect current grants with that ceiling. Never
20
38
  construct the scope, retention policy or budgets from incoming document labels.
21
39
 
22
40
  `receive` accepts bounded original byte buffers, with stable source part IDs,
23
- parent relationships and optional upstream identity/version. Source adapters own
41
+ parent relationships, optional upstream identity/version and an optional
42
+ original `filename` (kept for labels on `IntakeEvidenceDTO.filename`, never
43
+ part of receipt identity). The multipart upload source, email attachments and
44
+ the watched folder all supply it, as may a host `snapshotReference` part or a
45
+ vendor webhook's verified parts. Every source delivers through one place that
46
+ removes control, format (bidi, zero-width, tag), line-separator and
47
+ default-ignorable characters (Unicode properties, see
48
+ `src/presented-filename.ts`) and drops a name that is then empty or over 512
49
+ characters, so an odd name falls back to the default label instead of failing
50
+ the receipt (`receive` itself rejects such a name from a direct caller as
51
+ `invalid`, and names are cleaned again when read back). Removing every format
52
+ and default-ignorable character has display-only effects, since nothing uses
53
+ the name as identity: emoji ZWJ sequences and presentation selectors flatten,
54
+ ZWNJ/ZWJ are removed from Persian and Indic text (letters may join
55
+ differently), and ideographic variation selectors are removed (a CJK variant
56
+ shows its base glyph). Unicode normalization
57
+ (NFC/NFKC) and confusable look-alike characters are out of scope: the name is
58
+ still untrusted text, so render it as text, never as markup or a path. The
59
+ upstream `sourceReference` is stored but not exposed on the DTO: its ids are
60
+ upstream locators, not display text, and they are never cleaned. For email
61
+ they are the provider's message locator (plus the attachment part id); for the
62
+ watched folder the id is the raw folder entry name, whose cleaned form is what
63
+ `filename` exposes. Source adapters own
24
64
  streaming and upstream snapshot retrieval. Receipt identity is tenant + source +
25
65
  delivery key; changing bytes or authenticated scope conflicts. Content equality
26
66
  in a different conversation does not deduplicate an arrival. The returned item
@@ -157,11 +197,22 @@ Mount the `/svelte` inbox and review components through an authenticated host us
157
197
  the browser-safe `IntakeReviewHost` DTO contract. See [REVIEW.md](REVIEW.md) for
158
198
  exact revision-bound callbacks, live evidence access, logical split corrections,
159
199
  assignment ownership, and the maintained upload-to-domain-result browser fixture.
200
+ `IntakeWorkspace` puts the inbox, the selected item with its original inline,
201
+ the message box and approve-and-advance on one page with keyboard next/previous;
202
+ it uses the same callbacks and adds no authority (REVIEW.md, "Single-page triage
203
+ workspace").
160
204
 
161
205
  Optional explicit feedback and scoped example retrieval are documented in
162
206
  [FEEDBACK.md](FEEDBACK.md). Approval and execution outcomes remain separate from
163
207
  interpretation correctness; routing policy adoption is explicit and versioned.
164
208
 
209
+ Reviewers can start any offered handler for an item (`startReviewerAction`) or
210
+ close it as handled without action (`resolveItem`); the published host adapter
211
+ wires both as `startAction`/`resolve` and exposes the offered `catalog`. Hosts
212
+ with their own transport add the two POST operations behind the same session,
213
+ CSRF and reviewer checks as `decide`. The `intake_dispositions` table requires
214
+ a manifest migration before upgrading.
215
+
165
216
  ## Recipe preview
166
217
 
167
218
  `ingestion.inbox` and `ingestion.review` declare the user-facing inbox and
@@ -208,6 +259,121 @@ and `maxSuggestions` are each capped at 20; `timeoutMs` is capped at 2147483647.
208
259
  Invalid limits are rejected when constructing `IngestionService`, before receipt
209
260
  or interpretation. See [REVIEW.md](REVIEW.md) for the published server review host.
210
261
 
262
+ ## Trusted internal sources
263
+
264
+ A host with a trusted internal source, such as chat messages recorded in
265
+ smrt-messages, can receive **by reference**, interpret **inline**, and let the
266
+ **submitter** confirm their own proposal (#4085). None of these modes changes
267
+ copied receipts: their identity, validation, queueing and retention are
268
+ untouched, and a replay that switches mode conflicts.
269
+
270
+ **Receive by reference.** A part may be `{ partId, reference: { owner, id,
271
+ version? }, snapshot: { text } | { structured } }` instead of bytes. The
272
+ service derives UTF-8 bytes once (`text/plain` or canonical `application/json`;
273
+ text, reference fields and owner identity containing NUL or unpaired
274
+ surrogates are rejected `invalid` on every engine, and `expireOwned` refuses
275
+ them, so distinct owners can never collapse onto one stored key),
276
+ hashes them into the receipt identity, and stores only that snapshot in the
277
+ durable evidence row; no AssetStore object or asset record is created, and
278
+ `readEvidence` serves the snapshot after the same hash/length check. The
279
+ reference is identity and provenance, never authority. Such parts must be
280
+ text or structured; files still arrive as copied bytes.
281
+
282
+ **Owner-bound retention.** `retention: { kind: 'owner', owner: { kind, id },
283
+ version, expiresAt, replayUntil, acceptAfter }` binds the receipt to its
284
+ owner's lifecycle. The time deadlines remain required and validated exactly as
285
+ before: they are the hard privacy ceiling, and `sweepRetention` and execution's
286
+ policy ceiling still apply. When the owner is deleted the host calls
287
+ `expireOwned({ kind, id })` from its deletion path; every active receipt bound
288
+ to that owner is revoked and removed through the existing `expire` /
289
+ `repairDeletions` protocol, which redacts the snapshot together with every
290
+ other payload, and the deleted delivery is never re-admitted. The owner key is
291
+ recorded in the indexed `owner_key` column (`<kind>:<id>`, so `kind` cannot
292
+ contain `:`). Deletion redaction blanks `owner_key` and `submitter_id` together
293
+ with every payload: only the receipt identity (`delivery_key`) and the captured
294
+ ceiling survive, and the ceiling loses `execution.principalId` unless an
295
+ external action is still `executing` or `outcome_unknown`. So once nothing is
296
+ left to reconcile, a redacted row no longer links an owner to an actor, and a
297
+ repeated `expireOwned` finds nothing. Redaction also overwrites every
298
+ identity- or content-derived digest (`payload_digest`, `scope_digest`, analysis
299
+ `input_digest`, attempt `output_digest`, plan `digest` and proposal, decision
300
+ and execution `binding_hash`) with the fixed marker `redacted`; successful
301
+ result digests remain as the replay tombstone. The principal is retained only
302
+ for an external action that was still in flight when the item was redacted,
303
+ because reconciling it authorizes under that principal; it is dropped when that
304
+ reconciliation reaches a final state, or by the next `repairDeletions` /
305
+ `sweepRetention` pass. For a self-approvable receipt the principal is the
306
+ submitter, so until then it identifies the sender. Later deletion sweeps pass
307
+ this redacted ceiling to the host's `authorize({ operation: 'delete' })`, so
308
+ delete authorization must not depend on `execution.principalId`: a host that
309
+ requires it would refuse, and the sweep would stop with later deletion intents
310
+ still pending. A successful result digest
311
+ is a plain SHA-256 of the canonical result, so it can confirm a guessed result:
312
+ keep handler results to record ids or opaque references, never content.
313
+
314
+ A replay of a deleted delivery is rejected and never becomes fresh work. While
315
+ deletion is still pending it returns `rejected/expired`; once redaction has
316
+ completed the identity digests are gone, so it returns `rejected/conflict`.
317
+ Sources treat both as terminal and must not retry. `delivery_key` stays in plaintext because it is
318
+ the unique receipt identity and the package holds no secret to key it: hosts
319
+ must use an opaque value (for example a random UUID recorded with the message)
320
+ and must not encode owner ids, sender ids or content in `deliveryKey`.
321
+
322
+ Host ordering for owner deletion: first stop issuing receipts for the owner,
323
+ then call `expireOwned`, and call it again once in-flight receipts settle (or
324
+ from a periodic sweep). A receipt still in flight when its owner is expired
325
+ returns `{ kind: 'rejected', category: 'expired' }` rather than `accepted`, but
326
+ one that reserved its row after the first `expireOwned` is only found by the
327
+ later call; until then the time ceiling bounds it. Services are confidential-scope bound, so
328
+ a host whose owner has receipts in several confidential scopes calls
329
+ `expireOwned` from each scope's service; a foreign tenant or scope sees zero.
330
+
331
+ **Inline mode.** `receive({ ..., dispatch: 'inline' })` requires text/structured
332
+ parts only, records the route provenance but **no deliverable dispatch intent**
333
+ for any revision (every intent is retained as `completed`, so no worker or
334
+ `repairDispatches` ever picks it up), and does not enqueue. The host then calls
335
+ `interpretInline(itemId, { requestId, extraction: { configurationRevision,
336
+ limits } })`, which runs in-process exactly what the job path runs: an
337
+ immutable `stage: 'extract'` revision claimed under a fenced lease and
338
+ extracted with the built-in UTF-8 extractor, then `prepareGeneration` and the
339
+ `stage: 'interpret'` revision with the same analysis lease, catalog-live
340
+ checks, provenance and failure categories as `generateProposals`. A retry with
341
+ the same `requestId` replays the immutable revisions, including when two
342
+ identical calls race. The interpretation is appended under the item lock only
343
+ while its pinned extraction is still the latest revision; if another request
344
+ reprocessed the item meanwhile it fails with `GenerationSourceStaleError` (the
345
+ general latest-extraction check every interpretation append runs, #4228) or,
346
+ for a pin whose extraction attempt was replaced, `Generation source changed`. The result reports each
347
+ stage's attempt and safe error and, when generation completed, the authorized
348
+ `getCompletedAnalysis` snapshot. Inline is for send-triggered suggestions that
349
+ must not depend on the scheduler; it is not a bypass of review or execution.
350
+
351
+ **Submitter-scoped approval.** The service records `submitter_id` as the
352
+ authenticated `scope.actorId` of the receiving service, server-side; payload
353
+ and source metadata never supply it. `listOwnProposals({ cursor?, limit? })`
354
+ lists only this actor's own active submissions with their saved review pages,
355
+ reloaded under current item/handler/target authority, without the reviewer
356
+ inbox. A submission the actor can no longer read is skipped before projection
357
+ and never named by the cursor; a readable one whose reviews are denied by
358
+ ingestion authority is returned as `{ item, reviews: null, unavailable: true }`
359
+ with no reason, so one revoked grant never hides the others. `submitOwnDecision(input)` accepts `approve`, `reject` or `defer` (never
360
+ `correct`) under the `review_own` authority: the host's execution `authorize`
361
+ must return `submitter: true` for that actor, the application policy must opt
362
+ in with `submitterReview: true` (narrower layers may only disable it), and the
363
+ item's recorded submitter must equal the current actor. Every other check is
364
+ the ordinary one, and `applyAction` remains the single once-only effect path
365
+ under the approver's live grants. Because the effect runs as the receipt's
366
+ captured `capturedCeiling.execution.principalId`, `review_own` additionally
367
+ requires that principal to be the submitter itself (`submitter_id ===
368
+ scope.actorId === principalId`), checked under the item lock at decision time
369
+ and again at apply; a receipt captured for a bot or service principal can only
370
+ be approved by a reviewer. The decision is recorded with `mode` (`review` or
371
+ `review_own`) and `selfDecision`, which is true whenever the deciding actor is
372
+ the item's submitter, in either mode (mirrored as `decisionMode` /
373
+ `selfDecision` on feedback events), so a self-confirmation is never mistaken
374
+ for independent review and never contributes to automation eligibility. See
375
+ [REVIEW.md](REVIEW.md) and [execution integration](agents/execution.md).
376
+
211
377
  ## Dispatch route ownership and legacy recovery
212
378
 
213
379
  A receipt retains a versioned, server-owned `jobTarget` snapshot, and every