@happyvertical/smrt-ingestion 0.57.19 → 0.57.21

package/FEEDBACK.md

@@ -51,6 +51,39 @@ examples. A receiving item's provider grant cannot substitute for a source grant
 Superseded, revoked, deleted or incompatible sources are excluded. Current explicit human corrections
 take precedence: a generation carrying one does not retrieve learned examples.
 
+A judgment recorded on an action whose generation ran under a reviewer
+instruction (REVIEW.md) stores only a reference, `instruction: {id, digest}`,
+after verifying the generation's provenance against the authenticated
+extraction record. **Write side:** the reference is ~100 bytes, so instruction
+text never counts against the judgment's write budget (`feedback.maxBytes` /
+`policy.maxBytes`) and cannot make `recordFeedback` fail with `Feedback
+limit`. `retrieve` re-verifies the source record and joins the text, returning
+`{id, digest, text}`; a stored reference that no longer matches the source's
+record excludes the example. Because one long instruction joined at read time
+could otherwise push every selection that ranks it over budget and deny
+generation scope-wide, `recordFeedback` also builds that single joined example
+exactly as `retrieve` would and refuses with `Feedback instruction exceeds
+retrieval budget` (nothing is written) when it alone exceeds
+`min(feedback.maxBytes, handler policy.maxBytes)`. An example that fits exactly
+records and retrieves. **This guard covers only the single-example case** on the
+judged handler's own budget: retrieval still sums up to `maxExamples` examples
+and takes the minimum across *every* offered handler of the receiving item, so a
+multi-example selection, or a receiving item with a narrower handler budget, can
+still exceed the budget and fails closed as below. It is returned as part of the example so comparable later items see the guidance
+that produced the judged suggestion. It is example content only: it does not
+transfer to the receiving generation's configuration or prompt section, and it
+expands no catalog, candidate, permission or automation eligibility. **Read-side budget
+impact:** joined instruction text (up to 4,000 characters, more bytes for
+non-ASCII) counts toward the exact serialized selection budget — the narrowest
+of `feedback.maxBytes` and each offered handler's `policy.maxBytes` — like any
+other example field. The existing fail-closed rule is unchanged: an over-budget
+selection rejects with `Feedback limit` and generation records
+`provider_error` without provider I/O; examples are not silently trimmed. Hosts
+that enable reviewer instructions together with feedback should size
+`feedback.maxBytes` (and the policy ceilings) for at least
+`maxExamples × (4,000 characters of instruction + the example's own fields)`,
+or lower `maxExamples`.
+
 Full examples exist only in the authorized provider request. The SDK proposal
 adapter includes that typed selection in its serialized user message when
 present; disabled feedback leaves the original request shape unchanged. Saved generation

package/PROPOSALS.md

@@ -32,7 +32,8 @@ the authorized transaction executor. Query by tenant and confidential scope
 `limit + 1`) and report `hasMore`. The service projects candidate identity, model,
 revision and label, then checks every target with the owning execution callback.
 There are no hidden counts or pagination tokens. Truncated discovery forces
-`needs_review` and cannot enter generated preview.
+`needs_review`; a person may still preview it explicitly, and automation never
+becomes eligible.
 
 Completed generation reload checks process authority, the current attempt, and
 owner/candidate visibility in the same final transaction. Database-backed grant
@@ -64,6 +65,12 @@ that declaration and its target authorization semantics.
    or pass an explicit attempt ID to additionally reject stale client state.
    Queued/running revisions never return an older completed attempt.
 
+`analyze` refuses an interpretation configuration whose pinned source is not
+the item's latest extraction, checked under the item lock before anything is
+superseded (`GenerationSourceStaleError`, nothing appended). A preparation
+superseded by a later extraction therefore fails at append time rather than
+after a revision was created.
+
 A generation revision does not rewrite terminal extraction. Reprocessing starts
 with a new extraction revision, followed by interpretation, while the host keeps
 the same business intention keys. An interrupted still-current generation lease
@@ -99,6 +106,33 @@ for human preview; automatic eligibility always remains false. The service
 deadline aborts requests and prevents late publication; it does not claim to terminate computation at a remote
 provider or isolate native allocations. Extractor process limits remain separate.
 
+When the pinned extraction carries an authenticated reviewer instruction
+(REVIEW.md), `prepareGeneration` copies it into the interpretation configuration
+and `generateProposals` verifies that copy against the extraction record and its
+own digest before provider disclosure (`integrity` otherwise). The prompt then
+carries it in its own **trusted reviewer guidance** section after the fixed
+system instruction and handler contract: the guidance is named as authenticated
+human input, limited to the offered catalog, and the evidence remains untrusted
+data. `GenerativeProposalInput.reviewerInstruction` exposes `{id, text}` to custom
+generators; `provenance.reviewerInstruction` records `{id, digest}`. The
+instruction never changes `offered`, `catalogDigest`, argument schemas,
+permissions or `automaticActionEligible`; a reply that follows guidance outside
+the catalog fails validation like any other hostile output. **Budget cost:** the instruction counts once against the input budget
+(`maxInputBytes`, each offered handler's `maxBytes` and the live policy
+ceiling), as its JSON-escaped text inside the prompt's guidance section — up
+to 4,000 characters plus escaping and the fixed guidance sentence. The
+`reviewerInstruction` field duplicated for custom generators is not counted
+again. The decision stage also carries it once: its request state holds the
+evidence-side input with the un-guided base instructions under `evidence`, and
+the guidance only under a separate `reviewerGuidance` key, never under
+`evidence`; the decision budget measures that serialized request as sent. Size `maxInputBytes` with that headroom. Learned examples are
+still retrieved for an instructed generation; only a logical split correction
+suppresses them. Every instruction costs a fresh extraction revision before the
+new interpretation: the predecessor-pin contract above (source = immediately
+preceding revision, current-revision reads) is deliberately left intact rather
+than reused for a same-source second interpretation, so OCR/speech hosts budget
+one extraction per instruction, as for split corrections.
+
 Model output can contain only offered handler versions, schema-valid arguments,
 retained evidence locations and offered alternative handler/candidate keys.
 Candidate IDs must occur in the offered set, and evidence IDs in retained source
@@ -108,6 +142,26 @@ as `missingFields`, with `needs_review`; values are never invented to satisfy a
 schema. Observed page/time/source granularity is preserved. Logical scan splits
 use `proposeDocumentSplits`; originals and hashes remain intact.
 
+A rejected reply ends the attempt as `provider_error` with the `malformed_output`
+warning and a bounded, value-free `provenance.failure` `{category, reason}` (#4207).
+The reason lists schema-owned paths, keywords and schema constants, for example
+`/outcome enum (allowedValues=proposals|unknown|...)` and
+`/suggestions/0 required (missingProperty=evidence)`, or the fixed internal check
+and its path, such as `Unoffered handler: /suggestions/0`. Path segments are
+kept only when they are keys declared under the schema's `properties` or
+indexes into a value that actually is an array in the rejected reply; any other
+segment, numeric or not (for example a key under an open
+`additionalProperties` argument sub-schema, or a numeric key of an object),
+becomes `?`, and an extra property's name is never echoed.
+It never contains document text, argument values, evidence content or
+model-chosen keys, is cut at 1000 characters,
+and is dropped before the output-limit fallback. Hosts read it from the
+completed/partial analysis at `analysis.result.output.proposals.provenance.failure`.
+The SDK adapter constrains replies with a `json_schema` response format built
+from the validator's schema and the fixed instructions describe the shape; a
+custom `ProposalGenerator` receives no schema, so its raw reply meets the
+validator directly and the same reason is recorded.
+
 `createSDKProposalDecisionClient(client)` adapts an existing SDK client’s optional
 capability without changing its errors or protocol. Optional `decision` uses public core `executeDecision` and the existing SDK
 predicate, choice, score and batch contracts. Supported-effect predicates,
@@ -117,15 +171,76 @@ separately from generative provenance and unknown OCR certainty. The complete
 serialized decision request, including suggestions and questions, must fit the
 minimum configured/current policy byte ceiling before capability probing and
 again before deciding. A policy change between these boundaries fails closed.
-Ties, support probability or selected-handler confidence below
-`minimumDecisionProbability`, and actually incomplete extraction require review.
-Unknown provider
+Actually incomplete extraction always requires review. Unknown provider
 completion metadata is preserved as `source_completion_unknown`; it does not
 block otherwise schema-valid **human** preview or imply completeness. Automatic
 action eligibility remains false. An unconfigured decision path
 is recorded explicitly. Configured errors produce a durable safe failure outcome;
 they never silently switch to a generative decision fallback.
 
+### Decision readiness gate (#4209)
+
+For every suggestion `i` the request asks `supported_i` (predicate),
+`route_i` (choice over the offered `handlerId@version` keys plus `none`),
+`risk_i` (ordered score over `clear`, `uncertain`, `ambiguous`) and one
+`entity_i_<field>` choice per candidate-reference argument. The validated
+answers are always persisted under `provenance.decision.answers`. With the
+default `decisionReadiness: 'gate'` they are read as follows, and any failing
+check sets the suggestion to `needs_review` (a tie also sets the outcome to
+`ambiguous`):
+
+- `supported.probability >= minimumDecisionProbability` (default `0.8`).
+- `route.choice` equals the suggestion's own `handlerId@handlerVersion`,
+  `route.confidence >= minimumDecisionProbability`, and the gap between the two
+  largest route probabilities is greater than `tieMargin` (default `0.1`).
+- `risk.score < 1`: only the `clear` level passes; `uncertain` and `ambiguous`
+  require review.
+- Each entity match chooses the suggested candidate's key with confidence at or
+  above the threshold and a gap above `tieMargin`.
+
+These values are provider outputs read as if they were calibrated
+probabilities; nothing in this package calibrates them for this use. The
+teamworks-os measurement in #4209 (self-hosted Laya `typed-decisions`) found no
+separation between right and wrong suggestions: a correctly read vendor invoice
+scored `supported` 0.37, garbage input 0.45–0.50, a wrongly proposed purchase
+order 0.51, route confidence 0.003–0.15 and `risk` ≥ 0.94, so the gate sent
+every suggestion to review. Until a labelled sample shows separation, treat the
+answers as diagnostics and configure:
+
+- `decisionMaxLen`: positive integer forwarded unchanged as the per-request
+  `maxLen` of the decision client (the SDK Laya provider's token window; the
+  checkpoint default is 1024 and the serialized evidence plus suggestions is
+  otherwise truncated). Other clients ignore it.
+- `decisionReadiness: 'record'`: the decision still runs, is bounded and
+  authorized exactly as before and its answers, usage, `readiness` and `maxLen`
+  are recorded, but the gate never changes `outcome` or `disposition`; readiness
+  comes from the validator alone (`missingFields`, outcome, source completeness).
+  A client error or timeout is recorded as the `decision_unavailable`
+  warning. A reply that comes back and then fails the core `executeDecision`
+  identity or answer-shape check is recorded as `decision_malformed` with the
+  reply's usage; this is the path for injected `DecisionClient`s. SDK clients,
+  including the Laya provider behind `createSDKProposalDecisionClient`,
+  validate the reply's shape inside `client.decide` and throw, so a malformed
+  SDK reply is recorded as `decision_unavailable` with no usage. Every case
+  records the `provenance.decision` identity. The generated suggestions stay
+  as they were. The serialized request exceeding the byte budget is a
+  configuration failure, not a provider answer: it ends the attempt as `limit`
+  in both modes, and in `gate` mode every failure above still ends the attempt
+  as `provider_error`. Lease, catalog, policy and provider-allowlist revocation
+  fail closed in both modes.
+
+Successful output also records `provenance.decision.readiness` (`'gate'` or
+`'record'`) and `maxLen` when set; configuration digests are unchanged, but
+persisted output is therefore not byte-identical to earlier releases.
+
+Both settings are public configuration: they are part of
+`ProposalReadConfiguration`, the frozen `GenerationStageConfiguration.proposals`
+(present only when set, so existing digests are unchanged) and the completed
+snapshot authority check. Invalid values fail at proposal service construction.
+Neither mode affects automation: `automaticActionEligible` stays `false`, and
+`authorizeAutomatic` reads only the host's `evaluateAutomatic` callback and
+policy, never these answers.
+
 Output records zero, one or multiple suggestions, alternatives, missing fields,
 source links, proposed splits, versions, allowlisted usage, warnings and explicit
 `unknown`, `ambiguous`, `no_action`, `needs_review` or `provider_error` outcomes.
@@ -154,7 +269,14 @@ await service.previewGeneratedProposals({
 
 `intentionKey` is the host's stable business-intention identity across generation
 revisions. Never derive a fresh key from model output, suggestion order or a retry.
-Selections must reference current persisted validated `ready_for_review` output.
+Selections must reference current persisted validated `ready_for_review` or
+`needs_review` output (#4208); a `provider_error` attempt, an unoffered handler or
+a non-interpretation stage is rejected. A `needs_review` suggestion whose
+`missingFields` are still absent fails the handler schema inside the ordinary
+preview (`Invalid handler payload`) and publishes nothing; the reviewer supplies
+the missing values through the explicit human path (`previewProposal` /
+`previewPlan`, as the published review host does for `preview({args})`), and the
+handler's `validate` and `preview` run again on the completed arguments.
 Each batch requires distinct suggestion indexes and intention keys; duplicate
 indexes are rejected before any operation or plan preview is written.
 The service rechecks current catalog/schema/target/source provenance, then calls

package/README.md

@@ -10,7 +10,25 @@ methods through their authenticated application transport.
 
 Before starting a service, deploy the package manifest through the public s-m-r-t
 migration APIs or `smrt db:migrate`. `assertReady()` only checks existing tables.
-SQLite and PostgreSQL are supported; other engines fail construction.
+SQLite and PostgreSQL are supported; other engines fail construction. Index
+additions ship the same way: `intake_items_received_idx` (newest-first inbox
+paging on `tenant_id, confidential_scope_id, created_at, id`) is declared on the model and reaches existing deployments through the
+next manifest migration; there are no hand-written migration files in this
+package, and `smrt doctor --db` / `db:status --parity` report the index until
+it is applied.
+
+`listItemsPage({ limit?, cursor?, state?, itemIds?, scanLimit? })` returns
+`{ items: [{ item, state, cursor }], scanned, nextCursor? }`: one authorized
+inbox page newest receipt first with each row's durable inbox state and
+continuation cursor, and the number of rows the call examined. `listItems()`
+remains as a deprecated array-returning method that walks the private page
+scan to the end. Without the optional `cursorKey` service option every cursor
+names the last row the actor can read, and a page that finds no readable row
+within its scan limit fails with `IntakeScanLimitError`; with a server secret
+of at least 32 bytes, cursors are sealed and such a page resumes past what it
+skipped. `scanned` is for host
+accounting only. See [REVIEW.md](REVIEW.md) for the paging and authorization
+contract.
 
 A host supplies a database, AssetRuntime, authenticated tenant/actor/confidential
 scope, a live authorization callback, a registered jobs target, and an idempotent
@@ -20,7 +38,29 @@ transaction executor; it must intersect current grants with that ceiling. Never
 construct the scope, retention policy or budgets from incoming document labels.
 
 `receive` accepts bounded original byte buffers, with stable source part IDs,
-parent relationships and optional upstream identity/version. Source adapters own
+parent relationships, optional upstream identity/version and an optional
+original `filename` (kept for labels on `IntakeEvidenceDTO.filename`, never
+part of receipt identity). The multipart upload source, email attachments and
+the watched folder all supply it, as may a host `snapshotReference` part or a
+vendor webhook's verified parts. Every source delivers through one place that
+removes control, format (bidi, zero-width, tag), line-separator and
+default-ignorable characters (Unicode properties, see
+`src/presented-filename.ts`) and drops a name that is then empty or over 512
+characters, so an odd name falls back to the default label instead of failing
+the receipt (`receive` itself rejects such a name from a direct caller as
+`invalid`, and names are cleaned again when read back). Removing every format
+and default-ignorable character has display-only effects, since nothing uses
+the name as identity: emoji ZWJ sequences and presentation selectors flatten,
+ZWNJ/ZWJ are removed from Persian and Indic text (letters may join
+differently), and ideographic variation selectors are removed (a CJK variant
+shows its base glyph). Unicode normalization
+(NFC/NFKC) and confusable look-alike characters are out of scope: the name is
+still untrusted text, so render it as text, never as markup or a path. The
+upstream `sourceReference` is stored but not exposed on the DTO: its ids are
+upstream locators, not display text, and they are never cleaned. For email
+they are the provider's message locator (plus the attachment part id); for the
+watched folder the id is the raw folder entry name, whose cleaned form is what
+`filename` exposes. Source adapters own
 streaming and upstream snapshot retrieval. Receipt identity is tenant + source +
 delivery key; changing bytes or authenticated scope conflicts. Content equality
 in a different conversation does not deduplicate an arrival. The returned item

package/REVIEW.md

@@ -32,6 +32,41 @@ Plan membership contains the exact parent args/version so edits call
 `previewPlan`, not per-step correction. An item may have more review pages; the
 host must aggregate state across pages when projecting a production inbox state.
 
+A `needs_review` suggestion is completable, not only rejectable (#4208). The
+component shows its `missingFields`, an editable `Suggested arguments (JSON)`
+draft prefilled from the generated arguments, and the candidate picker for
+declared candidate fields; Prepare review stays disabled while required fields
+are still missing from an unchanged draft. An unchanged draft previews the
+generated snapshot as before; an edited draft is sent as `preview({..., args})`,
+which the published host publishes through the ordinary `previewProposal` /
+`previewPlan` with the same `ui:<index>` intention, never by rewriting the
+generation. The handler validates the human arguments again before a
+`waiting_review` action exists, approval remains a separate explicit decision,
+`ActionReview` lists required arguments missing from the current draft while
+editing, and no authority or automation eligibility is added: a failed attempt,
+an unoffered handler or a foreign stage is still refused by the host.
+`preview` throws on every failure, as before. When the handler's argument
+schema or `validate` refused the arguments being previewed (operation preview,
+which also carries corrections, or plan preview), the server throws
+`HandlerArgumentsInvalidError` (`name` set, messages unchanged). Apply-time and
+external-send revalidation and result-schema checks keep a plain `Error`. The component shows an inline message and
+keeps the view and draft when `preview` either rejects with an error whose
+`name` is `HandlerArgumentsInvalidError` or resolves `{ rejected: 'invalid' }`
+(`PreviewRejection`). A transport that crosses a process boundary preserves
+the inline UX by keeping that name (for example re-throwing an `Error` with
+`name = 'HandlerArgumentsInvalidError'` in the browser callback) or by
+returning `{ rejected: 'invalid' }` for that case only, as the maintained
+fixture's `previewForReview` does. Without either, the refusal behaves like
+any other failure: the view is cleared exactly as before this change, and the
+reviewer's in-memory draft returns after an authorized refresh. Every other
+thrown preview or candidate-search failure (denied, revoked, unavailable or
+unknown) clears the view.
+Unsent drafts live only in component memory, keyed by attempt and suggestion
+index. They survive Refresh, a failed callback, an action on another card and a
+transient `unavailable`/`stale` reload; a `revoked` or `expired` result
+discards them, a successful preview clears that card's draft, and an available
+reload drops drafts for attempts or cards no longer offered.
+
 The host's `load` callback supplies a current `ItemReviewView`, using
 `getCompletedAnalysis` with no historical fallback. Generated suggestions bind
 the **current generation attempt**, not `generation.source.attemptId` (the prior
@@ -98,6 +133,151 @@ with `human_split_not_respected`. New proposals require new explicit review.
 The reference host runs these stages synchronously for browser determinism;
 production hosts dispatch the existing durable analysis jobs.
 
+## Reviewer instructions
+
+`recordReviewerInstruction({itemId, attemptId, expectedRevision, text,
+requestId})` records a plain-text instruction for one item ("This is a customer
+PO from Badger; file it, do not bill it") as the second kind of **human
+correction**. It is authorized exactly like `reviseLogicalSplit`: the current
+generation must pass the same-executor catalog/candidate gate and the reviewer
+grant plus policy reviewer list, and authority is checked before mutation and
+again before publication. Process access, `analyze`, and a client payload cannot
+mint one: `analyze` rejects a `reviewerInstruction` key on an extraction
+configuration, and an interpretation copy that differs from the pinned
+extraction's record fails with `Generation evidence changed` before provider
+disclosure. Text is required, trimmed-nonempty and at most 4,000 characters.
+
+The record is immutable: `{id, text, actorId, recordedAt, supersedesId, digest}`
+plus the request binding, written into a new extraction configuration beside
+any existing split correction. Recording supersedes current suggestions and
+approvals, queues the normal fenced re-extraction and interpretation through the
+existing dispatch path, and returns immediately; the reviewer keeps working
+while the item reports `processing`. `prepareGeneration` copies the instruction
+into the interpretation configuration, the generator receives it only after its
+digest matches the authoritative extraction record, and saved generation
+provenance records `reviewerInstruction: {id, digest}`. A later instruction
+records `supersedesId` and replaces the earlier one for future interpretations;
+`listReviewerInstructions(itemId)` returns the full history, newest first, and
+history is never deleted. The instruction steers the model only: it cannot add
+handlers, candidates, schemas, permissions or automation eligibility, replies
+are validated as usual, and suggestions still require explicit preview and
+approval. A replay with the same request ID returns the same revision; a changed
+text under the same request ID conflicts.
+
+**Cost.** Each recorded instruction appends a new extraction revision, so the
+item is extracted again before it is interpreted again: cheap for embedded-text
+PDFs and plain text, but OCR/speech hosts spend provider budget per instruction,
+exactly as they do per split correction. The pinned extraction is not reused
+because the interpretation contract requires its source to be the immediately
+preceding revision and `getCompletedAnalysis(itemId, attemptId)` serves only
+the current revision (PROPOSALS.md, "Durable stages"); reusing it would mean
+weakening that predecessor pin, which protects every generation from an
+intervening reprocess, so this feature follows the split path instead.
+
+**State and failure.** An in-flight stage (queued/running) reports the item as
+`processing` even when earlier, now-stale actions exist. If the re-extraction or
+interpretation fails, `getReinterpretationState(itemId)` returns the latest
+attempt's safe failure category and the latest completed generation attempt;
+the host projects them as `instruction.failure` and `instruction.binding`, and
+`recordReviewerInstruction` accepts that historical attempt as its anchor when
+`expectedRevision` is the item's current (failed) revision and no later
+**interpretation** completed and nothing is queued or running, so the reviewer
+can retry or change the instruction after either stage fails. A completed
+extraction whose own interpretation failed belongs to the failed attempt, not a
+newer result; a completed extraction whose interpretation has **not been
+appended yet** is in flight (the host is between stages) and is refused with
+`ReviewerInstructionStaleError` (`Stale reviewer instruction`), so a second
+reviewer cannot pre-empt it. That gap expires: once the extraction attempt's
+completion plus the item's analysis lease duration (`limits.leaseMs`, the lease
+`claimAnalysis` grants) has passed with no interpretation appended,
+`getReinterpretationState` reports `failure: 'not_started'` with the retry
+binding and the guard allows the retry. `not_started` means **the
+interpretation did not start within the lease window**: the host crashed,
+threw between `prepareGeneration` and `analyze`, or its queue is merely slow.
+Pre-empting a slow but healthy host is safe because `analyze` refuses an
+interpretation whose pinned extraction is no longer the item's latest, under
+the item lock and before anything is superseded
+(`GenerationSourceStaleError`, nothing appended): the slow host's late append
+cannot clobber the retry. Missing or malformed timing (no attempt, no
+`completedAt`, non-finite `leaseMs`) counts as already expired rather than
+holding retries forever. The "still being re-read" note shown after a stale
+result is cleared by the next reload that shows the item settled; the draft
+then carries a persistent "This item changed while you were writing; your
+instruction was not recorded." note (any stale cause: another reinterpretation,
+a split, or a host reprocess) until the reviewer edits or resubmits it. The guard and
+`getReinterpretationState` share one predicate, and the latter's
+`reinterpreting` flag (projected as `instruction.reinterpreting`) is true in
+the gap too, so the component keeps polling and keeps Re-read disabled there.
+The published host maps two typed errors to `instruct` results:
+`ReviewerInstructionStaleError` → `{ stale: true }` (refresh-and-retry: reload
+with a note, draft kept) and `ReviewerInstructionInvalidError` → `{ invalid:
+true }` (empty, whitespace-only or over 4,000 characters: shown inline, draft
+kept). Neither is a denial. Custom hosts should map them the same way; a host
+that lets either propagate as a thrown error makes the component treat it like
+any other failed mutation, i.e. a whole-view deny with the "view unavailable"
+message, as before these results existed. The maintained fixture's
+`/api/instruct` now answers an invalid text with HTTP 200 `{ invalid: true }`
+instead of 409. A retry from an older anchor builds on the **whole configuration**
+of the newest extraction after it (replacing only the instruction), because that
+is the item's current configuration: corrections chain forward through it and a
+host reprocess replaces it. So a split followed by a failed stage is never
+silently dropped, and a failed reprocess's changed configuration is neither
+reverted nor overlaid with a split it dropped. **Concurrency.** `recordReviewerInstruction` and `reviseLogicalSplit` each
+run their whole decision in one transaction under the item row lock: the
+anchor snapshot read takes the lock first, the execution authorization of
+that snapshot takes it again (`UPDATE intake_items` row lock), the guard step
+re-asserts it explicitly, and
+the request-key replay check, the revision and later-work guards, the
+carried-forward split, the history read that sets `supersedesId`, and the
+append all run under it. On PostgreSQL a second reviewer at the same revision
+blocks on that lock and is then refused (`ReviewerInstructionStaleError` /
+`Stale logical split`), so the ledger cannot fork; an instruction racing a
+split (in either order) is refused rather than built from the pre-split
+configuration; and a double-click whose duplicate commits after the unlocked
+replay check is answered as a replay of the committed request, not a conflict
+or a denial.
+
+**Host reprocess.** A host reprocess (a plain `analyze()` extraction the host
+configures itself) cannot carry human corrections — `analyze` rejects
+`humanCorrection`/`reviewerInstruction` on extraction configurations — so it
+drops the split and the instruction alike, and neither is applied to its
+interpretations. The instruction is then reported as `dropped`, not pending:
+`getReinterpretationState().instructionInEffect` names the instruction the
+newest extraction carries (verified by digest) and is null after such a
+reprocess. History is unchanged; the reviewer can re-record the instruction,
+which supersedes the dropped one.
+
+`ItemReviewView.instruction` exposes `current` (the latest recorded instruction
+still in effect, i.e. carried by the newest extraction), optional `dropped`
+(the latest recorded instruction a reprocess dropped), `applied` (the
+instruction the displayed **generation** ran under; a completed extraction alone
+applies nothing), `stale` (the displayed suggestions predate the current
+instruction), optional `failure` and optional `binding`. Feedback examples that carry instruction
+text count it toward the feedback byte budget (FEEDBACK.md). The optional host callback
+`IntakeReviewHost.instruct` is implemented by the published adapter when the
+service supports it, with an optional `onInstruct(itemId, revision)` wakeup
+mirroring `onSplit`. The review component renders "Instructions for the
+assistant" with "Re-read with these instructions", shows "Reinterpreting…" while
+the item's stage is queued/running, repeats the existing authenticated `load`
+on `pollMs` (default 2 s, backing off to 5×) and patches the view in place —
+drafts, split edits, focus and the evidence viewer survive and no loading
+placeholder is shown — stops when the stage is no longer in flight, and then
+shows the instruction that produced the current suggestions, marking a
+superseded one as stale. An explicit host denial on a poll tick fails closed at
+once; a thrown tick is treated as transient (view and drafts kept, a
+"Couldn't refresh; retrying." note, backoff continues), but because hosts may
+also throw for revoked access, three consecutive thrown ticks fail closed.
+**Hosts must signal revocation by returning `availability: 'revoked'`** (or
+another non-`available` state) from `load`, not by throwing: a returned denial
+clears the view on that tick, while a host that throws on revocation leaves the
+already-displayed content visible until the third thrown tick (about 10 s at the
+default interval and backoff) before the view fails closed.
+Split drafts survive a refresh only when the reviewer edited them. A failed stage shows its safe category with the form
+still available. There is no
+streaming channel; status is read through the existing item/analysis APIs. The
+maintained fixture records the instruction durably and then runs the stages in
+the background so the browser proof exercises the in-place update.
+
 `feedback` is an optional explicit correctness callback, separate from approval
 choice and execution success. No learning, feedback storage or rule promotion is
 implemented by these components; #3676 owns those services. Do not pass feedback
@@ -147,14 +327,170 @@ action/plan preview CAS lookup, decisions, apply, corrections, candidates, split
 and feedback. Repeated preparation of an unchanged authorized suggestion retains
 its current review instead of creating a new revision. Changed arguments still
 require explicit re-preview and a fresh human decision.
-Lists scan 20 items per page; filtered pages may be empty with a `nextCursor`.
-Continue until the cursor is absent. Listing does not load original bytes.
+`list({ state?, assigneeId?, cursor?, limit? })` returns one page newest receipt
+first (`intake_items.created_at`, id tiebreak) through `service.listItemsPage`.
+`limit` defaults to 20 and is capped at 100; the cursor is an opaque
+(received-at, id) continuation, never a bare id. Tenant, confidential scope,
+retention and the `state` filter run inside the page query, so a page is full
+and `nextCursor` is present only when more rows follow.
+
+Authority is rechecked at the same two levels `load` applies, but per page
+rather than per row: the host authorize callback once per distinct captured
+ceiling in a batch, and the execution read gate (policy, ceiling, handler and
+operation allowlists, principal) once per distinct (item, handler) bound by
+the page's saved actions. The list gate is read-only: it evaluates the same
+checks as `listReviews` without the retention write, so an item that the item
+view would narrow past its deadline is simply not listed. The list gate also
+resolves each bound handler and rejects version drift against the saved
+binding and omits a cancelled item, as `listReviews` does; it does not
+evaluate per-action target,
+evidence or plan gates, so an item whose target became unreadable still lists
+and its item view refuses. An item whose bound
+handler the actor cannot currently read is not listed at all, rather than
+listed with a state derived from actions the actor may not see; actions
+without a proposal bind no handler and are not gated, as in `listReviews`.
+Handler bindings (handler id and version, projected in SQL, distinct per
+item) are read once per fetched batch for the rows that passed the item-level
+gate. Note the host cost plainly: the execution `authorize` callback runs once
+per distinct (item, handler) bound by a scanned row that passed the item-level
+gate. Each item is its own authorization subject, so nothing is deduplicated
+across items; the count is bounded by rows scanned (at most
+`ITEM_SCAN_CEILING`) and runs in host code rather than as N+1 database reads. Either host callback, the item-level `authorize` or the execution
+`authorize` behind the handler gate, follows the same rule: a refusal (false,
+or an access check that fails) hides the rows it covers, and a callback that
+throws rejects the whole page (and `listItems()`) with its error, as `load`
+does, so a transient failure never lets paging move past readable rows.
+Denied or unreadable rows are skipped without
+disclosure and the page refills from the next rows; refills grow from the page
+size to 100 rows per query, so a fully denied scope costs about a dozen page
+reads and as many binding reads per call. One service call examines at most
+`ITEM_SCAN_CEILING` (1000) rows, or the caller's smaller `scanLimit`, and
+reports the count as `scanned`; when a mostly-denied scope reaches the ceiling
+the page comes back short with a `nextCursor`. Rows are authorized first and
+the `state` filter is then applied to the fetched batch, so it is bounded by
+the same scan limit: a rare state yields short pages with a cursor rather than
+an unbounded walk inside one query. A row the actor can read whose state does
+not match discloses nothing and may anchor the cursor, so a filter with no
+match among the next 1000 rows returns an empty page with a cursor and still
+pages to the end. The host `list` passes its remaining budget as `scanLimit`, so the
+rows scanned across the service calls of one host call never exceed 1000: a
+short service page with a cursor, or an exhausted budget while filtering by
+assignee, ends the host call with the same cursor and the caller continues.
+Only rows the service wrote are supported: cursors carry the canonical
+millisecond ISO receipt time the service stores. A row whose stored receipt
+time is not that form (on SQLite anything but `YYYY-MM-DDTHH:MM:SS.mmmZ`,
+including a parseable value without milliseconds or with an offset; on
+PostgreSQL a value with a sub-millisecond part) is never listed, whatever the
+actor's access, and counts toward the scan, as does a row whose id is not a
+UUID. On SQLite a non-text stored value (an integer, say) and a receipt value
+or id longer than 64 characters or containing a NUL character are excluded by
+the page query itself: the first sorts after every text value so the scan
+could never move past it, the second could not be carried by any cursor, and
+the third is returned truncated at the NUL, so it would look canonical and
+sort above a legitimate row at the truncated value. Within a call the scan moves
+past a service-foreign row by its exact stored value, and an unsealed cursor
+names the last row before it that the actor can read. A call that finds no readable row within its scan limit (at
+most 1000) fails closed with `IntakeScanLimitError` unless a `cursorKey` is
+configured, as below. The host
+`list` returns what earlier service calls collected, with their cursor, when
+a later call fails that way.
+
+`listItems()` is deprecated and kept only for callers of the array-returning
+API. It walks every page, so its cost grows with the scope, including one
+execution authorization per (item, handler) across the whole scope. It follows
+the internal scan position, which never leaves the service, so runs of rows
+the actor cannot read never stop it and it returns `[]` where nothing is
+readable. Items are newest-first and gated exactly as on a page.
+
+The page's inbox state is the durable projection of the saved action journal
+and processing state, computed once per scanned row inside the page query;
+`load` remains the authoritative, live-gated projection for one item (it still
+projects individual stale actions, which the list does not). Cost is
+proportional to the page for unfiltered pages and common states, and to the
+number of items scanned until enough matches are found for rare states: the
+state is not persisted, because the action journal is append-only and
+projecting it into an indexed column would need a maintained denormalisation
+in the execution service plus a migration, which this change does not take.
+Listing does not load original bytes.
+
+Cursors never disclose an unreadable row. Without a `cursorKey` in the
+service options, every cursor (each row's and `nextCursor`) names a row the
+actor can read (the last one when a page stops early); a page that spends its scan limit before filling hands back
+the last such row, and one that found none fails closed with
+`IntakeScanLimitError` (`Intake scan limit`, no id or time). A run of rows the
+actor cannot read (refused by the captured ceiling or the execution gate, or
+stored with a non-canonical receipt time) at least as long as the scan limit
+therefore stops an unkeyed walk there; readable rows that the `state` filter
+excludes never count toward such a run. With a `cursorKey` (a server secret of at least 32 bytes), every
+cursor is sealed with AES-256-GCM, bound to the tenant, confidential scope and
+actor, and a budget-stopped page returns a sealed position past what it
+skipped, so the walk always progresses. Sealed cursors have a fixed length
+(the plaintext is padded to 1024 bytes) and a fresh nonce per seal, so neither
+their length nor their bytes reveal anything about the stop row. `scanned` is for trusted host
+accounting and must not be sent to clients; a short page with a cursor tells
+a caller only that unlisted rows exist, never which.
+
+Newest-first paging relies on `intake_items_received_idx`
+(`tenant_id, confidential_scope_id, created_at, id`), the full ordering key, so
+an unfiltered page is a backward index scan that stops at the page boundary;
+the foundation suite asserts that plan on both engines (SQLite
+`EXPLAIN QUERY PLAN`, PostgreSQL `EXPLAIN`), with the per-row state counters
+served by `intake_actions_item_idx`. On SQLite the page query pins the index with `INDEXED BY` (except under an
+id restriction, where direct id lookups are cheaper), because without planner
+statistics SQLite otherwise picks `intake_items_expiry_idx` plus a sort; the
+SQLite plan check runs without `ANALYZE`. A deployment must have applied the
+manifest migration, or the page query fails with a missing-index error. This package ships no
+migration files: indexes are declared on the model, the build emits them into
+`dist/manifest.json`, and `smrt db:migrate` / the public migration API create
+them, which is how `intake_items_receipt_idx` shipped. Deploy the manifest
+migration before upgrading hosts; `db:status --parity` reports the index as
+missing until then.
 
 Supply four application adapters: `upload(files, requestId)` (authenticated
 receipt plus first extraction request), `assignment(itemId)` (durable read),
 `assign(input)` (manager authorization, eligible assignees, durable CAS and request
 replay protection), and `evidence(part)` (safe label/text or authenticated
-same-origin URL). Optional `label(item)` changes presentation. Optional
+same-origin URL), which `load` calls for every part and `list` never calls;
+`load` sets no row preview on its entry.
+
+Row previews are opt-in through an optional fifth adapter,
+`rowEvidence(rows)`. When it is supplied, `list` reads the page's durable
+evidence metadata with one `service.listEvidence(itemIds)` query (at most 100
+ids; items the scope cannot read, and expired or inactive items, are omitted
+without disclosure; the host `authorize` runs once per distinct captured
+ceiling through the same gate as `listItemsPage`, so a refusal or a
+malformed captured ceiling hides the item and a failing `authorize` rejects
+the page; an item whose stored data does not parse is skipped by both
+`listItemsPage` and `listEvidence`, never fatal). It then calls `rowEvidence`
+once with each row's primary part (`{ itemId, evidence }`) and publishes the
+returned `{ label, viewUrl?, thumbnailUrl? }` as `InboxEntry.evidence`
+(`{ evidence, label, viewUrl?, thumbnailUrl? }`, never text). Build those URLs
+from metadata without reading bytes; if `rowEvidence` throws or returns a
+malformed answer, the page still lists without previews, and a row without
+durable parts never gets one. When it is omitted, rows show no original link.
+Either way `list` reads the page's evidence metadata once, in that same query,
+so the row label can use the original filename; it never reads original bytes
+and never calls `evidence(part)`. The primary part is chosen by role, not media
+type: the shallowest child part when any part has a parent (an upload's file
+under its capture record, an email's first attachment under the message, even
+when that child is JSON), else the first root part (a watched-folder file);
+part ids sort numerically. `IntakeInbox`/`InboxView` accept an
+optional `rowAside` snippet receiving the `InboxEntry`; without it a row renders
+the host's "Open original" link (new tab) and thumbnail when the URLs are
+same-origin; each link's accessible name is "Open original: <row label>", and
+a row shows no wrapper when neither URL is usable. The slot is a plain `div`,
+not an `aside` landmark, so a page does not add one landmark per row. Optional
+`label(item, evidence)` changes presentation; `evidence` is the item's durable
+parts with the primary part first, and the default label is the primary part's
+own original `filename`, else `Intake <8 hex>`. Optional
+`assignments(itemIds)` reads one page's assignments in one batch (otherwise
+`assignment` runs per row), and optional `assignedItemIds(assigneeId)` lets
+`list` push the assignee filter into the page query (junk ids are dropped, not
+fatal); without it an assignee filter scans further pages and filters by
+`assignment`, which stays correct but costs more rows. The page's continuation
+is always the last row actually returned, so an assignment that changes
+between the page read and the assignment read costs at most one extra scan and
+never skips or repeats a row. Optional
 `enableFeedback: true` exposes the optional feedback callback when the service
 has feedback configured. Optional `onSplit(itemId, revision)` wakes the host worker after the durable extraction
 request; repair dispatches if wakeup fails. The factory creates no assignment