@happyvertical/smrt-ingestion 0.0.0-stage → 0.55.6

package/AGENTS.md

@@ -0,0 +1,35 @@
+# @happyvertical/smrt-ingestion
+
+Durable foundation for ADR 0004. `src/models.ts` owns all ingestion tables;
+`src/server.ts` owns authenticated service boundaries and lifecycle primitives.
+`src/execution.ts` owns authoritative review/apply and imports server-only principals.
+See [execution integration](agents/execution.md) for host transaction guarantees,
+catalog contracts, deployment dependencies and continuation bindings.
+Root, `/models`, and `/dto` remain browser-safe; provider/network/filesystem code
+belongs under `/server`. Generated REST/CLI/MCP model mutation is disabled.
+
+See [README.md](README.md) for host integration and recovery/deletion operation.
+See [EXTRACTION.md](EXTRACTION.md) for extraction contracts, provider isolation
+requirements and corpus validation.
+See [PROPOSALS.md](PROPOSALS.md) for the optional server-only proposal stage,
+current catalog/candidate authority and immutable source pinning.
+
+All records require tenant and confidential scope. Hosts create service scopes
+from authenticated sessions; source metadata is never authority. Every read and
+write rechecks the host scope predicate. Analysis input revisions, attempts,
+proposals, decisions, executions and feedback are append-only except documented
+state projections and retention redaction. Terminal actions outlive job cleanup.
+
+Only SQLite and PostgreSQL are supported. Deploy manifest migrations before
+starting services; runtime performs no application DDL. Storage planning and
+preservation use AssetRuntime/AssetStore public methods. Blob I/O and queue
+submission never run inside ingestion database transactions.
+
+Validation: `pnpm test`, `pnpm test:postgres`, `pnpm test:providers`, `pnpm typecheck`, `pnpm build`.
+The PostgreSQL command provisions a real database and fails if unavailable.
+Build includes browser import validation. The provider lane exercises local SDK
+adapters and process isolation, without claiming live remote recognition quality.
+The component/e2e/evaluation
+commands are reserved by ADR 0004 and fail explicitly until their owning children
+supply real suites.
+See ../../docs/test-matrix/3670-ingestion-foundation.md for behavior coverage.

package/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

package/EXTRACTION.md

@@ -0,0 +1,120 @@
+# Extraction adapters
+
+`@happyvertical/smrt-ingestion/server` exports `createSDKExtractionAdapter`,
+`extractAnalysis`, extraction types, and `proposeDocumentSplits`. Root and DTO
+exports remain browser-safe. No extraction result authorizes a business action.
+
+Hosts construct the SDK adapter with explicit provider/model/version identities
+(`unknown` for unavailable model/version), credential-bearing SDK options, and
+`nativeMemoryIsolation: 'host-enforced'`. This last setting is an assertion by
+trusted deployment configuration, never a source/body/client option. Deploy the
+provider child under an OS/container native-memory ceiling. Node's heap limit
+cannot bound native allocations, and PDF's renderer has no pre-render pixel
+limit. Without that host guarantee, adapter construction refuses to run. A
+configured provider must be pinned, not `auto`; OCR fallback is disabled so
+capability provenance cannot silently change. Identity objects are projected onto
+exactly `provider`, `model`, and `version`, each a required nonblank string. Extra
+host fields never enter capability or segment identities, including through
+trusted custom adapter results at the publication boundary.
+
+The adapter forks one disposable Node process per retained part. Deadline or
+cancellation kills it (and its process group on POSIX); provider stdout/stderr
+are discarded. Calls are sequential and bounded by input bytes, page count,
+image decode pixels, serialized output, vision tokens, elapsed time and Node
+heap. Native memory remains the host boundary described above. Remote compute
+already accepted by a provider may continue/bill after the local process dies.
+Do not interpret a local cancellation as remote billing cancellation.
+
+Persist `{extraction: {configurationRevision, limits}}` using `service.analyze`,
+then claim its lease. Call `extractAnalysis(service, lease, adapter, options)`
+with exactly that revision and limits. The service reloads the immutable input
+snapshot and rechecks current processing authority before each provider and
+before publication. The production child requests a parent-side authorization
+handshake before each PDF/OCR/vision/speech operation, including calls on later
+pages. Denial kills the child; already transmitted material cannot be recalled.
+Custom trusted adapters must await `request.beforeProviderCall` before every
+provider invocation. Bytes are read through the service's integrity-checked
+reader; revoked, expired, foreign-scope, forged and stale leases fail closed.
+A retry keeps the same analysis input revision; changed provider/model/options
+must use a new configuration revision and `analyze` request key. Credentials are
+never placed in persisted configuration or provider identity.
+
+Results retain the original evidence ID/hash/part/parent references. Ordered
+segments form whole-document context; extraction does not split originals into
+business records. Valid UTF-8 JSON, text and HTML remain literal source text (render HTML
+only through an application sanitizer). JSON syntax is validated without rewriting
+its source text; malformed JSON or UTF-8 fails extraction. PDF text is extracted one page at a
+time, falling back to actual rendered-page OCR for empty pages. Embedded text
+is not evidence that every visual element was understood. TIFF is actually
+decoded page-by-page with sharp to PNG; each result cites its original TIFF page.
+Animated non-TIFF images are unsupported. Ordinary image OCR and scene vision
+are separately configured. Speech uses source-level text unless actual segment
+timestamps are returned; absent speech confidence calibration is unknown.
+
+Confidence and boxes are explicitly null unless an OCR provider advertises and
+returns them. Unlimited-OCR's synthetic common confidence is always discarded,
+even if an injected capability claims otherwise. Measured OCR values retain an
+unknown calibration label; they are not routing certainty or approval. Boxes
+are in decoded image pixels, attached only to that actual page/source. No tables
+are invented: the current PDF/OCR text APIs do not supply a table contract.
+
+The persisted receipt's `maxOutputBytes`, exposed by `getAnalysisInput`, bounds
+the complete serialized analysis envelope across all parts, including provenance,
+capabilities, usage, and failure metadata. Extraction reserves room for a `limit`
+outcome and retains fitting earlier parts/segments when the aggregate is too
+large. Top-level `truncated` and `omittedEvidenceCount` identify publication
+omissions; per-result omissions retain every discarded segment's exact page or
+time location. Existing errors and omitted ranges survive trimming unchanged.
+If those diagnostic records cannot fit, the entire result is omitted rather than
+publishing an incomplete account of its known failures. If the ceiling
+cannot fit the minimum result envelope, no provider runs: `failAnalysis` records
+only a fixed safe category and terminal lifecycle state, with no provider output
+or output digest. That service-owned failure accounting is outside the provider
+payload budget; `completeAnalysis` never accepts an oversized result.
+
+Partial page results survive later failures or process timeout. Warnings and
+omitted page/source references remain inspectable. `unsupported_type` covers
+video, unconfigured media and encrypted PDF; corrupt decoder failures without a
+safe typed SDK category are `unavailable`, never raw exception strings. A host
+can repair/reconfigure and create a fresh analysis without rewriting originals.
+`proposeDocumentSplits` produces a hashed, versioned, non-executable grouping of
+observed pages; overlap, unordered groups and nonexistent pages are rejected.
+
+## Evidence and limitations
+
+`pnpm --filter @happyvertical/smrt-ingestion test:providers` runs actual local PDF
+SDK extraction, actual sharp multipage TIFF decoding, and packaged process/HTTP
+adapter lifecycle tests. The local speech server tests adapter transport and
+termination, not transcription quality. The tiny TIFF fixture proves IFD/page
+mapping, not fax OCR accuracy. `extraction.test.ts` injects deterministic SDK
+boundaries for mixed/scanned PDF, Unlimited-OCR, camera observations, message
+attachments, audio timestamps and malformed/partial output. These fixtures are
+not real remote-provider integrations and are not the ADR evaluation corpus.
+
+No live Unlimited-OCR, speech recognition or vision quality run is claimed.
+No recorded-human-audio recognition accuracy, real camera-scene quality,
+calibrated confidence, table extraction, or automatic-action acceptance is
+established by these tests. Deployment owners must run the intended provider,
+model and representative retained corpus before making those claims. The
+[behavior matrix](../../docs/test-matrix/3672-ingestion-extraction.md) records the
+release commands and authority obligations.
+
+The versioned on-disk corpus manifest records exact hashes for valid embedded,
+raster and mixed PDFs, two-page TIFF, a synthetic camera-format JPEG, a valid
+PCM WAV recording of a generated tone, email text, corrupt PDF and unsupported
+video. The tone is not human speech; the illustrated scene is not a photograph.
+Real local HTTP fixture runs execute Unlimited-OCR's installed SDK and speech
+transport against deterministic responses/timeouts. They verify adapters, not
+remote models. Unlimited-OCR 0.61.6 rejects valid encoded images smaller than
+100 bytes; ingestion reports `unavailable` without altering them. This owning
+SDK defect is tracked in [happyvertical/ocr#143](https://github.com/happyvertical/ocr/issues/143).
+
+Each capability record explicitly reports `truncation` and `usage` as `unknown`
+or `reported`. Unlimited-OCR's current SDK discards upstream `finish_reason`
+and token usage, so both are `unknown` even for nonempty output. Downstream
+rules must not interpret `status: 'complete'` or `truncated: false` as proof of
+untruncated OCR: they describe completed input processing and *detected*
+truncation only. The adapter never bypasses the SDK to recover hidden fields.
+The missing SDK metadata is tracked in [happyvertical/ocr#144](https://github.com/happyvertical/ocr/issues/144).
+AI vision retains reported truncation when supplied by its SDK. Missing provider
+cost is unknown, not zero; no cost estimate is invented.

package/LICENSE

@@ -0,0 +1,7 @@
+Copyright <2025> <Happy Vertical Corporation>
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

package/PROPOSALS.md

@@ -0,0 +1,173 @@
+# Application-aware proposal generation
+
+`@happyvertical/smrt-ingestion/proposals` is server-only. Configure
+`IngestionOptions.proposals` and the existing `execution.handlers` catalog.
+Handlers opt into discovery with `discovery.mediaTypes`, `references`, and an
+optional `candidates` callback. There is one catalog: the same operation or
+ordered playbook adapter is discovered, validated, previewed, and later applied
+by the execution service. Generation never approves or applies.
+
+## Discovery and authority
+
+`listHandlers(itemId)` returns a bounded detached catalog. The owning execution
+service checks the current principal, captured permissions/handler/operation
+ceiling, policy, confidential scope, public registered operation facts, and
+ordered playbook definition. Every configured provider must occur in the current
+policy's `providers` allowlist. Missing grants remove handlers before model input.
+An authorized catalog larger than the configured bound is rejected; the host must
+narrow it explicitly instead of relying on silent truncation.
+
+`findCandidates({itemId,handlerId,handlerVersion,query,limit})` returns
+`{items,truncated}`. Its trusted handler callback receives `HandlerContext` with
+the authorized transaction executor. Query by tenant and confidential scope
+**before** selecting, ranking, or building labels. Use a bounded query (often
+`limit + 1`) and report `hasMore`. The service projects candidate identity, model,
+revision and label, then checks every target with the owning execution callback.
+There are no hidden counts or pagination tokens. Truncated discovery forces
+`needs_review` and cannot enter generated preview.
+
+Completed generation reload checks process authority, the current attempt, and
+owner/candidate visibility in the same final transaction. Database-backed grant
+callbacks must use the supplied executor. The owning retention lifecycle encloses
+that transaction so a denied read cannot roll back an authorized shorter privacy
+ceiling; restoration starts only after the outer transaction has rolled back.
+
+Candidate callbacks perform bounded database reads only: no provider I/O, network,
+or domain mutations. Target locks and existing retention enforcement remain
+owned by execution. Hosts configure their database's statement deadlines; an
+arbitrary trusted callback is not a sandbox. Each existing-record argument must
+be a declared top-level `references` field of kind `candidate` with its qualified
+model, or `evidence`. Closed JSON argument schemas are required. The host owns
+that declaration and its target authorization semantics.
+
+## Durable stages
+
+1. Complete extraction through the existing analysis ledger.
+2. Call `prepareGeneration(itemId, extractionAttemptId)`. It reads the current
+   completed/partial immutable extraction and returns configuration with
+   `stage: 'interpret'`, pinned source attempt/revision/input/output/evidence
+   digests, prompt/configuration/catalog/provider versions, limits and thresholds.
+3. Pass that configuration to `analyze(itemId, configuration, requestKey)` and
+   claim its new revision with `claimAnalysis`.
+4. The host's existing job callback routes on the frozen `configuration.stage`:
+   extraction invokes `extractAnalysis`; interpretation invokes
+   `generateProposals(lease)`.
+5. Reload the current completed generation using `getCompletedAnalysis(itemId)`,
+   or pass an explicit attempt ID to additionally reject stale client state.
+   Queued/running revisions never return an older completed attempt.
+
+A generation revision does not rewrite terminal extraction. Reprocessing starts
+with a new extraction revision, followed by interpretation, while the host keeps
+the same business intention keys. An interrupted still-current generation lease
+can be retried; terminal generation output remains immutable. `getGenerationInput`
+reads historical extraction only through the current generation lease and its
+frozen source pin. It verifies same-item scope, immutable hashes, durable evidence,
+and current process permission before returning detached data. The source must be
+the immediately preceding revision, so an intervening reprocess invalidates a
+prepared generation. Completed generation reads additionally recheck current
+catalog and target visibility before disclosing persisted candidate labels.
+Expired, cancelled, revoked, retried or superseded workers cannot read or publish late output. The
+current lease and each offered handler/candidate are rechecked immediately before
+each outbound generative/decision boundary and before publication.
+
+## Providers and validation
+
+`createSDKProposalGenerator(client, identity, {maxTokens,timeoutMs})` uses an
+existing AI SDK `chat` client. It sends a fixed system instruction and separate
+JSON evidence/catalog data, requests JSON, supplies cancellation and bounds, uses
+no tools or continuation, and rejects known truncation or tool calls,
+unexpected models, malformed JSON and oversized output. Configure SDK clients
+without automatic retry that would bypass application boundary checks. The
+adapter's public identity is exactly provider/model/version; credentials and raw
+provider errors are never provenance.
+
+A trusted custom `ProposalGenerator` must likewise honor cancellation, bound its
+response, and explicitly declare validated `complete` or `unknown` completion.
+The SDK currently normalizes missing/unknown finish reasons to `stop`
+([owning SDK follow-up](https://github.com/happyvertical/sdk/issues/1381)), so its
+adapter conservatively declares `unknown`, persisted as
+`generation_completion_unknown`. Otherwise valid suggestions remain available
+for human preview; automatic eligibility always remains false. The service
+deadline aborts requests and prevents late publication; it does not claim to terminate computation at a remote
+provider or isolate native allocations. Extractor process limits remain separate.
+
+Model output can contain only offered handler versions, schema-valid arguments,
+retained evidence locations and offered alternative handler/candidate keys.
+Candidate IDs must occur in the offered set, and evidence IDs in retained source
+results. Unknown enums, invented IDs, extra authority/tool/approval fields and
+malformed structures fail closed. Missing required top-level fields are retained
+as `missingFields`, with `needs_review`; values are never invented to satisfy a
+schema. Observed page/time/source granularity is preserved. Logical scan splits
+use `proposeDocumentSplits`; originals and hashes remain intact.
+
+`createSDKProposalDecisionClient(client)` adapts an existing SDK client’s optional
+capability without changing its errors or protocol. Optional `decision` uses public core `executeDecision` and the existing SDK
+predicate, choice, score and batch contracts. Supported-effect predicates,
+handler classification, candidate entity matching, and action ambiguity/risk
+are distinct questions. Their validated probabilities and provenance are stored
+separately from generative provenance and unknown OCR certainty. The complete
+serialized decision request, including suggestions and questions, must fit the
+minimum configured/current policy byte ceiling before capability probing and
+again before deciding. A policy change between these boundaries fails closed.
+Ties, support probability or selected-handler confidence below
+`minimumDecisionProbability`, and actually incomplete extraction require review.
+Unknown provider
+completion metadata is preserved as `source_completion_unknown`; it does not
+block otherwise schema-valid **human** preview or imply completeness. Automatic
+action eligibility remains false. An unconfigured decision path
+is recorded explicitly. Configured errors produce a durable safe failure outcome;
+they never silently switch to a generative decision fallback.
+
+Output records zero, one or multiple suggestions, alternatives, missing fields,
+source links, proposed splits, versions, allowlisted usage, warnings and explicit
+`unknown`, `ambiguous`, `no_action`, `needs_review` or `provider_error` outcomes.
+`automaticActionEligible` is always false. Provider input/output and the complete
+serialized persisted envelope are bounded by configuration, live policy and the
+receipt ceiling. Oversized output becomes a bounded review/limit result; when
+no safe envelope fits, the owning service records fixed `failAnalysis` metadata
+outside the provider payload budget without retaining provider output.
+
+## Explicit preview and stable intentions
+
+The host chooses persisted suggestion indexes and calls:
+
+```ts
+await service.previewGeneratedProposals({
+  itemId,
+  attemptId: generationAttemptId,
+  selections: [{
+    index: 0,
+    intentionKey: 'document:meeting-2026-10-09',
+    expectedRevision: 0,
+    requestId: 'review-request-1',
+  }],
+});
+```
+
+`intentionKey` is the host's stable business-intention identity across generation
+revisions. Never derive a fresh key from model output, suggestion order or a retry.
+Selections must reference current persisted validated `ready_for_review` output.
+Each batch requires distinct suggestion indexes and intention keys; duplicate
+indexes are rejected before any operation or plan preview is written.
+The service rechecks current catalog/schema/target/source provenance, then calls
+the existing operation or plan preview. Each result still requires the ordinary
+human review and execution APIs. Reprocessing creates no new business identity;
+successful-action replay protection stays with execution.
+
+## Evidence and limits
+
+The test matrix is `docs/test-matrix/3673-ingestion-proposals.md`. SQLite and real
+PostgreSQL suites exercise authority, revisions, abstention, hostile output,
+compound proposals, split provenance and reviewed domain effects. The upload
+reference composes authenticated multipart delivery of a generated embedded-text
+PDF through the actual packaged PDF child, durable interpretation, and the existing
+ordered ContentDocument draft/retained-asset plan. It checks duplicate delivery,
+unauthenticated and foreign-tenant denial, explicit approval of both steps,
+idempotent execution replay, page provenance, and unchanged original bytes.
+The compound-email case uses a declared messages-owner snapshot fixture.
+
+Local HTTP fixtures execute the actual AI SDK transport and cancellation contract.
+Generative responses and the separate injected PDF split boundaries establish
+deterministic orchestration, not model accuracy or OCR quality. No held-out routing
+evaluation or automatic approval is claimed. Missing/unknown AI completion remains
+explicit under [SDK #1381](https://github.com/happyvertical/sdk/issues/1381).

package/README.md

@@ -1,3 +1,151 @@
-# Temporary Holding Version
+# @happyvertical/smrt-ingestion
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Durable receipt, evidence, analysis attempts and action-journal foundations for
+[ADR 0004](../../docs/adr/0004-application-aware-ingestion.md).
+
+The root, `/models` and `/dto` entry points are browser-safe. Import
+`IngestionService` from `/server` in trusted host code. Generated REST, CLI and
+MCP access to the persistence models is disabled; hosts expose authorized DTO
+methods through their authenticated application transport.
+
+Before starting a service, deploy the package manifest through the public s-m-r-t
+migration APIs or `smrt db:migrate`. `assertReady()` only checks existing tables.
+SQLite and PostgreSQL are supported; other engines fail construction.
+
+A host supplies a database, AssetRuntime, authenticated tenant/actor/confidential
+scope, a live authorization callback, a registered jobs target, and an idempotent
+`purgeDerived` callback for host-owned previews, retrieval indexes and caches.
+The authorization callback receives the captured ceiling and the current
+transaction executor; it must intersect current grants with that ceiling. Never
+construct the scope, retention policy or budgets from incoming document labels.
+
+`receive` accepts bounded original byte buffers, with stable source part IDs,
+parent relationships and optional upstream identity/version. Source adapters own
+streaming and upstream snapshot retrieval. Receipt identity is tenant + source +
+delivery key; changing bytes or authenticated scope conflicts. Content equality
+in a different conversation does not deduplicate an arrival. The returned item
+ID and version are safe acknowledgment data, never a storage URL.
+
+Storage intents contain a stable asset ID and a public AssetStore-planned URI
+before byte I/O. Preservation verifies existing bytes instead of replacing them.
+A receipt becomes ready only after every part is durable and its dispatch intent
+commits. A source retries the same receive call after failure or lost
+acknowledgment. Hosts schedule `repairDispatches` after restart and periodically;
+it repairs missing/failed queue submissions through SmrtJobCollection. The host
+job restores the captured scope and calls the analysis lifecycle methods. Run
+TaskRunner with the `ingestion` queue enabled.
+
+`analyze` appends immutable input/configuration revisions. `claimAnalysis` uses
+leases, monotonic fences and attempt budgets; `completeAnalysis` rejects expired
+or superseded workers. Terminal attempt output/provenance is preserved. Queue
+submission budgets also stop repeated handler failures in visible
+`needs_attention`. Reprocessing does not mint actions. `createAction`,
+`publishProposal` and `appendPlan` persist explicit intentions and exact attempt
+bindings for the later review/execution layer. They do not approve or apply
+business operations. Decision/execution/feedback models are persistence
+foundations; policy, handler expansion, review and business apply belong to the
+later epic children.
+
+`expire` revokes access and pending work before removing bytes and calling
+`purgeDerived`. Interrupted deletion is retried by `repairDeletions` and
+`sweepRetention`. Minimal receipt/action identities remain, including successful
+action result digests; all content-bearing journal payloads are redacted. Private
+cleanup locators remain in deletion tombstones so repeated sweeps can remove a
+late stale writer's orphan bytes even after the asset owner row is gone. Source
+adapters must enforce their configured delivery-age cutoff; expired replay IDs
+never become fresh work. Continue running the cleanup sweep for the retained
+storage namespace. Upstream source mutation/deletion cannot change copied
+originals; explicit privacy deletion revokes every ingestion-derived payload and
+requires host cache/index purge.
+
+See [EXTRACTION.md](EXTRACTION.md) for bounded extraction, lease-bound input
+snapshots, provider isolation requirements and the extraction corpus.
+
+Validation commands are `pnpm test`, `pnpm test:postgres`, `pnpm test:providers`, `pnpm typecheck` and
+`pnpm build` (including built browser import inspection). The PostgreSQL lane
+requires the repository's configured database service and never silently skips.
+The provider lane exercises local SDK adapters and process isolation; it does not
+claim live remote recognition quality. Component, e2e and evaluation command names are reserved by the ADR and
+explicitly unavailable until their owning children deliver the corresponding
+capability. See the [behavior matrix](../../docs/test-matrix/3670-ingestion-foundation.md).
+
+## Authoritative review and execution
+
+Configure `IngestionOptions.execution` for policy, immutable previews, authenticated
+decisions and idempotent domain execution. See [execution integration](agents/execution.md)
+for the public handler catalog, required host authorization/target locks, deployment
+migrations and jobs continuation integration. The maintained application reference
+in `reference/handlers.ts` creates draft ContentDocuments and attaches retained
+evidence through public domain APIs; it is not a published runtime adapter.
+
+## Configured source adapters
+
+The server entry exports `createSourceDeliveryHandler`, `EmailSourceAdapter`,
+`WatchFolderSourceAdapter`, and `createVendorWebhookHandler`. Construct a
+`SourceBinding` from authenticated host configuration: enabled flag, scoped
+`IngestionService`, stable source ID/version, captured authority ceiling,
+retention policy, allowed media types and receipt limits. Re-resolve this binding
+when grants/configuration change. Source payloads never supply authority.
+
+The reference POST handler authenticates before reading bytes and requires
+`Idempotency-Key`. JSON accepts exactly `capturedAt` and one of `text`,
+`structured`, or `reference: { owner: 'asset'|'message', id, version }`.
+The host's `snapshotReference` must authorize current ownership and return copied
+bytes; an absent snapshot fails rather than acknowledging a mutable locator.
+Multipart accepts one `file` plus `captureId` (equal to the idempotency key),
+`capturedAt`, `captureSource` (`camera`, `native_picker`, `audio`, `document`), and optional
+lowercase SHA-256 `sha256`. This reuses mobile's durable UUID Idempotency-Key,
+MIME-bearing file part and explicit capture metadata convention. Captures retain
+original bytes and a parent metadata evidence record. Configure both the wire
+limit `maxRequestBytes` and receipt `maxBytes`/`maxParts`; include
+`application/json` in the media allowlist for capture/email metadata. Signature
+checks cover PDF, TIFF, PNG/JPEG/WebP and WAV/Ogg/WebM/MP3/MP4 audio; signature
+acceptance promises preservation, not extraction support or codec validity.
+
+Email is explicitly opt-in per configured account. `EmailSourceAdapter` takes a
+`readMessage` callback compatible with the messages owner's
+`EmailAccount.readIntakeMessage(providerLookup, maxBytes, imapIdentity?)` snapshot.
+The provider lookup is Gmail message ID, POP3 UIDL, or IMAP RFC Message-ID; it
+is separate from the delivery dedupe key. IMAP requires `{ folder, uidValidity,
+uid }`: the method selects the mailbox and verifies UIDVALIDITY and returned UID.
+Ambiguous sender IDs resolving to another UID fail explicitly. That method connects,
+fetches the full message including actual attachment bytes, copies the bytes,
+and disconnects. Metadata-only attachments fail explicitly. It never reads an
+attachment path supplied by a provider. A host lists provider deliveries using a
+stable account-specific locator (for IMAP include folder, UIDVALIDITY and UID),
+an explicit revision, and an opaque resumable checkpoint. Same locator/revision
+with changed bytes conflicts; an explicit new revision creates a new receipt.
+Do not use sender-controlled RFC Message-ID as the sole transport identity.
+`receiveBatch` processes in order and stops at the first failure. The host owns
+durable checkpoint storage and commits its `acknowledge` callback only after the
+adapter has received accepted/duplicate. If checkpoint persistence fails, retry
+returns the original receipt. Message/thread/attachment relationships are copied;
+credentials and arbitrary headers are excluded. The ordinary `syncFrom` metadata
+sync remains separate; `hasAttachments` is never proof of byte preservation.
+
+Watch-folder intake polls only an explicitly configured existing private local
+directory. Producers should close their files, preferably publishing with atomic
+rename. Two unchanged size/inode/mtime/ctime observations separated by
+`stabilityMs` are required, and the interval runs again after claim/restart.
+A private `.ingestion-claims/<UUID>/claim.json` intent is synced before atomically
+renaming the original into that directory. Keep the directory on one filesystem,
+under the host's ownership; symlink inputs are refused. A file descriptor still
+held by an uncooperative producer can write after any stability window, so the
+producer close/publish protocol is required for immutable input. Accepted or
+duplicate receipts move the entire claim to `.ingestion-processed`; do not delete
+claims to retry. Polling recovers persisted UUIDs after process death, so receipt
+replay deduplicates ready-without-ack. `failure.json` and poll results expose safe
+categories while originals remain available. Intent directories without an
+original mean death before claim or a competing poll won; the producer original
+has not been removed. Operators may remove these empty intents after confirming
+no active importer. Keep processed originals according to the host's retention
+policy. This adapter does not watch arbitrary paths or start background loops.
+
+Vendor webhooks use a host `verify(request, originalBytes)` callback after bounded
+raw-body reading. The callback must verify signature, delivery time/replay window
+and account ownership, then return trusted binding and durable delivery identity.
+No vendor implementation or hardware driver is selected. All adapters invoke
+only receipt preservation, never domain writes, AI tools or external actions.
+
+See [PROPOSALS.md](PROPOSALS.md) for bounded application catalog discovery, durable
+interpretation revisions, optional typed decisions and explicit generated preview.

package/agents/execution.md

@@ -0,0 +1,132 @@
+# Review and execution integration
+
+`/execution` exports `IntakeExecutionService`, handler/host types, policy
+resolution, binding digests and `continueIntakeReview`. `IngestionService` exposes
+these operations when its optional `execution` configuration is present. Its
+loader is lazy: importing `/server` alone does not import the principal stack.
+`/dto` exposes execution messages using type-only exports. No generated REST,
+CLI or MCP mutation endpoint is enabled; applications authenticate their own
+service surface and create `IngestionScope` from the session.
+
+## Catalog and authority
+
+Applications supply a trusted `IntakeExecutionOptions.handlers` catalog.
+`OperationHandler` has qualified ID, immutable version, description, object JSON
+schemas for arguments/results, allowlisted `resultModels`, `validate`, `preview`,
+a registered `{model, action, version}` operation and explicit effect/idempotency/
+open-world capability declaration. Preview returns display, exact normalized
+arguments and target revision preconditions. There is no arbitrary tool execution
+or expression evaluation. Reviewer display wraps application fields under `preview`
+and symbolic metadata under `dependencies`; plan steps additionally wrap that
+object under `step` alongside the plan preview. Application display keys cannot
+overwrite framework metadata or be silently overwritten by it. A version change
+requires a new preview and approval.
+`PlanHandler` references a real registered playbook key and resolved definition
+hash; expansion contains only ordered operation handlers and typed earlier-step
+result references. Discovery/proposal generation consumes this catalog rather
+than creating another registry. Semantic model output is never host configuration.
+
+Authenticated receipt context captures `capturedCeiling.execution` with principal
+ID and permission/handler/operation ceilings. The execution host's `authorize`
+resolves current grants and application/tenant/source/(optional request) policy
+on the supplied executor, checking the active requester and confidential scope.
+Lists intersect, budgets minimize, review requirements OR, and disabled policy
+wins. Policy versions must change when policy semantics change. Normal operation
+requires human review; machine authorization requires explicit evaluated opt-ins
+at all three authority layers and remains separate provenance from human review.
+The reference handlers always require review; no evaluation claim is made here.
+
+`mutationBoundary: 'serialized'` is an explicit host guarantee, not an inferred
+property of a permissions array: grant revocation must serialize with the supplied
+transaction, or the owning mutation must revalidate under its own lock. The host
+must implement that guarantee before enabling handlers. `assertTarget` similarly
+locks current tenant/confidential ownership and revision against concurrent
+writers using the supplied executor. The reference host demonstrates a scoped
+row lock before public `Contents` reads. A current `PrincipalRun` intersects live
+permissions with the captured ceiling and checks the exact registered collection
+operation at every effect. No `runOnce` atomicity is assumed.
+
+## Durable review and effects
+
+Preview appends a proposal bound to exact analysis attempt/evidence, handler,
+arguments, policy and target preconditions. Decision requests use unique durable
+request IDs; reusing an ID with changed content is rejected. First valid review
+wins under the item lock. Corrections append a new revision; reject/defer remain
+non-executable. Plan corrections use re-expansion. Human decisions expire and
+live authority is checked again before each step. Raw foundation proposals cannot
+bypass execution's schema, preview, evidence and approval verification.
+
+Database handlers must use `context.db` for every domain effect and must perform
+no external I/O. Domain result and successful execution commit together. A domain savepoint rolls failed effects
+back while preserving the item lock and atomically consuming the retry attempt. Completed
+actions survive job cleanup and evidence retention; expired originals leave a
+minimal result digest tombstone, preventing a second effect. Active result reads
+recheck host visibility and result target ownership.
+
+External handlers reserve a stable action idempotency key before submission.
+Timeout, process death or ambiguous outcome requires reconciliation; an expired
+lease never means permission to resend. `reconcile` must return authoritative
+`succeeded`, `not_applied` or `unknown`. `not_applied` is safe only when the provider
+can exclude a late acceptance of the original send; uncertainty stays `unknown`.
+The same key is reused after a conclusive non-application. Cancellation prevents
+pending work but cannot retract a send already issued. Applications must arrange
+provider reconciliation before privacy retention removes the binding; a redacted
+unknown action fails closed and cannot automatically execute again.
+
+Each plan step has a stable action identity. Completed steps are immutable and
+return their original result. Symbolic record IDs bind to exact earlier proposal
+revision, successful result digest/model and target revision; they cannot point
+across items or form cycles. Failed dependent steps never repeat predecessors.
+The plan preview and target preconditions are part of its digest and visible in
+each step review; changed preview/preconditions require a fresh review.
+Abort/continue follows the pinned public playbook definition, with fresh authority
+at every step.
+
+`continueIntakeReview` uses the existing jobs continuation context. Applications
+create tasks with owner, tenant and exact `{recordId, revision, inputKey}` binding
+and configure `authorizeMcpTask` to revalidate it. A continuation answer only wakes
+the task; the service reloads the authoritative decision. Waiting persists without
+an occupied worker and survives runner restart.
+
+## Deployment and evidence
+
+Apply normal manifest migrations before runtime. Execution imports the public
+agents/users principal stack, which registers profiles including `ProfileAsset`;
+execution-enabled hosts must migrate their dependency graph (including profile
+asset joins used by asset deletion). `/server` receipt-only imports register none
+of these additional models. Content is not a runtime dependency of ingestion.
+`reference/handlers.ts` is an application/test fixture, excluded from published
+entries and declarations; importing it deliberately adds Content and its asset
+joins, whose migrations the application then owns. No runtime DDL is performed.
+
+Real SQLite and PostgreSQL execution suites exercise the public Content draft and
+attachment APIs, transaction rollback, actor/context denials, simultaneous applies
+and reviewers, retention replay, external process death and actual TaskRunner
+restart. See the issue 3674 test matrix for the complete release evidence mapping.
+
+## Layered retention and request automation
+
+Every automation-bearing policy layer, including an optional request layer, must
+match the evaluated version. An absent request automation block adds no evaluator
+restriction; it cannot replace the three required application/tenant/source opt-ins.
+
+Authorized execution-policy access monotonically narrows the existing intake
+expiry to `min(existing expires_at, item.created_at + resolved retentionMs)`.
+The anchor is receipt creation, never review, retry or replay time. A longer later
+policy never extends a recorded deadline. This hard maximum applies to the whole
+intake item: the existing `sweepRetention` protocol removes retained originals and
+derivatives, preserving replay identities. No second expiry column or hidden clock
+is introduced, and no blob I/O occurs in an execution transaction.
+
+At an observed expired deadline, execution reads/completions erase proposal,
+decision, execution, plan, action and feedback payloads under the item lock. Review
+reads return existing identity/hash fields with `state: 'expired'` and empty
+`display`; completed action replay returns its digest-only tombstone. Failed writes, including ordinary validation/CAS/policy rejection and domain
+savepoint rollback, cannot roll back an already-authorized retention ceiling or
+redaction. Restoring a recorded authorized maximum needs no new content-read grant;
+later revocation cannot undo it. Initial denied authorization records no intent.
+Completed replay checks the deadline again after target authorization
+callbacks before returning cached content. Proposal publication and automatic
+authorization also recheck after preview/evaluator callbacks, before publishing
+retained proposal or decision payloads. Unknown
+external outcomes remain unknown/reconcile-only, never eligible for blind resend.

package/dist/__smrt-register__.d.ts

@@ -0,0 +1,2 @@
+export {};
+//# sourceMappingURL=__smrt-register__.d.ts.map

package/dist/__smrt-register__.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"__smrt-register__.d.ts","sourceRoot":"","sources":["../src/__smrt-register__.ts"],"names":[],"mappings":""}