@voltro/plugin-storage 0.85.0 → 0.87.0

package/README.md

@@ -20,7 +20,84 @@ Installation, guides, the full API reference, and examples all live in the
 **[Voltro documentation](https://docs.voltro.dev/docs/intro/getting-started)** — the single source of truth. This README is
 deliberately short so there is nothing here to drift out of date.
 
+## Server helpers
+
+Import transport connectors and signed grant/upload-session helpers from
+`@voltro/plugin-storage/server` in server code. This entry uses the same
+implementations as the plugin root without loading the plugin registration graph. Browser descriptors use `@voltro/plugin-storage/rpc` and
+`@voltro/plugin-storage/errors`; the server entry is not browser-safe.
+
+Server-side operation coordination can observe a durable put with
+`makeStorageObjectMetadata(store).lookup(scope, claim)` without acquiring another
+claim or accessing provider bytes. See the Storage docs for pending/deleted
+states and the unverified execution status.
+
+### Bounded HTTP source reader
+
+Server integrations may use `fetchIngestBody(url, maxBytes?, policy?)` to collect
+bounded, interruptible HTTP bytes before validating an expected digest or size.
+The default limit is 64 MiB. Redirects and internal addresses use `IngestPolicy`;
+`allowHosts` exempts named internal hosts, not an exclusive source allow-list.
+The reader creates no references and performs no authorization, scanning or
+persistence. Use `StorageService.put` after validation for the complete storage
+pipeline, bound request duration, and protect network egress against DNS rebinding.
+Redact capability URLs from errors. Bounded buffering is not a process-RSS limit.
+
 ## License
 
+
 Proprietary. © 2026 Voltro UG. Your use is governed by the
 [Terms of Service](https://voltro.cloud/legal/terms). See the bundled `LICENSE`.
+
+
+### Reference catalogues and combined replica inspection
+
+A physical provider namespace identifies object bytes. A reference catalogue identifies the metadata view that lists those objects. They are separate: independent reference stores may use the same provider, and one database may contain views for independent providers or named storage instances.
+
+`RefStore.catalogueId` is a required `Effect<string, StorageError>` with a stable SHA-256 identity for its actual catalogue view. Replicas reading the same view must agree; independent catalogues must differ even if their reference IDs or contents match. Custom stores must retain this identity with their metadata. Wrappers forward the underlying effect. `memoryRefStore()` creates one identity per actual map, not per read.
+
+Configured default stores initialize `_voltro_storage_reference_catalogue` after normal schema reconciliation. Production starts still require applying the normal schema first. Preserve the catalogue marker on restart, backup and restore. If database copies will run concurrently as independent catalogues, assign them distinct markers through a reviewed stopped-service data migration. Do not treat two independent copies as one catalogue merely because a backup copied its marker.
+
+Direct `dataStoreRefStore` consumers register `storageReferenceCatalogueTable`, reconcile their schema and call `ensureStorageReferenceCatalogue(store)` once at boot before inspection. The adapter's `catalogueId` read never writes or repairs missing or corrupt identity. Its view identity additionally includes the configured storage instance and physical provider namespace.
+
+The `/refs` and `/stats` inspect reads report their catalogue, provider namespace and storage instance in `source`. Combined reads expose `sources` and attach source identity and answering replica IDs to each reference. Shared catalogue rows and counters appear once; independent catalogues remain separate, including equal reference IDs. The observation envelope retains unavailable-replica evidence. Counts describe reference metadata, not a complete inventory of unindexed provider objects.
+
+Each catalogue contributes up to 50 references for the requested page offset. Combined pages can therefore contain more than 50 files. Page advancement moves the same 50-reference window across all catalogues; ordering is newest first within the combined window. This is not global pagination across a single merged index. Divergent snapshots from the same shared catalogue are refused with an explicit comparison/retry message; no responder wins silently and counters are not guessed.
+
+The shared Storage browser displays file provenance and catalogue page semantics. Choose one replica before uploading, downloading, sharing, revoking access or deleting. Multi-replica browsing does not fan out file operations. Custom hosts must pass `StoragePage.operationTargetAvailable={false}` while multiple replicas or all replicas are selected, and use a distinct upload scope when the target changes. Server authorization and explicit mutation-target checks remain authoritative.
+
+Combined instance choosers offer only identical plugin declarations from every answering replica. Differing or absent installations are named explicitly; select a single replica to inspect those instances. Missing replica answers remain visible in the observation notice.
+
+## Through-app ticket retries
+
+With the application's DataStore and default reference/grant stores bound, ordinary and Inspect through-app upload tickets carry one native object operation. The ordinary ticket response exposes its optional `operationId`. Keep the issued upload URL to retry the same transfer: repeated or concurrent POSTs with the same bytes converge on one reference in that physical storage namespace.
+
+A provider failure retains the operation's intent and reservation until it is resumed or explicitly retired through the server's object-operation lifecycle. Ticket expiry alone does not release that reservation. Its reference becomes visible after the bytes are confirmed; retrying the same ticket resumes that intent. Different bytes or immutable inputs under that operation are refused with HTTP 409. Upload and resumable HTTP responses retain a provider's declared error status.
+
+Pass the returned `operationId` in a fresh authorized `storage.mintUploadTicket` request with the same metadata to renew an expired ticket and resume its retained transfer. A caller can also choose a stable `operationId` before the first request to recover a lost ticket acknowledgement. This identity is scoped to the current upload owner, tenant and configured storage instance; it is not an authorization credential. Omitting `operationId` starts another operation. Custom reference or grant stores do not bind the native object-operation boundary and do not advertise its operation ID. Presigned, resumable and multipart transfers retain their separate transport contracts. Provider I/O stays outside retryable metadata transactions; the operation does not add a later business-authorization check.
+
+`useUpload` retains an advertised ticket `operationId` across its automatic through-app retries. To recover a lost first mint reply or an explicit later UI retry, choose `operationId` before the first `upload(file, { operationId, prefer: 'through-app' })` call and retain it with that same file and metadata. A new file or business scope needs a new identity. `uploadMany` rejects one explicit identity shared by multiple files before dispatch. Without native operation support, omit the explicit identity; custom stores do not acquire an exactly-once guarantee. This does not persist a browser transfer across a page reload or extend presigned, resumable or multipart contracts.
+
+Object keys are physical provider keys. Native operation-backed writes use their existing content-addressed namespace; use the reference ID and approved ticket binding instead of inferring authority from a logical path inside a physical key.
+
+## Observe a durable operation from trusted server code
+
+`StorageService.operationStatus({ operationId, tenantId })` reads the exact original put identity in the selected service's configured instance, actual reference catalogue, physical namespace and logical bucket. Supply the tenant explicitly, including `null` for a global operation. Every result includes an opaque `scope` identifying that destination; recovery code must compare it with its retained scope.
+
+The states are `absent`, `reserved`, `ready`, `deleting` and `deleted`. Only `ready` includes a matching readable `ref`. Unsupported, unbound, inconsistent or unavailable metadata fails with `StorageError`; it never becomes `absent`. The observation does not reserve, put, claim, delete, read provider bytes or retry a provider operation. Existing operation IDs and tables do not change.
+
+Authorize the business operation before using this trusted server API. It is not a client RPC or access-policy check. Absence is an observation of this metadata view, not proof that an earlier provider request did nothing. Readiness does not prove current provider bytes, continued liveness or application-level artifact validity. Verify those separately, and keep the original operation identity when the outcome is uncertain. Custom `StorageServiceShape` implementations must add this method or return a typed unsupported `StorageError`; the standard service implements it.
+
+## HTTPS-only source reads
+
+`IngestPolicy.requireHttps: true` requires HTTPS at the initial URL and every
+redirect before fetching. Host allowlisting does not bypass it. The default is
+false. This applies to `fetchIngestBody` and configured URL ingestion while
+preserving the existing byte, redirect, address and cancellation boundaries.
+
+## Signed token deadlines
+
+Native signed read grants and upload, finalize, resumable and multipart tickets expire at their signed Unix-second deadline. Verification refuses the exact deadline and every later instant; a zero-second lifetime grants no usable interval. Obtain a new token through the authorized issuer when another transfer or read is needed.
+
+The ticket keeps its approved owner, tenant, key and access policy. Its deadline does not turn redemption into a session login or rerun the application's authorization function. Provider-presigned URLs retain their provider's expiry contract; this rule concerns Voltro's native signed tokens.
+