@lossless.org/objectstorage 10.7.0 → 11.0.0

package/readme.md

@@ -69,13 +69,13 @@ described below.
 ### Clustering Features
 
 - 🔗 **Erasure coding** — Reed-Solomon (configurable k data + m parity shards) for storage efficiency and fault tolerance
-- 🚄 **QUIC transport** — multiplexed, encrypted inter-node communication via `quinn` with zero head-of-line blocking
+- 🚄 **QUIC transport** — multiplexed inter-node communication via `quinn` with zero head-of-line blocking. Encrypted but not authenticated: run it only on a trusted network (see Cluster transport trust).
 - 💽 **Multi-drive awareness** — each node manages multiple independent storage paths with health monitoring
 - 🩺 **Cluster health introspection** — query native node, drive, quorum, and healing status for product dashboards
 - 🤝 **Cluster membership** — static seed config + runtime join, heartbeat-based failure detection
 - ✍️ **Quorum writes** — data is only acknowledged after k+1 shards are persisted
 - 📖 **Quorum reads** — reconstruct from any k available shards, local-first fast path
-- 🩹 **Self-healing** — background scanner detects and reconstructs missing/corrupt shards
+- 🩹 **Self-healing** — background scanner rebuilds missing shards on the drives their manifest names
 
 ## Installation
 
@@ -114,7 +114,7 @@ glibc LGPL terms, Cargo licenses and native distribution inventory.
 ## Native builds and verification
 
 `pnpm build` builds the current host architecture (GNU and musl on Linux), then
-builds the TypeScript facade. Test preparation uses `tsrust --configured-targets`
+builds the TypeScript facade. Test preparation uses `rsrepo --configured-targets`
 to cover all configured host-OS binaries in the packaging checks. Rust 1.95.0 is
 pinned in `rust-toolchain.toml`. Mac builders
 need Xcode command-line tools, both Darwin Rust targets and Rosetta 2 for Intel
@@ -126,7 +126,7 @@ with aliases from their full `*-unknown-linux-musl-gcc` names to the names above
 Rust supplies its pinned, patched musl runtime. Native runtime notices are included
 in the package; retain them when distributing standalone binaries.
 
-Releases use `tsrust matrix build` to assemble all six binaries from one clean
+Releases use `rsrepo matrix build` to assemble all six binaries from one clean
 Git commit. Configure these environment variables on the release coordinator:
 
 ```sh
@@ -184,6 +184,10 @@ Cluster mode requires the control root and every drive root to exist before
 startup. The process user must own them, no path component may be a symlink,
 and group/world write permission must be disabled.
 
+The cluster transport does not authenticate nodes: run cluster mode only on a
+network where every host that can reach `quicPort` is a cluster node (see
+[Cluster transport trust](#cluster-transport-trust)).
+
 ```typescript
 import { ObjectStorage } from '@lossless.org/objectstorage';
 
@@ -211,6 +215,21 @@ const storage = await ObjectStorage.createAndStart({
 
 Objects are automatically split into chunks (default 4 MB), erasure-coded into 6 shards (4 data + 2 parity), and distributed across drives/nodes. Any 4 of 6 shards can reconstruct the original data.
 
+Each drive root counts as one independent drive, so startup refuses two drive
+roots on the same device (`st_dev`). Only the device number is checked:
+partitions of one disk, LVM volumes, btrfs subvolumes and datasets of one ZFS
+pool report distinct devices, are not detected, and keeping them apart is the
+operator's responsibility. The control root may share a device with a drive.
+For dev and test clusters whose drive roots are directories on one filesystem,
+opt in explicitly; startup then logs which roots share a device:
+
+```typescript
+drives: {
+  paths: ['/srv/objectstorage-dev/drive-1', '/srv/objectstorage-dev/drive-2', '/srv/objectstorage-dev/drive-3'],
+  allowSharedDevice: true, // dev/test only: these roots are not independent drives
+},
+```
+
 ## Configuration
 
 All config fields are optional — sensible defaults are applied automatically.
@@ -263,6 +282,7 @@ const config: IObjectStorageConfig = {
   multipart: {
     expirationDays: 7,             // Must be greater than zero
     cleanupIntervalMinutes: 60,    // Must be greater than zero
+    maxUploadsPerPrefixScope: 100, // Open uploads per prefix credential scope; must be non-zero
   },
   cluster: {                 // Optional — omit for standalone mode
     enabled: true,
@@ -272,13 +292,15 @@ const config: IObjectStorageConfig = {
     erasure: {
       dataShards: 4,         // Default: 4
       parityShards: 2,       // Default: 2
-      chunkSizeBytes: 4194304, // Default: 4 MB
+      chunkSizeBytes: 4194304, // Default: 4 MB; between 1 and 64 MiB
     },
     drives: {
       paths: ['/mnt/disk1', '/mnt/disk2'],
+      allowSharedDevice: false, // Default: false; true only for dev/test roots on one device
     },
     heartbeatIntervalMs: 5000,  // Default: 5000
     heartbeatTimeoutMs: 30000,  // Default: 30000
+    getDeadlineMs: 21600000,    // Default: 6 h; 60000 (1 min) to 604800000 (7 d)
   },
 };
 
@@ -460,6 +482,7 @@ interface IStorageCredential {
 - Requests that were already authenticated before the replacement keep running; auth is evaluated when each request starts.
 - No restart is required, and runtime-created credentials survive restart unless `storage.cleanSlate` clears the bounded storage contents; the owned root, maintenance-lock inode, and any durable provider/pool identity are preserved.
 - Replacement input must contain at least one credential, each `accessKeyId` and `secretAccessKey` must be non-empty, and `accessKeyId` values must be unique.
+- A new `accessKeyId` must not be held by a live or revoked prefix credential (`ECREDENTIAL_CONFLICT`); the same check applies to bucket tenant create, ensure and rotate. Prefix credentials are not part of `listCredentials()`; see [Prefix Credentials](#prefix-credentials).
 
 ## Bucket Tenants
 
@@ -499,6 +522,207 @@ const tenants = await storage.listBucketTenants();
 - Tenant credentials can list, read, write, and delete objects in their assigned bucket, but cannot list all buckets, access other buckets, copy from other buckets, delete buckets, or mutate bucket policies.
 - Bucket tenant APIs require `auth.enabled: true`.
 
+## Prefix Credentials
+
+A prefix credential is an ordinary SigV4 key that reaches only the keys below one
+prefix of one bucket, and only until it expires. It is meant for handing a
+short-lived key to a guest (a VM, a job, a mounted drive) that must see one
+folder and nothing else.
+
+```typescript
+const credential = await storage.createPrefixCredential({
+  bucketName: 'drives',
+  prefix: 'drives/user-42/',
+  ttlSeconds: 3600,          // 1 to 43200 (12 h)
+});
+// { accessKeyId, secretAccessKey, bucketName, prefix, region, createdAt, expiresAt }
+
+const client = new S3Client({
+  endpoint: 'http://localhost:3000',
+  region: credential.region,
+  credentials: {
+    accessKeyId: credential.accessKeyId,
+    secretAccessKey: credential.secretAccessKey,
+  },
+  forcePathStyle: true,
+});
+
+await storage.revokePrefixCredential({ accessKeyId: credential.accessKeyId }); // { revoked: true }
+const live = await storage.listPrefixCredentials({ bucketName: 'drives' });   // metadata, no secrets
+```
+
+```typescript
+interface IPrefixCredentialInput {
+  bucketName: string;
+  prefix: string;
+  ttlSeconds?: number;      // first node: lifetime in seconds
+  expiresAt?: number;       // further nodes: the first node's expiresAt (epoch ms)
+  accessKeyId?: string;     // supplied together with secretAccessKey and expiresAt only
+  secretAccessKey?: string;
+  region?: string;          // default: server.region
+}
+```
+
+**Input rules.**
+- Exactly one of `ttlSeconds` (an integer, 1 to 43 200) and `expiresAt` (an
+  integer epoch-ms time after now and at most 43 200 s ahead).
+- Key material is generated when omitted (`SP` plus 20 uppercase hex
+  characters, a 64-hex-character secret). Supplied material needs `expiresAt`,
+  and both `accessKeyId` and `secretAccessKey`: an ID of 16 to 128 characters
+  of `A-Z0-9`, and a secret of 32 to 128 printable ASCII characters without
+  spaces.
+- The prefix is validated, never normalised. It is 1 to 1023 bytes of
+  `A-Z a-z 0-9 - . _ ~ /` (RFC 3986 unreserved characters plus `/`, no
+  percent-encoding). It ends with `/` and does not start with `/`, and has no
+  empty, `.` or `..` segment.
+- The bucket must exist. The access key ID must not belong to a static or bucket
+  tenant credential, and it must not have been revoked.
+- The secret appears only in the create result. It is never listed, logged, or
+  stored in `storage.config`.
+- Errors are `ObjectStorageCredentialError` with a `code`:
+
+| Code | Meaning |
+|---|---|
+| `ECREDENTIAL_AUTH_DISABLED` | `auth.enabled` is false |
+| `ECREDENTIAL_INVALID_PREFIX` | the prefix fails the grammar |
+| `ECREDENTIAL_INVALID_TTL` | not exactly one of `ttlSeconds` and `expiresAt`, or `ttlSeconds` out of range |
+| `ECREDENTIAL_INVALID_EXPIRY` | `expiresAt` is not in the future, or is more than 12 h ahead |
+| `ECREDENTIAL_INVALID_KEY` | the key material or the region is malformed, or only part of it was supplied |
+| `ECREDENTIAL_NO_SUCH_BUCKET` | the bucket does not exist |
+| `ECREDENTIAL_BUCKET_UNAVAILABLE` | the bucket is a sealed or retired migration source |
+| `ECREDENTIAL_CONFLICT` | the ID is held by another credential, or a replay differs from the stored record |
+| `ECREDENTIAL_REVOKED` | the ID was revoked |
+| `ECREDENTIAL_NOT_PREFIX` | a revoke named a static or bucket tenant credential |
+| `ECREDENTIAL_LIMIT` | 10 000 live credentials and revocations are stored |
+| `ECREDENTIAL_PERSISTENCE` | the store could not be written |
+| `ECREDENTIAL_STORE_UNAVAILABLE` | the store file is unreadable (see below) |
+
+**Access.** A key is covered when its bytes start with the prefix; a key equal
+to the prefix (its folder marker) is covered.
+- **Objects.** GetObject, HeadObject, PutObject, DeleteObject and the multipart
+  operations work on covered keys.
+  - Writes also refuse keys with `.`, `..` or empty segments after the prefix,
+    and keys with control characters.
+  - CopyObject and UploadPartCopy need the source to be covered too.
+  - Object requests accept only the query parameters the operation uses:
+    `uploads`, `uploadId`, `partNumber`, `max-parts` and `part-number-marker`,
+    the six `response-*` header overrides on GetObject and HeadObject, plus
+    the `x-id` operation label that AWS SDKs append. Any other object query
+    (`?acl`, `?tagging`, `?versionId`, `?retention`, …) is refused with
+    `AccessDenied` (403), also a name S3 does not define (`?v=1`), and so is
+    any repeated query parameter. A prefix
+    credential never names a version: `?versionId` is refused even as `null`,
+    and so is a copy source with a `?versionId=` suffix.
+  - Requests that pass these checks still go through the router's allowlist
+    (see [Supported S3 Operations](#supported-s3-operations)), so a parameter
+    the operation does not implement gets `NotImplemented` (501).
+- **DeleteObjects.** Every member is checked on its own: covered keys are
+  deleted, and the others get a per-key `AccessDenied`.
+- **Listings.** ListObjects V1, ListObjectsV2, ListObjectVersions and
+  ListMultipartUploads need a `prefix` parameter that is covered. A missing or
+  wider prefix is refused, never clamped.
+- **Bucket level.**
+  - ListBuckets returns only the credential's bucket.
+  - HeadBucket and `GET /{bucket}?location` work on that bucket.
+  - Any other bucket-level query (`?acl`, `?versioning`, `?lifecycle`, …) is
+    refused with `AccessDenied`, and so is any repeated query parameter.
+  - Creating or deleting buckets and bucket policy operations are refused.
+- **Policies.** The scope check runs before bucket policies, so a policy Allow
+  (including `Principal: "*"`) never widens a prefix credential. A policy Deny
+  still applies.
+
+**Expiry and deadlines.**
+- A credential authenticates while the engine clock is before `expiresAt`.
+  After that, or after revocation, requests get `InvalidAccessKeyId` (403).
+- Requests that were admitted end at `expiresAt` as well: a trickled upload
+  fails with nothing stored, and a slowly read download is cut off.
+- Revocation affects only requests admitted after it. Requests already running
+  continue until they finish or until `expiresAt`.
+- There is no extend operation. To refresh, create a new key and revoke the old
+  one.
+- **Multipart cap.** All prefix credentials of one scope (the same bucket and
+  prefix) together hold at most `multipart.maxUploadsPerPrefixScope`
+  (default 100, must be non-zero) open multipart uploads. The next
+  CreateMultipartUpload of that scope gets `AccessDenied` (403) until one of
+  them ends: completed or aborted by any credential, or removed by the
+  multipart expiry cleanup. Revoking, expiring or replacing a credential
+  frees none of its scope's slots, so issuing new keys does not raise the cap.
+  The count is held in memory and resets when the engine restarts. The
+  engine-wide limit of 100 000 uploads still applies.
+
+**Bucket deletion.** Deletion attempts revoke first; callers re-issue.
+- Every path that deletes a bucket revokes the bucket's prefix credentials
+  before it removes anything: S3 DeleteBucket, `deleteBucket()`,
+  `deleteBucketTenant()` without an `accessKeyId`, and migration source
+  cleanup. A recreated bucket of the same name is not reachable with them.
+- S3 DeleteBucket checks that the bucket is empty before it revokes, so a
+  `BucketNotEmpty` (409) keeps the credentials.
+- If the revocation cannot be stored, the deletion is refused.
+- A deletion that fails or is refused after the revocation (for example a
+  stale resource fence, `EFENCE_STALE`) returns that error, never a success,
+  and leaves the credentials revoked. Callers that keep the bucket create new
+  prefix credentials.
+
+**Persistence.**
+- Credentials and revocations live in `.smartstorage/prefix-credentials.json`,
+  under the same root as `credentials.json`. The engine writes the file
+  atomically with mode `0600`, and `storage.cleanSlate` clears it.
+- A sweep every 60 seconds drops expired entries.
+- Engines older than prefix credentials never read the file, so a rollback
+  disables every prefix credential rather than widening it.
+- **Fail closed.** If the file is unreadable, larger than 32 MiB, malformed,
+  or inconsistent (duplicate IDs, an ID held by a static credential, a record
+  that fails validation), the store is unavailable:
+  - every prefix credential is refused;
+  - create, revoke and list fail with `ECREDENTIAL_STORE_UNAVAILABLE`;
+  - no new static or tenant access key ID can be introduced, because the ID
+    collision check cannot run: `replaceCredentials()`,
+    `createBucketTenant()`, `ensureBucketTenant()` and
+    `rotateBucketTenantCredentials()` throw `ObjectStorageCredentialError`
+    with code `ECREDENTIAL_STORE_UNAVAILABLE` when they would add an access
+    key ID that is not in the current set;
+  - bucket deletion is refused, because the bucket's credentials cannot be
+    revoked: `deleteBucket()` and `deleteBucketTenant()` without an
+    `accessKeyId` throw `ObjectStorageCredentialError` with code
+    `ECREDENTIAL_STORE_UNAVAILABLE`, and S3 DeleteBucket returns
+    `InvalidRequest` (400) with a message starting
+    `ECREDENTIAL_STORE_UNAVAILABLE:`;
+  - static and tenant credentials keep working;
+  - `getHealth().auth.prefixCredentialStoreAvailable` is `false`;
+  - the engine logs the condition without secrets and leaves the file as found.
+
+  To recover, repair or delete the file and restart. Deleting it revokes every
+  prefix credential and forgets every revocation, so never reuse a revoked ID
+  afterwards. Generated IDs are random.
+
+**Cluster mode.** Prefix credentials are node-local, like runtime credentials:
+nothing replicates them, and the caller fans out.
+1. Create on one node with `ttlSeconds`.
+2. Create on every other node with the first node's `accessKeyId`,
+   `secretAccessKey`, `expiresAt` and `region`.
+   - Each node records its own `createdAt`; every other field matches the first
+     node's record.
+   - An identical replay returns the stored record unchanged. Any difference
+     gives `ECREDENTIAL_CONFLICT`.
+   - A late create never extends the lifetime.
+3. To revoke, call `revokePrefixCredential` on every node and retry until each
+   node acknowledges.
+   - A node that never saw the ID records the revocation for 12 h, so a create
+     that reaches it later is refused with `ECREDENTIAL_REVOKED`.
+   - `revoked` is `true` only where a live credential was removed.
+
+**Revocation guarantee.**
+- Once a node has acknowledged a revocation, it refuses every request admitted
+  after the acknowledgement. It also refuses any later create for that ID until
+  the ID could no longer be created anyway: an `expiresAt` create must lie
+  within 12 h of the first node's create, and the revocation is kept at least
+  that long.
+- A node that never acknowledged serves the key until the caller's retry
+  reaches it, or until `expiresAt`.
+- The hard bound is `expiresAt`, at most 12 h after the first create. Each node
+  judges expiry by its own clock, so clock offsets shift these instants.
+- `listPrefixCredentials()` lets the caller reconcile nodes.
+
 ## Bucket Backup/Restore
 
 ```typescript
@@ -716,7 +940,9 @@ const metrics = await storage.getMetrics();
 ```
 
 - `getHealth()` reports running state, storage directory and pool observation, auth enabled state, credential counts, bucket count, object count, total bytes, and cluster health.
-- `getMetrics()` returns numeric counters and a Prometheus text snippet for bucket, object, byte, tenant credential, and cluster-enabled metrics.
+- `getHealth().auth` also reports `prefixCredentialCount` (live prefix credentials), `prefixCredentialTombstoneCount` (unexpired revocations) and `prefixCredentialStoreAvailable` (`false` while the prefix credential file is unreadable and every prefix credential is refused).
+- `getMetrics()` returns numeric counters and a Prometheus text snippet for bucket, object, byte, tenant credential, prefix credential (`objectstorage_prefix_credentials_total`, `objectstorage_prefix_credential_tombstones_total`, `objectstorage_prefix_credential_store_available`), and cluster-enabled metrics.
+- `getHealth()` reads the prefix credential figures through the `getPrefixCredentialStatus` IPC command (no TypeScript method of its own), and `getMetrics()` derives its prefix metrics from `getHealth()`. The command takes no parameters and returns `{ available, credentialCount, tombstoneCount }`: counts and the store state only, never an access key ID, secret, bucket or prefix.
 
 ## Runtime Stats
 
@@ -794,6 +1020,123 @@ interface IClusterHealth {
 - Drive health is based on live native probe checks on the configured local drive paths. Capacity values are cheap filesystem snapshots.
 - `quorumHealthy` means the local node currently sees majority quorum and enough available placements in every erasure set to satisfy the configured write quorum.
 - Repair fields expose the background healer's currently available runtime state. They are best-effort and limited to what the engine tracks today, such as whether a scan is active, the last completed run, and the last error.
+- `repairs.shardsChecked`, `shardsHealed` and `failed` count the last completed scan's placement checks, shards written and failures. `repairs.shardsPending` counts the pending shards (see [Self-Healing](#self-healing)) that scan left pending because their drive was out of reach, refused the write, or their object was busy. While a scan runs, `lastRunStartedAt` is the running scan's start and the counts are still the previous scan's; `runHealingScan()` returns one scan's start and counts together.
+
+## Supported S3 Operations
+
+The router fails closed, in standalone and cluster mode alike. A request
+selects one operation from its method, its path (`/`, `/{bucket}`,
+`/{bucket}/{key}`) and its query names, and each query name must be one that
+operation implements. Anything else gets `NotImplemented` (501, with the S3 XML
+error body) before the request takes a lock, reads its body or touches a bucket
+or object. A credential the request is not allowed for still gets
+`AccessDenied` (403) first. Until 11.0.0 such requests were served as another
+operation: `PUT ?tagging` overwrote the object with the XML body,
+`DELETE ?lifecycle` deleted an empty bucket, `DELETE ?versionId=…` deleted the
+current object, and `GET ?versioning` answered with an object listing.
+
+| Operation | Request | Query names besides the selector |
+|---|---|---|
+| ListBuckets | `GET /` | none |
+| CreateBucket, DeleteBucket, HeadBucket | `PUT`, `DELETE`, `HEAD /{bucket}` | none |
+| ListObjects | `GET /{bucket}` | `prefix`, `delimiter`, `max-keys`, `marker`, `encoding-type` |
+| ListObjectsV2 | `GET /{bucket}?list-type=2` | `prefix`, `delimiter`, `max-keys`, `continuation-token`, `start-after`, `fetch-owner`, `encoding-type` |
+| ListObjectVersions | `GET /{bucket}?versions` | `prefix`, `delimiter`, `max-keys`, `key-marker`, `version-id-marker`, `encoding-type` |
+| ListMultipartUploads | `GET /{bucket}?uploads` | `prefix`, `encoding-type` |
+| GetBucketLocation | `GET /{bucket}?location` | none |
+| GetBucketVersioning | `GET /{bucket}?versioning` | none |
+| GetBucketLifecycleConfiguration | `GET /{bucket}?lifecycle` | none |
+| GetBucketPolicy, PutBucketPolicy, DeleteBucketPolicy | `GET`, `PUT`, `DELETE /{bucket}?policy` | none |
+| DeleteObjects | `POST /{bucket}?delete` | none |
+| GetObject, HeadObject | `GET`, `HEAD /{bucket}/{key}` | `versionId`, `response-cache-control`, `response-content-disposition`, `response-content-encoding`, `response-content-language`, `response-content-type`, `response-expires` |
+| DeleteObject | `DELETE /{bucket}/{key}` | `versionId` |
+| PutObject, CopyObject | `PUT /{bucket}/{key}`, CopyObject with `x-amz-copy-source` | none |
+| CreateMultipartUpload | `POST /{bucket}/{key}?uploads` | none |
+| UploadPart, UploadPartCopy | `PUT /{bucket}/{key}?partNumber=…&uploadId=…` | none |
+| CompleteMultipartUpload | `POST /{bucket}/{key}?uploadId=…` | none |
+| AbortMultipartUpload | `DELETE /{bucket}/{key}?uploadId=…` | none |
+| ListParts | `GET /{bucket}/{key}?uploadId=…` | `max-parts`, `part-number-marker` |
+
+Every operation also accepts the `x-id` label that AWS SDKs append and the SigV4
+query-signing parameters of a presigned URL (`X-Amz-Algorithm`,
+`X-Amz-Credential`, `X-Amz-Date`, `X-Amz-Expires`, `X-Amz-SignedHeaders`,
+`X-Amz-Signature`, `X-Amz-Security-Token`, `X-Amz-Content-Sha256`).
+Query-string signatures are not verified: a presigned request is served as an
+unsigned one, which is anonymous when auth is enabled.
+
+An object GET or HEAD (GetObject, HeadObject, ListParts) ignores query names S3
+does not define, as S3 does, so links with cache-busters (`?v=1`) or tracking
+parameters (`?fbclid=…`) keep working. A name S3 defines that the operation
+does not implement (`?acl`, `?partNumber=1`, …) is still refused there, and
+every other request (bucket level, every write and delete) refuses unknown
+names too. The set of names S3 defines is taken from the AWS SDK v3 model
+(3.1039.0). With CORS enabled, the 501 and 405 responses carry the CORS headers.
+
+The `response-*` parameters of GetObject and HeadObject replace the
+`Cache-Control`, `Content-Disposition`, `Content-Encoding`, `Content-Language`,
+`Content-Type` and `Expires` headers of a 200 or 206 response with the given
+value, verbatim. A value that is not a valid HTTP header value (a control
+character such as CR or LF) returns `InvalidArgument` (400). With auth enabled, an anonymous
+request that carries one returns `InvalidRequest` (400), as S3 does, because an
+override such as `response-content-type=text/html` would let a public object
+serve script from the storage origin; a presigned URL is such a request. With
+auth disabled the overrides apply.
+
+Versioning is never enabled on a bucket:
+- `GET ?versioning` returns the empty `<VersioningConfiguration/>` of a bucket
+  that was never versioned. `PUT ?versioning` is `NotImplemented`.
+- `GET ?lifecycle` returns `NoSuchLifecycleConfiguration` (404). `PUT` and
+  `DELETE ?lifecycle` are `NotImplemented`.
+- `GET ?versions` lists every current object as its S3 null version
+  (`VersionId` `null`, `IsLatest` true), with `CommonPrefixes` and paging on
+  `key-marker` and `version-id-marker`. A truncated page returns the last key
+  or common prefix as `NextKeyMarker`, plus `NextVersionIdMarker` `null` after a
+  key. A `version-id-marker` needs a `key-marker`.
+- `versionId` on GET, HEAD and DELETE, and a `?versionId=` after the copy source
+  of CopyObject and UploadPartCopy: `null` is the current object. A
+  well-formed id (a lowercase UUID) names no version and returns
+  `NoSuchVersion` (404; `NoSuchBucket` when the bucket is missing). Anything
+  else, the empty string included, returns `InvalidArgument` (400). A `?`
+  inside a copy-source key must be encoded as `%3F`.
+- These reads are authorized as `s3:GetBucketVersioning`,
+  `s3:GetLifecycleConfiguration` and `s3:ListBucketVersions`. A `versionId`,
+  `null` included, is authorized as `s3:GetObjectVersion` (GET, HEAD, and a
+  copy source's `?versionId=`) or `s3:DeleteObjectVersion` (DELETE), not as
+  `s3:GetObject` or `s3:DeleteObject`, so a policy that allows DeleteObject and
+  denies DeleteObjectVersion keeps protecting history once buckets are
+  versioned. Bucket tenants may use both actions in their bucket.
+
+`encoding-type=url` percent-encodes, on ListObjects, ListObjectsV2,
+ListObjectVersions and ListMultipartUploads, every UTF-8 byte outside
+`A-Z a-z 0-9 - . _ ~` and `/` (a space is `%20`, never `+`) in the elements S3
+encodes for that operation, and the response carries
+`<EncodingType>url</EncodingType>`:
+- ListObjects: `Prefix`, `Marker`, `NextMarker`, `Delimiter`, `Key` and
+  `CommonPrefixes/Prefix`.
+- ListObjectsV2: `Prefix`, `Delimiter`, `StartAfter`, `Key` and
+  `CommonPrefixes/Prefix`; `ContinuationToken` and `NextContinuationToken`
+  are opaque tokens, the same with or without `encoding-type`.
+- ListObjectVersions: `Prefix`, `KeyMarker`, `NextKeyMarker`, `Delimiter`,
+  `Key` and `CommonPrefixes/Prefix`.
+- ListMultipartUploads: `Prefix` and `Key`.
+
+Without the parameter keys are written as they are and no `EncodingType` is
+returned; any other value returns `InvalidArgument` (400). An object key may
+hold any UTF-8 character, also those XML 1.0 cannot carry (U+0001 and the other
+control characters except tab, line feed and carriage return): a listing that
+contains such a key is valid XML only with `encoding-type=url`. ListObjectsV2
+continuation tokens use only `A-Z a-z 0-9 - _`, so they never break the XML.
+
+Accepted but not applied: owners are never returned (`fetch-owner`); responses
+say so by omitting `Owner`. Refused with `NotImplemented` although S3
+defines them: every other subresource (`?acl`, `?tagging`, `?retention`,
+`?legal-hold`, `?cors`, `?website`, `?object-lock`, `?attributes`, `?restore`,
+`?select`, …), `partNumber` on GET and HEAD, the paging parameters of
+ListMultipartUploads (`key-marker`, `upload-id-marker`, `max-uploads`,
+`delimiter`), ListObjects V1 with V2 cursors, the ListBuckets parameters
+(`prefix`, `max-buckets`, `continuation-token`, `bucket-region`), headers
+hoisted into a presigned URL's query (`x-amz-checksum-crc32=…`) on anything but
+an object read, and any `POST` that is not one of the operations above.
 
 ## Usage with AWS SDK v3
 
@@ -999,7 +1342,23 @@ Client ──HTTP PUT──▶ Node A (coordinator)
 2. **Objects are chunked** — large objects split into fixed-size pieces (default 4 MB)
 3. **Each chunk is erasure-coded** — Reed-Solomon produces k data + m parity shards
 4. **Shards are distributed** — placed across different nodes and drives for fault isolation
-5. **Quorum guarantees consistency** — writes need k+1 acks, reads need k shards
+5. **Quorum guarantees consistency** — writes need k+1 acks, reads need k shards.
+   A chunk with fewer than k+m written shards is published with the missing
+   ones as pending shards that healing completes on their assigned drives.
+   A node that membership marks offline is never contacted: its shards of a
+   write are pending at once, reads take their shards from the other nodes,
+   and below either quorum a request fails at once. Until a stopped node is
+   marked offline (five missed heartbeat rounds), each chunk that needs it
+   waits up to the QUIC idle timeout (10 s). Cluster QUIC connections send
+   keep-alives every 2 s from both ends, so a slow but live peer stays
+   connected however long a request takes; there is no whole-request
+   deadline. A heartbeat to a silent peer therefore fails after 10 s. Each
+   heartbeat round goes to every peer at once, so stopped nodes are marked
+   offline after about 50 s by default, however many stop together (a whole
+   location, for example). A node whose restart takes longer than that comes
+   back to pending shards and healing work. `heartbeatTimeoutMs` (default 30 s) still bounds how long a live
+   peer may take to answer a heartbeat, but cannot extend the wait for a
+   silent one beyond 10 s.
 
 ### Cluster Root and Upgrade Requirements
 
@@ -1007,7 +1366,8 @@ Cluster mode never creates configured control or drive roots. Before startup,
 create `storage.directory` and every `cluster.drives.paths` entry, make them
 owned by the ObjectStorage process user, and remove group/world write permission.
 Every configured root must be a real directory with no symlink path component,
-and every drive root must resolve to a distinct device/inode identity. The
+and every drive root must resolve to a distinct device/inode identity and,
+unless `cluster.drives.allowSharedDevice` is set, to a distinct device. The
 control root may be distinct or may be the same identity as one drive root; it
 cannot make two drive entries aliases of one another. Every root must support
 the filesystem durability and locking semantics validated at startup.
@@ -1027,6 +1387,17 @@ has written or migrated the new state, do not restart an older binary and do not
 run a mixed-version cluster. Back up the cluster roots before upgrading; rollback
 requires restoring those roots from the pre-upgrade backup.
 
+Manifests with pending shards (written while a drive refused writes, see
+[Self-Healing](#self-healing)) are refused by engines that predate pending
+shards, because their chunks are short of placements. Before rolling a cluster
+back to such an engine, stop writing, complete or abort every open multipart
+upload (part descriptors record pending shards too, the scan does not count
+them, and an older engine drops them, so such an upload could not complete
+after the rollback), and then, on every node, run
+`runHealingScan()` until it reports `shardsPending` 0. Its scan starts after
+the request arrives, so it covers every write that finished before the call; a
+scan that could not list every manifest fails instead of reporting counts.
+
 ### Erasure Coding
 
 With the default `4+2` configuration:
@@ -1040,14 +1411,61 @@ With the default `4+2` configuration:
 | 6+3 | 9 | 50% | 3 failures | 5 |
 | 2+1 | 3 | 50% | 1 failure | 2 |
 
+### Streaming Reads
+
+A cluster GET streams the object chunk by chunk; nothing is written to the
+control root. It reads from the generation of the manifest it loaded at its
+start, without holding the object's operation gate, so overwrites, deletes and
+other GETs of the same key run while it streams. Each chunk is rebuilt from the
+first k valid shards, read in parallel, and at most one rebuilt chunk waits
+ahead of the client.
+
+- **Errors after the headers abort the response.** The first chunk is rebuilt
+  before the status line, so an unreadable object still gets an error status.
+  A failure in a later chunk ends the body with an error (the connection is
+  closed), never with a clean end of body. A whole-object GET sends its last
+  bytes only after the total length and the object MD5 match the manifest;
+  range GETs are checked for length only.
+- **One deadline for the whole GET.** `cluster.getDeadlineMs` is the longest a
+  single GET may run, counted from its start and including the client's
+  transfer time; a GET that takes longer is aborted, and if its client stopped
+  reading, its connection is closed. The default is 6 hours,
+  the allowed range 1 minute to 7 days. The cleanup grace is derived from it:
+  superseded and deleted generations keep their shards for the deadline plus
+  the smaller of the deadline and one hour (7 hours by default, 2 minutes at
+  the 1-minute floor), so every GET that started on a generation has ended
+  before cleanup frees it. Writes in progress hold their object's gate and are
+  never cleaned up, however short the grace. The setting must be the same on
+  every node; a mismatch can abort streams, but never serves wrong bytes.
+  CopyObject and CompleteMultipartUpload read their sources under the object
+  gate with their own fixed 6-hour limit, counted from the start of the
+  operation.
+- **GET memory budget.** Rebuilding chunks is charged to an engine-wide budget
+  of `max(1 GiB, 16 × chunk footprint)` at the configured chunk size. A chunk's
+  footprint is ⌈chunk/k⌉ × (k + min(k, m)) + chunk: the k fetched shards, the
+  data shards rebuilt for missing ones (a read rebuilds no parity shards) and
+  the decoded chunk; 4+2 at 4 MiB is 10 MiB, 1+15 at 64 MiB is 192 MiB. A GET
+  whose client went away keeps its charge until its local shard reads have
+  ended, and shutdown waits for them before it releases the drive roots. The
+  budget is separate from the upload budget. The first chunk waits at most 5 seconds for
+  budget and is otherwise refused with `503 SlowDown` and `Retry-After: 1`;
+  later chunks wait. Slow downloaders hold up to two rebuilt chunks each, so
+  many of them can exhaust the budget and turn new GETs into `503`s under load.
+
 ### QUIC Transport
 
 Inter-node communication uses [QUIC](https://en.wikipedia.org/wiki/QUIC) via the `quinn` library:
-- 🔒 **Built-in TLS** — self-signed certs auto-generated at cluster init
+- **Encrypted, not authenticated** — each node generates a self-signed certificate at startup and accepts any peer; run it only on a trusted network (see [Cluster transport trust](#cluster-transport-trust))
 - 🔀 **Multiplexed streams** — concurrent shard transfers without head-of-line blocking
 - ⚡ **Connection pooling** — persistent connections to peer nodes
 - 🌊 **Natural backpressure** — QUIC flow control prevents overloading slow peers
 
+### Cluster transport trust
+
+The cluster transport does not authenticate nodes. Each node generates a self-signed certificate at startup and accepts any peer's certificate without verifying it. QUIC encrypts the traffic against passive eavesdropping, but not against an attacker on the path. Any host that can reach a node's `quicPort` can join the cluster, sync topology, and read, write, tombstone and purge shards. S3 credentials and bucket policies do not apply to this port.
+
+Run cluster mode only on a network where every host that can reach `quicPort` (UDP, default 4000) is a cluster node: a private network or VPN between the nodes, with `quicPort` firewalled from everything else. This has been the case since cluster mode was introduced, and 11.0.0 does not change it. Mutual TLS with a cluster CA and certificate-checked joins are planned.
+
 ### Cluster Membership
 
 - **Static seed nodes** — initial cluster defined in config
@@ -1057,13 +1475,40 @@ Inter-node communication uses [QUIC](https://en.wikipedia.org/wiki/QUIC) via the
 
 ### Self-Healing
 
-A background scanner periodically (default: every 24h):
-1. Checks shard checksums (CRC32C) for bit-rot detection
-2. Identifies shards on offline nodes
-3. Reconstructs missing shards from remaining data using Reed-Solomon
-4. Places healed shards on healthy drives
-
-Healing runs at low priority to avoid impacting foreground I/O.
+Every 24 hours, and whenever `runHealingScan()` asks for it, each node scans
+the manifests it coordinated (manifests live on the node that wrote them):
+1. Every shard placement is checked at the drive it names, locally or through
+   its node, against the checksum and size the manifest recorded. The check
+   reads the shard record's header and the record's length only, never its
+   payload, so a scan costs a few small reads per shard. Verifying payloads
+   is not the scan's job: a read detects a rotted payload through its CRC32C
+   and rebuilds the chunk from the other shards.
+2. A missing shard is rebuilt from k valid shards with Reed-Solomon, checked
+   against the recorded checksum, and written to the drive its placement
+   names. The manifest does not change.
+3. A pending shard is written to the drive its manifest recorded and then
+   moved into the chunk's placements, in the same version and generation; the
+   completed manifest is identical to one whose write never failed.
+
+Healing never places a shard on a drive of its own choosing. Placements and
+pending shards on an offline node wait for that node; a pending shard whose
+drive refuses writes stays pending and is reported in `repairs.shardsPending`
+(replacing a permanently lost drive is not supported yet). A shard record that
+cannot be read, such as a truncated one, is reported in `repairs.failed` and
+left in place, because shard records are immutable; a node that cannot read a
+record answers with an error, never as if the shard were missing. A minority
+partition does not heal: its scan fails. Placements are checked without the
+object's operation gate, as a GET reads them. The chunks of an object that need
+a write are healed in batches of up to 1 GiB of chunk data: each batch takes
+the object's gate, reloads the manifest, heals only chunks that are unchanged
+in the same generation, publishes its pending shards with one manifest store
+and releases the gate before the next batch. An object whose gate is busy is
+skipped until the next scan.
+
+Scans never overlap. A scan that outlasts the 24-hour interval delays the next
+one. `runHealingScan()` joins the next scan that starts after its request and
+never a scan that is already running; requests that arrive together share one
+scan.
 
 ### Erasure Set Formation
 
@@ -1083,7 +1528,7 @@ Drives are interleaved across nodes for maximum fault isolation. New nodes form
 
 ```typescript
 import { ObjectStorage } from '@lossless.org/objectstorage';
-import { tap, expect } from '@git.zone/tstest/tapbundle';
+import { tap, expect } from '@git.zone/tsrepo/tapbundle';
 
 let storage: ObjectStorage;
 
@@ -1153,8 +1598,8 @@ the scanned bucket and leaf counts, the total `orphanCount`, and up to
 `reportLimit` orphans with bucket, decoded key, leaf path and the leaf's first
 entry names.
 
-An orphan leaf makes reads, writes and deletes of its key and every listing of
-its bucket fail. A server startup removes empty orphan leaves; a leaf that
+An orphan leaf makes reads, writes and deletes of its key fail, and every
+listing of its bucket whose prefix and cursor range covers the key. A server startup removes empty orphan leaves; a leaf that
 still holds sidecar files stays in place until it is repaired.
 
 Engines before 10.7.0 create a key's leaf before its upload body is complete,
@@ -1266,6 +1711,24 @@ List scoped tenant credential metadata without returning secrets.
 
 Return endpoint, port, region, bucket, access key, secret key, SSL flag, legacy descriptor fields, and S3/AWS env values for the bucket tenant.
 
+#### `createPrefixCredential(input: IPrefixCredentialInput): Promise<IPrefixCredential>`
+
+Create an expiring credential limited to one prefix of one bucket, or install
+one created on another node (`accessKeyId`, `secretAccessKey`, `expiresAt`). An
+identical replay returns the stored record. The secret is returned only here.
+See [Prefix Credentials](#prefix-credentials) for the input rules and the
+`ObjectStorageCredentialError` codes.
+
+#### `revokePrefixCredential(options): Promise<{ revoked: boolean }>`
+
+Revoke a prefix credential and keep its ID refused until the latest moment it
+could still be created. Options: `{ accessKeyId }`. `revoked` is `true` only
+when a live credential was removed; unknown IDs are recorded as revoked too.
+
+#### `listPrefixCredentials(options?): Promise<IPrefixCredentialMetadata[]>`
+
+List live prefix credentials without secrets. Options: `{ bucketName? }`.
+
 #### `getBucketMigrationCapability(): Promise<IObjectStorageBucketMigrationCapability>`
 
 Return standalone provider identity and receipt-key evidence plus source seal,
@@ -1324,7 +1787,9 @@ the bucket's exclusive credential. Options:
 
 Delete a fenced bucket. Supplying `accessKeyId` also durably revokes that
 exact tenant credential and deletes the policy; omitting it preserves the
-legacy credential/policy behavior. Options:
+legacy credential/policy behavior. The bucket's prefix credentials are revoked
+first in both cases, and the deletion is refused if that revocation cannot be
+stored. Options:
 `{ bucketName, accessKeyId?, fence, holdPublication? }`. Setting
 `holdPublication: true` returns a durable `resourcePublicationBarrier`; omitted
 or `false` preserves immediate behavior.
@@ -1390,6 +1855,21 @@ address. With `cluster.quicPort: 0`, the operating system assigns its port; use
 Membership advertisements and peer `quicAddress` values contain that same bound
 address, including after a restart.
 
+#### `runHealingScan(options?: IRunHealingScanOptions): Promise<IClusterHealingScanResult>`
+
+Run a healing scan on this cluster node (see [Self-Healing](#self-healing)) and
+return its `startedAt`, `completedAt`, `durationMs`, `shardsChecked`,
+`shardsHealed`, `shardsPending` and `failed`. The request joins the next scan
+that starts after it arrives; a scan already running finishes first. It
+rejects in standalone mode, when the scan fails (without a majority of the
+cluster, or when a bucket's manifests cannot be listed) and when the engine
+shuts down before the scan completes, and it is refused while 16 requests
+already wait for a scan to start. The engine answers other management calls
+while the scan runs. `options.timeoutMs` (default 300,000) bounds the wait;
+the engine cannot cancel a scan, so one that outlasts it keeps running, its
+result is discarded, and a later call joins the next scan that starts after
+it.
+
 #### `getHealth(): Promise<IObjectStorageHealth>`
 
 Return running state, storage directory, per-location pool observations, auth
@@ -1416,7 +1896,7 @@ objectstorage uses a **hybrid Rust + TypeScript** architecture:
 │  │   ├─ FileStore (single-node mode)        │
 │  │   └─ DistributedStore (cluster mode)     │
 │  │       ├─ ErasureCoder (Reed-Solomon)     │
-│  │       ├─ ShardStore (per-drive storage)  │
+│  │       ├─ ShardBackend (per-drive shards) │
 │  │       ├─ QuicTransport (quinn)           │
 │  │       ├─ ClusterState & Membership       │
 │  │       └─ HealingService                  │
@@ -1433,7 +1913,7 @@ objectstorage uses a **hybrid Rust + TypeScript** architecture:
 
 **Why Rust?** The original TypeScript implementation had critical perf issues: OOM on multipart uploads (parts buffered in memory), double stream copying, file descriptor leaks on HEAD requests, full-file reads for range requests, and no backpressure. The Rust binary solves these with bounded-memory streaming I/O, backpressure, and direct `seek()` for range requests.
 
-**IPC Protocol:** TypeScript communicates with the `ruststorage` binary over newline-delimited JSON via stdin/stdout. The current management commands are `probeStoragePool`, `findOrphanObjectLeaves`, `getStoragePoolDiagnostic`, `start`, `stop`, `createBucket`, `createBucketTenant`, `ensureBucketTenant`, `deleteBucketTenant`, `rotateBucketTenantCredentials`, `listBucketTenants`, `getBucketTenantCredential`, `getBucketRetentionCapability`, `getBucketRetentionReceipt`, `getBucketMigrationCapability`, `createSourceMigration`, `createDestinationMigration`, `transitionSourceMigration`, `transitionDestinationMigration`, `cleanupSourceMigration`, `inspectBucketMigration`, `exportBucket`, `importBucket`, `deleteBucket`, `commitResourcePublication`, `getResourceFencingCapability`, `getBucketResourceFenceState`, `getStorageStats`, `listBucketSummaries`, `listCredentials`, `replaceCredentials`, and `getClusterHealth`. `getStoragePoolDiagnostic` reads a weak, cached diagnostic source and remains responsive while another management command owns the server; it does not retain the mounted root after server ownership completes.
+**IPC Protocol:** TypeScript communicates with the `ruststorage` binary over newline-delimited JSON via stdin/stdout. The current management commands are `probeStoragePool`, `findOrphanObjectLeaves`, `getStoragePoolDiagnostic`, `start`, `stop`, `createBucket`, `createBucketTenant`, `ensureBucketTenant`, `deleteBucketTenant`, `rotateBucketTenantCredentials`, `listBucketTenants`, `getBucketTenantCredential`, `getBucketRetentionCapability`, `getBucketRetentionReceipt`, `getBucketMigrationCapability`, `createSourceMigration`, `createDestinationMigration`, `transitionSourceMigration`, `transitionDestinationMigration`, `cleanupSourceMigration`, `inspectBucketMigration`, `exportBucket`, `importBucket`, `deleteBucket`, `commitResourcePublication`, `getResourceFencingCapability`, `getBucketResourceFenceState`, `getStorageStats`, `listBucketSummaries`, `listCredentials`, `replaceCredentials`, `createPrefixCredential`, `revokePrefixCredential`, `listPrefixCredentials`, `getPrefixCredentialStatus`, `getClusterHealth`, and `runHealingScan`. `getStoragePoolDiagnostic` reads a weak, cached diagnostic source and remains responsive while another management command owns the server; it does not retain the mounted root after server ownership completes. `runHealingScan` answers when its scan completes, with the request's `id`; the engine serves other commands meanwhile, and a shutdown answers a pending request with an error before the `stop` response.
 
 ### S3-Compatible Operations
 
@@ -1483,7 +1963,8 @@ those versions cannot read the canonical v2 objects.
 | ListParts | `GET /{bucket}/{key}?uploadId` | Optional `max-parts` and `part-number-marker` |
 | CompleteMultipartUpload | `POST /{bucket}/{key}?uploadId` | |
 | AbortMultipartUpload | `DELETE /{bucket}/{key}?uploadId` | |
-| ListMultipartUploads | `GET /{bucket}?uploads` | |
+| ListMultipartUploads | `GET /{bucket}?uploads` | Optional `prefix` |
+| GetBucketLocation | `GET /{bucket}?location` | Returns `server.region`; an empty constraint for `us-east-1` |
 | GetBucketPolicy | `GET /{bucket}?policy` | |
 | PutBucketPolicy | `PUT /{bucket}?policy` | |
 | DeleteBucketPolicy | `DELETE /{bucket}?policy` | |
@@ -1493,6 +1974,11 @@ those versions cannot read the canonical v2 objects.
 ListObjects pages identically in standalone and cluster mode, because both
 backends assemble a page through the same builder.
 
+A standalone listing walks only the part of the bucket its prefix and cursor
+can reach, and blocks only writes to the listed bucket while it walks: reads
+of every bucket, writes to other buckets and other listings of the same bucket
+proceed. A listing without a prefix still walks the whole bucket.
+
 **ListObjectsV2** (`?list-type=2`) reads `prefix`, `delimiter`, `max-keys`,
 `continuation-token` and `start-after`. Both cursors are exclusive: listing
 resumes strictly after the named key. A request may carry both; the
@@ -1502,6 +1988,14 @@ response echoes `ContinuationToken` and `StartAfter` when the request carried
 them, reports `KeyCount` as contents plus common prefixes, and returns
 `NextContinuationToken` whenever the page is truncated.
 
+A continuation token is opaque: pass `NextContinuationToken` back unchanged.
+It is the unpadded base64url (RFC 4648 §5) encoding of the UTF-8 key the next
+page resumes after, at most 1,366 characters, and is the same with or without
+`encoding-type=url`, so a client may change `encoding-type` between pages. A
+token the server could not have issued (outside the base64url alphabet, padded,
+not canonical, or not decoding to a 1 to 1,024-byte UTF-8 key) returns
+`InvalidArgument` (400); so do tokens issued before 11.0.0, which were raw keys.
+
 **ListObjects v1** (no `list-type`) reads `prefix`, `delimiter`, `max-keys` and
 `marker`, and ignores the V2 cursors. The response always carries `Marker`,
 empty when the request carried none, and carries `NextMarker` whenever the page
@@ -1518,14 +2012,16 @@ Paging rules shared by both versions:
   cursor.
 - `max-keys` above 1000 is clamped to 1000 and the response echoes the clamped
   value in `MaxKeys`, as S3 does. A `max-keys` value that is not a non-negative
-  integer is rejected with `InvalidRequest`, and so is a cursor longer than the
-  1,024-byte S3 key bound.
+  integer is rejected with `InvalidRequest`, and so is a `start-after` or
+  `marker` longer than the 1,024-byte S3 key bound.
 - An empty `start-after`, `continuation-token` or `marker` value means no cursor.
 - Keys are returned in ascending UTF-8 byte order, and a key is never returned
   twice across pages of an unchanged bucket.
 
 ListMultipartUploads is not paginated: it returns every in-flight upload of the
-bucket with `IsTruncated` false.
+bucket with `IsTruncated` false. An optional `prefix` parameter limits the
+result to uploads whose keys start with it and is echoed in the response;
+prefix credentials must supply one inside their prefix.
 
 ### On-Disk Format
 
@@ -1547,8 +2043,20 @@ bucket with `IsTruncated` false.
     {bucket}.policy.json                 # Bucket policy (IAM JSON format)
   .smartstorage/staging/v1/              # Private staging: a write lands here first; its
                                          # key leaf appears only with the complete payload
+  .smartstorage/credentials.json         # Runtime and bucket tenant credentials (0600)
+  .smartstorage/prefix-credentials.json  # Prefix credentials and revocations (0600)
 ```
 
+Rolling a standalone root back from 10.7.0 or later to 8.0.3–10.6.0: a crash
+between staging and publication of a PUT, CopyObject or CompleteMultipartUpload
+can leave a staged leaf directory
+`.smartstorage/staging/v1/.smartstorage-<uuid>.object.tmp.leaf/`. Startup of
+10.7.0 or later removes it; 8.0.3–10.6.0 refuse to start with "Standalone
+staging directory contains an unowned entry". Start and stop the newer engine
+once on the root before rolling back. If it cannot start, back up the root and
+then remove only those `*.object.tmp.leaf` directories: they hold payloads that
+were never published.
+
 **Cluster mode:**
 ```
 {storage.directory}/
@@ -1560,7 +2068,8 @@ bucket with `IsTruncated` false.
       {object-address}.state.json
                                          # Active/tombstoned object manifest
     .multipart/{upload-id}/               # Descriptor-anchored multipart state
-    .get-tmp/                             # Unlinked GET reconstruction files
+  .smartstorage/credentials.json          # This node's runtime credentials (not replicated)
+  .smartstorage/prefix-credentials.json   # This node's prefix credentials (not replicated)
   .smartstorage/cluster/
     identity.json                         # Node and cluster identity
     topology.json                         # Last durable topology
@@ -1573,8 +2082,12 @@ bucket with `IsTruncated` false.
 ```
 
 Superseded physical generations are queued with their exact shard placements.
-Cluster reads are bounded, cleanup waits a 24-hour reader grace period, and the
-hourly cleanup worker purges both shard records and tombstones. Multipart
+They keep their shards for the cleanup grace derived from `cluster.getDeadlineMs`
+(see Streaming Reads), and the cleanup worker, which runs once per grace period
+and at least hourly, then purges both shard records and tombstones. Earlier
+engines spooled GET responses into `.manifests/.get-tmp/`; startup removes a
+leftover spool directory once it holds only spool files, and otherwise leaves
+it in place, unused. Multipart
 uploads older than `multipart.expirationDays`, plus committed uploads whose
 first cleanup attempt failed, are retried during bounded cleanup passes
 scheduled every `multipart.cleanupIntervalMinutes`; large registries are
@@ -1597,7 +2110,7 @@ established.
 
 - [`@push.rocks/smartbucket`](https://code.foss.global/push.rocks/smartbucket) — High-level S3-compatible abstraction layer
 - [`@push.rocks/smartrust`](https://code.foss.global/push.rocks/smartrust) — TypeScript ↔ Rust IPC bridge
-- [`@git.zone/tsrust`](https://code.foss.global/git.zone/tsrust) — Rust cross-compilation for npm packages
+- [`@git.zone/rsrepo`](https://code.foss.global/git.zone/rsrepo) — Rust builds and cross-compilation for npm packages
 
 ## License and Legal Information