@nestm/storage 0.1.0-alpha.7 → 0.1.0-alpha.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +135 -0
- package/README.md +302 -18
- package/SECURITY.md +34 -5
- package/dist/ai-sdk/ai-sdk-workspace-tools.d.ts +21 -3
- package/dist/ai-sdk/ai-sdk-workspace-tools.d.ts.map +1 -1
- package/dist/ai-sdk/ai-sdk-workspace-tools.js +163 -73
- package/dist/ai-sdk/ai-sdk-workspace-tools.js.map +1 -1
- package/dist/ai-sdk/index.d.ts +1 -1
- package/dist/ai-sdk/index.d.ts.map +1 -1
- package/dist/ai-sdk/index.js.map +1 -1
- package/dist/files-sdk/files-sdk.driver.d.ts +54 -10
- package/dist/files-sdk/files-sdk.driver.d.ts.map +1 -1
- package/dist/files-sdk/files-sdk.driver.js +831 -63
- package/dist/files-sdk/files-sdk.driver.js.map +1 -1
- package/dist/files-sdk/fs/index.d.ts +4 -4
- package/dist/files-sdk/fs/index.d.ts.map +1 -1
- package/dist/files-sdk/fs/index.js +242 -9
- package/dist/files-sdk/fs/index.js.map +1 -1
- package/dist/files-sdk/index.d.ts +1 -1
- package/dist/files-sdk/index.d.ts.map +1 -1
- package/dist/files-sdk/index.js.map +1 -1
- package/dist/files-sdk/provider/index.d.ts +9 -5
- package/dist/files-sdk/provider/index.d.ts.map +1 -1
- package/dist/files-sdk/provider/index.js +53 -15
- package/dist/files-sdk/provider/index.js.map +1 -1
- package/dist/files-sdk/s3/construction-metadata.d.ts +7 -0
- package/dist/files-sdk/s3/construction-metadata.d.ts.map +1 -0
- package/dist/files-sdk/s3/construction-metadata.js +13 -0
- package/dist/files-sdk/s3/construction-metadata.js.map +1 -0
- package/dist/files-sdk/s3/index.d.ts +39 -19
- package/dist/files-sdk/s3/index.d.ts.map +1 -1
- package/dist/files-sdk/s3/index.js +936 -113
- package/dist/files-sdk/s3/index.js.map +1 -1
- package/dist/gateway/storage-gateway.controller.d.ts.map +1 -1
- package/dist/gateway/storage-gateway.controller.js +25 -5
- package/dist/gateway/storage-gateway.controller.js.map +1 -1
- package/dist/storage-etag.d.ts +13 -0
- package/dist/storage-etag.d.ts.map +1 -0
- package/dist/storage-etag.js +31 -0
- package/dist/storage-etag.js.map +1 -0
- package/dist/storage.client.d.ts +3 -1
- package/dist/storage.client.d.ts.map +1 -1
- package/dist/storage.client.js +87 -22
- package/dist/storage.client.js.map +1 -1
- package/dist/storage.driver.d.ts +21 -1
- package/dist/storage.driver.d.ts.map +1 -1
- package/dist/storage.driver.js.map +1 -1
- package/dist/storage.types.d.ts +90 -19
- package/dist/storage.types.d.ts.map +1 -1
- package/dist/storage.types.js.map +1 -1
- package/dist/testing/index.d.ts +1 -0
- package/dist/testing/index.d.ts.map +1 -1
- package/dist/testing/index.js +1 -0
- package/dist/testing/index.js.map +1 -1
- package/dist/testing/provider-conformance.d.ts +65 -0
- package/dist/testing/provider-conformance.d.ts.map +1 -0
- package/dist/testing/provider-conformance.js +853 -0
- package/dist/testing/provider-conformance.js.map +1 -0
- package/dist/workspace/index.d.ts +2 -1
- package/dist/workspace/index.d.ts.map +1 -1
- package/dist/workspace/index.js +1 -0
- package/dist/workspace/index.js.map +1 -1
- package/dist/workspace/storage-workspace.cursor.d.ts +68 -3
- package/dist/workspace/storage-workspace.cursor.d.ts.map +1 -1
- package/dist/workspace/storage-workspace.cursor.js +402 -39
- package/dist/workspace/storage-workspace.cursor.js.map +1 -1
- package/dist/workspace/storage-workspace.d.ts.map +1 -1
- package/dist/workspace/storage-workspace.js +162 -62
- package/dist/workspace/storage-workspace.js.map +1 -1
- package/dist/workspace/storage-workspace.types.d.ts +31 -5
- package/dist/workspace/storage-workspace.types.d.ts.map +1 -1
- package/dist/workspace/storage-workspace.types.js +2 -0
- package/dist/workspace/storage-workspace.types.js.map +1 -1
- package/package.json +3 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,140 @@
|
|
|
1
1
|
# @nestm/storage
|
|
2
2
|
|
|
3
|
+
## 0.1.0-alpha.9
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- aec25d6: Add explicit last-write-wins workspace write, copy, and unconditional-delete
|
|
8
|
+
operations that traverse the ordinary Files SDK plugin, hook, and receipt
|
|
9
|
+
pipeline while retaining the existing native conditional create, replace,
|
|
10
|
+
copy, move, and delete variants. Unconditional delete requires both `write` and
|
|
11
|
+
`delete`; move remains conditional-only because a non-atomic
|
|
12
|
+
download/upload/delete sequence could delete a newer source generation. Add a
|
|
13
|
+
separate `write` permission and an AI tool factory mutation-mode switch whose
|
|
14
|
+
default remains conditional.
|
|
15
|
+
|
|
16
|
+
Add bounded binary workspace reads through `readBytes`, alongside the existing
|
|
17
|
+
UTF-8 `readText` API. `readBytes` is a required `StorageWorkspace` member, so
|
|
18
|
+
custom interface implementations and typed test doubles must add it when
|
|
19
|
+
upgrading; workspaces returned by `mountStorageWorkspace` need no changes.
|
|
20
|
+
|
|
21
|
+
### Patch Changes
|
|
22
|
+
|
|
23
|
+
- 37e0d8d: Fail conditional storage operations closed when caller-configured Files SDK
|
|
24
|
+
plugins, hooks, or receipts would be bypassed by native adapter extensions.
|
|
25
|
+
Ordinary operations continue through Files SDK while incompatible conditional
|
|
26
|
+
capabilities are hidden until Files SDK exposes one shared interception boundary.
|
|
27
|
+
|
|
28
|
+
## 0.1.0-alpha.8
|
|
29
|
+
|
|
30
|
+
### Minor Changes
|
|
31
|
+
|
|
32
|
+
- d81c6f4: Add a typed `mapCreateConflict` hook to the AI SDK workspace adapter so
|
|
33
|
+
applications can represent atomic create collisions as domain results without
|
|
34
|
+
mutating generated tools. Keep replace/ETag conflicts fail-closed and sanitize
|
|
35
|
+
mapper failures at the tool boundary.
|
|
36
|
+
|
|
37
|
+
Mark workspace tools with optional inputs or a combined create/replace union as
|
|
38
|
+
non-strict for provider schema generation while retaining strict Zod runtime
|
|
39
|
+
validation.
|
|
40
|
+
|
|
41
|
+
- a0ea392: Add injectable, replica-safe workspace pagination cursors. The workspace now
|
|
42
|
+
binds versioned cursor payloads to stable store, mount, tenant/workspace, prefix,
|
|
43
|
+
operation, query, limit, and expiry context; authorizes non-consuming replay
|
|
44
|
+
before that expiry; and rejects altered or cross-context continuations.
|
|
45
|
+
Successful continuation still depends on the embedded provider cursor remaining
|
|
46
|
+
valid and available.
|
|
47
|
+
|
|
48
|
+
Export an AES-256-GCM key-ring codec for stateless multi-replica deployments and
|
|
49
|
+
an asynchronous byte-payload codec contract for shared durable opaque-token
|
|
50
|
+
stores. Cursor payloads and tokens are bounded, provider continuations remain
|
|
51
|
+
opaque, and pagination fails closed when no cursor mechanism is configured.
|
|
52
|
+
Compatible replicas rely on the universal driver contract for non-consuming,
|
|
53
|
+
instance-portable provider cursors whose position is independent of page size
|
|
54
|
+
while the provider token remains valid. Cursor expiry is not a provider-token
|
|
55
|
+
retention, snapshot-isolation, or uptime promise; provider invalidation remains
|
|
56
|
+
an operational list failure.
|
|
57
|
+
|
|
58
|
+
- d996b92: Split the aggregate S3 conditional-mutation and copy declarations into exact
|
|
59
|
+
create, replace, delete, read, source-copy, destination-copy, atomic-promotion,
|
|
60
|
+
and multipart-completion capabilities. Add independent AWS S3, Cloudflare R2,
|
|
61
|
+
and fail-closed custom-endpoint profiles that force unverified drivers
|
|
62
|
+
read-only; enforce complete physical-key byte budgets; normalize provider
|
|
63
|
+
errors without retaining raw provider payloads or causes; and publish a reusable
|
|
64
|
+
provider conformance contract with gated filesystem, AWS, R2, and custom suites.
|
|
65
|
+
|
|
66
|
+
Normalize every provider ETag to a canonical bare 1–1024-byte visible ASCII
|
|
67
|
+
token and serialize exactly one HTTP quote pair at S3 request boundaries.
|
|
68
|
+
Quoted, weak, wildcard, list-shaped, control-bearing, non-ASCII, and otherwise
|
|
69
|
+
unsafe values now fail closed instead of being accepted as arbitrary non-empty
|
|
70
|
+
strings. Applications that persisted quoted ETags must refresh them from
|
|
71
|
+
provider metadata before conditional mutation; this prevents wildcard/list and
|
|
72
|
+
header-ambiguity inputs from widening an exact-match precondition.
|
|
73
|
+
|
|
74
|
+
Specify and test provider list cursors as non-consuming replayable tokens for
|
|
75
|
+
unchanged provider state, so higher-level replayable pagination can fail closed
|
|
76
|
+
when a provider cannot meet that contract.
|
|
77
|
+
|
|
78
|
+
Exercise the complete advertised source-condition by destination-condition
|
|
79
|
+
promotion matrix, including stale-state preservation and competing stale/valid
|
|
80
|
+
requests. Provider documentation or audited implementation evidence remains
|
|
81
|
+
required for the internal one-linearization-point claim.
|
|
82
|
+
|
|
83
|
+
Ordinary and conditional provider failures now expose stable public messages
|
|
84
|
+
and preserve only normalized codes and retry flags; raw provider bodies,
|
|
85
|
+
request metadata, and nested causes are not retained in loggable error shapes.
|
|
86
|
+
|
|
87
|
+
Raise the `@aws-sdk/client-s3` peer floor to 3.919.0, the first release that
|
|
88
|
+
serializes destination conditions for `CopyObject`, and verify the real wire
|
|
89
|
+
headers in the packed minimum-peer consumer. Native AWS construction now
|
|
90
|
+
disables environment and shared-config endpoint URL overrides, while the public
|
|
91
|
+
capability helper derives custom-endpoint provenance from the actual SDK client
|
|
92
|
+
instead of a duplicated caller hint. Capability decoration is now single-use
|
|
93
|
+
per raw S3 adapter, preventing broader operations from surviving a later
|
|
94
|
+
narrower profile application.
|
|
95
|
+
|
|
96
|
+
Bind S3 provider authority to package-private raw-client and adapter-method
|
|
97
|
+
identity plus the exact surface snapshot installed by capability decoration.
|
|
98
|
+
Structurally S3-backed raw adapters are rejected until they pass through the
|
|
99
|
+
package helper, regardless of adapter name, proxying, or forged global symbols;
|
|
100
|
+
same-client aliases cannot replace their raw client, ordinary methods, policies,
|
|
101
|
+
or conditional operations. Unverified custom endpoints and noncanonical
|
|
102
|
+
S3-backed provider slugs are forced read-only, while an explicit branded profile
|
|
103
|
+
unlocks only its declared conditional operations. Endpoint and public-URL
|
|
104
|
+
provenance now follows the adapter actually produced by the provider loader,
|
|
105
|
+
including `configJson`.
|
|
106
|
+
|
|
107
|
+
Retain `publicBaseUrl` construction policy in the package-owned `s3()` helper
|
|
108
|
+
so omitted decorator hints cannot re-enable an expiring-download claim. Unknown
|
|
109
|
+
foreign S3 construction conservatively disables that claim. Validate the exact
|
|
110
|
+
physical adapter key without stripping leading slashes, and include configured
|
|
111
|
+
separators plus list/search-derived prefixes in the provider byte budget before
|
|
112
|
+
dispatch. Search uses files-sdk's own inferred glob-prefix and zero-result
|
|
113
|
+
semantics instead of duplicating its matcher logic. The innermost dispatch guard
|
|
114
|
+
repeats these checks after supported in-process plugins have transformed an
|
|
115
|
+
operation; adapters and plugins remain trusted code rather than a sandbox
|
|
116
|
+
boundary.
|
|
117
|
+
|
|
118
|
+
Derive signed-upload policy claims from the branded provider profile instead
|
|
119
|
+
of granting them to every S3-compatible endpoint. Native AWS proves content
|
|
120
|
+
type and POST size-range enforcement; Cloudflare R2 proves content type but not
|
|
121
|
+
POST form size ranges; omitted custom declarations normalize to false/false so
|
|
122
|
+
the gateway fails closed. Profile authority is backed by a package-private
|
|
123
|
+
WeakSet after deep freezing, so reflecting and copying the nominal brand symbol
|
|
124
|
+
cannot forge conformance evidence.
|
|
125
|
+
|
|
126
|
+
Treat the built-in AWS profile as an immutable ceiling for every SDK client
|
|
127
|
+
with native endpoint provenance, independent of mutable adapter display names.
|
|
128
|
+
Explicit profiles may narrow its operations, policy bits, and key limit but
|
|
129
|
+
cannot raise the 1,024-byte physical-key budget or add unsupported claims.
|
|
130
|
+
|
|
131
|
+
Enforce every requested signed-upload constraint at URL creation time.
|
|
132
|
+
Content-type-constrained PUT URLs now sign the `content-type` header; bounded
|
|
133
|
+
AWS uploads use exact POST MIME and byte-range conditions; unsupported profile
|
|
134
|
+
constraints and lower-only S3 ranges fail before signing. Bounded POST uploads
|
|
135
|
+
also reject physical keys ending in AWS's `${filename}` template so an exact
|
|
136
|
+
authorized key cannot be widened into a prefix policy.
|
|
137
|
+
|
|
3
138
|
## 0.1.0-alpha.7
|
|
4
139
|
|
|
5
140
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -36,6 +36,14 @@ such as `files-sdk/gcs`:
|
|
|
36
36
|
pnpm add files-sdk@2.2.3
|
|
37
37
|
```
|
|
38
38
|
|
|
39
|
+
Pass AWS-SDK-backed S3 adapters through the package-owned `s3()` and
|
|
40
|
+
`withS3Capabilities()` helpers (or use `createS3StorageDriver()`).
|
|
41
|
+
`createFilesSdkDriver()` rejects a structurally S3-backed raw adapter that has
|
|
42
|
+
not crossed this provenance boundary, even when a wrapper renames or proxies
|
|
43
|
+
the adapter. This prevents callers from bypassing endpoint and provider-profile
|
|
44
|
+
checks by omitting the capability decorator or replacing the public `raw`
|
|
45
|
+
property while retaining methods closed over the original client.
|
|
46
|
+
|
|
39
47
|
Install only the native SDKs required by the chosen provider. For example:
|
|
40
48
|
|
|
41
49
|
```sh
|
|
@@ -50,6 +58,11 @@ pnpm add @google-cloud/storage google-auth-library
|
|
|
50
58
|
pnpm add @azure/storage-blob @azure/core-auth @azure/identity
|
|
51
59
|
```
|
|
52
60
|
|
|
61
|
+
`@aws-sdk/client-s3` 3.919.0 or newer is required. Earlier releases omit
|
|
62
|
+
destination `If-Match` and `If-None-Match` from the serialized `CopyObject`
|
|
63
|
+
request even when those fields appear in a newer compile-time command shape;
|
|
64
|
+
the package peer range and packed minimum-peer smoke test enforce this floor.
|
|
65
|
+
|
|
53
66
|
`files-sdk` currently declares its optional Nest peer for Nest 10 and 11. This
|
|
54
67
|
library does not import `files-sdk/nestjs`; the Nest 12 integration is entirely
|
|
55
68
|
owned here. A package manager may nevertheless report that temporary optional
|
|
@@ -110,16 +123,51 @@ control characters, repeated separators, and `.` or `..` segments are rejected
|
|
|
110
123
|
rather than normalized. Keys and provider cursors returned by a driver are also
|
|
111
124
|
checked before they are converted back to logical paths.
|
|
112
125
|
|
|
126
|
+
Pagination requires a server-owned cursor configuration. The built-in
|
|
127
|
+
`Aes256GcmStorageWorkspaceCursorCodec` produces versioned, authenticated,
|
|
128
|
+
encrypted tokens that can resume on another request, process, or replica when
|
|
129
|
+
every replica constructs an equivalent codec from the same key ring and uses
|
|
130
|
+
the same stable store identity, physical prefix, mount ID, trusted scope, and
|
|
131
|
+
effective limits. Use one codec instance per process, use a dedicated 32-byte
|
|
132
|
+
key, retain rotated decryption keys for at least one cursor TTL, and derive
|
|
133
|
+
`mountId` and `scope` only from authenticated server context.
|
|
134
|
+
|
|
135
|
+
The underlying driver must also implement the universal replayable list-cursor
|
|
136
|
+
contract against the same logical backend namespace: its cursor cannot be
|
|
137
|
+
consumed or tied to one driver instance. While the provider cursor remains
|
|
138
|
+
valid and available, an outer cursor can be retried until its authenticated
|
|
139
|
+
expiry. That expiry is only an authorization ceiling: it does not extend a
|
|
140
|
+
provider token's lifetime or promise snapshot isolation, provider availability,
|
|
141
|
+
network access, or valid credentials. Provider invalidation is an operational
|
|
142
|
+
list failure, and concurrent object changes remain subject to provider
|
|
143
|
+
continuation semantics. Without a codec, single-page operations still work but
|
|
144
|
+
a continuation fails closed.
|
|
145
|
+
|
|
113
146
|
```ts
|
|
114
|
-
import {
|
|
147
|
+
import {
|
|
148
|
+
Aes256GcmStorageWorkspaceCursorCodec,
|
|
149
|
+
mountStorageWorkspace,
|
|
150
|
+
} from '@nestm/storage/workspace';
|
|
151
|
+
|
|
152
|
+
// cursorKey is a separately validated 32-byte secret from deployment config.
|
|
153
|
+
const cursorCodec = new Aes256GcmStorageWorkspaceCursorCodec({
|
|
154
|
+
activeKeyId: 'v1',
|
|
155
|
+
keys: { v1: cursorKey },
|
|
156
|
+
});
|
|
115
157
|
|
|
116
158
|
const workspace = mountStorageWorkspace(agentFiles, {
|
|
117
159
|
// Use an opaque server-derived run id, never a value selected by the model.
|
|
118
160
|
prefix: `workspaces/${runId}`,
|
|
161
|
+
cursor: {
|
|
162
|
+
codec: cursorCodec,
|
|
163
|
+
mountId: `agent-workspace:${runId}`,
|
|
164
|
+
scope: `organization:${organizationId}/workspace:${workspaceId}`,
|
|
165
|
+
},
|
|
119
166
|
permissions: [
|
|
120
167
|
'list',
|
|
121
168
|
'read',
|
|
122
169
|
'search',
|
|
170
|
+
'write',
|
|
123
171
|
'create',
|
|
124
172
|
'replace',
|
|
125
173
|
'copy',
|
|
@@ -127,6 +175,8 @@ const workspace = mountStorageWorkspace(agentFiles, {
|
|
|
127
175
|
'delete',
|
|
128
176
|
],
|
|
129
177
|
limits: {
|
|
178
|
+
cursorTtlMs: 5 * 60 * 1000,
|
|
179
|
+
maxCursorBytes: 4096,
|
|
130
180
|
maxReadBytes: 1024 * 1024,
|
|
131
181
|
maxWriteBytes: 1024 * 1024,
|
|
132
182
|
maxPageSize: 100,
|
|
@@ -150,8 +200,16 @@ await workspace.writeFile('src/main.ts', 'export const ready = false;\n', {
|
|
|
150
200
|
etag: created.etag,
|
|
151
201
|
contentType: 'text/typescript',
|
|
152
202
|
});
|
|
203
|
+
|
|
204
|
+
const image = await workspace.readBytes('assets/logo.png');
|
|
205
|
+
console.log(image.bytes.byteLength);
|
|
153
206
|
```
|
|
154
207
|
|
|
208
|
+
`readBytes` is now a required member of the exported `StorageWorkspace`
|
|
209
|
+
interface. Workspaces returned by `mountStorageWorkspace` provide it
|
|
210
|
+
automatically; custom implementations and typed test doubles must add the
|
|
211
|
+
method when adopting this alpha minor.
|
|
212
|
+
|
|
155
213
|
Create, replace, and delete are conditional operations. A driver that cannot
|
|
156
214
|
enforce the requested not-exists or ETag precondition fails with
|
|
157
215
|
`NOT_SUPPORTED`; the workspace never substitutes an `exists()`/`head()` check
|
|
@@ -166,6 +224,32 @@ the call returns `CONFLICT`; inspect both logical paths before retrying. This
|
|
|
166
224
|
preserves at least one copy across provider timeouts and post-operation hook
|
|
167
225
|
failures, but does not pretend a multi-object move is transactionally atomic.
|
|
168
226
|
|
|
227
|
+
Callers that prefer the ordinary Files pipeline can opt into explicit
|
|
228
|
+
last-write-wins variants. The `write` permission is separate from conditional
|
|
229
|
+
`create` and `replace` authority:
|
|
230
|
+
|
|
231
|
+
```ts
|
|
232
|
+
await workspace.writeFile('notes.txt', 'latest contents', {
|
|
233
|
+
mode: 'overwrite',
|
|
234
|
+
});
|
|
235
|
+
await workspace.copyFile('notes.txt', 'backup.txt', { mode: 'overwrite' });
|
|
236
|
+
await workspace.deleteFile('backup.txt', { mode: 'unconditional' });
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
Overwrite copy reads the latest source through the ordinary download pipeline,
|
|
240
|
+
enforces `maxWriteBytes` while collecting it, and uploads it through the
|
|
241
|
+
ordinary upload pipeline. It never substitutes the provider's server-side
|
|
242
|
+
copy. These paths compose with Files SDK plugins, hooks, and receipts, including
|
|
243
|
+
the built-in `encryption()` plugin. That plugin is useful compatibility
|
|
244
|
+
evidence, not an Artifact-specific security policy: strict encrypted-only
|
|
245
|
+
reads, tenant/path-bound AAD, key custody and rotation, and copy/move rules
|
|
246
|
+
remain application-owned.
|
|
247
|
+
|
|
248
|
+
Move remains conditional-only. A last-write-wins download/upload/delete
|
|
249
|
+
sequence could copy one source generation and then delete a newer generation
|
|
250
|
+
written during the transfer. Use the ETag-conditional `moveFile` variant when a
|
|
251
|
+
move is required.
|
|
252
|
+
|
|
169
253
|
A child mount may further restrict a directory, permissions, or limits, but it
|
|
170
254
|
cannot widen any of them:
|
|
171
255
|
|
|
@@ -220,10 +304,13 @@ import type { ToolSet } from 'ai';
|
|
|
220
304
|
// A validated, opaque coordinate from trusted auth/run state.
|
|
221
305
|
// It is never accepted from a prompt or tool input.
|
|
222
306
|
prefix: context.storagePrefix,
|
|
307
|
+
// Includes the singleton codec plus stable mountId and scope.
|
|
308
|
+
cursor: context.cursorConfiguration,
|
|
223
309
|
permissions: [
|
|
224
310
|
'list',
|
|
225
311
|
'read',
|
|
226
312
|
'search',
|
|
313
|
+
'write',
|
|
227
314
|
'create',
|
|
228
315
|
'replace',
|
|
229
316
|
'copy',
|
|
@@ -261,6 +348,32 @@ workspace capability remains the authorization boundary even when approval is
|
|
|
261
348
|
disabled. The module's `AiSdkService.files()` API is the model provider's file
|
|
262
349
|
upload facility and is unrelated to storage workspaces.
|
|
263
350
|
|
|
351
|
+
`mutationMode` defaults to `'conditional'`. A trusted composition can instead
|
|
352
|
+
select `{ mutationMode: 'last-write-wins' }`; generated mutation schemas then
|
|
353
|
+
omit ETags and modes, hardcode the explicit overwrite/unconditional workspace
|
|
354
|
+
variants, and require `write` permission for destination mutations.
|
|
355
|
+
Unconditional delete requires both `write` and `delete`. The move tool is
|
|
356
|
+
omitted in last-write-wins mode because Workspace move remains conditional-only.
|
|
357
|
+
|
|
358
|
+
Atomic create collisions remain sanitized tool errors by default. Applications
|
|
359
|
+
that model an existing destination as a normal tool result can map that one
|
|
360
|
+
case while preserving replace/ETag conflicts as failures:
|
|
361
|
+
|
|
362
|
+
```ts
|
|
363
|
+
const tools = createAiSdkWorkspaceTools({
|
|
364
|
+
workspace,
|
|
365
|
+
mapCreateConflict: ({ path }) => ({
|
|
366
|
+
kind: 'artifact-conflict' as const,
|
|
367
|
+
path,
|
|
368
|
+
status: 'already-exists' as const,
|
|
369
|
+
}),
|
|
370
|
+
});
|
|
371
|
+
```
|
|
372
|
+
|
|
373
|
+
The mapper receives only the logical workspace path; provider errors, object
|
|
374
|
+
keys, and mount coordinates are never exposed. `mapCreateConflict` is valid
|
|
375
|
+
only in conditional mode and is rejected with last-write-wins mode.
|
|
376
|
+
|
|
264
377
|
This logical confinement is sufficient for a `ToolLoopAgent` whose only file
|
|
265
378
|
capabilities are these tools. It cannot constrain a coding harness that already
|
|
266
379
|
has shell, `node:fs`, or subprocess access. For Codex/Claude-style harnesses,
|
|
@@ -463,19 +576,77 @@ listStorageProviderSecretEnvVars('s3').map((variable) => variable.key);
|
|
|
463
576
|
The catalog is pure data and pulls in no adapter, so it is safe in config UIs,
|
|
464
577
|
health checks, and startup validation.
|
|
465
578
|
|
|
466
|
-
The `s3` slug additionally carries the
|
|
467
|
-
capabilities described under
|
|
468
|
-
[
|
|
469
|
-
|
|
470
|
-
known at build time, import
|
|
471
|
-
`@nestm/storage/files-sdk/fs` directly and skip
|
|
579
|
+
The `s3` slug additionally carries the verified per-operation profile and
|
|
580
|
+
signed-policy capabilities described under
|
|
581
|
+
[Exact provider conditions and staged-object promotion](#exact-provider-conditions-and-staged-object-promotion);
|
|
582
|
+
every provider not backed by the AWS S3 SDK exposes what its adapter declares.
|
|
583
|
+
When the provider _is_ known at build time, import
|
|
584
|
+
`@nestm/storage/files-sdk/s3` or `@nestm/storage/files-sdk/fs` directly and skip
|
|
585
|
+
the indirection.
|
|
586
|
+
|
|
587
|
+
S3 endpoint and public-URL provenance is resolved from the adapter that
|
|
588
|
+
`files-sdk` actually constructs, including values merged from `configJson`.
|
|
589
|
+
An unaudited endpoint forces the driver read-only, and a `publicBaseUrl` removes
|
|
590
|
+
the signed-download TTL guarantee because the resulting public URL does not
|
|
591
|
+
expire. AWS-SDK-backed noncanonical provider slugs (for example an S3-compatible
|
|
592
|
+
provider wrapper) also default to unverified/read-only; only the canonical
|
|
593
|
+
`s3` provider may infer the native AWS profile, and a custom endpoint becomes
|
|
594
|
+
writable only with an explicit branded `S3ProviderProfile`.
|
|
595
|
+
|
|
596
|
+
Before enabling conditional operations for a custom S3-compatible endpoint,
|
|
597
|
+
run the reusable
|
|
598
|
+
[provider conformance contract](https://github.com/nestm-dev/storage/blob/main/docs/provider-conformance.md)
|
|
599
|
+
against dedicated test credentials. Unknown endpoints are forced read-only and
|
|
600
|
+
receive no inferred conditional capabilities.
|
|
601
|
+
|
|
602
|
+
## Files SDK responsibility boundary
|
|
603
|
+
|
|
604
|
+
Files SDK is the upstream authority for the generic storage data plane:
|
|
605
|
+
provider adapters, generic CRUD, bulk and list operations, retries, transfers
|
|
606
|
+
and sync, its plugin pipeline, and framework-neutral gateway mechanics.
|
|
607
|
+
`@nestm/storage` retains the guarantees that Files SDK does not currently
|
|
608
|
+
provide: NestJS 12 named stores, exact native conditional/CAS capabilities,
|
|
609
|
+
`StorageWorkspace` permissions and limits, bounded storage errors, and
|
|
610
|
+
capability-scoped AI tools.
|
|
611
|
+
|
|
612
|
+
On the alpha.8 base, ordinary `FilesSdkStorageDriver` operations already
|
|
613
|
+
delegate to the Files SDK pipeline. The exception is the native conditional
|
|
614
|
+
adapter extensions: the current Files SDK operation union does not include
|
|
615
|
+
them, so they cannot run through caller-configured Files plugins, hooks, or
|
|
616
|
+
receipts. Until Files SDK provides one interception boundary for ordinary and
|
|
617
|
+
conditional operations, the driver applies this interim fail-closed
|
|
618
|
+
compatibility rule:
|
|
619
|
+
|
|
620
|
+
| Caller Files configuration | Ordinary operations | Conditional operations |
|
|
621
|
+
| ------------------------------------------------- | ------------------------------------ | -------------------------------------------------------- |
|
|
622
|
+
| No plugins, active hooks, or receipts | Files pipeline | Advertised when the adapter supports the exact primitive |
|
|
623
|
+
| One or more plugins | Files pipeline, including transforms | Hidden; direct invocation returns `NOT_SUPPORTED` |
|
|
624
|
+
| Any active hook | Files pipeline and hook callbacks | Hidden; direct invocation returns `NOT_SUPPORTED` |
|
|
625
|
+
| Receipts enabled with `true` or an options object | Files pipeline and receipts | Hidden; direct invocation returns `NOT_SUPPORTED` |
|
|
626
|
+
|
|
627
|
+
An empty plugin list, an empty hooks object, and `receipts: false` do not trigger
|
|
628
|
+
the gate. NestM's internal physical-key guard does not trigger it either. When
|
|
629
|
+
available, direct conditional paths independently apply prefixing, the
|
|
630
|
+
physical-key budget, mutation read-only restrictions, default
|
|
631
|
+
retry/signal/timeout options, and bounded error mapping. `StoragePlugin` remains
|
|
632
|
+
a separate veto/observation boundary; it is not a substitute for Files body or
|
|
633
|
+
result transforms. This compatibility gate is intended to be removed once
|
|
634
|
+
native CAS can traverse the upstream operation and plugin pipeline rather than
|
|
635
|
+
becoming a second generic CRUD facade here.
|
|
636
|
+
|
|
637
|
+
`StorageWorkspace` therefore exposes both contracts without weakening either:
|
|
638
|
+
its existing create/replace, exact-read copy/move, and conditional-delete paths
|
|
639
|
+
retain native CAS and this fail-closed gate, while explicit overwrite and
|
|
640
|
+
unconditional-delete variants use the ordinary Files pipeline. Lower-level
|
|
641
|
+
conditional client and driver APIs remain available to callers that need them.
|
|
472
642
|
|
|
473
643
|
## Storage API
|
|
474
644
|
|
|
475
645
|
`StorageClient` exposes:
|
|
476
646
|
|
|
477
647
|
- `upload`, `downloadStream`, `head`, `exists`, `delete`, `copy`, and `move`;
|
|
478
|
-
-
|
|
648
|
+
- exact `uploadConditional`, `downloadConditional`, `deleteConditional`, and
|
|
649
|
+
staged-object `promote` operations when the driver advertises each primitive;
|
|
479
650
|
- `list`, cursor-aware `listAll`, and lazy `search`;
|
|
480
651
|
- `signDownload` and discriminated PUT/POST `signUpload`;
|
|
481
652
|
- `uploadMany`, `downloadMany`, `headMany`, `existsMany`, and `deleteMany`;
|
|
@@ -483,6 +654,24 @@ known at build time, import `@nestm/storage/files-sdk/s3` or
|
|
|
483
654
|
- provider capability inspection; and
|
|
484
655
|
- pause/resume/abort through `StorageUploadControl`.
|
|
485
656
|
|
|
657
|
+
Provider list cursors are opaque, non-consuming continuation tokens. Replaying
|
|
658
|
+
the same cursor and page limit against unchanged provider-visible state must
|
|
659
|
+
return an equivalent page and continuation position, even after a descendant
|
|
660
|
+
cursor has been used. A cursor is bound to the logical store, `prefix`, and
|
|
661
|
+
`delimiter`, but not to `limit`, retries, timeout, or abort signal; callers may
|
|
662
|
+
change those transport/page-size options while resuming the same position.
|
|
663
|
+
|
|
664
|
+
The cursor must work through a newly constructed compatible driver targeting
|
|
665
|
+
the same backend namespace while the provider token remains valid and
|
|
666
|
+
available; it cannot depend on process-, client-, or session-local state. An
|
|
667
|
+
adapter for a consuming or instance-bound provider token must materialize a
|
|
668
|
+
stable continuation before it can provide conforming paginated
|
|
669
|
+
`StorageDriver.list` results. This contract lets a caller safely retry, replay,
|
|
670
|
+
or resume pagination on another replica. It does not promise a provider-token
|
|
671
|
+
lifetime, snapshot isolation across concurrent mutations, or provider,
|
|
672
|
+
network, credential, or authorization availability. Provider invalidation is
|
|
673
|
+
an ordinary list-operation failure.
|
|
674
|
+
|
|
486
675
|
Downloads are streaming by default:
|
|
487
676
|
|
|
488
677
|
```ts
|
|
@@ -505,11 +694,54 @@ Node `Readable` uploads are accepted and converted to Web streams without
|
|
|
505
694
|
buffering. Provider capability gaps fail closed with `StorageError` rather than
|
|
506
695
|
silently discarding a range, metadata, or cache-control request.
|
|
507
696
|
|
|
508
|
-
###
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
697
|
+
### Exact provider conditions and staged-object promotion
|
|
698
|
+
|
|
699
|
+
Capabilities distinguish conditional create, replace, delete, read, source
|
|
700
|
+
copy, destination copy, atomic source-and-destination promotion, and multipart
|
|
701
|
+
completion. They also declare the complete physical-key byte budget. Callers
|
|
702
|
+
must check the exact primitive they need; a missing field is unsupported and is
|
|
703
|
+
never widened from another operation.
|
|
704
|
+
|
|
705
|
+
The physical-key budget applies to the exact key sent to the adapter. It
|
|
706
|
+
therefore counts leading slashes for unprefixed drivers, the separator added to
|
|
707
|
+
a configured driver prefix, and provider prefixes derived for `list` or
|
|
708
|
+
`search`. Over-budget object keys and explicit list/search prefixes fail before
|
|
709
|
+
provider dispatch rather than being normalized into a shorter key. A glob's
|
|
710
|
+
provider prefix is the exact prefix inferred by files-sdk itself, and that
|
|
711
|
+
derived list operation passes through the same final guard. A non-positive
|
|
712
|
+
`maxResults` performs no provider walk and therefore dispatches no prefix.
|
|
713
|
+
|
|
714
|
+
Adapters and plugins execute as trusted in-process code; this package does not
|
|
715
|
+
attempt to sandbox a plugin that performs its own network or filesystem I/O.
|
|
716
|
+
For supported plugin pipelines that forward operations through `next`, the
|
|
717
|
+
physical-key guard is the innermost wrapper, after caller plugins and before
|
|
718
|
+
the adapter call. A plugin therefore cannot widen an upload key or list/search
|
|
719
|
+
prefix past the declared byte ceiling while still using the normal dispatch
|
|
720
|
+
pipeline.
|
|
721
|
+
|
|
722
|
+
Storage-facing ETags have one canonical representation: a bare, case-sensitive
|
|
723
|
+
opaque token with no HTTP quotes. Canonical values contain 1–1024 visible
|
|
724
|
+
ASCII bytes and exclude commas, backslashes, whitespace, control characters,
|
|
725
|
+
`DEL`, non-ASCII text, the `*` wildcard, and case-insensitive `W/` weak-tag
|
|
726
|
+
prefixes. Treat the value as opaque: preserve the exact string returned by
|
|
727
|
+
`head`, reads, or writes and pass it back unchanged to a conditional operation.
|
|
728
|
+
Do not add or remove quotes in application code. S3-compatible drivers remove
|
|
729
|
+
exactly one valid provider-owned quote pair on ingress and add exactly one pair
|
|
730
|
+
when serializing the HTTP header.
|
|
731
|
+
|
|
732
|
+
This tightens the precondition boundary. Quoted or otherwise non-canonical
|
|
733
|
+
ETags that older versions happened to accept now fail with `INVALID_ARGUMENT`,
|
|
734
|
+
and an unsafe provider result fails with a sanitized `PROVIDER` error instead
|
|
735
|
+
of being exposed as a usable validator. Applications that persisted quoted
|
|
736
|
+
ETags must refresh them with `head` rather than trimming them heuristically.
|
|
737
|
+
Strict normalization prevents a caller-controlled wildcard, entity-tag list,
|
|
738
|
+
weak validator, or malformed header value from widening an operation that
|
|
739
|
+
promises one exact strong match.
|
|
740
|
+
|
|
741
|
+
The native AWS S3 profile advertises ETag- and version-conditioned server-side
|
|
742
|
+
copy. This lets an application validate a staged object and copy that exact
|
|
743
|
+
source to its final key instead of re-reading whichever bytes occupy the
|
|
744
|
+
staging key later:
|
|
513
745
|
|
|
514
746
|
```ts
|
|
515
747
|
import { StorageError, StorageErrorCode } from '@nestm/storage';
|
|
@@ -532,9 +764,58 @@ await media.delete(stagingKey);
|
|
|
532
764
|
```
|
|
533
765
|
|
|
534
766
|
`sourceVersion` can select an immutable S3 version and may be combined with
|
|
535
|
-
`sourceEtag`. A
|
|
536
|
-
|
|
537
|
-
|
|
767
|
+
`sourceEtag`. A destination condition can independently require create-only or
|
|
768
|
+
replacement of an exact ETag. Combining source and destination predicates also
|
|
769
|
+
requires `capabilities.conditionalCopyDestination.atomicWithSource`; otherwise
|
|
770
|
+
the request fails with `NOT_SUPPORTED`. A promotion must contain at least one
|
|
771
|
+
source or destination predicate.
|
|
772
|
+
|
|
773
|
+
Cloudflare R2 has a separate stable profile: create, replace, ETag-conditioned
|
|
774
|
+
read, and ETag-conditioned source copy are enabled, while conditional delete,
|
|
775
|
+
destination copy, atomic promotion, version predicates, and conditional
|
|
776
|
+
multipart completion remain absent. R2 proves content-type binding for
|
|
777
|
+
presigned PUT requests but not POST-form size ranges, so its signed-upload
|
|
778
|
+
policy is `{ contentType: true, sizeRange: false }` and the gateway refuses to
|
|
779
|
+
mint its POST upload form. Cloudflare documents that presigned `POST` form
|
|
780
|
+
uploads are not supported in its
|
|
781
|
+
[presigned URL contract](https://developers.cloudflare.com/r2/api/s3/presigned-urls/).
|
|
782
|
+
Direct R2 signed-upload calls that request a size bound likewise fail with
|
|
783
|
+
`NOT_SUPPORTED` before signing.
|
|
784
|
+
Custom S3-compatible endpoints start with no conditional operations and the
|
|
785
|
+
entire driver is forced read-only until an explicit conformance-verified
|
|
786
|
+
`S3ProviderProfile` is supplied. Omitting `signedUploadPolicy` while defining a
|
|
787
|
+
custom profile normalizes both policy claims to `false`; providers may opt in
|
|
788
|
+
only to constraints their conformance evidence proves.
|
|
789
|
+
|
|
790
|
+
Successful S3 signed uploads enforce every requested constraint. A request
|
|
791
|
+
with `contentType` and no `maxSize` uses a presigned PUT whose signature
|
|
792
|
+
includes the `content-type` header. A request with `maxSize` uses a POST policy
|
|
793
|
+
with `content-length-range` and, when present, an exact `Content-Type`
|
|
794
|
+
condition. S3 cannot express a lower-only `minSize` through this contract, so
|
|
795
|
+
that shape fails with `NOT_SUPPORTED`; an unclaimed profile constraint also
|
|
796
|
+
fails before credentials are resolved or a URL is minted. Literal physical
|
|
797
|
+
keys ending in AWS's `${filename}` POST template are rejected for bounded
|
|
798
|
+
uploads because the SDK otherwise widens the exact key condition to a prefix.
|
|
799
|
+
|
|
800
|
+
`withS3Capabilities()` decorates a raw S3 adapter in place and may be applied
|
|
801
|
+
only once. Construct a fresh raw adapter when selecting a different profile;
|
|
802
|
+
reapplying the helper is rejected so a previous broader profile cannot survive
|
|
803
|
+
a later narrower declaration. The selected profile is bound to the exact
|
|
804
|
+
reserved capability and operation members installed by that decoration;
|
|
805
|
+
same-client aliases may change display metadata but cannot add or replace those
|
|
806
|
+
members to widen the profile.
|
|
807
|
+
|
|
808
|
+
An explicit profile applied to a native AWS SDK endpoint may only narrow the
|
|
809
|
+
immutable `AWS_S3_PROVIDER_PROFILE`. It cannot raise the complete-key budget
|
|
810
|
+
above 1,024 bytes or claim an operation/policy bit absent from the built-in
|
|
811
|
+
profile. This containment follows actual SDK endpoint provenance even when an
|
|
812
|
+
adapter alias changes its display name.
|
|
813
|
+
|
|
814
|
+
The package-owned `s3()` factory also retains whether `publicBaseUrl` was
|
|
815
|
+
configured even when the second `withS3Capabilities()` options object is
|
|
816
|
+
omitted. In that case `signedDownloadPolicy.expiresIn` is false. Foreign S3
|
|
817
|
+
adapters whose construction metadata is unavailable receive the same
|
|
818
|
+
conservative false value instead of claiming an enforceable TTL.
|
|
538
819
|
|
|
539
820
|
### Resumable uploads
|
|
540
821
|
|
|
@@ -676,8 +957,11 @@ by `maxSignedUploadBytes`, and require an exact lowercase MIME type from
|
|
|
676
957
|
`attachment` or `inline` response disposition; arbitrary response-header text
|
|
677
958
|
and filenames are rejected. A driver must also advertise
|
|
678
959
|
`signedUploadPolicy.contentType` and `signedUploadPolicy.sizeRange`; otherwise
|
|
679
|
-
the gateway refuses to mint the URL.
|
|
680
|
-
|
|
960
|
+
the gateway refuses to mint the URL. Native AWS advertises both and uses S3
|
|
961
|
+
POST policy conditions. R2 advertises content-type enforcement but not a POST
|
|
962
|
+
size range, while omitted custom declarations normalize both claims to false;
|
|
963
|
+
the gateway therefore fails closed for those profiles. Signed downloads
|
|
964
|
+
similarly require
|
|
681
965
|
`signedDownloadPolicy.expiresIn`. The S3 factory advertises it only when no
|
|
682
966
|
permanent `publicBaseUrl` was configured, preventing a configured TTL from
|
|
683
967
|
silently returning a non-expiring public link.
|
package/SECURITY.md
CHANGED
|
@@ -56,11 +56,40 @@ untrusted agent code. A tenant id, run id, prefix, provider cursor, snapshot id,
|
|
|
56
56
|
or fork id supplied by a model is not a safe mount coordinate.
|
|
57
57
|
|
|
58
58
|
The workspace accepts only canonical mount-relative POSIX paths and rechecks
|
|
59
|
-
every key returned by a driver before unscoping it.
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
59
|
+
every key returned by a driver before unscoping it. Pagination cursors are bound
|
|
60
|
+
to the store, normalized physical prefix, stable mount identity, trusted
|
|
61
|
+
tenant/workspace scope, operation, complete effective limits, normalized query,
|
|
62
|
+
and expiry. Provider continuations and prefixes stay inside the encrypted or
|
|
63
|
+
server-side payload. Both layers are non-consuming: a durable opaque token
|
|
64
|
+
store must read rather than consume a record, and an embedded provider cursor
|
|
65
|
+
must remain replayable through a fresh compatible driver against the same
|
|
66
|
+
backend namespace while that provider cursor remains valid and available.
|
|
67
|
+
Provider cursors cannot depend on process-local state; an adapter for a
|
|
68
|
+
consuming or instance-bound backend token must materialize a stable continuation
|
|
69
|
+
before exposing paginated `StorageDriver.list` results.
|
|
70
|
+
|
|
71
|
+
Under unchanged provider-visible state, cursors are reusable while their
|
|
72
|
+
provider continuation remains valid and available. The authenticated expiry is
|
|
73
|
+
an authorization ceiling, not a guarantee of provider-token lifetime, snapshot
|
|
74
|
+
isolation, provider/network availability, or valid credentials. Provider
|
|
75
|
+
invalidation remains an operational failure, and concurrent mutations remain
|
|
76
|
+
subject to provider ordering, duplicate, and omission semantics.
|
|
77
|
+
|
|
78
|
+
Production pagination must configure either the built-in AES-256-GCM codec with
|
|
79
|
+
a dedicated shared 32-byte key ring or an authenticated shared durable token
|
|
80
|
+
store. Do not reuse an authentication, session, storage-provider, or encryption
|
|
81
|
+
key from another purpose. All replicas must use the same stable identities and
|
|
82
|
+
key ring, compatible driver configuration, and logical backend namespace. New
|
|
83
|
+
cursors use the active key id; retain prior keys for at least the maximum cursor
|
|
84
|
+
TTL during rotation. Dropping a key, changing a binding field, or restarting
|
|
85
|
+
with a different ephemeral key intentionally invalidates outstanding cursors.
|
|
86
|
+
Tokens and decoded payloads are bounded, and malformed, altered, expired,
|
|
87
|
+
cross-query, cross-operation, cross-workspace, and cross-store cursors fail
|
|
88
|
+
closed.
|
|
89
|
+
|
|
90
|
+
Permissions, byte limits, result limits, and conditional mutation preconditions
|
|
91
|
+
are enforced inside the capability; tool omission and user approval are
|
|
92
|
+
additional workflow controls, not the authorization boundary.
|
|
64
93
|
|
|
65
94
|
Conditional mutations can still have an ambiguous outcome when a remote
|
|
66
95
|
provider commits and then loses or violates its response, or when a configured
|