streamby-core 0.39.0 β†’ 0.41.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (63) hide show
  1. package/CHANGELOG.md +84 -0
  2. package/LICENSE +661 -661
  3. package/README.md +785 -573
  4. package/dist/adapters/database/connectionManager.js +182 -124
  5. package/dist/adapters/database/sql.js +1 -1
  6. package/dist/index.d.ts +3 -1
  7. package/dist/index.js +9 -1
  8. package/dist/mcp/audit.d.ts +5 -2
  9. package/dist/mcp/audit.js +8 -0
  10. package/dist/mcp/server.js +3 -3
  11. package/dist/mcp/tools.js +41 -0
  12. package/dist/middleware/createRouter.js +14 -1
  13. package/dist/middleware/routes/ai.js +3 -3
  14. package/dist/middleware/routes/deliverable.d.ts +4 -2
  15. package/dist/middleware/routes/deliverable.js +16 -7
  16. package/dist/middleware/routes/mcp.js +11 -4
  17. package/dist/middleware/routes/media.js +7 -6
  18. package/dist/middleware/routes/{apiConnection.d.ts β†’ oauthResource.d.ts} +1 -1
  19. package/dist/middleware/routes/oauthResource.js +18 -0
  20. package/dist/middleware/routes/project.js +5 -1
  21. package/dist/middleware/routes/renderFarm.js +4 -77
  22. package/dist/middleware/routes/storage.js +13 -27
  23. package/dist/middleware/routes/worker.d.ts +7 -0
  24. package/dist/middleware/routes/worker.js +125 -0
  25. package/dist/middleware/routes/workerCallback.d.ts +5 -0
  26. package/dist/middleware/routes/workerCallback.js +31 -0
  27. package/dist/models/Model.js +9 -9
  28. package/dist/services/aiProcessor.js +11 -11
  29. package/dist/services/apiTokenPermissions.js +6 -5
  30. package/dist/services/auth.d.ts +2 -0
  31. package/dist/services/auth.js +36 -10
  32. package/dist/services/executor.js +10 -10
  33. package/dist/services/jobQueue.d.ts +6 -4
  34. package/dist/services/jobQueue.js +28 -16
  35. package/dist/services/jobStore.d.ts +27 -0
  36. package/dist/services/jobStore.js +177 -0
  37. package/dist/services/mediaProcessor.js +16 -16
  38. package/dist/services/oauthTokens.d.ts +60 -0
  39. package/dist/services/oauthTokens.js +164 -0
  40. package/dist/services/vfxProcessor.js +12 -12
  41. package/dist/services/workerJobs.d.ts +73 -0
  42. package/dist/services/workerJobs.js +459 -0
  43. package/dist/services/workers.d.ts +78 -0
  44. package/dist/services/workers.js +179 -0
  45. package/dist/types/index.d.ts +64 -2
  46. package/dist/utils/assetCategories.d.ts +4 -0
  47. package/dist/utils/assetCategories.js +21 -0
  48. package/dist/utils/builtinAccess.d.ts +1 -1
  49. package/dist/utils/builtinAccess.js +4 -3
  50. package/dist/utils/privateAddress.d.ts +10 -0
  51. package/dist/utils/privateAddress.js +85 -0
  52. package/package.json +1 -1
  53. package/dist/middleware/routes/apiConnection.js +0 -100
  54. package/dist/middleware/routes/credential.d.ts +0 -3
  55. package/dist/middleware/routes/credential.js +0 -80
  56. package/dist/services/apiConnection.d.ts +0 -18
  57. package/dist/services/apiConnection.js +0 -65
  58. package/dist/services/credential.d.ts +0 -8
  59. package/dist/services/credential.js +0 -70
  60. package/dist/services/pipeline.d.ts +0 -2
  61. package/dist/services/pipeline.js +0 -292
  62. package/dist/utils/crypto.d.ts +0 -3
  63. package/dist/utils/crypto.js +0 -32
package/README.md CHANGED
@@ -1,573 +1,785 @@
1
- # streamby-core
2
-
3
- [![License: AGPL v3](https://img.shields.io/badge/License-AGPLv3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
4
-
5
- Middleware framework for building storage-agnostic media APIs.
6
- Part of the [StreamBy](https://streamby.nhexa.cl) ecosystem by TerminalCore Labs.
7
-
8
- ---
9
-
10
- ## πŸš€ What is it?
11
-
12
- `streamby-core` is a plug-and-play middleware for Express (or compatible frameworks) that enables file uploads, listings and **multi-project access** over services like:
13
-
14
- - βœ… AWS S3
15
- - βœ… Google Cloud Storage (soon)
16
- - βœ… Cloudflare R2 (soon)
17
- - βœ… Local/Personal servers (soon)
18
-
19
- It's designed to be installed inside your existing API as a library β€” **no need to run a separate backend**.
20
-
21
- ---
22
-
23
- ## πŸ“¦ Installation
24
-
25
- ```bash
26
- npm install streamby-core ws
27
- npm install -D @types/ws
28
- ```
29
-
30
- ---
31
-
32
- ## 🧱 Basic usage
33
-
34
- ```ts
35
- import http from 'http';
36
- import express from 'express';
37
- import { WebSocketServer } from 'ws';
38
- import { createStreamByRouter } from 'streamby-core';
39
-
40
- const app = express();
41
- const server = http.createServer(app);
42
- const wss = new WebSocketServer({ server, path: '/streamby/ws' });
43
-
44
- app.use('/streamby', express.json(), createStreamByRouter({
45
- authProvider: async (req) => {
46
- // validate token/session and return user info
47
- return { userId: 'user-id', username: 'username', role: 'admin' };
48
- },
49
- databases: [
50
- { id: 'mongo', type: 'nosql', connectionString: process.env.MONGO_URI! },
51
- { id: 'postgres', type: 'sql', connectionString: process.env.POSTGRES_URI!, main: true },
52
- ],
53
- storageProviders: [
54
- {
55
- type: 's3',
56
- config: {
57
- bucket: process.env.AWS_BUCKET!,
58
- region: process.env.AWS_REGION!,
59
- accessKeyId: process.env.AWS_ACCESS_KEY!,
60
- secretAccessKey: process.env.AWS_SECRET_KEY!,
61
- },
62
- },
63
- ],
64
- encrypt: process.env.STREAMBY_ENCRYPTION_KEY, // 64 hex characters; see "Encryption and key rotation"
65
- websocket: { server: wss },
66
- }));
67
-
68
- server.listen(3000);
69
- ```
70
-
71
- > **Important:** use `http.createServer(app)` and pass `server.listen()` instead of `app.listen()`.
72
- > This is required so the `WebSocketServer` shares the same port as the HTTP server.
73
-
74
- ---
75
-
76
- ## πŸ”‘ Encryption and key rotation
77
-
78
- Credentials stored by StreamBy (connection secrets, export credentials, user integrations,
79
- distribution and render farm credentials) are encrypted with the key you pass as `encrypt`:
80
- 32 bytes, written as 64 hexadecimal characters. Each stored value records the id of the key
81
- that encrypted it (a fingerprint, never the key itself), so you can rotate the key:
82
-
83
- 1. Generate a new key, for example with `openssl rand -hex 32`.
84
- 2. Pass the new key as `encrypt` and the previous one in `encryptPreviousKeys`.
85
- 3. Restart. Values encrypted with a previous key still decrypt, and at startup StreamBy
86
- re-encrypts all of them with the new key.
87
- 4. Once that's done (the startup log says how many records it re-encrypted, and none are left
88
- on the old key), you can remove the old key from `encryptPreviousKeys`.
89
-
90
- ```ts
91
- createStreamByRouter({
92
- // ...
93
- encrypt: process.env.STREAMBY_ENCRYPTION_KEY, // the current key
94
- encryptPreviousKeys: [process.env.STREAMBY_PREVIOUS_KEY!], // only while rotating
95
- });
96
- ```
97
-
98
- > Upgrading from a version before 0.34.0: deploy with your current key first, and rotate only
99
- > after that deploy (see the [changelog](CHANGELOG.md)).
100
-
101
- ---
102
-
103
- ## 🎫 API tokens
104
-
105
- Agents and services work with StreamBy through API tokens, without a browser session. There
106
- are two kinds:
107
-
108
- - **A user's token** acts as its user. The user manages their own (`GET`, `POST` and
109
- `DELETE /streamby/user/api-tokens`). It reaches a list of the user's projects, or `'*'` for
110
- every project they're a member of, including ones they join later. It never does more than
111
- its user can: their role in each project still applies.
112
- - **A project's token** acts on its project alone, as the project's, not as whoever made it:
113
- it keeps working after they leave the project, for permanent integrations (CI, services, the
114
- project's agents). Only the project's admins list, make and revoke them (`GET`, `POST` and
115
- `DELETE /streamby/projects/:id/api-tokens`), and each one records who made it. When they
116
- leave, the project's admins are notified that it still works. Deleting the project ends
117
- its tokens.
118
-
119
- Tokens are managed only with a session: a token can't manage tokens.
120
-
121
- Each token has a name, **its permissions**, and **an expiry** (7, 30, 90 or 365 days, or
122
- none). It's `sbt_` followed by 32 random bytes, shown once, when it's made. Only its SHA-256
123
- hash is stored, with a short prefix to tell it apart.
124
-
125
- **Permissions**
126
-
127
- | Resource | Permissions | Covers |
128
- |---|---|---|
129
- | Project | `project:read`, `project:update`, `project:delete` | The project, its settings and its connections (API, database, storage, distribution, render farm), integrations and allowed origins. |
130
- | Members | `members:read`, `members:manage` | Listing members; inviting, changing and removing them, and searching users. |
131
- | Assets | `assets:read`, `assets:write`, `assets:delete` | Files and folders, upload URLs, metadata, versions, reviews and annotations, QC reports. |
132
- | Production | `production:read`, `production:write`, `production:delete` | Sequences, shots and tasks. |
133
- | Workflow | `workflow:read`, `workflow:write` | The workflow and its pipelines. |
134
- | Exports | `exports:read`, `exports:write` | The project's exports. |
135
- | Rights | `rights:read`, `rights:write` | Assets' rights. |
136
- | Jobs | `jobs:read`, `jobs:run` | Following jobs; starting them (transcoding, rendering, AI, publishing a deliverable). |
137
- | Account | `projects:create` | Creating projects. A user's token only; the token reaches what it creates. |
138
-
139
- `PROJECT_PERMISSIONS` and `ACCOUNT_PERMISSIONS` are exported. A route no permission covers
140
- (the user's notifications, integrations and tokens, Explore, accepting an invitation) isn't
141
- reachable with a token.
142
-
143
- **Resolving a token**
144
-
145
- Requests send it as `Authorization: Bearer sbt_...`. Your `authProvider` resolves it with
146
- `resolveApiToken`, and returns its `userId` with the token's scope in `apiToken`. A project's
147
- token isn't a user: its `userId` is the token's own (on its project), and it goes by its name.
148
-
149
- ```ts
150
- import { createStreamByRouter, resolveApiToken, API_TOKEN_PREFIX } from 'streamby-core';
151
-
152
- createStreamByRouter({
153
- // ...
154
- authProvider: async (req) => {
155
- const bearer = req.headers.authorization?.replace(/^Bearer /, '');
156
- if (bearer?.startsWith(API_TOKEN_PREFIX)) {
157
- const resolved = await resolveApiToken(bearer); // null once revoked or expired
158
- if (!resolved) throw new Error('Invalid API token');
159
- if (resolved.scope.kind === 'project') {
160
- return { userId: resolved.userId, username: resolved.name, role: 'viewer', apiToken: resolved.scope };
161
- }
162
- const user = await findUser(resolved.userId); // your own user lookup
163
- return { userId: user.id, username: user.username, role: user.role, apiToken: resolved.scope };
164
- }
165
- // ...your session-based authentication
166
- },
167
- });
168
- ```
169
-
170
- A request made with a token goes through only when its route's permission is among the
171
- token's (otherwise 403), and a project outside the token's projects answers 404; the project
172
- list shows only the token's projects.
173
-
174
- ---
175
-
176
- ## πŸ€– MCP server
177
-
178
- StreamBy serves an [MCP](https://modelcontextprotocol.io) server at `POST /streamby/mcp`
179
- (Streamable HTTP, stateless). Agents such as Claude Code use it to read and work on StreamBy
180
- projects with an API token (see above). It takes only an API token; a browser session gets 401.
181
-
182
- Each tool calls StreamBy's own REST API with the caller's token. So it reaches exactly what the
183
- token allows (and, for a user's token, what its user's role in each project allows), and a
184
- change is saved just like one made from the UI, which shows it when it loads the project.
185
-
186
- **Tools**
187
-
188
- A token sees only the tools whose permissions it has.
189
-
190
- | Tools | Permissions |
191
- |---|---|
192
- | `list_projects` | any token |
193
- | `get_project` | `project:read` |
194
- | `create_project` | `projects:create` |
195
- | `delete_project` | `project:delete` |
196
- | `list_assets` (by category, optionally in one storage connection, and filtered by name), `get_asset_metadata` | `assets:read` |
197
- | `update_asset_metadata` | `assets:write` |
198
- | `delete_asset` | `assets:delete` |
199
- | `get_asset_rights` / `set_asset_rights` | `rights:read` / `rights:write` |
200
- | `get_workflow`, `list_pipelines`, `get_pipeline` | `workflow:read` |
201
- | `create_pipeline`, `update_pipeline` | `workflow:write` |
202
- | `list_exports` | `project:read` and `exports:read` |
203
- | `get_export` | `exports:read` |
204
- | `get_job` | `jobs:read` |
205
- | `list_sequences`, `list_shots`, `list_tasks` | `production:read` |
206
- | `update_sequence`, `update_shot`, `update_task` | `production:write` |
207
- | `create_sequence`, `create_shot`, `create_task` | `production:write` and `workflow:write` |
208
- | `delete_sequence`, `delete_shot`, `delete_task` | `production:delete` |
209
- | `list_asset_versions`, `get_asset_download_url`, `list_deliverables` | `assets:read` |
210
- | `request_asset_upload`, `mark_deliverable`, `remove_deliverable` | `assets:write` |
211
-
212
- The tools take the path a user does in the UI, and each one's description says where its
213
- result shows there. Production data lives under a production node of the project's workflow:
214
- if the workflow has none yet, `create_sequence`, `create_shot` and `create_task` first add a
215
- Shot node to it, as dropping one on the canvas does (so they need `workflow:write`), and make
216
- nothing if it can't be added.
217
-
218
- The tools that delete are marked as destructive (`destructiveHint`), so a client can ask before
219
- running them.
220
-
221
- **Files**
222
-
223
- A file never goes through the conversation. `request_asset_upload` makes a new asset (or a new
224
- version of one) and gives a signed URL to `PUT` its bytes to; `get_asset_download_url` gives a
225
- signed URL to `GET` an asset's file, its latest version or any earlier one. The tool that has
226
- the file uploads or downloads it directly, for example Blender's or Godot's own MCP server,
227
- which can run code. Each URL is for that one file, and given to an API token it works only
228
- briefly: 15 minutes to upload, 5 to download (`mcp.signedUrlTtl`).
229
-
230
- **Versions.** Replacing an asset doesn't change its URL: the new version is uploaded to the
231
- asset's own key, so a site that links to the asset shows the new one with no deploy. Before
232
- that, the current version is copied to a key of its own (`<key>.v<n>.<ext>`) and recorded, so
233
- every earlier version can still be downloaded by its id; if it can't be copied, nothing is
234
- replaced. A storage adapter keeps versions only if it can copy (`copyFile`; the S3 one does);
235
- otherwise asking for a new version answers 501. Deleting the asset deletes every key of it.
236
-
237
- Since the URL stays the same, a browser or a CDN may keep serving the earlier version for a
238
- while. StreamBy doesn't set `Cache-Control` on uploads (a signed URL that did would require
239
- every client to send the same header): set it where the files are served, for example a short
240
- `max-age` or `no-cache` on the CDN in front of the bucket, or invalidate the path when an asset
241
- is replaced.
242
-
243
- A **deliverable** is an asset, or one of its versions, delivered for a
244
- production shot or for an export; a shot or an export can have several, and the shot's panel
245
- in the UI shows them.
246
-
247
- For example, with Claude connected to StreamBy and to Blender's MCP server, asking *"export the
248
- scene as glTF and deliver it for shot SH010 of Portfolio"* goes:
249
-
250
- 1. `list_projects`, `list_sequences`, `list_shots`: the project and the shot's id.
251
- 2. `request_asset_upload` with `category: '3d-models'`, `fileName: 'SH010_layout.glb'` and
252
- `contentType: 'model/gltf-binary'`: the new asset's id, and where to upload it.
253
- 3. In Blender, code that exports the scene and uploads it:
254
-
255
- ```python
256
- import bpy, urllib.request
257
- path = bpy.app.tempdir + "SH010_layout.glb"
258
- bpy.ops.export_scene.gltf(filepath=path, export_format="GLB")
259
- with open(path, "rb") as f:
260
- req = urllib.request.Request(UPLOAD_URL, data=f.read(), method="PUT",
261
- headers={"Content-Type": "model/gltf-binary"})
262
- urllib.request.urlopen(req)
263
- ```
264
-
265
- 4. `mark_deliverable` with the asset's id and `shotId`: it shows in the shot's panel.
266
-
267
- Bringing an asset into Blender or Godot is the other way around: `get_asset_download_url`, and
268
- code there that downloads the URL and imports the file (in Blender,
269
- `bpy.ops.import_scene.gltf(filepath=...)`). A new version of the same asset is
270
- `request_asset_upload` with its `assetId`.
271
-
272
- **Limits and audit**
273
-
274
- - Each token can make at most 120 requests a minute (a sliding window). Past that, StreamBy
275
- answers 429 with `Retry-After`. Each instance counts its own requests. Change the limit with
276
- `mcp.rateLimit`.
277
- - Every tool call is recorded in the `mcp_audit` table, created at startup in the main database.
278
- Each row has the user (a project's token: the token itself), the token's id, the project, the
279
- tool, whether it went through, the status, the error and the duration. A row never includes
280
- the arguments or the token.
281
-
282
- ```ts
283
- createStreamByRouter({
284
- // ...
285
- mcp: {
286
- rateLimit: { requests: 120, windowMs: 60_000 }, // the default
287
- signedUrlTtl: { upload: 900, download: 300 }, // the default, in seconds
288
- // apiBaseUrl: 'http://127.0.0.1:3000/streamby', // where tools reach the REST API;
289
- // by default this same process, at the request's local port and mount path
290
- },
291
- });
292
- ```
293
-
294
- ### Connecting from Claude Code
295
-
296
- Make an API token in StreamBy, with only the permissions the agent needs: your own
297
- (Settings β†’ Security β†’ Personal API tokens), or a project's (in the project, Settings β†’ API
298
- Tokens). Keep it in an environment variable, never in a file:
299
-
300
- ```bash
301
- export STREAMBY_TOKEN=sbt_...
302
- ```
303
-
304
- Add the server to your project's `.mcp.json`. Claude Code expands `${STREAMBY_TOKEN}` from the
305
- environment:
306
-
307
- ```json
308
- {
309
- "mcpServers": {
310
- "streamby": {
311
- "type": "http",
312
- "url": "https://your-host/streamby/mcp",
313
- "headers": { "Authorization": "Bearer ${STREAMBY_TOKEN}" }
314
- }
315
- }
316
- }
317
- ```
318
-
319
- ### Connecting from Claude Desktop
320
-
321
- Claude Desktop runs local (stdio) servers from `claude_desktop_config.json`. Reach StreamBy
322
- through [`mcp-remote`](https://www.npmjs.com/package/mcp-remote), passing the token in its
323
- `env`. This file holds it in plain text, so use a token with only the permissions you need,
324
- and an expiry:
325
-
326
- ```json
327
- {
328
- "mcpServers": {
329
- "streamby": {
330
- "command": "npx",
331
- "args": ["mcp-remote", "https://your-host/streamby/mcp", "--header", "Authorization:${STREAMBY_AUTH}"],
332
- "env": { "STREAMBY_AUTH": "Bearer sbt_..." }
333
- }
334
- }
335
- }
336
- ```
337
-
338
- The claude.ai connector needs OAuth, which isn't available yet: it's for API tokens only for now.
339
-
340
- ---
341
-
342
- ## πŸ‘₯ Project roles
343
-
344
- What a user can do in a project depends on their role in that project, as an active member
345
- (an invitation counts once it's accepted). It never depends on the platform role your
346
- `authProvider` returns.
347
-
348
- | Role | Can |
349
- |---|---|
350
- | `viewer` | See the project and its content, and archive the project in their own list. |
351
- | `editor` | Everything a viewer can, plus change the project: its details, connections, exports, files and folders, workflow, pipelines, production, reviews, rights, and start jobs. |
352
- | `admin` | Everything an editor can, plus manage the project's members and delete the project. |
353
-
354
- A project's public exports (`GET /streamby/:projectId/export/:exportName`) are read without a
355
- session. They only run nodes that read data: never a node that uses an account's credential,
356
- nor one that starts a job.
357
-
358
- ---
359
-
360
- ## πŸ—„οΈ Database Schema Setup
361
-
362
- `streamby-core` requires a pre-configured database schema for its internal operations. It does not manage database connections directly; instead, it expects the host application to provide initialized connection instances. This ensures full control and flexibility for the host environment.
363
-
364
- The library will only interact with the specified schema and will not touch other schemas in your database. The `reset` option is opt-in and will only drop and recreate the specified `streamby` schema if explicitly set to `true`.
365
-
366
- ### PostgreSQL Example
367
-
368
- ```ts
369
- import { Pool } from "pg";
370
- import { setupStreambyPg } from "streamby-core/pg/setup"; // Adjust path as per your package structure
371
-
372
- const pool = new Pool({ connectionString: process.env.DATABASE_URL });
373
-
374
- // Call setupStreambyPg to ensure the schema and tables are created
375
- // reset: false (default) ensures idempotency and no data loss
376
- await setupStreambyPg({ pool, schema: "streamby", reset: false });
377
- ```
378
-
379
- ### MongoDB Example
380
-
381
- ```ts
382
- import { MongoClient } from "mongodb";
383
- import { setupStreambyMongo } from "streamby-core/mongo/setup"; // Adjust path as per your package structure
384
-
385
- const client = new MongoClient(process.env.MONGO_URL!);
386
-
387
- // Connect the MongoDB client
388
- await client.connect();
389
-
390
- // Call setupStreambyMongo to ensure collections and indexes are created
391
- await setupStreambyMongo({ client, dbName: "streamby" });
392
- ```
393
-
394
- ```ts
395
- import express from 'express';
396
- import { createStreamByRouter } from 'streamby-core';
397
-
398
- const app = express();
399
-
400
- app.use('/streamby', createStreamByRouter({
401
- storageProvider: {
402
- type: 's3',
403
- config: {
404
- bucket: 'your-bucket-name',
405
- region: 'your-region',
406
- accessKeyId: 'YOUR_AWS_KEY',
407
- secretAccessKey: 'YOUR_AWS_SECRET'
408
- }
409
- },
410
- authProvider: async (req) => {
411
- // extract token/cookie and validate
412
- return {
413
- userId: 'demo',
414
- projects: ['demo-project'],
415
- role: 'admin',
416
- };
417
- },
418
- projectProvider: async (projectId) => {
419
- // fetch project from your DB
420
- return {
421
- id: projectId,
422
- name: 'Demo Project',
423
- description: 'Demo for StreamBy integration',
424
- rootFolders: [],
425
- settings: { allowUpload: true }
426
- };
427
- }
428
- }));
429
-
430
- app.listen(3000);
431
- ```
432
-
433
- ---
434
-
435
- ## πŸ§ͺ Testing
436
-
437
- ```bash
438
- npm run test
439
- ```
440
-
441
- Unit tests use [Vitest](https://vitest.dev) and [Supertest](https://www.npmjs.com/package/supertest).
442
-
443
- ---
444
-
445
- ## πŸ“ Features
446
-
447
- - πŸ“‚ Multi-project access control
448
- - πŸ” Project-aware file uploads & listings
449
- - 🧹 Modular adapters per storage provider
450
- - 🧰 Built-in testability with mock auth & storage
451
- - ✨ Simple to extend for custom business logic
452
-
453
- ---
454
-
455
- ## πŸ—ΊοΈ API Endpoints
456
-
457
- Here's a summary of the key API endpoints provided by `streamby-core`:
458
-
459
- ### `GET /streamby/projects`
460
-
461
- Lists all projects accessible by the authenticated user.
462
-
463
- **Query Parameters:**
464
- - `archived`: (Optional) Filter projects by their archived status.
465
- - `true`: Returns only archived projects.
466
- - `false`: Returns only unarchived projects.
467
- - (Omitted): Returns all projects (both archived and unarchived).
468
-
469
- **Example Response:**
470
- ```json
471
- {
472
- "projects": [
473
- {
474
- "id": "project1_id",
475
- "dbType": "nosql",
476
- "name": "My NoSQL Project",
477
- "image": "url_to_image",
478
- "archived": false
479
- },
480
- {
481
- "id": "project2_id",
482
- "dbType": "sql",
483
- "name": "My SQL Project",
484
- "image": "url_to_image",
485
- "archived": true
486
- }
487
- ]
488
- }
489
- ```
490
-
491
- ### `PATCH /streamby/projects/:id/archive`
492
-
493
- Archives a specific project. After archiving, it returns the complete list of projects for the user, including the updated project.
494
-
495
- **Example Response:**
496
- ```json
497
- {
498
- "success": true,
499
- "projects": [
500
- {
501
- "id": "project1_id",
502
- "dbType": "nosql",
503
- "name": "My NoSQL Project",
504
- "image": "url_to_image",
505
- "archived": true
506
- },
507
- {
508
- "id": "project2_id",
509
- "dbType": "sql",
510
- "name": "My SQL Project",
511
- "image": "url_to_image",
512
- "archived": false
513
- }
514
- ]
515
- }
516
- ```
517
-
518
- ### `PATCH /streamby/projects/:id/unarchive`
519
-
520
- Unarchives a specific project. After unarchiving, it returns the complete list of projects for the user, including the updated project.
521
-
522
- **Example Response:**
523
- ```json
524
- {
525
- "success": true,
526
- "projects": [
527
- {
528
- "id": "project1_id",
529
- "dbType": "nosql",
530
- "name": "My NoSQL Project",
531
- "image": "url_to_image",
532
- "archived": false
533
- },
534
- {
535
- "id": "project2_id",
536
- "dbType": "sql",
537
- "name": "My SQL Project",
538
- "image": "url_to_image",
539
- "archived": false
540
- }
541
- ]
542
- }
543
- ```
544
-
545
- ---
546
-
547
- ## 🚣 Roadmap
548
-
549
- - [ ] Support for Google Cloud Storage
550
- - [ ] CLI tool for uploading files
551
- - [ ] Role-based access control middleware
552
- - [ ] Plugin system for extended routes
553
- - [ ] Streaming support (HLS, audio, etc.)
554
-
555
- ---
556
-
557
- ## πŸ§‘β€πŸ’» Maintained by
558
-
559
- **TerminalCore Labs** – part of [Nhexa Entertainment](https://nhexa.cl)
560
- Developed as open-source infrastructure for creative and media-oriented apps.
561
-
562
- Visit us [here](https://terminalcore.cl)
563
-
564
- ---
565
-
566
- ## πŸ“Ÿ License
567
-
568
- Copyright (C) 2026 TerminalCore Labs
569
-
570
- Licensed under the [GNU Affero General Public License v3.0](./LICENSE) (AGPL-3.0-only).
571
-
572
- > **Note for consumers:** unlike most installable packages, this one is intentionally AGPL rather than MIT. AGPL's copyleft applies to any project that depends on it -- if you install `streamby-core`, your own project must also be licensed under AGPL-3.0 (or a compatible copyleft license). Confirm this is acceptable for your use case before adding this dependency.
573
-
1
+ # streamby-core
2
+
3
+ [![License: AGPL v3](https://img.shields.io/badge/License-AGPLv3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
4
+
5
+ Middleware framework for building storage-agnostic media APIs.
6
+ Part of the [StreamBy](https://streamby.nhexa.cl) ecosystem by TerminalCore Labs.
7
+
8
+ ---
9
+
10
+ ## πŸš€ What is it?
11
+
12
+ `streamby-core` is a plug-and-play middleware for Express (or compatible frameworks) that enables file uploads, listings and **multi-project access** over services like:
13
+
14
+ - βœ… AWS S3
15
+ - βœ… Google Cloud Storage (soon)
16
+ - βœ… Cloudflare R2 (soon)
17
+ - βœ… Local/Personal servers (soon)
18
+
19
+ It's designed to be installed inside your existing API as a library β€” **no need to run a separate backend**.
20
+
21
+ ---
22
+
23
+ ## πŸ“¦ Installation
24
+
25
+ ```bash
26
+ npm install streamby-core ws
27
+ npm install -D @types/ws
28
+ ```
29
+
30
+ ---
31
+
32
+ ## 🧱 Basic usage
33
+
34
+ ```ts
35
+ import http from 'http';
36
+ import express from 'express';
37
+ import { WebSocketServer } from 'ws';
38
+ import { createStreamByRouter } from 'streamby-core';
39
+
40
+ const app = express();
41
+ const server = http.createServer(app);
42
+ const wss = new WebSocketServer({ server, path: '/streamby/ws' });
43
+
44
+ app.use('/streamby', express.json(), createStreamByRouter({
45
+ authProvider: async (req) => {
46
+ // validate token/session and return user info
47
+ return { userId: 'user-id', username: 'username', role: 'admin' };
48
+ },
49
+ databases: [
50
+ { id: 'mongo', type: 'nosql', connectionString: process.env.MONGO_URI! },
51
+ { id: 'postgres', type: 'sql', connectionString: process.env.POSTGRES_URI!, main: true },
52
+ ],
53
+ storageProviders: [
54
+ {
55
+ type: 's3',
56
+ config: {
57
+ bucket: process.env.AWS_BUCKET!,
58
+ region: process.env.AWS_REGION!,
59
+ accessKeyId: process.env.AWS_ACCESS_KEY!,
60
+ secretAccessKey: process.env.AWS_SECRET_KEY!,
61
+ },
62
+ },
63
+ ],
64
+ encrypt: process.env.STREAMBY_ENCRYPTION_KEY, // 64 hex characters; see "Encryption and key rotation"
65
+ websocket: { server: wss },
66
+ }));
67
+
68
+ server.listen(3000);
69
+ ```
70
+
71
+ > **Important:** use `http.createServer(app)` and pass `server.listen()` instead of `app.listen()`.
72
+ > This is required so the `WebSocketServer` shares the same port as the HTTP server.
73
+
74
+ ---
75
+
76
+ ## πŸ”‘ Encryption and key rotation
77
+
78
+ Credentials stored by StreamBy (connection secrets, export credentials, user integrations,
79
+ distribution and render farm credentials) are encrypted with the key you pass as `encrypt`:
80
+ 32 bytes, written as 64 hexadecimal characters. Each stored value records the id of the key
81
+ that encrypted it (a fingerprint, never the key itself), so you can rotate the key:
82
+
83
+ 1. Generate a new key, for example with `openssl rand -hex 32`.
84
+ 2. Pass the new key as `encrypt` and the previous one in `encryptPreviousKeys`.
85
+ 3. Restart. Values encrypted with a previous key still decrypt, and at startup StreamBy
86
+ re-encrypts all of them with the new key.
87
+ 4. Once that's done (the startup log says how many records it re-encrypted, and none are left
88
+ on the old key), you can remove the old key from `encryptPreviousKeys`.
89
+
90
+ ```ts
91
+ createStreamByRouter({
92
+ // ...
93
+ encrypt: process.env.STREAMBY_ENCRYPTION_KEY, // the current key
94
+ encryptPreviousKeys: [process.env.STREAMBY_PREVIOUS_KEY!], // only while rotating
95
+ });
96
+ ```
97
+
98
+ > Upgrading from a version before 0.34.0: deploy with your current key first, and rotate only
99
+ > after that deploy (see the [changelog](CHANGELOG.md)).
100
+
101
+ ---
102
+
103
+ ## 🎫 API tokens
104
+
105
+ Agents and services work with StreamBy through API tokens, without a browser session. There
106
+ are two kinds:
107
+
108
+ - **A user's token** acts as its user. The user manages their own (`GET`, `POST` and
109
+ `DELETE /streamby/user/api-tokens`). It reaches a list of the user's projects, or `'*'` for
110
+ every project they're a member of, including ones they join later. It never does more than
111
+ its user can: their role in each project still applies.
112
+ - **A project's token** acts on its project alone, as the project's, not as whoever made it:
113
+ it keeps working after they leave the project, for permanent integrations (CI, services, the
114
+ project's agents). Only the project's admins list, make and revoke them (`GET`, `POST` and
115
+ `DELETE /streamby/projects/:id/api-tokens`), and each one records who made it. When they
116
+ leave, the project's admins are notified that it still works. Deleting the project ends
117
+ its tokens.
118
+
119
+ Tokens are managed only with a session: a token can't manage tokens.
120
+
121
+ Each token has a name, **its permissions**, and **an expiry** (7, 30, 90 or 365 days, or
122
+ none). It's `sbt_` followed by 32 random bytes, shown once, when it's made. Only its SHA-256
123
+ hash is stored, with a short prefix to tell it apart.
124
+
125
+ **Permissions**
126
+
127
+ | Resource | Permissions | Covers |
128
+ |---|---|---|
129
+ | Project | `project:read`, `project:update`, `project:delete` | The project, its settings and its connections (API, database, storage, distribution, render farm), integrations and allowed origins. |
130
+ | Members | `members:read`, `members:manage` | Listing members; inviting, changing and removing them, and searching users. |
131
+ | Assets | `assets:read`, `assets:write`, `assets:delete` | Files and folders, upload URLs, metadata, versions, reviews and annotations, QC reports. |
132
+ | Production | `production:read`, `production:write`, `production:delete` | Sequences, shots and tasks. |
133
+ | Workflow | `workflow:read`, `workflow:write` | The workflow and its pipelines. |
134
+ | Exports | `exports:read`, `exports:write` | The project's exports. |
135
+ | Rights | `rights:read`, `rights:write` | Assets' rights. |
136
+ | Jobs | `jobs:read`, `jobs:run` | Following jobs; starting them (transcoding, rendering, AI, publishing a deliverable). |
137
+ | Account | `projects:create` | Creating projects. A user's token only; the token reaches what it creates. |
138
+
139
+ `PROJECT_PERMISSIONS` and `ACCOUNT_PERMISSIONS` are exported. A route no permission covers
140
+ (the user's notifications, integrations and tokens, Explore, accepting an invitation) isn't
141
+ reachable with a token.
142
+
143
+ **Resolving a token**
144
+
145
+ Requests send it as `Authorization: Bearer sbt_...`. Your `authProvider` resolves it with
146
+ `resolveApiToken`, and returns its `userId` with the token's scope in `apiToken`. A project's
147
+ token isn't a user: its `userId` is the token's own (on its project), and it goes by its name.
148
+
149
+ ```ts
150
+ import { createStreamByRouter, resolveApiToken, API_TOKEN_PREFIX } from 'streamby-core';
151
+
152
+ createStreamByRouter({
153
+ // ...
154
+ authProvider: async (req) => {
155
+ const bearer = req.headers.authorization?.replace(/^Bearer /, '');
156
+ if (bearer?.startsWith(API_TOKEN_PREFIX)) {
157
+ const resolved = await resolveApiToken(bearer); // null once revoked or expired
158
+ if (!resolved) throw new Error('Invalid API token');
159
+ if (resolved.scope.kind === 'project') {
160
+ return { userId: resolved.userId, username: resolved.name, role: 'viewer', apiToken: resolved.scope };
161
+ }
162
+ const user = await findUser(resolved.userId); // your own user lookup
163
+ return { userId: user.id, username: user.username, role: user.role, apiToken: resolved.scope };
164
+ }
165
+ // ...your session-based authentication
166
+ },
167
+ });
168
+ ```
169
+
170
+ A request made with a token goes through only when its route's permission is among the
171
+ token's (otherwise 403), and a project outside the token's projects answers 404; the project
172
+ list shows only the token's projects.
173
+
174
+ ---
175
+
176
+ ## πŸ” OAuth: apps the user signs in to
177
+
178
+ An app the user signs in to with their account, such as claude.ai's connector, works with
179
+ StreamBy through an **OAuth access token**, not an API token. StreamBy is the protected resource
180
+ (RFC 9728); the tokens are issued by the host's authorization server (for Nhexa, Nhexa-API).
181
+
182
+ A token is a JWT the server signs with RS256 for one resource (its audience, RFC 8707), checked
183
+ with the server's published keys (JWKS) alone: no secret is shared. It reaches what the user
184
+ approved, and never more than the user can: their role in each project still applies.
185
+
186
+ | Scope | Permissions |
187
+ |---|---|
188
+ | `streamby:read` | Every `:read` permission: `project:read`, `members:read`, `assets:read`, `production:read`, `workflow:read`, `exports:read`, `rights:read`, `jobs:read`. |
189
+ | `streamby:write` | Those, and making and changing: `project:update`, `assets:write`, `production:write`, `workflow:write`, `exports:write`, `rights:write`, `jobs:run`, `projects:create`. |
190
+
191
+ No scope grants deleting (`*:delete`, `project:delete`) nor managing members. And since some
192
+ write permissions also reach routes that delete (`project:update` a connection, a worker or an
193
+ integration; `workflow:write` a pipeline; `exports:write` an export), **an app's `DELETE` is
194
+ refused (403) whatever its route**, but taking a deliverable's mark off, which only forgets the
195
+ mark (the asset stays). API tokens aren't affected. Other scopes (`profile`, `email`) grant
196
+ nothing in StreamBy. `OAUTH_SCOPE_PERMISSIONS` is exported.
197
+
198
+ **Telling clients where to sign in**
199
+
200
+ Set `oauth` in the router's config. A request without a valid token then gets a 401 with
201
+ `WWW-Authenticate: Bearer resource_metadata="..."`, and the metadata says which authorization
202
+ server to use. The router serves it at `GET /streamby/.well-known/oauth-protected-resource`;
203
+ serve it at your root too, where RFC 9728 puts it (`resourceMetadataUrl` defaults to that):
204
+
205
+ ```ts
206
+ import express from 'express';
207
+ import { createStreamByRouter, protectedResourceMetadata } from 'streamby-core';
208
+
209
+ const oauth = { issuer: 'https://api.example.com', resource: 'https://api.example.com/streamby/mcp' };
210
+ app.get('/.well-known/oauth-protected-resource/streamby/mcp', (_req, res) => res.json(protectedResourceMetadata(oauth)));
211
+ app.use('/streamby', createStreamByRouter({ /* ... */ oauth }));
212
+ ```
213
+
214
+ **Validating a token (your `authProvider`)**
215
+
216
+ Your `authProvider` checks the token with `createOAuthTokenVerifier` and returns the user, with
217
+ the token's scope in `apiToken` and the app in `oauth`. A request with `oauth` but no `apiToken`
218
+ is refused, so an app never gets all of its user's access. `looksLikeOAuthAccessToken` tells the
219
+ token from your own session tokens.
220
+
221
+ ```ts
222
+ import { createOAuthTokenVerifier, looksLikeOAuthAccessToken, API_TOKEN_PREFIX } from 'streamby-core';
223
+
224
+ const verifyOAuthToken = createOAuthTokenVerifier({
225
+ issuer: 'https://api.example.com',
226
+ // What a token may be for: StreamBy's resources.
227
+ audiences: ['https://api.example.com/streamby', 'https://api.example.com/streamby/mcp'],
228
+ // The server's JWKS URL, or a function giving its keys (when the host is the server itself).
229
+ jwks: 'https://api.example.com/.well-known/jwks.json',
230
+ });
231
+
232
+ createStreamByRouter({
233
+ // ...
234
+ authProvider: async (req) => {
235
+ const bearer = req.headers.authorization?.replace(/^Bearer /, '');
236
+ if (bearer?.startsWith(API_TOKEN_PREFIX)) { /* an API token, as above */ }
237
+ if (looksLikeOAuthAccessToken(bearer)) {
238
+ const verified = await verifyOAuthToken(bearer); // null: forged, expired, or for another resource
239
+ if (!verified) throw new Error('Invalid OAuth access token');
240
+ // Optional but recommended: has the user taken the app's access away? (your own records)
241
+ if (!(await stillConnected(verified.userId, verified.clientId))) throw new Error('Access revoked');
242
+ const user = await findUser(verified.userId);
243
+ return {
244
+ userId: user.id, username: user.username, role: user.role,
245
+ apiToken: verified.scope,
246
+ oauth: { clientId: verified.clientId, clientName: 'Claude' }, // the name, if you know it
247
+ };
248
+ }
249
+ // ...your session-based authentication
250
+ },
251
+ });
252
+ ```
253
+
254
+ The verifier keeps the keys for 5 minutes (`jwksCacheMs`) and fetches them again for a key id it
255
+ doesn't know (a rotation), at most once a minute. It doesn't know whether the user has since
256
+ taken the app's access away: without that check, a revoked app keeps its access until its access
257
+ token expires (Nhexa's last an hour). Nhexa-API checks the user's consent on every request, so
258
+ taking the access away in accounts.nhexa.cl ends it at once.
259
+
260
+ ---
261
+
262
+ ## πŸ€– MCP server
263
+
264
+ StreamBy serves an [MCP](https://modelcontextprotocol.io) server at `POST /streamby/mcp`
265
+ (Streamable HTTP, stateless). Agents such as Claude Code use it to read and work on StreamBy
266
+ projects with an API token, and apps the user signed in to (claude.ai's connector) with an OAuth
267
+ access token (see above). A browser session gets 401.
268
+
269
+ Each tool calls StreamBy's own REST API with the caller's token. So it reaches exactly what the
270
+ token allows (and, for a user's token or an app, what the user's role in each project allows), and a
271
+ change is saved just like one made from the UI, which shows it when it loads the project.
272
+
273
+ **Tools**
274
+
275
+ A token sees only the tools whose permissions it has (an app: those its scopes grant).
276
+
277
+ | Tools | Permissions |
278
+ |---|---|
279
+ | `list_projects` | any token |
280
+ | `get_project` | `project:read` |
281
+ | `create_project` | `projects:create` |
282
+ | `delete_project` | `project:delete` |
283
+ | `list_assets` (by category, optionally in one storage connection, and filtered by name), `get_asset_metadata` | `assets:read` |
284
+ | `update_asset_metadata` | `assets:write` |
285
+ | `delete_asset` | `assets:delete` |
286
+ | `get_asset_rights` / `set_asset_rights` | `rights:read` / `rights:write` |
287
+ | `get_workflow`, `list_pipelines`, `get_pipeline` | `workflow:read` |
288
+ | `create_pipeline`, `update_pipeline` | `workflow:write` |
289
+ | `list_exports` | `project:read` and `exports:read` |
290
+ | `get_export` | `exports:read` |
291
+ | `get_job`, `list_jobs` | `jobs:read` |
292
+ | `list_workers` | `project:read` |
293
+ | `process_asset`, `retry_job` | `jobs:run` |
294
+ | `list_sequences`, `list_shots`, `list_tasks` | `production:read` |
295
+ | `update_sequence`, `update_shot`, `update_task` | `production:write` |
296
+ | `create_sequence`, `create_shot`, `create_task` | `production:write` and `workflow:write` |
297
+ | `delete_sequence`, `delete_shot`, `delete_task` | `production:delete` |
298
+ | `list_asset_versions`, `get_asset_download_url`, `list_deliverables` | `assets:read` |
299
+ | `request_asset_upload`, `mark_deliverable`, `remove_deliverable` | `assets:write` |
300
+
301
+ The tools take the path a user does in the UI, and each one's description says where its
302
+ result shows there. Production data lives under a production node of the project's workflow:
303
+ if the workflow has none yet, `create_sequence`, `create_shot` and `create_task` first add a
304
+ Shot node to it, as dropping one on the canvas does (so they need `workflow:write`), and make
305
+ nothing if it can't be added.
306
+
307
+ The tools that delete are marked as destructive (`destructiveHint`), so a client can ask before
308
+ running them.
309
+
310
+ **Files**
311
+
312
+ A file never goes through the conversation. `request_asset_upload` makes a new asset (or a new
313
+ version of one) and gives a signed URL to `PUT` its bytes to; `get_asset_download_url` gives a
314
+ signed URL to `GET` an asset's file, its latest version or any earlier one. The tool that has
315
+ the file uploads or downloads it directly, for example Blender's or Godot's own MCP server,
316
+ which can run code. Each URL is for that one file, and given to an API token it works only
317
+ briefly: 15 minutes to upload, 5 to download (`mcp.signedUrlTtl`).
318
+
319
+ **Versions.** Replacing an asset doesn't change its URL: the new version is uploaded to the
320
+ asset's own key, so a site that links to the asset shows the new one with no deploy. Before
321
+ that, the current version is copied to a key of its own (`<key>.v<n>.<ext>`) and recorded, so
322
+ every earlier version can still be downloaded by its id; if it can't be copied, nothing is
323
+ replaced. A storage adapter keeps versions only if it can copy (`copyFile`; the S3 one does);
324
+ otherwise asking for a new version answers 501. Deleting the asset deletes every key of it.
325
+
326
+ Since the URL stays the same, a browser or a CDN may keep serving the earlier version for a
327
+ while. StreamBy doesn't set `Cache-Control` on uploads (a signed URL that did would require
328
+ every client to send the same header): set it where the files are served, for example a short
329
+ `max-age` or `no-cache` on the CDN in front of the bucket, or invalidate the path when an asset
330
+ is replaced.
331
+
332
+ A **deliverable** is an asset, or one of its versions, delivered for a
333
+ production shot or for an export; a shot or an export can have several, and the shot's panel
334
+ in the UI shows them.
335
+
336
+ For example, with Claude connected to StreamBy and to Blender's MCP server, asking *"export the
337
+ scene as glTF and deliver it for shot SH010 of Portfolio"* goes:
338
+
339
+ 1. `list_projects`, `list_sequences`, `list_shots`: the project and the shot's id.
340
+ 2. `request_asset_upload` with `category: '3d-models'`, `fileName: 'SH010_layout.glb'` and
341
+ `contentType: 'model/gltf-binary'`: the new asset's id, and where to upload it.
342
+ 3. In Blender, code that exports the scene and uploads it:
343
+
344
+ ```python
345
+ import bpy, urllib.request
346
+ path = bpy.app.tempdir + "SH010_layout.glb"
347
+ bpy.ops.export_scene.gltf(filepath=path, export_format="GLB")
348
+ with open(path, "rb") as f:
349
+ req = urllib.request.Request(UPLOAD_URL, data=f.read(), method="PUT",
350
+ headers={"Content-Type": "model/gltf-binary"})
351
+ urllib.request.urlopen(req)
352
+ ```
353
+
354
+ 4. `mark_deliverable` with the asset's id and `shotId`: it shows in the shot's panel.
355
+
356
+ Bringing an asset into Blender or Godot is the other way around: `get_asset_download_url`, and
357
+ code there that downloads the URL and imports the file (in Blender,
358
+ `bpy.ops.import_scene.gltf(filepath=...)`). A new version of the same asset is
359
+ `request_asset_upload` with its `assetId`.
360
+
361
+ **Limits and audit**
362
+
363
+ - Each token can make at most 120 requests a minute (a sliding window); an app, per user it acts
364
+ for. Past that, StreamBy answers 429 with `Retry-After`. Each instance counts its own
365
+ requests. Change the limit with `mcp.rateLimit`.
366
+ - Every tool call is recorded in the `mcp_audit` table, created at startup in the main database.
367
+ Each row has the user (a project's token: the token itself), the token's id (an app's call:
368
+ its OAuth client, in `oauthClientId`, and no token), the project, the tool, whether it went
369
+ through, the status, the error and the duration. A row never includes the arguments or the
370
+ token.
371
+
372
+ ```ts
373
+ createStreamByRouter({
374
+ // ...
375
+ mcp: {
376
+ rateLimit: { requests: 120, windowMs: 60_000 }, // the default
377
+ signedUrlTtl: { upload: 900, download: 300 }, // the default, in seconds
378
+ // apiBaseUrl: 'http://127.0.0.1:3000/streamby', // where tools reach the REST API;
379
+ // by default this same process, at the request's local port and mount path
380
+ },
381
+ });
382
+ ```
383
+
384
+ ### Connecting from Claude Code
385
+
386
+ Make an API token in StreamBy, with only the permissions the agent needs: your own
387
+ (Settings β†’ Security β†’ Personal API tokens), or a project's (in the project, Settings β†’ API
388
+ Tokens). Keep it in an environment variable, never in a file:
389
+
390
+ ```bash
391
+ export STREAMBY_TOKEN=sbt_...
392
+ ```
393
+
394
+ Add the server to your project's `.mcp.json`. Claude Code expands `${STREAMBY_TOKEN}` from the
395
+ environment:
396
+
397
+ ```json
398
+ {
399
+ "mcpServers": {
400
+ "streamby": {
401
+ "type": "http",
402
+ "url": "https://your-host/streamby/mcp",
403
+ "headers": { "Authorization": "Bearer ${STREAMBY_TOKEN}" }
404
+ }
405
+ }
406
+ }
407
+ ```
408
+
409
+ ### Connecting from Claude Desktop
410
+
411
+ Claude Desktop runs local (stdio) servers from `claude_desktop_config.json`. Reach StreamBy
412
+ through [`mcp-remote`](https://www.npmjs.com/package/mcp-remote), passing the token in its
413
+ `env`. This file holds it in plain text, so use a token with only the permissions you need,
414
+ and an expiry:
415
+
416
+ ```json
417
+ {
418
+ "mcpServers": {
419
+ "streamby": {
420
+ "command": "npx",
421
+ "args": ["mcp-remote", "https://your-host/streamby/mcp", "--header", "Authorization:${STREAMBY_AUTH}"],
422
+ "env": { "STREAMBY_AUTH": "Bearer sbt_..." }
423
+ }
424
+ }
425
+ }
426
+ ```
427
+
428
+ ### Connecting from claude.ai
429
+
430
+ claude.ai connects through a custom connector, which signs in with OAuth (see above): nobody on
431
+ the team sets up a token or a file. Each person signs in with their own account and approves what
432
+ Claude may do, and Claude reaches only that, and only what their role in each project allows.
433
+
434
+ - **For yourself**: in claude.ai, Settings β†’ Connectors β†’ Add custom connector. Give it a name
435
+ (StreamBy) and the server's URL, `https://your-host/streamby/mcp`, and add it. Then Connect:
436
+ claude.ai opens the sign-in page, and after you sign in, the consent screen, with what it asks
437
+ for (`streamby:read`, and `streamby:write` to make and change things). Approve it.
438
+ - **For your organization** (Team and Enterprise plans): an owner adds it once in Organization
439
+ settings β†’ Connectors, with the same URL; each member then connects it with their own account
440
+ from their Settings β†’ Connectors.
441
+
442
+ claude.ai registers itself with the authorization server (dynamic client registration or a
443
+ client ID metadata document), so there's nothing to set up there. To take its access away, the
444
+ person removes it in their account's connected apps (in Nhexa, accounts.nhexa.cl β†’ Aplicaciones
445
+ β†’ Quitar acceso), or disconnects it in claude.ai.
446
+
447
+ ---
448
+
449
+ ## βš™οΈ Processing workers
450
+
451
+ StreamBy doesn't process assets itself (optimizing a glTF, making LODs, transcoding...): it
452
+ sends the work to a **worker**, an external service, and keeps what it makes. Anyone can write a
453
+ compatible worker by following the contract below.
454
+
455
+ **Where workers come from**
456
+
457
+ - **The host's**, offered to every project, in the router config (each one gated, like the other
458
+ built-ins, by `canUseBuiltin` with kind `'worker'`):
459
+
460
+ ```ts
461
+ createStreamByRouter({
462
+ // ...
463
+ workers: [{
464
+ id: 'gltf', name: 'glTF optimizer',
465
+ endpoint: 'https://worker.example.com/jobs', credential: process.env.GLTF_WORKER_TOKEN,
466
+ kinds: ['gltf-optimize', 'lod'], // what it does
467
+ categories: ['3d-models'], // the assets it takes
468
+ autoKind: 'gltf-optimize', // optional: run on its own (see below)
469
+ timeoutSeconds: 3600, // optional: how long it may take a job
470
+ }],
471
+ workerOptions: {
472
+ callbackBaseUrl: 'https://api.example.com/streamby', // where workers call back (see below)
473
+ maxAttempts: 3, // how many times a job is sent before it fails
474
+ sweepIntervalMs: 30_000, // how often each instance looks for jobs to send again
475
+ },
476
+ });
477
+ ```
478
+
479
+ - **A project's own**, connected from the project (`GET`, `POST` and
480
+ `DELETE /projects/:projectId/workers`; a project's render farms are workers too). Its endpoint
481
+ must be https and never reach a local or private address: not as written, nor as its name
482
+ resolves, which is checked when it's connected and again right before each job is sent (a
483
+ name can point at the server's own network, or start pointing there later). A job for one
484
+ that resolves to such an address fails for good. `workerOptions.allowLocalEndpoints` lifts
485
+ all this, for development. Its credential is stored encrypted, and used only while whoever
486
+ connected it is still in the project.
487
+
488
+ The host's own workers aren't checked this way: the host chose them, and they may well be on
489
+ its private network.
490
+
491
+ `GET /projects/:projectId/workers` lists the workers a member can use.
492
+
493
+ **Asking for a job**
494
+
495
+ `POST /projects/:projectId/jobs/process` with
496
+ `{ assetId, versionId?, worker: { source: 'builtin' | 'project', id }, kind, profile?, resultMode? }`
497
+ (editors and admins; an API token needs `jobs:run`, and the MCP server has `process_asset`).
498
+ `resultMode` decides where the result goes:
499
+
500
+ - `'derived'` (the default): a new asset, linked to the original (`derivedFrom`), which is never
501
+ touched;
502
+ - `'version'`: the asset's new version, at its own key, so its URL serves the result; the
503
+ version it replaces is archived, and can still be downloaded.
504
+
505
+ A worker with an `autoKind` also runs by itself on an asset of its categories when it's marked
506
+ as a deliverable (by a session, or by an API token that has `jobs:run`: one without it marks the
507
+ deliverable and starts nothing).
508
+
509
+ Jobs are kept in the main database (a `jobs` table or collection): they survive restarts, and
510
+ every instance sees them. `GET /projects/:projectId/jobs` lists them (`?assetId=` for one asset's);
511
+ `POST /projects/:projectId/jobs/:jobId/retry` sends a failed one again.
512
+
513
+ **The contract: what a worker receives**
514
+
515
+ StreamBy sends each job with a `POST` to the worker's endpoint
516
+ (`Authorization: Bearer <the worker's credential>`):
517
+
518
+ ```json
519
+ {
520
+ "jobId": "…", "kind": "gltf-optimize", "profile": { "textureFormat": "ktx2" }, "attempt": 1,
521
+ "asset": { "id": "…", "versionId": null, "fileName": "hero.glb", "contentType": "model/gltf-binary", "category": "3d-models" },
522
+ "download": { "url": "https://…signed…", "expiresAt": "…" },
523
+ "callback": { "url": "https://api.example.com/streamby/worker-jobs/<jobId>", "token": "…" },
524
+ "deadline": "…"
525
+ }
526
+ ```
527
+
528
+ The worker answers `2xx` to take it (any other answer, or none, and the job is sent again later),
529
+ downloads the asset with `download.url`, and does its work before `deadline`.
530
+
531
+ `callback.url` is `workerOptions.callbackBaseUrl` plus the job's path. Without it, it's made
532
+ from the request that asked for the job (`req.protocol` and its host): behind a proxy that ends
533
+ TLS (and without Express's `trust proxy`), that comes out as `http://`, which a worker can't call
534
+ back to. Set `callbackBaseUrl` to the router's public https URL.
535
+
536
+ **The contract: what a worker calls back**
537
+
538
+ Every callback is a `POST` to `callback.url` plus an action, with
539
+ `Authorization: Bearer <callback.token>`. The token is the job's for this attempt alone: a
540
+ callback without it, with another, or after the job ended answers 401.
541
+
542
+ | Action | Body | Answer |
543
+ |---|---|---|
544
+ | `/progress` | `{ "progress": 0-100, "stage": "optimizing" }` (either) | the job |
545
+ | `/upload-url` | `{ "fileName": "hero.glb", "contentType": "model/gltf-binary" }` (of the asset's category) | `{ "url", "method": "PUT", "headers", "expiresAt" }`: upload the result there |
546
+ | `/complete` | `{ "report": { … } }` (optional; any JSON object, kept with the result) | the job, completed, with its `result`: `{ assetId, versionId, mode }` |
547
+ | `/fail` | `{ "error": "…", "retryable": false }` | the job, failed with its error (sent again if `retryable` and attempts are left) |
548
+
549
+ A job its worker doesn't complete before its deadline is sent again, up to
550
+ `workerOptions.maxAttempts` times; then it fails, with its error, and can be retried.
551
+
552
+ ---
553
+
554
+ ## πŸ‘₯ Project roles
555
+
556
+ What a user can do in a project depends on their role in that project, as an active member
557
+ (an invitation counts once it's accepted). It never depends on the platform role your
558
+ `authProvider` returns.
559
+
560
+ | Role | Can |
561
+ |---|---|
562
+ | `viewer` | See the project and its content, and archive the project in their own list. |
563
+ | `editor` | Everything a viewer can, plus change the project: its details, connections, exports, files and folders, workflow, pipelines, production, reviews, rights, and start jobs. |
564
+ | `admin` | Everything an editor can, plus manage the project's members and delete the project. |
565
+
566
+ A project's public exports (`GET /streamby/:projectId/export/:exportName`) are read without a
567
+ session. They only run nodes that read data: never a node that uses an account's credential,
568
+ nor one that starts a job.
569
+
570
+ ---
571
+
572
+ ## πŸ—„οΈ Database Schema Setup
573
+
574
+ `streamby-core` requires a pre-configured database schema for its internal operations. It does not manage database connections directly; instead, it expects the host application to provide initialized connection instances. This ensures full control and flexibility for the host environment.
575
+
576
+ The library will only interact with the specified schema and will not touch other schemas in your database. The `reset` option is opt-in and will only drop and recreate the specified `streamby` schema if explicitly set to `true`.
577
+
578
+ ### PostgreSQL Example
579
+
580
+ ```ts
581
+ import { Pool } from "pg";
582
+ import { setupStreambyPg } from "streamby-core/pg/setup"; // Adjust path as per your package structure
583
+
584
+ const pool = new Pool({ connectionString: process.env.DATABASE_URL });
585
+
586
+ // Call setupStreambyPg to ensure the schema and tables are created
587
+ // reset: false (default) ensures idempotency and no data loss
588
+ await setupStreambyPg({ pool, schema: "streamby", reset: false });
589
+ ```
590
+
591
+ ### MongoDB Example
592
+
593
+ ```ts
594
+ import { MongoClient } from "mongodb";
595
+ import { setupStreambyMongo } from "streamby-core/mongo/setup"; // Adjust path as per your package structure
596
+
597
+ const client = new MongoClient(process.env.MONGO_URL!);
598
+
599
+ // Connect the MongoDB client
600
+ await client.connect();
601
+
602
+ // Call setupStreambyMongo to ensure collections and indexes are created
603
+ await setupStreambyMongo({ client, dbName: "streamby" });
604
+ ```
605
+
606
+ ```ts
607
+ import express from 'express';
608
+ import { createStreamByRouter } from 'streamby-core';
609
+
610
+ const app = express();
611
+
612
+ app.use('/streamby', createStreamByRouter({
613
+ storageProvider: {
614
+ type: 's3',
615
+ config: {
616
+ bucket: 'your-bucket-name',
617
+ region: 'your-region',
618
+ accessKeyId: 'YOUR_AWS_KEY',
619
+ secretAccessKey: 'YOUR_AWS_SECRET'
620
+ }
621
+ },
622
+ authProvider: async (req) => {
623
+ // extract token/cookie and validate
624
+ return {
625
+ userId: 'demo',
626
+ projects: ['demo-project'],
627
+ role: 'admin',
628
+ };
629
+ },
630
+ projectProvider: async (projectId) => {
631
+ // fetch project from your DB
632
+ return {
633
+ id: projectId,
634
+ name: 'Demo Project',
635
+ description: 'Demo for StreamBy integration',
636
+ rootFolders: [],
637
+ settings: { allowUpload: true }
638
+ };
639
+ }
640
+ }));
641
+
642
+ app.listen(3000);
643
+ ```
644
+
645
+ ---
646
+
647
+ ## πŸ§ͺ Testing
648
+
649
+ ```bash
650
+ npm run test
651
+ ```
652
+
653
+ Unit tests use [Vitest](https://vitest.dev) and [Supertest](https://www.npmjs.com/package/supertest).
654
+
655
+ ---
656
+
657
+ ## πŸ“ Features
658
+
659
+ - πŸ“‚ Multi-project access control
660
+ - πŸ” Project-aware file uploads & listings
661
+ - 🧹 Modular adapters per storage provider
662
+ - 🧰 Built-in testability with mock auth & storage
663
+ - ✨ Simple to extend for custom business logic
664
+
665
+ ---
666
+
667
+ ## πŸ—ΊοΈ API Endpoints
668
+
669
+ Here's a summary of the key API endpoints provided by `streamby-core`:
670
+
671
+ ### `GET /streamby/projects`
672
+
673
+ Lists all projects accessible by the authenticated user.
674
+
675
+ **Query Parameters:**
676
+ - `archived`: (Optional) Filter projects by their archived status.
677
+ - `true`: Returns only archived projects.
678
+ - `false`: Returns only unarchived projects.
679
+ - (Omitted): Returns all projects (both archived and unarchived).
680
+
681
+ **Example Response:**
682
+ ```json
683
+ {
684
+ "projects": [
685
+ {
686
+ "id": "project1_id",
687
+ "dbType": "nosql",
688
+ "name": "My NoSQL Project",
689
+ "image": "url_to_image",
690
+ "archived": false
691
+ },
692
+ {
693
+ "id": "project2_id",
694
+ "dbType": "sql",
695
+ "name": "My SQL Project",
696
+ "image": "url_to_image",
697
+ "archived": true
698
+ }
699
+ ]
700
+ }
701
+ ```
702
+
703
+ ### `PATCH /streamby/projects/:id/archive`
704
+
705
+ Archives a specific project. After archiving, it returns the complete list of projects for the user, including the updated project.
706
+
707
+ **Example Response:**
708
+ ```json
709
+ {
710
+ "success": true,
711
+ "projects": [
712
+ {
713
+ "id": "project1_id",
714
+ "dbType": "nosql",
715
+ "name": "My NoSQL Project",
716
+ "image": "url_to_image",
717
+ "archived": true
718
+ },
719
+ {
720
+ "id": "project2_id",
721
+ "dbType": "sql",
722
+ "name": "My SQL Project",
723
+ "image": "url_to_image",
724
+ "archived": false
725
+ }
726
+ ]
727
+ }
728
+ ```
729
+
730
+ ### `PATCH /streamby/projects/:id/unarchive`
731
+
732
+ Unarchives a specific project. After unarchiving, it returns the complete list of projects for the user, including the updated project.
733
+
734
+ **Example Response:**
735
+ ```json
736
+ {
737
+ "success": true,
738
+ "projects": [
739
+ {
740
+ "id": "project1_id",
741
+ "dbType": "nosql",
742
+ "name": "My NoSQL Project",
743
+ "image": "url_to_image",
744
+ "archived": false
745
+ },
746
+ {
747
+ "id": "project2_id",
748
+ "dbType": "sql",
749
+ "name": "My SQL Project",
750
+ "image": "url_to_image",
751
+ "archived": false
752
+ }
753
+ ]
754
+ }
755
+ ```
756
+
757
+ ---
758
+
759
+ ## 🚣 Roadmap
760
+
761
+ - [ ] Support for Google Cloud Storage
762
+ - [ ] CLI tool for uploading files
763
+ - [ ] Role-based access control middleware
764
+ - [ ] Plugin system for extended routes
765
+ - [ ] Streaming support (HLS, audio, etc.)
766
+
767
+ ---
768
+
769
+ ## πŸ§‘β€πŸ’» Maintained by
770
+
771
+ **TerminalCore Labs** – part of [Nhexa Entertainment](https://nhexa.cl)
772
+ Developed as open-source infrastructure for creative and media-oriented apps.
773
+
774
+ Visit us [here](https://terminalcore.cl)
775
+
776
+ ---
777
+
778
+ ## πŸ“Ÿ License
779
+
780
+ Copyright (C) 2026 TerminalCore Labs
781
+
782
+ Licensed under the [GNU Affero General Public License v3.0](./LICENSE) (AGPL-3.0-only).
783
+
784
+ > **Note for consumers:** unlike most installable packages, this one is intentionally AGPL rather than MIT. AGPL's copyleft applies to any project that depends on it -- if you install `streamby-core`, your own project must also be licensed under AGPL-3.0 (or a compatible copyleft license). Confirm this is acceptable for your use case before adding this dependency.
785
+