streamby-core 0.39.0 → 0.40.0

package/CHANGELOG.md

@@ -1,5 +1,57 @@
 # Changelog
 
+## 0.40.0
+
+### ⚠️ Breaking changes
+
+- **A project's render farm connections are its workers.** `GET`, `POST` and
+  `DELETE /projects/:projectId/render-farm-connections` are now
+  `/projects/:projectId/workers`, which takes `{ name, provider?, endpoint, credential?, kinds,
+  categories, autoKind?, timeoutSeconds?, description? }`. They're still kept in the project's
+  `renderFarmConnections` list.
+- **Jobs are kept in the main database** (a new `jobs` table, created at startup, or collection),
+  not in memory: they survive restarts and every instance sees them. `GET /jobs/:jobId` answers
+  from there.
+- `canUseBuiltin` may be called with kind `'worker'`.
+
+### Added
+
+- **Processing workers**: StreamBy sends jobs to external workers (the host's, in the router
+  config's `workers`; or a project's own) and keeps what they make, as a new asset derived from
+  the original or as the asset's new version, with the worker's report. The README documents the
+  contract a worker follows.
+  - `POST /projects/:projectId/jobs/process` asks for one; `GET /projects/:projectId/jobs` lists
+    a project's jobs; `POST /projects/:projectId/jobs/:jobId/retry` sends a failed one again.
+  - Workers call back to `/worker-jobs/:jobId/{progress,upload-url,complete,fail}` with a token
+    of that job's attempt alone.
+  - A job not taken, or not finished in time, is sent again (up to `workerOptions.maxAttempts`);
+    every instance looks for them (`workerOptions.sweepIntervalMs`), and only one sends each.
+  - A worker with an `autoKind` runs by itself on an asset of its categories marked as a
+    deliverable, unless it's marked with an API token without `jobs:run`.
+  - **Behind a proxy that ends TLS, set `workerOptions.callbackBaseUrl`** to the router's public
+    https URL: without it, the callback URL is made from the request (`req.protocol`), and comes
+    out as `http://`.
+  - A project's worker is reached only over https and never at a local, private, shared,
+    benchmarking, multicast or reserved address: neither as its endpoint is written nor as its
+    name resolves, checked when it's connected and again right before each job is sent. A job
+    for one that resolves to such an address fails for good. `workerOptions.allowLocalEndpoints`
+    lifts that, for development. Its credential is used only while whoever connected it is
+    still in the project.
+  - A worker's answer to a job it doesn't take is logged, not shown: the job says "The worker
+    refused the job".
+  - A job is sent as whoever asked for it, with the platform role they had then (a host's
+    `canUseBuiltin` decides on sending it as on asking for it).
+- MCP tools `list_workers`, `process_asset`, `list_jobs` and `retry_job`.
+- Deleting a project deletes its jobs.
+- Job events (`jobEvent`) carry the job's `projectId` and `status`.
+
+### Fixed
+
+- **Instances starting together don't deadlock on StreamBy's tables.** The statements that make
+  them (or bring them up to date) at startup run one instance at a time, in a transaction that
+  holds a transaction-level advisory lock (so it works behind a pooler in transaction mode, such
+  as Supabase's).
+
 ## 0.39.0
 
 ### Added