streamby-core 0.41.0 → 0.42.0

package/CHANGELOG.md

@@ -1,5 +1,33 @@
 # Changelog
 
+## 0.42.0
+
+### Added
+
+- **Project documents**: markdown a project's team reads and edits in the UI, and agents read and
+  keep up to date through the MCP server, with every save a version (who saved it, and when). The
+  README's Documents section documents them.
+  - **New tables: `documents` and `document_versions`.** They're created at startup in the main
+    database (collections in MongoDB; in memory without one).
+  - Routes under `/projects/:projectId/documents`: the tree, a document, creating, a new version
+    (`PATCH` with `baseVersion`; 409 with the document as it is now when it changed since), a
+    section's (`PATCH .../sections`), moving and marking exportable, the history
+    (`.../versions[/:version]`), publishing a version (`POST .../publish`) and deleting (a folder
+    with everything in it). Deleting a project deletes its documents.
+  - Permissions `docs:read`, `docs:write`, `docs:publish` and `docs:delete`. Publishing (a
+    version for the exports, or whether they deliver it: `POST .../publish`, `PATCH` with
+    `exportable`) needs `docs:publish`, not `docs:write`. The OAuth scopes grant reading
+    (`streamby:read`) and writing (`streamby:write`); an app publishes and deletes none, whatever
+    its scopes: what it edits stays a draft.
+  - In MongoDB, a save writes the document and its version in a transaction when the server has
+    them (a replica set or a sharded cluster); on a standalone server, two writes.
+  - MCP tools `list_documents`, `get_document`, `list_document_versions`, `create_document`,
+    `update_document` and `update_document_section`.
+  - An export's `documentsNode` delivers the published version of the project's exportable
+    documents (all, one folder's, or some), never a draft.
+  - A `documentEvent` to the project's active members on each change.
+  - Types `ProjectDocument`, `DocumentVersion`, `DocumentAuthor` and `DocumentKind`.
+
 ## 0.41.0
 
 ### Added

package/README.md

@@ -134,6 +134,7 @@ hash is stored, with a short prefix to tell it apart.
 | Exports | `exports:read`, `exports:write` | The project's exports. |
 | Rights | `rights:read`, `rights:write` | Assets' rights. |
 | Jobs | `jobs:read`, `jobs:run` | Following jobs; starting them (transcoding, rendering, AI, publishing a deliverable). |
+| Documents | `docs:read`, `docs:write`, `docs:publish`, `docs:delete` | The project's documents: reading them and their history; creating, editing and moving them; publishing them (the version its exports deliver, and whether they deliver it); deleting them. |
 | Account | `projects:create` | Creating projects. A user's token only; the token reaches what it creates. |
 
 `PROJECT_PERMISSIONS` and `ACCOUNT_PERMISSIONS` are exported. A route no permission covers
@@ -185,15 +186,16 @@ approved, and never more than the user can: their role in each project still app
 
 | Scope | Permissions |
 |---|---|
-| `streamby:read` | Every `:read` permission: `project:read`, `members:read`, `assets:read`, `production:read`, `workflow:read`, `exports:read`, `rights:read`, `jobs:read`. |
-| `streamby:write` | Those, and making and changing: `project:update`, `assets:write`, `production:write`, `workflow:write`, `exports:write`, `rights:write`, `jobs:run`, `projects:create`. |
+| `streamby:read` | Every `:read` permission: `project:read`, `members:read`, `assets:read`, `production:read`, `workflow:read`, `exports:read`, `rights:read`, `jobs:read`, `docs:read`. |
+| `streamby:write` | Those, and making and changing: `project:update`, `assets:write`, `production:write`, `workflow:write`, `exports:write`, `rights:write`, `jobs:run`, `projects:create`, `docs:write`. |
 
 No scope grants deleting (`*:delete`, `project:delete`) nor managing members. And since some
 write permissions also reach routes that delete (`project:update` a connection, a worker or an
 integration; `workflow:write` a pipeline; `exports:write` an export), **an app's `DELETE` is
 refused (403) whatever its route**, but taking a deliverable's mark off, which only forgets the
-mark (the asset stays). API tokens aren't affected. Other scopes (`profile`, `email`) grant
-nothing in StreamBy. `OAUTH_SCOPE_PERMISSIONS` is exported.
+mark (the asset stays). Nor does an app publish a document (`POST .../publish`, or a `PATCH`
+changing `exportable`), whatever its scopes: what it edits stays a draft until someone publishes
+it. API tokens aren't affected. Other scopes (`profile`, `email`) grant nothing in StreamBy. `OAUTH_SCOPE_PERMISSIONS` is exported.
 
 **Telling clients where to sign in**
 
@@ -297,6 +299,8 @@ A token sees only the tools whose permissions it has (an app: those its scopes g
 | `delete_sequence`, `delete_shot`, `delete_task` | `production:delete` |
 | `list_asset_versions`, `get_asset_download_url`, `list_deliverables` | `assets:read` |
 | `request_asset_upload`, `mark_deliverable`, `remove_deliverable` | `assets:write` |
+| `list_documents`, `get_document`, `list_document_versions` | `docs:read` |
+| `create_document`, `update_document`, `update_document_section` | `docs:write` |
 
 The tools take the path a user does in the UI, and each one's description says where its
 result shows there. Production data lives under a production node of the project's workflow:
@@ -446,6 +450,49 @@ person removes it in their account's connected apps (in Nhexa, accounts.nhexa.cl
 
 ---
 
+## 📄 Documents
+
+Each project keeps its documents: markdown its team reads and edits (its concept, its
+architecture, its decisions), and agents read and keep up to date through the MCP server. They're
+kept in the main database (the `documents` and `document_versions` tables, created at startup,
+or collections), each one in a single project, in folders.
+
+- **Every save is a version**, with who saved it and when: a user, an API token, or an app on a
+  user's behalf (`{ userId, username, via: 'session' | 'apiToken' | 'oauth', tokenId?, clientId?, clientName? }`).
+- **Nothing is overwritten without notice.** A change says the version it was made on
+  (`baseVersion`); if someone saved another one since, it's refused with a 409 and the document
+  as it is now, to read again.
+- **A section** is what's under a heading (`## Architecture`), up to the next one of the same
+  level or above: it can be replaced alone, the rest untouched.
+- **Drafts and published versions.** A document marked exportable delivers, through the project's
+  exports, only the version published from its history; editing it changes nothing exported until
+  another version is published. An export's `documentsNode` (on the StreamBy node's data input)
+  delivers them, `[{ id, title, parentId, position, version, content, publishedAt }]`: all the
+  project's, those in one folder (`data.folderId`), or some (`data.documentIds`).
+- **In real time**, a `documentEvent` reaches the project's active members on each change:
+  `{ projectId, action: 'created' | 'saved' | 'changed' | 'deleted', documentIds, version?, title?, author }`
+  (`saved` is a new version; `changed`, a move or a publication).
+
+| Route | Permission |
+|---|---|
+| `GET /projects/:projectId/documents`: the tree, without contents | `docs:read` |
+| `POST /projects/:projectId/documents` `{ title, content?, kind?: 'document' \| 'folder', parentId?, position? }` | `docs:write` |
+| `GET /projects/:projectId/documents/:docId` | `docs:read` |
+| `PATCH /projects/:projectId/documents/:docId` `{ baseVersion, title?, content? }` (a new version) and/or `{ parentId?, position? }` | `docs:write` |
+| `PATCH /projects/:projectId/documents/:docId` `{ exportable }` | `docs:write` and `docs:publish` |
+| `PATCH /projects/:projectId/documents/:docId/sections` `{ baseVersion, heading, content }` | `docs:write` |
+| `GET /projects/:projectId/documents/:docId/versions` and `/versions/:version` | `docs:read` |
+| `POST /projects/:projectId/documents/:docId/publish` `{ version }` (`null`: none) | `docs:publish` |
+| `DELETE /projects/:projectId/documents/:docId` (a folder, with everything in it) | `docs:delete` |
+
+Editors and admins create, edit, publish and delete them; viewers read them. An app connected
+with OAuth reads and edits them, but publishes and deletes none: what it edits stays a draft.
+
+In MongoDB, a save writes the document and its version in a transaction when the server has them
+(a replica set or a sharded cluster). A standalone server doesn't: there they're two writes.
+
+---
+
 ## ⚙️ Processing workers
 
 StreamBy doesn't process assets itself (optimizing a glTF, making LODs, transcoding...): it
@@ -560,7 +607,7 @@ What a user can do in a project depends on their role in that project, as an act
 | Role | Can |
 |---|---|
 | `viewer` | See the project and its content, and archive the project in their own list. |
-| `editor` | Everything a viewer can, plus change the project: its details, connections, exports, files and folders, workflow, pipelines, production, reviews, rights, and start jobs. |
+| `editor` | Everything a viewer can, plus change the project: its details, connections, exports, files and folders, workflow, pipelines, production, reviews, rights, documents, and start jobs. |
 | `admin` | Everything an editor can, plus manage the project's members and delete the project. |
 
 A project's public exports (`GET /streamby/:projectId/export/:exportName`) are read without a

package/dist/adapters/database/connectionManager.js

@@ -254,6 +254,38 @@ const createTables = async (pool) => {
       ALTER TABLE streamby.mcp_audit ALTER COLUMN "tokenId" DROP NOT NULL;
     `);
         console.log('✅ "mcp_audit" table ensured to exist.');
+        // Projects' documents and their versions (see services/documents). Main db only.
+        await pool.query(`
+      CREATE TABLE IF NOT EXISTS streamby.documents (
+        id                  TEXT PRIMARY KEY,
+        "projectId"         VARCHAR(255) NOT NULL,
+        kind                VARCHAR(20) NOT NULL DEFAULT 'document',
+        title               TEXT NOT NULL,
+        "parentId"          TEXT,
+        position            INTEGER NOT NULL DEFAULT 0,
+        content             TEXT NOT NULL DEFAULT '',
+        version             INTEGER NOT NULL,
+        exportable          BOOLEAN NOT NULL DEFAULT FALSE,
+        "publishedVersion"  INTEGER,
+        "createdAt"         TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
+        "createdBy"         JSONB NOT NULL,
+        "updatedAt"         TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
+        "updatedBy"         JSONB NOT NULL
+      );
+      CREATE INDEX IF NOT EXISTS documents_project_idx ON streamby.documents ("projectId");
+      CREATE TABLE IF NOT EXISTS streamby.document_versions (
+        "documentId"  TEXT NOT NULL,
+        "projectId"   VARCHAR(255) NOT NULL,
+        version       INTEGER NOT NULL,
+        title         TEXT NOT NULL,
+        content       TEXT NOT NULL,
+        author        JSONB NOT NULL,
+        "createdAt"   TIMESTAMP WITH TIME ZONE NOT NULL DEFAULT NOW(),
+        PRIMARY KEY ("documentId", version)
+      );
+      CREATE INDEX IF NOT EXISTS document_versions_project_idx ON streamby.document_versions ("projectId");
+    `);
+        console.log('✅ "documents" and "document_versions" tables ensured to exist.');
     }
     catch (error) {
         console.error('❌ Error ensuring tables exist:', error);

package/dist/index.d.ts

@@ -1,7 +1,7 @@
 export { createStreamByRouter } from './middleware/createRouter';
 export { emitToUser } from './services/wsHub';
 export { createNotification } from './services/notification';
-export type { UserPlan, UserSubscription, Auth, ApiTokenScope, ApiTokenKind, ApiTokenPermission, ProjectPermission, AccountPermission, OAuthClientInfo, OAuthResourceConfig } from './types';
+export type { UserPlan, UserSubscription, Auth, ApiTokenScope, ApiTokenKind, ApiTokenPermission, ProjectPermission, AccountPermission, OAuthClientInfo, OAuthResourceConfig, ProjectDocument, DocumentVersion, DocumentAuthor, DocumentKind } from './types';
 export { resolveApiToken, API_TOKEN_PREFIX } from './services/apiTokens';
 export { PROJECT_PERMISSIONS, ACCOUNT_PERMISSIONS } from './services/apiTokenPermissions';
 export type { ResolvedApiToken } from './services/apiTokens';

package/dist/mcp/tools.js

@@ -358,6 +358,74 @@ exports.TOOLS = [
         call: ({ projectId, jobId }) => ({ method: 'POST', path: `/projects/${enc(projectId)}/jobs/${enc(jobId)}/retry` }),
         pick: (data) => data?.job,
     }),
+    // ── Documents ──
+    tool({
+        name: 'list_documents', title: 'List documents', permissions: ['docs:read'],
+        description: "A project's documents (its concept, architecture, decisions...), as the tree of its Documents section shows them: each one's id, title, folder (parentId; a folder's kind is 'folder'), position, current version and the version published through its exports, without their content (read one with get_document).",
+        input: { projectId },
+        projectOf: byProject,
+        call: ({ projectId }) => ({ method: 'GET', path: `/projects/${enc(projectId)}/documents` }),
+        pick: (data) => data?.documents ?? [],
+    }),
+    tool({
+        name: 'get_document', title: 'Read a document', permissions: ['docs:read'],
+        description: "A document's markdown and its current version, the one an update is made on. With `version`, that past version instead (see list_document_versions).",
+        input: { projectId, documentId: id('document'), version: zod_1.z.number().int().min(1).optional().describe('A past version to read; the current one if not given') },
+        projectOf: byProject,
+        call: ({ projectId, documentId, version }) => ({
+            method: 'GET',
+            path: `/projects/${enc(projectId)}/documents/${enc(documentId)}${version ? `/versions/${version}` : ''}`,
+        }),
+        pick: (data) => data?.document ?? data?.version,
+    }),
+    tool({
+        name: 'list_document_versions', title: "List a document's versions", permissions: ['docs:read'],
+        description: "A document's history, the latest first: each version's number, title, who saved it (a user, an API token, or an app on a user's behalf) and when.",
+        input: { projectId, documentId: id('document') },
+        projectOf: byProject,
+        call: ({ projectId, documentId }) => ({ method: 'GET', path: `/projects/${enc(projectId)}/documents/${enc(documentId)}/versions` }),
+        pick: (data) => data?.versions ?? [],
+    }),
+    tool({
+        name: 'create_document', title: 'Create a document', permissions: ['docs:write'], write: true,
+        description: "Creates a markdown document (or, with kind 'folder', a folder to group them) in a project's Documents section, at the end of its folder. It's a draft: nothing is exported until someone publishes a version of it in the UI.",
+        input: {
+            projectId,
+            title: zod_1.z.string().min(1),
+            content: zod_1.z.string().optional().describe('Its markdown'),
+            kind: zod_1.z.enum(['document', 'folder']).optional(),
+            parentId: zod_1.z.string().optional().describe("The folder it goes in (a folder's id); at the top if not given"),
+        },
+        projectOf: byProject,
+        call: ({ projectId, ...fields }) => ({ method: 'POST', path: `/projects/${enc(projectId)}/documents`, body: defined(fields) }),
+        pick: (data) => data?.document,
+    }),
+    tool({
+        name: 'update_document', title: 'Update a document', permissions: ['docs:write'], write: true,
+        description: "Saves a new version of a document: its whole markdown, its title, or both. `baseVersion` is the version it was read at (get_document): if someone saved another one since, nothing is saved and the answer says so; read it again and make the change on the new version. To change one part, update_document_section rewrites less.",
+        input: {
+            projectId, documentId: id('document'),
+            baseVersion: zod_1.z.number().int().min(1).describe('The version the change was made on'),
+            title: zod_1.z.string().min(1).optional(),
+            content: zod_1.z.string().optional().describe('The whole new markdown'),
+        },
+        projectOf: byProject,
+        call: ({ projectId, documentId, ...fields }) => ({ method: 'PATCH', path: `/projects/${enc(projectId)}/documents/${enc(documentId)}`, body: defined(fields) }),
+        pick: (data) => data?.document,
+    }),
+    tool({
+        name: 'update_document_section', title: "Update a document's section", permissions: ['docs:write'], write: true,
+        description: "Saves a new version of a document with only what's under one of its headings replaced (up to the next heading of the same level or above), the rest untouched. `heading` is the heading as written (\"## Architecture\") or its title alone. `baseVersion` is the version it was read at: if the document changed since, nothing is saved; read it again.",
+        input: {
+            projectId, documentId: id('document'),
+            baseVersion: zod_1.z.number().int().min(1).describe('The version the change was made on'),
+            heading: zod_1.z.string().min(1).describe('The section\'s heading, e.g. "## Architecture"'),
+            content: zod_1.z.string().describe("The section's new markdown, without its heading"),
+        },
+        projectOf: byProject,
+        call: ({ projectId, documentId, ...fields }) => ({ method: 'PATCH', path: `/projects/${enc(projectId)}/documents/${enc(documentId)}/sections`, body: fields }),
+        pick: (data) => data?.document,
+    }),
     // ── Projects ──
     tool({
         name: 'create_project', title: 'Create a project', permissions: ['projects:create'], write: true,

package/dist/middleware/createRouter.js

@@ -44,11 +44,14 @@ const worker_1 = require("./routes/worker");
 const workerCallback_1 = require("./routes/workerCallback");
 const workerJobs_1 = require("../services/workerJobs");
 const deliverable_1 = require("./routes/deliverable");
+const document_1 = require("./routes/document");
+const documents_1 = require("../services/documents");
 function createStreamByRouter(config) {
     const router = express_1.default.Router();
     if (config.encrypt)
         (0, encryption_1.setEncryptionKey)(config.encrypt, config.encryptPreviousKeys);
     (0, jobStore_1.setJobStoreDatabase)(config);
+    (0, documents_1.setDocumentStoreDatabase)(config);
     // Establish DB connections asynchronously (Postgres/Mongo + schema migrations).
     // Until they're ready, HOLD incoming requests instead of letting them through —
     // route handlers resolve projects/exports (e.g. the /menu-list export) against a
@@ -149,5 +152,6 @@ function createStreamByRouter(config) {
     router.use((0, pipeline_1.pipelineRouter)(config));
     router.use((0, worker_1.workerRouter)({ ...config, adapter }));
     router.use((0, deliverable_1.deliverableRouter)({ ...config, adapter }));
+    router.use((0, document_1.documentRouter)(config));
     return router;
 }

package/dist/middleware/routes/document.d.ts

@@ -0,0 +1,3 @@
+import { Router } from 'express';
+import { StreamByConfig } from '../../types';
+export declare function documentRouter(_config: StreamByConfig): Router;

package/dist/middleware/routes/document.js

@@ -0,0 +1,343 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.documentRouter = documentRouter;
+const express_1 = require("express");
+const crypto_1 = __importDefault(require("crypto"));
+const manager_1 = require("../../models/manager");
+const auth_1 = require("../../utils/auth");
+const wsHub_1 = require("../../services/wsHub");
+const documents_1 = require("../../services/documents");
+const auth_2 = require("../../services/auth");
+const MAX_TITLE = 200;
+const MAX_CONTENT = 1000000;
+// What happened to a project's documents, to whoever is in it now: a view of a document
+// refreshes itself, an editor warns of a newer version. `version` and `author` come with a save.
+function broadcast(project, data) {
+    const projectId = String(project._id ?? project.id);
+    for (const member of project.members ?? []) {
+        if ((0, auth_1.activeMemberOf)(project, member.userId))
+            (0, wsHub_1.emitToUser)(member.userId, { type: 'documentEvent', data: { projectId, ...data } });
+    }
+}
+const isVersion = (value) => Number.isInteger(value) && value >= 1;
+function checkTitle(value) {
+    if (typeof value !== 'string' || !value.trim())
+        return { error: 'A title is required' };
+    if (value.trim().length > MAX_TITLE)
+        return { error: `A title has at most ${MAX_TITLE} characters` };
+    return value.trim();
+}
+function checkContent(value) {
+    if (typeof value !== 'string')
+        return { error: 'content must be text (markdown)' };
+    if (value.length > MAX_CONTENT)
+        return { error: `A document has at most ${MAX_CONTENT} characters` };
+    return value;
+}
+// Where a document may go: the top (null), or a folder of the same project that isn't the
+// document itself nor inside it.
+function checkParent(value, documents, movingId) {
+    if (value === null)
+        return null;
+    if (typeof value !== 'string')
+        return { error: 'parentId must be a folder\'s id, or null' };
+    const parent = documents.find(d => d.id === value);
+    if (!parent)
+        return { error: 'No such folder in this project' };
+    if (parent.kind !== 'folder')
+        return { error: 'A document goes in a folder, not in another document' };
+    if (movingId && (0, documents_1.withDescendants)(documents, movingId).includes(value))
+        return { error: 'A folder can\'t go inside itself' };
+    return value;
+}
+const failed = (value) => typeof value === 'object' && value !== null && 'error' in value;
+function documentRouter(_config) {
+    const router = (0, express_1.Router)();
+    const Project = (0, manager_1.getModel)('projects');
+    // The project, if the caller may see it (or, with `edit`, change it); otherwise the answer.
+    async function projectFor(req, res, edit = false) {
+        const auth = req.auth;
+        const project = await Project.findOne({ _id: req.params.projectId });
+        if (!project) {
+            res.status(404).json({ message: 'Project not found' });
+            return null;
+        }
+        if (!(edit ? auth_1.canEditProject : auth_1.isProjectMember)(project, auth.userId)) {
+            res.status(403).json({ message: 'Unauthorized' });
+            return null;
+        }
+        return { project, auth, projectId: String(req.params.projectId) };
+    }
+    const conflict = (res, current) => res.status(409).json({
+        message: `The document changed: it's at version ${current.version} now. Read it again, and make the change on that version.`,
+        document: current,
+    });
+    router.get('/projects/:projectId/documents', async (req, res) => {
+        try {
+            const found = await projectFor(req, res);
+            if (!found)
+                return;
+            res.json({ documents: await (0, documents_1.documentStore)().list(found.projectId) });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    router.post('/projects/:projectId/documents', async (req, res) => {
+        try {
+            const found = await projectFor(req, res, true);
+            if (!found)
+                return;
+            const { project, auth, projectId } = found;
+            const kind = req.body?.kind === 'folder' ? 'folder' : 'document';
+            if (req.body?.kind !== undefined && !['document', 'folder'].includes(req.body.kind))
+                return res.status(400).json({ message: "kind is 'document' or 'folder'" });
+            const title = checkTitle(req.body?.title);
+            if (failed(title))
+                return res.status(400).json({ message: title.error });
+            if (kind === 'folder' && req.body?.content)
+                return res.status(400).json({ message: 'A folder has no content' });
+            const content = kind === 'folder' ? '' : checkContent(req.body?.content ?? '');
+            if (failed(content))
+                return res.status(400).json({ message: content.error });
+            const store = (0, documents_1.documentStore)();
+            const documents = await store.list(projectId);
+            const parentId = checkParent(req.body?.parentId ?? null, documents);
+            if (failed(parentId))
+                return res.status(400).json({ message: parentId.error });
+            if (req.body?.position !== undefined && !Number.isInteger(req.body.position))
+                return res.status(400).json({ message: 'position must be a whole number' });
+            // At the end of its folder, unless placed.
+            const siblings = documents.filter(d => d.parentId === parentId);
+            const position = req.body?.position ?? (siblings.length ? Math.max(...siblings.map(d => d.position)) + 1 : 0);
+            const now = new Date();
+            const author = (0, documents_1.authorOf)(auth);
+            const document = await store.create({
+                id: crypto_1.default.randomUUID(), projectId, kind, title, parentId, position, content, version: 1,
+                exportable: false, publishedVersion: null, createdAt: now, createdBy: author, updatedAt: now, updatedBy: author,
+            });
+            broadcast(project, { action: 'created', documentIds: [document.id], version: 1, title, author });
+            res.status(201).json({ document });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    router.get('/projects/:projectId/documents/:docId', async (req, res) => {
+        try {
+            const found = await projectFor(req, res);
+            if (!found)
+                return;
+            const document = await (0, documents_1.documentStore)().get(found.projectId, req.params.docId);
+            if (!document)
+                return res.status(404).json({ message: 'Document not found' });
+            res.json({ document });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    // Its title or content (a new version, made on `baseVersion`), and where it is in the tree
+    // or whether it's exportable (not a version).
+    router.patch('/projects/:projectId/documents/:docId', async (req, res) => {
+        try {
+            const found = await projectFor(req, res, true);
+            if (!found)
+                return;
+            const { project, auth, projectId } = found;
+            const { docId } = req.params;
+            const body = req.body ?? {};
+            const store = (0, documents_1.documentStore)();
+            const current = await store.get(projectId, docId);
+            if (!current)
+                return res.status(404).json({ message: 'Document not found' });
+            const changesText = body.title !== undefined || body.content !== undefined;
+            const changesPlace = body.parentId !== undefined || body.position !== undefined || body.exportable !== undefined;
+            if (!changesText && !changesPlace)
+                return res.status(400).json({ message: 'Nothing to change: title, content, parentId, position or exportable' });
+            // Everything is checked before anything changes.
+            let text;
+            if (changesText) {
+                if (!isVersion(body.baseVersion))
+                    return res.status(400).json({ message: 'baseVersion is required: the version the change was made on' });
+                const title = body.title === undefined ? current.title : checkTitle(body.title);
+                if (failed(title))
+                    return res.status(400).json({ message: title.error });
+                if (current.kind === 'folder' && body.content)
+                    return res.status(400).json({ message: 'A folder has no content' });
+                const content = body.content === undefined ? current.content : checkContent(body.content);
+                if (failed(content))
+                    return res.status(400).json({ message: content.error });
+                text = { title, content };
+            }
+            const place = {};
+            if (body.parentId !== undefined) {
+                const parentId = checkParent(body.parentId, await store.list(projectId), docId);
+                if (failed(parentId))
+                    return res.status(400).json({ message: parentId.error });
+                place.parentId = parentId;
+            }
+            if (body.position !== undefined) {
+                if (!Number.isInteger(body.position))
+                    return res.status(400).json({ message: 'position must be a whole number' });
+                place.position = body.position;
+            }
+            if (body.exportable !== undefined) {
+                // Whether its exports deliver it is publishing it: a token needs docs:publish.
+                if (auth.apiToken && !(0, auth_2.tokenAllows)(auth.apiToken, 'docs:publish')) {
+                    return res.status(403).json({ error: "This API token doesn't have the 'docs:publish' permission" });
+                }
+                if (typeof body.exportable !== 'boolean')
+                    return res.status(400).json({ message: 'exportable is true or false' });
+                if (current.kind === 'folder')
+                    return res.status(400).json({ message: 'A folder isn\'t exported: its documents are' });
+                place.exportable = body.exportable;
+            }
+            const author = (0, documents_1.authorOf)(auth);
+            let document = current;
+            if (text) {
+                document = await store.save(projectId, docId, body.baseVersion, text, author, new Date());
+                if (!document) {
+                    const now = await store.get(projectId, docId);
+                    return now ? conflict(res, now) : res.status(404).json({ message: 'Document not found' });
+                }
+            }
+            if (changesPlace)
+                document = await store.update(projectId, docId, place);
+            if (!document)
+                return res.status(404).json({ message: 'Document not found' });
+            broadcast(project, text
+                ? { action: 'saved', documentIds: [docId], version: document.version, title: document.title, author }
+                : { action: 'changed', documentIds: [docId], author });
+            res.json({ document });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    // What's under one of its headings, the rest untouched: a new version, made on `baseVersion`.
+    router.patch('/projects/:projectId/documents/:docId/sections', async (req, res) => {
+        try {
+            const found = await projectFor(req, res, true);
+            if (!found)
+                return;
+            const { project, auth, projectId } = found;
+            const { docId } = req.params;
+            const { baseVersion, heading, content } = req.body ?? {};
+            if (!isVersion(baseVersion))
+                return res.status(400).json({ message: 'baseVersion is required: the version the change was made on' });
+            if (typeof heading !== 'string' || !heading.trim())
+                return res.status(400).json({ message: 'heading is required: the section\'s heading, e.g. "## Architecture"' });
+            const body = checkContent(content);
+            if (failed(body))
+                return res.status(400).json({ message: body.error });
+            const store = (0, documents_1.documentStore)();
+            const current = await store.get(projectId, docId);
+            if (!current)
+                return res.status(404).json({ message: 'Document not found' });
+            if (current.kind === 'folder')
+                return res.status(400).json({ message: 'A folder has no content' });
+            // A section of another version than the one it was read on could be another text.
+            if (current.version !== baseVersion)
+                return conflict(res, current);
+            const replaced = (0, documents_1.replaceSection)(current.content, heading, body);
+            if ('error' in replaced) {
+                return replaced.error === 'not-found'
+                    ? res.status(404).json({ message: `No section "${heading.trim()}" in the document` })
+                    : res.status(400).json({ message: `More than one section is "${heading.trim()}": give the heading with its #s, or update the whole document` });
+            }
+            const author = (0, documents_1.authorOf)(auth);
+            const document = await store.save(projectId, docId, baseVersion, { title: current.title, content: replaced.content }, author, new Date());
+            if (!document) {
+                const now = await store.get(projectId, docId);
+                return now ? conflict(res, now) : res.status(404).json({ message: 'Document not found' });
+            }
+            broadcast(project, { action: 'saved', documentIds: [docId], version: document.version, title: document.title, author });
+            res.json({ document });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    router.get('/projects/:projectId/documents/:docId/versions', async (req, res) => {
+        try {
+            const found = await projectFor(req, res);
+            if (!found)
+                return;
+            const store = (0, documents_1.documentStore)();
+            if (!(await store.get(found.projectId, req.params.docId)))
+                return res.status(404).json({ message: 'Document not found' });
+            res.json({ versions: await store.versions(found.projectId, req.params.docId) });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    router.get('/projects/:projectId/documents/:docId/versions/:version', async (req, res) => {
+        try {
+            const found = await projectFor(req, res);
+            if (!found)
+                return;
+            const number = Number(req.params.version);
+            const version = isVersion(number) ? await (0, documents_1.documentStore)().version(found.projectId, req.params.docId, number) : null;
+            if (!version)
+                return res.status(404).json({ message: 'Version not found' });
+            res.json({ version });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    // The version its exports deliver (null: none, it's no longer published). Editing it later
+    // changes nothing exported until another version is published.
+    router.post('/projects/:projectId/documents/:docId/publish', async (req, res) => {
+        try {
+            const found = await projectFor(req, res, true);
+            if (!found)
+                return;
+            const { project, auth, projectId } = found;
+            const { docId } = req.params;
+            const version = req.body?.version;
+            if (version !== null && !isVersion(version))
+                return res.status(400).json({ message: 'version is the version to publish, or null to publish none' });
+            const store = (0, documents_1.documentStore)();
+            const current = await store.get(projectId, docId);
+            if (!current)
+                return res.status(404).json({ message: 'Document not found' });
+            if (current.kind === 'folder')
+                return res.status(400).json({ message: 'A folder isn\'t published: its documents are' });
+            if (version !== null && version > current.version)
+                return res.status(404).json({ message: 'Version not found' });
+            const document = await store.update(projectId, docId, { publishedVersion: version });
+            broadcast(project, { action: 'changed', documentIds: [docId], author: (0, documents_1.authorOf)(auth) });
+            res.json({ document });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    // A document, or a folder with everything in it.
+    router.delete('/projects/:projectId/documents/:docId', async (req, res) => {
+        try {
+            const found = await projectFor(req, res, true);
+            if (!found)
+                return;
+            const { project, auth, projectId } = found;
+            const store = (0, documents_1.documentStore)();
+            const documents = await store.list(projectId);
+            if (!documents.some(d => d.id === req.params.docId))
+                return res.status(404).json({ message: 'Document not found' });
+            const ids = (0, documents_1.withDescendants)(documents, req.params.docId);
+            await store.remove(projectId, ids);
+            broadcast(project, { action: 'deleted', documentIds: ids, author: (0, documents_1.authorOf)(auth) });
+            res.json({ deleted: ids });
+        }
+        catch (err) {
+            res.status(500).json({ message: err.message });
+        }
+    });
+    return router;
+}

package/dist/middleware/routes/project.js

@@ -13,6 +13,7 @@ const apiTokens_1 = require("../../services/apiTokens");
 const nosqlDb_1 = require("../../utils/nosqlDb");
 const production_1 = require("../../services/production");
 const jobStore_1 = require("../../services/jobStore");
+const documents_1 = require("../../services/documents");
 function projectRouter(config) {
     const router = (0, express_1.Router)();
     const Project = (0, manager_1.getModel)('projects');
@@ -261,6 +262,8 @@ function projectRouter(config) {
             await (0, production_1.deleteProductionData)(String(projectId));
             if (projectId)
                 await (0, jobStore_1.jobStore)().deleteProject(String(projectId));
+            if (projectId)
+                await (0, documents_1.documentStore)().deleteProject(String(projectId));
             const allProjects = await Project.find({});
             const projects = allProjects
                 .filter(project => {

package/dist/services/apiTokenPermissions.d.ts

@@ -1,5 +1,5 @@
 /** Permissions on a project, for the projects a token reaches. */
-export declare const PROJECT_PERMISSIONS: readonly ["project:read", "project:update", "project:delete", "members:read", "members:manage", "assets:read", "assets:write", "assets:delete", "production:read", "production:write", "production:delete", "workflow:read", "workflow:write", "exports:read", "exports:write", "rights:read", "rights:write", "jobs:read", "jobs:run"];
+export declare const PROJECT_PERMISSIONS: readonly ["project:read", "project:update", "project:delete", "members:read", "members:manage", "assets:read", "assets:write", "assets:delete", "production:read", "production:write", "production:delete", "workflow:read", "workflow:write", "exports:read", "exports:write", "rights:read", "rights:write", "jobs:read", "jobs:run", "docs:read", "docs:write", "docs:publish", "docs:delete"];
 /** Permissions on the account: only a user's token has them. */
 export declare const ACCOUNT_PERMISSIONS: readonly ["projects:create"];
 export type ProjectPermission = typeof PROJECT_PERMISSIONS[number];

package/dist/services/apiTokenPermissions.js

@@ -15,6 +15,7 @@ exports.PROJECT_PERMISSIONS = [
     'exports:read', 'exports:write',
     'rights:read', 'rights:write',
     'jobs:read', 'jobs:run',
+    'docs:read', 'docs:write', 'docs:publish', 'docs:delete',
 ];
 /** Permissions on the account: only a user's token has them. */
 exports.ACCOUNT_PERMISSIONS = ['projects:create'];
@@ -94,6 +95,13 @@ const RULES = [
     // Jobs: transcoding, rendering, AI, publishing a deliverable.
     rule('read', `${P}/jobs`, 'jobs:read'),
     rule(['POST'], `${P}/(jobs/[^/]+(/retry)?|pipeline-suggest|deliverables/[^/]+/publish)`, 'jobs:run'),
+    // Documents: writing them is saving a version or moving them; publishing one (choosing what
+    // the exports deliver, or whether they deliver it at all) is apart. A PATCH that changes
+    // `exportable` needs docs:publish too (see routes/document).
+    rule('read', `${P}/documents(/.+)?`, 'docs:read'),
+    rule(['POST'], `${P}/documents/[^/]+/publish`, 'docs:publish'),
+    rule('write', `${P}/documents(/[^/]+(/sections)?)?`, 'docs:write'),
+    rule('delete', `${P}/documents/[^/]+`, 'docs:delete'),
 ];
 /** The rule for a request made with an API token, or undefined: not reachable with one. */
 function routeRuleFor(method, path) {

package/dist/services/auth.d.ts

@@ -8,5 +8,7 @@ export declare const scopeReachesProject: (scope: Auth["apiToken"], projectId: s
 export declare const tokenAllows: (scope: Auth["apiToken"], permission: ApiTokenPermission) => boolean;
 /** Whether an app (an OAuth request) may make this request's method on its path. */
 export declare const oauthMayDelete: (method: string, path: string) => boolean;
+/** Whether a request publishes a document (its version, or whether it's exported). */
+export declare const publishesDocument: (method: string, path: string, body: unknown) => boolean;
 export declare const enforceApiTokenScope: (req: Request, res: Response, next: NextFunction) => void | Response<any, Record<string, any>>;
 export declare const authenticate: (config: StreamByConfig) => (req: Request, res: Response, next: NextFunction) => Promise<void | Response<any, Record<string, any>>>;

package/dist/services/auth.js

@@ -1,6 +1,6 @@
 "use strict";
 Object.defineProperty(exports, "__esModule", { value: true });
-exports.authenticate = exports.enforceApiTokenScope = exports.oauthMayDelete = exports.tokenAllows = exports.scopeReachesProject = exports.isPublicRoute = void 0;
+exports.authenticate = exports.enforceApiTokenScope = exports.publishesDocument = exports.oauthMayDelete = exports.tokenAllows = exports.scopeReachesProject = exports.isPublicRoute = void 0;
 const auth_1 = require("../utils/auth");
 const apiTokenPermissions_1 = require("./apiTokenPermissions");
 const oauthTokens_1 = require("./oauthTokens");
@@ -46,6 +46,19 @@ const OAUTH_DELETE_ALLOWED = new RegExp(`^/projects/[^/]+/deliverables/[^/]+/?$`
 /** Whether an app (an OAuth request) may make this request's method on its path. */
 const oauthMayDelete = (method, path) => method.toUpperCase() !== 'DELETE' || OAUTH_DELETE_ALLOWED.test(path);
 exports.oauthMayDelete = oauthMayDelete;
+// Nor does an app publish: what it edits stays a draft until someone publishes it. Publishing
+// is choosing a document's version for its exports (POST .../publish) or whether they deliver
+// it (`exportable`). Refused whatever its scopes grant, as deleting is.
+const DOCUMENT_PUBLISH = /^\/projects\/[^/]+\/documents\/[^/]+\/publish\/?$/;
+const DOCUMENT = /^\/projects\/[^/]+\/documents\/[^/]+\/?$/;
+/** Whether a request publishes a document (its version, or whether it's exported). */
+const publishesDocument = (method, path, body) => {
+    const upper = method.toUpperCase();
+    if (upper === 'POST' && DOCUMENT_PUBLISH.test(path))
+        return true;
+    return upper === 'PATCH' && DOCUMENT.test(path) && typeof body === 'object' && body !== null && 'exportable' in body;
+};
+exports.publishesDocument = publishesDocument;
 const enforceApiTokenScope = (req, res, next) => {
     const auth = req.auth;
     const scope = auth?.apiToken;
@@ -57,6 +70,9 @@ const enforceApiTokenScope = (req, res, next) => {
     if (auth?.oauth && !(0, exports.oauthMayDelete)(req.method, req.path)) {
         return res.status(403).json({ error: 'An app connected with OAuth can\'t delete' });
     }
+    if (auth?.oauth && (0, exports.publishesDocument)(req.method, req.path, req.body)) {
+        return res.status(403).json({ error: 'An app connected with OAuth can\'t publish: what it edits stays a draft' });
+    }
     const rule = (0, apiTokenPermissions_1.routeRuleFor)(req.method, req.path);
     if (!rule || (rule.userTokenOnly && scope.kind !== 'user')) {
         return res.status(403).json({ error: 'Not available with an API token' });

package/dist/services/documents.d.ts

@@ -0,0 +1,66 @@
+import { Auth, DocumentAuthor, DocumentVersion, ProjectDocument, StreamByConfig } from '../types';
+/** A document as the tree lists it: all but its content. */
+export type DocumentSummary = Omit<ProjectDocument, 'content'>;
+/** A version as the history lists it: all but its content. */
+export type VersionSummary = Omit<DocumentVersion, 'content'>;
+type Structure = Partial<Pick<ProjectDocument, 'parentId' | 'position' | 'exportable' | 'publishedVersion'>>;
+export interface DocumentStore {
+    list(projectId: string): Promise<DocumentSummary[]>;
+    get(projectId: string, id: string): Promise<ProjectDocument | null>;
+    /** A new document, and its first version. */
+    create(document: ProjectDocument): Promise<ProjectDocument>;
+    /**
+     * A new version, only if the document is still at `baseVersion`: null if it isn't (someone
+     * saved in between) or there's no such document.
+     */
+    save(projectId: string, id: string, baseVersion: number, change: {
+        title: string;
+        content: string;
+    }, author: DocumentAuthor, at: Date): Promise<ProjectDocument | null>;
+    /** Where it is in the tree, and what's published of it: not a version. */
+    update(projectId: string, id: string, patch: Structure): Promise<ProjectDocument | null>;
+    /** Its versions, the latest first. */
+    versions(projectId: string, id: string): Promise<VersionSummary[]>;
+    version(projectId: string, id: string, version: number): Promise<DocumentVersion | null>;
+    /** Documents and their versions. */
+    remove(projectId: string, ids: string[]): Promise<void>;
+    /** A deleted project's documents. */
+    deleteProject(projectId: string): Promise<void>;
+}
+/** The router's main database, where documents are kept. */
+export declare const setDocumentStoreDatabase: (config: StreamByConfig) => void;
+export declare function documentStore(): DocumentStore;
+/** Who's saving: the user, through the UI, an API token, or an app acting for them. */
+export declare function authorOf(auth: Auth): DocumentAuthor;
+/** A document and everything inside it (a folder's documents, at any depth). */
+export declare function withDescendants(documents: DocumentSummary[], id: string): string[];
+export type SectionResult = {
+    content: string;
+} | {
+    error: 'not-found' | 'ambiguous';
+};
+/**
+ * The markdown with what's under `heading` replaced by `body`, the rest untouched. `heading` is
+ * its title, with or without its #s ("## Architecture", "Architecture"): with them, only a
+ * heading of that level matches.
+ */
+export declare function replaceSection(markdown: string, heading: string, body: string): SectionResult;
+/** A document as an export delivers it: its published version, never a draft. */
+export interface PublishedDocument {
+    id: string;
+    title: string;
+    parentId: string | null;
+    position: number;
+    version: number;
+    content: string;
+    publishedAt: Date;
+}
+/**
+ * The project's exportable documents that have a published version (in one folder, at any
+ * depth, or only those given), as that version was saved.
+ */
+export declare function publishedDocuments(projectId: string, scope?: {
+    folderId?: string;
+    documentIds?: string[];
+}): Promise<PublishedDocument[]>;
+export {};

package/dist/services/documents.js

@@ -0,0 +1,347 @@
+"use strict";
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.setDocumentStoreDatabase = void 0;
+exports.documentStore = documentStore;
+exports.authorOf = authorOf;
+exports.withDescendants = withDescendants;
+exports.replaceSection = replaceSection;
+exports.publishedDocuments = publishedDocuments;
+const connectionManager_1 = require("../adapters/database/connectionManager");
+const withoutContent = ({ content: _content, ...rest }) => rest;
+const byPlace = (a, b) => a.position - b.position || new Date(a.createdAt).getTime() - new Date(b.createdAt).getTime();
+// ── PostgreSQL ──
+const sqlStore = (pool) => {
+    const DOCS = 'streamby.documents';
+    const VERSIONS = 'streamby.document_versions';
+    const SUMMARY = `id, "projectId", kind, title, "parentId", position, version, exportable, "publishedVersion", "createdAt", "createdBy", "updatedAt", "updatedBy"`;
+    const insertVersion = (client, v) => client.query(`INSERT INTO ${VERSIONS} ("documentId", "projectId", version, title, content, author, "createdAt") VALUES ($1, $2, $3, $4, $5, $6, $7)`, [v.documentId, v.projectId, v.version, v.title, v.content, JSON.stringify(v.author), v.createdAt]);
+    // A save changes two tables: both, or neither.
+    const transaction = async (work) => {
+        const client = await pool.connect();
+        try {
+            await client.query('BEGIN');
+            const result = await work(client);
+            await client.query(result === null ? 'ROLLBACK' : 'COMMIT');
+            return result;
+        }
+        catch (err) {
+            await client.query('ROLLBACK').catch(() => { });
+            throw err;
+        }
+        finally {
+            client.release();
+        }
+    };
+    return {
+        async list(projectId) {
+            const { rows } = await pool.query(`SELECT ${SUMMARY} FROM ${DOCS} WHERE "projectId" = $1 ORDER BY position, "createdAt"`, [projectId]);
+            return rows;
+        },
+        async get(projectId, id) {
+            const { rows } = await pool.query(`SELECT * FROM ${DOCS} WHERE "projectId" = $1 AND id = $2`, [projectId, id]);
+            return rows[0] ?? null;
+        },
+        async create(d) {
+            return (await transaction(async (client) => {
+                const { rows } = await client.query(`INSERT INTO ${DOCS} (id, "projectId", kind, title, "parentId", position, content, version, exportable, "publishedVersion", "createdAt", "createdBy", "updatedAt", "updatedBy")
+           VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14) RETURNING *`, [d.id, d.projectId, d.kind, d.title, d.parentId, d.position, d.content, d.version, d.exportable, d.publishedVersion,
+                    d.createdAt, JSON.stringify(d.createdBy), d.updatedAt, JSON.stringify(d.updatedBy)]);
+                await insertVersion(client, { documentId: d.id, projectId: d.projectId, version: d.version, title: d.title, content: d.content, author: d.createdBy, createdAt: d.createdAt });
+                return rows[0];
+            }));
+        },
+        async save(projectId, id, baseVersion, change, author, at) {
+            return transaction(async (client) => {
+                const { rows } = await client.query(`UPDATE ${DOCS} SET title = $4, content = $5, version = version + 1, "updatedAt" = $6, "updatedBy" = $7
+            WHERE "projectId" = $1 AND id = $2 AND version = $3 RETURNING *`, [projectId, id, baseVersion, change.title, change.content, at, JSON.stringify(author)]);
+                const saved = rows[0];
+                if (!saved)
+                    return null;
+                await insertVersion(client, { documentId: id, projectId, version: saved.version, title: saved.title, content: saved.content, author, createdAt: at });
+                return saved;
+            });
+        },
+        async update(projectId, id, patch) {
+            const entries = Object.entries(patch).filter(([, value]) => value !== undefined);
+            if (!entries.length)
+                return this.get(projectId, id);
+            const sets = entries.map(([key], i) => `"${key}" = $${i + 3}`).join(', ');
+            const { rows } = await pool.query(`UPDATE ${DOCS} SET ${sets} WHERE "projectId" = $1 AND id = $2 RETURNING *`, [projectId, id, ...entries.map(([, value]) => value)]);
+            return rows[0] ?? null;
+        },
+        async versions(projectId, id) {
+            const { rows } = await pool.query(`SELECT "documentId", "projectId", version, title, author, "createdAt" FROM ${VERSIONS}
+          WHERE "projectId" = $1 AND "documentId" = $2 ORDER BY version DESC`, [projectId, id]);
+            return rows;
+        },
+        async version(projectId, id, version) {
+            const { rows } = await pool.query(`SELECT * FROM ${VERSIONS} WHERE "projectId" = $1 AND "documentId" = $2 AND version = $3`, [projectId, id, version]);
+            return rows[0] ?? null;
+        },
+        async remove(projectId, ids) {
+            if (!ids.length)
+                return;
+            await transaction(async (client) => {
+                await client.query(`DELETE FROM ${VERSIONS} WHERE "projectId" = $1 AND "documentId" = ANY($2)`, [projectId, ids]);
+                await client.query(`DELETE FROM ${DOCS} WHERE "projectId" = $1 AND id = ANY($2)`, [projectId, ids]);
+                return true;
+            });
+        },
+        async deleteProject(projectId) {
+            await pool.query(`DELETE FROM ${VERSIONS} WHERE "projectId" = $1`, [projectId]);
+            await pool.query(`DELETE FROM ${DOCS} WHERE "projectId" = $1`, [projectId]);
+        },
+    };
+};
+// ── MongoDB ──
+// A save writes the document and then its version: in a transaction, both or neither, when
+// the server supports them (a replica set or a sharded cluster, as Atlas is). A standalone
+// server doesn't: there they're two writes, and if the version's failed after the document's,
+// the document would be at a version its history lacks.
+let mongoTransactions;
+const noTransactions = (err) => {
+    const { code, message } = (err ?? {});
+    return code === 20 || /Transaction numbers are only allowed|replica set/i.test(message ?? '');
+};
+const mongoStore = (client) => {
+    const docs = () => client.db().collection('documents');
+    const versions = () => client.db().collection('document_versions');
+    const strip = (doc) => {
+        if (!doc)
+            return null;
+        const { _id, ...rest } = doc;
+        return rest;
+    };
+    const together = async (work) => {
+        if (mongoTransactions !== false && typeof client.startSession === 'function') {
+            const session = client.startSession();
+            try {
+                let result;
+                await session.withTransaction(async () => { result = await work(session); });
+                mongoTransactions = true;
+                return result;
+            }
+            catch (err) {
+                if (!noTransactions(err))
+                    throw err;
+                mongoTransactions = false;
+            }
+            finally {
+                await session.endSession();
+            }
+        }
+        return work();
+    };
+    return {
+        async list(projectId) {
+            const found = await docs().find({ projectId }, { projection: { content: 0 } }).toArray();
+            return found.map(d => strip(d)).sort(byPlace);
+        },
+        async get(projectId, id) {
+            return strip(await docs().findOne({ projectId, id }));
+        },
+        async create(d) {
+            return together(async (session) => {
+                await docs().insertOne({ ...d }, { session });
+                await versions().insertOne({ documentId: d.id, projectId: d.projectId, version: d.version, title: d.title, content: d.content, author: d.createdBy, createdAt: d.createdAt }, { session });
+                return d;
+            });
+        },
+        async save(projectId, id, baseVersion, change, author, at) {
+            return together(async (session) => {
+                // Conditional on the version: of two saves made on the same one, only the first goes through.
+                const saved = strip(await docs().findOneAndUpdate({ projectId, id, version: baseVersion }, { $set: { ...change, version: baseVersion + 1, updatedAt: at, updatedBy: author } }, { returnDocument: 'after', session }));
+                if (!saved)
+                    return null;
+                await versions().insertOne({ documentId: id, projectId, version: saved.version, ...change, author, createdAt: at }, { session });
+                return saved;
+            });
+        },
+        async update(projectId, id, patch) {
+            const $set = Object.fromEntries(Object.entries(patch).filter(([, value]) => value !== undefined));
+            if (!Object.keys($set).length)
+                return this.get(projectId, id);
+            return strip(await docs().findOneAndUpdate({ projectId, id }, { $set }, { returnDocument: 'after' }));
+        },
+        async versions(projectId, id) {
+            const found = await versions().find({ projectId, documentId: id }, { projection: { content: 0 } }).toArray();
+            return found.map(v => strip(v)).sort((a, b) => b.version - a.version);
+        },
+        async version(projectId, id, version) {
+            return strip(await versions().findOne({ projectId, documentId: id, version }));
+        },
+        async remove(projectId, ids) {
+            if (!ids.length)
+                return;
+            await together(async (session) => {
+                await versions().deleteMany({ projectId, documentId: { $in: ids } }, { session });
+                await docs().deleteMany({ projectId, id: { $in: ids } }, { session });
+            });
+        },
+        async deleteProject(projectId) {
+            await versions().deleteMany({ projectId });
+            await docs().deleteMany({ projectId });
+        },
+    };
+};
+// ── In memory (no main database) ──
+const memoryStore = () => {
+    const docs = new Map();
+    const versions = [];
+    const own = (projectId, id) => {
+        const d = docs.get(id);
+        return d && d.projectId === projectId ? d : undefined;
+    };
+    return {
+        async list(projectId) {
+            return [...docs.values()].filter(d => d.projectId === projectId).map(withoutContent).sort(byPlace);
+        },
+        async get(projectId, id) {
+            const d = own(projectId, id);
+            return d ? { ...d } : null;
+        },
+        async create(d) {
+            docs.set(d.id, { ...d });
+            versions.push({ documentId: d.id, projectId: d.projectId, version: d.version, title: d.title, content: d.content, author: d.createdBy, createdAt: d.createdAt });
+            return { ...d };
+        },
+        async save(projectId, id, baseVersion, change, author, at) {
+            const d = own(projectId, id);
+            if (!d || d.version !== baseVersion)
+                return null;
+            Object.assign(d, change, { version: baseVersion + 1, updatedAt: at, updatedBy: author });
+            versions.push({ documentId: id, projectId, version: d.version, ...change, author, createdAt: at });
+            return { ...d };
+        },
+        async update(projectId, id, patch) {
+            const d = own(projectId, id);
+            if (!d)
+                return null;
+            Object.assign(d, Object.fromEntries(Object.entries(patch).filter(([, value]) => value !== undefined)));
+            return { ...d };
+        },
+        async versions(projectId, id) {
+            return versions.filter(v => v.projectId === projectId && v.documentId === id).map(withoutContent).sort((a, b) => b.version - a.version);
+        },
+        async version(projectId, id, version) {
+            const v = versions.find(x => x.projectId === projectId && x.documentId === id && x.version === version);
+            return v ? { ...v } : null;
+        },
+        async remove(projectId, ids) {
+            for (const id of ids)
+                if (own(projectId, id))
+                    docs.delete(id);
+            for (let i = versions.length - 1; i >= 0; i--)
+                if (versions[i].projectId === projectId && ids.includes(versions[i].documentId))
+                    versions.splice(i, 1);
+        },
+        async deleteProject(projectId) {
+            for (const [id, d] of docs)
+                if (d.projectId === projectId)
+                    docs.delete(id);
+            for (let i = versions.length - 1; i >= 0; i--)
+                if (versions[i].projectId === projectId)
+                    versions.splice(i, 1);
+        },
+    };
+};
+let mainDbId;
+let memory;
+/** The router's main database, where documents are kept. */
+const setDocumentStoreDatabase = (config) => {
+    mainDbId = config.databases?.find(db => db.main)?.id;
+};
+exports.setDocumentStoreDatabase = setDocumentStoreDatabase;
+function documentStore() {
+    if (mainDbId && (0, connectionManager_1.getConnectedIds)().includes(mainDbId)) {
+        const entry = (0, connectionManager_1.getConnection)(mainDbId);
+        return entry.type === 'sql' ? sqlStore(entry.client) : mongoStore(entry.client);
+    }
+    return (memory ?? (memory = memoryStore()));
+}
+// ── Who saved it ──
+/** Who's saving: the user, through the UI, an API token, or an app acting for them. */
+function authorOf(auth) {
+    if (auth.oauth)
+        return { userId: auth.userId, username: auth.username, via: 'oauth', clientId: auth.oauth.clientId, clientName: auth.oauth.clientName };
+    if (auth.apiToken)
+        return { userId: auth.userId, username: auth.username, via: 'apiToken', tokenId: auth.apiToken.tokenId };
+    return { userId: auth.userId, username: auth.username, via: 'session' };
+}
+// ── The tree ──
+/** A document and everything inside it (a folder's documents, at any depth). */
+function withDescendants(documents, id) {
+    const ids = [id];
+    for (let i = 0; i < ids.length; i++) {
+        for (const d of documents)
+            if (d.parentId === ids[i])
+                ids.push(d.id);
+    }
+    return ids;
+}
+// ── Sections ──
+// A section is what's under a markdown heading ("## Architecture"), up to the next heading of
+// the same level or above. Headings inside a fenced code block aren't headings.
+const HEADING = /^(#{1,6})[ \t]+(.*?)[ \t]*#*[ \t]*$/;
+const FENCE = /^[ \t]*(```|~~~)/;
+const headingOf = (text) => {
+    const match = text.trim().match(/^(#{1,6})[ \t]+(.*?)[ \t]*#*$/);
+    return match ? { level: match[1].length, title: match[2].trim() } : { level: null, title: text.trim() };
+};
+/**
+ * The markdown with what's under `heading` replaced by `body`, the rest untouched. `heading` is
+ * its title, with or without its #s ("## Architecture", "Architecture"): with them, only a
+ * heading of that level matches.
+ */
+function replaceSection(markdown, heading, body) {
+    const wanted = headingOf(heading);
+    const lines = markdown.split('\n');
+    const headings = [];
+    let fenced = false;
+    lines.forEach((line, i) => {
+        if (FENCE.test(line))
+            fenced = !fenced;
+        const match = !fenced && line.match(HEADING);
+        if (match)
+            headings.push({ line: i, level: match[1].length, title: match[2].trim() });
+    });
+    const matches = headings.filter(h => h.title === wanted.title && (wanted.level === null || h.level === wanted.level));
+    if (!matches.length)
+        return { error: 'not-found' };
+    if (matches.length > 1)
+        return { error: 'ambiguous' };
+    const [target] = matches;
+    const next = headings.find(h => h.line > target.line && h.level <= target.level);
+    const end = next ? next.line : lines.length;
+    const text = body.replace(/^(\s*\n)+/, '').replace(/\s+$/, '');
+    const replaced = [
+        ...lines.slice(0, target.line + 1),
+        ...(text ? ['', ...text.split('\n')] : []),
+        ...(next ? [''] : []),
+        ...lines.slice(end),
+    ];
+    // A document's last line ends it, as it did.
+    if (!next && markdown.endsWith('\n'))
+        replaced.push('');
+    return { content: replaced.join('\n') };
+}
+/**
+ * The project's exportable documents that have a published version (in one folder, at any
+ * depth, or only those given), as that version was saved.
+ */
+async function publishedDocuments(projectId, scope = {}) {
+    const store = documentStore();
+    const documents = await store.list(projectId);
+    const within = scope.folderId ? new Set(withDescendants(documents, scope.folderId)) : null;
+    const chosen = documents.filter(d => d.kind === 'document' && d.exportable && d.publishedVersion !== null
+        && (!within || within.has(d.id))
+        && (!scope.documentIds?.length || scope.documentIds.includes(d.id)));
+    const published = await Promise.all(chosen.map(async (d) => {
+        const version = await store.version(projectId, d.id, d.publishedVersion);
+        return version && {
+            id: d.id, title: version.title, parentId: d.parentId, position: d.position,
+            version: version.version, content: version.content, publishedAt: version.createdAt,
+        };
+    }));
+    return published.filter((d) => !!d);
+}

package/dist/services/executor.js

@@ -8,6 +8,7 @@ const connectionResolver_1 = require("./connectionResolver");
 const userIntegration_1 = require("./userIntegration");
 const oauthClient_1 = require("./oauthClient");
 const jobQueue_1 = require("./jobQueue");
+const documents_1 = require("./documents");
 const mediaProcessor_1 = require("./mediaProcessor");
 const vfxProcessor_1 = require("./vfxProcessor");
 const aiProcessor_1 = require("./aiProcessor");
@@ -143,6 +144,13 @@ async function executeExport(nodeSchema, project, config, { public: isPublic = t
                 await (0, connectionResolver_1.releaseDbClient)(resolved);
             }
         }
+        else if (node.type === 'documentsNode') {
+            // The project's own documents, as published: an edit is a draft, delivered only once a
+            // version of it is published. Its own data, so public exports read it too.
+            const documentIds = Array.isArray(node.data?.documentIds) ? node.data.documentIds : undefined;
+            const folderId = node.data?.folderId || undefined;
+            dataResults.push(await (0, documents_1.publishedDocuments)(project._id?.toString() ?? project.id, { folderId, documentIds }));
+        }
         else if (node.type === 'connectionNode') {
             const connection = project.connections?.find((c) => c.id === node.data?.connectionId);
             if (!connection)

package/dist/services/oauthTokens.js

@@ -16,12 +16,12 @@ const crypto_1 = __importDefault(require("crypto"));
 exports.OAUTH_SCOPES = ['streamby:read', 'streamby:write'];
 const READ_PERMISSIONS = [
     'project:read', 'members:read', 'assets:read', 'production:read', 'workflow:read',
-    'exports:read', 'rights:read', 'jobs:read',
+    'exports:read', 'rights:read', 'jobs:read', 'docs:read',
 ];
 // Making and changing, as the consent screen says: never deleting, nor managing members.
 const WRITE_PERMISSIONS = [
     'project:update', 'assets:write', 'production:write', 'workflow:write', 'exports:write',
-    'rights:write', 'jobs:run', 'projects:create',
+    'rights:write', 'jobs:run', 'projects:create', 'docs:write',
 ];
 /** The permissions each scope grants. */
 exports.OAUTH_SCOPE_PERMISSIONS = {

package/dist/types/index.d.ts

@@ -529,3 +529,43 @@ export interface AssetVersionRecord {
     changeNote?: string;
     createdAt: Date;
 }
+/**
+ * Who saved a document's version: a user in the UI, an API token, or an app the user let act
+ * for them (OAuth, e.g. Claude in claude.ai).
+ */
+export interface DocumentAuthor {
+    userId: string;
+    username: string;
+    via: 'session' | 'apiToken' | 'oauth';
+    tokenId?: string;
+    clientId?: string;
+    clientName?: string;
+}
+export type DocumentKind = 'document' | 'folder';
+/** A project's document as it is now (a folder only groups others: it has no content). */
+export interface ProjectDocument {
+    id: string;
+    projectId: string;
+    kind: DocumentKind;
+    title: string;
+    parentId: string | null;
+    position: number;
+    content: string;
+    version: number;
+    exportable: boolean;
+    publishedVersion: number | null;
+    createdAt: Date;
+    createdBy: DocumentAuthor;
+    updatedAt: Date;
+    updatedBy: DocumentAuthor;
+}
+/** One save of a document: its title and content then, who saved it and when. */
+export interface DocumentVersion {
+    documentId: string;
+    projectId: string;
+    version: number;
+    title: string;
+    content: string;
+    author: DocumentAuthor;
+    createdAt: Date;
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "streamby-core",
-  "version": "0.41.0",
+  "version": "0.42.0",
   "description": "StreamBy middleware framework for media storage management",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",