@mengine/medeo-client 2.0.1 → 2.1.1-dsl.0

package/README.md

@@ -1,35 +1,150 @@
-# @mengine/medeo-client
+# `@mengine/medeo-client`
+
+The default client edits one Medeo DSL document. `DslEditor` writes entities, relations, native Text/Cursors and the containment Tree through one audited transaction. VideoDraft is a read view, never a second writable document.
+
+## Open, edit and synchronize
+
+```ts
+import { MengineDocSession, MengineHttpClient } from '@mengine/medeo-client';
 
-Medeo domain client for the `mengine` document path.
+const session = new MengineDocSession({ docId, client: new MengineHttpClient({ docId, httpOrigin, authToken }) });
+await session.start(); // The server must have initialized the DSL document.
+const [clipId] = session.editor.addClips({
+  trackId: 'main_track',
+  clips: [{ media: { kind: 'video', system: 'memota', key: mediaId, durationMs: 5000 } }],
+});
+session.editor.setClipVolume(clipId, -3, { actor: { user_id: userId, role: 'user' } });
+await session.waitForServerAck(); // Local completion is separate from durable confirmation.
+const { content, diagnostics } = session.projectVideoDraft();
+session.destroy();
+```
 
-This package owns the shared `VideoDocument` contract, semantic editing layer, and runtime-neutral HTTP/SSE client used by medeo-web, agent-harness, and server-side validation code. UI adapters such as `DraftMutationDriver` stay in product repositories.
+`start()`, `snapshot()` and subscription events expose `MedeoDslSnapshot`. `session.dsl` exposes controlled `transact`/`applyMutations`; ordinary business callers use `session.editor`. Rich text uses the native text adapter. `readonlyMode` blocks editing and Undo/Redo. Closing a session also invalidates captured editor/document references.
 
-`MengineDocSession.start()` waits for DocManager to load a local snapshot, or for the first remote sync if no local snapshot exists, then validates the content. A cached document can open before SSE connects; a cold start waits for SSE and the existing sync cycle. Session does not prefetch or import an initial snapshot itself.
+For explicit synchronization, use `ManualSyncDoc.open({client, peerId})`, then `pull()`, `editor`, `push()` and `dispose()`. Both lifecycles use the same binary snapshot/update and version-coverage contracts. Neither silently upgrades old documents or overwrites an incompatible cache.
 
-`getState().loaded` means the first local load completed. Once loaded, `loadedFromLocal` reports whether that load imported a snapshot, including a valid empty document. It stays unchanged by later edits and remote updates. Document readiness does not depend on metadata or nonempty tracks.
+## Editing and conversion
 
-## Session undo/redo
+- Track: create, reorder, hide and delete.
+- Clip: batch insert with registered sources or full media facts; move, trim, speed, volume, replace and delete.
+- Placement: sequential, absolute or anchored. Reordering and replacement require an explicit associated-content policy.
+- Voiceover: `applyVoiceover` consumes materialized recording/script/caption results in one commit. Upload, ASR and TTS belong to the host.
+- BGM: `setBgm` requires full resource facts and explicitly fills the Timeline; trim/speed are not enabled.
+- Caption: independent native Text or AudioScript Cursor reference; style and source switching.
 
-After `await session.start()`, `session.undo(options?)` and `session.redo(options?)` synchronously apply local history and return whether they produced a document update. `options` accepts the same actor/intent metadata as semantic edits. Read the resulting document through `session.snapshot()`; use `waitForServerAck()` only when durable confirmation is required.
+`resolveDslLayout`, `toVideoDraft`, `fromVideoDraft`, `MedeoDsl` and the model types are exported from the root and `/dsl`. Projection returns diagnostics; unrepresentable legacy output throws `DslProjectionError`. `MengineHttpClient.fetchDraft()` / `fetchDraftAt(seq)` read the server's content, diagnostics and matching version.
 
-`getUndoState()` returns the current `{ canUndo, canRedo }` immediately. `subscribeUndoState(callback)` immediately provides that state, then coalesces subsequent stack changes in a microtask after mutations settle. Several synchronous commands may produce only one availability notification; use the getter when reading directly after a command. Teardown publishes empty availability synchronously and cancels pending notifications. Dispose the returned subscription with the host. Loading and closed sessions report an empty history; history commands require a ready session.
+## The DSL document
 
-History belongs to one document/session peer. The loaded baseline and remote imports are not recorded. Each semantic commit is a separate step (up to 100); a new local edit invalidates redo. Reconnecting preserves history, while reopening starts empty. Empty or ineffective history actions return false. Never reuse a peer ID across concurrent sessions.
+The live document lives here: `src/dsl/document` holds the `MedeoDsl` handle, transactions, draft guards and native reads; `src/dsl/projection` holds layout solving, VideoDraft conversion and the generated IDL types. Definitions, the Loro schema and change constraints come from the private `@mengine/medeo-dsl` and are re-exported by name, so the published API does not grow when the DSL does. Transactions ask the DSL's constraint functions; only what needs a live Loro document — caption ranges in the current body, Tree identities, Mirror's K03 limit — is checked here. See [RFC 04](../../docs/projects/medeo-dsl/rfc/04-entity-relation-to-loro-mapping.md).
 
-Use `beginUndoGroup()` / `endUndoGroup()` to combine existing semantic operations into one undo step, or to combine repeated commits during one gesture. Always end in `finally`. Every constituent commit is immediately visible locally and enters synchronization independently; if a later operation fails, earlier commits remain applied. Conflicting remote imports may split/end a group early, and unrelated local edits in the same session would join an open group.
+### Using the editor
 
-Mirror's draft callback and commit are synchronous. SemanticEditor methods return promises, but their document mutations currently finish synchronously before the promise resolves. Await external asset generation before opening a group, then compose the local semantic operations. Keeping a group open across asynchronous work does not provide transaction isolation, deferred publication, or rollback. If several changes must be atomic, implement one semantic operation with one synchronous draft callback. A gesture that uses only local previews and commits once on release already needs no group.
+```ts
+import { DslEditor, MedeoDsl } from '@mengine/medeo-client/dsl';
 
-Flush pending local preview/debounce work before undo/redo. Mutating again from a document-update observer during a mutation is rejected; schedule the next command after the current one returns.
+const dsl = MedeoDsl.create();
+const editor = new DslEditor(dsl);
+const trackId = editor.createTrack('video_clip');
+const sourceId = editor.registerMedia({
+  kind: 'video',
+  system: 'memota',
+  key: 'video-1',
+  durationMs: 5000,
+});
+const [first, second] = editor.addClips({
+  trackId,
+  clips: [
+    { sourceId, outMs: 3000 },
+    { sourceId, inMs: 1000, outMs: 3000 },
+  ],
+});
+editor.moveClip(second, trackId, 0);
+editor.setClipVolume(first, -3, { intent: 'Lower the opening clip', actor: { user_id: 'user-1', role: 'user' } });
+const snapshot = dsl.doc.export({ mode: 'snapshot' });
+dsl.dispose();
+```
 
-The session assembles SemanticEditor with MirrorVideoDocumentAdapter directly. DocumentUndoManager independently observes the same LoroDoc and owns undo/redo, grouping and audit metadata. Both write paths share a synchronous DocumentMutationGuard; the session closes it before teardown callbacks, and each component releases its own resources. Standalone adapters create their own guard by default and reject use after disposal while leaving the borrowed LoroDoc usable.
+`MedeoDsl.create()` owns its fresh LoroDoc and frees it on dispose. `new MedeoDsl(doc)` borrows an existing doc and only releases subscriptions. Mirror registers the schema's empty root handles on opening; this adds no CRDT operations or audit entries. The caller uses native binary snapshots/updates for persistence and sync; Mirror JSON is not a backup of text marks or Cursor identities. Opening a legacy content document is rejected.
 
-Undo/redo append ordinary synchronized updates and retain audit metadata (`Undo`/`Redo`, actor, and `payload.source` identifying an original local change). Existing revision bindings remain immutable. Redo reuses already materialized assets without repeating generation, billing, or attachment requests. Network failures do not automatically reverse history.
+### Interface and invariants
 
-Loro history uses new writes: undo on a shared LWW field restores its value before the local edit, potentially overwriting a later remote write to the same field; redo restores the value visible before undo. Unrelated remote fields remain intact. Undoing a local insertion removes the object and makes dependent remote edits invisible until redo restores the object. Caption creation includes its style container, even when empty, so a later style edit only changes fields and undo preserves another peer's independently edited fields. Initial document seeding preserves supplied styles and creates empty maps for captions without styles. No write-time repair or migration of older CRDT documents is provided.
+- `getSnapshot()` returns a frozen, read-only `MedeoDslSnapshot`: entity and relation tables, effective containment, rich-text runs, resolved Caption text and diagnostics. It does not compute layout or VideoDraft. Reads reuse the same snapshot until a DSL content event invalidates it; no-op Actions and repeated imports retain it. Reading never commits changes.
+- `subscribe(listener)` includes text-only and marks-only changes. The returned function unsubscribes; dispose is idempotent. As with other synchronous document observers, listeners should not throw or initiate nested writes.
+- `applyMutations(mutations, audit)` is the preferred interface for typed entity, relation, Tree, Text and Caption-selection commands. It opens one `transact`; `tx.applyMutations` inside an existing callback uses the same draft and commit.
+- `transact(edit, audit)` exposes a guarded `tx.draft` for ordinary fields and Tree, `tx.editText(scriptId, edits, expectedBodyId?)`, `tx.editCaptionText(captionId, edits, expectedBodyId?)`, and `tx.setCaptionContent({captionId, audioScriptId, start, end})`. The synchronous callback and every draft reference expire on return. Async callbacks, nested writes, identity changes, direct overwrites of existing bodies, and same-transaction entity deletion/recreation are rejected. Catching a guard/command error inside the callback does not allow a partial commit.
+- `editText(scriptId, expectedBodyId, edits, audit)` delegates to the same transaction. Text supports insert/delete/mark/unmark/editing Delta; offsets are UTF-16 and cannot split surrogate pairs. `tx.draft` exposes the text preview after preceding edits, while existing body strings are never sent through Mirror's full-text diff. A supplied body identity must match the current Text; Actions editing an existing body retain that stale-view check.
+- Caption selections are interpreted at their position in the command sequence. Their Cursors are created after earlier native edits, and follow later edits. New selections are deferred commands, not synthetic Cursor values in `tx.draft`; do not read that relation table to retrieve a pending selection. Explicit Cursor relation writes and deletes also replay in sequence with Text edits.
+- `textRange(scriptId, start, end)` returns native Cursors for a nonempty continuous selection. Caption content stores their native bytes through the central codec. The permanent first-boundary limitation remains deferred.
 
-## Large-update resynchronization
+Every business write requires a structured `DslAudit`: `{semantic_op, payload, intent?, actor?}`. The transaction exit serializes the existing server envelope exactly once; initialization alone uses an internal unaudited path. No-op transactions add no history. External pending native operations must be committed separately first. Known preparation failures write nothing; unexpected native/runtime failures do not receive database-style rollback.
 
-`MengineDocSession` handles SSE `event: resync` by scheduling the existing version-vector sync on the current connection. The event carries no Loro bytes and does not create an edit or undo step. A hint arriving while an HTTP sync is in flight schedules another diff so changes newer than that response are still fetched. Low-level users of `readMengineEventStream` can handle the same hint with `onResync`; `MedeoHttpDocStorage` exposes `subscribeResync`.
+```ts
+const audit = {
+  semantic_op: 'createCaptionFromScript',
+  payload: { scriptId: 'script-1', captionId: 'caption-1' },
+  actor: { user_id: 'user-1', role: 'user' as const },
+};
 
-Deploy this client before enabling server-side resync notifications. Older clients ignore the event. Large HTTP pushes remain valid: the server chooses inline update or resync notification after persistence, rather than requiring client-side fragmentation. Base64 encoding uses `Uint8Array.toBase64()` where available, with a chunked fallback preserving the same padded wire representation.
+dsl.transact((tx) => {
+  tx.draft['audio-script-entities']['script-1'] = { body: 'hello' };
+  tx.draft['caption-entities']['caption-1'] = {};
+  tx.editText('script-1', [{ type: 'insert', at: 5, text: ' world' }]);
+  // Reads the prepared preview: "hello world". The native Text does not yet exist.
+  tx.setCaptionContent({ captionId: 'caption-1', audioScriptId: 'script-1', start: 0, end: 5 });
+}, audit);
+```
+
+`medeoDslLoroShape` defines the persisted containers; `MedeoDslLoroState` is directly inferred from it. `MedeoDslDraft` derives a writable view with optional absent fields and unallocated Tree IDs, accounting for Mirror's input-type limitations. `$cid`/existing TreeID are protected at runtime. `MedeoDslSnapshot` remains the separate read-only domain view, including diagnostics, rich text and resolved Cursors; it is not a schema alias or a second SSOT.
+
+Default fields stay absent. Explicit `seqMarker.rate: 1` or `volume: 0` remain explicit; resetting them deletes the field. A Clip's `seqMarker` — window, rate and timeline fill — is one atomic value: concurrent edits of the same marker keep one whole marker, while `volume` merges on its own. Editor inputs keep their flat `inMs`/`outMs`/`speed`/`fillTimeline` and are translated to the marker. Existing Caption style Maps stay present when their last field is reset. Resource locator changes require a new resource identity; optional facts can be updated independently.
+
+Ordinary entity IDs are fresh allocations. Deleting and recreating the same ordinary entity in one transaction is rejected so Mirror cannot collapse replacement into an in-place field diff; later independent recreation does not inherit old Caption references. Do not split a single business Action into multiple transactions to bypass this restriction. Enhanced collaborative Undo is not enabled. Duplicate containment references are excluded from the effective tree with diagnostics, while the original Tree remains unchanged; this is not a repair or an Undo resolution policy.
+
+`DslEditor` accepts the existing client `CommitOptions` (`intent`, `actor`). Each effective Action writes `{semantic_op, payload, intent, actor}` in one native commit; generated IDs accompany business inputs, and explicit resets use `null` in audit payloads. No-op Actions leave no audit entry. Transport can combine several Actions into one binary update without combining their audit records.
+
+First-phase Track admission is Video/Image on `video_clip`, Audio/VoiceoverRecording on `speech`, Caption on `caption`, and Audio on `bgm`. `fillTimeline: true` is explicit on a Clip input and survives moving tracks; the role does not manufacture this fact. Filling uses currently reject trim/speed edits. Image and Caption uses require an explicit window.
+
+### Implementation and verification boundary
+
+Each handle owns one long-lived Mirror. Ordinary fields and Tree read from its in-memory state; rich-text runs and Cursor resolution use the same native Text. Content events invalidate a lazily rebuilt snapshot, including marks-only events. There is no full native field scan or JSON fingerprint on every read. Rebuilding a changed snapshot still traverses the document in JavaScript and reads native Text; fine-grained read caches, UI binding/IME and large-document performance remain future work.
+
+Mirror 2.3.3 cannot safely move an existing node under a parent created in the same draft. The document transaction rejects this combination before writing. When a business Action actually needs it, implement its scoped native path with one audited commit as specified by [K03](../../docs/projects/medeo-dsl/known-unfixed.md#k03-mirror-同批新建父节点并移入已有节点); that fallback is not implemented yet.
+
+Loro 1.16.1 and loro-mirror 2.3.3 are the lockfile baseline. Mirror is pinned with a [workspace dependency patch](../../patches/README.md): its optional `setState(..., {applyNative})` hook runs after the ordinary draft reaches Loro and before one commit. The full native event updates Mirror before DSL subscribers are notified. The hook is internal to the DSL adapter; Actions never receive a native-write callback. It supports non-ephemeral Mirror instances, and must not commit/export/import or await. Native writes preserve marks and actual container identities; a plain Mirror string snapshot still does not contain rich-text marks. Tests cover snapshot/update round trips, concurrent moves and replacements, field merging, optional reset, malformed codecs, deletion/recreation, rich text, Cursor ownership, subscriptions and Action failure preflight. The conversion library is implemented; production server/editor/player entry points have not switched.
+
+Run the package test/build tasks through `vp run @mengine/medeo-client#test` and `vp run @mengine/medeo-client#build`.
+
+### Layout and VideoDraft conversion
+
+```ts
+import { fromVideoDraft, resolveDslLayout, toVideoDraft } from '@mengine/medeo-client/dsl';
+
+const { dsl, diagnostics } = fromVideoDraft(legacyContent, {
+  mediaFacts, // physical Image/Video/Audio facts supplied by the host
+  audit: { semantic_op: 'importVideoDraft', payload: { source: 'legacy' } },
+});
+try {
+  const layout = resolveDslLayout(dsl.getSnapshot());
+  const { content, diagnostics: projectionDiagnostics } = toVideoDraft(dsl.getSnapshot());
+  // The host owns business metadata and persistence; content has no revision/owner/settings.
+} finally {
+  dsl.dispose();
+}
+```
+
+`resolveDslLayout` supports multiple Tracks, sequential/absolute/anchored placement, fallback, window intersection, bounded overlap constraints and timeline fill. `toVideoDraft` directly emits the legacy content contract. Unsupported representations (multiple visual Tracks, audio trim/speed, etc.) throw `DslProjectionError`; recoverable states return diagnostics. Legacy consumer support is not implied by successful conversion.
+
+`fromVideoDraft` creates a new document, never overwrites an existing one. Its required audit is used by one content transaction after Timeline initialization. Loro may split a large commit into multiple Changes carrying the same message; the current server lists these separately ([K04](../../docs/projects/medeo-dsl/known-unfixed.md#k04-大提交的原生-change-分段与审计展示)). It restores legacy positioning (main Track sequential, aggregation/caption anchors, otherwise absolute), so original gaps are not guaranteed to survive. Media facts must state intrinsic source duration. Unsupported curves, timed caption sub-cues, orphan parts and inconsistent identities/ownership reject explicitly.
+
+Caption text is matched forward in `speech.caption_ids` order. A failed exact match preserves the text in the Caption's own optional **LoroText**, without advancing the match cursor or inventing another AudioScript. Inline text is also available through `DslEditor.createCaption({text, style?})`, `setCaptionText`, and `editCaptionText`. Switching to a reference clears the inline field in the same transaction; direct `transact` callers must clear the other source explicitly. Concurrent coexistence reads the reference, even when it later becomes invalid (no stale-text fallback).
+
+`richText` is keyed by AudioScript or inline Caption entity ID; `captionText` resolves either content source. Optional Caption Texts have ordinary identities and are created in the native hook, preserving mergeable style/font while avoiding duplicate text on mergeable-Text deletion Undo. See [RFC 02](../../docs/projects/medeo-dsl/rfc/02-videodraft-projection.md) for layout, import normalization and compatibility limits.
+
+## Legacy consumers
+
+`SemanticEditor`, `VideoDocument`, their adapters and old timeline projection have been removed. `@mengine/medeo-client/legacy` retains only the old EntityTimelineEditor, entity transport and its direct VideoDraft projection. Old entity tool/sandbox imports use this entry until their business adaptation is implemented. The obsolete timeline-journal replay rejects nonempty plans before writing. They cannot open new DSL documents. `/schemas` remains the old tool's schema contract, not the new Editor input model.
+
+`/relay` and `/testing` remain host/test-only entrypoints. The private DSL package is bundled into the client distribution.
+
+See [DSL index](../../docs/projects/medeo-dsl/README.md), [capability mapping and cutover](../../docs/projects/medeo-dsl/rfc/03-editor-and-document-cutover.md) and [known limits](../../docs/projects/medeo-dsl/known-unfixed.md). K04 audit segmentation is unchanged; one native commit can contain multiple Changes.

package/dist/base64-D-jd-dr5.js

@@ -0,0 +1,16 @@
+//#region src/client/base64.ts
+function bytesToBase64(bytes) {
+	const native = bytes.toBase64;
+	if (typeof native === "function") return native.call(bytes);
+	const chunks = [];
+	for (let offset = 0; offset < bytes.length; offset += 8192) chunks.push(String.fromCharCode(...bytes.subarray(offset, offset + 8192)));
+	return btoa(chunks.join(""));
+}
+function base64ToBytes(base64) {
+	const binary = atob(base64);
+	const bytes = new Uint8Array(binary.length);
+	for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i);
+	return bytes;
+}
+//#endregion
+export { bytesToBase64 as n, base64ToBytes as t };