@animastor/orchestration 0.1.7 → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,107 @@
+# Changelog
+
+## 0.2.0 (2026-10-27)
+
+Version-line split for `stage143a-foundation-phase1`: the branch line moves
+`0.1.6 → 0.2.0` so it can never collide with, or be shadowed by, the hotfix
+line. **No functional change** — the content is the `0.1.6` code below (D1 +
+A1 already in this branch).
+
+**The hotfix line is separate:** `0.1.7` (branch `hotfix/d1-d2-a1`) is cut
+from `0.1.4`, not from this line, and carries only D1 + the A1 version-gate
+fallback (D2 and the host side of A1 live in `backend`). `^0.1.x` therefore
+resolves within the `0.1.x` line, while this branch's backend depends on
+`^0.2.0`.
+
+## 0.1.7 (2026-10-27) — hotfix line (содержимое входит в 0.2.0)
+
+Hotfix line (branch `hotfix/d1-d2-a1`, base `78e8f1ae` = `0.1.4`): only the
+D1 and A1-gate fixes are carried in this package. It is a **separate line from
+the `0.1.6` entry of the `stage143a-foundation-phase1` branch** — this package
+is `0.1.4` + the two fixes, not a descendant of `0.1.5`/`0.1.6`.
+
+**In this package:** D1, A1-gate. **In `backend` (not here):** D2 and the host
+side of A1 (`scene-assets-repo.getAssetNullBuild` + runtime adapter + port op).
+
+### Fixed
+
+- `orchestration/orchestrator.js` (D1, stage 146) — the
+  `require('@animastor/generation').artifactNaming` binding lived INSIDE
+  `beginStage()`, but `resetScenes()` also reads it in the stale-PNG/preview
+  pre-delete branch (step 6). Any regeneration that passed `cleanPngUnitIds`
+  therefore threw `ReferenceError: artifactNaming is not defined`, and
+  `/regenerate` returned HTTP 500. The binding now sits at module scope, next
+  to the other module-level requires; `beginStage()` keeps its own
+  `dispatchEngine` lazy require. Behavior is otherwise unchanged.
+- `orchestration/orchestrator.js` (`completeStage` version gate, A1 /
+  stage 146 §5.4) — when `sceneAssets.getAsset(..., buildId)` found no row for
+  THIS build, the gate went fail-closed (DIRTY) even though the key's only row
+  was a legacy `build_id IS NULL` row: `getAsset` filters `build_id = $5`, and
+  `markReady` — the only writer able to stamp `build_id`
+  (`buildIdFromArtifactPath`) — runs BEHIND the gate. The scene then
+  re-dispatched every 5 s tick (`cache_hit` → callback → `stale` → DIRTY)
+  forever. The gate now performs exactly ONE additional read of the SAME key
+  with `build_id IS NULL` (never another build's row, never "latest any row")
+  through the PersistencePort op `sceneAssets.getAssetNullBuild`, applies the
+  SAME version rules (`scene_content_version`, plus
+  `scene_audio_config_version` for audio), and lets `markReady` stamp
+  `build_id` as before. A NULL row whose version is below the scene version
+  still fails the gate (DIRTY), so the loop converges after the next real
+  dispatch instead of spinning. PG errors stay fail-closed.
+
+### Shipped by backend, not by this package
+
+- D2 — `/api/v1/debug/runtime/*` middleware now calls `next()`
+  (`backend/src/routes/debug-routes.cjs`).
+- A1 host side — `sceneAssets.getAssetNullBuild` implementation and delegation
+  (`backend/src/storage/postgres/repositories/scene-assets-repo.js`,
+  `backend/src/storage/runtime-persistence-adapter.js`) plus the op
+  declaration in `runtime/persistence-port.js`.
+
+## 0.1.6 (2026-10-26)
+
+Stage 146 — D1 fix: `artifactNaming` hoisted to module scope.
+Stage 147 — A1 fix: version-gate sees the legacy `build_id IS NULL` row.
+
+### Fixed
+
+- `orchestration/orchestrator.js` — the `require('@animastor/generation')
+  .artifactNaming` binding lived INSIDE `beginStage()`, but
+  `resetScenes()` also reads it in the stale-PNG/preview pre-delete branch
+  (step 6). Any regeneration that passed `cleanPngUnitIds` therefore threw
+  `ReferenceError: artifactNaming is not defined`, and `/regenerate`
+  returned HTTP 500. The binding now sits at module scope, next to the
+  other module-level requires; `beginStage()` keeps its own
+  `dispatchEngine` lazy require. Behavior is otherwise unchanged.
+- `orchestration/orchestrator.js` (`completeStage` version gate, A1 /
+  stage146 §5.4) — when `sceneAssets.getAsset(..., buildId)` found no row
+  for THIS build, the gate went fail-closed (DIRTY) even though the key's
+  only row was a legacy `build_id IS NULL` row: `getAsset` filters
+  `build_id = $5`, and `markReady` — the only writer able to stamp
+  `build_id` (`buildIdFromArtifactPath`) — runs BEHIND the gate. The scene
+  then re-dispatched every 5 s tick (`cache_hit` → callback → `stale` →
+  DIRTY) forever. The gate now performs exactly ONE additional read of the
+  SAME key with `build_id IS NULL` (never another build's row, never
+  "latest any row") through the same PersistencePort op
+  `sceneAssets.getAssetNullBuild`, applies the SAME version rules
+  (`scene_content_version`, plus `scene_audio_config_version` for audio),
+  and lets `markReady` stamp `build_id` as before. A NULL row whose
+  version is below the scene version still fails the gate (DIRTY) — the
+  correct version arrives from `stampSceneVersionsForDispatch` (dispatch
+  step 5.5 stamps ALL rows of the key, incl. `build_id IS NULL`, before
+  every generation/cache_hit), so the loop converges after the next real
+  dispatch instead of spinning. PG errors stay fail-closed.
+
+### Changed (host adapter, A1)
+
+- `backend/src/storage/postgres/repositories/scene-assets-repo.js` — new
+  `getAssetNullBuild` (same key, `build_id IS NULL`, latest NULL row).
+- `backend/src/storage/runtime-persistence-adapter.js` — delegates the new
+  op; `runtime/persistence-port.js` — declares `sceneAssets.getAssetNullBuild`
+  in `OPS` (no SQL crosses the port).
+
+## 0.1.5 (2026-10-26)
+
+Stage 143a — versioned-pointer readers + atomic publication. See the
+player 0.1.2/0.1.3 entries (same stage) for the flag-gated IU-media
+contour this package complements.

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "@animastor/orchestration",
-    "version": "0.1.7",
+    "version": "0.2.0",
     "description": "Animastor Orchestration — the generation orchestration/runtime contour of the Animastor backend, physically extracted from the host (§32.30): the unified generation lifecycle owner (orchestrator facade, stage executor, scene callbacks/restoration), the dispatch engine and its lease/quota/retry managers, the reconciliation engine and counter reconciliation, the scene window/scheduler/worker-health, the runtime-result emitter/consumer, and the eight O-2..O-10 port contracts (Persistence, SceneData, PlaceholderAudio, ProgressEvents, AudioFsm, VideoFsm, HubCancel, LayerConfig). The package reaches the host (runtime config slices, Redis-domain state layer, media facades, the S-5 seam registry, the artifact root, the PW-2 workspace resolver) ONLY through the composition-root host-bindings seam — no backend/src require, no storage/book/routes/services, no Redis, no PG, no Express inside.",
     "keywords": [
         "animastor",
@@ -18,7 +18,8 @@
         "src/",
         "test/",
         "README.md",
-        "LICENSE"
+        "LICENSE",
+        "CHANGELOG.md"
     ],
     "scripts": {
         "test": "mocha --exit --require test/test-bindings.js test/*.test.js"

package/src/orchestration/scene-callbacks.js

@@ -277,9 +277,17 @@ async function handleImageCompleted(redis, bookId, chapterId, sceneId, buildId)
     // T1: возвращает { ok, retryable, reason, artifact }
     log(`IMAGE_CALLBACK: ${bookId}/${chapterId}/${sceneId}`);
 
-    const sceneImage = image.resolveCanonicalSceneImage(
+    // Stage 143a: legacy canonical scan first (unchanged behavior, also the
+    // path every host stub knows), then — flag on and scan empty — the
+    // published pointer of this scene build as the completion proof.
+    let sceneImage = image.resolveCanonicalSceneImage(
         artifactRoot(), buildId, bookId, chapterId, sceneId
     );
+    if (!sceneImage && typeof image.resolveVersionedSceneProof === 'function') {
+        sceneImage = await image.resolveVersionedSceneProof(
+            artifactRoot(), buildId, bookId, chapterId, sceneId
+        );
+    }
 
     // Per-asset check
     const assetStates = await state.getAssetStates(redis, bookId, chapterId, sceneId);
@@ -334,7 +342,28 @@ async function handleImageCompleted(redis, bookId, chapterId, sceneId, buildId)
             const stillPending = [];
             for (const uid of dirtyIds) {
                 const pngPath = path.join(buildDir, artifactNaming.sceneImageName(bookId, chapterId, sceneId, uid));
-                if (!fs.existsSync(pngPath)) {
+                if (fs.existsSync(pngPath)) continue;
+                // ── Stage 143a (phase 2a · N1) ─────────────────────
+                // A versioned-only build never writes the canonical file, so
+                // the published pointer is the completion proof for this
+                // unit. Without it the dirty marker is never cleared and the
+                // unit is force-regenerated on EVERY reconcile pass (infinite
+                // regeneration loop with the flag on).
+                // Flag OFF ⇒ the facade answers null with zero PG calls → the
+                // legacy check above decides, exactly as before.
+                // A pointer-store ERROR keeps the unit pending (never cleared
+                // on an unknown verdict) and is logged.
+                let provenByPointer = false;
+                try {
+                    if (image && typeof image.resolveIuImagePath === 'function') {
+                        const published = await image.resolveIuImagePath(
+                            artifactRoot(), buildId, bookId, chapterId, sceneId, uid);
+                        provenByPointer = !!(published && published.status === 'ok');
+                    }
+                } catch (pgErr) {
+                    warn(`[DIRTY-UNITS] pointer proof unavailable for ${uid}: ${pgErr.message} — keeping it pending`);
+                }
+                if (!provenByPointer) {
                     stillPending.push(uid);
                 }
             }
@@ -343,8 +372,31 @@ async function handleImageCompleted(redis, bookId, chapterId, sceneId, buildId)
                 log(`[DIRTY-UNITS-CLEARED] ${bookId}/${chapterId}/${sceneId}: all ${dirtyIds.length} dirty unit(s) completed, cleared`);
                 try {
                     const iuPrefix = artifactNaming.iuImagePrefix(bookId, chapterId, sceneId);
+                    const scenePrefix = artifactNaming.scenePrefix(bookId, chapterId, sceneId);
                     const allFiles = fs.existsSync(buildDir) ? fs.readdirSync(buildDir) : [];
-                    const iuFileCount = allFiles.filter(f => f.startsWith(iuPrefix) && f.endsWith('.png')).length;
+                    // ── Stage 143a (phase 2a · N2) ─────────────────
+                    // Ready units = canonical file names ∪ published pointer
+                    // ids (versioned files are invisible to this scan). The
+                    // counter is a DISPLAY source (consumers clamp it), so a
+                    // pointer-store error here only logs and leaves the
+                    // legacy count — never a readiness decision.
+                    const readyIds = new Set();
+                    for (const f of allFiles) {
+                        if (f.startsWith(iuPrefix) && f.endsWith('.png')) {
+                            readyIds.add(f.startsWith(`${scenePrefix}_`)
+                                ? f.slice(scenePrefix.length + 1, -4)
+                                : f);
+                        }
+                    }
+                    if (image && typeof image.listPublishedIuIdsForScene === 'function') {
+                        try {
+                            const publishedIds = await image.listPublishedIuIdsForScene(buildId, bookId, chapterId, sceneId);
+                            for (const id of publishedIds) readyIds.add(String(id));
+                        } catch (pgErr) {
+                            warn(`[IU-COUNTER-RESET] published ids unavailable for ${bookId}/${chapterId}/${sceneId}: ${pgErr.message} — counting canonical files only`);
+                        }
+                    }
+                    const iuFileCount = readyIds.size;
                     if (iuFileCount > 0) {
                         const progKey = `animastor:iu-progress:${bookId}:${chapterId}:${sceneId}:image`;
                         await redis.set(progKey, String(iuFileCount), 'EX', 14400);

package/src/orchestration/scene-orchestrator.js

@@ -13,10 +13,15 @@ const chunks = lazyHostBinding('media.audio.chunks');
 // orchestrator never requires gpu-dispatcher or the workflow connector.
 const provider = require('@animastor/generation').comfyuiProvider;
 const runtimeScheduler = require('../runtime/runtime-scheduler');
+// Stage 118 (M3): the shared outcome taxonomy — the dispatch engine (runtime
+// tier) and the stage executors (orchestration tier) must classify a
+// "nothing was enqueued" outcome IDENTICALLY.
+const { classifyNotSentReasons } = require('../runtime/dispatch-outcomes');
 // O-2: persistence arrives ONLY through the PersistencePort (host adapter:
 // storage/runtime-persistence-adapter) — the scene-assets repository left
 // this file. SQL lives host-side.
 const persist = require('../runtime/persistence-port').persist;
+const isPersistencePortWired = require('../runtime/persistence-port').isPersistencePortWired;
 // O-3: scene content arrives ONLY through the SceneDataPort (host adapter:
 // storage/scene-data-adapter) — the Book Model facade (../book →
 // @animastor/vbook-runtime) left this file. Aliased `sceneDataPort` because
@@ -300,8 +305,22 @@ async function executeAudioDispatch(redis, scene, loadedBook, buildId, dispatchI
         const completion = await completeStage(redis, bookId, chapterId, sceneId, 'audio', buildId, dispatchId);
         return { dispatched: false, jobs: 0, completed: completion.completed, reason: 'no_segments' };
     } else {
-        warn(`AUDIO_DISPATCH: generateSceneAudio did not send jobs for ${bookId}/${chapterId}/${sceneId}`);
-        return { dispatched: false, jobs: 0, reason: 'no_jobs_generated' };
+        // Stage 116 (D3) / Stage 118 (B3, M3): nothing was enqueued — carry
+        // the precise reason (no live worker / everything already queued /
+        // ambiguous POST) so the dispatch engine can apply a BOUNDED deferral
+        // instead of a fatal cancel.
+        //
+        // Stage 118 (B3): the merged (pure-dialogue) path and the per-segment
+        // path must produce the SAME classification — the merged branch used
+        // to leave the reason empty, so a pure-dialogue scene kept the
+        // 5-second retry loop. `deferReason` is computed by the host (audio
+        // generation) for EVERY path; the fallback below keeps the historic
+        // 'no_jobs_generated' semantics when the host could not classify it.
+        const deferReason = (result && result.reason === 'no_jobs_accepted' && result.deferReason)
+            ? result.deferReason
+            : 'no_jobs_generated';
+        warn(`AUDIO_DISPATCH: generateSceneAudio did not send jobs for ${bookId}/${chapterId}/${sceneId} (${deferReason})`);
+        return { dispatched: false, jobs: 0, reason: deferReason, retryable: true };
     }
 }
 
@@ -352,8 +371,11 @@ async function executeImageDispatch(redis, scene, loadedBook, buildId, dispatchI
 
     // T3: generateSceneIUImages возвращает количество отправленных jobs
     let jobsSent = 0;
+    // Stage 116 (D3): the result is needed AFTER the try/catch (the deferral
+    // reason) — it must be visible outside the block.
+    let genResult = null;
     try {
-        const genResult = await image.generateSceneIUImages(
+        genResult = await image.generateSceneIUImages(
             redis,
             sceneData,
             bookData,
@@ -377,6 +399,16 @@ async function executeImageDispatch(redis, scene, loadedBook, buildId, dispatchI
         return { dispatched: true, jobs: jobsSent, reason: null };
     }
 
+    // Stage 116 (D3) / Stage 118 (M3): the executor sent NOTHING. That is
+    // not a lost task and not a fatal cancellation — it is a bounded
+    // deferral. The precise reason (no live worker / everything already
+    // queued / ambiguous POST) is carried through so the deferral can be
+    // classified by the shared taxonomy (runtime/dispatch-outcomes).
+    if (genResult && genResult.deferReason) {
+        warn(`IMAGE_DISPATCH: ${bookId}/${chapterId}/${sceneId} — nothing enqueued (${genResult.deferReason}); deferring`);
+        return { dispatched: false, jobs: 0, reason: genResult.deferReason, retryable: true };
+    }
+
     log(`IMAGE_DISPATCH: ${bookId}/${chapterId}/${sceneId} — no jobs sent (cache hit or empty units)`);
     return { dispatched: false, jobs: 0, reason: 'no_jobs_sent' };
 }
@@ -551,15 +583,41 @@ async function executeVideoDispatch(redis, scene, loadedBook, buildId, dispatchI
     await videoOrch.setWaitingChunks(redis, bookId, chapterId, sceneId);
 
     let sentCount = 0;
+    // Stage 116 (D3) / Stage 118 (M3): collect WHY a job was not sent — the
+    // dominant reason becomes the deferral reason (bounded backoff) instead
+    // of the generic 'send_failed'.
+    const notSentReasons = new Map();
     for (const jobSpec of jobsToSend) {
         try {
-            const sendResult = await provider.generate(jobSpec);
+            let sendResult = await provider.generate(jobSpec);
+            // Stage 122 (N2): the group file was probed absent above (a cached
+            // group never reaches this loop), so an `already_completed` verdict
+            // is by construction an ORPHANED completion marker. Free it and
+            // retry ONCE so the video regenerates instead of looping on a
+            // 900 s deferral with the scene stuck in the active index (audit
+            // 121 · N2). The release rides the persistence port — the
+            // orchestration tier cannot require the backend helper directly.
+            // Skipped when the port is unwired (harnesses that don't exercise
+            // orphan cleanup); production always wires it in backend.cjs.
+            if (isPersistencePortWired()) {
+                let released = false;
+                try {
+                    released = await persist('idem.releaseTaskDoneIfOrphan')(redis, sendResult);
+                } catch (_) { released = false; }
+                if (released) {
+                    log(`VIDEO_DISPATCH: orphaned completion marker released for ${jobSpec.job_id} — regenerating`);
+                    sendResult = await provider.generate(jobSpec);
+                }
+            }
             if (sendResult.sent) {
                 sentCount++;
             } else {
-                warn(`VIDEO_DISPATCH: Hub rejected job ${jobSpec.job_id}: ${sendResult.error || 'unknown'}`);
+                const reason = sendResult.reason || 'send_failed';
+                notSentReasons.set(reason, (notSentReasons.get(reason) || 0) + 1);
+                warn(`VIDEO_DISPATCH: Hub rejected job ${jobSpec.job_id}: ${sendResult.error || reason}`);
             }
         } catch (sendErr) {
+            notSentReasons.set('send_failed', (notSentReasons.get('send_failed') || 0) + 1);
             warn(`VIDEO_DISPATCH: provider generate failed for job ${jobSpec.job_id}: ${sendErr.message}`);
         }
     }
@@ -569,8 +627,13 @@ async function executeVideoDispatch(redis, scene, loadedBook, buildId, dispatchI
         return { dispatched: true, jobs: sentCount, reason: null };
     }
 
-    warn(`VIDEO_DISPATCH: ${bookId}/${chapterId}/${sceneId}: all provider.generate calls failed`);
-    return { dispatched: false, jobs: 0, reason: 'send_failed' };
+    // Stage 116 (D3) / Stage 118 (M3): the shared taxonomy decides whether
+    // the outcome is a bounded deferral. EVERY reason must be deferrable —
+    // a single real failure (send_failed) keeps the historic 'send_failed'
+    // semantics (an immediate retry on the next tick).
+    const deferReason = classifyNotSentReasons(notSentReasons) || 'send_failed';
+    warn(`VIDEO_DISPATCH: ${bookId}/${chapterId}/${sceneId}: nothing enqueued (${deferReason})`);
+    return { dispatched: false, jobs: 0, reason: deferReason, retryable: true };
 }
 
 // T4: dispatchStage принимает dispatchId (из dispatch-engine) и передаёт executors