scene-capability-engine 3.0.7 → 3.0.8

package/CHANGELOG.md

@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Added
 - **Capability lexicon hard gate end-to-end defaultization**: `sce auto handoff run` and `sce auto handoff capability-matrix` now enforce unknown Moqui capability alias blocking by default (expected/provided), emit lexicon gate telemetry into release evidence, and promote those signals into governance risk/concern/recommendation/close-loop block decisions.
+- **Moqui release summary report helper**: Added `node scripts/moqui-release-summary.js` (and npm alias `npm run report:moqui-summary`) to consolidate handoff evidence + baseline + lexicon + capability-matrix into a single release-gate verdict (`passed|failed|incomplete`) with remediation commands.
 - **SCE naming consolidation + compatibility bridge**: Rebranded product naming to `Scene Capability Engine`, moved package to `scene-capability-engine`, promoted `sce` as the primary CLI command, and preserved `sco` / `sce` / `scene-capability-engine` aliases for migration continuity.
 - **Official template library v1.5.0 alignment**: Synced with `scene-capability-engine-templates` `v1.5.0`, adding scene orchestration template coverage for canvas visualization, interaction hardening, execution playbook, dependency drilldown, decision cockpit, runbook export, action queue orchestration, action pack export, and unified scene governance closure.
 - **Branding consistency release guard**: Added `test:brand-consistency` to block publish when legacy repository/package/product naming reappears in tracked source files.
@@ -138,6 +139,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Autonomous operator UX**: Expanded docs with semantic decomposition and live stream behavior for close-loop command usage.
 
 ### Fixed
+- **Orchestrator 429 backoff stall behavior**: `orchestration-engine` now clamps `Retry-After` retry waits by `rateLimitBackoffMaxMs` and interrupts pending retry sleeps immediately on `stop()`, preventing long backoff waits from appearing as deadlocks.
 - **Scene runtime 429 failure behavior under high request pressure**: Moqui HTTP client now retries on `429` with `Retry-After` support and bounded exponential backoff, reducing multi-agent stalls caused by transient service-side request limits.
 - **Moqui adapter matching fallback safety**: `spec.erp.*` bindings no longer get captured by Moqui handler when no adapter config exists, preventing false hard-fail paths and restoring expected simulator fallback behavior.
 - **Controller summary semantics**: `close-loop-controller` now reports final `pending_goals` from the persisted queue snapshot and only marks `cycle-limit-reached` as exhausted when pending work remains (or empty-polling mode explicitly consumes cycle budget).

package/docs/331-poc-weekly-delivery-checklist.md

@@ -23,6 +23,12 @@ Use this checklist before each integration batch.
 - `ontology_validation` exists and is recent.
 - Dependency relations (`depends_on`) are present for multi-spec batches.
 - High-risk gaps have mitigation notes.
+- Moqui baseline matrix has no hard-gate regressions:
+  - `compare.coverage_matrix_regressions.length == 0`
+  - `summary.coverage_matrix.baseline_passed.rate_percent == 100`
+- Capability lexicon has no unknown aliases:
+  - `summary.expected_unknown_count == 0`
+  - `summary.provided_unknown_count == 0`
 
 ## sce Acceptance Commands
 
@@ -34,6 +40,10 @@ npx sce auto handoff run --manifest ../331-poc/docs/handoffs/handoff-manifest.js
 
 npx sce auto handoff template-diff --manifest ../331-poc/docs/handoffs/handoff-manifest.json --json
 npx sce auto handoff regression --session-id latest --json
+npx sce auto handoff capability-matrix --manifest ../331-poc/docs/handoffs/handoff-manifest.json --fail-on-gap --json
+npx sce scene moqui-baseline --compare-with .kiro/reports/release-evidence/moqui-template-baseline-prev.json --fail-on-portfolio-fail --json
+node scripts/moqui-lexicon-audit.js --manifest ../331-poc/docs/handoffs/handoff-manifest.json --fail-on-gap --json
+node scripts/moqui-release-summary.js --fail-on-gate-fail --json
 ```
 
 ## Scene Package Gate Commands

package/docs/command-reference.md

@@ -602,7 +602,7 @@ Close-loop batch (`sce auto close-loop-batch <goals-file>`) options:
 - Batch summary includes `resource_plan` (budget/effective parallel/per-goal maxParallel/scheduling strategy/aging/starvation wait metrics/criticality summary) and `metrics` (`success_rate_percent`, `status_breakdown`, `average_sub_specs_per_goal`, `average_replan_cycles_per_goal`, `total_rate_limit_signals`, `average_rate_limit_signals_per_goal`, `total_rate_limit_backoff_ms`)
   - Under budget mode, scheduler is complexity-weighted (`goal_weight`/`scheduling_weight`) so higher-complexity goals consume more shared slots and can reduce same-batch concurrency.
   - Batch summary includes `batch_retry` telemetry (strategy, until-complete mode, configured/max/performed rounds, exhausted flag, per-round history).
-  - Under `--batch-retry-strategy adaptive`, retry history includes rate-limit pressure and next-round backpressure decisions (`applied_batch_parallel`, `next_batch_parallel`, `adaptive_backpressure_applied`).
+  - Under `--batch-retry-strategy adaptive`, retry history includes rate-limit pressure and next-round backpressure decisions (`applied_batch_parallel`, `next_batch_parallel`, `adaptive_backpressure_applied`, `backpressure_level`). Severe pressure automatically halves `batch_parallel` / `batch_agent_budget` for the next retry round.
   - Batch summary includes `batch_session` metadata when persisted (session id + file path).
   - When using `--decompose-goal`, summary includes `generated_from_goal` metadata (strategy, target count, produced count, clause/category diagnostics, decomposition `quality`, and refinement telemetry).
 
@@ -833,6 +833,18 @@ Moqui template library lexicon audit (script-level governance helper):
   - By default, template capability auditing is scoped to `manifest.templates` (when matched), reducing noise from unrelated templates.
   - Template scope matching normalizes `sce.scene--*` / `kse.scene--*` prefixes, so renamed template namespaces still map correctly.
 
+Moqui release summary helper (script-level consolidated gate view):
+- `node scripts/moqui-release-summary.js [--evidence <path>] [--baseline <path>] [--lexicon <path>] [--capability-matrix <path>] [--out <path>] [--markdown-out <path>] [--fail-on-gate-fail] [--json]`: merge handoff release-evidence + baseline + lexicon + capability-matrix into one Moqui release gate summary (`passed | failed | incomplete`) with executable remediation recommendations.
+  - Default inputs:
+    - `.kiro/reports/release-evidence/handoff-runs.json`
+    - `.kiro/reports/release-evidence/moqui-template-baseline.json`
+    - `.kiro/reports/release-evidence/moqui-lexicon-audit.json`
+    - `.kiro/reports/handoff-capability-matrix.json`
+  - Default outputs:
+    - `.kiro/reports/release-evidence/moqui-release-summary.json`
+    - `.kiro/reports/release-evidence/moqui-release-summary.md`
+  - `--fail-on-gate-fail` exits with code `2` when summary gate is `failed`.
+
 Moqui standard rebuild helper (script-level recovery bootstrap):
 - `node scripts/moqui-standard-rebuild.js [--metadata <path>] [--out <path>] [--markdown-out <path>] [--bundle-out <path>] [--json]`: build a standard Moqui recovery bundle from metadata, including recommended SCE template matrix, recovery spec plan, handoff manifest seed, ontology seed, and page-copilot context contract.
   - Output now includes `recovery.readiness_matrix`, `recovery.readiness_summary`, and `recovery.prioritized_gaps` for template capability matrix scoring and remediation planning.
@@ -872,7 +884,7 @@ Recommended `.kiro/config/orchestrator.json`:
 }
 ```
 
-`rateLimit*` settings provide dedicated retry/backoff and adaptive parallel throttling when providers return 429 / too-many-requests errors. Engine retry now also honors `Retry-After` / `try again in ...` hints from provider error messages when present, and pauses launching new pending specs during the active backoff window to reduce request bursts (launch hold remains active even if adaptive parallel throttling is disabled).
+`rateLimit*` settings provide dedicated retry/backoff and adaptive parallel throttling when providers return 429 / too-many-requests errors. Engine retry honors `Retry-After` / `try again in ...` hints from provider error messages and clamps final retry waits by `rateLimitBackoffMaxMs` to avoid unbounded pause windows. During active backoff windows, new pending spec launches are paused to reduce request bursts (launch hold remains active even if adaptive parallel throttling is disabled). `orchestrate stop` now interrupts pending retry waits immediately so long backoff does not look like a deadlock.
 
 ### Scene Template Engine
 
@@ -957,6 +969,9 @@ sce scene moqui-baseline --json
 # Script alias (same behavior)
 npm run report:moqui-baseline
 
+# Consolidated Moqui release gate summary (recommended before tag/publish)
+npm run report:moqui-summary
+
 # Score all scene templates instead of the default Moqui + orchestration subset
 sce scene moqui-baseline --include-all --json
 
@@ -977,6 +992,7 @@ sce scene moqui-baseline \
 
 Release workflow default:
 - Publishes `moqui-template-baseline.json` + `moqui-template-baseline.md` as release assets.
+- Publishes `moqui-release-summary.json` + `moqui-release-summary.md` as release review assets.
 - Enforces baseline portfolio gate by default (`KSE_MOQUI_BASELINE_ENFORCE` defaults to `true` when unset).
 
 ### Moqui ERP Integration

package/docs/moqui-template-core-library-playbook.md

@@ -33,6 +33,42 @@ Emergency bypass exists but is not recommended:
 - `--no-require-moqui-baseline`
 - `--no-require-capability-coverage`
 
+## Template Capability Matrix Contract
+
+Use the baseline report as the canonical matrix contract (`.kiro/reports/moqui-template-baseline.json`):
+
+| Matrix Dimension | Meaning | Default Gate Target |
+| --- | --- | --- |
+| `graph_valid.rate_percent` | Ontology graph structural validity | `100%` |
+| `score_passed.rate_percent` | Semantic score above baseline threshold | `100%` |
+| `entity_coverage.rate_percent` | Entity model coverage | `100%` |
+| `relation_coverage.rate_percent` | Entity relation coverage | `100%` |
+| `business_rule_coverage.rate_percent` | Business-rule presence coverage | `100%` |
+| `business_rule_closed.among_covered_rate_percent` | No unmapped rules among covered templates | `100%` |
+| `decision_coverage.rate_percent` | Decision logic presence coverage | `100%` |
+| `decision_closed.among_covered_rate_percent` | No undecided decisions among covered templates | `100%` |
+| `baseline_passed.rate_percent` | Full matrix closure rate | `100%` |
+
+Trend regression should stay hard-gated by default:
+
+- `compare.coverage_matrix_regressions.length` must be `0`
+- `sce auto handoff run` default `--max-moqui-matrix-regressions 0`
+
+## External POC Handoff Requirements
+
+When an upstream business project (e.g. a POC) feeds templates into sce, require the following by default:
+
+- Every handoff spec contains:
+  - `custom/scene.yaml` with entity/service/screen binding refs
+  - `custom/scene-package.json` with explicit `capabilities.expected/provided`
+  - ontology semantics for entity, relation, business-rule, and decision dimensions
+- Handoff evidence includes:
+  - latest `moqui-template-baseline.json|.md`
+  - latest `handoff-capability-matrix` report
+  - latest `moqui-lexicon-audit` report
+  - ontology batch report from `scene package-publish-batch --dry-run`
+- New/updated templates must provide deterministic IDs and version bumps so baseline comparisons are stable across releases.
+
 ## One-Shot Intake Flow
 
 ```bash
@@ -60,6 +96,11 @@ node scripts/moqui-lexicon-audit.js \
   --fail-on-gap \
   --json
 
+# 0.4) Consolidated release gate summary (single-file pass/fail/incomplete verdict)
+node scripts/moqui-release-summary.js \
+  --fail-on-gate-fail \
+  --json
+
 # 1) Handoff close-loop
 sce auto handoff run --manifest docs/handoffs/handoff-manifest.json --json
 
@@ -99,6 +140,8 @@ Required artifacts for each intake batch:
 - `.kiro/reports/release-evidence/moqui-capability-coverage.md`
 - `.kiro/reports/release-evidence/moqui-lexicon-audit.json`
 - `.kiro/reports/release-evidence/moqui-lexicon-audit.md`
+- `.kiro/reports/release-evidence/moqui-release-summary.json`
+- `.kiro/reports/release-evidence/moqui-release-summary.md`
 - `.kiro/reports/handoff-capability-matrix.md` (or JSON equivalent from `sce auto handoff capability-matrix`)
 - `.kiro/reports/handoff-runs/<session>.json`
 - `.kiro/reports/scene-package-ontology-batch.json`

package/lib/commands/auto.js

@@ -4701,6 +4701,67 @@ function buildBatchMetrics(results, totalGoals) {
   };
 }
 
+function applyAdaptiveRateLimitBackpressure({
+  currentBatchParallel,
+  currentBatchAgentBudget,
+  rateLimitSignals,
+  rateLimitBackoffMs,
+  rateLimitLaunchHoldMs,
+  inputGoals
+}) {
+  const next = {
+    batchParallel: Number.isInteger(currentBatchParallel) ? currentBatchParallel : 1,
+    batchAgentBudget: currentBatchAgentBudget,
+    applied: false,
+    level: 'none',
+    signalsPerGoal: 0
+  };
+
+  const safeGoals = Math.max(1, Number(inputGoals) || 1);
+  const normalizedSignals = Math.max(0, Number(rateLimitSignals) || 0);
+  const normalizedBackoffMs = Math.max(0, Number(rateLimitBackoffMs) || 0);
+  const normalizedLaunchHoldMs = Math.max(0, Number(rateLimitLaunchHoldMs) || 0);
+  const signalsPerGoal = normalizedSignals / safeGoals;
+  next.signalsPerGoal = Number(signalsPerGoal.toFixed(2));
+
+  if (normalizedSignals <= 0 && normalizedBackoffMs <= 0 && normalizedLaunchHoldMs <= 0) {
+    return next;
+  }
+
+  const severePressure = (
+    signalsPerGoal >= 1.5 ||
+    normalizedBackoffMs >= 4000 ||
+    normalizedLaunchHoldMs >= 2000
+  );
+  next.level = severePressure ? 'severe' : 'mild';
+
+  const normalizePositiveInteger = (value, fallback = 1) => {
+    const parsed = Number(value);
+    if (!Number.isInteger(parsed) || parsed < 1) {
+      return fallback;
+    }
+    return parsed;
+  };
+
+  const currentParallel = normalizePositiveInteger(next.batchParallel, 1);
+  const reducedParallel = severePressure
+    ? Math.max(1, Math.floor(currentParallel / 2))
+    : Math.max(1, currentParallel - 1);
+  next.batchParallel = reducedParallel;
+  next.applied = reducedParallel !== currentParallel;
+
+  if (currentBatchAgentBudget !== null && currentBatchAgentBudget !== undefined) {
+    const currentBudget = normalizePositiveInteger(currentBatchAgentBudget, 1);
+    const reducedBudget = severePressure
+      ? Math.max(1, Math.floor(currentBudget / 2))
+      : Math.max(1, currentBudget - 1);
+    next.batchAgentBudget = reducedBudget;
+    next.applied = next.applied || reducedBudget !== currentBudget;
+  }
+
+  return next;
+}
+
 async function runCloseLoopBatchGoals(goals, options) {
   const continueOnError = Boolean(options.continueOnError);
   const batchParallel = normalizeBatchParallel(options.batchParallel);
@@ -4996,16 +5057,25 @@ async function runCloseLoopBatchWithRetries(goals, options) {
     let nextBatchParallel = adaptiveBatchParallel;
     let nextBatchAgentBudget = adaptiveBatchAgentBudget;
     let adaptiveBackpressureApplied = false;
+    let backpressureLevel = 'none';
+    let roundRateLimitSignalsPerGoal = Number(
+      (roundRateLimitSignals / Math.max(1, pendingEntries.length)).toFixed(2)
+    );
     const hasRateLimitPressure = roundRateLimitSignals > 0 || roundRateLimitBackoffMs > 0 || roundMaxLaunchHoldMs > 0;
     if (retryStrategy === 'adaptive' && retryEntries.length > 0 && hasRateLimitPressure) {
-      if (nextBatchParallel > 1) {
-        nextBatchParallel -= 1;
-        adaptiveBackpressureApplied = true;
-      }
-      if (nextBatchAgentBudget !== null && nextBatchAgentBudget > 1) {
-        nextBatchAgentBudget -= 1;
-        adaptiveBackpressureApplied = true;
-      }
+      const backpressure = applyAdaptiveRateLimitBackpressure({
+        currentBatchParallel: adaptiveBatchParallel,
+        currentBatchAgentBudget: adaptiveBatchAgentBudget,
+        rateLimitSignals: roundRateLimitSignals,
+        rateLimitBackoffMs: roundRateLimitBackoffMs,
+        rateLimitLaunchHoldMs: roundMaxLaunchHoldMs,
+        inputGoals: pendingEntries.length
+      });
+      nextBatchParallel = backpressure.batchParallel;
+      nextBatchAgentBudget = backpressure.batchAgentBudget;
+      adaptiveBackpressureApplied = backpressure.applied;
+      backpressureLevel = backpressure.level;
+      roundRateLimitSignalsPerGoal = backpressure.signalsPerGoal;
     }
 
     retryHistory.push({
@@ -5019,9 +5089,11 @@ async function runCloseLoopBatchWithRetries(goals, options) {
       unprocessed_goals: unprocessedCount,
       stopped_early: Boolean(run.stoppedEarly),
       rate_limit_signals: roundRateLimitSignals,
+      rate_limit_signals_per_goal: roundRateLimitSignalsPerGoal,
       rate_limit_backoff_ms: roundRateLimitBackoffMs,
       rate_limit_launch_hold_ms: roundMaxLaunchHoldMs,
       adaptive_backpressure_applied: adaptiveBackpressureApplied,
+      backpressure_level: backpressureLevel,
       next_batch_parallel: nextBatchParallel,
       next_batch_agent_budget: nextBatchAgentBudget
     });

package/lib/orchestrator/orchestration-engine.js

@@ -107,6 +107,8 @@ class OrchestrationEngine extends EventEmitter {
     this._launchBudgetLastHoldSignalAt = 0;
     /** @type {number} last launch-budget hold duration emitted to telemetry (ms) */
     this._launchBudgetLastHoldMs = 0;
+    /** @type {Set<{timer: NodeJS.Timeout|null, resolve: (() => void)|null}>} cancellable sleep waiters */
+    this._pendingSleeps = new Set();
     /** @type {number} fallback wait timeout to avoid indefinite hangs when lifecycle events are missing */
     this._agentWaitTimeoutMs = (DEFAULT_AGENT_WAIT_TIMEOUT_SECONDS * 1000) + AGENT_WAIT_TIMEOUT_GRACE_MS;
     /** @type {() => number} */
@@ -230,6 +232,7 @@ class OrchestrationEngine extends EventEmitter {
    */
   async stop() {
     this._stopped = true;
+    this._cancelPendingSleeps();
 
     if (this._state !== 'running') {
       return;
@@ -520,10 +523,7 @@ class OrchestrationEngine extends EventEmitter {
       this._statusMonitor.updateSpecStatus(specName, 'pending', null, resolvedError);
 
       const retryDelayMs = isRateLimitError
-        ? Math.max(
-          this._calculateRateLimitBackoffMs(retryCount),
-          this._extractRateLimitRetryAfterMs(resolvedError)
-        )
+        ? this._resolveRateLimitRetryDelayMs(resolvedError, retryCount)
         : 0;
       if (retryDelayMs > 0) {
         this._onRateLimitSignal(retryDelayMs);
@@ -1156,16 +1156,73 @@ class OrchestrationEngine extends EventEmitter {
     return Math.max(1, Math.round(cappedBaseDelay * jitterFactor));
   }
 
+  /**
+   * Resolve final retry delay for rate-limit failures.
+   * Uses larger of computed backoff and retry-after hint, then clamps to configured max.
+   *
+   * @param {string} error
+   * @param {number} retryCount
+   * @returns {number}
+   * @private
+   */
+  _resolveRateLimitRetryDelayMs(error, retryCount) {
+    const computedBackoffMs = this._calculateRateLimitBackoffMs(retryCount);
+    const hintedRetryAfterMs = this._extractRateLimitRetryAfterMs(error);
+    const candidateDelayMs = Math.max(computedBackoffMs, hintedRetryAfterMs);
+    const maxDelayMs = this._toPositiveInteger(
+      this._rateLimitBackoffMaxMs,
+      DEFAULT_RATE_LIMIT_BACKOFF_MAX_MS
+    );
+    return Math.max(1, Math.min(candidateDelayMs, maxDelayMs));
+  }
+
   /**
    * @param {number} ms
    * @returns {Promise<void>}
    * @private
    */
   _sleep(ms) {
-    if (!ms || ms <= 0) {
+    if (!ms || ms <= 0 || this._stopped) {
       return Promise.resolve();
     }
-    return new Promise((resolve) => setTimeout(resolve, ms));
+    return new Promise((resolve) => {
+      let settled = false;
+      const entry = { timer: null, resolve: null };
+      entry.resolve = () => {
+        if (settled) {
+          return;
+        }
+        settled = true;
+        if (entry.timer) {
+          clearTimeout(entry.timer);
+        }
+        this._pendingSleeps.delete(entry);
+        resolve();
+      };
+      entry.timer = setTimeout(() => {
+        if (entry.resolve) {
+          entry.resolve();
+        }
+      }, ms);
+      this._pendingSleeps.add(entry);
+    });
+  }
+
+  /**
+   * Cancel all pending sleeps so stop() does not block behind long retry waits.
+   *
+   * @private
+   */
+  _cancelPendingSleeps() {
+    if (!this._pendingSleeps || this._pendingSleeps.size === 0) {
+      return;
+    }
+    for (const entry of Array.from(this._pendingSleeps)) {
+      if (entry && typeof entry.resolve === 'function') {
+        entry.resolve();
+      }
+    }
+    this._pendingSleeps.clear();
   }
 
   /**
@@ -1300,6 +1357,7 @@ class OrchestrationEngine extends EventEmitter {
    * @private
    */
   _reset() {
+    this._cancelPendingSleeps();
     this._runningAgents.clear();
     this._retryCounts.clear();
     this._failedSpecs.clear();

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "scene-capability-engine",
-  "version": "3.0.7",
+  "version": "3.0.8",
   "description": "SCE (Scene Capability Engine) - A CLI tool and npm package for spec-driven development with AI coding assistants.",
   "main": "index.js",
   "bin": {
@@ -39,6 +39,7 @@
     "report:moqui-rebuild": "node scripts/moqui-standard-rebuild.js --json",
     "gate:moqui-rebuild": "node scripts/moqui-rebuild-gate.js",
     "report:moqui-baseline": "node scripts/moqui-template-baseline-report.js --json",
+    "report:moqui-summary": "node scripts/moqui-release-summary.js --json",
     "report:moqui-core-regression": "node scripts/moqui-core-regression-suite.js --json",
     "prepublishOnly": "npm run test:full && npm run test:skip-audit && npm run test:brand-consistency",
     "publish:manual": "npm publish --access public",