motionloom 2.0.0 → 2.2.0

package/docs/audits/2.1.0-deep-stress-evaluation.md

@@ -0,0 +1,97 @@
+# MotionLoom 2.1.0 — Deep Stress and Agent Intelligence Evaluation
+
+## Executive conclusion
+
+MotionLoom đã vượt qua một đợt deep stress **6.900 logical trials** trên canonical smoke task `browser-review-smoke-task`, với seed `20260813`, wall time `25.883117s`, `6.900/6.900` pass, `0` false positive và `0` false negative. Các thử nghiệm gọi trực tiếp validator/helper production và fault-injection trên artifact thật; đây **không phải** tuyên bố rằng đã render 6.900 animation trong browser.
+
+Kết luận hiện tại là MotionLoom có nền tảng **trust-boundary, context binding, provenance và review-first** mạnh. Nó đã đủ đáng tin để giảm những vòng chỉnh sửa sai do stale evidence, cross-task mix-up, tamper, wrong scene, replay path escape và attestation sai chữ ký. Tuy nhiên, nó chưa thể được gọi là hệ thống “hiểu chất lượng hình ảnh animation” hoàn chỉnh: visual comparison có nhãn provenance, historical learning, device/browser matrix và đo số vòng sửa thực tế vẫn còn thiếu.
+
+## Test matrix and evidence
+
+| Contract surface | Trials | Expected result | Observed result | p95 / max |
+| --- | ---: | --- | --- | ---: |
+| Canonical JSON metamorphic ordering | 600 | Reordered mappings serialize identically | 600 pass, 0 FP/FN | 0.027 / 0.216 ms |
+| DSSE Ed25519 round-trip | 250 | Valid signature verifies | 250 pass | 0.144 / 0.163 ms |
+| DSSE payload tamper | 250 | Modified payload is rejected | 250 pass | 0.140 / 0.179 ms |
+| Statement contract | 600 | 200 valid accepted; 400 mutated rejected | 600 pass | 0.008 / 0.060 ms |
+| Attestation verifier boundary | 1,500 | Codes 0, 11, 14 and 13 remain stable | 1,500 pass; 375 per code | 0.505 / 0.842 ms |
+| Strict quality gate | 400 | Canonical production bundle accepted | 400 pass | 29.717 / 45.588 ms |
+| Intelligence P1 | 400 | Graph/P1 reports accepted | 400 pass | 21.977 / 32.027 ms |
+| Replay hash validation | 475 | Clean replay remains clean | 475 pass | 10.210 / 13.316 ms |
+| Semantic lint clean path | 475 | Canonical lint report remains valid | 475 pass | 0.063 / 0.121 ms |
+| Continuity clean path | 475 | Canonical multi-scene continuity remains valid | 475 pass | 0.054 / 0.101 ms |
+| Approval invariant | 475 | Attestation and verifier report remain `approval=false` | 475 pass | 0.149 / 0.203 ms |
+| Fault injection: graph, IR, provenance, replay, P1, continuity, semantic, attestation | 1,000 | Every controlled mutation is rejected | 1,000 pass, 0 FN | 0.095–16.092 ms p95 by case |
+
+The machine-readable source is [`data/2.1.0-deep-stress-6900.json`](data/2.1.0-deep-stress-6900.json). Its SHA-256 is `8e4b967a87cf1dab2268c23fc15a98b5ebde1699d81a102505d98ccb89bf3a2d`.
+
+The fault corpus deliberately checks task ID substitution, scene substitution, provenance substitution, replay path escape, fix-plan source hash tampering, wrong-scene P1 validation, continuity structural drift, semantic structural drift, attestation approval mutation and payload mutation. The semantic/continuity validator hardening added during this audit closed a real malformed-report acceptance gap; the continuity regression fixture was then corrected to preserve the production SHA-256 context binding rather than a legacy empty placeholder.
+
+## Runtime and Dev Lab cross-check
+
+The in-process stress run was complemented by a real browser runtime test. Rive, GSAP and Framer Motion all passed the runtime adapter harness with 0/50/100% snapshots, no console errors and telemetry at approximately 16.7–16.8 ms maximum/p95 RAF interval in the sampled run. This establishes that the adapter evidence path works for the canonical examples, but it is not a 5.000-run visual or cross-device benchmark.
+
+The Dev Lab production build passed after the attestation rail and Timeline Desk review-driven pass. Two non-blocking operational warnings remain: pnpm ignores the legacy `pnpm` field in `package.json`, and the main JavaScript bundle is approximately 780.38 kB before gzip / 225.37 kB gzip, above the 500 kB advisory threshold. These do not invalidate the current build, but code splitting should be planned before the Lab grows further.
+
+## Scorecard
+
+The following scores are an engineering rubric based on the evidence above, not a universal benchmark or a claim of autonomous animation taste.
+
+| Capability | Score | Evidence-based assessment |
+| --- | ---: | --- |
+| Determinism and contract integrity | **9.5/10** | 6,900/6,900 pass; canonicalization, replay, semantic and continuity paths remained stable. |
+| Trust boundary and evidence identity | **9.4/10** | 1,500 verifier cases plus 1,000 cross-boundary mutations rejected correctly; approval invariant preserved. |
+| Project-aware binding | **8.8/10** | Graph, provenance, Motion IR, task/scene and fix-plan hashes are actively checked; more project archetypes are needed. |
+| Runtime truthfulness | **7.9/10** | Three real adapters pass with telemetry; visual correctness and environment diversity are not yet measured at scale. |
+| Agent handoff and review-first collaboration | **7.8/10** | Reports, issues, fix-plan, continuity, handoff and Dev Lab review rail exist; remediation-cycle outcomes are not yet learned from history. |
+| Performance and operational ergonomics | **8.2/10** | In-process p95 is low and strict gate is under 30 ms in the fixture; browser/device scaling and bundle size remain open. |
+| Release usability | **8.9/10** | `motionloom@2.0.0` is public on npm with a CLI wrapper and workstation release guide; Python/runtime dependencies remain external prerequisites. |
+
+**Overall engineering readiness: 8.6/10.** The score is high for evidence integrity and deterministic orchestration, not for end-to-end visual intelligence. The strongest differentiator is that MotionLoom can explain why an artifact is unsafe to trust instead of merely generating another animation.
+
+## What is working well
+
+MotionLoom now has a credible chain from project context to graph, provenance, Motion IR, runtime evidence, semantic feedback, continuity, signed attestation and review-first handoff. That chain is materially more useful to an Agent than a collection of framework snippets because the Agent can bind an artifact to a task and explain whether a failure is stale context, wrong scene, tampering, unsupported capability or an actual quality issue.
+
+The new hardening also demonstrates an important behavior: the system distinguishes a valid signature from user approval. A cryptographically verified bundle remains `approval=false`, and the Dev Lab remains the place where the user sees, tests, fixes and explicitly confirms a PR-side effect. This protects against the common failure mode where automation quietly treats “technically valid” as “approved.”
+
+The fix-plan and handoff surfaces are promising for reducing repeated edits. The system can identify a root cause, constrain patch scope and select a rerun scope instead of asking an Agent to regenerate the whole scene. That is the right architecture for reducing accidental churn, even though the audit did not yet measure a historical reduction percentage across real user iterations.
+
+## What is not yet proven
+
+The 6.900 trials are predominantly deterministic contract tests over one canonical smoke task. They prove boundary behavior, not broad semantic understanding across many project types, animation styles, body rigs, asset families, browser engines, device pixel ratios, reduced-motion settings or long-running timelines.
+
+The harness does not yet perform pixel-diff or perceptual comparison against labeled expected frames. Therefore it can prove that a frame was captured, bound and verified, but not that a character’s hand pose, easing feel, silhouette, staging or brand motion is visually correct. A second limitation is that clean-path repetition can hide state leaks that only appear after many different task bundles; a future matrix must rotate fixtures, not only repeat the same fixture.
+
+The Agent collaboration score is also necessarily provisional. The repository emits structured reports and actionable handoff data, but it does not yet have a benchmark of “first-pass acceptance rate,” “mean corrections per scene,” “rerender avoidance rate,” or “time from issue to accepted fix.” Without those measures, the claim that MotionLoom reduces user workload is architecturally plausible but not yet empirically quantified.
+
+## Prioritized development recommendations
+
+| Priority | Recommendation | Why it reduces wrong animation edits | Acceptance metric |
+| --- | --- | --- | --- |
+| P0 | Promote `deep-stress.py` to a documented nightly/PR audit with fixed seeds, thresholds and retained JSON artifacts | Prevents trust-boundary regressions from returning silently | 5,000+ logical trials, 0 FN, 0 FP, stable p95 budget |
+| P0 | Add visual-comparison contract with provenance-labeled 0/50/100% fixtures, pixel diff and perceptual hash | Distinguishes “evidence is valid” from “motion looks correct” | Labeled fixture pass/fail with explainable region-level diff |
+| P1 | Add multi-project corpus: landing UI, product onboarding, body rig, Lottie export, Rive state machine and mixed framework scene | Tests whether project-aware binding generalizes beyond one smoke bundle | At least 10 project archetypes and cross-task mutation coverage |
+| P1 | Add browser matrix for Chromium/WebKit/Firefox, DPR 1/2, reduced motion and 30/60/120 Hz timing | Captures runtime failures that in-process validators cannot see | Zero console errors; telemetry threshold per environment |
+| P1 | Create append-only remediation ledger: issue, proposed fix, patch scope, rerun scope, user outcome | Lets the Agent learn which fixes reduce repeated edits instead of regenerating blindly | Track first-pass acceptance, correction count and rerender avoidance |
+| P1 | Add benchmark history and aggregate metrics by project/context/framework/capability | Detects drift and capability regressions over time | Trend report, outlier alert and reproducible run hash |
+| P2 | Split Dev Lab public route and heavy workbench bundles with dynamic imports; migrate pnpm config to supported settings | Reduces load cost as evidence rails grow | Main chunk below advisory threshold and no pnpm warning |
+| P2 | Add capability confidence and “why this adapter” explanation to Agent handoff | Makes unsupported or stale capability selection visible before rendering | Every selected capability has evidence hash, freshness and reason |
+
+## Recommended next milestone
+
+The next milestone should be **2.2.0 Visual Truth and Remediation Learning**. It should not add more framework adapters first. It should add labeled visual fixtures, perceptual comparison, multi-project context corpus and the remediation ledger. Only after those are working should historical outcomes influence recommendations; otherwise the Agent risks learning from unverified visual outputs.
+
+## Reproduction
+
+From the repository root:
+
+```bash
+npm run audit:deep
+python3 tests/scripts/run_tests.py
+python3 scripts/eval-intelligence.py
+python3 scripts/skill-doctor.py --json
+npm run runtime:test
+```
+
+The deep audit command writes a machine-readable report under `docs/audits/data/` when invoked with the documented script defaults. The strict production checks remain separate so a fast developer test does not accidentally imply that browser review, telemetry, attestation or user approval has occurred.

package/docs/audits/ci-replay-remediation-2026-08-13.md

@@ -0,0 +1,33 @@
+# CI Replay Remediation — 2026-08-13
+
+## Evidence
+
+The `MotionLoom Quality` workflow for commit [`d148f21`](https://github.com/lenhonbp/MotionLoom/commit/d148f2175e47e3fc32f7156fc83af82c749b47ed) completed with failure in run [`31709781319`](https://github.com/lenhonbp/MotionLoom/actions/runs/31709781319). Documentation and Package Hygiene and Security Analysis completed successfully on the same commit.
+
+The failing step was **Enforce context-bound quality gate**. The recorded rejection was:
+
+> `replay bundle has 11 mismatch(es)`
+
+The preceding Intelligence, project graph, Motion IR and provenance checks were valid. The failure therefore represented stale replay hashes, not an approval decision or a request to weaken the verifier.
+
+## Root cause
+
+The Quality workflow treated `replay-bundle.json` as a committed integrity snapshot, but earlier steps intentionally regenerate runtime telemetry, semantic-lint benchmark output, report/manifest output, signed attestation output and rendered runtime evidence for changed scenes. The quality gate then verified the old replay bundle against those newly generated files. A local verification before the generated-artifact phase could pass while the same sequence in CI correctly reported mismatches.
+
+## Remediation
+
+The workflow now runs the canonical command below after runtime snapshot rendering and before the context-bound quality gate:
+
+```text
+python3 scripts/intelligence.py replay capture \
+  --task-dir "artifacts/${scene}-task" --root . \
+  --output "artifacts/${scene}-task/replay-bundle.json"
+```
+
+The change does not remove replay verification, expand exclusions, or turn heuristic evidence into approval. A regression assertion checks that replay capture follows runtime rendering and precedes the quality gate. A temporary task-bundle test also confirms that tampered evidence is rejected, canonical capture rebuilds its hashes, and the rebuilt bundle verifies successfully.
+
+## Local verification
+
+After the remediation, the following checks passed locally: engine regression, docs audit, Skill Doctor, skill-creator validator, runtime adapter smoke, Agent discovery, installation matrix, Visual Truth validation, Remediation Learning validation, quality attestation with Visual Truth required, npm dry-run, Python syntax compilation and `git diff --check`.
+
+This audit records the CI incident and its fix as evidence; it does not change the published `motionloom@2.1.0` version or grant user approval.

package/docs/audits/data/2.1.0-deep-stress-6900.json

@@ -0,0 +1,329 @@
+{
+  "schema_version": "1.0",
+  "audit": "motionloom-deep-stress",
+  "generated_at": "2026-08-13T08:44:23.535422Z",
+  "seed": 20260813,
+  "requested_iterations": 5000,
+  "actual_iterations": 6900,
+  "fixture": {
+    "task_id": "browser-review-smoke-task",
+    "scene": "browser-review-smoke"
+  },
+  "summary": {
+    "passed": 6900,
+    "failed": 0,
+    "pass_rate": 1.0,
+    "false_positive": 0,
+    "false_negative": 0,
+    "clean_contracts_preserved": true,
+    "faults_rejected_without_false_negative": true,
+    "approval_false_invariant_preserved": true
+  },
+  "metrics": {
+    "canonical-json-metamorphic": {
+      "attempts": 600,
+      "passed": 600,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "equal": 600
+      },
+      "p50_ms": 0.018963,
+      "p95_ms": 0.027028,
+      "max_ms": 0.215561,
+      "pass_rate": 1.0
+    },
+    "dsse-ed25519-roundtrip": {
+      "attempts": 250,
+      "passed": 250,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "verified": 250
+      },
+      "p50_ms": 0.128048,
+      "p95_ms": 0.144034,
+      "max_ms": 0.163253,
+      "pass_rate": 1.0
+    },
+    "dsse-ed25519-tamper": {
+      "attempts": 250,
+      "passed": 250,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "invalid-signature": 250
+      },
+      "p50_ms": 0.130132,
+      "p95_ms": 0.140209,
+      "max_ms": 0.178904,
+      "pass_rate": 1.0
+    },
+    "statement-contract": {
+      "attempts": 600,
+      "passed": 600,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "valid": 200,
+        "rejected": 400
+      },
+      "p50_ms": 0.007181,
+      "p95_ms": 0.008413,
+      "max_ms": 0.059516,
+      "pass_rate": 1.0
+    },
+    "attestation-verifier-boundary": {
+      "attempts": 1500,
+      "passed": 1500,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "0": 375,
+        "11": 375,
+        "14": 375,
+        "13": 375
+      },
+      "p50_ms": 0.303064,
+      "p95_ms": 0.505011,
+      "max_ms": 0.842255,
+      "pass_rate": 1.0
+    },
+    "strict-quality-gate-clean": {
+      "attempts": 400,
+      "passed": 400,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "accepted": 400
+      },
+      "p50_ms": 24.511679,
+      "p95_ms": 29.717216,
+      "max_ms": 45.587579,
+      "pass_rate": 1.0
+    },
+    "intelligence-p1-clean": {
+      "attempts": 400,
+      "passed": 400,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "accepted": 400
+      },
+      "p50_ms": 17.860905,
+      "p95_ms": 21.976583,
+      "max_ms": 32.027403,
+      "pass_rate": 1.0
+    },
+    "replay-hash-clean": {
+      "attempts": 475,
+      "passed": 475,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "clean": 475
+      },
+      "p50_ms": 8.02739,
+      "p95_ms": 10.20982,
+      "max_ms": 13.31553,
+      "pass_rate": 1.0
+    },
+    "semantic-lint-clean": {
+      "attempts": 475,
+      "passed": 475,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "clean": 475
+      },
+      "p50_ms": 0.044806,
+      "p95_ms": 0.062948,
+      "max_ms": 0.120889,
+      "pass_rate": 1.0
+    },
+    "continuity-clean": {
+      "attempts": 475,
+      "passed": 475,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "clean": 475
+      },
+      "p50_ms": 0.036813,
+      "p95_ms": 0.053581,
+      "max_ms": 0.1014,
+      "pass_rate": 1.0
+    },
+    "approval-invariant": {
+      "attempts": 475,
+      "passed": 475,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "approval=false": 475
+      },
+      "p50_ms": 0.103737,
+      "p95_ms": 0.148545,
+      "max_ms": 0.202869,
+      "pass_rate": 1.0
+    },
+    "fault-graph-task-id": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 14.465091,
+      "p95_ms": 16.091659,
+      "max_ms": 16.891256,
+      "pass_rate": 1.0
+    },
+    "fault-motion-ir-scene": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 2.367015,
+      "p95_ms": 2.776738,
+      "max_ms": 3.812271,
+      "pass_rate": 1.0
+    },
+    "fault-provenance-task-id": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 0.913747,
+      "p95_ms": 1.226514,
+      "max_ms": 1.544532,
+      "pass_rate": 1.0
+    },
+    "fault-replay-path-escape": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 8.533108,
+      "p95_ms": 10.344097,
+      "max_ms": 11.235017,
+      "pass_rate": 1.0
+    },
+    "fault-p1-fix-plan-hash": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 0.827828,
+      "p95_ms": 1.029271,
+      "max_ms": 1.439878,
+      "pass_rate": 1.0
+    },
+    "fault-p1-wrong-scene": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 0.710469,
+      "p95_ms": 1.010883,
+      "max_ms": 1.146831,
+      "pass_rate": 1.0
+    },
+    "fault-continuity-structure": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 0.069471,
+      "p95_ms": 0.091397,
+      "max_ms": 0.11505,
+      "pass_rate": 1.0
+    },
+    "fault-semantic-structure": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "rejected": 100
+      },
+      "p50_ms": 0.068215,
+      "p95_ms": 0.095822,
+      "max_ms": 0.118719,
+      "pass_rate": 1.0
+    },
+    "fault-attestation-approval": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "10": 100
+      },
+      "p50_ms": 0.220256,
+      "p95_ms": 0.316655,
+      "max_ms": 0.349047,
+      "pass_rate": 1.0
+    },
+    "fault-attestation-payload": {
+      "attempts": 100,
+      "passed": 100,
+      "failed": 0,
+      "false_positive": 0,
+      "false_negative": 0,
+      "codes": {
+        "11": 100
+      },
+      "p50_ms": 0.284957,
+      "p95_ms": 0.388469,
+      "max_ms": 0.452502,
+      "pass_rate": 1.0
+    }
+  },
+  "failures": [],
+  "interpretation": {
+    "logical_trials": "Each trial executes a real repository validator/helper against canonical fixture bytes or a controlled mutation; this is not a claim that 5,000 browser renders were performed.",
+    "false_positive": "A clean canonical case was rejected.",
+    "false_negative": "A controlled tamper, binding, revocation or invalid-contract case was accepted."
+  },
+  "wall_time_s": 25.883117
+}