@ccslabs/xtend 0.1.0-rc.1 → 0.1.2

package/docs/en/xtendrmt-app-dsl.md

@@ -0,0 +1,331 @@
+# XTendRMT App DSL Reference
+
+- Status: current after Epic 05 completion, updated vNext-first
+- Contract: `xtend.docs.xtendrmt-app-dsl.v1`
+- Schema source: `xtendrmt/rmt.schema.json`
+- Normalizer: `createRmtFormat().normalizeDocument(...)`
+
+## Purpose
+
+The App DSL describes a renderable application as an RMT document. New
+authoring work uses RMT vNext; legacy/App-DSL JSON remains the normalized core
+output, runtime registry, and compatibility surface. XTend UI, XRouter, Vanilla
+JS, and other hosts are connected through adapter records without the kernel
+importing host runtime.
+
+Since `WP-E13-09`, [RMT Production Readiness](./rmt-production-readiness.md)
+bundles this App DSL under `xtend.epic13.rmt-production-readiness.v1` as the
+RC1 boundary for shell-first app shell, native routes, components, Fabric/lanes,
+lifecycle telemetry, diagnostics, and artifact parity.
+
+## Minimal vNext Document
+
+```rmt
+template app.shell {
+  state app.ready type boolean initial true
+
+  portal surface.root root "#app-root" layer surface
+
+  surface app.home kind page component x-section {
+    source state app.ready
+    portal surface.root
+
+    lane visible weight 80 {
+      hydrate app-shell from state app.ready
+    }
+  }
+}
+```
+
+The compiler lowers this source into core domains such as `adapters`,
+`components`, `routes`, `schedules`, `surfaces`, and `templates`. These domains
+are stable for runtime adapters, but they are no longer the preferred writing
+surface.
+
+## Native Domains as Compiler Output
+
+| Domain | Responsibility |
+|--------|----------------|
+| `adapters` | host capabilities, runtime surface, capability negotiation |
+| `components` | domain component records, host adapter, hydration hints |
+| `routes` | navigation, route targets, query/params, schedule references |
+| `schedules` | reusable scheduler policies |
+| `templates` | `dom_descriptor`, props, slots, bindings, hydration contracts |
+| `surfaces` | SurfaceManager, window, panel, and overlay handoffs |
+
+`manifest.metadata` remains valid for descriptions, handoff, history, and demo
+notes. New operational routes, components, and schedules belong in vNext source
+and are projected from there into the registry.
+
+## Adapter Records
+
+In vNext, adapters become visible implicitly through surface, route, and
+endpoint usage. The registry then contains stable adapter records for hosts:
+
+- `xtend.xrouter`
+- `xtend.component`
+- `xtend.surface`
+- `rmt.state-scheduler-diagnostics`
+- `vanilla.component`
+
+`kernelVisible: false` is the default for host-specific adapter data. The
+kernel may index these records, but it must not load host runtime.
+
+## Component Records
+
+A vNext surface describes the domain host element:
+
+```rmt
+template settings.components {
+  portal surface.root root "#settings-root" layer surface
+
+  surface settings.card kind card component x-card {
+    portal surface.root
+
+    lane visible weight 80 {
+      mount x-card
+      hydrate settings-card from endpoint xtendrmt.component.hydrate
+    }
+  }
+}
+```
+
+The normalizer turns this into runtime registry entries that can be consumed
+through `componentRegistry.byAdapter["xtend.component"]` and
+`componentRegistry.byTag["x-card"]`.
+
+## Route Records
+
+Route metadata remains declarative and can be tied to surfaces:
+
+```rmt
+template settings.routes {
+  state settings.tab type string initial "profile"
+
+  portal surface.root root "#app-root" layer surface
+
+  surface settings.page kind page component x-section {
+    source state settings.tab
+    portal surface.root
+
+    lane visible weight 80 {
+      hydrate route-view from endpoint xtendrmt.route.render
+      hydrate settings-shell from state settings.tab
+    }
+  }
+}
+```
+
+The normalizer creates route and schedule indexes for
+`routeRegistry.byRouter["xtend.xrouter"]`, `routeRegistry.byId[...]`, and
+`routeRegistry.byPath[...]`. `title`, `documentTitle`, `titleTemplate`,
+`metaDescription`, and `metaKeywords` remain declarative route metadata:
+XRouter writes `document.title` and `description`/`keywords` from them without
+RMT importing XTend or XRouter.
+
+## Schedule Records
+
+Schedules are `lane` and lifecycle clauses in vNext:
+
+```rmt
+template scheduler.page {
+  portal surface.root root "#app-root" layer surface
+
+  surface app.shell kind page component x-section {
+    portal surface.root
+
+    lane visible weight 80 {
+      hydrate route-shell from endpoint xtendrmt.route.render
+    }
+
+    lane idle weight 20 {
+      hydrate help-panel from endpoint xtendrmt.component.hydrate
+    }
+  }
+}
+```
+
+Proven endpoint names remain:
+
+- `xtendrmt.route.render`
+- `xtendrmt.component.mount`
+- `xtendrmt.component.hydrate`
+- `xtendrmt.vanilla.mount`
+- `xtendrmt.diagnostics.snapshot`
+- `xtendrmt.template.inspect`
+
+## Template Records and Trusted DOM
+
+New templates are lowered from vNext surfaces and slots into
+`dom_descriptor` output:
+
+```rmt
+template settings.template {
+  portal surface.root root "#settings-root" layer surface
+
+  surface settings.shell kind page component x-card {
+    portal surface.root
+
+    lane visible weight 80 {
+      hydrate settings-card from endpoint xtendrmt.component.hydrate {
+        slot header hydrate settings-header
+        slot body hydrate settings-body
+      }
+    }
+  }
+}
+```
+
+`html_fragment` remains compatible, but requires an explicit Trusted DOM
+boundary:
+
+```rmt
+template settings.legacyHtml {
+  surface settings.legacy kind card component x-card {
+    lane visible weight 60 {
+      hydrate legacy-fragment from endpoint docs.parse {
+        trust boundary "xtend.security.sanitizing-boundary.v1"
+        sanitize html_fragment
+      }
+    }
+  }
+}
+```
+
+The kernel may normalize and schedule such records. Sanitizing, Trusted DOM,
+and concrete DOM sinks remain host adapter work. See
+[Trusted DOM and Sanitizing](./trusted-dom-sanitizing.md).
+
+## Shell-First Host Apps
+
+The Docs App uses this path productively as a shell-first pilot.
+`docs/xtendrmt-parsedown-docs.rmt` describes `docs.app.shell` as a
+`dom_descriptor`, `docs.header.search` as the header search slot template, and
+`docs.media.lazy` as a future-ready slot for XPlayer tutorials.
+`docs/utils/pageloader.js` renders the RMT shell first and then inserts
+Parsedown HTML only into the `data-rmt-slot="content"` slot.
+
+Important: RMT remains framework-agnostic in this mode too. Parsedown,
+rich-HTML sinks, XPlayer lazy loading, and concrete DOM events are executed by
+the host adapter. RMT provides shell records, slots, schedules, and diagnostics.
+
+## RMT-First XTend Apps
+
+Since Epic 10, app authoring for complete XTend apps is described by contract
+`xtend.rmt.first-class-app-authoring.v1`. The goal is an app whose shell,
+routes, components, templates, events, commands, hydration policies, Fabric
+lanes, and diagnostics all live in RMT.
+
+The reference path is `tests/fixtures/rmt-first-class-xtend-app.rmt`. The gate
+is:
+
+```bash
+node scripts/run_xtend_tests.js rmt-first-class-app --json
+```
+
+The contract intentionally stays host-neutral. RMT knows `xtend.component`,
+`xtend.xrouter`, and `rmt.state-scheduler-diagnostics` as adapter records, but
+it imports no XTend components and no XRouter module into the kernel. Details
+live in `development/XTend-RMT-First-Class-App-Authoring.md`.
+
+The canonical developer guide for complete XTend apps is
+[RMT-First XTend Apps](./rmt-first-xtend-apps.md). The Epic 10 completion and
+release gates are documented in
+[Epic 10 Release Handoff](./epic10-release-handoff.md).
+
+## Component Fabric Context
+
+XTend Components can receive Fabric hints from vNext lanes and legacy metadata:
+
+```rmt
+template pages.settings {
+  portal surface.root root "#app-root" layer surface
+
+  surface pages.settings kind page component x-form {
+    portal surface.root
+
+    lane idle weight 40 {
+      hydrate settings-form from endpoint xtendrmt.component.hydrate
+    }
+  }
+}
+```
+
+The productive adapter resolves this data through
+`xtend.component.fabric-lane-ingestion.v2`. RMT schedule records take priority
+over component metadata, runtime overrides, static contracts, and scaffold
+defaults. The gate is
+`node scripts/run_xtend_tests.js rmt-component-fabric-ingestion --json`.
+
+## Component Lifecycle Telemetry
+
+Starting with `xtend.component.lifecycle-telemetry.v1`, the adapter creates
+lifecycle records for component work. RMT documents do not need to import XTend
+code for this; they only provide component, route, schedule, and Fabric context.
+The host can pass `telemetryCollector`, `recordTelemetry`, or a Fabric instance.
+
+```js
+const records = [];
+adapter.mountComponent(root, 'pages.settings', model, {
+  mapping,
+  telemetryCollector: records
+});
+
+const snapshot = fabric.createTelemetrySnapshot({
+  componentTelemetry: records
+});
+```
+
+`snapshot.componentTelemetry` aggregates `mount`, `hydrate`, `render`,
+`update`, `event`, `unmount`, and `error` by operation, component, and lane.
+Component errors, deadline misses, and explicit `backpressureSignal` metadata
+can create backpressure. The gate is
+`node scripts/run_xtend_tests.js rmt-component-lifecycle-telemetry --json`.
+
+Hosts can pass Fabric snapshots directly to the productive RMT bridge:
+
+```js
+bridge.recordTelemetrySnapshot(snapshot, {
+  scheduleRef: "diagnostics.snapshot"
+});
+```
+
+The bridge mirrors `rmt.telemetry.lastSnapshot` and `rmt.backpressure.*` from
+that and schedules the diagnostics snapshot endpoint when needed.
+
+## Runtime Registry
+
+The registry is the consumable boundary between DSL and adapter. Adapters read
+registry entries, not raw demo metadata:
+
+```js
+const format = createRmtFormat();
+const normalizedDocument = format.normalizeDocument(document);
+const registry = format.createRuntimeRegistries(normalizedDocument, {
+  requiredRoutes: ["settings", "/settings"],
+  requiredComponents: ["settings.card", "x-card"]
+});
+```
+
+## Diagnostics
+
+The App DSL normalizer creates diagnostics instead of forcing host execution.
+Important groups:
+
+- `rmt.dsl.reference.*` for missing or invalid references
+- `rmt.runtime.registry.*` for registry conflicts or missing required refs
+- `rmt.xrouter.*` for route mapping and navigation
+- `rmt.xtend.component.*` for component mapping, mounting, and hydration
+- `rmt.bridge.*` for the state, scheduler, and diagnostics bridge
+
+## Review Checklist
+
+- App shell examples are `rmt` and vNext-first.
+- Runtime registry JSON is classified as generated output or compatibility
+  surface.
+- Routes reference components and schedules only by ID.
+- Components reference host adapters only by ID.
+- Schedule policies are central and reusable.
+- XTend-specific data stays outside the kernel.
+- Non-XTend hosts receive their own adapters instead of XTend fallbacks.
+- `node scripts/run_xtend_tests.js rmt-compatibility --json` passes.

package/docs/en/xtendrmt-migration-guide.md

@@ -0,0 +1,256 @@
+# XTendRMT Native Migration Guide
+
+- Status: production after Epic 05 completion
+- Contract: `xtend.rmt.native-migration-guide.v1`
+- Minimum gates:
+  - `node scripts/run_xtend_tests.js rmt-compatibility --json`
+  - `node scripts/run_xtend_tests.js references --json`
+  - `npm test`
+
+## Purpose
+
+This guide describes the migration from early XTendRMT metadata paths to native RMT top-level domains and onward to RMT vNext. The migration is additive and opt-in: existing XTend, React, Vue, Vanilla JS and custom apps may keep running while new app shells are written in RMT vNext. `adapters`, `components`, `routes`, `schedules` and `templates` remain runtime registry, compiler output and compatibility mirror.
+
+The current product overview is in [XTendRMT Developer Overview](./xtendrmt-overview.md). App DSL details are in [XTendRMT App-DSL Reference](./xtendrmt-app-dsl.md), and production adapter/bridge wiring is in [XTendRMT Runtime Bridge](./xtendrmt-runtime-bridge.md).
+
+## Target State
+
+New documents should use these sources:
+
+| Area | Target Source |
+|------|---------------|
+| Host Adapter | vNext surface/endpoint usage, output: `adapters` |
+| XTend Components | `surface ... component x-*`, output: `components` with `adapter: "xtend.component"` |
+| XRouter Routes | vNext shell/route surface, output: `routes` with `router: "xtend.xrouter"` |
+| Scheduler Policies | `lane` and lifecycle operations, output: `schedules` |
+| Markup or Fragments | `surface`, `slot`, `trust boundary`, output: `templates` |
+| Description, Handoff, History | `manifest.metadata` |
+
+`manifest.metadata` remains valid for product descriptions, handoff notes, demo history and deliberately historical pilot data. Operative routes, components and schedules should not be created there anymore. Template-only documents remain compatible.
+
+## vNext Target Shape
+
+```rmt
+template settings.migration {
+  state settings.tab type string initial "profile"
+
+  portal surface.root root "#settings-root" layer surface
+
+  surface settings.card kind card component x-card {
+    source state settings.tab
+    portal surface.root
+
+    lane visible weight 80 {
+      hydrate settings-card from state settings.tab
+    }
+  }
+}
+```
+
+The JSON examples in the following steps are intentionally marked as legacy input or runtime-registry output. They show migration evidence, not the new writing style.
+
+## Migration Matrix
+
+| Starting Point | Migration |
+|----------------|-----------|
+| Template-only `.rmt` document | remains valid; add native domains only when App DSL requires them |
+| `manifest.metadata.routes` | move to `routes` |
+| `manifest.metadata.components` | move to `components` |
+| `manifest.metadata.schedules` | move to `schedules` |
+| XRouter-specific demo initialization | replace with `createRmtXRouterAdapter` |
+| XTend-specific demo mount logic | replace with `createRmtXtendComponentAdapter` |
+| manual scheduler/state bridge | replace with `createRmtStateSchedulerDiagnosticsBridge` |
+| non-XTend host | declare a custom adapter such as `vanilla.component` |
+
+## Step 1: Make Adapters Explicit in Registry Output
+
+Old demo metadata often contains implicit knowledge such as "this route uses XRouter" or "this component is XTend". The first migration step is an explicit adapter record.
+
+```json
+{
+  "id": "xtend.component",
+  "kind": "component_adapter",
+  "runtimeSurface": ["esm", "browser_classic"],
+  "providedCapabilities": ["components", "customElements", "hydration", "scheduleRefs"],
+  "kernelVisible": false
+}
+```
+
+`kernelVisible: false` is mandatory for host-specific adapter data. It means the kernel may validate and index the record, but must not import the XTend runtime.
+
+## Step 2: Lift Components out of Legacy Metadata
+
+Before: legacy `manifest.metadata`
+
+```json
+{
+  "manifest": {
+    "metadata": {
+      "components": [
+        {
+          "id": "settings.card",
+          "tag": "x-card"
+        }
+      ]
+    }
+  }
+}
+```
+
+After: compatibility/registry output
+
+```json
+{
+  "components": [
+    {
+      "id": "settings.card",
+      "kind": "custom_element",
+      "adapter": "xtend.component",
+      "tag": "x-card",
+      "schedule": "component.idle.hydrate"
+    }
+  ]
+}
+```
+
+This lets `createRmtFormat().createRuntimeRegistries(...)` expose the component through `componentRegistry.byAdapter["xtend.component"]`.
+
+## Step 3: Lift Routes out of Legacy Metadata
+
+Before: legacy `manifest.metadata`
+
+```json
+{
+  "manifest": {
+    "metadata": {
+      "routes": [
+        {
+          "path": "/settings",
+          "component": "settings.card"
+        }
+      ]
+    }
+  }
+}
+```
+
+After: compatibility/registry output
+
+```json
+{
+  "routes": [
+    {
+      "id": "settings",
+      "path": "/settings",
+      "router": "xtend.xrouter",
+      "component": "settings.card",
+      "template": "settings.shell",
+      "schedule": "route.visible.render"
+    }
+  ]
+}
+```
+
+This lets `createRmtXRouterAdapter` map the route through `routeRegistry.byRouter["xtend.xrouter"]` and pass it to XRouter with `registerRoutes`.
+
+## Step 4: Centralize Schedules
+
+Endpoint hints often used to live in route, component or template metadata. After migration, the policy lives centrally in `schedules`.
+
+```json
+{
+  "schedules": [
+    {
+      "id": "route.visible.render",
+      "endpointName": "xtendrmt.route.render",
+      "lane": "visible",
+      "priority": 80,
+      "preferIdle": false
+    },
+    {
+      "id": "component.idle.hydrate",
+      "endpointName": "xtendrmt.component.hydrate",
+      "lane": "idle",
+      "priority": 40,
+      "preferIdle": true
+    }
+  ]
+}
+```
+
+Routes and components now only reference `schedule`. Execution runs through `createRmtStateSchedulerDiagnosticsBridge`.
+
+## Step 5: Remove Demo Bridge Logic
+
+Permanent demo bridge logic should no longer carry new product logic. A migrated host start uses the production factories:
+
+```js
+const format = createRmtFormat();
+const normalizedDocument = format.normalizeDocument(document);
+const registry = format.createRuntimeRegistries(normalizedDocument);
+
+const routes = createRmtXRouterAdapter({ routerElement }).registerRoutes(registry);
+const components = createRmtXtendComponentAdapter({ document, manifest }).mapComponents(registry);
+const bridge = createRmtStateSchedulerDiagnosticsBridge({ schedules: normalizedDocument.schedules });
+
+bridge.recordAdapterResult(routes, { scheduleRef: 'route.visible.render' });
+```
+
+If special logic is still necessary, it belongs in an adapter or in upstream RMT source, not in a demo file.
+
+## Best-Case Reference
+
+`xtendrmt/xtendrmt-bestcase-demo.rmt` is the production authoring reference for RMT vNext:
+
+- the `.rmt` file uses `template`, `surface`, `lane`, lifecycle operations, slots, event actions, state, selectors, actions, data sources, portals, overlays, resources and remote surfaces instead of JSON
+- `xtendrmt/xtendrmt-bestcase-demo.core.json` is the byte-stable vNext Core output
+- the browser demo projects vNext Core to `adapters`, `components`, `routes`, `schedules`, the Component Capability Registry, the Player Contract and resource ownership at runtime
+- `createRmtXRouterAdapter`, `createRmtXtendComponentAdapter`, `createRmtStateSchedulerDiagnosticsBridge` and `createRmtComponentCapabilityRegistry` remain the production adapter paths
+- `nativeDemoMigration` is retained in the runtime projection as handoff metadata
+
+`tests/browser/fixtures/rmt-xrouter-xtend-smoke.html` is the browser-near regression for the migrated path. It additionally verifies `vanilla.component` so the target state remains framework-agnostic.
+
+For the official Docs App, Parsedown remains the active Markdown parser, but the visible app shell is rendered shell-first from `docs.app.shell` in the RMT document. The RMT scheduling and shell path for Parsedown, search and future-ready media slots is documented through [XTendRMT Parsedown Scheduling Pilot](./xtendrmt-parsedown-scheduling.md) and remains host-neutral.
+
+## SurfaceManager Migration
+
+Starting with `WP-SM-09`, the SurfaceManager has its own migration guide: [SurfaceManager Migration Guide](./surface-manager-migration-guide.md) (`docs/surface-manager-migration-guide.md`).
+
+The Surface path is additive:
+
+- existing `components[*].metadata.surface` records remain valid
+- native `surfaces[*]` records are preferred for complex app shells
+- dual records keep `id`, `type`, `manager`, `component`, `route`, `schedule` and `stateKey` synchronized
+- `xtend.surface` remains a `surface_adapter` handoff until a production adapter runtime is implemented
+
+## What Does Not Need Migration
+
+Not every file immediately needs native domains.
+
+Do not migrate:
+
+- pure template-only documents without routing or component needs
+- historical demos with `manual-legacy` status
+- metadata that only contains product descriptions or handoff notes
+- React, Vue, Vanilla or custom hosts that do not yet use RMT as scheduler
+
+Migrate:
+
+- new `.rmt` App DSL documents
+- demo code with operative route/component bridges
+- production route or component flows
+- host paths that need scheduler endpoint policies
+
+## Review Checklist
+
+Before closing a migration, verify that:
+
+- `manifest.metadata.routes -> routes` has been completed
+- `manifest.metadata.components -> components` has been completed
+- `manifest.metadata.schedules -> schedules` has been completed
+- `xtend.xrouter` and `xtend.component` are adapter records, not kernel knowledge
+- `route.visible.render` and `component.idle.hydrate` are central policies
+- `createRmtXRouterAdapter`, `createRmtXtendComponentAdapter` and `createRmtStateSchedulerDiagnosticsBridge` replace demo bridge logic
+- template-only compatibility remains intact
+- React, Vue, Vanilla JS and custom hosts are not forced into the XTend migration
+- `node scripts/run_xtend_tests.js rmt-compatibility --json` and `node scripts/run_xtend_tests.js references --json` run