@ccslabs/xtend 0.1.0-rc.1 → 0.1.2

package/docs/en/rmt-vnext-primitive-lowering.md

@@ -0,0 +1,91 @@
+# RMT vNext Primitive Lowering
+
+- Contract: `xtend.rmt.vnext.primitive-lowering.v1`
+- Workpackage: `RMT-VNEXT-PRIM-04`
+- Status: `completed`
+- Source: Media Manager downstream transfer, `2026-05-19`
+
+## Goal
+
+`RMT-VNEXT-PRIM-04` lowers validated vNext primitives from the PRIM-03 semantic graph into deterministic Core, App Platform and Kernel records. App authors stay in RMT vNext; JSON remains compiler output, not the authoring path.
+
+The lowering stage is part of the existing vNext compiler:
+
+```js
+const {
+  compileRmtVNextSource
+} = require('./tools/rmt-language/vnext-compiler');
+
+const result = compileRmtVNextSource({ text, filePath });
+```
+
+For primitive sources, the compiler first builds `buildRmtVNextPrimitiveSemanticGraph(...)`. Error diagnostics from the graph stop lowering with `status: "semantic_error"` and without a Core document.
+
+## Core Domains
+
+The vNext Core receives additional primitive domains:
+
+- `states`
+- `selectors`
+- `actions`
+- `effects`
+- `portals`
+- `overlays`
+- `resources`
+
+Existing domains such as `surfaces`, `lanes`, `operations`, `events` and `dataSources` are enriched for primitives instead of creating a second runtime path.
+
+## App Platform Artifact
+
+When primitives are present, the Core document contains `appPlatform`:
+
+```json
+{
+  "schema": "xtend.rmt.vnext.app-platform-records.v1",
+  "sourceSyntax": "rmt-vnext",
+  "surfaces": [],
+  "events": [],
+  "resources": []
+}
+```
+
+This artifact is intended for host and scaffold adapters. Host-specific imports, such as lazy component imports, remain visible there and carry `kernelVisible: false`.
+
+## Kernel Records
+
+For primitive sources, the Core document also contains `kernelRecords`:
+
+```json
+{
+  "schema": "xtend.rmt.vnext.kernel-records.v1",
+  "boundary": "no-rmt-kernel-import-of-host-runtime-types",
+  "schedules": [],
+  "fibers": [],
+  "lifecycleRecords": []
+}
+```
+
+The Kernel record slice contains lane, fiber, lifecycle, state, selector, action, data source and resource facts without host-runtime imports. This prepares PRIM-05: Fabric can derive lane/fiber evidence from compiler output without the kernel importing Fabric or XTend.
+
+## Evidence
+
+The compiler suite verifies that:
+
+- the positive primitive fixture compiles to state, selector, data source, action, effect, portal, overlay, resource, surface and event records;
+- App Platform artifact and Kernel records are created;
+- keyed surface repeaters and event payload contracts remain intact;
+- host imports stay outside kernel visibility;
+- the negative PRIM-03 fixture stops before lowering with semantic diagnostics;
+- Core JSON remains byte-stable.
+
+Targeted local gates:
+
+```bash
+node --check tools/rmt-language/vnext-compiler.js
+node --check tests/rmt-language/rmt_vnext_compiler_suite.js
+node -e "const suite=require('./tests/rmt-language/rmt_vnext_compiler_suite'); const result=suite.runRmtVNextCompilerSuite({rootDir:process.cwd()}); process.exit(result.ok ? 0 : 1);"
+```
+
+## PRIM-06 Follow-up
+
+`RMT-VNEXT-PRIM-06` builds on `kernelRecords.schedules` and `kernelRecords.fibers`. The first source-to-sea slice produces evidence that correlates vNext source maps, Kernel schedules, derivable Fabric fibers, UI markers and a browser-smoke fixture under the same primitive ID.

package/docs/en/rmt-vnext-primitive-migration.md

@@ -0,0 +1,93 @@
+# RMT vNext Primitive Migration
+
+- Contract: `xtend.rmt.vnext.primitive-migration-preview.v1`
+- Workpackage: `RMT-VNEXT-PRIM-08`
+- Status: `in_progress`
+- Source fixture: `tests/fixtures/rmt-app-platform-tooling.rmt`
+- Local gate: `node scripts/run_xtend_tests.js rmt-vnext-compatibility --json`
+
+## Goal
+
+`RMT-VNEXT-PRIM-08` pushes Legacy and App Platform JSON authoring into the background. Existing App Platform primitive records remain available as compiler targets and compatibility evidence, but the primary authoring surface is RMT vNext.
+
+The migration path therefore does not create a new legacy work mode. It creates a vNext preview draft from existing App Platform records and immediately compiles that draft through the vNext compiler again. Only when this draft can lower state, data sources, actions, events, portals, overlays, resources, surfaces, lanes and Kernel records is the migration considered compatible.
+
+## Preview and Apply-Plan Contract
+
+The API `createAppPlatformPrimitiveMigrationPreview(...)` detects App Platform JSON documents with primitive domains such as `state`, `dataSources`, `actions`, `events`, `portals`, `overlays`, `resources` and `surfaces`.
+
+The preview report contains:
+
+- `schema: "xtend.rmt.vnext.primitive-migration-preview.v1"`
+- `workpackage: "RMT-VNEXT-PRIM-08"`
+- `languageMode: "legacy-app-platform-json"`
+- `vNextAuthoring.role: "default"`
+- `legacyAuthoring.role: "compiler-target"`
+- `legacyAuthoring.backgrounded: true`
+- `authoringDraft` with vNext primitive syntax
+- `authoringDraftCompileStatus: "compiled"`
+- `projection.appPlatform` and `projection.kernelRecords`
+- `domainMapping` for the detected primitive families
+
+Report-only mode writes no source. Instead, it reports `rmt.vnext.migration.opt_in_required` and `rmt.vnext.primitive_migration.preview_available`. Preview mode is deliberately opt-in so existing downstream fixtures are not migrated silently.
+
+The API `createAppPlatformPrimitiveMigrationApplyPlan(...)` builds on that and creates a deterministic apply plan:
+
+- `schema: "xtend.rmt.vnext.primitive-migration-apply-plan.v1"`
+- `status: "apply-plan-ready"` when the vNext draft compiles and a projection can be produced
+- `status: "blocked"` when JSON parsing or vNext compilation fails
+- `targetPath` as target path hint, by default `<source>.vnext.rmt`
+- `automaticWrite: false`
+- `writePolicy: "manual-apply-only"`
+
+The apply plan performs no file write. It is a verifiable handoff for agents, CLIs or editors that want to adopt the vNext draft into a new authoring path after review. Legacy/App Platform JSON remains mirror and compiler target, not the user-facing authoring surface.
+
+## Authoring Draft
+
+The App Platform fixture is converted into a vNext shell of this shape:
+
+```rmt
+template epic18.app-platform-tooling.fixture {
+  state items type collection initial []
+
+  datasource items from fixture records.generic-items {
+    contract "domain.record.generic-item.v1[]"
+  }
+
+  action load-items {
+    effect fetch datasource items
+    reduce state.items = result.records
+  }
+
+  portal app root "#app-root" layer surface
+
+  surface workspace kind window component workspace {
+    repeat from datasource items
+    portal app
+
+    lane visible weight 70 {
+      hydrate workspace from datasource items
+    }
+
+    on open-detail target ref.row -> action open-detail {
+      payload id from detail.id
+    }
+  }
+}
+```
+
+The preview normalizes known App Platform prefixes for vNext authoring: `state.items` becomes `state items`, `datasource.items` becomes `datasource items`, `surface.workspace` becomes `surface workspace` and `action.open-detail` becomes `action open-detail`. Runtime and Kernel records remain correlatable through the compiler projection.
+
+## Gate Expectation
+
+The compatibility gate verifies for PRIM-08 that:
+
+- App Platform primitive JSON is detected as `legacy-app-platform-json`.
+- Report-only mode requires explicit preview opt-in.
+- Preview mode creates a vNext draft and compiles it.
+- Apply-plan mode creates the same draft, a target path hint and a blocking compile/parse check without writing files.
+- The projection contains App Platform and Kernel records.
+- Legacy is marked as `compiler-target`, not as the authoring path.
+- The compatibility matrix accepts App Platform preview and native vNext source together.
+
+This makes the upgrade rule testable: Legacy remains provably compatible, but the developer experience for new app-shell work lives in vNext.

package/docs/en/rmt-vnext-primitive-parser-ast.md

@@ -0,0 +1,59 @@
+# RMT vNext Primitive Parser AST
+
+- Contract: `xtend.rmt.vnext-primitive-parser-ast.v1`
+- Workpackage: `RMT-VNEXT-PRIM-02`
+- Status: `completed`
+- Parent backlog: [RMT vNext Primitive Compiler Backlog](./rmt-vnext-primitives-compiler-backlog.md)
+- Grammar contract: [RMT vNext Primitive Grammar Design](./rmt-vnext-primitive-grammar-design.md)
+- Parser module: `tools/rmt-language/vnext-parser.js`
+- Parser suite: `tests/rmt-language/rmt_vnext_parser_suite.js`
+- Fixture: `tests/rmt-language/fixtures/vnext-primitives-grammar-design.rmt`
+
+## Goal
+
+PRIM-02 extends the existing vNext parser so the primitive syntax defined in PRIM-01 becomes visible as real AST nodes. The slice remains backward-compatible with the previous `template/surface/lane` syntax.
+
+Compiler lowering and kernel ingestion remain follow-up work in `RMT-VNEXT-PRIM-04`. The first app graph semantics started in `RMT-VNEXT-PRIM-03`.
+
+## Implemented Parser Scope
+
+The parser now accepts the design fixture `vnext-primitives-grammar-design.rmt` without syntax diagnostics and creates initial AST nodes for:
+
+| Primitive | AST nodes |
+|-----------|-----------|
+| State | `RmtStateDeclaration`, `RmtInitialBlock`, `RmtInitialValueEntry`, `RmtPrimitiveValue`, `RmtTypeReference` |
+| Selector | `RmtSelectorDeclaration`, `RmtSelectorWhereClause`, `RmtSelectorFindClause`, `RmtSelectorSortClause`, `RmtSelectorOutputClause` |
+| DataSource | `RmtDataSourceDeclaration`, `RmtPrimitiveSourceReference`, `RmtDataSourceMethodClause`, `RmtDataSourceContractClause`, `RmtDataSourceResultClause`, `RmtDataSourceFallbackClause` |
+| Action | `RmtActionDeclaration`, `RmtActionInputClause`, `RmtActionStatusClause`, `RmtEffectStatement`, `RmtReducerStatement`, `RmtEmitStatement`, `RmtActionResultHandler` |
+| Portal | `RmtPortalDeclaration`, `RmtPrimitiveAttribute`, `RmtPortalPolicyClause` |
+| Overlay | `RmtOverlayDeclaration`, `RmtOverlayPolicyClause` |
+| Resource | `RmtResourceDeclaration`, `RmtResourceImportClause`, `RmtResourceSourceClause`, `RmtResourceDisposeClause` |
+| Surface | extended `RmtSurfaceDeclaration`, `RmtSurfaceHeaderClause`, `RmtSurfaceSourceClause`, `RmtSurfaceRepeatClause`, `RmtSurfaceKeyClause`, `RmtSurfacePortalClause`, `RmtSurfaceBoundsClause`, `RmtSurfacePreserveClause`, `RmtSurfaceDestroyClause` |
+| Event | extended `RmtEventBinding`, `RmtEventSelector`, `RmtEventPayloadBlock`, `RmtEventPayloadMapping`, `RmtEventOptionClause` |
+
+## Parser Boundaries
+
+- Selector expressions such as `contains(record.name, state.media.filters.query)` are preserved in PRIM-02 as declarative raw clauses with source range. The semantic operator AST follows in PRIM-03.
+- Action result handlers such as `on success -> reduce ...` keep the effect text and source range. Typed effect nodes follow in PRIM-03/04.
+- The compiler continues to ignore unknown template body nodes so existing vNext Core builds remain stable. Primitive lowering is PRIM-04.
+- The RMT kernel is not ingested in PRIM-02. The source-to-sea gate is PRIM-06.
+
+## Verification
+
+Direct local checks:
+
+```bash
+node --check tools/rmt-language/vnext-parser.js
+node --check tests/rmt-language/rmt_vnext_parser_suite.js
+node -e "const { runRmtVNextParserSuite } = require('./tests/rmt-language/rmt_vnext_parser_suite'); const result = runRmtVNextParserSuite(); process.exit(result.ok ? 0 : 1)"
+node -e "const { runRmtVNextCompilerSuite } = require('./tests/rmt-language/rmt_vnext_compiler_suite'); const result = runRmtVNextCompilerSuite(); process.exit(result.ok ? 0 : 1)"
+```
+
+The global runner `node scripts/run_xtend_tests.js rmt-vnext-parser --json` is currently blocked by existing merge conflict markers in `tests/references/reference_path_suite.js`, because the runner imports this suite at startup.
+
+## Handoff
+
+- `RMT-VNEXT-PRIM-03`: [RMT vNext Primitive Semantic Graph](./rmt-vnext-primitive-semantic-graph.md)
+  builds indexes, references and pre-runtime diagnostics for the primitive AST nodes.
+- `RMT-VNEXT-PRIM-04`: lower primitive AST and semantic graph into Core/App Platform records.
+- Update the Language Server: completions, hover, symbols and code actions for the new node types.

package/docs/en/rmt-vnext-primitive-semantic-graph.md

@@ -0,0 +1,103 @@
+# RMT vNext Primitive Semantic Graph
+
+- Contract: `xtend.rmt.vnext.primitive-semantic-graph.v1`
+- Workpackage: `RMT-VNEXT-PRIM-03`
+- Status: `completed`
+- Source: Media Manager downstream transfer, `2026-05-19`
+
+## Goal
+
+`RMT-VNEXT-PRIM-03` makes vNext primitives semantically visible before runtime. The parser provides typed AST nodes; the new graph builds indexes, references, diagnostics and completion hints for the App Platform primitives from them.
+
+The API is additive:
+
+```js
+const {
+  buildRmtVNextPrimitiveSemanticGraph
+} = require('./tools/rmt-language/semantic-graph');
+
+const graph = buildRmtVNextPrimitiveSemanticGraph({ text, filePath });
+```
+
+The existing `buildSemanticGraph(...)` for Legacy/Core RMT remains stable. The vNext primitive graph is the new handoff between parser and later lowering into Kernel records.
+
+## Index Domains
+
+The graph creates dedicated domains for:
+
+- `states`
+- `selectors`
+- `dataSources`
+- `actions`
+- `surfaces`
+- `portals`
+- `overlays`
+- `resources`
+- `events`
+
+Each domain provides `records`, `byId`, `ids` and duplicate diagnostics. In addition, `catalogHints` provides directly usable IDs for completions and docs.
+
+## Reference Model
+
+PRIM-03 validates the most important vNext primitive edges:
+
+- Selector -> State
+- Action Effect -> DataSource
+- Reducer -> State
+- Surface Source/Repeat/Lifecycle -> Selector or State
+- Surface -> Portal
+- Overlay -> Portal
+- Resource -> Owner Surface/Overlay
+- Resource Source -> Selector/State/Resource
+- Surface Destroy -> Resource
+- Event -> Action
+
+The references follow the same structure as the existing semantic graph:
+
+- `references.records`
+- `references.bySourcePointer`
+- `references.byTargetId`
+- `references.unresolved`
+
+This keeps the later language-server path for go-to-definition, references and completion consistent.
+
+## Diagnostics
+
+PRIM-03 introduces these vNext primitive diagnostics:
+
+| Code | Meaning |
+|------|---------|
+| `rmt.vnext.primitive.duplicate-id` | A primitive is defined more than once in the same domain. |
+| `rmt.vnext.primitive.unknown-reference` | A primitive reference cannot be resolved. |
+| `rmt.vnext.primitive.owner-missing` | A resource has no surface or overlay owner. |
+| `rmt.vnext.primitive.unkeyed-repeat` | A surface repeater has no `key` clause. |
+| `rmt.vnext.primitive.payload-contract-missing` | An event or emit leads to a runtime payload without a contract. |
+| `rmt.vnext.primitive.initial-missing` | A state has no initial value. |
+| `rmt.vnext.primitive.resource-kind-missing` | A resource has no `kind`. |
+| `rmt.vnext.primitive.action-reducer-missing` | An action has input, status or effect but no reducer target. |
+| `rmt.vnext.primitive.effect-source-missing` | An `effect fetch` has no data source, selector or resource source. |
+| `rmt.vnext.primitive.unsafe-html` | Unsafe HTML becomes visible without a trust boundary. |
+| `rmt.vnext.primitive.kernel-boundary` | A primitive import violates Kernel/Fabric boundaries. |
+
+The negative fixture `tests/rmt-language/fixtures/vnext-primitives-semantic-invalid.rmt` proves cross-reference, ownership, event-payload, repeater and kernel-boundary errors as pre-runtime diagnostics.
+
+## Evidence
+
+The semantic-graph suite verifies both directions:
+
+- `tests/rmt-language/fixtures/vnext-primitives-grammar-design.rmt` must index as a positive vNext primitive graph without error diagnostics.
+- `tests/rmt-language/fixtures/vnext-primitives-semantic-invalid.rmt` must produce the expected PRIM-03 diagnostics.
+
+Targeted local gates:
+
+```bash
+node --check tools/rmt-language/semantic-graph.js
+node --check tests/rmt-language/rmt_semantic_graph_suite.js
+node -e "const suite=require('./tests/rmt-language/rmt_semantic_graph_suite'); const result=suite.runRmtSemanticGraphSuite({rootDir:process.cwd()}); process.exit(result.ok ? 0 : 1);"
+```
+
+## Next Handoff
+
+`RMT-VNEXT-PRIM-04` uses the new graph as lowering input. The compiler stops primitive sources with semantic errors before Core output and produces deterministic Kernel/App Platform records for valid sources.
+
+Handoff: [RMT vNext Primitive Lowering](./rmt-vnext-primitive-lowering.md)