solid-tag-runtime 0.0.1

package/ARCHITECTURE.md

@@ -0,0 +1,608 @@
+# solid-tag-runtime Architecture
+
+This file is the engineering handoff and design record for `solid-tag-runtime`.
+
+**Maintenance rule:** update this document whenever we change the public runtime API, module record model, module resolution/linking semantics, execution backend, compiler adapter contract, caching/invalidation behavior, supported source formats, host-module behavior, or an important architectural invariant.
+
+## 1. Purpose
+
+`solid-tag-runtime` provides a runtime ES-module-like environment for source modules that may contain ordinary Solid JSX.
+
+It is intentionally separate from `solid-tag`:
+
+```text
+solid-tag
+  syntax transformation
+  JSX → @solidjs/html tagged templates
+
+solid-tag-runtime
+  module definition
+  dependency resolution
+  linking
+  execution
+  caching
+  invalidation
+  host-module injection
+```
+
+The runtime's primary abstraction is a **module**, not a component. Components are normal exports of modules.
+
+## 2. Goals
+
+The runtime should allow applications to:
+
+1. define JSX source modules from strings
+2. import those modules and receive module namespace objects
+3. allow one runtime module to import another runtime module
+4. expose the host application's existing Solid runtime to dynamic source
+5. expose host components, signals, callbacks, services, and data as modules
+6. reuse compiled/evaluated modules through caching
+7. invalidate and rebuild a module graph when source changes
+8. inspect dependencies and compiled source
+9. support normal JavaScript modules in the same graph
+10. keep the JSX compiler replaceable behind a narrow adapter
+
+## 3. Non-goals for the first version
+
+The initial package does not attempt to provide:
+
+- a Vite/Rollup/esbuild integration
+- a security sandbox
+- Node package resolution
+- automatic npm installation
+- automatic unresolved-variable scope rewriting
+- circular runtime source-module support
+- transparent live replacement of already-held component references
+- full HMR propagation semantics
+- script-tag discovery/registration
+
+These may be added later without changing the module-first public model.
+
+## 4. Package relationship
+
+```text
+Application
+   │
+   ├── host Solid runtime
+   │     ├── solid-js
+   │     ├── @solidjs/web
+   │     └── @solidjs/html
+   │
+   └── solid-tag-runtime
+          │
+          ├── solid-tag
+          │    └── JSX → html``
+          │
+          └── runtime module graph
+```
+
+`solid-tag-runtime` depends on `solid-tag` but deliberately does not depend on or bundle its own copy of Solid.
+
+The host application can register its existing Solid namespaces:
+
+```ts
+runtime.defineModule("solid-js", Solid);
+runtime.defineModule("@solidjs/web", SolidWeb);
+runtime.defineModule("@solidjs/html", { default: html });
+```
+
+This avoids duplicate Solid runtimes.
+
+## 5. Public API model
+
+The first API surface is centered on `createRuntime()`:
+
+```ts
+const runtime = createRuntime(options);
+```
+
+### Source modules
+
+```ts
+runtime.define(id, source, options?);
+runtime.update(id, source, options?);
+```
+
+### Host namespace modules
+
+```ts
+runtime.defineModule(id, namespace);
+```
+
+### URL/native modules
+
+```ts
+runtime.defineUrl(id, url);
+```
+
+### Loading
+
+```ts
+await runtime.import(id);
+await runtime.toModule(source, options?);
+await runtime.toComponent(source, options?);
+```
+
+### Tooling/introspection
+
+```ts
+await runtime.compile(id);
+runtime.modules();
+runtime.dependencies(id);
+runtime.dependents(id);
+runtime.getModuleInfo(id);
+```
+
+### Lifecycle
+
+```ts
+runtime.invalidate(id);
+runtime.dispose();
+```
+
+## 6. Module kinds
+
+The runtime currently has three module record kinds.
+
+### 6.1 Source module
+
+A string containing JavaScript with optional JSX:
+
+```ts
+runtime.define("/Button.jsx", source);
+```
+
+Source modules are analyzed, linked, optionally transformed with `solid-tag`, converted to a module URL, and evaluated by the native module loader.
+
+### 6.2 Host module
+
+An already-existing namespace/object from the application:
+
+```ts
+runtime.defineModule("@app/components", {
+  Button,
+  Card,
+});
+```
+
+The runtime generates a tiny bridge ES module which reads those values from a runtime-specific registry on `globalThis` and re-exports them.
+
+The bridge lets native dynamic modules import application-owned JavaScript references without serializing them.
+
+### 6.3 URL module
+
+A module ID mapped directly to a URL:
+
+```ts
+runtime.defineUrl("lib", "https://example.test/lib.js");
+```
+
+The runtime rewrites imports of `lib` to that URL and lets the native module loader handle it.
+
+## 7. Source formats
+
+### `jsx`
+
+Default.
+
+The runtime links imports, then passes source through the compiler adapter. The default compiler adapter is `solid-tag` and injects `@solidjs/html` when JSX is transformed.
+
+### `js`
+
+No JSX transformation. Imports are still resolved and rewritten through the runtime module graph.
+
+This permits ordinary JavaScript and JSX modules to coexist in one graph.
+
+## 8. Compiler adapter boundary
+
+The runtime engine itself does not depend on Acorn or the `solid-tag` AST.
+
+It requires a compiler adapter with two operations:
+
+```ts
+interface RuntimeCompiler {
+  analyze(source, context): {
+    imports: ImportReference[];
+    needsHtmlRuntime?: boolean;
+  };
+
+  transform(source, context): {
+    code: string;
+    diagnostics?: unknown[];
+  };
+}
+```
+
+The default adapter lives in `src/compiler.js` and uses:
+
+```text
+solid-tag.parseJSX()
+solid-tag.transformModule()
+```
+
+### Why this boundary exists
+
+It allows future compiler implementations without rewriting the runtime module system.
+
+Possible future adapters include:
+
+- a TSX-capable `solid-tag` backend
+- another JSX parser
+- pre-tagged source processing
+- reverse/tagged transformations for tooling
+
+The module graph must remain independent of parser details.
+
+## 9. Linking pipeline
+
+For a source module `/features/Counter.jsx`:
+
+```text
+source
+  ↓
+compiler.analyze()
+  ↓
+collect static imports / re-exports / literal dynamic imports
+  ↓
+resolve each specifier
+  ↓
+ensure virtual dependencies have module URLs
+  ↓
+rewrite import string literals to linked URLs
+  ↓
+solid-tag transformation (for JSX format)
+  ↓
+create native module URL
+  ↓
+import(moduleUrl)
+```
+
+Example graph:
+
+```text
+/features/Counter.jsx
+  ├── solid-js             → host bridge module
+  ├── @solidjs/html        → host bridge module
+  └── ../ui/Button.jsx     → compiled virtual source URL
+```
+
+## 10. Resolution rules
+
+Resolution order is conceptually:
+
+1. custom `resolve()` callback
+2. exact registered module ID
+3. absolute URL
+4. absolute virtual path
+5. relative path resolved against the importing virtual module
+6. unresolved bare import left native when `allowNativeImports === true`
+7. otherwise resolution error
+
+Relative virtual resolution uses POSIX-style paths regardless of the host operating system.
+
+This keeps module IDs stable in browsers and on Windows hosts.
+
+## 11. Native ESM execution backend
+
+The first execution backend deliberately relies on the JavaScript engine's native ESM evaluator rather than `new Function()`.
+
+Browser default:
+
+```text
+compiled source
+  ↓
+Blob
+  ↓
+blob: URL
+  ↓
+import(blobUrl)
+```
+
+Node/test default:
+
+```text
+compiled source
+  ↓
+data:text/javascript URL
+  ↓
+import(dataUrl)
+```
+
+The URL creation mechanism is abstracted by `ModuleUrlBackend` so another execution strategy can be supplied later.
+
+### Why native modules
+
+Benefits:
+
+- real module namespace objects
+- native `import` / `export` behavior for the supported graph
+- top-level code is evaluated as a module
+- no `eval` / `new Function()` runtime wrapper
+- clean `toModule()` semantics
+
+## 12. Circular dependency limitation
+
+Virtual source module cycles are explicitly rejected in v0.0.1.
+
+Reason: the current linker must know the final URL of dependencies before it can create a module's immutable Blob/data URL. A cycle requires module identities to exist before final source URLs can be constructed.
+
+Example currently rejected:
+
+```text
+/A.jsx → /B.jsx → /A.jsx
+```
+
+Future options:
+
+1. runtime linker/backend instead of native URL rewriting
+2. service-worker-backed stable virtual URLs
+3. an internal module server
+4. another backend capable of stable preallocated module identities
+
+Do not add ad-hoc cycle handling that breaks module semantics; change the backend deliberately.
+
+## 13. Host module bridges
+
+`defineModule()` stores the namespace in a runtime-specific registry:
+
+```text
+globalThis.__solidTagRuntimeHostModules__
+  ↓
+runtimeId
+  ↓
+moduleId
+  ↓
+namespace object
+```
+
+A generated bridge module re-exports identifier-safe properties.
+
+### Important invariant
+
+Host values are passed as **references**, not serialized copies.
+
+This is what allows dynamic source to import:
+
+- signal accessor functions
+- setters
+- stores
+- components
+- callbacks
+- objects/services
+
+### Current limitation
+
+The ES module bridge captures each exported property value when the bridge module evaluates. Mutating the namespace object later does not create ESM live bindings.
+
+To replace exports, call `defineModule()` again. This invalidates dependent runtime modules.
+
+## 14. Scope injection design choice
+
+The runtime does not automatically turn unknown identifiers into scope lookups.
+
+Rejected initial design:
+
+```tsx
+<div>{hostVariable}</div>
+```
+
+with compiler magic rewriting `hostVariable` to `scope.hostVariable`.
+
+Chosen design:
+
+```ts
+runtime.defineModule("@app/scope", {
+  hostVariable,
+});
+```
+
+and runtime source explicitly imports it:
+
+```ts
+import { hostVariable } from "@app/scope";
+```
+
+Reasons:
+
+- explicit dependencies
+- no lexical-scope guessing
+- easier module graph introspection
+- actual JavaScript references cross the boundary
+- components and state use the same mechanism
+- fewer compiler responsibilities
+
+A future convenience API may create scope modules automatically, but it should still compile down to module semantics.
+
+## 15. Dependency graph
+
+Each record tracks:
+
+```text
+record.dependencies
+record.dependents
+```
+
+When linking `/App.jsx` imports `/Button.jsx`:
+
+```text
+/App.jsx.dependencies includes /Button.jsx
+/Button.jsx.dependents includes /App.jsx
+```
+
+These edges power invalidation and tooling.
+
+## 16. Caching
+
+A source module caches:
+
+- analyzed/linked compiled source
+- generated module URL
+- import promise
+- loaded namespace when directly imported through the runtime
+
+Repeated imports use the same module URL/import promise until invalidated.
+
+Native ESM also caches imports by URL.
+
+## 17. Invalidation and updates
+
+Updating a module invalidates:
+
+1. that module's generated URL/import cache
+2. dependent runtime modules recursively
+
+The next `runtime.import(entry)` rebuilds the affected graph with new URLs.
+
+### Important invariant
+
+Already-held module namespaces/components are not mutated.
+
+Example:
+
+```ts
+const old = await runtime.import("/App.jsx");
+runtime.update("/Button.jsx", newSource);
+const fresh = await runtime.import("/App.jsx");
+```
+
+`old.App` remains the old module identity. `fresh.App` comes from the rebuilt graph.
+
+This is deliberate and matches the immutable identity of native evaluated modules.
+
+## 18. `toModule()` semantics
+
+`toModule(source)` is sugar for:
+
+```text
+generate anonymous module ID
+  ↓
+runtime.define(id, source)
+  ↓
+runtime.import(id)
+  ↓
+return module namespace
+```
+
+It is asynchronous by design.
+
+## 19. `toComponent()` semantics
+
+`toComponent()` calls `toModule()` and returns a selected callable export.
+
+Default:
+
+```text
+exportName = "default"
+```
+
+Named exports require an explicit `exportName`.
+
+The runtime does not guess between multiple component exports.
+
+## 20. Security model
+
+`solid-tag-runtime` is **not a sandbox**.
+
+Dynamic source executes as JavaScript in the page/module environment and has whatever capabilities its imports/globals provide.
+
+Do not use this package to execute untrusted hostile code without a separate isolation mechanism such as a suitably sandboxed iframe/worker/process boundary.
+
+## 21. CSP considerations
+
+The browser backend currently uses Blob module URLs by default.
+
+Applications with strict Content Security Policy may need:
+
+- an allowed `blob:` module policy, or
+- a custom `ModuleUrlBackend`, or
+- a future stable virtual-module backend
+
+The public module API should not depend on Blob URLs so this backend can change later.
+
+## 22. Future directions
+
+Potential additions, in rough architectural order:
+
+### Stable/cycle-capable module backend
+
+Needed before claiming full ESM graph compatibility.
+
+### URL fetching/loading
+
+Possible API:
+
+```ts
+await runtime.load("/App.jsx");
+```
+
+with configurable fetch/source providers.
+
+### Script-tag module registration
+
+Possible declarative zero-build API:
+
+```html
+<script type="solid-jsx" module="/Button.jsx">...</script>
+```
+
+implemented as a thin source-provider layer over the same runtime graph.
+
+### Runtime update/HMR helpers
+
+Graph invalidation already exists. Future APIs can add lifecycle hooks and entry re-rendering without changing module identity rules.
+
+### Tagged-source modules
+
+A format for source that already contains `html\`\`` templates and only needs module linking/execution.
+
+### TSX compiler adapter
+
+The runtime compiler boundary allows a future TSX-capable transformer without coupling module linking to TypeScript or another parser.
+
+### Bidirectional JSX/tagged tooling
+
+If `solid-tag` later gains tagged-template → JSX transformation, runtime/editor tooling can expose source views without changing module execution semantics.
+
+## 23. Decision log
+
+### 2026-10-02 — Separate runtime package
+
+Decision: create `solid-tag-runtime` instead of adding module execution to `solid-tag`.
+
+Reason: keep syntax transformation small and reusable while allowing the runtime package to grow a module graph, execution backend, caching, and lifecycle APIs.
+
+### 2026-10-02 — Modules are the primary abstraction
+
+Decision: components are module exports; there is no separate component registry at the linker level.
+
+Reason: normal imports/exports naturally solve component composition and also work for utilities, state, and services.
+
+### 2026-10-02 — Host Solid is injected as modules
+
+Decision: do not bundle a private Solid runtime in `solid-tag-runtime`.
+
+Reason: dynamically compiled components must share the host application's reactive ownership/runtime graph.
+
+### 2026-10-02 — Scope is represented as modules
+
+Decision: prefer `defineModule("@app/scope", values)` plus explicit imports instead of compiler rewriting of unresolved identifiers.
+
+Reason: explicit, inspectable dependency semantics and fewer compiler responsibilities.
+
+### 2026-10-02 — Native ESM URL backend first
+
+Decision: use Blob URLs in browsers and data URLs in Node/tests.
+
+Reason: preserve real module namespaces and avoid `new Function()` while the API remains backend-independent.
+
+### 2026-10-02 — Circular virtual dependencies deferred
+
+Decision: reject cycles rather than emulate them incorrectly.
+
+Reason: the current immutable URL linker cannot assign stable module identities before the full cyclic graph is linked.
+
+### 2026-10-02 — Compiler adapter boundary
+
+Decision: the runtime engine consumes `analyze()` and `transform()` rather than importing parser internals directly.
+
+Reason: preserve the option to add TSX, tagged-source, or alternative transformer backends later.