nowebcli 0.2.1

package/README.md

@@ -0,0 +1,188 @@
+# Noweb
+
+Noweb ist eine Next.js-App fuer Website-Templates, Projekte, Auth, Checkout und
+Deployment. Nutzer koennen Templates auswaehlen, Inhalte im Editor anpassen,
+Bilder hochladen und Projekte mit eigener Domain veroeffentlichen.
+
+## Schnellstart
+
+```bash
+npm install
+npm run dev
+```
+
+Die App laeuft lokal unter `http://localhost:3000`.
+
+Vor einem Pull Request oder Deploy:
+
+```bash
+npm run ci
+```
+
+## Community-Templates
+
+Die vollständige Anleitung für Erstellung, Assets, lokale Vorschau und
+automatische Veröffentlichung steht in [`templates/README.md`](templates/README.md).
+
+## Wichtige Befehle
+
+- `npm run dev` startet Next lokal mit Webpack.
+- `npm run build` baut die Produktionsversion.
+- `npm run start` startet den gebauten Next-Server.
+- `npm run lint` prueft den Code mit ESLint und erlaubt keine Warnings.
+- `npm run typecheck` prueft TypeScript ohne Emit.
+- `npm run test` fuehrt Unit-Tests aus.
+- `npm run test:e2e` fuehrt den Playwright-Smoke gegen `npm run start` aus.
+- `npx nowebcli create` erstellt ein Community-Template; danach laufen die Befehle lokal ueber `npx noweb`.
+- `npm run ci` fuehrt Audit, Typecheck, Lint, Tests, Build und E2E aus.
+
+## Struktur
+
+- `app/` enthaelt Seiten, Routen und Server Actions.
+- `app/page.tsx`, `app/HomeSections.tsx`, `app/Footer.tsx` und `app/menu.tsx`
+  bilden die Startseite und Navigation.
+- `app/docs/` enthaelt die oeffentliche Doku im Produkt.
+- `lib/auth/` konfiguriert Better Auth.
+- `lib/db.ts` stellt die Postgres-Verbindung bereit.
+- `lib/object-storage.ts` kapselt RustFS/R2-kompatiblen Object Storage.
+- `lib/env.ts` validiert serverseitige Runtime-Konfiguration.
+- `lib/domain.ts` normalisiert und validiert Domains.
+- `lib/templates/` enthaelt Template-Typen, Runtime-Helfer und lokale Templates.
+- `templates/index.ts` ist generiert und soll nicht per Hand editiert werden.
+- `db.txt` dokumentiert das aktuelle Datenbankschema.
+- `scripts/deploy-worker.mjs` veroeffentlicht Projekte als statische Sites.
+- `.github/workflows/deploy.yaml` deployed `main` per SSH auf den Server.
+
+## Umgebung
+
+Mindestens benoetigt:
+
+```env
+DATABASE_URL=postgresql://...
+BETTER_AUTH_SECRET=...
+```
+
+Je nach Feature zusaetzlich:
+
+```env
+RESEND_API_KEY=...
+EMAIL_FROM=Noweb <mail@example.com>
+
+GOOGLE_CLIENT_ID=...
+GOOGLE_CLIENT_SECRET=...
+GITHUB_CLIENT_ID=...
+GITHUB_CLIENT_SECRET=...
+
+POLAR_ACCESS_TOKEN=...
+POLAR_PRODUCT_ID=...
+POLAR_SERVER=sandbox
+
+RUSTFS_ACCESS_KEY=...
+RUSTFS_SECRET_KEY=...
+RUSTFS_BUCKET_NAME=...
+RUSTFS_ENDPOINT=...
+RUSTFS_PUBLIC_URL=...
+
+SENTRY_DSN=...
+```
+
+Object Storage kann alternativ ueber die `R2_*` Variablen konfiguriert werden.
+Details stehen in [docs/object-storage.md](docs/object-storage.md).
+
+## Wie die Hauptteile zusammenhaengen
+
+Auth laeuft ueber Better Auth in `lib/auth/server.ts`. Die Daten liegen in
+Postgres, E-Mails werden ueber Resend verschickt, OAuth ist optional ueber
+Google und GitHub aktiv.
+
+Projekte und Templates werden in Postgres gespeichert. Das aktuelle Schema
+steht in `db.txt`; Schema-Aenderungen werden direkt in Postgres ausgefuehrt und
+danach dort dokumentiert.
+
+Checkout laeuft ueber Polar. Der kanonische Einstieg ist
+`app/checkout/route.ts`, nicht direkt der Better-Auth-Checkout. Details stehen
+in [docs/checkout.md](docs/checkout.md).
+
+Bild-Uploads gehen ueber `app/api/uploads/r2/route.ts` in einen S3-kompatiblen
+Storage. Die oeffentliche URL kommt aus `RUSTFS_PUBLIC_URL`, `R2_PUBLIC_URL`
+oder `PUBLIC_DEV_URL`.
+
+Template-Previews laufen in einer sandboxed `iframe/srcDoc` Runtime. Oeffentlich
+sichtbar werden Community-Templates nach erfolgreicher automatischer Validierung.
+
+Der Deploy Worker nimmt unveroeffentlichte Projekte aus der Datenbank, liest die
+kompilierten Template-Dateien, validiert Domains und schreibt statische Sites
+nach `SITES_DIR` Standard: `/etc/caddy/sites`.
+
+## Deployment
+
+Bei jedem Push auf `main` fuehrt GitHub Actions `.github/workflows/deploy.yaml`
+aus:
+
+1. Per SSH auf den Server verbinden und `origin/main` auschecken.
+2. `npm ci` ausfuehren.
+3. Repository-Templates in die Datenbank und den Preview-Storage publizieren.
+4. `npm run build` ausfuehren.
+5. Den `noweb`-Service neu starten.
+
+Die GitHub-Secrets muessen gesetzt sein:
+
+- `SERVER_HOST`
+- `SERVER_USER`
+- `SSH_PRIVATE_KEY`
+
+Auf dem Server muessen ausserdem Node, npm, die Runtime-Env-Variablen und der
+`noweb` systemd-Service vorhanden sein.
+
+## Aenderungs-Checkliste
+
+Wenn du etwas aenderst, pruefe die abhaengigen Stellen mit:
+
+- Startseite, Preise oder Marketingtext:
+  `app/page.tsx`, `app/HomeSections.tsx`, `app/Footer.tsx`, `app/menu.tsx`,
+  `docs/checkout.md` bei Preis-/Checkout-Aenderungen.
+- Polar-Produkte oder Checkout:
+  `app/checkout/route.ts`, `lib/auth/server.ts`, `lib/polar.ts`,
+  `docs/checkout.md`, Env-Variablen auf Server/GitHub.
+- Auth, Login, Verifizierung oder OAuth:
+  `lib/auth/server.ts`, `app/auth/**`, Datenbanktabellen und Resend/OAuth-Env.
+- Datenbank-Schema oder Query-Format:
+  `lib/db.ts`, Server Actions in `app/**/actions.ts`, `scripts/deploy-worker.mjs`,
+  `db.txt`.
+- Template-Felder oder Customizations:
+  Template-Definition, Editor-Komponenten in `lib/editor/` und
+  `app/my-templates/[id]/`, Deploy Worker, `templates/index.ts` nur ueber den
+  Sync-Prozess aktualisieren.
+- Template-Doku:
+  passende Seite in `app/docs/` aktualisieren und den Eintrag in
+  `app/docs/guides.ts` pruefen.
+- Bild-Uploads oder Storage:
+  `lib/object-storage.ts`, `app/api/uploads/r2/route.ts`,
+  `docs/object-storage.md`, Storage-Env auf dem Server.
+- Projekt-Deployment oder Domains:
+  `scripts/deploy-worker.mjs`, Caddy/systemd-Konfiguration, `docs/deploy-worker.md`.
+- GitHub Deploy:
+  `.github/workflows/deploy.yaml`, GitHub-Secrets, Serverpfad `~/noweb`,
+  systemd-Service `noweb`.
+- Neue oder geaenderte Produktfunktion:
+  vorhandene Doku aktualisieren oder eine kleine neue Seite in `docs/` oder
+  `app/docs/` anlegen.
+
+## Launch-Blocker
+
+- Echte Betreiberangaben in `app/imprint/page.tsx`, `app/privacy/page.tsx` und
+  `app/terms/page.tsx` eintragen.
+- Lokale und Server-Env pruefen. `POLAR_SERVER` darf nur `sandbox` oder
+  `production` sein.
+- Monitoring setzen: `SENTRY_DSN` oder dokumentierte Alternative.
+- `npm run ci` muss vor Production gruen sein.
+
+## Wichtige Hinweise
+
+- Dieses Projekt nutzt Next.js 16. Vor groesseren Next-Aenderungen die lokalen
+  Docs in `node_modules/next/dist/docs/` lesen.
+- `templates/index.ts` ist generiert. Nicht manuell pflegen.
+- JSON-Spalten muessen beim Schreiben mit `JSON.stringify(...)` gespeichert
+  werden.
+- Secrets gehoeren nicht ins Repository.
+- Vor einem Deploy `npm run ci` ausfuehren.

package/lib/customizations.ts

@@ -0,0 +1,252 @@
+export type ScalarType =
+  | "text"
+  | "markdown"
+  | "image"
+  | "icon"
+  | "color"
+  | "number"
+  | "range"
+  | "boolean"
+  | "select";
+
+export type ScalarValue = string | number | boolean;
+export type CollectionItem = Record<string, ScalarValue>;
+export type CustomizationValue = ScalarValue | string[] | CollectionItem[];
+export type CustomizationValues = Record<string, CustomizationValue>;
+
+type FieldOptions = {
+  min?: number;
+  max?: number;
+  step?: number;
+  options?: string[];
+};
+
+export type CollectionField = FieldOptions & {
+  key: string;
+  name: string;
+  type: ScalarType;
+  defaultValue: ScalarValue;
+};
+
+export type TemplateCustomization = FieldOptions & {
+  fieldId: number;
+  name: string;
+  type: ScalarType | "array" | "collection";
+  defaultValue: CustomizationValue;
+  fields?: CollectionField[];
+};
+
+const SCALAR_TYPES: ScalarType[] = [
+  "text", "markdown", "image", "icon", "color",
+  "number", "range", "boolean", "select",
+];
+
+export function normalizeTemplateCustomizations(value: unknown) {
+  const parsed = typeof value === "string" ? parseJson(value, []) : value;
+  return Array.isArray(parsed)
+    ? parsed.flatMap((item) => {
+        const field = readField(item, true);
+        return field ? [field] : [];
+      })
+    : [];
+}
+
+export function validateTemplateCustomizations(value: unknown) {
+  if (!Array.isArray(value)) throw new Error("Template customizations must be an array.");
+  return value.map((item, index) => {
+    const field = readField(item, false);
+    if (!field) throw new Error(`Customization ${index + 1} is invalid.`);
+    return field;
+  });
+}
+
+export function normalizeCustomizationValues(value: unknown): CustomizationValues | null {
+  const parsed = typeof value === "string" ? parseJson(value, null) : value;
+  return isRecord(parsed) && Object.values(parsed).every(isCustomizationValue)
+    ? parsed as CustomizationValues
+    : null;
+}
+
+export function restoreCustomizationValues(
+  current: CustomizationValues,
+  previous: CustomizationValues,
+  fieldIds: string[],
+) {
+  const restored = { ...current };
+  for (const fieldId of fieldIds) {
+    if (Object.hasOwn(previous, fieldId)) restored[fieldId] = previous[fieldId];
+    else delete restored[fieldId];
+  }
+  return restored;
+}
+
+export function normalizeCustomizationValue(
+  field: TemplateCustomization,
+  value: unknown,
+): CustomizationValue | undefined {
+  if (field.type === "array") {
+    return Array.isArray(value) && value.length <= 50 &&
+      value.every((item) => typeof item === "string" && item.length <= 20_000)
+      ? value
+      : undefined;
+  }
+
+  if (field.type === "collection") {
+    if (!Array.isArray(value) || value.length > 50 || !field.fields?.length) return undefined;
+    const items = value.map((item) => {
+      if (!isRecord(item)) return null;
+      const normalized: CollectionItem = {};
+      for (const child of field.fields!) {
+        const childValue = normalizeScalarValue(child, item[child.key]);
+        if (childValue === undefined) return null;
+        normalized[child.key] = childValue;
+      }
+      return normalized;
+    });
+    return items.some((item) => item === null) ? undefined : items as CollectionItem[];
+  }
+
+  const type = field.type;
+  if (!isScalarType(type)) return undefined;
+  return normalizeScalarValue({ ...field, type }, value);
+}
+
+function readField(value: unknown, loose: boolean): TemplateCustomization | null {
+  if (!isRecord(value)) return null;
+  const fieldId = Number(value.fieldId);
+  const name = text(value.name);
+  const type = value.type === "option" ? "select" : value.type;
+  if (!Number.isInteger(fieldId) || fieldId < 1 || !name) return null;
+
+  if (type === "array") {
+    const defaultValue = strings(value.defaultValue);
+    if (!loose && !defaultValue) return null;
+    return { fieldId, name, type, defaultValue: defaultValue ?? [] };
+  }
+
+  if (type === "collection") {
+    const fields = Array.isArray(value.fields)
+      ? value.fields.flatMap((item) => {
+          const field = readCollectionField(item, loose);
+          return field ? [field] : [];
+        })
+      : [];
+    if (!loose && new Set(fields.map((field) => field.key)).size !== fields.length) return null;
+    const defaultValue = collection(value.defaultValue, fields);
+    if (!loose && (!Array.isArray(value.fields) || !defaultValue)) return null;
+    return { fieldId, name, type, fields, defaultValue: defaultValue ?? [] };
+  }
+
+  if (!isScalarType(type)) return null;
+  const defaultValue = scalar(value.defaultValue, type);
+  if (!loose && defaultValue === null) return null;
+  return { fieldId, name, type, defaultValue: defaultValue ?? scalarDefault(type), ...fieldOptions(value, type) };
+}
+
+function readCollectionField(value: unknown, loose: boolean): CollectionField | null {
+  if (!isRecord(value) || !isScalarType(value.type)) return null;
+  const key = text(value.key);
+  const name = text(value.name);
+  const defaultValue = scalar(value.defaultValue, value.type);
+  if (!key || !name || (!loose && defaultValue === null)) return null;
+  return { key, name, type: value.type, defaultValue: defaultValue ?? scalarDefault(value.type), ...fieldOptions(value, value.type) };
+}
+
+function fieldOptions(value: Record<string, unknown>, type: ScalarType) {
+  return {
+    ...(number(value.min) !== null ? { min: number(value.min)! } : {}),
+    ...(number(value.max) !== null ? { max: number(value.max)! } : {}),
+    ...(number(value.step) !== null && number(value.step)! > 0 ? { step: number(value.step)! } : {}),
+    ...(type === "select" ? { options: strings(value.options) ?? [] } : {}),
+  };
+}
+
+function collection(value: unknown, fields: CollectionField[]) {
+  if (!Array.isArray(value)) return null;
+  const items = value.flatMap((item) => {
+    if (!isRecord(item)) return [];
+    return [Object.fromEntries(fields.map((field) => [
+      field.key,
+      scalar(item[field.key], field.type) ?? field.defaultValue,
+    ])) as CollectionItem];
+  });
+  return items.length === value.length ? items : null;
+}
+
+function isCustomizationValue(value: unknown): value is CustomizationValue {
+  if (typeof value === "string" || typeof value === "boolean") return true;
+  if (typeof value === "number") return Number.isFinite(value);
+  if (!Array.isArray(value)) return false;
+  return value.every((item) => typeof item === "string") ||
+    value.every((item) => isRecord(item) && Object.values(item).every(isScalarValue));
+}
+
+function scalar(value: unknown, type: ScalarType): ScalarValue | null {
+  if (type === "number" || type === "range") return number(value);
+  if (type === "boolean") return typeof value === "boolean" ? value : null;
+  return typeof value === "string" ? value : null;
+}
+
+function normalizeScalarValue(
+  field: { type: ScalarType; min?: number; max?: number; options?: string[] },
+  value: unknown,
+): ScalarValue | undefined {
+  if (field.type === "number" || field.type === "range") {
+    if (typeof value !== "number" || !Number.isFinite(value)) return undefined;
+    if (field.min !== undefined && value < field.min) return undefined;
+    if (field.max !== undefined && value > field.max) return undefined;
+    return value;
+  }
+  if (field.type === "boolean") return typeof value === "boolean" ? value : undefined;
+  if (typeof value !== "string" || value.length > 20_000) return undefined;
+  if (field.type === "select" && field.options?.length && !field.options.includes(value)) return undefined;
+  if (field.type === "color" && !/^#[\da-f]{3}([\da-f]{3})?$/i.test(value)) return undefined;
+  if (field.type === "icon" && !/^[a-z0-9]+(?:-[a-z0-9]+)*$/.test(value)) return undefined;
+  return value;
+}
+
+function scalarDefault(type: ScalarType): ScalarValue {
+  if (type === "number" || type === "range") return 0;
+  if (type === "boolean") return false;
+  if (type === "icon") return "circle";
+  return "";
+}
+
+function isScalarValue(value: unknown): value is ScalarValue {
+  return typeof value === "string" || typeof value === "boolean" ||
+    (typeof value === "number" && Number.isFinite(value));
+}
+
+function isScalarType(value: unknown): value is ScalarType {
+  return SCALAR_TYPES.includes(value as ScalarType);
+}
+
+function number(value: unknown) {
+  return typeof value === "number" && Number.isFinite(value) ? value : null;
+}
+
+function strings(value: unknown) {
+  return Array.isArray(value) && value.every((item) => typeof item === "string") ? value : null;
+}
+
+function text(value: unknown) {
+  return typeof value === "string" ? value.trim() : "";
+}
+
+function isRecord(value: unknown): value is Record<string, unknown> {
+  return Boolean(value) && typeof value === "object" && !Array.isArray(value);
+}
+
+function parseJson<T>(value: string, fallback: T) {
+  let parsed: unknown = value;
+
+  for (let i = 0; i < 4 && typeof parsed === "string"; i++) {
+    try {
+      parsed = JSON.parse(parsed) as unknown;
+    } catch {
+      return fallback;
+    }
+  }
+
+  return parsed;
+}

package/package.json

@@ -0,0 +1,103 @@
+{
+  "name": "nowebcli",
+  "version": "0.2.1",
+  "description": "Create, preview and publish community templates for Noweb",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/lnny8/noweb.git"
+  },
+  "files": [
+    "lib/customizations.ts",
+    "scripts/noweb.mjs",
+    "templates/compileTailwind.ts",
+    "templates/shadersRuntime.tsx",
+    "templates/templateCompiler.ts",
+    "templates/templateRuntime.tsx",
+    "templates/validateTemplateCode.ts",
+    "tsconfig.json"
+  ],
+  "publishConfig": {
+    "access": "public"
+  },
+  "bin": {
+    "noweb": "scripts/noweb.mjs",
+    "nowebcli": "scripts/noweb.mjs"
+  },
+  "scripts": {
+    "dev": "next dev --webpack",
+    "build": "next build",
+    "start": "next start",
+    "lint": "eslint --max-warnings=0",
+    "typecheck": "tsc --noEmit",
+    "test": "vitest run",
+    "test:e2e": "playwright test",
+    "screenshots": "tsx scripts/generate-template-screenshots.ts",
+    "template": "node scripts/noweb.mjs",
+    "template:publish": "node --env-file-if-exists=.env --env-file-if-exists=.env.local scripts/noweb.mjs publish",
+    "migrate:cli-auth": "node --env-file-if-exists=.env --env-file-if-exists=.env.local --import tsx scripts/migrate-cli-auth.ts",
+    "migrate:project-snapshots": "node --env-file-if-exists=.env --env-file-if-exists=.env.local --import tsx scripts/migrate-project-snapshots.ts",
+    "migrate:project-data": "node --env-file-if-exists=.env --env-file-if-exists=.env.local --import tsx scripts/migrate-project-data.ts",
+    "migrate:template-ids": "tsx scripts/migrate-positive-template-ids.ts",
+    "ci": "npm audit --omit=dev --audit-level=high && npm run typecheck && npm run lint && npm run test && npm run build && npm run test:e2e"
+  },
+  "dependencies": {
+    "@ai-sdk/openai": "^4.0.24",
+    "@ai-sdk/react": "^4.0.45",
+    "@aws-sdk/client-s3": "^3.1055.0",
+    "@heroicons/react": "^2.2.0",
+    "@monaco-editor/react": "^4.7.0",
+    "@polar-sh/better-auth": "^1.8.4",
+    "@polar-sh/nextjs": "^0.9.6",
+    "@polar-sh/sdk": "^0.47.1",
+    "@react-three/drei": "^10.7.7",
+    "@react-three/fiber": "^9.6.1",
+    "@react-three/rapier": "^2.2.0",
+    "@tailwindcss/postcss": "^4",
+    "@types/pg": "^8.20.0",
+    "@uiw/react-textarea-code-editor": "^3.1.1",
+    "ai": "^7.0.42",
+    "better-auth": "1.7.2",
+    "clsx": "^2.1.1",
+    "esbuild": "^0.28.1",
+    "fumadocs-core": "^16.15.11",
+    "fumadocs-ui": "^16.15.11",
+    "geist": "^1.7.2",
+    "gsap": "^3.15.0",
+    "html2canvas-pro": "^2.3.3",
+    "lucide-react": "^1.14.0",
+    "monaco-editor": "^0.53.0",
+    "motion": "^12.40.0",
+    "next": "^16.2.10",
+    "ogl": "^1.0.11",
+    "pg": "^8.22.0",
+    "playwright": "^1.61.1",
+    "postcss": "^8.5.6",
+    "postgres": "^3.4.9",
+    "react": "19.2.4",
+    "react-dom": "19.2.4",
+    "react-markdown": "^10.1.0",
+    "remark-gfm": "^4.0.1",
+    "shaders": "^3.0.445",
+    "shiki": "^4.3.0",
+    "tailwindcss": "^4",
+    "three": "^0.184.0",
+    "tsx": "^4.22.4",
+    "typescript": "^5",
+    "zod": "^4.4.3"
+  },
+  "devDependencies": {
+    "@playwright/test": "^1.61.1",
+    "@tailwindcss/vite": "^4.3.1",
+    "@types/node": "^20",
+    "@types/react": "^19",
+    "@types/react-dom": "^19",
+    "@types/three": "^0.184.1",
+    "@vitejs/plugin-react": "^6.0.3",
+    "eslint": "^9",
+    "eslint-config-next": "^16.2.10",
+    "shadcn": "^4.16.1",
+    "vite": "^8.1.0",
+    "vitest": "^4.1.10"
+  }
+}