docs-i18n 0.8.1 → 0.8.3

package/template/content/en/deployment.md

@@ -0,0 +1,209 @@
+---
+title: Deployment
+description: Using docs-i18n in CI/CD pipelines, deploying the admin dashboard, and runtime translation with Cloudflare D1.
+---
+
+# Deployment
+
+## CI/CD with GitHub Actions
+
+docs-i18n works well in CI/CD pipelines. Since translations are cached in SQLite, you only pay for API calls on new or changed content.
+
+### Basic translation workflow
+
+```yaml
+name: Translate Docs
+on:
+  push:
+    branches: [main]
+    paths:
+      - 'content/**'
+
+jobs:
+  translate:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      - run: npm ci
+
+      # Restore the SQLite cache from a previous run
+      - uses: actions/cache@v4
+        with:
+          path: .cache
+          key: i18n-cache-${{ hashFiles('content/**') }}
+          restore-keys: |
+            i18n-cache-
+
+      # Rescan source files to detect changes
+      - run: npx docs-i18n rescan
+
+      # Translate all configured languages
+      - run: npx docs-i18n translate --lang zh-hans
+        env:
+          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
+
+      - run: npx docs-i18n translate --lang ja
+        env:
+          OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
+
+      # Assemble output files
+      - run: npx docs-i18n assemble
+
+      # Commit translated files (or use them in a build step)
+      - uses: stefanzweifel/git-auto-commit-action@v5
+        with:
+          commit_message: 'chore: update translations'
+          file_pattern: '.cache/content/**'
+```
+
+### Tips for CI/CD
+
+- **Cache the SQLite database.** The `.cache/translations.db` file contains all cached translations. Restoring it between runs avoids re-translating unchanged content. Use `actions/cache` with a key based on your content files.
+- **Run `rescan` before `translate`.** This detects new, changed, and deleted source files, and cleans up orphaned translations.
+- **Use `--dry-run` in PR checks.** Add a CI step that runs `docs-i18n translate --lang zh-hans --dry-run` to report how many nodes need translation without spending API credits.
+- **Limit concurrency.** In CI, you may want `--concurrency 1` to avoid rate limiting from your LLM provider.
+- **Set `--max` for budgeting.** Use `--max 10` to cap the number of API calls per run if you want to translate incrementally across multiple CI runs.
+
+### PR status check
+
+```yaml
+name: Translation Status
+on:
+  pull_request:
+    paths:
+      - 'content/**'
+
+jobs:
+  check:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+      - run: npm ci
+
+      - uses: actions/cache@v4
+        with:
+          path: .cache
+          key: i18n-cache-${{ hashFiles('content/**') }}
+          restore-keys: |
+            i18n-cache-
+
+      - run: npx docs-i18n rescan
+      - run: npx docs-i18n status
+      - run: npx docs-i18n translate --lang zh-hans --dry-run
+```
+
+## Admin Dashboard Deployment
+
+The admin dashboard is a TanStack Start application that runs locally. It is designed for development use, not production deployment.
+
+To start it:
+
+```bash
+npx docs-i18n admin
+# or
+npx docs-i18n admin --port 4000
+```
+
+The dashboard starts a Vite dev server pointed at the `src/admin` directory within the docs-i18n package. It reads your `docs-i18n.config.ts` to discover projects, versions, and languages.
+
+**Prerequisites:**
+
+Your project must have `vite` and `@vitejs/plugin-react` installed as dev dependencies:
+
+```bash
+npm install -D vite @vitejs/plugin-react
+```
+
+## Runtime Translation with D1
+
+For SSR sites that fetch markdown at runtime (e.g., TanStack Start on Cloudflare Workers), docs-i18n provides a lightweight runtime translator that queries Cloudflare D1 for cached translations.
+
+### How it works
+
+The `createTranslator` function from `docs-i18n/serve` creates a translator instance. When `translate()` is called:
+
+1. The markdown is parsed into AST nodes using the same parser as the CLI.
+2. Translatable nodes' MD5 keys are collected.
+3. A batch query is sent to D1 to look up all translations for the given language.
+4. The content is reassembled: translated nodes use the D1 cache value, untranslated nodes fall back to the original English text.
+
+### Setup
+
+#### 1. Export translations to D1
+
+First, translate your content using the CLI as normal. Then export the SQLite cache to D1. You can use Wrangler to create a D1 database and import the translations:
+
+```bash
+# Create a D1 database
+wrangler d1 create docs-translations
+
+# Export the translations table from your local SQLite
+sqlite3 .cache/translations.db ".dump translations" > translations.sql
+
+# Import into D1
+wrangler d1 execute docs-translations --file=translations.sql
+```
+
+Make sure the D1 database has the same `translations` table schema:
+
+```sql
+CREATE TABLE IF NOT EXISTS translations (
+  lang       TEXT NOT NULL,
+  key        TEXT NOT NULL,
+  value      TEXT NOT NULL,
+  created_at INTEGER NOT NULL DEFAULT (unixepoch()),
+  updated_at INTEGER NOT NULL DEFAULT (unixepoch()),
+  PRIMARY KEY (lang, key)
+);
+```
+
+#### 2. Use the translator in your Worker
+
+```ts
+import { createTranslator } from 'docs-i18n/serve';
+
+const translator = createTranslator();
+
+// In your request handler (e.g., TanStack Start server function):
+export async function getTranslatedDoc(
+  repo: string,
+  branch: string,
+  filePath: string,
+  lang: string,
+  env: { DB: D1Database },
+) {
+  // Fetch the English markdown from GitHub
+  const enMarkdown = await fetchFromGitHub(repo, branch, filePath);
+
+  // If the requested language is English, return as-is
+  if (lang === 'en') return enMarkdown;
+
+  // Translate using D1 cache
+  const translated = await translator.translate(enMarkdown, lang, env.DB);
+  return translated;
+}
+```
+
+#### 3. Bind D1 in wrangler.toml
+
+```toml
+[[d1_databases]]
+binding = "DB"
+database_name = "docs-translations"
+database_id = "your-database-id"
+```
+
+### Performance considerations
+
+- The translator uses a single batch query to D1, so latency scales with the number of translatable nodes (typically under 50ms for a standard documentation page).
+- English content (`lang === 'en'`) short-circuits immediately without parsing or querying.
+- The parser runs on every request since the English source is fetched at runtime. For heavy traffic, consider caching the translated output at the edge.
+- Untranslated nodes gracefully fall back to English, so partial translations work without errors.

package/template/content/en/getting-started.md

@@ -0,0 +1,168 @@
+---
+title: Getting Started
+description: Install docs-i18n and translate your first documentation file in minutes.
+---
+
+# Getting Started
+
+## What is docs-i18n?
+
+docs-i18n is a universal documentation translation engine. It parses markdown and MDX files into translatable AST nodes, translates them via LLM, caches results in SQLite, and assembles translated output files. It is framework-agnostic and works with any documentation site built on Next.js, Astro, TanStack Start, or similar tools.
+
+Key characteristics:
+
+- **Incremental** -- only changed or new content is sent to the LLM.
+- **AST-based** -- uses remark to parse markdown into nodes, producing stable MD5 keys for deduplication.
+- **SQLite-backed** -- concurrent-safe cache with WAL mode. No manual save steps.
+- **Multi-project** -- supports multiple documentation projects and versions in a single config.
+- **Admin dashboard** -- web UI for monitoring progress, managing jobs, and previewing translations.
+- **Runtime serve** -- optional D1-compatible translator for SSR sites on Cloudflare Workers.
+
+## Requirements
+
+- Node.js 20+ or Bun
+- An API key for an LLM provider (OpenRouter, OpenAI, or Anthropic)
+- For the admin dashboard: `vite` and `@vitejs/plugin-react` as dev dependencies
+
+## Installation
+
+```bash
+npm install docs-i18n
+```
+
+Or with Bun:
+
+```bash
+bun add docs-i18n
+```
+
+## Quick Start
+
+### 1. Create a configuration file
+
+Create `docs-i18n.config.ts` in your project root:
+
+```ts
+import { defineConfig } from 'docs-i18n';
+
+export default defineConfig({
+  projects: {
+    mydocs: {
+      sources: {
+        latest: 'content/docs',
+      },
+    },
+  },
+  languages: ['zh-hans', 'ja', 'es'],
+  llm: {
+    provider: 'openrouter',
+    model: 'deepseek/deepseek-chat-v3-0324:free',
+    apiKey: process.env.OPENROUTER_API_KEY,
+  },
+});
+```
+
+The `projects` object maps project names to their source directories. Each project can have multiple versions (e.g., `latest`, `v1`, `v2`). The `languages` array lists the target language codes to translate into.
+
+### 2. Scan your source files
+
+Before translating, scan your English source files to build the source index:
+
+```bash
+npx docs-i18n rescan
+```
+
+This parses every markdown/MDX file in your source directories, extracts translatable nodes, and stores them in the SQLite cache.
+
+### 3. Check translation status
+
+```bash
+npx docs-i18n status
+```
+
+This displays a progress bar for each language showing how many nodes have been translated.
+
+### 4. Run your first translation
+
+```bash
+npx docs-i18n translate --lang zh-hans
+```
+
+This sends untranslated nodes to your configured LLM, receives translations, and stores them in the cache. Only new or changed content is translated -- cached translations are reused.
+
+### 5. Assemble output files
+
+```bash
+npx docs-i18n assemble
+```
+
+This produces translated markdown files by combining your English sources with cached translations. Output is written to `.cache/content/<version>/<lang>/` by default.
+
+## Minimal Working Example
+
+Given this project structure:
+
+```
+my-docs/
+  content/
+    docs/
+      getting-started.md
+      api-reference.md
+  docs-i18n.config.ts
+  package.json
+```
+
+With this config:
+
+```ts
+import { defineConfig } from 'docs-i18n';
+
+export default defineConfig({
+  projects: {
+    docs: {
+      sources: { latest: 'content/docs' },
+    },
+  },
+  languages: ['zh-hans'],
+  llm: {
+    provider: 'openrouter',
+    apiKey: process.env.OPENROUTER_API_KEY,
+    model: 'deepseek/deepseek-chat-v3-0324:free',
+    contextLength: 32768,
+    maxTokens: 16384,
+  },
+  context: 'MyProject is a TypeScript library for building web apps.',
+});
+```
+
+Run these commands:
+
+```bash
+npx docs-i18n rescan
+npx docs-i18n translate --lang zh-hans
+npx docs-i18n assemble --lang zh-hans
+```
+
+The translated files will appear at:
+
+```
+.cache/content/latest/zh-hans/getting-started.md
+.cache/content/latest/zh-hans/api-reference.md
+```
+
+## Adding npm scripts
+
+Add these convenience scripts to your `package.json`:
+
+```json
+{
+  "scripts": {
+    "i18n": "docs-i18n",
+    "i18n:status": "docs-i18n status",
+    "i18n:translate": "docs-i18n translate",
+    "i18n:assemble": "docs-i18n assemble",
+    "i18n:rescan": "docs-i18n rescan",
+    "i18n:admin": "docs-i18n admin"
+  }
+}
+```

package/template/content/form/docs.config.json

@@ -0,0 +1,18 @@
+{
+  "sections": [
+    {
+      "label": "Getting Started",
+      "children": [
+        { "label": "Overview", "to": "overview" },
+        { "label": "Installation", "to": "installation" },
+        { "label": "Quick Start", "to": "quick-start" }
+      ]
+    },
+    {
+      "label": "Guides",
+      "children": [
+        { "label": "Validation", "to": "guides/validation" }
+      ]
+    }
+  ]
+}

package/template/content/form/en/guides/validation.md

@@ -0,0 +1,175 @@
+---
+title: Validation
+description: Learn how to validate form fields with TanStack Form
+---
+
+# Validation
+
+TanStack Form provides powerful, flexible validation that works at both the field level and form level. Validation can run on change, on blur, on submit, or asynchronously.
+
+## Field-Level Validation
+
+The most common approach is to validate individual fields. You can add validators directly to each field:
+
+```tsx
+import { useForm } from '@tanstack/react-form'
+
+function SignupForm() {
+  const form = useForm({
+    defaultValues: {
+      username: '',
+      email: '',
+      password: '',
+    },
+    onSubmit: async ({ value }) => {
+      console.log('Form submitted:', value)
+    },
+  })
+
+  return (
+    <form
+      onSubmit={(e) => {
+        e.preventDefault()
+        form.handleSubmit()
+      }}
+    >
+      <form.Field
+        name="username"
+        validators={{
+          onChange: ({ value }) => {
+            if (value.length < 3) {
+              return 'Username must be at least 3 characters'
+            }
+            return undefined
+          },
+        }}
+      >
+        {(field) => (
+          <div>
+            <label htmlFor={field.name}>Username</label>
+            <input
+              id={field.name}
+              value={field.state.value}
+              onChange={(e) => field.handleChange(e.target.value)}
+              onBlur={field.handleBlur}
+            />
+            {field.state.meta.errors.length > 0 && (
+              <p className="text-red-500">{field.state.meta.errors.join(', ')}</p>
+            )}
+          </div>
+        )}
+      </form.Field>
+
+      <button type="submit">Sign Up</button>
+    </form>
+  )
+}
+```
+
+## Validation Timing
+
+TanStack Form supports multiple validation timings:
+
+| Timing | When it runs | Use case |
+|---|---|---|
+| `onChange` | Every keystroke | Real-time feedback |
+| `onBlur` | When field loses focus | Less intrusive UX |
+| `onSubmit` | When form is submitted | Final validation only |
+| `onChangeAsync` | Debounced on change | API-backed validation |
+| `onBlurAsync` | Async on blur | Server-side checks |
+
+```tsx
+<form.Field
+  name="email"
+  validators={{
+    onChange: ({ value }) => {
+      if (!value.includes('@')) return 'Invalid email format'
+    },
+    onChangeAsyncDebounceMs: 500,
+    onChangeAsync: async ({ value }) => {
+      const available = await checkEmailAvailable(value)
+      if (!available) return 'Email already taken'
+    },
+  }}
+>
+  {(field) => <input {...field} />}
+</form.Field>
+```
+
+## Schema Validation with Adapters
+
+TanStack Form integrates with popular schema validation libraries through adapters:
+
+### Zod Adapter
+
+```bash
+npm install @tanstack/zod-form-adapter zod
+```
+
+```tsx
+import { zodValidator } from '@tanstack/zod-form-adapter'
+import { z } from 'zod'
+
+const form = useForm({
+  defaultValues: { email: '' },
+  onSubmit: async ({ value }) => console.log(value),
+  validatorAdapter: zodValidator(),
+})
+
+// Then in fields:
+<form.Field
+  name="email"
+  validators={{
+    onChange: z.string().email('Invalid email'),
+  }}
+>
+  {(field) => <input />}
+</form.Field>
+```
+
+### Valibot Adapter
+
+```bash
+npm install @tanstack/valibot-form-adapter valibot
+```
+
+```tsx
+import { valibotValidator } from '@tanstack/valibot-form-adapter'
+import * as v from 'valibot'
+
+const form = useForm({
+  defaultValues: { email: '' },
+  onSubmit: async ({ value }) => console.log(value),
+  validatorAdapter: valibotValidator(),
+})
+```
+
+> [!NOTE]
+> Schema adapters let you use the same schema definitions for both form validation and API validation, keeping your validation logic DRY.
+
+## Form-Level Validation
+
+You can also validate the entire form at once, useful for cross-field validation:
+
+```tsx
+const form = useForm({
+  defaultValues: {
+    password: '',
+    confirmPassword: '',
+  },
+  validators: {
+    onChange: ({ value }) => {
+      if (value.password !== value.confirmPassword) {
+        return 'Passwords do not match'
+      }
+      return undefined
+    },
+  },
+  onSubmit: async ({ value }) => {
+    // handle submit
+  },
+})
+```
+
+> [!WARNING]
+> Form-level validators run on every field change by default. For expensive validation logic, use `onBlur` or `onSubmit` timing instead.

package/template/content/form/en/installation.md

@@ -0,0 +1,63 @@
+---
+title: Installation
+description: How to install TanStack Form in your project
+---
+
+# Installation
+
+## React
+
+```bash
+npm install @tanstack/react-form
+```
+
+```bash
+pnpm add @tanstack/react-form
+```
+
+```bash
+yarn add @tanstack/react-form
+```
+
+```bash
+bun add @tanstack/react-form
+```
+
+## Vue
+
+```bash
+npm install @tanstack/vue-form
+```
+
+## Angular
+
+```bash
+npm install @tanstack/angular-form
+```
+
+## Validation Adapters
+
+TanStack Form supports popular validation libraries via adapters:
+
+### Zod
+
+```bash
+npm install @tanstack/zod-form-adapter zod
+```
+
+### Valibot
+
+```bash
+npm install @tanstack/valibot-form-adapter valibot
+```
+
+### Yup
+
+```bash
+npm install @tanstack/yup-form-adapter yup
+```
+
+## Requirements
+
+- TypeScript 5.0+
+- React 18+ / Vue 3+ / Angular 17+

package/template/content/form/en/overview.md

@@ -0,0 +1,71 @@
+---
+title: TanStack Form Overview
+description: Headless, performant, and type-safe form state management
+---
+
+# TanStack Form
+
+TanStack Form is a headless, performant, and type-safe form state management library for TS/JS, React, Vue, and Angular.
+
+## Why TanStack Form?
+
+Most form libraries either sacrifice type safety for convenience or are tightly coupled to a specific framework. TanStack Form gives you both — full type safety from form values to validation errors, with adapters for React, Vue, and Angular.
+
+## Features
+
+- **Type-Safe** — Full TypeScript inference for form values, field names, and validation
+- **Framework Agnostic** — Works with React, Vue, Angular, and more
+- **Headless** — No UI opinions, bring your own components
+- **Performant** — Field-level subscriptions, no unnecessary re-renders
+- **Validation** — Built-in sync and async validation with adapter support for Zod, Valibot, and Yup
+- **Arrays & Dynamic Fields** — First-class support for array fields
+
+## Basic Example
+
+```tsx
+import { useForm } from '@tanstack/react-form'
+
+function SignupForm() {
+  const form = useForm({
+    defaultValues: {
+      firstName: '',
+      lastName: '',
+      email: '',
+    },
+    onSubmit: async ({ value }) => {
+      console.log(value)
+    },
+  })
+
+  return (
+    <form
+      onSubmit={(e) => {
+        e.preventDefault()
+        form.handleSubmit()
+      }}
+    >
+      <form.Field
+        name="firstName"
+        validators={{
+          onChange: ({ value }) =>
+            !value ? 'First name is required' : undefined,
+        }}
+      >
+        {(field) => (
+          <div>
+            <label>First Name</label>
+            <input
+              value={field.state.value}
+              onChange={(e) => field.handleChange(e.target.value)}
+            />
+            {field.state.meta.errors.length > 0 && (
+              <span>{field.state.meta.errors.join(', ')}</span>
+            )}
+          </div>
+        )}
+      </form.Field>
+      <button type="submit">Submit</button>
+    </form>
+  )
+}
+```