@llm-translate/cli 1.0.0-next.1

package/tests/fixtures/input/lynq-installation.md

@@ -0,0 +1,350 @@
+# Installation Guide
+
+This guide covers various installation methods for Lynq.
+
+[[toc]]
+
+::: tip Trying it locally?
+Use the [Quick Start with Minikube](quickstart.md) guide for an automated setup tailored to first-time users.
+:::
+
+## Prerequisites
+
+### Required
+
+| Component | Minimum version | Notes |
+| --- | --- | --- |
+| Kubernetes cluster | v1.11.3+ | API compatibility tested with recent releases |
+| `kubectl` | Matches cluster | Must target the cluster where you deploy |
+| **cert-manager** | **v1.13.0+** | **REQUIRED for all installations** (production, development, local) |
+
+::: danger cert-manager is REQUIRED
+**cert-manager v1.13.0+** is **REQUIRED for ALL installations** (production, development, and local environments). It provisions webhook TLS certificates, handles automatic renewal, and injects CA bundles into webhook configurations.
+
+**Webhooks are no longer optional.** They provide essential validation and defaulting at admission time.
+
+Install before deploying Lynq:
+```bash
+kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
+```
+:::
+
+### Optional
+
+- **MySQL database** for node data source (PostgreSQL support planned for v1.2)
+
+## Kubernetes Compatibility
+
+### Supported Versions
+
+The operator relies only on GA/stable Kubernetes APIs and controller-runtime patterns, making it compatible across the supported upstream version skew.
+
+**Validated versions** (end-to-end tested and production-verified):
+
+| Kubernetes Version | Status |
+|--------------------|--------|
+| v1.28              | ✅ Validated |
+| v1.29              | ✅ Validated |
+| v1.30              | ✅ Validated |
+| v1.31              | ✅ Validated |
+| v1.32              | ✅ Validated |
+| v1.33              | ✅ Validated |
+| Other GA releases  | ⚠️ Expected to work |
+
+::: tip Compatibility Philosophy
+The operator is designed to work across Kubernetes version skew. Earlier or newer versions are expected to function, but validate in a staging environment before rolling out broadly.
+:::
+
+## Installation Methods
+
+### Method 1: Install with Helm (Recommended)
+
+**cert-manager is REQUIRED** for all installations.
+
+```bash
+# Step 1: Install cert-manager (REQUIRED)
+kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
+
+# Step 2: Wait for cert-manager to be ready
+kubectl wait --for=condition=Available --timeout=300s -n cert-manager \
+  deployment/cert-manager \
+  deployment/cert-manager-webhook \
+  deployment/cert-manager-cainjector
+
+# Step 3: Add Helm repository
+helm repo add lynq https://k8s-lynq.github.io/lynq
+helm repo update
+
+# Step 4: Install Lynq
+helm install lynq lynq/lynq \
+  --namespace lynq-system \
+  --create-namespace
+```
+
+See the [Helm Chart README](https://github.com/k8s-lynq/lynq/blob/main/chart/README.md) for detailed configuration options.
+
+---
+
+### Method 2: Install with Kustomize
+
+**cert-manager is REQUIRED** for webhook TLS certificate management.
+
+```bash
+# Step 1: Install cert-manager (if not already installed)
+kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
+
+# Step 2: Wait for cert-manager to be ready
+kubectl wait --for=condition=Available --timeout=300s -n cert-manager deployment/cert-manager
+kubectl wait --for=condition=Available --timeout=300s -n cert-manager deployment/cert-manager-webhook
+
+# Step 3: Install Lynq
+# cert-manager will automatically issue and manage webhook TLS certificates
+kubectl apply -k https://github.com/k8s-lynq/lynq/config/default
+```
+
+::: info What cert-manager handles
+- Issues TLS certificates for the webhook server
+- Renews certificates before expiration
+- Injects the CA bundle into webhook configurations
+- Provides battle-tested certificate automation for Kubernetes clusters
+:::
+
+### Method 3: Install from Source
+
+```bash
+# Clone repository
+git clone https://github.com/k8s-lynq/lynq.git
+cd lynq
+
+# Install CRDs
+make install
+
+# Install cert-manager first if not already installed
+kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
+
+# Deploy operator
+make deploy IMG=ghcr.io/k8s-lynq/lynq:latest
+```
+
+::: warning Remember TLS
+Even when deploying from source, install cert-manager before applying the operator manifests, otherwise webhooks will fail to start.
+:::
+
+### Method 4: Local Development with Minikube
+
+For local development, use Minikube with automated setup scripts. **cert-manager is automatically installed** by the setup script.
+
+See [Local Development with Minikube](local-development-minikube.md) for detailed instructions.
+
+```bash
+# Quick setup (cert-manager included)
+./scripts/setup-minikube.sh      # Create cluster with cert-manager
+./scripts/deploy-to-minikube.sh  # Build and deploy operator
+```
+
+::: tip cert-manager in Local Development
+The setup script automatically installs cert-manager. You don't need to install it manually for local development when using the provided scripts.
+:::
+
+## Verification
+
+Check that the operator is running:
+
+```bash
+# Check operator deployment
+kubectl get deployment -n lynq-system lynq-controller-manager
+
+# Check operator logs
+kubectl logs -n lynq-system deployment/lynq-controller-manager -f
+
+# Verify CRDs are installed
+kubectl get crd | grep operator.lynq.sh
+```
+
+Expected output:
+```
+lynqhubs.operator.lynq.sh    2025-01-15T10:00:00Z
+lynqnodes.operator.lynq.sh             2025-01-15T10:00:00Z
+lynqforms.operator.lynq.sh     2025-01-15T10:00:00Z
+```
+
+::: tip Troubleshooting
+If the deployment is not ready, inspect `kubectl describe deployment/lynq-controller-manager` for webhook, RBAC, or image issues.
+:::
+
+## Configuration Options
+
+### Webhook TLS Configuration
+
+Webhook TLS is managed automatically by cert-manager. The default configuration includes:
+
+```yaml
+# config/default/kustomization.yaml
+# Webhook patches are enabled by default
+patches:
+- path: manager_webhook_patch.yaml
+- path: webhookcainjection_patch.yaml
+```
+
+::: info cert-manager responsibilities
+- Issue TLS certificates for the webhook server
+- Inject CA bundles into webhook configurations
+- Renew certificates before expiration
+:::
+
+### Resource Limits
+
+Adjust operator resource limits based on your cluster size:
+
+```yaml
+# config/manager/manager.yaml
+resources:
+  limits:
+    cpu: 500m      # Increase for large clusters
+    memory: 512Mi  # Increase for many nodes
+  requests:
+    cpu: 100m
+    memory: 128Mi
+```
+
+### Concurrency Settings
+
+Configure concurrent reconciliation workers:
+
+```yaml
+spec:
+  template:
+    spec:
+      containers:
+      - name: manager
+        args:
+        - --node-concurrency=10        # Concurrent LynqNode reconciliations (default: 10)
+        - --form-concurrency=5       # Concurrent Template reconciliations (default: 5)
+        - --hub-concurrency=3       # Concurrent Hub syncs (default: 3)
+        - --leader-elect                 # Enable leader election
+```
+
+## Multi-Platform Support
+
+The operator supports multiple architectures:
+
+- `linux/amd64` (Intel/AMD 64-bit)
+- `linux/arm64` (ARM 64-bit, Apple Silicon)
+
+Container images are automatically pulled for your platform.
+
+## Namespace Isolation
+
+By default, the operator is installed in `lynq-system` namespace:
+
+```bash
+# Check operator namespace
+kubectl get all -n lynq-system
+
+# View RBAC
+kubectl get clusterrole | grep lynq
+kubectl get clusterrolebinding | grep lynq
+```
+
+## Upgrading
+
+### Upgrade CRDs First
+
+```bash
+# Upgrade CRDs (safe, preserves existing data)
+make install
+
+# Or with kubectl
+kubectl apply -f config/crd/bases/
+```
+
+### Upgrade Operator
+
+```bash
+# Update operator deployment
+kubectl set image -n lynq-system \
+  deployment/lynq-controller-manager \
+  manager=ghcr.io/k8s-lynq/lynq:v1.1.0
+
+# Or use make
+make deploy IMG=ghcr.io/k8s-lynq/lynq:v1.1.0
+```
+
+### Rolling Back
+
+```bash
+# Rollback to previous version
+kubectl rollout undo -n lynq-system \
+  deployment/lynq-controller-manager
+
+# Check rollout status
+kubectl rollout status -n lynq-system \
+  deployment/lynq-controller-manager
+```
+
+## Uninstallation
+
+```bash
+# Delete operator deployment
+kubectl delete -k config/default
+
+# Or with make
+make undeploy
+
+# Delete CRDs (WARNING: This deletes all LynqNode data!)
+make uninstall
+
+# Or with kubectl
+kubectl delete crd lynqhubs.operator.lynq.sh
+kubectl delete crd lynqforms.operator.lynq.sh
+kubectl delete crd lynqnodes.operator.lynq.sh
+```
+
+**Warning:** Deleting CRDs will delete all LynqHub, LynqForm, and LynqNode resources. Ensure you have backups if needed.
+
+## Troubleshooting Installation
+
+### Webhook TLS Errors
+
+**Error:** `open /tmp/k8s-webhook-server/serving-certs/tls.crt: no such file or directory`
+
+**Solution:** Install cert-manager to automatically manage webhook TLS certificates.
+
+```bash
+# Install cert-manager
+kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.13.0/cert-manager.yaml
+
+# Wait for cert-manager to be ready
+kubectl wait --for=condition=Available --timeout=300s -n cert-manager deployment/cert-manager
+
+# Restart operator to pick up certificates
+kubectl rollout restart -n lynq-system deployment/lynq-controller-manager
+```
+
+### CRD Already Exists
+
+**Error:** `Error from server (AlreadyExists): customresourcedefinitions.apiextensions.k8s.io "lynqnodes.operator.lynq.sh" already exists`
+
+**Solution:** This is normal during upgrades. CRD updates are applied automatically.
+
+### Image Pull Errors
+
+**Error:** `Failed to pull image "ghcr.io/k8s-lynq/lynq:latest"`
+
+**Solution:** Ensure your cluster can access GitHub Container Registry (ghcr.io). Check network policies and image pull secrets if needed.
+
+### Permission Denied
+
+**Error:** `Error from server (Forbidden): User "system:serviceaccount:lynq-system:lynq-controller-manager" cannot create resource`
+
+**Solution:** Ensure RBAC resources are installed:
+```bash
+kubectl apply -f config/rbac/
+```
+
+## Next Steps
+
+- [Create your first LynqHub](quickstart.md#step-4-deploy-lynqhub)
+- [Learn about Templates](templates.md)
+- [Configure Monitoring](monitoring.md)
+- [Set up Security](security.md)

package/tests/fixtures/input/simple.ko.md

@@ -0,0 +1,27 @@
+# 머신러닝 시작하기
+
+이 가이드는 인공지능과 딥러닝의 기초를 소개합니다.
+
+## 사전 요구사항
+
+- Python 3.8+
+- OpenAI의 API 키
+
+## 설치
+
+```bash
+pip install tensorflow
+```
+
+신경망 모델을 생성하세요:
+
+```python
+model = Sequential([
+    Dense(128, activation='relu'),
+    Dense(10, activation='softmax')
+])
+```
+
+## 다음 단계
+
+딥러닝 기법과 SDK 사용 방법에 대해 더 자세히 알아보세요.

package/tests/fixtures/input/simple.md

@@ -0,0 +1,27 @@
+# Getting Started with Machine Learning
+
+This guide introduces the basics of artificial intelligence and deep learning.
+
+## Prerequisites
+
+- Python 3.8+
+- An API key from OpenAI
+
+## Installation
+
+```bash
+pip install tensorflow
+```
+
+Create a neural network model:
+
+```python
+model = Sequential([
+    Dense(128, activation='relu'),
+    Dense(10, activation='softmax')
+])
+```
+
+## Next Steps
+
+Learn more about deep learning techniques and how to use the SDK.

package/tests/unit/chunker.test.ts

@@ -0,0 +1,229 @@
+import { describe, it, expect } from "vitest";
+import {
+  chunkContent,
+  reassembleChunks,
+  getChunkStats,
+} from "../../src/core/chunker.js";
+
+describe("chunkContent", () => {
+  it("should return single chunk for small content", () => {
+    const content = "This is a small piece of text.";
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+
+    expect(chunks).toHaveLength(1);
+    expect(chunks[0]?.type).toBe("translatable");
+    expect(chunks[0]?.content).toBe(content);
+  });
+
+  it("should preserve code blocks as separate chunks", () => {
+    const content = `# Title
+
+Some text before code.
+
+\`\`\`javascript
+const x = 1;
+console.log(x);
+\`\`\`
+
+Some text after code.`;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+
+    // Should have at least one preserved chunk (code block)
+    const preservedChunks = chunks.filter((c) => c.type === "preserve");
+    expect(preservedChunks.length).toBeGreaterThanOrEqual(1);
+
+    // Code block should contain the JS code
+    const codeChunk = preservedChunks.find((c) =>
+      c.content.includes("const x = 1")
+    );
+    expect(codeChunk).toBeDefined();
+  });
+
+  it("should split large content into multiple chunks", () => {
+    // Create content that exceeds token limit
+    const paragraph =
+      "This is a paragraph with enough words to make it substantial. ".repeat(
+        50
+      );
+    const content = `${paragraph}\n\n${paragraph}\n\n${paragraph}`;
+
+    const chunks = chunkContent(content, { maxTokens: 100 });
+
+    expect(chunks.length).toBeGreaterThan(1);
+  });
+
+  it("should extract header hierarchy metadata", () => {
+    const content = `# Main Title
+
+Introduction text.
+
+## Section One
+
+Content in section one.
+
+### Subsection 1.1
+
+More detailed content.
+
+## Section Two
+
+Content in section two.`;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+
+    // Find chunk containing "More detailed content"
+    const detailChunk = chunks.find((c) =>
+      c.content.includes("More detailed content")
+    );
+    expect(detailChunk?.metadata?.headerHierarchy).toBeDefined();
+
+    // Should have headers in hierarchy
+    if (detailChunk?.metadata?.headerHierarchy) {
+      expect(detailChunk.metadata.headerHierarchy.length).toBeGreaterThan(0);
+    }
+  });
+
+  it("should include previousContext for subsequent chunks", () => {
+    const paragraph =
+      "This is the first paragraph with important context. ".repeat(30);
+    const content = `${paragraph}\n\nThis is the second paragraph that should have previous context.`;
+
+    const chunks = chunkContent(content, { maxTokens: 100 });
+
+    // If there are multiple chunks, later ones should have previousContext
+    if (chunks.length > 1) {
+      const laterChunk = chunks[chunks.length - 1];
+      expect(laterChunk?.metadata?.previousContext).toBeDefined();
+    }
+  });
+
+  it("should handle empty content", () => {
+    const chunks = chunkContent("", { maxTokens: 1000 });
+    expect(chunks).toHaveLength(0);
+  });
+
+  it("should handle content with only code blocks", () => {
+    const content = `\`\`\`python
+print("Hello")
+\`\`\``;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+
+    expect(chunks).toHaveLength(1);
+    expect(chunks[0]?.type).toBe("preserve");
+  });
+});
+
+describe("reassembleChunks", () => {
+  it("should reassemble chunks in correct order", () => {
+    const content = "First part. Second part. Third part.";
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+    const reassembled = reassembleChunks(chunks);
+
+    expect(reassembled).toBe(content);
+  });
+
+  it("should handle multiple chunks correctly", () => {
+    const content = `# Title
+
+First paragraph.
+
+\`\`\`code
+block
+\`\`\`
+
+Second paragraph.`;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+    const reassembled = reassembleChunks(chunks);
+
+    expect(reassembled).toContain("# Title");
+    expect(reassembled).toContain("First paragraph");
+    expect(reassembled).toContain("```code");
+    expect(reassembled).toContain("Second paragraph");
+  });
+
+  it("should preserve line breaks between code blocks and text", () => {
+    const content = `\`\`\`yaml
+key: value
+\`\`\`
+
+::: info Note
+Some info here
+:::`;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+    const reassembled = reassembleChunks(chunks);
+
+    // Should preserve the empty line between code block and info block
+    expect(reassembled).toBe(content);
+    expect(reassembled).toContain("```\n\n:::");
+  });
+
+  it("should preserve whitespace-only segments between preserved sections", () => {
+    const content = `\`\`\`js
+code1
+\`\`\`
+
+\`\`\`js
+code2
+\`\`\``;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+    const reassembled = reassembleChunks(chunks);
+
+    // Should preserve the empty lines between code blocks
+    expect(reassembled).toBe(content);
+  });
+
+  it("should preserve exact line breaks between code block and following text (issue #187)", () => {
+    // This is the exact pattern from lynq-installation.md line 187
+    const content = `\`\`\`yaml
+patches:
+- path: manager_webhook_patch.yaml
+\`\`\`
+
+::: info cert-manager responsibilities
+- Issue TLS certificates
+:::`;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+    const reassembled = reassembleChunks(chunks);
+
+    // The reassembled content must be exactly equal to original
+    expect(reassembled).toBe(content);
+    // Specifically verify the line break between code block and info block
+    expect(reassembled).toContain("```\n\n:::");
+    expect(reassembled).not.toContain("```:::"); // No missing line break
+  });
+});
+
+describe("getChunkStats", () => {
+  it("should return correct statistics", () => {
+    const content = `Some text before.
+
+\`\`\`js
+code block
+\`\`\`
+
+Some text after.`;
+
+    const chunks = chunkContent(content, { maxTokens: 1000 });
+    const stats = getChunkStats(chunks);
+
+    expect(stats.totalChunks).toBe(chunks.length);
+    expect(stats.preservedChunks).toBeGreaterThanOrEqual(1);
+    expect(stats.translatableChunks).toBeGreaterThanOrEqual(1);
+    expect(stats.totalTokens).toBeGreaterThan(0);
+  });
+
+  it("should handle empty chunk array", () => {
+    const stats = getChunkStats([]);
+
+    expect(stats.totalChunks).toBe(0);
+    expect(stats.preservedChunks).toBe(0);
+    expect(stats.translatableChunks).toBe(0);
+    expect(stats.averageTokens).toBe(0);
+  });
+});