melaka 0.0.0

package/docs/INTEGRATION.md

@@ -0,0 +1,477 @@
+# Integration Guide
+
+This guide walks you through integrating Melaka into your Firebase project.
+
+## Prerequisites
+
+- Firebase project with Firestore enabled
+- Firebase CLI installed (`npm install -g firebase-tools`)
+- Cloud Functions enabled (requires Blaze plan for external API calls)
+- Node.js 18+
+
+## Quick Start
+
+### 1. Install Melaka CLI
+
+```bash
+npm install -g melaka
+```
+
+### 2. Initialize in Your Project
+
+Navigate to your Firebase project root (where `firebase.json` lives):
+
+```bash
+cd my-firebase-project
+melaka init
+```
+
+This creates `melaka.config.ts`:
+
+```typescript
+import { defineConfig } from 'melaka';
+
+export default defineConfig({
+  languages: ['ms-MY', 'zh-CN'],
+  
+  ai: {
+    provider: 'gemini',
+    model: 'gemini-2.5-flash',
+    apiKeySecret: 'GEMINI_API_KEY',
+  },
+  
+  collections: [
+    {
+      path: 'articles',
+      fields: ['title', 'content'],
+    },
+  ],
+});
+```
+
+### 3. Configure Your Collections
+
+Edit `melaka.config.ts` to match your Firestore structure:
+
+```typescript
+import { defineConfig } from 'melaka';
+
+export default defineConfig({
+  languages: ['ms-MY', 'zh-CN', 'ja-JP'],
+  
+  ai: {
+    provider: 'gemini',
+    model: 'gemini-2.5-flash',
+    apiKeySecret: 'GEMINI_API_KEY',
+  },
+  
+  region: 'asia-southeast1',  // Match your Firestore region
+  
+  collections: [
+    {
+      path: 'products',
+      fields: ['name', 'description', 'features'],
+      prompt: 'E-commerce product descriptions. Keep brand names unchanged.',
+    },
+    {
+      path: 'articles',
+      fields: ['title', 'content', 'summary'],
+      prompt: 'Blog articles. Maintain markdown formatting.',
+    },
+    {
+      path: 'categories',
+      fields: ['name', 'description'],
+    },
+  ],
+  
+  glossary: {
+    'Acme Store': 'Acme Store',  // Don't translate brand
+  },
+});
+```
+
+### 4. Set Up AI Provider API Key
+
+Store your API key as a Firebase secret:
+
+```bash
+# For Gemini (recommended)
+firebase functions:secrets:set GEMINI_API_KEY
+
+# Or for OpenAI
+firebase functions:secrets:set OPENAI_API_KEY
+
+# Or for Claude
+firebase functions:secrets:set ANTHROPIC_API_KEY
+```
+
+### 5. Deploy
+
+```bash
+melaka deploy
+```
+
+This command:
+1. Reads your `melaka.config.ts`
+2. Generates Cloud Functions code in your `functions/` directory
+3. Deploys the functions to Firebase
+
+### 6. Done!
+
+Your translations will now happen automatically when documents are created or updated.
+
+---
+
+## What Gets Deployed
+
+After `melaka deploy`, your `functions/` folder will have:
+
+```
+functions/
+├── src/
+│   ├── index.ts              # Your existing code (unchanged)
+│   └── melaka/               # Generated by Melaka
+│       ├── triggers.ts       # Firestore triggers for each collection
+│       ├── task-handler.ts   # Cloud Task processor
+│       └── index.ts          # Exports all Melaka functions
+```
+
+### Generated Functions
+
+For each collection in your config, Melaka generates:
+
+| Function | Type | Purpose |
+|----------|------|---------|
+| `melakaTranslate{Collection}` | `onDocumentWritten` | Triggered when doc changes, enqueues translation |
+| `melakaTranslateTask` | `onTaskDispatched` | Processes translation tasks asynchronously |
+
+### Example: Generated Trigger
+
+```typescript
+// Auto-generated: melaka/triggers.ts
+import { onDocumentWritten } from 'firebase-functions/v2/firestore';
+
+export const melakaTranslateProducts = onDocumentWritten(
+  { document: 'products/{docId}', region: 'asia-southeast1' },
+  async (event) => {
+    if (!event.data?.after.exists) return; // Skip deletes
+    
+    await enqueueTranslation({
+      collection: 'products',
+      documentId: event.params.docId,
+      languages: ['ms-MY', 'zh-CN', 'ja-JP'],
+    });
+  }
+);
+```
+
+---
+
+## How Cloud Tasks Work
+
+Melaka uses Firebase Cloud Tasks for reliable, async translation processing.
+
+### Why Cloud Tasks?
+
+- **No timeout issues** — Triggers return immediately, translation happens async
+- **Automatic retries** — Failed translations retry with exponential backoff
+- **Rate limiting** — Prevents overwhelming the AI API
+- **Reliable delivery** — Tasks persist even if functions restart
+
+### Automatic Setup
+
+**You don't need to configure Cloud Tasks manually.** Firebase Functions v2's `onTaskDispatched` automatically:
+
+1. Creates the task queue when deployed
+2. Configures HTTP targets and authentication
+3. Handles IAM permissions
+
+### Task Flow
+
+```
+┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
+│  Document       │     │  Firestore      │     │  Cloud Task     │
+│  Updated        │────▶│  Trigger        │────▶│  Enqueued       │
+└─────────────────┘     └─────────────────┘     └────────┬────────┘
+                                                         │
+                        ┌─────────────────┐              │
+                        │  i18n Subcolls  │              │
+                        │  Updated        │◀─────────────┤
+                        └─────────────────┘              │
+                                                         ▼
+                        ┌─────────────────┐     ┌─────────────────┐
+                        │  AI             │     │  Task Handler   │
+                        │  Translation    │◀────│  Processes      │
+                        └─────────────────┘     └─────────────────┘
+```
+
+### Task Configuration
+
+Default settings (configurable in `melaka.config.ts`):
+
+```typescript
+{
+  retryConfig: {
+    maxAttempts: 3,
+    minBackoffSeconds: 60,
+    maxBackoffSeconds: 300,
+  },
+  rateLimits: {
+    maxConcurrentDispatches: 10,
+  },
+}
+```
+
+---
+
+## Translation Storage
+
+Translations are stored as subcollections under each document:
+
+```
+/products/product-123                    ← Source document
+/products/product-123/i18n/ms-MY         ← Malay translation
+/products/product-123/i18n/zh-CN         ← Chinese translation
+/products/product-123/i18n/ja-JP         ← Japanese translation
+```
+
+### Translation Document Structure
+
+```typescript
+// /products/product-123/i18n/ms-MY
+{
+  // Translated fields
+  name: "Telefon Pintar XYZ",
+  description: "Telefon pintar terbaru dengan...",
+  features: ["Kamera 48MP", "Bateri 5000mAh"],
+  
+  // Non-translatable fields (copied from source)
+  price: 999,
+  sku: "XYZ-001",
+  category_ref: "/categories/electronics",
+  
+  // Melaka metadata
+  _melaka: {
+    source_hash: "a1b2c3d4...",
+    translated_at: Timestamp,
+    model: "gemini-2.5-flash",
+    status: "completed",
+    reviewed: false,
+  }
+}
+```
+
+---
+
+## Reading Translations in Your App
+
+### Web/Mobile Client
+
+```typescript
+import { doc, getDoc } from 'firebase/firestore';
+
+async function getProduct(productId: string, locale: string) {
+  // Get translated version
+  const i18nRef = doc(db, 'products', productId, 'i18n', locale);
+  const i18nSnap = await getDoc(i18nRef);
+  
+  if (i18nSnap.exists()) {
+    return i18nSnap.data();
+  }
+  
+  // Fallback to source document
+  const sourceRef = doc(db, 'products', productId);
+  const sourceSnap = await getDoc(sourceRef);
+  return sourceSnap.data();
+}
+```
+
+### With Fallback Helper
+
+```typescript
+async function getLocalizedDoc(
+  collection: string,
+  docId: string,
+  locale: string,
+  fallbackLocale = 'en'
+) {
+  // Try requested locale
+  const i18nSnap = await getDoc(
+    doc(db, collection, docId, 'i18n', locale)
+  );
+  if (i18nSnap.exists()) return i18nSnap.data();
+  
+  // Try fallback locale
+  if (locale !== fallbackLocale) {
+    const fallbackSnap = await getDoc(
+      doc(db, collection, docId, 'i18n', fallbackLocale)
+    );
+    if (fallbackSnap.exists()) return fallbackSnap.data();
+  }
+  
+  // Return source document
+  const sourceSnap = await getDoc(doc(db, collection, docId));
+  return sourceSnap.data();
+}
+```
+
+---
+
+## Firestore Security Rules
+
+Add rules to protect i18n subcollections:
+
+```javascript
+rules_version = '2';
+service cloud.firestore {
+  match /databases/{database}/documents {
+    
+    // Your existing rules for main collections
+    match /products/{productId} {
+      allow read: if true;
+      allow write: if request.auth != null;
+      
+      // i18n subcollection: read-only for clients
+      match /i18n/{locale} {
+        allow read: if true;
+        allow write: if false;  // Only Cloud Functions can write
+      }
+    }
+    
+    match /articles/{articleId} {
+      allow read: if true;
+      allow write: if request.auth != null;
+      
+      match /i18n/{locale} {
+        allow read: if true;
+        allow write: if false;
+      }
+    }
+  }
+}
+```
+
+---
+
+## Manual Translation
+
+For existing documents or re-translation:
+
+```bash
+# Translate all documents in a collection
+melaka translate --collection products
+
+# Translate specific language only
+melaka translate --collection products --language ms-MY
+
+# Force re-translate (ignore content hash)
+melaka translate --collection products --force
+
+# Dry run (show what would be translated)
+melaka translate --collection products --dry-run
+```
+
+---
+
+## Monitoring
+
+### Check Translation Status
+
+```bash
+melaka status
+
+# Output:
+# products
+#   ms-MY: 142/150 (94.7%) ✓ 142  ✗ 3  ⏳ 5
+#   zh-CN: 150/150 (100%)  ✓ 150  ✗ 0  ⏳ 0
+#   ja-JP: 148/150 (98.7%) ✓ 148  ✗ 2  ⏳ 0
+#
+# articles
+#   ms-MY: 45/50 (90%)     ✓ 45   ✗ 0  ⏳ 5
+#   ...
+```
+
+### Retry Failed Translations
+
+```bash
+# Retry all failed
+melaka retry
+
+# Retry specific collection/language
+melaka retry --collection products --language ms-MY
+```
+
+---
+
+## Project Structure Example
+
+After integration, your project looks like:
+
+```
+my-firebase-project/
+├── firebase.json
+├── firestore.rules
+├── melaka.config.ts          # Your Melaka configuration
+├── functions/
+│   ├── package.json
+│   ├── src/
+│   │   ├── index.ts          # Your existing functions
+│   │   └── melaka/           # Generated by Melaka
+│   │       ├── index.ts
+│   │       ├── triggers.ts
+│   │       └── task-handler.ts
+│   └── tsconfig.json
+└── ...
+```
+
+---
+
+## Updating Configuration
+
+When you change `melaka.config.ts`:
+
+```bash
+# Re-deploy with new config
+melaka deploy
+```
+
+This regenerates the Cloud Functions and redeploys.
+
+---
+
+## Troubleshooting
+
+### "Function deployment failed"
+
+- Ensure you're on Blaze plan (required for external API calls)
+- Check that your Firebase CLI is logged in: `firebase login`
+- Verify region matches your Firestore region
+
+### "API key not found"
+
+- Set the secret: `firebase functions:secrets:set GEMINI_API_KEY`
+- Ensure `apiKeySecret` in config matches the secret name
+
+### "Translations not appearing"
+
+- Check Cloud Functions logs: `firebase functions:log`
+- Verify the collection path matches your Firestore structure
+- Run `melaka status` to see pending/failed translations
+
+### "Rate limit exceeded"
+
+- Reduce `maxConcurrency` in config:
+  ```typescript
+  defaults: {
+    maxConcurrency: 5,  // Lower from default 10
+  }
+  ```
+
+---
+
+## Next Steps
+
+- [Configuration Reference](./CONFIGURATION.md) — All config options
+- [CLI Reference](./CLI.md) — Available commands
+- [AI Providers](./AI_PROVIDERS.md) — Provider-specific setup
+- [Architecture](./ARCHITECTURE.md) — System design details

package/docs/ROADMAP.md

@@ -0,0 +1,248 @@
+# Roadmap
+
+This document outlines the planned development phases for Melaka.
+
+---
+
+## Current Status: 🚧 Planning
+
+We're in the architecture and planning phase. No code has been written yet.
+
+---
+
+## Phase 1: MVP (v0.1.0)
+
+**Goal:** Minimal viable product that can translate Firestore documents.
+
+### Core Features
+
+- [ ] **@melaka/core**
+  - [ ] Config file parsing (`melaka.config.ts`)
+  - [ ] TypeScript types and interfaces
+  - [ ] Zod schema generation from field types
+  - [ ] Content hashing for change detection
+  - [ ] Glossary handling
+
+- [ ] **@melaka/ai**
+  - [ ] Gemini adapter (primary)
+  - [ ] Translation facade with structured output
+  - [ ] Prompt construction with glossary
+
+- [ ] **@melaka/firestore**
+  - [ ] i18n subcollection read/write
+  - [ ] Cloud Task handler for translations
+  - [ ] Trigger generator for `onDocumentWritten`
+  - [ ] Batch processing with rate limiting
+
+- [ ] **@melaka/cli**
+  - [ ] `melaka init` — Create config file
+  - [ ] `melaka deploy` — Deploy triggers
+  - [ ] `melaka translate` — Manual translation
+  - [ ] `melaka status` — Check progress
+
+### Documentation
+
+- [ ] README with quick start
+- [ ] Architecture documentation
+- [ ] Configuration reference
+- [ ] Basic examples
+
+### Milestone
+
+✅ MVP is complete when you can:
+1. Run `melaka init` in a Firebase project
+2. Configure collections in `melaka.config.ts`
+3. Run `melaka deploy` to deploy triggers
+4. See automatic translations in `i18n` subcollections
+
+---
+
+## Phase 2: Production Ready (v0.2.0)
+
+**Goal:** Ready for production use with reliability features.
+
+### Features
+
+- [ ] **Additional AI Providers**
+  - [ ] OpenAI adapter
+  - [ ] Claude adapter
+
+- [ ] **Reliability**
+  - [ ] Retry logic with exponential backoff
+  - [ ] Error recording and status tracking
+  - [ ] `melaka retry` command
+
+- [ ] **Monitoring**
+  - [ ] Detailed status output
+  - [ ] Progress tracking
+  - [ ] JSON output for CI/CD
+
+- [ ] **CLI Improvements**
+  - [ ] `melaka validate` — Config validation
+  - [ ] `melaka cleanup` — Remove old translations
+  - [ ] `--dry-run` flag for all commands
+  - [ ] Verbose and quiet modes
+
+### Documentation
+
+- [ ] AI provider comparison and setup
+- [ ] Troubleshooting guide
+- [ ] CI/CD integration guide
+
+### Milestone
+
+✅ Phase 2 is complete when Melaka can be used in production with confidence.
+
+---
+
+## Phase 3: Developer Experience (v0.3.0)
+
+**Goal:** Great developer experience and tooling.
+
+### Features
+
+- [ ] **Collection Groups**
+  - [ ] Support for `isCollectionGroup: true`
+  - [ ] Subcollection translation
+
+- [ ] **Export/Import**
+  - [ ] `melaka export` — Export translations to JSON/CSV
+  - [ ] `melaka import` — Import reviewed translations
+  - [ ] Human review workflow support
+
+- [ ] **Field Mappings**
+  - [ ] Detailed field configuration
+  - [ ] Field-level descriptions for AI context
+  - [ ] Custom schema type overrides
+
+- [ ] **Local Development**
+  - [ ] Firebase emulator support
+  - [ ] Local translation preview
+  - [ ] Watch mode for development
+
+### Documentation
+
+- [ ] Advanced configuration examples
+- [ ] Human review workflow guide
+- [ ] Local development guide
+
+---
+
+## Phase 4: Dashboard (v0.4.0)
+
+**Goal:** Web UI for translation management.
+
+### Features
+
+- [ ] **Review Dashboard**
+  - [ ] Web interface for reviewing translations
+  - [ ] Side-by-side source/translation view
+  - [ ] Mark as reviewed functionality
+  - [ ] Edit and save translations
+
+- [ ] **Analytics**
+  - [ ] Translation statistics
+  - [ ] Cost tracking
+  - [ ] Error monitoring
+
+- [ ] **Team Features**
+  - [ ] Multiple reviewers
+  - [ ] Review assignments
+  - [ ] Comment system
+
+### Technical
+
+- [ ] React/Next.js dashboard
+- [ ] Firebase Auth integration
+- [ ] Hosted version option
+
+---
+
+## Phase 5: Enterprise (v1.0.0)
+
+**Goal:** Enterprise-ready features for large-scale deployments.
+
+### Features
+
+- [ ] **Translation Memory**
+  - [ ] Store and reuse translations
+  - [ ] Similarity matching
+  - [ ] Cost savings through reuse
+
+- [ ] **Advanced Glossary**
+  - [ ] Glossary management UI
+  - [ ] Import from industry standards (TBX)
+  - [ ] Per-language glossaries
+
+- [ ] **Compliance**
+  - [ ] Audit logging
+  - [ ] Data residency options
+  - [ ] SSO integration
+
+- [ ] **Performance**
+  - [ ] Incremental field translation
+  - [ ] Parallel processing improvements
+  - [ ] Large document handling
+
+---
+
+## Future Ideas
+
+These are ideas for future consideration, not committed features:
+
+### Database Adapters
+- Supabase/PostgreSQL adapter
+- MongoDB adapter
+- PlanetScale adapter
+
+### Integrations
+- GitHub Action for CI/CD
+- VS Code extension
+- Slack notifications
+
+### AI Features
+- Quality scoring
+- Auto-detect source language
+- Translation suggestions
+- Custom fine-tuned models
+
+### Localization
+- RTL language support
+- Pluralization rules
+- Date/number formatting
+
+---
+
+## Contributing
+
+We welcome contributions! See [CONTRIBUTING.md](../CONTRIBUTING.md) for guidelines.
+
+### Priority Areas
+
+1. **Core functionality** — Help build the MVP
+2. **Documentation** — Improve docs and examples
+3. **Testing** — Add unit and integration tests
+4. **AI adapters** — Add OpenAI and Claude support
+
+---
+
+## Release Schedule
+
+| Version | Target | Focus |
+|---------|--------|-------|
+| v0.1.0 | Q2 2026 | MVP |
+| v0.2.0 | Q2 2026 | Production ready |
+| v0.3.0 | Q3 2026 | Developer experience |
+| v0.4.0 | Q3 2026 | Dashboard |
+| v1.0.0 | Q4 2026 | Enterprise |
+
+*Dates are estimates and subject to change.*
+
+---
+
+## Feedback
+
+Have ideas or feature requests? 
+
+- Open an issue on [GitHub](https://github.com/rizahassan/melaka/issues)
+- Start a discussion in [GitHub Discussions](https://github.com/rizahassan/melaka/discussions)