@easyling/sanity-connector 1.4.0 → 2.0.0

package/README.md

@@ -1,154 +1,624 @@
-# Sanity Translation Plugin
+# @easyling/sanity-connector
 
-A Sanity Studio plugin that enables seamless document translation workflows. It extracts content as HTML, manages translation requests to external services, and handles the re-import of translated content while preserving Portable Text structures.
+Sanity Studio v3 plugin for document translation with Easyling. Provides UI-based translation functionality with support for single document translation, bulk operations, and granular control over translatable content.
 
 ## Features
 
-- **Single Document Translation**: Translate individual documents directly from the document form.
-- **Bulk Translation**: Batch process multiple documents via a dedicated tool.
-- **Smart Content Extraction**: Converts Sanity Portable Text to HTML for translation and back again.
-- **Translation Collision Handling**: Intelligent detection and resolution when re-translating to the same locale.
-  - **Replace Mode**: Overwrite existing translated documents with new translations.
-  - **Create Mode**: Create new documents while managing existing ones (draft or delete).
-- **Translation Invariance**: granular control over what gets translated via "Do Not Translate" (DNT) flags.
-  - **Inline Badges**: Toggle translation status for individual fields directly in the editor.
-  - **Field Management**: dedicated action to manage DNT status across the whole document.
-- **Authentication Support**: Secure OAuth integration with translation services, with a legacy mode for local testing.
-- **Auto-Configuration**: Intelligent defaults for new installations to get you started quickly.
-- **Progress Tracking**: Real-time feedback for long-running bulk operations.
+- **Single Document Translation**: Translate individual documents directly from the document form
+- **Bulk Translation Tool**: Batch process multiple documents via a dedicated tool
+- **Smart Content Extraction**: Converts Sanity Portable Text to HTML for translation and back again
+- **Translation Collision Handling**: Intelligent detection and resolution when re-translating to the same locale
+- **Do Not Translate (DNT) Control**: Granular field-level control over what gets translated
+  - Inline badges for toggling translation status
+  - Dedicated field management action
+  - Smart defaults based on field types and names
+- **OAuth Integration**: Secure authentication with translation services
+- **Real-time Progress Tracking**: Visual feedback for long-running operations
+- **Debug Mode**: Detailed field information for troubleshooting
 
 ## Installation
 
-Install the plugin in your Sanity Studio project:
-
 ```bash
-npm install sanity-translation-plugin
+npm install @easyling/sanity-connector
 ```
 
-Add the plugin to your `sanity.config.ts` file:
+### Peer Dependencies
+
+The plugin requires the following peer dependencies:
+
+- `sanity` ^3.0.0
+- `react` ^18.0.0
+- `react-dom` ^18.0.0
+
+These are typically already installed in your Sanity Studio project.
+
+## Setup
+
+### Basic Configuration
+
+Add the plugin to your `sanity.config.ts` or `sanity.config.js`:
 
 ```typescript
 import { defineConfig } from 'sanity'
-import translationPlugin from 'sanity-translation-plugin'
+import translationPlugin from '@easyling/sanity-connector'
 
 export default defineConfig({
-  // ... your other config
+  // ... other config
   plugins: [
-    // ... other plugins
     translationPlugin()
   ]
 })
 ```
 
-## Configuration
+That's it! The plugin will automatically:
+- Register document actions (Translate, Manage DNT Fields)
+- Add the Bulk Translate tool to your Studio
+- Create configuration document types
+- Initialize services on Studio load
+
+### Initial Configuration
+
+After installing the plugin, you'll need to configure:
+
+1. **OAuth Credentials** (required for translation)
+2. **Target Locales** (languages to translate to)
+3. **Translation API Endpoint** (optional, defaults to Easyling)
+
+Configuration is stored in your Sanity dataset as an `EL_PluginConfiguration` document. You can configure through:
+
+- The Studio UI (recommended)
+- Direct document editing
+- The UnifiedConfigStorage API
+
+## Configuration Options
+
+### OAuth Configuration
+
+Configure authentication with your translation service:
+
+1. Navigate to your Studio
+2. Create or edit the `EL_PluginConfiguration` document
+3. Fill in the OAuth section:
+   - **Project ID**: Your translation project identifier
+   - **Access Token**: Long-lived access token (hidden for security)
+
+```typescript
+// Example: Programmatic configuration
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+import { createClient } from 'sanity'
+
+const client = createClient({
+  projectId: 'your-project-id',
+  dataset: 'production',
+  apiVersion: '2024-01-01',
+  token: 'your-token'
+})
+
+const configStorage = new UnifiedConfigStorage(client)
+
+await configStorage.update({
+  projectId: 'your-easyling-project-id',
+  accessToken: 'your-access-token'
+})
+```
+
+### Locale Configuration
+
+Define which languages you want to translate to:
+
+```typescript
+await configStorage.update({
+  locales: [
+    {
+      code: 'en',
+      title: 'English',
+      enabled: true,
+      isDefault: true
+    },
+    {
+      code: 'ja',
+      title: 'Japanese',
+      enabled: true,
+      isDefault: false
+    },
+    {
+      code: 'de',
+      title: 'German',
+      enabled: true,
+      isDefault: false
+    }
+  ]
+})
+```
+
+**Locale Fields:**
+- `code`: Language code (e.g., 'en', 'ja-JP')
+- `title`: Display name for the language
+- `enabled`: Whether this locale is active for translation
+- `isDefault`: Whether this is the source language (only one should be true)
+
+### Translation API Configuration
+
+Customize the translation service endpoint and format:
+
+```typescript
+await configStorage.update({
+  translationApiEndpoint: 'https://api.easyling.com/translate',
+  requestContentType: 'application/x-protobuf', // or 'application/json'
+  responseAcceptHeader: 'application/x-protobuf' // or 'application/json'
+})
+```
+
+**Default values:**
+- Endpoint: Easyling translation API
+- Content Type: `application/x-protobuf`
+- Accept Header: `application/x-protobuf`
+
+### Document Creation Options
+
+Control how translated documents are created:
+
+```typescript
+await configStorage.update({
+  defaultDocumentCreationMode: 'draft', // or 'published'
+  collisionResolutionMode: 'skip', // or 'replace', 'merge'
+  existingDocumentHandling: 'keep' // or 'archive', 'delete'
+})
+```
+
+**Creation Modes:**
+- `draft`: Create translated documents as drafts (recommended)
+- `published`: Create translated documents as published
 
-### Authentication & Connection
+**Collision Resolution:**
+- `skip`: Don't create if translation already exists
+- `replace`: Overwrite existing translation
+- `merge`: Merge with existing translation (preserves untranslated fields)
 
-The plugin supports two modes of operation:
+**Existing Document Handling:**
+- `keep`: Keep existing document as-is
+- `archive`: Move existing document to drafts
+- `delete`: Delete existing document
 
-1.  **OAuth Mode (Recommended for Production)**:
-    -   Uses secure token-based authentication.
-    -   Configurable via the **OAuth Configuration** tool within Sanity Studio.
-    -   Supports project-specific access control.
+### DNT (Do Not Translate) Configuration
 
-2.  **Legacy/Development Mode**:
-    -   Default behavior out of the box.
-    -   Points to `http://app.easyling.com` by default.
-    -   Useful for local development and testing without setting up full auth.
+Configure which fields should not be translated by default:
 
-To configure the plugin for production use, navigate to the **Easyling Translation Plugin Configuration** document type in your Sanity Studio after installation, and create a new configuration. Enter your project code and access token (both are available in the Easyling platform), and publish your configuration.
+```typescript
+await configStorage.setDNTField('article', 'author.name', true)
+await configStorage.setDNTField('article', 'publishDate', true)
+```
 
-Next, navigate to the **Locales** tab of the configuration, and add the locales you wish to handle. These are the ones you will be able to translate your content into.
+The plugin automatically applies smart defaults for common non-translatable fields:
+- Author names
+- Dates and timestamps
+- IDs and references
+- Technical metadata
 
-At this point, you're ready to begin processing content, but if you want, you can add translation-invariant fields for your content types, by toggling the badges on the individual fields.
+You can override these defaults on a per-field basis using the inline DNT badges or the Manage DNT Fields action.
 
-### Defaults
+### Debug Mode
 
-The plugin comes with "Auto-Default" behavior. When you install it for the first time, it will automatically select sensible defaults (like "Draft" mode for new translated documents) so you can start using it immediately without manual configuration.
+Enable debug mode to see detailed field information:
+
+```typescript
+await configStorage.update({
+  debugMode: true
+})
+```
+
+When enabled, all fields will show debug badges with:
+- Field path
+- Field type
+- DNT status (default and stored)
+- Group/fieldset information
 
 ## Usage
 
 ### Single Document Translation
 
-1.  Open a document in Sanity Studio.
-2.  Click the **Translate** icon in the document toolbar.
-3.  Review the document details in the confirmation dialog.
-4.  Click **Translate**.
-5.  The plugin will process the translation and inform you when complete.
+1. Open any document in your Studio
+2. Click the **Translate** button in the document toolbar
+3. Select target languages
+4. Configure translation options (creation mode, collision handling)
+5. Click **Translate**
+
+The plugin will:
+- Extract translatable content from the document
+- Send translation request to the configured API
+- Create new documents for each target language
+- Show progress and results
 
 ### Bulk Translation
 
-1.  Open the **Bulk Translate** tool from the Sanity Studio tools menu.
-2.  Select the document types you wish to translate.
-3.  Filter and select specific documents from the list.
-4.  Click the **Translate** button to process the batch.
-5.  A summary report will be displayed upon completion.
+1. Open the **Bulk Translate** tool from the Studio tools menu
+2. Select document type to translate
+3. Choose documents (or select all)
+4. Select target languages
+5. Configure translation options
+6. Click **Start Translation**
+
+The tool provides:
+- Real-time progress tracking
+- Success/failure counts
+- Detailed results for each document
+- Rollback capability if needed
 
-### Managing Translatable Fields
+### Managing DNT Fields
 
-Sometimes you need to keep specific fields (like product codes, technical IDs, or proper names) from being translated.
+#### Inline Badges
 
-**Method 1: Inline Badges**
-In the document editor, you will see small badges below translatable fields:
--   **Green "Will Translate"**: The field will be included in the translation request.
--   **Red "Do Not Translate"**: The field will be skipped.
-Click the badge to toggle the status.
+For translatable fields (string, text, array), you'll see a DNT badge:
+- **Green badge**: Field will be translated
+- **Red badge**: Field will NOT be translated
+- Click the badge to toggle the DNT status
 
-**Method 2: Field Management Action**
-For a high-level view:
-1.  Click the document menu (three dots) in the top right.
-2.  Select **Manage Translation Fields**.
-3.  View and toggle the translation status for all fields in the document.
+#### Manage DNT Fields Action
 
-### Translation Collision Handling
+1. Open any document
+2. Click **Manage DNT Fields** in the document toolbar
+3. View all fields with their DNT status
+4. Toggle individual fields or apply bulk changes
+5. Save changes
 
-When translating a document to a locale that already has an existing translation, the plugin detects this "collision" and offers resolution options.
+The action shows:
+- Field path and type
+- Current DNT status
+- Smart default recommendations
+- Ability to reset to defaults
 
-**Collision Detection**
-A collision is detected when a translated document already exists for the same source document and target locale. The plugin identifies existing translations by:
-1.  **Parent Reference**: Documents linked via `parentDocument._ref` to the source document.
-2.  **Slug + Locale**: Documents with matching slug and locale (as a fallback).
+## Migration from Previous Version
 
-**Resolution Modes**
+If you're upgrading from the single-package version (pre-monorepo), follow these steps:
 
-When a collision is detected, you can choose how to handle it:
+### 1. Update Package Name
 
--   **Replace existing translated document**: Overwrites the existing translation with the new content. The document ID remains the same.
--   **Create new translated document**: Creates a new document with a unique ID. You can also specify what happens to the existing document:
-    -   **Set existing document to draft**: Converts the existing published translation to a draft.
-    -   **Delete existing document**: Permanently removes the existing translation.
+The plugin package name remains the same, but the internal structure has changed:
 
-**Configuration**
+```bash
+# Uninstall old version
+npm uninstall @easyling/sanity-connector
 
-Default collision resolution settings can be configured in the **Easyling Translation Plugin Configuration** document:
+# Install new version
+npm install @easyling/sanity-connector
+```
 
-1.  Navigate to the plugin configuration document in Sanity Studio.
-2.  Find the **Translation** settings group.
-3.  Set the **Default Collision Resolution Mode** (Replace or Create).
-4.  If using Create mode, set the **Existing Document Handling** preference (Draft or Delete).
+### 2. Update Imports
 
-These defaults can be overridden on a per-translation basis in the locale selection dialog.
+Most imports remain the same, but some have moved to the shared library:
+
+**Before:**
+```typescript
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+```
+
+**After:**
+```typescript
+// Still works - re-exported from plugin
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+
+// Or import directly from shared library
+import { UnifiedConfigStorage } from '@easyling/sanity-connector-shared'
+```
+
+### 3. Configuration Migration
+
+The plugin automatically migrates legacy configuration documents:
+
+- **Legacy OAuth Config** (`EL_oauth-config`) → Merged into `EL_PluginConfiguration`
+- **Legacy Locale Config** (`EL_locale-config`) → Merged into `EL_PluginConfiguration`
+
+Migration happens automatically on first load. Legacy documents are hidden from the UI but preserved for rollback if needed.
+
+### 4. Verify Configuration
+
+After migration, verify your configuration:
+
+1. Open the `EL_PluginConfiguration` document in your Studio
+2. Check that OAuth credentials are present
+3. Verify locale settings
+4. Test a translation to ensure everything works
+
+### 5. Update Custom Code (if applicable)
+
+If you have custom code using the plugin's APIs:
+
+**Service Imports:**
+```typescript
+// Before
+import { ContentExtractor } from '@easyling/sanity-connector/services'
+
+// After - import from shared library
+import { ContentExtractor } from '@easyling/sanity-connector-shared'
+```
+
+**Type Imports:**
+```typescript
+// Before
+import type { TranslationRequest } from '@easyling/sanity-connector/types'
+
+// After - still works, or use shared library
+import type { TranslationRequest } from '@easyling/sanity-connector'
+// or
+import type { TranslationRequest } from '@easyling/sanity-connector-shared'
+```
+
+### Breaking Changes
+
+The monorepo restructuring introduces minimal breaking changes:
+
+1. **Internal file structure**: If you were importing from internal paths (not recommended), update to use public exports
+2. **Service initialization**: Some services now require explicit client initialization
+3. **Configuration storage**: Legacy config documents are deprecated (but still supported)
+
+### Rollback
+
+If you need to rollback to the previous version:
+
+```bash
+npm install @easyling/sanity-connector@1.3.0
+```
+
+Your configuration will remain intact as the document structure hasn't changed.
+
+## Advanced Usage
+
+### Custom Translation Workflow
+
+You can build custom translation workflows using the exported services:
+
+```typescript
+import {
+  ContentExtractor,
+  TranslationService,
+  DocumentCreationService,
+  UnifiedConfigStorage
+} from '@easyling/sanity-connector-shared'
+import { useClient } from 'sanity'
+
+function MyCustomTranslationComponent() {
+  const client = useClient()
+  
+  const handleCustomTranslation = async (document) => {
+    // Initialize services
+    const configStorage = new UnifiedConfigStorage(client)
+    const extractor = new ContentExtractor(client)
+    const translationService = new TranslationService(client)
+    const docCreationService = new DocumentCreationService(client)
+    
+    // Load configuration
+    const config = await configStorage.load()
+    translationService.setEndpoint(config.translationApiEndpoint)
+    translationService.setAuthToken(config.accessToken, config.projectId)
+    
+    // Extract content
+    const structured = await extractor.extractStructuredContent(
+      document,
+      configStorage
+    )
+    
+    // Translate
+    const response = await translationService.translateDocument(
+      structured,
+      document._type,
+      'en',
+      undefined,
+      ['ja', 'de']
+    )
+    
+    // Create translated documents
+    for (const translatedDoc of response.translatedDocuments) {
+      await docCreationService.createDocumentFromTranslation(
+        document,
+        { ...response, translatedDocuments: [translatedDoc] },
+        {
+          targetLanguage: translatedDoc.locale,
+          creationMode: 'draft'
+        }
+      )
+    }
+  }
+  
+  return <button onClick={() => handleCustomTranslation(doc)}>Translate</button>
+}
+```
+
+### Custom DNT Logic
+
+Implement custom DNT logic for your schema:
+
+```typescript
+import { DNTServiceManager } from '@easyling/sanity-connector'
+import { useClient } from 'sanity'
+
+function MyComponent() {
+  const client = useClient()
+  const dntService = DNTServiceManager.getInstance(client)
+  
+  const checkDNTStatus = async (documentType, fieldPath) => {
+    const isDNT = await dntService.getDNTFieldStatus(documentType, fieldPath)
+    console.log(`Field ${fieldPath} is ${isDNT ? 'NOT' : ''} translatable`)
+  }
+  
+  const setDNTStatus = async (documentType, fieldPath, isDNT) => {
+    await dntService.setDNTField(documentType, fieldPath, isDNT)
+  }
+  
+  return <div>Custom DNT UI</div>
+}
+```
+
+### Accessing Configuration
+
+Access plugin configuration in your custom components:
+
+```typescript
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+import { useClient } from 'sanity'
+import { useEffect, useState } from 'react'
+
+function MyConfigComponent() {
+  const client = useClient()
+  const [config, setConfig] = useState(null)
+  
+  useEffect(() => {
+    const loadConfig = async () => {
+      const configStorage = new UnifiedConfigStorage(client)
+      const cfg = await configStorage.load()
+      setConfig(cfg)
+    }
+    loadConfig()
+  }, [client])
+  
+  return (
+    <div>
+      <h2>Current Configuration</h2>
+      <pre>{JSON.stringify(config, null, 2)}</pre>
+    </div>
+  )
+}
+```
 
 ## Troubleshooting
 
-**Plugin not appearing in Studio**
--   Verify `npm install sanity-translation-plugin` completed successfully.
--   Ensure `translationPlugin()` is in the `plugins` array in `sanity.config.ts`.
--   Restart the Sanity Studio development server.
+### Translation Fails with Authentication Error
+
+**Problem**: Translation requests fail with 401 or 403 errors
+
+**Solution**:
+1. Verify OAuth credentials in `EL_PluginConfiguration`
+2. Check that Project ID and Access Token are correct
+3. Ensure access token hasn't expired
+4. Contact your translation service provider for new credentials
+
+### DNT Badges Not Showing
+
+**Problem**: DNT badges don't appear on fields
+
+**Solution**:
+1. Check that the field type is translatable (string, text, array)
+2. Verify the field isn't blacklisted (locale, _id, _type, etc.)
+3. Enable debug mode to see all fields
+4. Check browser console for errors
+
+### Translated Documents Not Created
 
-**Translation action not visible**
--   The action only appears on documents that have translatable fields defined in their schema.
+**Problem**: Translation succeeds but no documents are created
 
-**Authentication Issues**
--   If using OAuth, check the **OAuth Configuration** tool to ensure your tokens are valid.
--   If using local testing, ensure your mock translation service is running on the expected port.
+**Solution**:
+1. Check collision resolution mode - may be set to 'skip'
+2. Verify you have write permissions in the dataset
+3. Check browser console for document creation errors
+4. Ensure target locale codes match your schema
+
+### Bulk Translation Stalls
+
+**Problem**: Bulk translation stops or hangs
+
+**Solution**:
+1. Check network connection
+2. Verify translation API is responding
+3. Try translating a single document to isolate the issue
+4. Check browser console for errors
+5. Reduce batch size if translating many documents
+
+### Configuration Not Persisting
+
+**Problem**: Configuration changes don't save
+
+**Solution**:
+1. Verify you have write permissions in the dataset
+2. Check that `EL_PluginConfiguration` document exists
+3. Look for validation errors in the Studio
+4. Check browser console for save errors
+
+### Migration Issues
+
+**Problem**: Legacy configuration not migrated
+
+**Solution**:
+1. Check that legacy documents exist (`EL_oauth-config`, `EL_locale-config`)
+2. Manually create `EL_PluginConfiguration` document
+3. Copy values from legacy documents
+4. Contact support if automatic migration fails
+
+## API Reference
+
+### Exported Components
+
+- `DNTFieldBadge`: Standalone DNT badge component
+- `withDNTBadge`: HOC for adding DNT badge to custom fields
+- `DNTFieldInput`: Complete field input with DNT badge
+
+### Exported Services
+
+- `UnifiedConfigStorage`: Configuration management
+- `DNTServiceManager`: DNT field management
+- `DNTStorageAdapter`: DNT storage interface
+- `LegacyDNTStorageAdapter`: Legacy DNT storage support
+
+### Exported Types
+
+```typescript
+import type {
+  LocaleDefinition,
+  LocaleConfig,
+  TranslationRequestPayload,
+  TranslationResponse,
+  TranslatableField,
+  DNTFieldPreference,
+  DNTPreferences,
+  DNTTypeConfig,
+  DNTStorage,
+  UnifiedPluginConfig
+} from '@easyling/sanity-connector'
+```
+
+See the [shared library documentation](../shared/README.md) for complete API reference.
+
+## Development
+
+### Building
+
+```bash
+npm run build        # Production build
+npm run dev          # Development build with watch
+npm run clean        # Remove build artifacts
+```
+
+### Testing
+
+```bash
+npm test             # Run all tests
+npm run test:watch   # Run tests in watch mode
+npm run test:coverage # Generate coverage report
+```
+
+### Code Quality
+
+```bash
+npm run lint         # Check for linting issues
+npm run lint:fix     # Auto-fix linting issues
+npm run typecheck    # Type check without emitting
+```
 
 ## Support
 
-Our support team can be reached at support@easyling.com.
+For questions, issues, or feature requests:
 
-**Note:** In order to investigate compatibility or interoperability issues, we may need access to your Sanity data lake and Studio codebase. We are happy to work with you to establish secure access for debugging purposes.
+- **Email**: support@easyling.com
+- **Issues**: [GitHub Issues](https://github.com/easyling/el-sanity-connector/issues)
+- **Documentation**: [Main README](../README.md) | [Shared Library](../shared/README.md)
 
 ## License
 
-MIT License - see [LICENSE](LICENSE) file for details.
+MIT
+
+## Related Packages
+
+- [@easyling/sanity-connector-shared](../shared/README.md) - Core translation logic
+- [@easyling/sanity-function-auto-translate](../function/README.md) - Automatic translation on publish
+
+## Changelog
+
+See [CHANGELOG.md](CHANGELOG.md) for version history and migration notes.