@easyling/sanity-connector 1.3.0 → 2.0.0

package/README.md

@@ -1,122 +1,624 @@
-# Sanity Translation Plugin
+# @easyling/sanity-connector
 
-A Sanity Studio plugin that enables seamless document translation workflows. It extracts content as HTML, manages translation requests to external services, and handles the re-import of translated content while preserving Portable Text structures.
+Sanity Studio v3 plugin for document translation with Easyling. Provides UI-based translation functionality with support for single document translation, bulk operations, and granular control over translatable content.
 
 ## Features
 
-- **Single Document Translation**: Translate individual documents directly from the document form.
-- **Bulk Translation**: Batch process multiple documents via a dedicated tool.
-- **Smart Content Extraction**: Converts Sanity Portable Text to HTML for translation and back again.
-- **Translation Invariance**: granular control over what gets translated via "Do Not Translate" (DNT) flags.
-  - **Inline Badges**: Toggle translation status for individual fields directly in the editor.
-  - **Field Management**: dedicated action to manage DNT status across the whole document.
-- **Authentication Support**: Secure OAuth integration with translation services, with a legacy mode for local testing.
-- **Auto-Configuration**: Intelligent defaults for new installations to get you started quickly.
-- **Progress Tracking**: Real-time feedback for long-running bulk operations.
+- **Single Document Translation**: Translate individual documents directly from the document form
+- **Bulk Translation Tool**: Batch process multiple documents via a dedicated tool
+- **Smart Content Extraction**: Converts Sanity Portable Text to HTML for translation and back again
+- **Translation Collision Handling**: Intelligent detection and resolution when re-translating to the same locale
+- **Do Not Translate (DNT) Control**: Granular field-level control over what gets translated
+  - Inline badges for toggling translation status
+  - Dedicated field management action
+  - Smart defaults based on field types and names
+- **OAuth Integration**: Secure authentication with translation services
+- **Real-time Progress Tracking**: Visual feedback for long-running operations
+- **Debug Mode**: Detailed field information for troubleshooting
 
 ## Installation
 
-Install the plugin in your Sanity Studio project:
-
 ```bash
-npm install sanity-translation-plugin
+npm install @easyling/sanity-connector
 ```
 
-Add the plugin to your `sanity.config.ts` file:
+### Peer Dependencies
+
+The plugin requires the following peer dependencies:
+
+- `sanity` ^3.0.0
+- `react` ^18.0.0
+- `react-dom` ^18.0.0
+
+These are typically already installed in your Sanity Studio project.
+
+## Setup
+
+### Basic Configuration
+
+Add the plugin to your `sanity.config.ts` or `sanity.config.js`:
 
 ```typescript
 import { defineConfig } from 'sanity'
-import translationPlugin from 'sanity-translation-plugin'
+import translationPlugin from '@easyling/sanity-connector'
 
 export default defineConfig({
-  // ... your other config
+  // ... other config
   plugins: [
-    // ... other plugins
     translationPlugin()
   ]
 })
 ```
 
-## Configuration
+That's it! The plugin will automatically:
+- Register document actions (Translate, Manage DNT Fields)
+- Add the Bulk Translate tool to your Studio
+- Create configuration document types
+- Initialize services on Studio load
+
+### Initial Configuration
+
+After installing the plugin, you'll need to configure:
+
+1. **OAuth Credentials** (required for translation)
+2. **Target Locales** (languages to translate to)
+3. **Translation API Endpoint** (optional, defaults to Easyling)
+
+Configuration is stored in your Sanity dataset as an `EL_PluginConfiguration` document. You can configure through:
+
+- The Studio UI (recommended)
+- Direct document editing
+- The UnifiedConfigStorage API
+
+## Configuration Options
+
+### OAuth Configuration
+
+Configure authentication with your translation service:
+
+1. Navigate to your Studio
+2. Create or edit the `EL_PluginConfiguration` document
+3. Fill in the OAuth section:
+   - **Project ID**: Your translation project identifier
+   - **Access Token**: Long-lived access token (hidden for security)
+
+```typescript
+// Example: Programmatic configuration
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+import { createClient } from 'sanity'
+
+const client = createClient({
+  projectId: 'your-project-id',
+  dataset: 'production',
+  apiVersion: '2024-01-01',
+  token: 'your-token'
+})
+
+const configStorage = new UnifiedConfigStorage(client)
+
+await configStorage.update({
+  projectId: 'your-easyling-project-id',
+  accessToken: 'your-access-token'
+})
+```
+
+### Locale Configuration
+
+Define which languages you want to translate to:
+
+```typescript
+await configStorage.update({
+  locales: [
+    {
+      code: 'en',
+      title: 'English',
+      enabled: true,
+      isDefault: true
+    },
+    {
+      code: 'ja',
+      title: 'Japanese',
+      enabled: true,
+      isDefault: false
+    },
+    {
+      code: 'de',
+      title: 'German',
+      enabled: true,
+      isDefault: false
+    }
+  ]
+})
+```
+
+**Locale Fields:**
+- `code`: Language code (e.g., 'en', 'ja-JP')
+- `title`: Display name for the language
+- `enabled`: Whether this locale is active for translation
+- `isDefault`: Whether this is the source language (only one should be true)
+
+### Translation API Configuration
+
+Customize the translation service endpoint and format:
+
+```typescript
+await configStorage.update({
+  translationApiEndpoint: 'https://api.easyling.com/translate',
+  requestContentType: 'application/x-protobuf', // or 'application/json'
+  responseAcceptHeader: 'application/x-protobuf' // or 'application/json'
+})
+```
 
-### Authentication & Connection
+**Default values:**
+- Endpoint: Easyling translation API
+- Content Type: `application/x-protobuf`
+- Accept Header: `application/x-protobuf`
 
-The plugin supports two modes of operation:
+### Document Creation Options
 
-1.  **OAuth Mode (Recommended for Production)**:
-    -   Uses secure token-based authentication.
-    -   Configurable via the **OAuth Configuration** tool within Sanity Studio.
-    -   Supports project-specific access control.
+Control how translated documents are created:
 
-2.  **Legacy/Development Mode**:
-    -   Default behavior out of the box.
-    -   Points to `http://app.easyling.com` by default.
-    -   Useful for local development and testing without setting up full auth.
+```typescript
+await configStorage.update({
+  defaultDocumentCreationMode: 'draft', // or 'published'
+  collisionResolutionMode: 'skip', // or 'replace', 'merge'
+  existingDocumentHandling: 'keep' // or 'archive', 'delete'
+})
+```
 
-To configure the plugin for production use, navigate to the **Easyling Translation Plugin Configuration** document type in your Sanity Studio after installation, and create a new configuration. Enter your project code and access token (both are available in the Easyling platform), and publish your configuration.
+**Creation Modes:**
+- `draft`: Create translated documents as drafts (recommended)
+- `published`: Create translated documents as published
 
-Next, navigate to the **Locales** tab of the configuration, and add the locales you wish to handle. These are the ones you will be able to translate your content into.
+**Collision Resolution:**
+- `skip`: Don't create if translation already exists
+- `replace`: Overwrite existing translation
+- `merge`: Merge with existing translation (preserves untranslated fields)
 
-At this point, you're ready to begin processing content, but if you want, you can add translation-invariant fields for your content types, by toggling the badges on the individual fields.
+**Existing Document Handling:**
+- `keep`: Keep existing document as-is
+- `archive`: Move existing document to drafts
+- `delete`: Delete existing document
 
-### Defaults
+### DNT (Do Not Translate) Configuration
 
-The plugin comes with "Auto-Default" behavior. When you install it for the first time, it will automatically select sensible defaults (like "Draft" mode for new translated documents) so you can start using it immediately without manual configuration.
+Configure which fields should not be translated by default:
+
+```typescript
+await configStorage.setDNTField('article', 'author.name', true)
+await configStorage.setDNTField('article', 'publishDate', true)
+```
+
+The plugin automatically applies smart defaults for common non-translatable fields:
+- Author names
+- Dates and timestamps
+- IDs and references
+- Technical metadata
+
+You can override these defaults on a per-field basis using the inline DNT badges or the Manage DNT Fields action.
+
+### Debug Mode
+
+Enable debug mode to see detailed field information:
+
+```typescript
+await configStorage.update({
+  debugMode: true
+})
+```
+
+When enabled, all fields will show debug badges with:
+- Field path
+- Field type
+- DNT status (default and stored)
+- Group/fieldset information
 
 ## Usage
 
 ### Single Document Translation
 
-1.  Open a document in Sanity Studio.
-2.  Click the **Translate** icon in the document toolbar.
-3.  Review the document details in the confirmation dialog.
-4.  Click **Translate**.
-5.  The plugin will process the translation and inform you when complete.
+1. Open any document in your Studio
+2. Click the **Translate** button in the document toolbar
+3. Select target languages
+4. Configure translation options (creation mode, collision handling)
+5. Click **Translate**
+
+The plugin will:
+- Extract translatable content from the document
+- Send translation request to the configured API
+- Create new documents for each target language
+- Show progress and results
 
 ### Bulk Translation
 
-1.  Open the **Bulk Translate** tool from the Sanity Studio tools menu.
-2.  Select the document types you wish to translate.
-3.  Filter and select specific documents from the list.
-4.  Click the **Translate** button to process the batch.
-5.  A summary report will be displayed upon completion.
+1. Open the **Bulk Translate** tool from the Studio tools menu
+2. Select document type to translate
+3. Choose documents (or select all)
+4. Select target languages
+5. Configure translation options
+6. Click **Start Translation**
+
+The tool provides:
+- Real-time progress tracking
+- Success/failure counts
+- Detailed results for each document
+- Rollback capability if needed
+
+### Managing DNT Fields
+
+#### Inline Badges
+
+For translatable fields (string, text, array), you'll see a DNT badge:
+- **Green badge**: Field will be translated
+- **Red badge**: Field will NOT be translated
+- Click the badge to toggle the DNT status
+
+#### Manage DNT Fields Action
+
+1. Open any document
+2. Click **Manage DNT Fields** in the document toolbar
+3. View all fields with their DNT status
+4. Toggle individual fields or apply bulk changes
+5. Save changes
+
+The action shows:
+- Field path and type
+- Current DNT status
+- Smart default recommendations
+- Ability to reset to defaults
+
+## Migration from Previous Version
+
+If you're upgrading from the single-package version (pre-monorepo), follow these steps:
+
+### 1. Update Package Name
+
+The plugin package name remains the same, but the internal structure has changed:
+
+```bash
+# Uninstall old version
+npm uninstall @easyling/sanity-connector
+
+# Install new version
+npm install @easyling/sanity-connector
+```
+
+### 2. Update Imports
+
+Most imports remain the same, but some have moved to the shared library:
+
+**Before:**
+```typescript
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+```
+
+**After:**
+```typescript
+// Still works - re-exported from plugin
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+
+// Or import directly from shared library
+import { UnifiedConfigStorage } from '@easyling/sanity-connector-shared'
+```
+
+### 3. Configuration Migration
+
+The plugin automatically migrates legacy configuration documents:
 
-### Managing Translatable Fields
+- **Legacy OAuth Config** (`EL_oauth-config`) → Merged into `EL_PluginConfiguration`
+- **Legacy Locale Config** (`EL_locale-config`) → Merged into `EL_PluginConfiguration`
 
-Sometimes you need to keep specific fields (like product codes, technical IDs, or proper names) from being translated.
+Migration happens automatically on first load. Legacy documents are hidden from the UI but preserved for rollback if needed.
 
-**Method 1: Inline Badges**
-In the document editor, you will see small badges below translatable fields:
--   **Green "Will Translate"**: The field will be included in the translation request.
--   **Red "Do Not Translate"**: The field will be skipped.
-Click the badge to toggle the status.
+### 4. Verify Configuration
 
-**Method 2: Field Management Action**
-For a high-level view:
-1.  Click the document menu (three dots) in the top right.
-2.  Select **Manage Translation Fields**.
-3.  View and toggle the translation status for all fields in the document.
+After migration, verify your configuration:
+
+1. Open the `EL_PluginConfiguration` document in your Studio
+2. Check that OAuth credentials are present
+3. Verify locale settings
+4. Test a translation to ensure everything works
+
+### 5. Update Custom Code (if applicable)
+
+If you have custom code using the plugin's APIs:
+
+**Service Imports:**
+```typescript
+// Before
+import { ContentExtractor } from '@easyling/sanity-connector/services'
+
+// After - import from shared library
+import { ContentExtractor } from '@easyling/sanity-connector-shared'
+```
+
+**Type Imports:**
+```typescript
+// Before
+import type { TranslationRequest } from '@easyling/sanity-connector/types'
+
+// After - still works, or use shared library
+import type { TranslationRequest } from '@easyling/sanity-connector'
+// or
+import type { TranslationRequest } from '@easyling/sanity-connector-shared'
+```
+
+### Breaking Changes
+
+The monorepo restructuring introduces minimal breaking changes:
+
+1. **Internal file structure**: If you were importing from internal paths (not recommended), update to use public exports
+2. **Service initialization**: Some services now require explicit client initialization
+3. **Configuration storage**: Legacy config documents are deprecated (but still supported)
+
+### Rollback
+
+If you need to rollback to the previous version:
+
+```bash
+npm install @easyling/sanity-connector@1.3.0
+```
+
+Your configuration will remain intact as the document structure hasn't changed.
+
+## Advanced Usage
+
+### Custom Translation Workflow
+
+You can build custom translation workflows using the exported services:
+
+```typescript
+import {
+  ContentExtractor,
+  TranslationService,
+  DocumentCreationService,
+  UnifiedConfigStorage
+} from '@easyling/sanity-connector-shared'
+import { useClient } from 'sanity'
+
+function MyCustomTranslationComponent() {
+  const client = useClient()
+  
+  const handleCustomTranslation = async (document) => {
+    // Initialize services
+    const configStorage = new UnifiedConfigStorage(client)
+    const extractor = new ContentExtractor(client)
+    const translationService = new TranslationService(client)
+    const docCreationService = new DocumentCreationService(client)
+    
+    // Load configuration
+    const config = await configStorage.load()
+    translationService.setEndpoint(config.translationApiEndpoint)
+    translationService.setAuthToken(config.accessToken, config.projectId)
+    
+    // Extract content
+    const structured = await extractor.extractStructuredContent(
+      document,
+      configStorage
+    )
+    
+    // Translate
+    const response = await translationService.translateDocument(
+      structured,
+      document._type,
+      'en',
+      undefined,
+      ['ja', 'de']
+    )
+    
+    // Create translated documents
+    for (const translatedDoc of response.translatedDocuments) {
+      await docCreationService.createDocumentFromTranslation(
+        document,
+        { ...response, translatedDocuments: [translatedDoc] },
+        {
+          targetLanguage: translatedDoc.locale,
+          creationMode: 'draft'
+        }
+      )
+    }
+  }
+  
+  return <button onClick={() => handleCustomTranslation(doc)}>Translate</button>
+}
+```
+
+### Custom DNT Logic
+
+Implement custom DNT logic for your schema:
+
+```typescript
+import { DNTServiceManager } from '@easyling/sanity-connector'
+import { useClient } from 'sanity'
+
+function MyComponent() {
+  const client = useClient()
+  const dntService = DNTServiceManager.getInstance(client)
+  
+  const checkDNTStatus = async (documentType, fieldPath) => {
+    const isDNT = await dntService.getDNTFieldStatus(documentType, fieldPath)
+    console.log(`Field ${fieldPath} is ${isDNT ? 'NOT' : ''} translatable`)
+  }
+  
+  const setDNTStatus = async (documentType, fieldPath, isDNT) => {
+    await dntService.setDNTField(documentType, fieldPath, isDNT)
+  }
+  
+  return <div>Custom DNT UI</div>
+}
+```
+
+### Accessing Configuration
+
+Access plugin configuration in your custom components:
+
+```typescript
+import { UnifiedConfigStorage } from '@easyling/sanity-connector'
+import { useClient } from 'sanity'
+import { useEffect, useState } from 'react'
+
+function MyConfigComponent() {
+  const client = useClient()
+  const [config, setConfig] = useState(null)
+  
+  useEffect(() => {
+    const loadConfig = async () => {
+      const configStorage = new UnifiedConfigStorage(client)
+      const cfg = await configStorage.load()
+      setConfig(cfg)
+    }
+    loadConfig()
+  }, [client])
+  
+  return (
+    <div>
+      <h2>Current Configuration</h2>
+      <pre>{JSON.stringify(config, null, 2)}</pre>
+    </div>
+  )
+}
+```
 
 ## Troubleshooting
 
-**Plugin not appearing in Studio**
--   Verify `npm install sanity-translation-plugin` completed successfully.
--   Ensure `translationPlugin()` is in the `plugins` array in `sanity.config.ts`.
--   Restart the Sanity Studio development server.
+### Translation Fails with Authentication Error
+
+**Problem**: Translation requests fail with 401 or 403 errors
+
+**Solution**:
+1. Verify OAuth credentials in `EL_PluginConfiguration`
+2. Check that Project ID and Access Token are correct
+3. Ensure access token hasn't expired
+4. Contact your translation service provider for new credentials
+
+### DNT Badges Not Showing
 
-**Translation action not visible**
--   The action only appears on documents that have translatable fields defined in their schema.
+**Problem**: DNT badges don't appear on fields
 
-**Authentication Issues**
--   If using OAuth, check the **OAuth Configuration** tool to ensure your tokens are valid.
--   If using local testing, ensure your mock translation service is running on the expected port.
+**Solution**:
+1. Check that the field type is translatable (string, text, array)
+2. Verify the field isn't blacklisted (locale, _id, _type, etc.)
+3. Enable debug mode to see all fields
+4. Check browser console for errors
+
+### Translated Documents Not Created
+
+**Problem**: Translation succeeds but no documents are created
+
+**Solution**:
+1. Check collision resolution mode - may be set to 'skip'
+2. Verify you have write permissions in the dataset
+3. Check browser console for document creation errors
+4. Ensure target locale codes match your schema
+
+### Bulk Translation Stalls
+
+**Problem**: Bulk translation stops or hangs
+
+**Solution**:
+1. Check network connection
+2. Verify translation API is responding
+3. Try translating a single document to isolate the issue
+4. Check browser console for errors
+5. Reduce batch size if translating many documents
+
+### Configuration Not Persisting
+
+**Problem**: Configuration changes don't save
+
+**Solution**:
+1. Verify you have write permissions in the dataset
+2. Check that `EL_PluginConfiguration` document exists
+3. Look for validation errors in the Studio
+4. Check browser console for save errors
+
+### Migration Issues
+
+**Problem**: Legacy configuration not migrated
+
+**Solution**:
+1. Check that legacy documents exist (`EL_oauth-config`, `EL_locale-config`)
+2. Manually create `EL_PluginConfiguration` document
+3. Copy values from legacy documents
+4. Contact support if automatic migration fails
+
+## API Reference
+
+### Exported Components
+
+- `DNTFieldBadge`: Standalone DNT badge component
+- `withDNTBadge`: HOC for adding DNT badge to custom fields
+- `DNTFieldInput`: Complete field input with DNT badge
+
+### Exported Services
+
+- `UnifiedConfigStorage`: Configuration management
+- `DNTServiceManager`: DNT field management
+- `DNTStorageAdapter`: DNT storage interface
+- `LegacyDNTStorageAdapter`: Legacy DNT storage support
+
+### Exported Types
+
+```typescript
+import type {
+  LocaleDefinition,
+  LocaleConfig,
+  TranslationRequestPayload,
+  TranslationResponse,
+  TranslatableField,
+  DNTFieldPreference,
+  DNTPreferences,
+  DNTTypeConfig,
+  DNTStorage,
+  UnifiedPluginConfig
+} from '@easyling/sanity-connector'
+```
+
+See the [shared library documentation](../shared/README.md) for complete API reference.
+
+## Development
+
+### Building
+
+```bash
+npm run build        # Production build
+npm run dev          # Development build with watch
+npm run clean        # Remove build artifacts
+```
+
+### Testing
+
+```bash
+npm test             # Run all tests
+npm run test:watch   # Run tests in watch mode
+npm run test:coverage # Generate coverage report
+```
+
+### Code Quality
+
+```bash
+npm run lint         # Check for linting issues
+npm run lint:fix     # Auto-fix linting issues
+npm run typecheck    # Type check without emitting
+```
 
 ## Support
 
-Our support team can be reached at support@easyling.com.
+For questions, issues, or feature requests:
 
-**Note:** In order to investigate compatibility or interoperability issues, we may need access to your Sanity data lake and Studio codebase. We are happy to work with you to establish secure access for debugging purposes.
+- **Email**: support@easyling.com
+- **Issues**: [GitHub Issues](https://github.com/easyling/el-sanity-connector/issues)
+- **Documentation**: [Main README](../README.md) | [Shared Library](../shared/README.md)
 
 ## License
 
-MIT License - see [LICENSE](LICENSE) file for details.
+MIT
+
+## Related Packages
+
+- [@easyling/sanity-connector-shared](../shared/README.md) - Core translation logic
+- [@easyling/sanity-function-auto-translate](../function/README.md) - Automatic translation on publish
+
+## Changelog
+
+See [CHANGELOG.md](CHANGELOG.md) for version history and migration notes.