@contentstorage/core 1.2.0 → 2.1.0

package/README.md

@@ -1,31 +1,33 @@
 # @contentstorage/core
 
-> A key-value based CMS core library that fetches content from ContentStorage and generates TypeScript types
+> CLI tool for managing translations and generating TypeScript types from ContentStorage
 
 [![npm version](https://img.shields.io/npm/v/@contentstorage/core.svg)](https://www.npmjs.com/package/@contentstorage/core)
 [![License](https://img.shields.io/npm/l/@contentstorage/core.svg)](https://github.com/kaidohussar/contentstorage-core/blob/master/LICENSE)
 
+## Overview
+
+ContentStorage Core is a powerful CLI tool for managing translations and content. It pulls content from ContentStorage CDN, generates TypeScript types, and integrates seamlessly with popular i18n libraries.
+
 ## Features
 
-- **Key-Value Storage** - Organize content with hierarchical dot-notation paths
+- **Translation Management** - Pull content from ContentStorage CDN
+- **TypeScript Generation** - Automatic type generation from your content
+- **Translation Statistics** - Analyze translation completeness across languages
 - **Multi-Language Support** - Built-in support for 40+ languages
-- **TypeScript First** - Automatic type generation from your content
-- **Type Safety** - Full autocompletion and type checking for content access
-- **Live Editor Integration** - Real-time content editing without page reload
-- **Special Content Types** - Support for text, images, and variations
-- **Variable Substitution** - Dynamic content with template variables
-- **CLI Tools** - Easy content management with professional CLI
+- **CLI Tools** - Professional command-line interface
+- **Plugin Ecosystem** - Integrate with i18next, react-intl, vue-i18n, and more
 
 ## Installation
 
 ```bash
-npm install @contentstorage/core
+npm install -D @contentstorage/core
 ```
 
 or
 
 ```bash
-yarn add @contentstorage/core
+yarn add -D @contentstorage/core
 ```
 
 ## Quick Start
@@ -53,30 +55,23 @@ npx contentstorage pull
 npx contentstorage generate-types
 ```
 
-### 3. Initialize and Use in Your App
-
-```typescript
-import { initContentStorage, fetchContent, getText, getImage } from '@contentstorage/core';
-
-// Initialize
-initContentStorage({
-  contentKey: 'your-content-key',
-  languageCodes: ['EN', 'FR', 'DE']
-});
-
-// Fetch content for a language
-await fetchContent('EN');
+### 3. Use with Your i18n Library
 
-// Access content with full type safety
-const title = getText('HomePage.title');
-// → { contentId: 'HomePage.title', text: 'Welcome!' }
+**With i18next:**
+```bash
+npm install @contentstorage/plugin-i18next
+npx contentstorage-i18next export
+```
 
-const heroImage = getImage('HomePage.hero');
-// → { contentId: 'HomePage.hero', data: { url: '...', altText: '...' } }
+**With react-intl:**
+```bash
+npm install @contentstorage/plugin-react-intl
+npx contentstorage-react-intl export
+```
 
-// Use variables in text
-const greeting = getText('HomePage.greeting', { name: 'John' });
-// → { contentId: 'HomePage.greeting', text: 'Hello John!' }
+**With ContentStorage SDK (for advanced features like variations and images):**
+```bash
+npm install @contentstorage/sdk
 ```
 
 ## CLI Commands
@@ -122,6 +117,39 @@ npx contentstorage generate-types --output src/types.ts
 npx contentstorage generate-types --content-key abc123 --lang EN
 ```
 
+### `contentstorage stats`
+
+Show translation completeness statistics across all configured languages
+
+```bash
+npx contentstorage stats [options]
+```
+
+**Options:**
+- `--content-key <key>` - Content key for your project
+- `--content-dir <dir>` - Directory with content files
+- `--pending-changes` - Analyze pending/draft content
+
+**Examples:**
+```bash
+npx contentstorage stats
+npx contentstorage stats --content-key abc123
+npx contentstorage stats --pending-changes
+```
+
+**What it shows:**
+- Total number of content items per language
+- Number of translated vs untranslated items
+- Completion percentage for each language
+- Detailed list of untranslated item IDs grouped by language
+- Overall translation completion across all languages
+
+**What counts as "untranslated":**
+- Empty strings (`""`)
+- Keys that exist in the reference language but are missing in target languages
+
+The stats command uses the first language in your `languageCodes` array as the reference/baseline for comparison.
+
 ### `contentstorage --help`
 
 Show all available commands and options
@@ -130,6 +158,7 @@ Show all available commands and options
 npx contentstorage --help
 npx contentstorage pull --help
 npx contentstorage generate-types --help
+npx contentstorage stats --help
 ```
 
 ## Configuration
@@ -157,106 +186,47 @@ export default {
 
 ### Supported Languages
 
-The library supports 40+ languages including:
+The CLI supports 40+ languages including:
 EN, FR, DE, ES, IT, PT, NL, PL, RU, TR, SV, NO, DA, FI, CS, SK, HU, RO, BG, HR, SL, SR, and more.
 
-## API Reference
-
-### Initialization
+## Integration Options
 
-#### `initContentStorage(config)`
+### Option 1: Use with i18n Libraries (Recommended for most projects)
 
-Initialize the content storage system.
+For standard i18n needs, use ContentStorage CLI with popular i18n libraries:
 
-```typescript
-initContentStorage({
-  contentKey: string,
-  languageCodes: LanguageCode[]
-});
-```
+- **[@contentstorage/plugin-i18next](https://www.npmjs.com/package/@contentstorage/plugin-i18next)** - i18next integration
+- **[@contentstorage/plugin-react-intl](https://www.npmjs.com/package/@contentstorage/plugin-react-intl)** - react-intl (FormatJS) integration
+- **[@contentstorage/plugin-vue-i18n](https://www.npmjs.com/package/@contentstorage/plugin-vue-i18n)** - Vue i18n integration
 
-#### `setContentLanguage(config)`
+### Option 2: Use with ContentStorage SDK (Advanced features)
 
-Set content for a specific language.
+If you need advanced features like variations (A/B testing) and image management:
 
-```typescript
-setContentLanguage({
-  languageCode: LanguageCode,
-  contentJson: object
-});
-```
-
-### Content Retrieval
-
-#### `getText<Path>(contentId, variables?)`
-
-Get localized text with optional variable substitution.
-
-```typescript
-const result = getText('HomePage.title');
-// → { contentId: 'HomePage.title', text: 'Welcome!' }
-
-const greeting = getText('HomePage.greeting', { name: 'John', count: 5 });
-// → { contentId: 'HomePage.greeting', text: 'Hello John, you have 5 items' }
-```
-
-**Returns:** `{ contentId: string, text: string }`
-
-#### `getImage(contentId)`
-
-Get image content with CDN URL.
-
-```typescript
-const image = getImage('HomePage.hero');
-// → {
-//     contentId: 'HomePage.hero',
-//     data: {
-//       contentstorage_type: 'image',
-//       url: 'https://cdn.contentstorage.app/...',
-//       altText: 'Hero image'
-//     }
-//   }
-```
-
-**Returns:** `{ contentId: string, data: ImageObject } | undefined`
-
-#### `getVariation(contentId, variationKey?, variables?)`
-
-Get content variation for A/B testing.
-
-```typescript
-const cta = getVariation('HomePage.cta', 'mobile');
-// → { contentId: 'HomePage.cta', text: 'Tap Now' }
-
-// Defaults to 'default' variation if not specified
-const ctaDefault = getVariation('HomePage.cta');
+```bash
+npm install @contentstorage/sdk
 ```
 
-**Returns:** `{ contentId: string, text: string }`
+See [@contentstorage/sdk](https://www.npmjs.com/package/@contentstorage/sdk) for documentation on:
+- Content variations (A/B testing)
+- Image management with CDN URLs
+- Live editor integration
+- Runtime content fetching
 
-### Content Fetching
+## TypeScript Support
 
-#### `fetchContent(languageCode)`
-
-Fetch content from ContentStorage CDN.
+After running `generate-types`, you get full TypeScript support:
 
 ```typescript
-await fetchContent('EN');
-```
+// Generated types augment the ContentStructure interface
+import type { ContentStructure } from '@contentstorage/core';
 
-## TypeScript Integration
-
-The library uses interface augmentation for type-safe content access:
-
-```typescript
-// After running: npx contentstorage generate-types
-// The generated types augment the ContentStructure interface
-
-import { getText } from '@contentstorage/core';
+// Use with your i18n library
+import i18next from 'i18next';
 
 // TypeScript knows all available content paths
-const title = getText('HomePage.title'); // ✅ Autocomplete works
-const invalid = getText('Invalid.path'); // ❌ TypeScript error
+i18next.t('HomePage.title'); // ✅ Autocomplete works
+i18next.t('Invalid.path');   // ❌ TypeScript error
 ```
 
 ## Content Structure
@@ -268,28 +238,20 @@ Content is organized in a hierarchical key-value structure:
   "HomePage": {
     "title": "Welcome to Our App",
     "greeting": "Hello {name}, you have {count} items",
-    "hero": {
-      "contentstorage_type": "image",
-      "url": "hero.jpg",
-      "altText": "Hero image"
-    },
-    "cta": {
-      "contentstorage_type": "variation",
-      "data": {
-        "default": "Click Here",
-        "mobile": "Tap Now",
-        "desktop": "Click to Continue"
-      }
-    }
+    "description": "Get started with our platform"
+  },
+  "Navigation": {
+    "home": "Home",
+    "about": "About",
+    "contact": "Contact"
   }
 }
 ```
 
 **Access with dot notation:**
-- `getText('HomePage.title')` → "Welcome to Our App"
-- `getText('HomePage.greeting', { name: 'John', count: 5 })` → "Hello John, you have 5 items"
-- `getImage('HomePage.hero')` → Image object with CDN URL
-- `getVariation('HomePage.cta', 'mobile')` → "Tap Now"
+- `HomePage.title` → "Welcome to Our App"
+- `HomePage.greeting` → "Hello {name}, you have {count} items"
+- `Navigation.home` → "Home"
 
 ## Package.json Scripts
 
@@ -314,16 +276,43 @@ npm run content:types   # Generate types
 npm run content:sync    # Pull and generate in one command
 ```
 
-## Integration with React
+## Workflow Example
+
+1. **Pull latest content:**
+   ```bash
+   npx contentstorage pull
+   ```
+
+2. **Generate TypeScript types:**
+   ```bash
+   npx contentstorage generate-types
+   ```
+
+3. **Use in your app with i18next:**
+   ```typescript
+   import i18next from 'i18next';
+   import enContent from './content/json/EN.json';
+
+   i18next.init({
+     lng: 'EN',
+     resources: {
+       EN: { translation: enContent }
+     }
+   });
+
+   // Use translations
+   const title = i18next.t('HomePage.title');
+   ```
 
-For React integration, use [@contentstorage/react](https://www.npmjs.com/package/@contentstorage/react) which builds on top of this core library and provides React-specific components and hooks.
+## SDK Extract
 
-## Live Editor Mode
+The `/sdk-extract` folder contains the ContentStorage SDK code ready to be moved to a separate repository. This SDK provides runtime features like:
+- getText/getImage/getVariation functions
+- Content variations (A/B testing)
+- Image management
+- Live editor integration
 
-When running in an iframe with live editor parameters, the library automatically:
-- Loads the live editor script from CDN
-- Tracks content usage via `window.memoryMap`
-- Enables real-time content updates without page reload
+To use the SDK, it will be published as `@contentstorage/sdk` in a separate package.
 
 ## Requirements
 
@@ -346,4 +335,4 @@ Kaido Hussar - [kaidohus@gmail.com](mailto:kaidohus@gmail.com)
 
 ## Support
 
-For issues and questions, please [open an issue](https://github.com/kaidohussar/contentstorage-core/issues) on GitHub.
+For issues and questions, please [open an issue](https://github.com/kaidohussar/contentstorage-core/issues) on GitHub.

package/dist/commands/cli.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export {};

package/dist/commands/cli.js

@@ -0,0 +1,110 @@
+#!/usr/bin/env node
+import chalk from 'chalk';
+import { pullContent } from './pull.js';
+import { generateTypes } from './generate-types.js';
+import { showStats } from './stats.js';
+const COMMANDS = {
+    pull: {
+        name: 'pull',
+        description: 'Pull content from Contentstorage CDN',
+        usage: 'contentstorage pull [options]',
+        options: [
+            '  --content-key <key>    Content key for your project',
+            '  --content-dir <dir>    Directory to save content files',
+            '  --lang <code>          Language code (e.g., EN, FR)',
+            '  --pending-changes      Fetch pending/draft content',
+        ],
+    },
+    'generate-types': {
+        name: 'generate-types',
+        description: 'Generate TypeScript type definitions from content',
+        usage: 'contentstorage generate-types [options]',
+        options: [
+            '  --output <file>        Output file for generated types',
+            '  --content-key <key>    Content key for your project',
+            '  --lang <code>          Language code (e.g., EN, FR)',
+            '  --pending-changes      Use pending/draft content',
+        ],
+    },
+    stats: {
+        name: 'stats',
+        description: 'Show translation completeness statistics',
+        usage: 'contentstorage stats [options]',
+        options: [
+            '  --content-key <key>    Content key for your project',
+            '  --content-dir <dir>    Directory with content files',
+            '  --pending-changes      Analyze pending/draft content',
+        ],
+    },
+};
+function showHelp() {
+    console.log(chalk.bold('\nContentStorage CLI'));
+    console.log(chalk.dim('Manage content and generate TypeScript types\n'));
+    console.log(chalk.bold('Usage:'));
+    console.log('  contentstorage <command> [options]\n');
+    console.log(chalk.bold('Commands:'));
+    Object.values(COMMANDS).forEach((cmd) => {
+        console.log(`  ${chalk.cyan(cmd.name.padEnd(20))} ${cmd.description}`);
+    });
+    console.log(chalk.bold('\nOptions:'));
+    console.log('  --help                 Show help for a command\n');
+    console.log(chalk.dim('Examples:'));
+    console.log(chalk.dim('  contentstorage pull --content-key abc123'));
+    console.log(chalk.dim('  contentstorage generate-types --output types.ts'));
+    console.log(chalk.dim('  contentstorage pull --help    # Show help for pull command\n'));
+}
+function showCommandHelp(commandName) {
+    const cmd = COMMANDS[commandName];
+    if (!cmd) {
+        console.error(chalk.red(`Unknown command: ${commandName}`));
+        process.exit(1);
+    }
+    console.log(chalk.bold(`\n${cmd.name}`));
+    console.log(chalk.dim(cmd.description + '\n'));
+    console.log(chalk.bold('Usage:'));
+    console.log(`  ${cmd.usage}\n`);
+    if (cmd.options.length > 0) {
+        console.log(chalk.bold('Options:'));
+        cmd.options.forEach((opt) => console.log(opt));
+        console.log('');
+    }
+}
+async function main() {
+    const args = process.argv.slice(2);
+    // No arguments - show help
+    if (args.length === 0) {
+        showHelp();
+        process.exit(0);
+    }
+    const command = args[0];
+    // Global --help flag
+    if (command === '--help' || command === '-h') {
+        showHelp();
+        process.exit(0);
+    }
+    // Command-specific --help
+    if (args.includes('--help') || args.includes('-h')) {
+        showCommandHelp(command);
+        process.exit(0);
+    }
+    // Route to commands
+    switch (command) {
+        case 'pull':
+            await pullContent();
+            break;
+        case 'generate-types':
+            await generateTypes();
+            break;
+        case 'stats':
+            await showStats();
+            break;
+        default:
+            console.error(chalk.red(`Unknown command: ${command}\n`));
+            console.log(chalk.dim('Run "contentstorage --help" for usage'));
+            process.exit(1);
+    }
+}
+main().catch((error) => {
+    console.error(chalk.red('Unexpected error:'), error);
+    process.exit(1);
+});

package/dist/commands/generate-types.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export declare function generateTypes(): Promise<void>;

package/dist/commands/generate-types.js

@@ -0,0 +1,177 @@
+#!/usr/bin/env node
+// ^ Ensures the script is executed with Node.js
+import fs from 'fs/promises';
+import path from 'path';
+import axios from 'axios';
+import chalk from 'chalk'; // Optional: for colored output
+import { loadConfig } from '../core/config-loader.js';
+import { flattenJson } from '../utils/flatten-json.js';
+import { CONTENTSTORAGE_CONFIG } from '../utils/constants.js';
+import { jsonToTS } from '../type-generation/index.js';
+export async function generateTypes() {
+    console.log(chalk.blue('Starting type generation...'));
+    const args = process.argv.slice(2);
+    const cliConfig = {};
+    for (let i = 0; i < args.length; i++) {
+        const arg = args[i];
+        if (arg.startsWith('--')) {
+            const key = arg.substring(2);
+            const value = args[i + 1];
+            if (key === 'pending-changes') {
+                cliConfig.pendingChanges = true;
+            }
+            else if (value && !value.startsWith('--')) {
+                if (key === 'lang') {
+                    cliConfig.languageCodes = [value.toUpperCase()];
+                }
+                else if (key === 'content-key') {
+                    cliConfig.contentKey = value;
+                }
+                else if (key === 'output') {
+                    cliConfig.typesOutputFile = value;
+                }
+                // Skip the value in the next iteration
+                i++;
+            }
+        }
+    }
+    let fileConfig = {};
+    try {
+        fileConfig = await loadConfig();
+    }
+    catch {
+        console.log(chalk.yellow('Could not load a configuration file. Proceeding with CLI arguments.'));
+    }
+    const config = { ...fileConfig, ...cliConfig };
+    if (!config.typesOutputFile) {
+        console.error(chalk.red.bold("Configuration error: 'typesOutputFile' is missing."));
+        process.exit(1);
+    }
+    if (!config.languageCodes ||
+        !Array.isArray(config.languageCodes) ||
+        config.languageCodes.length === 0) {
+        console.error(chalk.red.bold("Configuration error: 'languageCodes' must be a non-empty array."));
+        process.exit(1);
+    }
+    console.log(chalk.blue(`TypeScript types will be saved to: ${config.typesOutputFile}`));
+    let jsonObject; // To hold the JSON data from either local or remote source
+    let dataSourceDescription = ''; // For clearer logging
+    const firstLanguageCode = config.languageCodes[0];
+    try {
+        let attemptLocalLoad = false;
+        if (config.contentDir) {
+            try {
+                await fs.stat(config.contentDir); // Check if directory exists
+                attemptLocalLoad = true;
+                console.log(chalk.blue(`Local content directory found: ${config.contentDir}`));
+            }
+            catch (statError) {
+                if (statError.code === 'ENOENT') {
+                    console.log(chalk.yellow(`Local content directory specified but not found: ${config.contentDir}. Will attempt to fetch from URL.`));
+                    // attemptLocalLoad remains false
+                }
+                else {
+                    // Other errors accessing contentDir (e.g., permissions)
+                    throw new Error(`Error accessing content directory ${config.contentDir}: ${statError.message}`);
+                }
+            }
+        }
+        else {
+            console.log(chalk.yellow(`Local content directory (config.contentDir) not specified. Attempting to fetch from URL.`));
+        }
+        if (attemptLocalLoad && config.contentDir) {
+            // --- Load from local file system ---
+            const targetFilename = `${firstLanguageCode}.json`;
+            const jsonFilePath = path.join(config.contentDir, targetFilename);
+            dataSourceDescription = `local file (${jsonFilePath})`;
+            console.log(chalk.blue(`Attempting to read JSON from: ${jsonFilePath}`));
+            try {
+                const jsonContentString = await fs.readFile(jsonFilePath, 'utf-8');
+                console.log(chalk.blue('Parsing JSON'));
+                const parsendJsonObject = JSON.parse(jsonContentString);
+                console.log(chalk.blue('Flattening JSON for type generation'));
+                jsonObject = flattenJson(parsendJsonObject);
+                console.log(chalk.green(`Successfully read and parsed JSON from ${jsonFilePath}.`));
+            }
+            catch (fileError) {
+                if (fileError.code === 'ENOENT') {
+                    throw new Error(`Target JSON file not found at ${jsonFilePath}. ` +
+                        `Ensure content for language code '${firstLanguageCode}' has been pulled and exists locally, ` +
+                        `or ensure 'contentDir' is not set if you intend to fetch from URL.`);
+                }
+                throw new Error(`Failed to read or parse JSON from ${jsonFilePath}: ${fileError.message}`);
+            }
+        }
+        else {
+            if (!config.contentKey) {
+                throw new Error('Cannot generate types: contentKey is missing');
+            }
+            let fileUrl;
+            const requestConfig = { responseType: 'json' };
+            if (config.pendingChanges) {
+                fileUrl = `${CONTENTSTORAGE_CONFIG.API_URL}/pending-changes/get-json?languageCode=${firstLanguageCode}`;
+                requestConfig.headers = {
+                    'X-Content-Key': config.contentKey,
+                };
+            }
+            else {
+                fileUrl = `${CONTENTSTORAGE_CONFIG.BASE_URL}/${config.contentKey}/content/${firstLanguageCode}.json`;
+            }
+            dataSourceDescription = `remote URL (${fileUrl})`;
+            console.log(chalk.blue(`Attempting to fetch JSON from: ${fileUrl}`));
+            try {
+                const response = await axios.get(fileUrl, requestConfig);
+                let jsonResponse = response.data;
+                // Handle API response structure - API returns { data: actualContent }
+                if (config.pendingChanges &&
+                    jsonResponse &&
+                    typeof jsonResponse === 'object' &&
+                    'data' in jsonResponse) {
+                    jsonResponse = jsonResponse.data;
+                }
+                console.log(chalk.blue('Flattening JSON for type generation'));
+                jsonObject = flattenJson(jsonResponse);
+                if (typeof jsonObject !== 'object' || jsonObject === null) {
+                    throw new Error(`Workspaceed data from ${fileUrl} is not a valid JSON object. Received type: ${typeof jsonObject}`);
+                }
+                console.log(chalk.green(`Successfully fetched and parsed JSON from ${fileUrl}. This content will not be saved locally.`));
+            }
+            catch (fetchError) {
+                let errorDetail = fetchError.message;
+                if (axios.isAxiosError(fetchError)) {
+                    errorDetail = `Status: ${fetchError.response?.status}, Response: ${JSON.stringify(fetchError.response?.data)}`;
+                }
+                throw new Error(`Failed to fetch JSON from ${fileUrl}: ${errorDetail}`);
+            }
+        }
+        // Validate the obtained jsonObject (must be an object or array for json-to-ts)
+        if (typeof jsonObject !== 'object' || jsonObject === null) {
+            // jsonToTS can handle root arrays too, but if it's primitive it's an issue.
+            // Allowing arrays here explicitly based on jsonToTS capability.
+            if (!Array.isArray(jsonObject)) {
+                throw new Error(`The content obtained from ${dataSourceDescription} is not a JSON object or array (type: ${typeof jsonObject}). Cannot generate types.`);
+            }
+        }
+        // Generate TypeScript interfaces using json-to-ts
+        const rootTypeName = 'ContentRoot';
+        console.log(chalk.blue(`Generating TypeScript types with root name '${rootTypeName}'...`));
+        const typeDeclarations = jsonToTS(jsonObject, {
+            rootName: rootTypeName,
+        });
+        if (!typeDeclarations || typeDeclarations.length === 0) {
+            throw new Error(`Could not generate types from the content of ${dataSourceDescription}. 'json-to-ts' returned no declarations.`);
+        }
+        const outputContent = typeDeclarations.join('\n\n');
+        // Ensure the output directory exists for the types file
+        const outputDir = path.dirname(config.typesOutputFile);
+        await fs.mkdir(outputDir, { recursive: true });
+        // Write the generated types to the output file
+        await fs.writeFile(config.typesOutputFile, outputContent, 'utf-8');
+        console.log(chalk.green(`TypeScript types generated successfully at ${config.typesOutputFile} using data from ${dataSourceDescription}.`));
+    }
+    catch (error) {
+        console.error(chalk.red.bold('\nError generating TypeScript types:'));
+        console.error(chalk.red(error.message));
+        process.exit(1);
+    }
+}

package/dist/commands/pull.d.ts

@@ -0,0 +1,2 @@
+#!/usr/bin/env node
+export declare function pullContent(): Promise<void>;