toon-formatter 2.0.1 → 2.2.0

package/CHANGELOG.md

@@ -5,6 +5,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [2.1.1] - 2025-12-18
+
+### Added
+- **🔄 Unified Format Converters**
+  - New classes: `JsonConverter`, `YamlConverter`, `XmlConverter`, and `CsvConverter`
+  - Direct translation between formats (e.g., XML ↔ JSON, CSV ↔ YAML)
+  - Full encryption middleware support for all unified converters
+  - Static and Instance API parity across all converters
+- **📝 Enhanced Mixed-Text Support**
+  - Standardized mixed-text extraction for JSON, XML, and CSV
+  - Preserves surrounding text while converting embedded data blocks
+- **📦 Improved Packaging**
+  - Specific exports for all converters in `package.json`
+  - Fixed Node.js XML support by moving `xmldom` to production dependencies
+- **📚 Documentation Hub**
+  - Comprehensive README updates with Unified Converter examples
+  - New `toon-formatter.json` documentation for the web platform
+
+### Changed
+- Standardized `returnJson` default to `false` (returns objects/arrays) for consistency
+- Refined internal conversion logic to better handle edge cases in mixed-text scenarios
+- Improved package keywords and description for better SEO
+
 ## [2.0.0] - 2025-12-08
 
 ### Added
@@ -65,6 +88,21 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Migration Guide
 
+### From v2.0 to v2.1
+
+**To use Unified Converters:**
+
+```javascript
+import { JsonConverter, XmlConverter } from 'toon-formatter';
+
+// Direct format-to-format
+const xml = JsonConverter.toXml({ name: 'Alice' });
+
+// With encryption
+const converter = new JsonConverter(encryptor);
+const encryptedXml = converter.toXml({ name: 'Alice' }, { conversionMode: 'export' });
+```
+
 ### From v1.x to v2.0
 
 **No changes required!** Version 2.0 is fully backward compatible.
@@ -73,7 +111,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ```javascript
 // Old code (still works)
-import { ToonConverter } from 'toon-formatter';
+import ToonConverter from 'toon-formatter';
 const toon = ToonConverter.fromJson(data);
 
 // New code (with encryption)

package/README.md

@@ -1,5 +1,9 @@
 # 🚀 TOON Converter
 
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Node >=16.0.0](https://img.shields.io/badge/Node-%3E%3D16.0.0-blue.svg)](https://nodejs.org/en/download)
+[![LLM APIs cost reduction](https://img.shields.io/badge/LLM%20APIs-Up%20to%2040%25%20cost%20reduction-orange)](https://toonformatter.net/)
+
 A lightweight library to convert between **TOON** (Token-Oriented Object Notation) and popular data formats (JSON, YAML, XML, CSV).
 
 **Reduce your LLM token costs by up to 40%** using the TOON format!
@@ -55,6 +59,64 @@ users[3]{id,name,active}:
 
 ---
 
+## 💻 CLI Utility
+
+**NEW in v2.1.0**: TOON Formatter now includes a powerful CLI utility for fast data conversion directly from your terminal!
+
+### Global Installation
+To use the `toon-formatter` command anywhere:
+```bash
+npm install -g toon-formatter
+```
+Or run it instantly without installation using `npx`:
+```bash
+npx toon-formatter --help
+```
+
+### Basic Usage
+Convert files or piped data easily:
+```bash
+# Convert JSON file to TOON
+toon-formatter --from json --to toon -i data.json -o data.toon
+
+# Piping data (JSON -> TOON)
+echo '{"name": "Alice"}' | toon-formatter --from json --to toon
+
+# Convert XML to JSON (prettified by default)
+cat profile.xml | toon-formatter --from xml --to json
+```
+
+### Advanced Features
+The CLI supports all library features, including validation and encryption:
+
+```bash
+# Validate a TOON string
+cat data.toon | toon-formatter --validate toon
+
+# Encrypt data during conversion (XOR)
+echo '{"secret": "data"}' | toon-formatter --from json --to toon --mode export --key "mykey" --algo xor
+
+# Decrypt data (AES-256-GCM)
+cat encrypted.txt | toon-formatter --from toon --to json --mode ingestion --key "your-32-byte-key" --algo aes-256-gcm
+```
+
+### CLI Options
+| Flag | Short | Description |
+|------|-------|-------------|
+| `--from` | `-f` | Input format (json, yaml, xml, csv, toon) |
+| `--to` | `-t` | Output format (json, yaml, xml, csv, toon) |
+| `--input` | `-i` | Input file path (defaults to stdin) |
+| `--output` | `-o` | Output file path (defaults to stdout) |
+| `--validate` | `-v` | Validate the input format and exit |
+| `--mode` | `-m` | Encryption mode (middleware, ingestion, export) |
+| `--key` | `-k` | Encryption key |
+| `--algo` | `-a` | Encryption algorithm (aes-256-gcm, xor, base64) |
+| `--async` | | Use asynchronous conversion mode |
+| `--no-parse` | | Skip parsing of objects (returns raw strings) |
+| `--help` | `-h` | Show help information |
+
+---
+
 ## 🚀 Quick Start
 
 ### Basic Usage (Synchronous)
@@ -1086,6 +1148,152 @@ ToonConverter.toJsonAsync(toonString, returnJson?)
 
 ---
 
+### Unified Format Converters (v2.0.0+)
+
+The library now includes specialized, unified converters for each major format. These are perfect when you need to convert between non-TOON formats (like XML to JSON or CSV to YAML) while still having access to TOON and encryption features.
+
+#### Available Unified Converters:
+- `JsonConverter`: Specialized in JSON input/output
+- `YamlConverter`: Specialized in YAML input/output
+- `XmlConverter`: Specialized in XML input/output
+- `CsvConverter`: Specialized in CSV input/output
+
+#### Example: Cross-Format Conversion
+```javascript
+import { XmlConverter, YamlConverter } from 'toon-formatter';
+
+// Convert XML directly to YAML
+const xmlData = '<user><name>Alice</name></user>';
+const yamlData = XmlConverter.toYaml(xmlData);
+
+// Convert YAML directly to CSV
+const csvData = YamlConverter.toCsv("name: Alice\nrole: admin");
+```
+
+---
+
+### JsonConverter Class
+
+Specialized for JSON-centric workflows. It can convert JSON to any format and any format back to JSON.
+
+#### Instance Methods (with Encryption)
+```javascript
+import { JsonConverter, Encryptor } from 'toon-formatter';
+
+const converter = new JsonConverter(new Encryptor(key, 'aes-256-gcm'));
+
+// JSON -> TOON (Encrypted)
+const encryptedToon = converter.toToon(data, { conversionMode: 'export' });
+
+// XML -> JSON (Decrypted)
+const jsonData = converter.fromXml(encryptedXml, { conversionMode: 'ingestion' });
+```
+
+#### Static Methods (No Encryption)
+```javascript
+// Convert TOON to JSON object
+const obj = JsonConverter.fromToon(toonString);
+
+// Convert TOON to JSON string
+const json = JsonConverter.fromToon(toonString, { returnJson: true });
+
+// Convert JSON to XML
+const xml = JsonConverter.toXml({ name: "Alice" });
+```
+
+**Available Methods:**
+- `fromToon(toonString, options?)` / `fromToonAsync(toonString, options?)`
+- `toToon(data, options?)` / `toToonAsync(data, options?)`
+- `fromYaml(yamlString, options?)` / `fromYamlAsync(yamlString, options?)`
+- `toYaml(data, options?)` / `toYamlAsync(data, options?)`
+- `fromXml(xmlString, options?)` / `fromXmlAsync(xmlString, options?)`
+- `toXml(data, options?)` / `toXmlAsync(data, options?)`
+- `fromCsv(csvString, options?)` / `fromCsvAsync(csvString, options?)`
+- `toCsv(data, options?)` / `toCsvAsync(data, options?)`
+
+---
+
+### YamlConverter Class
+
+Specialized for YAML workflows.
+
+#### Usage Example
+```javascript
+import { YamlConverter } from 'toon-formatter';
+
+// YAML to TOON
+const toon = YamlConverter.fromToon(testToon);
+
+// YAML to JSON
+const json = YamlConverter.toJson(yamlString, { returnJson: true });
+```
+
+**Available Methods:**
+- `fromToon(toonString, options?)` / `fromToonAsync(toonString, options?)`
+- `toToon(yamlString, options?)` / `toToonAsync(yamlString, options?)`
+- `fromJson(jsonData, options?)` / `fromJsonAsync(jsonData, options?)`
+- `toJson(yamlString, options?)` / `toJsonAsync(yamlString, options?)`
+- `fromXml(xmlString, options?)` / `fromXmlAsync(xmlString, options?)`
+- `toXml(yamlString, options?)` / `toXmlAsync(yamlString, options?)`
+- `fromCsv(csvString, options?)` / `fromCsvAsync(csvString, options?)`
+- `toCsv(yamlString, options?)` / `toCsvAsync(yamlString, options?)`
+
+---
+
+### XmlConverter Class
+
+Specialized for XML workflows. Supports mixed-text extraction automatically.
+
+#### Usage Example
+```javascript
+import { XmlConverter } from 'toon-formatter';
+
+// XML to TOON
+const toon = XmlConverter.fromToon(xmlString);
+
+// JSON to XML
+const xml = XmlConverter.fromJson(jsonData);
+```
+
+**Available Methods:**
+- `fromToon(toonString, options?)` / `fromToonAsync(toonString, options?)`
+- `toToon(xmlString, options?)` / `toToonAsync(xmlString, options?)`
+- `fromJson(jsonData, options?)` / `fromJsonAsync(jsonData, options?)`
+- `toJson(xmlString, options?)` / `toJsonAsync(xmlString, options?)`
+- `fromYaml(yamlString, options?)` / `fromYamlAsync(yamlString, options?)`
+- `toYaml(xmlString, options?)` / `toYamlAsync(xmlString, options?)`
+- `fromCsv(csvString, options?)` / `fromCsvAsync(csvString, options?)`
+- `toCsv(xmlString, options?)` / `toCsvAsync(xmlString, options?)`
+
+---
+
+### CsvConverter Class
+
+Specialized for CSV workflows.
+
+#### Usage Example
+```javascript
+import { CsvConverter } from 'toon-formatter';
+
+// CSV to TOON
+const toon = CsvConverter.fromToon(csvString);
+
+// JSON to CSV
+const csv = CsvConverter.fromJson(jsonData);
+```
+
+**Available Methods:**
+- `fromToon(toonString, options?)` / `fromToonAsync(toonString, options?)`
+- `toToon(csvString, options?)` / `toToonAsync(csvString, options?)`
+- `fromJson(jsonData, options?)` / `fromJsonAsync(jsonData, options?)`
+- `toJson(csvString, options?)` / `toJsonAsync(csvString, options?)`
+- `fromYaml(yamlString, options?)` / `fromYamlAsync(yamlString, options?)`
+- `toYaml(csvString, options?)` / `toYamlAsync(csvString, options?)`
+- `fromXml(xmlString, options?)` / `fromXmlAsync(xmlString, options?)`
+- `toXml(csvString, options?)` / `toXmlAsync(csvString, options?)`
+
+---
+
 ## 🎨 TOON Format Guide
 
 ### Primitives
@@ -1234,7 +1442,10 @@ fs.writeFileSync('config.yaml', improvedYaml);
 
 ## 🧪 Testing
 
+The library includes a comprehensive test suite with 150+ unit and integration tests.
+
 ```bash
+# Run all tests (Unit, Integration, and CLI)
 npm test
 ```
 

package/bin/toon-formatter.js

@@ -0,0 +1,3 @@
+#!/usr/bin/env node
+
+import '../src/cli.js';

package/package.json

@@ -1,13 +1,16 @@
 {
     "name": "toon-formatter",
-    "version": "2.0.1",
+    "version": "2.2.0",
     "funding": {
         "type": "github",
         "url": "https://github.com/sponsors/ankitpal181"
     },
-    "description": "A lightweight library to convert between TOON (Token-Oriented Object Notation) and popular data formats (JSON, YAML, XML, CSV) with end-to-end encryption support. Reduce LLM token costs by up to 40%.",
+    "description": "A powerful library to convert between TOON, JSON, YAML, XML, and CSV with end-to-end encryption. Includes Unified Converters for direct format-to-format translation and LLM token optimization.",
     "main": "src/index.js",
     "type": "module",
+    "bin": {
+        "toon-formatter": "./bin/toon-formatter.js"
+    },
     "scripts": {
         "test": "node --test test/*.test.js",
         "test:watch": "node --test --watch test/*.test.js"
@@ -30,7 +33,11 @@
         "security",
         "crypto",
         "secure-data",
-        "end-to-end-encryption"
+        "end-to-end-encryption",
+        "unified-converter",
+        "json-to-xml",
+        "csv-to-yaml",
+        "xml-to-json"
     ],
     "author": "Ankit Pal",
     "license": "MIT",
@@ -47,11 +54,10 @@
     },
     "dependencies": {
         "js-yaml": "^4.1.0",
-        "papaparse": "^5.4.1"
-    },
-    "devDependencies": {
+        "papaparse": "^5.4.1",
         "xmldom": "^0.6.0"
     },
+    "devDependencies": {},
     "exports": {
         ".": "./src/index.js",
         "./json": "./src/json.js",
@@ -60,6 +66,10 @@
         "./csv": "./src/csv.js",
         "./validator": "./src/validator.js",
         "./utils": "./src/utils.js",
-        "./encryptor": "./src/encryptor.js"
+        "./encryptor": "./src/encryptor.js",
+        "./json-converter": "./src/json_formatter/index.js",
+        "./yaml-converter": "./src/yaml_formatter/index.js",
+        "./xml-converter": "./src/xml_formatter/index.js",
+        "./csv-converter": "./src/csv_formatter/index.js"
     }
 }

package/src/cli.js

@@ -0,0 +1,271 @@
+#!/usr/bin/env node
+
+import fs from 'fs';
+import path from 'path';
+import {
+    ToonConverter,
+    JsonConverter,
+    YamlConverter,
+    XmlConverter,
+    CsvConverter,
+    Encryptor
+} from './index.js';
+
+// Polyfill DOMParser for Node.js environments (required for XML conversion)
+if (typeof DOMParser === 'undefined') {
+    try {
+        const { DOMParser: NodeDOMParser } = await import('xmldom');
+        global.DOMParser = NodeDOMParser;
+    } catch (e) {
+        // xmldom might not be available or other import error
+    }
+}
+
+async function readStdin() {
+    return new Promise((resolve, reject) => {
+        let data = '';
+        process.stdin.setEncoding('utf8');
+        process.stdin.on('readable', () => {
+            let chunk;
+            while ((chunk = process.stdin.read()) !== null) {
+                data += chunk;
+            }
+        });
+        process.stdin.on('end', () => resolve(data));
+        process.stdin.on('error', reject);
+    });
+}
+
+const showHelp = () => {
+    console.log(`
+TOON Formatter CLI - Convert between TOON, JSON, YAML, XML, and CSV.
+
+Usage:
+  toon-formatter --from <format> --to <format> [options]
+  toon-formatter --validate <format> [options]
+
+Options:
+  --from <format>       Source format (json, yaml, xml, csv, toon)
+  --to <format>         Target format (json, yaml, xml, csv, toon)
+  -i, --input <file>    Input file path (default: stdin)
+  -o, --output <file>   Output file path (default: stdout)
+  --async               Use asynchronous converters
+  -m, --mode <mode>     Conversion mode (no_encryption, middleware, ingestion, export)
+  -k, --key <key>       Encryption key
+  -a, --algo <algo>     Encryption algorithm (aes-256-gcm, xor, base64)
+  --no-parse            Return raw strings for applicable conversions
+  --validate <format>   Validate the given format
+  -h, --help            Show this help message
+
+Examples:
+  echo '{"name": "Alice"}' | toon-formatter --from json --to toon
+  toon-formatter --from xml --to json -i data.xml -o data.json
+  toon-formatter --validate toon -i data.toon
+`);
+};
+
+function parseArgs(args) {
+    const config = {
+        from: null,
+        to: null,
+        input: null,
+        output: null,
+        isAsync: false,
+        mode: 'no_encryption',
+        key: null,
+        algo: 'aes-256-gcm',
+        noParse: false,
+        validate: null
+    };
+
+    for (let i = 0; i < args.length; i++) {
+        const arg = args[i];
+        switch (arg) {
+            case '--from':
+                config.from = args[++i];
+                break;
+            case '--to':
+                config.to = args[++i];
+                break;
+            case '-i':
+            case '--input':
+                config.input = args[++i];
+                break;
+            case '-o':
+            case '--output':
+                config.output = args[++i];
+                break;
+            case '--async':
+                config.isAsync = true;
+                break;
+            case '-m':
+            case '--mode':
+                config.mode = args[++i];
+                break;
+            case '-k':
+            case '--key':
+                config.key = args[++i];
+                break;
+            case '-a':
+            case '--algo':
+                config.algo = args[++i];
+                break;
+            case '--no-parse':
+                config.noParse = true;
+                break;
+            case '--validate':
+                config.validate = args[++i];
+                break;
+            case '-h':
+            case '--help':
+                showHelp();
+                process.exit(0);
+            default:
+                if (arg.startsWith('-')) {
+                    console.error(`Unknown option: ${arg}`);
+                    process.exit(1);
+                }
+        }
+    }
+
+    return config;
+}
+
+async function run() {
+    const args = process.argv.slice(2);
+    if (args.length === 0) {
+        showHelp();
+        process.exit(0);
+    }
+
+    const config = parseArgs(args);
+
+    if (!config.validate && (!config.from || !config.to)) {
+        console.error("Error: --from and --to are required unless using --validate");
+        process.exit(1);
+    }
+
+    let inputData = '';
+    if (config.input) {
+        try {
+            inputData = fs.readFileSync(path.resolve(config.input), 'utf8');
+        } catch (err) {
+            console.error(`Error reading input file: ${err.message}`);
+            process.exit(1);
+        }
+    } else {
+        inputData = await readStdin();
+    }
+
+    if (!inputData && !config.validate) {
+        console.error("Error: No input data provided");
+        process.exit(1);
+    }
+
+    let encryptor = null;
+    if (config.key || config.algo === 'base64') {
+        try {
+            encryptor = new Encryptor(config.key, config.algo);
+        } catch (err) {
+            console.error(`Error initializing encryptor: ${err.message}`);
+            process.exit(1);
+        }
+    }
+
+    try {
+        let result;
+        if (config.validate) {
+            result = ToonConverter.validate(inputData);
+            if (config.output) {
+                fs.writeFileSync(path.resolve(config.output), JSON.stringify(result, null, 2));
+            } else {
+                console.log(JSON.stringify(result, null, 2));
+            }
+            return;
+        }
+
+        const options = {
+            conversionMode: config.mode,
+            returnJson: !config.noParse
+        };
+
+        result = await handleConversion(config, inputData, encryptor);
+
+        if (typeof result === 'object' && result !== null) {
+            result = JSON.stringify(result, null, 2);
+        } else if (typeof result === 'string' && config.to === 'json' && !config.noParse) {
+            try {
+                const parsed = JSON.parse(result);
+                result = JSON.stringify(parsed, null, 2);
+            } catch (e) {
+                // Not a valid JSON string, leave as is
+            }
+        }
+
+        if (config.output) {
+            fs.writeFileSync(path.resolve(config.output), result);
+        } else {
+            process.stdout.write(result + '\n');
+        }
+
+    } catch (err) {
+        console.error(`Conversion error: ${err.message}`);
+        process.exit(1);
+    }
+}
+
+async function handleConversion(config, data, encryptor) {
+    const { from, to, isAsync, mode, noParse } = config;
+
+    // Use instance if encryption is involved
+    const toonConv = new ToonConverter(encryptor);
+    const jsonConv = new JsonConverter(encryptor);
+    const yamlConv = new YamlConverter(encryptor);
+    const xmlConv = new XmlConverter(encryptor);
+    const csvConv = new CsvConverter(encryptor);
+
+    const options = { conversionMode: mode, returnJson: !noParse };
+
+    if (from === 'toon') {
+        if (to === 'json') return isAsync ? toonConv.toJsonAsync(data, options) : toonConv.toJson(data, options);
+        if (to === 'yaml') return isAsync ? toonConv.toYamlAsync(data, options) : toonConv.toYaml(data, options);
+        if (to === 'xml') return isAsync ? toonConv.toXmlAsync(data, options) : toonConv.toXml(data, options);
+        if (to === 'csv') return isAsync ? toonConv.toCsvAsync(data, options) : toonConv.toCsv(data, options);
+    }
+
+    if (to === 'toon') {
+        if (from === 'json') return isAsync ? toonConv.fromJsonAsync(data, options) : toonConv.fromJson(data, options);
+        if (from === 'yaml') return isAsync ? toonConv.fromYamlAsync(data, options) : toonConv.fromYaml(data, options);
+        if (from === 'xml') return isAsync ? toonConv.fromXmlAsync(data, options) : toonConv.fromXml(data, options);
+        if (from === 'csv') return isAsync ? toonConv.fromCsvAsync(data, options) : toonConv.fromCsv(data, options);
+    }
+
+    // Unified Converters for direct translation
+    if (from === 'json') {
+        if (to === 'xml') return isAsync ? jsonConv.toXmlAsync(data, options) : jsonConv.toXml(data, options);
+        if (to === 'csv') return isAsync ? jsonConv.toCsvAsync(data, options) : jsonConv.toCsv(data, options);
+        if (to === 'yaml') return isAsync ? jsonConv.toYamlAsync(data, options) : jsonConv.toYaml(data, options);
+    }
+
+    if (from === 'yaml') {
+        if (to === 'json') return isAsync ? yamlConv.toJsonAsync(data, options) : yamlConv.toJson(data, options);
+        if (to === 'xml') return isAsync ? yamlConv.toXmlAsync(data, options) : yamlConv.toXml(data, options);
+        if (to === 'csv') return isAsync ? yamlConv.toCsvAsync(data, options) : yamlConv.toCsv(data, options);
+    }
+
+    if (from === 'xml') {
+        if (to === 'json') return isAsync ? xmlConv.toJsonAsync(data, options) : xmlConv.toJson(data, options);
+        if (to === 'csv') return isAsync ? xmlConv.toCsvAsync(data, options) : xmlConv.toCsv(data, options);
+        if (to === 'yaml') return isAsync ? xmlConv.toYamlAsync(data, options) : xmlConv.toYaml(data, options);
+    }
+
+    if (from === 'csv') {
+        if (to === 'json') return isAsync ? csvConv.toJsonAsync(data, options) : csvConv.toJson(data, options);
+        if (to === 'xml') return isAsync ? csvConv.toXmlAsync(data, options) : csvConv.toXml(data, options);
+        if (to === 'yaml') return isAsync ? csvConv.toYamlAsync(data, options) : csvConv.toYaml(data, options);
+    }
+
+    throw new Error(`Unsupported conversion from ${from} to ${to}`);
+}
+
+run();