toon-formatter 1.1.1 → 2.0.1

package/CHANGELOG.md

@@ -0,0 +1,95 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [2.0.0] - 2025-12-08
+
+### Added
+- **🔐 End-to-End Encryption Support**
+  - New `Encryptor` class with AES-256-GCM, XOR, and Base64 algorithms
+  - `Encryptor.generateKey()` static method for secure key generation
+  - Instance-based `ToonConverter` API with encryption support
+  - Four conversion modes: `no_encryption`, `middleware`, `ingestion`, `export`
+  - Comprehensive encryption documentation and examples
+
+- **New Features**
+  - `returnJson` parameter for `toJson()` and `toJsonAsync()` methods
+  - Support for encrypted data pipelines
+  - Key rotation capabilities
+  - Authentication tag validation (AES-256-GCM)
+
+- **Testing**
+  - 33 new encryption unit tests
+  - 32 new integration tests
+  - 8 backward compatibility tests
+  - Total: 96 tests (all passing)
+
+- **Documentation**
+  - Complete encryption guide in README
+  - Security best practices
+  - Real-world examples
+  - Migration guide
+  - API reference for Encryptor and ToonConverter
+
+### Changed
+- `ToonConverter` now supports both static methods (backward compatible) and instance methods (with encryption)
+- `toonToJsonSync()` and `toonToJson()` now accept optional `returnJson` parameter
+- Updated package description to mention encryption support
+- Enhanced package keywords for better discoverability
+
+### Backward Compatibility
+- ✅ All existing static methods work exactly as before
+- ✅ Default behavior unchanged (`returnJson=false`)
+- ✅ No breaking changes for existing users
+- ✅ Full backward compatibility maintained
+
+### Security
+- AES-256-GCM encryption with random IV per operation
+- Authentication tag prevents data tampering
+- Secure key generation using Node.js crypto
+- No external cryptographic dependencies
+
+## [1.1.1] - Previous Release
+
+### Features
+- JSON, YAML, XML, CSV conversion support
+- Mixed text extraction
+- Synchronous and asynchronous APIs
+- TOON validation
+- Comprehensive test suite
+
+---
+
+## Migration Guide
+
+### From v1.x to v2.0
+
+**No changes required!** Version 2.0 is fully backward compatible.
+
+**To use new encryption features:**
+
+```javascript
+// Old code (still works)
+import { ToonConverter } from 'toon-formatter';
+const toon = ToonConverter.fromJson(data);
+
+// New code (with encryption)
+import { ToonConverter, Encryptor } from 'toon-formatter';
+const key = Encryptor.generateKey();
+const encryptor = new Encryptor(key, 'aes-256-gcm');
+const converter = new ToonConverter(encryptor);
+const encrypted = converter.fromJson(data, { conversionMode: 'export' });
+```
+
+**To use returnJson parameter:**
+
+```javascript
+// Returns object (default, backward compatible)
+const obj = ToonConverter.toJson(toonString);
+
+// Returns JSON string (new feature)
+const jsonString = ToonConverter.toJson(toonString, true);
+```

package/README.md

@@ -1,9 +1,14 @@
 # 🚀 TOON Converter
 
-A lightweight, zero-dependency* library to convert between **TOON** (Token-Oriented Object Notation) and popular data formats (JSON, YAML, XML, CSV).
+A lightweight library to convert between **TOON** (Token-Oriented Object Notation) and popular data formats (JSON, YAML, XML, CSV).
 
 **Reduce your LLM token costs by up to 40%** using the TOON format!
 
+- **Documentation**: https://toonformatter.net/docs.html?package=toon-formatter
+- **Source Code**: https://github.com/ankitpal181/toon-formatter-lib
+- **Bug Reports**: https://github.com/ankitpal181/toon-formatter-lib/issues
+- **POC Tool**: https://toonformatter.net/
+
 \* *Only external dependencies: `js-yaml` and `papaparse` for YAML and CSV parsing*
 
 ---
@@ -288,9 +293,374 @@ console.log(converted);
 // price: 29.99
 ```
 
+```
+
+---
+
+## 🔐 Encryption Support
+
+**NEW in v2.0.0**: The TOON Converter now supports end-to-end encryption for secure data transmission and storage!
+
+### Overview
+
+The encryption feature allows you to:
+- **Encrypt data before transmission** to protect sensitive information
+- **Store encrypted TOON data** securely
+- **Process encrypted data** without exposing plaintext
+- **Support multiple encryption algorithms**: AES-256-GCM, XOR, Base64
+
+### Quick Start with Encryption
+
+```javascript
+import { ToonConverter, Encryptor } from 'toon-formatter';
+
+// 1. Generate a secure encryption key
+const key = Encryptor.generateKey(); // 32-byte key for AES-256-GCM
+
+// 2. Create an encryptor
+const encryptor = new Encryptor(key, 'aes-256-gcm');
+
+// 3. Create a converter with encryption support
+const converter = new ToonConverter(encryptor);
+
+// 4. Convert and encrypt data
+const data = { user: "Alice", role: "admin" };
+const encryptedToon = converter.fromJson(data, { 
+    conversionMode: 'export' 
+});
+
+console.log(encryptedToon); // Encrypted string
+
+// 5. Decrypt and convert back
+const decrypted = encryptor.decrypt(encryptedToon);
+console.log(decrypted); // Plain TOON string
+```
+
+### Encryption Algorithms
+
+#### AES-256-GCM (Recommended)
+High-security authenticated encryption with Galois/Counter Mode.
+
+```javascript
+const key = Encryptor.generateKey(); // Generates 32-byte key
+const encryptor = new Encryptor(key, 'aes-256-gcm');
+```
+
+**Features:**
+- ✅ Military-grade encryption
+- ✅ Authentication tag prevents tampering
+- ✅ Random IV for each encryption
+- ✅ No external dependencies (uses Node.js crypto)
+
+#### XOR Cipher
+Simple obfuscation (not cryptographically secure).
+
+```javascript
+const encryptor = new Encryptor('my-secret-key', 'xor');
+```
+
+**Use cases:**
+- Quick obfuscation
+- Non-sensitive data
+- Deterministic encryption
+
+#### Base64 Encoding
+Simple encoding (not encryption).
+
+```javascript
+const encryptor = new Encryptor(null, 'base64');
+```
+
+**Use cases:**
+- Data encoding
+- Testing
+- Non-sensitive transformations
+
+### Conversion Modes
+
+The encryption system supports **4 conversion modes** for different data flow scenarios:
+
+#### 1. `no_encryption` (Default)
+No encryption applied - standard conversion.
+
+```javascript
+const converter = new ToonConverter(encryptor);
+const toon = converter.fromJson(data); // Plain TOON
+```
+
+#### 2. `middleware` Mode
+**Encrypted → Encrypted** (Decrypt → Convert → Re-encrypt)
+
+Perfect for middleware services that need to convert format without exposing data.
+
+```javascript
+// Input: Encrypted JSON
+const encryptedJson = '...'; // From client
+
+// Convert to encrypted TOON (never see plaintext)
+const encryptedToon = converter.fromJson(encryptedJson, {
+    conversionMode: 'middleware'
+});
+
+// Output: Encrypted TOON (can be stored or forwarded)
+```
+
+**Use case:** API gateway converting encrypted client data to encrypted storage format.
+
+#### 3. `ingestion` Mode
+**Encrypted → Plain** (Decrypt → Convert)
+
+For ingesting encrypted data into your system.
+
+```javascript
+// Input: Encrypted JSON from external source
+const encryptedJson = '...';
+
+// Convert to plain TOON for processing
+const plainToon = converter.fromJson(encryptedJson, {
+    conversionMode: 'ingestion'
+});
+
+// Output: Plain TOON (ready for processing)
+```
+
+**Use case:** Receiving encrypted data from clients and converting to internal format.
+
+#### 4. `export` Mode
+**Plain → Encrypted** (Convert → Encrypt)
+
+For exporting data securely.
+
+```javascript
+// Input: Plain JSON data
+const data = { user: "Alice", role: "admin" };
+
+// Convert and encrypt for transmission
+const encryptedToon = converter.fromJson(data, {
+    conversionMode: 'export'
+});
+
+// Output: Encrypted TOON (safe to transmit)
+```
+
+**Use case:** Sending data to external systems or clients securely.
+
+### Real-World Example: Secure API Pipeline
+
+```javascript
+import { ToonConverter, Encryptor } from 'toon-formatter';
+
+// Setup (same key on client and server)
+const key = Encryptor.generateKey();
+const encryptor = new Encryptor(key, 'aes-256-gcm');
+
+// CLIENT SIDE
+// ============
+const clientConverter = new ToonConverter(encryptor);
+
+// 1. User submits sensitive data
+const userData = {
+    ssn: "123-45-6789",
+    creditCard: "4111-1111-1111-1111",
+    email: "alice@example.com"
+};
+
+// 2. Encrypt before sending
+const encryptedPayload = encryptor.encrypt(JSON.stringify(userData));
+
+// 3. Send to server
+await fetch('/api/user', {
+    method: 'POST',
+    body: encryptedPayload
+});
+
+// SERVER SIDE (Middleware)
+// =========================
+const serverConverter = new ToonConverter(encryptor);
+
+// 4. Receive encrypted data
+const encryptedJson = await request.text();
+
+// 5. Convert to encrypted TOON for storage (middleware mode)
+const encryptedToon = serverConverter.fromJson(encryptedJson, {
+    conversionMode: 'middleware'
+});
+
+// 6. Store encrypted TOON in database
+await db.save(encryptedToon);
+
+// SERVER SIDE (Processing)
+// =========================
+// 7. Retrieve encrypted TOON
+const storedToon = await db.get(userId);
+
+// 8. Convert back to plain JSON for processing (ingestion mode)
+const plainData = serverConverter.toJson(storedToon, {
+    conversionMode: 'ingestion',
+    returnJson: true
+});
+
+// 9. Process data
+const user = JSON.parse(plainData);
+console.log(user.email); // alice@example.com
+```
+
+### Working with `returnJson` Parameter
+
+By default, `toJson()` returns a JavaScript object. For encryption modes, you need a string. Use `returnJson: true`:
+
+```javascript
+// Returns object (default)
+const obj = converter.toJson(toonString);
+console.log(obj); // { name: "Alice" }
+
+// Returns JSON string (for encryption)
+const jsonString = converter.toJson(toonString, { returnJson: true });
+console.log(jsonString); // '{"name":"Alice"}'
+
+// With encryption
+const encrypted = converter.toJson(toonString, {
+    conversionMode: 'export',
+    returnJson: true  // Required for encryption!
+});
+```
+
+### Key Management Best Practices
+
+#### 🔑 Generating Keys
+
+```javascript
+// Generate a secure random key
+const key = Encryptor.generateKey();
+
+// Store as Base64 (e.g., in environment variables)
+const keyBase64 = key.toString('base64');
+process.env.ENCRYPTION_KEY = keyBase64;
+
+// Load from storage
+const loadedKey = Buffer.from(process.env.ENCRYPTION_KEY, 'base64');
+const encryptor = new Encryptor(loadedKey, 'aes-256-gcm');
+```
+
+#### 🔒 Security Best Practices
+
+1. **Never hardcode keys** in source code
+2. **Use environment variables** or secure key management systems
+3. **Rotate keys periodically** for long-term security
+4. **Use AES-256-GCM** for production (not XOR or Base64)
+5. **Protect keys at rest** with proper file permissions
+6. **Use HTTPS** for transmitting encrypted data
+7. **Implement key rotation** strategy
+
+#### 🔄 Key Rotation Example
+
+```javascript
+// Old system
+const oldKey = Buffer.from(process.env.OLD_KEY, 'base64');
+const oldEncryptor = new Encryptor(oldKey, 'aes-256-gcm');
+
+// New system
+const newKey = Encryptor.generateKey();
+const newEncryptor = new Encryptor(newKey, 'aes-256-gcm');
+
+// Migrate data
+const encryptedData = await db.getAllEncrypted();
+
+for (const item of encryptedData) {
+    // Decrypt with old key
+    const plaintext = oldEncryptor.decrypt(item.data);
+    
+    // Re-encrypt with new key
+    const reEncrypted = newEncryptor.encrypt(plaintext);
+    
+    // Update database
+    await db.update(item.id, reEncrypted);
+}
+
+// Update environment variable
+process.env.ENCRYPTION_KEY = newKey.toString('base64');
+```
+
+### Error Handling
+
+```javascript
+try {
+    const encrypted = encryptor.encrypt(data);
+    const decrypted = encryptor.decrypt(encrypted);
+} catch (error) {
+    if (error.message.includes('decryption failed')) {
+        console.error('Wrong key or tampered data');
+    } else if (error.message.includes('Invalid encrypted data format')) {
+        console.error('Corrupted ciphertext');
+    } else {
+        console.error('Encryption error:', error.message);
+    }
+}
+```
+
+### Async Encryption
+
+All encryption operations work with async methods:
+
+```javascript
+const converter = new ToonConverter(encryptor);
+
+// Async conversion with encryption
+const encrypted = await converter.fromJsonAsync(data, {
+    conversionMode: 'export'
+});
+
+const decrypted = await converter.toJsonAsync(encrypted, {
+    conversionMode: 'ingestion',
+    returnJson: true
+});
+```
+
+### Migration Guide
+
+#### From Static to Instance API
+
+**Before (no encryption):**
+```javascript
+import { ToonConverter } from 'toon-formatter';
+
+const toon = ToonConverter.fromJson(data);
+const json = ToonConverter.toJson(toon);
+```
+
+**After (with encryption):**
+```javascript
+import { ToonConverter, Encryptor } from 'toon-formatter';
+
+// Create encryptor
+const key = Encryptor.generateKey();
+const encryptor = new Encryptor(key, 'aes-256-gcm');
+
+// Create converter instance
+const converter = new ToonConverter(encryptor);
+
+// Use instance methods
+const encrypted = converter.fromJson(data, { conversionMode: 'export' });
+const plain = converter.toJson(encrypted, { conversionMode: 'ingestion' });
+```
+
+**Note:** Static methods still work for backward compatibility (no encryption).
+
+### Performance Considerations
+
+- **AES-256-GCM**: ~0.5-1ms per operation (recommended)
+- **XOR**: ~0.1ms per operation (fast but insecure)
+- **Base64**: ~0.05ms per operation (fastest, no security)
+
+For high-throughput applications, consider:
+- Batch processing
+- Caching decrypted data (with proper TTL)
+- Using middleware mode to avoid double encryption/decryption
+
 ---
 
-## �📚 API Reference
+## 📚 API Reference
+
 
 ### JSON Converters
 
@@ -335,25 +705,27 @@ Converts JSON data to TOON format (asynchronous).
 
 **Returns:** `Promise<string>` - TOON formatted string
 
-#### `toonToJsonSync(toonString)`
+#### `toonToJsonSync(toonString, returnJson?)`
 Converts TOON string to JSON (synchronous).
 
 **Supports:** ❌ Pure TOON data only (no mixed text)
 
 **Parameters:**
 - `toonString` (string): TOON formatted string
+- `returnJson` (boolean, optional): If `true`, returns JSON string; if `false` (default), returns object
 
-**Returns:** `any` - Parsed JSON data
+**Returns:** `any | string` - Parsed JSON data (object by default, string if `returnJson=true`)
 
-#### `toonToJson(toonString)`
+#### `toonToJson(toonString, returnJson?)`
 Converts TOON string to JSON (asynchronous).
 
 **Supports:** ❌ Pure TOON data only (no mixed text)
 
 **Parameters:**
 - `toonString` (string): TOON formatted string
+- `returnJson` (boolean, optional): If `true`, returns JSON string; if `false` (default), returns object
 
-**Returns:** `Promise<any>` - Parsed JSON data
+**Returns:** `Promise<any | string>` - Parsed JSON data (object by default, string if `returnJson=true`)
 
 ---
 
@@ -554,6 +926,166 @@ Validates a TOON string for syntax and structural correctness (asynchronous).
 
 ---
 
+
+---
+
+### Encryptor Class
+
+The `Encryptor` class provides encryption and decryption capabilities.
+
+#### `new Encryptor(key, algorithm)`
+Creates a new Encryptor instance.
+
+**Parameters:**
+- `key` (Buffer | string | null): Encryption key
+  - For `aes-256-gcm`: 32-byte Buffer (use `Encryptor.generateKey()`)
+  - For `xor`: String or Buffer
+  - For `base64`: null (no key needed)
+- `algorithm` (string): Encryption algorithm - `'aes-256-gcm'`, `'xor'`, or `'base64'`
+
+**Example:**
+```javascript
+// AES-256-GCM (recommended)
+const key = Encryptor.generateKey();
+const encryptor = new Encryptor(key, 'aes-256-gcm');
+
+// XOR
+const xorEncryptor = new Encryptor('my-secret-key', 'xor');
+
+// Base64
+const base64Encryptor = new Encryptor(null, 'base64');
+```
+
+#### `Encryptor.generateKey()`
+Static method to generate a secure 32-byte encryption key for AES-256-GCM.
+
+**Returns:** `Buffer` - 32-byte random key
+
+**Example:**
+```javascript
+const key = Encryptor.generateKey();
+console.log(key.length); // 32
+
+// Store as Base64
+const keyBase64 = key.toString('base64');
+process.env.ENCRYPTION_KEY = keyBase64;
+
+// Load from Base64
+const loadedKey = Buffer.from(process.env.ENCRYPTION_KEY, 'base64');
+```
+
+#### `encryptor.encrypt(data)`
+Encrypts a string.
+
+**Parameters:**
+- `data` (string): Plaintext string to encrypt
+
+**Returns:** `string` - Encrypted string (hex-encoded for AES-256-GCM and XOR, Base64 for base64)
+
+**Throws:** Error if data is not a string or key is missing (for AES/XOR)
+
+**Example:**
+```javascript
+const encrypted = encryptor.encrypt('Hello, World!');
+console.log(encrypted); // Hex string (AES-256-GCM)
+```
+
+#### `encryptor.decrypt(encryptedData)`
+Decrypts an encrypted string.
+
+**Parameters:**
+- `encryptedData` (string): Encrypted string
+
+**Returns:** `string` - Decrypted plaintext
+
+**Throws:** Error if decryption fails, wrong key, or tampered data
+
+**Example:**
+```javascript
+const decrypted = encryptor.decrypt(encrypted);
+console.log(decrypted); // 'Hello, World!'
+```
+
+---
+
+### ToonConverter Class (with Encryption)
+
+The `ToonConverter` class now supports both static methods (backward compatible) and instance methods (with encryption).
+
+#### `new ToonConverter(encryptor?)`
+Creates a new ToonConverter instance.
+
+**Parameters:**
+- `encryptor` (Encryptor | null, optional): Encryptor instance for encryption support
+
+**Example:**
+```javascript
+// Without encryption
+const converter = new ToonConverter();
+
+// With encryption
+const key = Encryptor.generateKey();
+const encryptor = new Encryptor(key, 'aes-256-gcm');
+const converter = new ToonConverter(encryptor);
+```
+
+#### Instance Methods with Encryption Support
+
+All instance methods accept an `options` object with:
+- `conversionMode` (string): `'no_encryption'` (default), `'middleware'`, `'ingestion'`, or `'export'`
+- `returnJson` (boolean, for `toJson` methods): If `true`, returns JSON string; if `false` (default), returns object
+
+**Example:**
+```javascript
+// fromJson with encryption
+const encrypted = converter.fromJson(data, {
+    conversionMode: 'export'
+});
+
+// toJson with encryption and JSON string output
+const jsonString = converter.toJson(toonString, {
+    conversionMode: 'ingestion',
+    returnJson: true
+});
+
+// fromYaml with middleware mode
+const encryptedToon = converter.fromYaml(encryptedYaml, {
+    conversionMode: 'middleware'
+});
+```
+
+**Available instance methods:**
+- `fromJson(data, options?)` / `fromJsonAsync(data, options?)`
+- `toJson(toonString, options?)` / `toJsonAsync(toonString, options?)`
+- `fromYaml(yamlString, options?)` / `fromYamlAsync(yamlString, options?)`
+- `toYaml(toonString, options?)` / `toYamlAsync(toonString, options?)`
+- `fromXml(xmlString, options?)` / `fromXmlAsync(xmlString, options?)`
+- `toXml(toonString, options?)` / `toXmlAsync(toonString, options?)`
+- `fromCsv(csvString, options?)` / `fromCsvAsync(csvString, options?)`
+- `toCsv(toonString, options?)` / `toCsvAsync(toonString, options?)`
+- `validate(toonString)` / `validateAsync(toonString)`
+
+#### Static Methods (Backward Compatible)
+
+Static methods work exactly as before, with no encryption support:
+
+```javascript
+// Static usage (no encryption)
+const toon = ToonConverter.fromJson(data);
+const json = ToonConverter.toJson(toon);
+
+// Static with returnJson parameter
+const jsonString = ToonConverter.toJson(toon, true);
+```
+
+**Note:** For `toJson` and `toJsonAsync` static methods, you can pass `returnJson` as the second parameter:
+```javascript
+ToonConverter.toJson(toonString, returnJson?)
+ToonConverter.toJsonAsync(toonString, returnJson?)
+```
+
+---
+
 ## 🎨 TOON Format Guide
 
 ### Primitives