jtcsv 1.2.0 → 2.1.0

package/README.md

@@ -1,140 +1,116 @@
-"# jtcsv - **The Complete JSON↔CSV Converter for Node.js**
+# jtcsv - Complete JSON ↔ CSV Converter
 
-⚡ **Zero dependencies** | 🚀 **Streaming for large files** | 🔄 **Bidirectional conversion** | 🔒 **Security built-in** | 📊 **100% test coverage**
+**Version 2.0** - Now with full browser support, Web Workers, and streaming!
 
-## 🚀 Quick Start
+[![npm version](https://img.shields.io/npm/v/jtcsvps://www.npmjs.com/package/jtcsv)jtcsv
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
+[![Zero Dependencies](https://img.shields.io/badge/dependencies-zero-brightgreen.svg)](https://www.npmjs.com/package/jtcsv
 
-### JSON → CSV
-```javascript
-const { jsonToCsv } = require('jtcsv');
+A lightweight, efficient, and secure library for converting between JSON and CSV formats with full browser support, Web Workers for large files, and streaming capabilities.
 
-const csv = jsonToCsv([
-  { id: 1, name: 'John Doe' },
-  { id: 2, name: 'Jane Smith' }
-], { delimiter: ',' });
+## ✨ Features
 
-console.log(csv);
-// Output:
-// id,name
-// 1,John Doe
-// 2,Jane Smith
-```
+### 🚀 Core Features
+- **Bidirectional Conversion**: JSON ↔ CSV with full type preservation
+- **Zero Dependencies**: Pure JavaScript/TypeScript, no external dependencies
+- **TypeScript Support**: Full type definitions included
+- **Security First**: Built-in CSV injection protection
+- **RFC 4180 Compliant**: Proper CSV formatting and escaping
 
-### CSV → JSON
-```javascript
-const { csvToJson } = require('jtcsv');
+### 🌐 Browser Support
+- **Full Browser Compatibility**: Chrome, Firefox, Safari, Edge, Mobile
+- **Web Workers**: Process large files without blocking UI
+- **File API Integration**: Direct file upload/download support
+- **Streaming Processing**: Handle files of any size
+- **Progress Tracking**: Real-time progress updates
 
-// Auto-detect delimiter (no need to specify)
-const csv = 'id,name\\n1,John\\n2,Jane';
-const json = csvToJson(csv); // Automatically detects comma delimiter
+### ⚡ Performance
+- **High Speed**: Optimized parsing algorithms
+- **Memory Efficient**: Streaming and chunked processing
+- **Worker Pool**: Reusable Web Workers for parallel processing
+- **Caching**: Intelligent caching for repeated operations
 
-console.log(json);
-// Output: [{id: '1', name: 'John'}, {id: '2', name: 'Jane'}]
-
-// Works with any delimiter
-const csvSemicolon = 'id;name;email\\n1;John;john@example.com';
-const json2 = csvToJson(csvSemicolon); // Automatically detects semicolon
-
-// Disable auto-detect if needed
-const csvCustom = 'id|name|age\\n1|John|30';
-const json3 = csvToJson(csvCustom, { 
-  delimiter: '|', 
-  autoDetect: false 
-});
-```
+### 🔧 Advanced Features
+- **Auto-detection**: Automatic delimiter detection
+- **Custom Headers**: Flexible header mapping and renaming
+- **Nested Objects**: Support for complex nested structures
+- **Multiple Formats**: CSV, TSV, Excel-compatible output
+- **Error Handling**: Comprehensive error reporting and recovery
 
 ## 📦 Installation
 
+### Node.js
 ```bash
 npm install jtcsv
 ```
 
-## ✨ Key Features
-
-### ✅ **Complete JSON↔CSV Conversion**
-- **JSON → CSV**: Convert arrays of objects to CSV format
-- **CSV → JSON**: Parse CSV strings back to JSON arrays
-- **File Operations**: Read/write CSV files with security validation
+### Browser (CDN)
+```html
+<!-- UMD version (global jtcsv variable) -->
+<script src="https://cdn.jsdelivr.net/npm/jtcsvst/jtcsv.umd.js"></script>
 
-### ✅ **Streaming API for Large Files**
-- Process files >100MB without loading into memory
-- Real-time transformation with backpressure handling
-- Schema validation during streaming
-
-### ✅ **Enterprise-Grade Security**
-- **CSV Injection Protection**: Automatic escaping of Excel formulas
-- **Path Traversal Protection**: Safe file path validation
-- **Input Validation**: Type checking and size limits
-
-### ✅ **Performance Optimized**
-- Zero dependencies, ~8KB package size
-- Memory-efficient streaming
-- RFC 4180 compliant output
-
-### ✅ **TypeScript Ready**
-- Full TypeScript definitions included
-- IntelliSense support in modern editors
+<!-- ESM version -->
+<script type="module">
+  import { jsonToCsv } from 'https://cdn.jsdelivr.net/npm/jtcsvst/jtcsv.esm.js';
+</script>
+```
 
-## 📊 Real-World Examples
+## 🚀 Quick Start
 
-### 1. Database Export to CSV
+### Node.js Usage
 ```javascript
-const { saveAsCsv } = require('jtcsv');
+const { jsonToCsv, csvToJson } = require('jtcsv
 
-// Export users from database
-const users = await db.query('SELECT * FROM users');
-await saveAsCsv(users, './exports/users.csv', {
+// JSON to CSV
+const data = [
+  { id: 1, name: 'John', email: 'john@example.com' },
+  { id: 2, name: 'Jane', email: 'jane@example.com' }
+];
+
+const csv = jsonToCsv(data, {
   delimiter: ',',
-  renameMap: {
-    id: 'User ID',
-    email: 'Email Address',
-    created_at: 'Registration Date'
-  }
+  includeHeaders: true,
+  preventCsvInjection: true
 });
-```
 
-### 2. CSV Import to Database
-```javascript
-const { readCsvAsJson } = require('jtcsv');
+console.log(csv);
+// id,name,email
+// 1,John,john@example.com
+// 2,Jane,jane@example.com
 
-// Import users from CSV file
-const users = await readCsvAsJson('./imports/users.csv', {
+// CSV to JSON
+const csvString = 'id,name,email\n1,John,john@example.com\n2,Jane,jane@example.com';
+const json = csvToJson(csvString, {
   delimiter: ',',
-  parseNumbers: true,
-  parseBooleans: true
+  parseNumbers: true
 });
 
-await db.insert('users', users);
+console.log(json);
+// [
+//   { id: 1, name: 'John', email: 'john@example.com' },
+//   { id: 2, name: 'Jane', email: 'jane@example.com' }
+// ]
 ```
 
-### 3. Streaming Large Dataset
+### Browser Usage
 ```javascript
-const { createJsonToCsvStream, saveJsonStreamAsCsv } = require('./stream-json-to-csv.js');
-const fs = require('fs');
+// Using global variable (UMD)
+const csv = window.jtcsv.jsonToCsv(data, { delimiter: ',' });
+
+// Download as file
+window.jtcsv.downloadAsCsv(data, 'export.csv', { delimiter: ',' });
 
-// Process 1GB JSON file without loading into memory
-const jsonStream = fs.createReadStream('./large-data.jsonl', 'utf8');
-await saveJsonStreamAsCsv(jsonStream, './output.csv', {
-  delimiter: ','
+// Parse uploaded file
+const fileInput = document.querySelector('input[type="file"]');
+const json = await window.jtcsv.parseCsvFile(fileInput.files[0], {
+  delimiter: ',',
+  parseNumbers: true
 });
-```
 
-### 4. API Response Conversion
-```javascript
-const { jsonToCsv } = require('jtcsv');
-
-// Convert API response to downloadable CSV
-app.get('/api/users/export', async (req, res) => {
-  const users = await fetchUsersFromAPI();
-  const csv = jsonToCsv(users, {
-    delimiter: ',',
-    preventCsvInjection: true,
-    rfc4180Compliant: true
-  });
-  
-  res.setHeader('Content-Type', 'text/csv');
-  res.setHeader('Content-Disposition', 'attachment; filename=\"users.csv\"');
-  res.send(csv);
+// Use Web Workers for large files
+const largeFile = document.querySelector('input[type="file"]').files[0];
+const result = await window.jtcsv.parseCSVWithWorker(largeFile, {}, (progress) => {
+  console.log(`Progress: ${progress.percentage.toFixed(1)}%`);
 });
 ```
 
@@ -143,294 +119,233 @@ app.get('/api/users/export', async (req, res) => {
 ### Core Functions
 
 #### `jsonToCsv(data, options)`
-Convert JSON array to CSV string.
+Converts an array of objects to CSV string.
 
 **Options:**
-- `delimiter` (default: ';') - CSV delimiter character
-- `includeHeaders` (default: true) - Include headers row
-  - `renameMap` - Rename column headers `{ oldKey: newKey }`
-  - `template` - Ensure consistent column order
-  - `maxRecords` (optional) - Maximum records to process (no limit by default)
-  - `preventCsvInjection` (default: true) - Escape Excel formulas
-  - `rfc4180Compliant` (default: true) - RFC 4180 compliance
+- `delimiter` (string, default: ';'): CSV delimiter character
+- `includeHeaders` (boolean, default: true): Include header row
+- `renameMap` (object): Map for renaming column headers
+- `preventCsvInjection` (boolean, default: true): Escape formulas for security
+- `rfc4180Compliant` (boolean, default: true): RFC 4180 compliance
+- `maxRecords` (number): Maximum records to process
 
 #### `csvToJson(csv, options)`
-Convert CSV string to JSON array.
+Converts CSV string to array of objects.
 
 **Options:**
-- `delimiter` (default: auto-detected) - CSV delimiter character
-- `autoDetect` (default: true) - Auto-detect delimiter if not specified
-- `candidates` (default: [';', ',', '\t', '|']) - Candidate delimiters for auto-detection
-- `hasHeaders` (default: true) - CSV has headers row
-- `renameMap` - Rename column headers `{ newKey: oldKey }`
-- `parseNumbers` (default: false) - Parse numeric values
-- `parseBooleans` (default: false) - Parse boolean values
-- `maxRows` (optional) - Maximum rows to process (no limit by default)
-
-#### `autoDetectDelimiter(csv, candidates)`
-Auto-detect CSV delimiter from content.
+- `delimiter` (string): Delimiter (auto-detected if not specified)
+- `autoDetect` (boolean, default: true): Auto-detect delimiter
+- `hasHeaders` (boolean, default: true): CSV has header row
+- `parseNumbers` (boolean, default: false): Parse numeric values
+- `parseBooleans` (boolean, default: false): Parse boolean values
+- `trim` (boolean, default: true): Trim whitespace
+- `maxRows` (number): Maximum rows to process
 
-**Parameters:**
-- `csv` - CSV content string
-- `candidates` (optional) - Array of candidate delimiters (default: [';', ',', '\t', '|'])
+### Browser-Specific Functions
 
-**Returns:** Detected delimiter string
+#### `downloadAsCsv(data, filename, options)`
+Converts and downloads JSON as CSV file.
 
-**Example:**
-```javascript
-const { autoDetectDelimiter } = require('jtcsv');
-
-const delimiter = autoDetectDelimiter('id,name,age\\n1,John,30');
-console.log(delimiter); // Output: ','
-```
+#### `parseCsvFile(file, options)`
+Parses CSV File object to JSON.
 
-#### `saveAsCsv(data, filePath, options)`
-Save JSON data as CSV file with security validation.
+#### `createCsvBlob(data, options)`
+Creates CSV Blob without downloading.
 
-#### `readCsvAsJson(filePath, options)`
-Read CSV file and convert to JSON array.
+#### `parseCsvBlob(blob, options)`
+Parses CSV Blob to JSON.
 
-#### `readCsvAsJsonSync(filePath, options)`
-Synchronous version of `readCsvAsJson`.
+### Web Workers Functions
 
-### Streaming API (stream-json-to-csv.js)
+#### `createWorkerPool(options)`
+Creates a pool of Web Workers for parallel processing.
 
-#### `createJsonToCsvStream(options)`
-Create transform stream for JSON→CSV conversion.
-
-#### `streamJsonToCsv(inputStream, outputStream, options)`
-Pipe JSON stream through CSV transformation.
-
-#### `saveJsonStreamAsCsv(inputStream, filePath, options)`
-Stream JSON to CSV file.
-
-#### `createJsonReadableStream(data)`
-Create readable stream from JSON array.
-
-#### `createCsvCollectorStream()`
-Create writable stream that collects CSV data.
-
-### Error Handling
+**Options:**
+- `workerCount` (number, default: 4): Number of workers
+- `maxQueueSize` (number, default: 100): Maximum queue size
+- `autoScale` (boolean, default: true): Auto-scale workers
+- `idleTimeout` (number, default: 60000): Idle timeout in ms
 
-Custom error classes for better debugging:
-- `JtcsvError` - Base error class
-- `ValidationError` - Input validation errors
-- `SecurityError` - Security violations
-- `FileSystemError` - File system operations
-- `ParsingError` - CSV/JSON parsing errors
-- `LimitError` - Size limit exceeded
-- `ConfigurationError` - Invalid configuration
+#### `parseCSVWithWorker(csvInput, options, onProgress)`
+Parses CSV using Web Workers with progress tracking.
 
-## 🛡️ Security Features
+## 💡 Examples
 
-### CSV Injection Protection
-```javascript
-// Dangerous data with Excel formulas
-const dangerous = [
-  { id: 1, formula: '=HYPERLINK(\"http://evil.com\",\"Click\")' },
-  { id: 2, formula: '@IMPORTANT' }
-];
+### React Component Example
+```jsx
+import React, { useState } from 'react';
+import { parseCsvFile, downloadAsCsv } from 'jtcsv
 
-// Automatically escaped
-const safeCsv = jsonToCsv(dangerous);
-// Formulas are prefixed with ' to prevent execution
+export function CSVProcessor() {
+  const [data, setData] = useState([]);
+  const [loading, setLoading] = useState(false);
+  
+  const handleFileUpload = async (event) => {
+    const file = event.target.files[0];
+    if (!file) return;
+    
+    setLoading(true);
+    try {
+      const jsonData = await parseCsvFile(file, {
+        delimiter: ',',
+        parseNumbers: true
+      });
+      setData(jsonData);
+    } catch (error) {
+      console.error('Error:', error);
+    } finally {
+      setLoading(false);
+    }
+  };
+  
+  const handleExport = () => {
+    downloadAsCsv(data, 'export.csv', { delimiter: ',' });
+  };
+  
+  return (
+    <div>
+      <input type="file" accept=".csv" onChange={handleFileUpload} />
+      <button onClick={handleExport} disabled={!data.length}>
+        Export to CSV
+      </button>
+      {loading && <div>Processing...</div>}
+      <pre>{JSON.stringify(data.slice(0, 5), null, 2)}</pre>
+    </div>
+  );
+}
 ```
 
-### Path Traversal Protection
+### Large File Processing with Progress
 ```javascript
-try {
-  // This will throw SecurityError
-  await saveAsCsv(data, '../../../etc/passwd.csv');
-} catch (error) {
-  console.error('Security violation:', error.message);
+import { parseCSVWithWorker } from 'jtcsv
+
+async function processLargeFile(file) {
+  const progressBar = document.getElementById('progress-bar');
+  const status = document.getElementById('status');
+  
+  try {
+    const result = await parseCSVWithWorker(file, {}, (progress) => {
+      const percent = Math.round(progress.percentage);
+      progressBar.style.width = percent + '%';
+      progressBar.textContent = percent + '%';
+      
+      status.textContent = 
+        `Processing: ${progress.processed.toLocaleString()} of ${progress.total.toLocaleString()} rows ` +
+        `(${Math.round(progress.speed)} rows/sec)`;
+    });
+    
+    status.textContent = `Processed ${result.length.toLocaleString()} rows successfully`;
+    return result;
+  } catch (error) {
+    status.textContent = `Error: ${error.message}`;
+    throw error;
+  }
 }
 ```
 
-### Input Validation
+### Security: CSV Injection Protection
 ```javascript
-// All inputs are validated
-jsonToCsv('not an array'); // throws ValidationError
-jsonToCsv([], { delimiter: 123 }); // throws ConfigurationError
-jsonToCsv(largeArray, { maxRecords: 100 }); // throws LimitError if >100 records
-```
-
-## 📈 Performance
-
-### Memory Efficiency
-- **In-memory**: Unlimited records (with performance warning for >1M)
-- **Streaming**: Unlimited size with constant memory
-- **Zero-copy**: Efficient buffer management
+const dangerousData = [
+  { formula: '=SUM(1,2)', command: '=cmd|"/c calc"!A1' }
+];
 
-### Benchmark Results
-```
-10,000 records: ~15ms
-100,000 records: ~120ms
-1,000,000 records: ~1.2s
-Streaming 1GB file: ~45s (22MB/s)
+// With protection enabled (default)
+const safeCsv = jsonToCsv(dangerousData, { preventCsvInjection: true });
+// formula,command
+// "'=SUM(1,2)","'=cmd|"/c calc"!A1"
+// Formulas are prefixed with single quote to prevent execution
+
+// Without protection
+const unsafeCsv = jsonToCsv(dangerousData, { preventCsvInjection: false });
+// formula,command
+// =SUM(1,2),=cmd|"/c calc"!A1
+// WARNING: This could execute commands in Excel!
 ```
 
-## 🔄 Complete Roundtrip Example
+## 📊 Performance
 
-```javascript
-const { jsonToCsv, csvToJson } = require('jtcsv');
-
-// Original data
-const original = [
-  { id: 1, name: 'John', active: true, score: 95.5 },
-  { id: 2, name: 'Jane', active: false, score: 88.0 }
-];
-
-// Convert to CSV
-const csv = jsonToCsv(original, {
-  delimiter: ',',
-  parseNumbers: true,
-  parseBooleans: true
-});
+### Benchmark Results
 
-// Convert back to JSON
-const restored = csvToJson(csv, {
-  delimiter: ',',
-  parseNumbers: true,
-  parseBooleans: true
-});
+| File Size | Rows | Without Workers | With Workers | Improvement |
+|-----------|------|-----------------|--------------|-------------|
+| 1 MB      | 10K  | 120 ms          | 80 ms        | 33% faster  |
+| 10 MB     | 100K | 1.2 sec         | 0.8 sec      | 33% faster  |
+| 100 MB    | 1M   | 12 sec          | 7 sec        | 42% faster  |
+| 500 MB    | 5M   | 65 sec          | 35 sec       | 46% faster  |
 
-// restored is identical to original
-console.assert(JSON.stringify(original) === JSON.stringify(restored));
-```
+### Memory Usage
+- **Without streaming**: Loads entire file into memory
+- **With streaming**: Processes in chunks (default 10K rows)
+- **With Web Workers**: Distributes memory across workers
 
-## 🧪 Testing
+## 🛠️ Development
 
+### Building from Source
 ```bash
-# Run all tests (108 tests)
-npm test
+# Clone repository
+git clone https://github.com/Linol-Hamelton/jtcsv.git
+cd jtcsv
 
-# Test with coverage
-npm run test:coverage
+# Install dependencies
+npm install
 
-# Run specific test suites
-npm test -- --testPathPattern=csv-to-json
-npm test -- --testPathPattern=stream
+# Build browser version
+npm run build
 
-# Lint code
-npm run lint
+# Run tests
+npm test
 
-# Security audit
-npm run security-check
+# Start demo server
+npm run demo
 ```
 
-**Test Coverage: 100%** (108 passing tests)
-
-## 📁 Project Structure
-
+### Project Structure
 ```
 jtcsv/
-├── index.js              # Main entry point
-├── index.d.ts           # TypeScript definitions
-├── json-to-csv.js       # JSON→CSV conversion
-├── csv-to-json.js       # CSV→JSON conversion
-├── errors.js            # Error classes
-├── stream-json-to-csv.js # Streaming API
-├── examples/            # Usage examples
-│   ├── express-api.js   # Express server example
-│   ├── cli-tool.js      # Command line tool
-│   └── large-dataset-example.js
-├── __tests__/           # Test suites
-└── package.json
+├── src/browser/           # Browser-specific code
+│   ├── index.js          # Browser entry point
+│   ├── *.js              # Browser modules
+│   └── workers/          # Web Workers implementation
+├── dist/                 # Built distributions
+│   ├── jtcsv.umd.js     # UMD bundle
+│   ├── jtcsv.esm.js     # ESM bundle
+│   └── jtcsv.cjs.js     # CJS bundle
+├── demo/                 # Demo application
+├── __tests__/           # Test files
+├── rollup.config.mjs     # Build configuration
+└── package.json         # Project configuration
 ```
 
-## 🚀 Getting Started
-
-### Basic Usage
-```javascript
-const jtcsv = require('jtcsv');
-
-// Convert JSON to CSV
-const csv = jtcsv.jsonToCsv(data);
-
-// Convert CSV to JSON
-const json = jtcsv.csvToJson(csv);
-
-// Save to file
-await jtcsv.saveAsCsv(data, 'output.csv');
+## 🤝 Contributing
 
-// Read from file
-const data = await jtcsv.readCsvAsJson('input.csv');
-```
+Contributions are welcome! Please feel free to submit a Pull Request.
 
-### TypeScript Usage
-```typescript
-import { jsonToCsv, csvToJson } from 'jtcsv';
+1. Fork the repository
+2. Create your feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add some amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
 
-interface User {
-  id: number;
-  name: string;
-  email: string;
-}
+## 📄 License
 
-const users: User[] = [...];
-const csv = jsonToCsv(users);
-const parsed = csvToJson<User>(csv);
-```
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
 
-## 🤝 Contributing
+## 🙏 Acknowledgments
 
-1. Fork the repository
-2. Create a feature branch
-3. Add tests for new functionality
-4. Ensure all tests pass: `npm test`
-5. Submit a Pull Request
+- Inspired by the need for secure, efficient CSV processing in browsers
+- Thanks to all contributors who have helped improve this library
+- Special thanks to the open source community for invaluable tools and libraries
 
-## 📄 License
+## 📞 Support
 
-MIT © Ruslan Fomenko
+- **GitHub Issues**: [Report bugs or request features](https://github.com/Linol-Hamelton/jtcsv/issues)
+- **Documentation**: [Full API documentation](https://github.com/Linol-Hamelton/jtcsv#readme)
+- **Examples**: [Example code and demos](https://github.com/Linol-Hamelton/jtcsv/tree/main/demo)
 
-## 🔗 Links
+---
 
-- **GitHub**: https://github.com/Linol-Hamelton/jtcsv
-- **npm**: https://www.npmjs.com/package/jtcsv
-- **Issues**: https://github.com/Linol-Hamelton/jtcsv/issues
+**Happy coding!** If you find this library useful, please consider giving it a star on GitHub ⭐
 
----
 
-## 🎯 When to Use jtcsv
-
-### ✅ **Perfect For:**
-- Simple JSON↔CSV conversion needs
-- Security-conscious applications
-- Large file processing (via streaming)
-- Embedding in other packages (zero deps)
-- TypeScript projects
-- Enterprise applications requiring RFC compliance
-
-### ⚠️ **Consider Alternatives For:**
-- Browser-only applications (use PapaParse)
-- Extremely complex CSV formats
-- Real-time streaming in browsers
-
-## 📊 Comparison with Alternatives
-
-| Feature | jtcsv | json2csv | PapaParse | csv-parser |
-|---------|-------|----------|-----------|------------|
-| **Size** | 8KB | 45KB | 35KB | 1.5KB |
-| **Dependencies** | 0 | 4 | 0 | 0 |
-| **JSON→CSV** | ✅ | ✅ | ✅ | ❌ |
-| **CSV→JSON** | ✅ | ✅ | ✅ | ✅ |
-| **Streaming** | ✅ | ❌ | ✅ | ✅ |
-| **Auto-detect Delimiter** | ✅ | ❌ | ✅ | ❌ |
-| **CSV Injection Protection** | ✅ | ❌ | ⚠️ | ❌ |
-| **TypeScript** | ✅ | ✅ | ✅ | ❌ |
-| **RFC 4180** | ✅ | ✅ | ✅ | ✅ |
-
-## 🆕 What's New in v1.0.0
-
-- **Complete bidirectional conversion** (JSON↔CSV)
-- **Streaming API** for large files (>100MB)
-- **Enhanced security** with CSV injection protection
-- **TypeScript definitions** for all functions
-- **100% test coverage** (108 passing tests)
-- **CI/CD pipeline** with GitHub Actions
-- **Comprehensive documentation**
 
----
 
-**Ready for production use with enterprise-grade security and performance.**"