jtcsv 1.2.0 → 2.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +272 -329
- package/bin/jtcsv.js +1092 -97
- package/cli-tui.js +0 -0
- package/csv-to-json.js +385 -311
- package/dist/jtcsv.cjs.js +1619 -0
- package/dist/jtcsv.cjs.js.map +1 -0
- package/dist/jtcsv.esm.js +1599 -0
- package/dist/jtcsv.esm.js.map +1 -0
- package/dist/jtcsv.umd.js +1625 -0
- package/dist/jtcsv.umd.js.map +1 -0
- package/examples/cli-tool.js +186 -0
- package/examples/express-api.js +167 -0
- package/examples/large-dataset-example.js +185 -0
- package/examples/plugin-excel-exporter.js +407 -0
- package/examples/simple-usage.js +280 -0
- package/examples/streaming-example.js +419 -0
- package/index.d.ts +288 -1
- package/index.js +23 -0
- package/json-save.js +1 -1
- package/json-to-csv.js +130 -89
- package/package.json +139 -13
- package/plugins/README.md +373 -0
- package/plugins/express-middleware/README.md +306 -0
- package/plugins/express-middleware/example.js +136 -0
- package/plugins/express-middleware/index.d.ts +114 -0
- package/plugins/express-middleware/index.js +360 -0
- package/plugins/express-middleware/package.json +52 -0
- package/plugins/fastify-plugin/index.js +406 -0
- package/plugins/fastify-plugin/package.json +55 -0
- package/plugins/nextjs-api/README.md +452 -0
- package/plugins/nextjs-api/examples/ConverterComponent.jsx +386 -0
- package/plugins/nextjs-api/examples/api-convert.js +69 -0
- package/plugins/nextjs-api/index.js +388 -0
- package/plugins/nextjs-api/package.json +63 -0
- package/plugins/nextjs-api/route.js +372 -0
- package/src/browser/browser-functions.js +189 -0
- package/src/browser/csv-to-json-browser.js +442 -0
- package/src/browser/errors-browser.js +194 -0
- package/src/browser/index.js +79 -0
- package/src/browser/json-to-csv-browser.js +309 -0
- package/src/browser/workers/csv-parser.worker.js +359 -0
- package/src/browser/workers/worker-pool.js +467 -0
- package/src/core/delimiter-cache.js +186 -0
- package/src/core/plugin-system.js +472 -0
- package/src/core/transform-hooks.js +350 -0
- package/src/engines/fast-path-engine-new.js +338 -0
- package/src/engines/fast-path-engine.js +836 -0
- package/src/formats/ndjson-parser.js +419 -0
- package/src/formats/tsv-parser.js +336 -0
- package/src/index-with-plugins.js +371 -0
- package/stream-csv-to-json.js +1 -1
- package/stream-json-to-csv.js +1 -1
package/README.md
CHANGED
|
@@ -1,140 +1,116 @@
|
|
|
1
|
-
|
|
1
|
+
# jtcsv - Complete JSON ↔ CSV Converter
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**Version 2.0** - Now with full browser support, Web Workers, and streaming!
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
```javascript
|
|
9
|
-
const { jsonToCsv } = require('jtcsv');
|
|
5
|
+
[jtcsv
|
|
6
|
+
[](https://opensource.org/licenses/MIT)
|
|
7
|
+
[](https://www.npmjs.com/package/jtcsv
|
|
10
8
|
|
|
11
|
-
|
|
12
|
-
{ id: 1, name: 'John Doe' },
|
|
13
|
-
{ id: 2, name: 'Jane Smith' }
|
|
14
|
-
], { delimiter: ',' });
|
|
15
|
-
|
|
16
|
-
console.log(csv);
|
|
17
|
-
// Output:
|
|
18
|
-
// id,name
|
|
19
|
-
// 1,John Doe
|
|
20
|
-
// 2,Jane Smith
|
|
21
|
-
```
|
|
9
|
+
A lightweight, efficient, and secure library for converting between JSON and CSV formats with full browser support, Web Workers for large files, and streaming capabilities.
|
|
22
10
|
|
|
23
|
-
|
|
24
|
-
```javascript
|
|
25
|
-
const { csvToJson } = require('jtcsv');
|
|
11
|
+
## ✨ Features
|
|
26
12
|
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
13
|
+
### 🚀 Core Features
|
|
14
|
+
- **Bidirectional Conversion**: JSON ↔ CSV with full type preservation
|
|
15
|
+
- **Zero Dependencies**: Pure JavaScript/TypeScript, no external dependencies
|
|
16
|
+
- **TypeScript Support**: Full type definitions included
|
|
17
|
+
- **Security First**: Built-in CSV injection protection
|
|
18
|
+
- **RFC 4180 Compliant**: Proper CSV formatting and escaping
|
|
30
19
|
|
|
31
|
-
|
|
32
|
-
|
|
20
|
+
### 🌐 Browser Support
|
|
21
|
+
- **Full Browser Compatibility**: Chrome, Firefox, Safari, Edge, Mobile
|
|
22
|
+
- **Web Workers**: Process large files without blocking UI
|
|
23
|
+
- **File API Integration**: Direct file upload/download support
|
|
24
|
+
- **Streaming Processing**: Handle files of any size
|
|
25
|
+
- **Progress Tracking**: Real-time progress updates
|
|
33
26
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
27
|
+
### ⚡ Performance
|
|
28
|
+
- **High Speed**: Optimized parsing algorithms
|
|
29
|
+
- **Memory Efficient**: Streaming and chunked processing
|
|
30
|
+
- **Worker Pool**: Reusable Web Workers for parallel processing
|
|
31
|
+
- **Caching**: Intelligent caching for repeated operations
|
|
37
32
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
```
|
|
33
|
+
### 🔧 Advanced Features
|
|
34
|
+
- **Auto-detection**: Automatic delimiter detection
|
|
35
|
+
- **Custom Headers**: Flexible header mapping and renaming
|
|
36
|
+
- **Nested Objects**: Support for complex nested structures
|
|
37
|
+
- **Multiple Formats**: CSV, TSV, Excel-compatible output
|
|
38
|
+
- **Error Handling**: Comprehensive error reporting and recovery
|
|
45
39
|
|
|
46
40
|
## 📦 Installation
|
|
47
41
|
|
|
42
|
+
### Node.js
|
|
48
43
|
```bash
|
|
49
44
|
npm install jtcsv
|
|
50
45
|
```
|
|
51
46
|
|
|
52
|
-
|
|
47
|
+
### Browser (CDN)
|
|
48
|
+
```html
|
|
49
|
+
<!-- UMD version (global jtcsv variable) -->
|
|
50
|
+
<script src="https://cdn.jsdelivr.net/npm/jtcsvst/jtcsv.umd.js"></script>
|
|
53
51
|
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
### ✅ **Streaming API for Large Files**
|
|
60
|
-
- Process files >100MB without loading into memory
|
|
61
|
-
- Real-time transformation with backpressure handling
|
|
62
|
-
- Schema validation during streaming
|
|
63
|
-
|
|
64
|
-
### ✅ **Enterprise-Grade Security**
|
|
65
|
-
- **CSV Injection Protection**: Automatic escaping of Excel formulas
|
|
66
|
-
- **Path Traversal Protection**: Safe file path validation
|
|
67
|
-
- **Input Validation**: Type checking and size limits
|
|
68
|
-
|
|
69
|
-
### ✅ **Performance Optimized**
|
|
70
|
-
- Zero dependencies, ~8KB package size
|
|
71
|
-
- Memory-efficient streaming
|
|
72
|
-
- RFC 4180 compliant output
|
|
73
|
-
|
|
74
|
-
### ✅ **TypeScript Ready**
|
|
75
|
-
- Full TypeScript definitions included
|
|
76
|
-
- IntelliSense support in modern editors
|
|
52
|
+
<!-- ESM version -->
|
|
53
|
+
<script type="module">
|
|
54
|
+
import { jsonToCsv } from 'https://cdn.jsdelivr.net/npm/jtcsvst/jtcsv.esm.js';
|
|
55
|
+
</script>
|
|
56
|
+
```
|
|
77
57
|
|
|
78
|
-
##
|
|
58
|
+
## 🚀 Quick Start
|
|
79
59
|
|
|
80
|
-
###
|
|
60
|
+
### Node.js Usage
|
|
81
61
|
```javascript
|
|
82
|
-
const {
|
|
62
|
+
const { jsonToCsv, csvToJson } = require('jtcsv
|
|
63
|
+
|
|
64
|
+
// JSON to CSV
|
|
65
|
+
const data = [
|
|
66
|
+
{ id: 1, name: 'John', email: 'john@example.com' },
|
|
67
|
+
{ id: 2, name: 'Jane', email: 'jane@example.com' }
|
|
68
|
+
];
|
|
83
69
|
|
|
84
|
-
|
|
85
|
-
const users = await db.query('SELECT * FROM users');
|
|
86
|
-
await saveAsCsv(users, './exports/users.csv', {
|
|
70
|
+
const csv = jsonToCsv(data, {
|
|
87
71
|
delimiter: ',',
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
email: 'Email Address',
|
|
91
|
-
created_at: 'Registration Date'
|
|
92
|
-
}
|
|
72
|
+
includeHeaders: true,
|
|
73
|
+
preventCsvInjection: true
|
|
93
74
|
});
|
|
94
|
-
```
|
|
95
75
|
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
76
|
+
console.log(csv);
|
|
77
|
+
// id,name,email
|
|
78
|
+
// 1,John,john@example.com
|
|
79
|
+
// 2,Jane,jane@example.com
|
|
99
80
|
|
|
100
|
-
//
|
|
101
|
-
const
|
|
81
|
+
// CSV to JSON
|
|
82
|
+
const csvString = 'id,name,email\n1,John,john@example.com\n2,Jane,jane@example.com';
|
|
83
|
+
const json = csvToJson(csvString, {
|
|
102
84
|
delimiter: ',',
|
|
103
|
-
parseNumbers: true
|
|
104
|
-
parseBooleans: true
|
|
85
|
+
parseNumbers: true
|
|
105
86
|
});
|
|
106
87
|
|
|
107
|
-
|
|
88
|
+
console.log(json);
|
|
89
|
+
// [
|
|
90
|
+
// { id: 1, name: 'John', email: 'john@example.com' },
|
|
91
|
+
// { id: 2, name: 'Jane', email: 'jane@example.com' }
|
|
92
|
+
// ]
|
|
108
93
|
```
|
|
109
94
|
|
|
110
|
-
###
|
|
95
|
+
### Browser Usage
|
|
111
96
|
```javascript
|
|
112
|
-
|
|
113
|
-
const
|
|
97
|
+
// Using global variable (UMD)
|
|
98
|
+
const csv = window.jtcsv.jsonToCsv(data, { delimiter: ',' });
|
|
99
|
+
|
|
100
|
+
// Download as file
|
|
101
|
+
window.jtcsv.downloadAsCsv(data, 'export.csv', { delimiter: ',' });
|
|
114
102
|
|
|
115
|
-
//
|
|
116
|
-
const
|
|
117
|
-
await
|
|
118
|
-
delimiter: ','
|
|
103
|
+
// Parse uploaded file
|
|
104
|
+
const fileInput = document.querySelector('input[type="file"]');
|
|
105
|
+
const json = await window.jtcsv.parseCsvFile(fileInput.files[0], {
|
|
106
|
+
delimiter: ',',
|
|
107
|
+
parseNumbers: true
|
|
119
108
|
});
|
|
120
|
-
```
|
|
121
109
|
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
const
|
|
125
|
-
|
|
126
|
-
// Convert API response to downloadable CSV
|
|
127
|
-
app.get('/api/users/export', async (req, res) => {
|
|
128
|
-
const users = await fetchUsersFromAPI();
|
|
129
|
-
const csv = jsonToCsv(users, {
|
|
130
|
-
delimiter: ',',
|
|
131
|
-
preventCsvInjection: true,
|
|
132
|
-
rfc4180Compliant: true
|
|
133
|
-
});
|
|
134
|
-
|
|
135
|
-
res.setHeader('Content-Type', 'text/csv');
|
|
136
|
-
res.setHeader('Content-Disposition', 'attachment; filename=\"users.csv\"');
|
|
137
|
-
res.send(csv);
|
|
110
|
+
// Use Web Workers for large files
|
|
111
|
+
const largeFile = document.querySelector('input[type="file"]').files[0];
|
|
112
|
+
const result = await window.jtcsv.parseCSVWithWorker(largeFile, {}, (progress) => {
|
|
113
|
+
console.log(`Progress: ${progress.percentage.toFixed(1)}%`);
|
|
138
114
|
});
|
|
139
115
|
```
|
|
140
116
|
|
|
@@ -143,294 +119,261 @@ app.get('/api/users/export', async (req, res) => {
|
|
|
143
119
|
### Core Functions
|
|
144
120
|
|
|
145
121
|
#### `jsonToCsv(data, options)`
|
|
146
|
-
|
|
122
|
+
Converts an array of objects to CSV string.
|
|
147
123
|
|
|
148
124
|
**Options:**
|
|
149
|
-
- `delimiter` (default: ';')
|
|
150
|
-
- `includeHeaders` (default: true)
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
- `rfc4180Compliant` (default: true) - RFC 4180 compliance
|
|
125
|
+
- `delimiter` (string, default: ';'): CSV delimiter character
|
|
126
|
+
- `includeHeaders` (boolean, default: true): Include header row
|
|
127
|
+
- `renameMap` (object): Map for renaming column headers
|
|
128
|
+
- `preventCsvInjection` (boolean, default: true): Escape formulas for security
|
|
129
|
+
- `rfc4180Compliant` (boolean, default: true): RFC 4180 compliance
|
|
130
|
+
- `maxRecords` (number): Maximum records to process
|
|
156
131
|
|
|
157
132
|
#### `csvToJson(csv, options)`
|
|
158
|
-
|
|
133
|
+
Converts CSV string to array of objects.
|
|
159
134
|
|
|
160
135
|
**Options:**
|
|
161
|
-
|
|
162
|
-
- `
|
|
163
|
-
- `
|
|
164
|
-
- `hasHeaders` (default: true)
|
|
165
|
-
- `
|
|
166
|
-
- `
|
|
167
|
-
- `
|
|
168
|
-
- `maxRows` (
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
- `candidates` (optional) - Array of candidate delimiters (default: [';', ',', '\t', '|'])
|
|
176
|
-
|
|
177
|
-
**Returns:** Detected delimiter string
|
|
136
|
+
Fast path parsing is the default pipeline; use `fastPathMode` to control row shape.
|
|
137
|
+
- `delimiter` (string): Delimiter (auto-detected if not specified)
|
|
138
|
+
- `autoDetect` (boolean, default: true): Auto-detect delimiter
|
|
139
|
+
- `hasHeaders` (boolean, default: true): CSV has header row
|
|
140
|
+
- `parseNumbers` (boolean, default: false): Parse numeric values
|
|
141
|
+
- `parseBooleans` (boolean, default: false): Parse boolean values
|
|
142
|
+
- `trim` (boolean, default: true): Trim whitespace
|
|
143
|
+
- `maxRows` (number): Maximum rows to process
|
|
144
|
+
- `useFastPath` (boolean, default: true): Enable fast-path parser (set `false` to force quote-aware path)
|
|
145
|
+
- `fastPathMode` (string, default: 'objects'): `'objects'` for object rows, `'compact'` for arrays (lower memory), `'stream'` to return an async iterator
|
|
146
|
+
|
|
147
|
+
#### `csvToJsonIterator(csv, options)`
|
|
148
|
+
Convert CSV to JSON rows as an async iterator for large inputs.
|
|
149
|
+
You can also call `csvToJson(csv, { fastPathMode: 'stream' })` to get the same async iterator.
|
|
178
150
|
|
|
179
151
|
**Example:**
|
|
180
152
|
```javascript
|
|
181
|
-
const {
|
|
153
|
+
const { csvToJsonIterator } = require('jtcsv');
|
|
182
154
|
|
|
183
|
-
const
|
|
184
|
-
|
|
155
|
+
const csv = 'id,name\n1,Jane\n2,John';
|
|
156
|
+
for await (const row of csvToJsonIterator(csv, { fastPathMode: 'compact' })) {
|
|
157
|
+
console.log(row);
|
|
158
|
+
}
|
|
185
159
|
```
|
|
186
160
|
|
|
187
|
-
|
|
188
|
-
Save JSON data as CSV file with security validation.
|
|
189
|
-
|
|
190
|
-
#### `readCsvAsJson(filePath, options)`
|
|
191
|
-
Read CSV file and convert to JSON array.
|
|
161
|
+
### Browser-Specific Functions
|
|
192
162
|
|
|
193
|
-
#### `
|
|
194
|
-
|
|
163
|
+
#### `downloadAsCsv(data, filename, options)`
|
|
164
|
+
Converts and downloads JSON as CSV file.
|
|
195
165
|
|
|
196
|
-
|
|
166
|
+
#### `parseCsvFile(file, options)`
|
|
167
|
+
Parses CSV File object to JSON.
|
|
197
168
|
|
|
198
|
-
#### `
|
|
199
|
-
|
|
169
|
+
#### `createCsvBlob(data, options)`
|
|
170
|
+
Creates CSV Blob without downloading.
|
|
200
171
|
|
|
201
|
-
#### `
|
|
202
|
-
|
|
172
|
+
#### `parseCsvBlob(blob, options)`
|
|
173
|
+
Parses CSV Blob to JSON.
|
|
203
174
|
|
|
204
|
-
|
|
205
|
-
Stream JSON to CSV file.
|
|
175
|
+
### Web Workers Functions
|
|
206
176
|
|
|
207
|
-
#### `
|
|
208
|
-
|
|
177
|
+
#### `createWorkerPool(options)`
|
|
178
|
+
Creates a pool of Web Workers for parallel processing.
|
|
209
179
|
|
|
210
|
-
|
|
211
|
-
|
|
212
|
-
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
Custom error classes for better debugging:
|
|
216
|
-
- `JtcsvError` - Base error class
|
|
217
|
-
- `ValidationError` - Input validation errors
|
|
218
|
-
- `SecurityError` - Security violations
|
|
219
|
-
- `FileSystemError` - File system operations
|
|
220
|
-
- `ParsingError` - CSV/JSON parsing errors
|
|
221
|
-
- `LimitError` - Size limit exceeded
|
|
222
|
-
- `ConfigurationError` - Invalid configuration
|
|
180
|
+
**Options:**
|
|
181
|
+
- `workerCount` (number, default: 4): Number of workers
|
|
182
|
+
- `maxQueueSize` (number, default: 100): Maximum queue size
|
|
183
|
+
- `autoScale` (boolean, default: true): Auto-scale workers
|
|
184
|
+
- `idleTimeout` (number, default: 60000): Idle timeout in ms
|
|
223
185
|
|
|
224
|
-
|
|
186
|
+
#### `parseCSVWithWorker(csvInput, options, onProgress)`
|
|
187
|
+
Parses CSV using Web Workers with progress tracking.
|
|
225
188
|
|
|
226
|
-
|
|
227
|
-
```javascript
|
|
228
|
-
// Dangerous data with Excel formulas
|
|
229
|
-
const dangerous = [
|
|
230
|
-
{ id: 1, formula: '=HYPERLINK(\"http://evil.com\",\"Click\")' },
|
|
231
|
-
{ id: 2, formula: '@IMPORTANT' }
|
|
232
|
-
];
|
|
189
|
+
## 💡 Examples
|
|
233
190
|
|
|
234
|
-
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
191
|
+
### React Component Example
|
|
192
|
+
```jsx
|
|
193
|
+
import React, { useState } from 'react';
|
|
194
|
+
import { parseCsvFile, downloadAsCsv } from 'jtcsv
|
|
238
195
|
|
|
239
|
-
|
|
240
|
-
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
196
|
+
export function CSVProcessor() {
|
|
197
|
+
const [data, setData] = useState([]);
|
|
198
|
+
const [loading, setLoading] = useState(false);
|
|
199
|
+
|
|
200
|
+
const handleFileUpload = async (event) => {
|
|
201
|
+
const file = event.target.files[0];
|
|
202
|
+
if (!file) return;
|
|
203
|
+
|
|
204
|
+
setLoading(true);
|
|
205
|
+
try {
|
|
206
|
+
const jsonData = await parseCsvFile(file, {
|
|
207
|
+
delimiter: ',',
|
|
208
|
+
parseNumbers: true
|
|
209
|
+
});
|
|
210
|
+
setData(jsonData);
|
|
211
|
+
} catch (error) {
|
|
212
|
+
console.error('Error:', error);
|
|
213
|
+
} finally {
|
|
214
|
+
setLoading(false);
|
|
215
|
+
}
|
|
216
|
+
};
|
|
217
|
+
|
|
218
|
+
const handleExport = () => {
|
|
219
|
+
downloadAsCsv(data, 'export.csv', { delimiter: ',' });
|
|
220
|
+
};
|
|
221
|
+
|
|
222
|
+
return (
|
|
223
|
+
<div>
|
|
224
|
+
<input type="file" accept=".csv" onChange={handleFileUpload} />
|
|
225
|
+
<button onClick={handleExport} disabled={!data.length}>
|
|
226
|
+
Export to CSV
|
|
227
|
+
</button>
|
|
228
|
+
{loading && <div>Processing...</div>}
|
|
229
|
+
<pre>{JSON.stringify(data.slice(0, 5), null, 2)}</pre>
|
|
230
|
+
</div>
|
|
231
|
+
);
|
|
246
232
|
}
|
|
247
233
|
```
|
|
248
234
|
|
|
249
|
-
###
|
|
235
|
+
### Large File Processing with Progress
|
|
250
236
|
```javascript
|
|
251
|
-
|
|
252
|
-
jsonToCsv('not an array'); // throws ValidationError
|
|
253
|
-
jsonToCsv([], { delimiter: 123 }); // throws ConfigurationError
|
|
254
|
-
jsonToCsv(largeArray, { maxRecords: 100 }); // throws LimitError if >100 records
|
|
255
|
-
```
|
|
256
|
-
|
|
257
|
-
## 📈 Performance
|
|
237
|
+
import { parseCSVWithWorker } from 'jtcsv
|
|
258
238
|
|
|
259
|
-
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
|
|
266
|
-
|
|
267
|
-
|
|
268
|
-
|
|
269
|
-
|
|
239
|
+
async function processLargeFile(file) {
|
|
240
|
+
const progressBar = document.getElementById('progress-bar');
|
|
241
|
+
const status = document.getElementById('status');
|
|
242
|
+
|
|
243
|
+
try {
|
|
244
|
+
const result = await parseCSVWithWorker(file, {}, (progress) => {
|
|
245
|
+
const percent = Math.round(progress.percentage);
|
|
246
|
+
progressBar.style.width = percent + '%';
|
|
247
|
+
progressBar.textContent = percent + '%';
|
|
248
|
+
|
|
249
|
+
status.textContent =
|
|
250
|
+
`Processing: ${progress.processed.toLocaleString()} of ${progress.total.toLocaleString()} rows ` +
|
|
251
|
+
`(${Math.round(progress.speed)} rows/sec)`;
|
|
252
|
+
});
|
|
253
|
+
|
|
254
|
+
status.textContent = `Processed ${result.length.toLocaleString()} rows successfully`;
|
|
255
|
+
return result;
|
|
256
|
+
} catch (error) {
|
|
257
|
+
status.textContent = `Error: ${error.message}`;
|
|
258
|
+
throw error;
|
|
259
|
+
}
|
|
260
|
+
}
|
|
270
261
|
```
|
|
271
262
|
|
|
272
|
-
|
|
273
|
-
|
|
263
|
+
### Security: CSV Injection Protection
|
|
274
264
|
```javascript
|
|
275
|
-
const
|
|
276
|
-
|
|
277
|
-
// Original data
|
|
278
|
-
const original = [
|
|
279
|
-
{ id: 1, name: 'John', active: true, score: 95.5 },
|
|
280
|
-
{ id: 2, name: 'Jane', active: false, score: 88.0 }
|
|
265
|
+
const dangerousData = [
|
|
266
|
+
{ formula: '=SUM(1,2)', command: '=cmd|"/c calc"!A1' }
|
|
281
267
|
];
|
|
282
268
|
|
|
283
|
-
//
|
|
284
|
-
const
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
288
|
-
|
|
289
|
-
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
parseBooleans: true
|
|
295
|
-
});
|
|
296
|
-
|
|
297
|
-
// restored is identical to original
|
|
298
|
-
console.assert(JSON.stringify(original) === JSON.stringify(restored));
|
|
269
|
+
// With protection enabled (default)
|
|
270
|
+
const safeCsv = jsonToCsv(dangerousData, { preventCsvInjection: true });
|
|
271
|
+
// formula,command
|
|
272
|
+
// "'=SUM(1,2)","'=cmd|"/c calc"!A1"
|
|
273
|
+
// Formulas are prefixed with single quote to prevent execution
|
|
274
|
+
|
|
275
|
+
// Without protection
|
|
276
|
+
const unsafeCsv = jsonToCsv(dangerousData, { preventCsvInjection: false });
|
|
277
|
+
// formula,command
|
|
278
|
+
// =SUM(1,2),=cmd|"/c calc"!A1
|
|
279
|
+
// WARNING: This could execute commands in Excel!
|
|
299
280
|
```
|
|
300
281
|
|
|
301
|
-
##
|
|
282
|
+
## 📊 Performance
|
|
302
283
|
|
|
303
|
-
|
|
304
|
-
# Run all tests (108 tests)
|
|
305
|
-
npm test
|
|
284
|
+
### Benchmark Results (Node.js 22, 10K rows/records)
|
|
306
285
|
|
|
307
|
-
|
|
308
|
-
npm run test:coverage
|
|
286
|
+
**CSV → JSON (10K rows)**
|
|
309
287
|
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
288
|
+
| Library | Time | Memory | Rank |
|
|
289
|
+
|---------|------|--------|------|
|
|
290
|
+
| **JTCSV (FastPath Compact)** | 16.79 ms | 4.47 MB | 🥇 1st |
|
|
291
|
+
| **JTCSV (FastPath Stream)** | 18.27 ms | 6.03 MB | 🥈 2nd |
|
|
292
|
+
| **JTCSV** | 19.76 ms | 8.96 MB | 🥉 3rd |
|
|
293
|
+
| PapaParse | 21.57 ms | 6.97 MB | 4th |
|
|
294
|
+
| csv-parser | 30.52 ms | 6.53 MB | 5th |
|
|
313
295
|
|
|
314
|
-
|
|
315
|
-
npm run lint
|
|
296
|
+
**JSON → CSV (10K records)**
|
|
316
297
|
|
|
317
|
-
|
|
318
|
-
|
|
319
|
-
|
|
298
|
+
| Library | Time | Memory | Rank |
|
|
299
|
+
|---------|------|--------|------|
|
|
300
|
+
| **JTCSV** | 11.21 ms | 4.77 MB | 🥇 1st |
|
|
301
|
+
| json2csv | 12.27 ms | 12.11 MB | 🥈 2nd |
|
|
320
302
|
|
|
321
|
-
|
|
303
|
+
### Scaling (JTCSV only)
|
|
322
304
|
|
|
323
|
-
|
|
305
|
+
| Rows/Records | CSV→JSON Time (FastPath Compact) | JSON→CSV Time (JTCSV) | CSV→JSON Memory | JSON→CSV Memory |
|
|
306
|
+
|--------------|----------------------------------|-----------------------|-----------------|-----------------|
|
|
307
|
+
| 1,000 | 2.06 ms | 1.04 ms | 2.15 MB | 0.52 MB |
|
|
308
|
+
| 10,000 | 14.68 ms | 8.23 ms | 2.11 MB | 4.14 MB |
|
|
309
|
+
| 100,000 | 164.18 ms | 90.93 ms | 44.93 MB | 34.79 MB |
|
|
324
310
|
|
|
325
|
-
|
|
326
|
-
jtcsv/
|
|
327
|
-
├── index.js # Main entry point
|
|
328
|
-
├── index.d.ts # TypeScript definitions
|
|
329
|
-
├── json-to-csv.js # JSON→CSV conversion
|
|
330
|
-
├── csv-to-json.js # CSV→JSON conversion
|
|
331
|
-
├── errors.js # Error classes
|
|
332
|
-
├── stream-json-to-csv.js # Streaming API
|
|
333
|
-
├── examples/ # Usage examples
|
|
334
|
-
│ ├── express-api.js # Express server example
|
|
335
|
-
│ ├── cli-tool.js # Command line tool
|
|
336
|
-
│ └── large-dataset-example.js
|
|
337
|
-
├── __tests__/ # Test suites
|
|
338
|
-
└── package.json
|
|
339
|
-
```
|
|
311
|
+
See `BENCHMARK-RESULTS.md` and `docs/PERFORMANCE.md` for environment details and methodology.
|
|
340
312
|
|
|
341
|
-
##
|
|
313
|
+
## 🛠️ Development
|
|
342
314
|
|
|
343
|
-
###
|
|
344
|
-
```
|
|
345
|
-
|
|
315
|
+
### Building from Source
|
|
316
|
+
```bash
|
|
317
|
+
# Clone repository
|
|
318
|
+
git clone https://github.com/Linol-Hamelton/jtcsv.git
|
|
319
|
+
cd jtcsv
|
|
346
320
|
|
|
347
|
-
|
|
348
|
-
|
|
321
|
+
# Install dependencies
|
|
322
|
+
npm install
|
|
349
323
|
|
|
350
|
-
|
|
351
|
-
|
|
324
|
+
# Build browser version
|
|
325
|
+
npm run build
|
|
352
326
|
|
|
353
|
-
|
|
354
|
-
|
|
327
|
+
# Run tests
|
|
328
|
+
npm test
|
|
355
329
|
|
|
356
|
-
|
|
357
|
-
|
|
330
|
+
# Start demo server
|
|
331
|
+
npm run demo
|
|
358
332
|
```
|
|
359
333
|
|
|
360
|
-
###
|
|
361
|
-
```
|
|
362
|
-
|
|
363
|
-
|
|
364
|
-
|
|
365
|
-
|
|
366
|
-
|
|
367
|
-
|
|
368
|
-
|
|
369
|
-
|
|
370
|
-
|
|
371
|
-
|
|
372
|
-
|
|
334
|
+
### Project Structure
|
|
335
|
+
```
|
|
336
|
+
jtcsv/
|
|
337
|
+
├── src/browser/ # Browser-specific code
|
|
338
|
+
│ ├── index.js # Browser entry point
|
|
339
|
+
│ ├── *.js # Browser modules
|
|
340
|
+
│ └── workers/ # Web Workers implementation
|
|
341
|
+
├── dist/ # Built distributions
|
|
342
|
+
│ ├── jtcsv.umd.js # UMD bundle
|
|
343
|
+
│ ├── jtcsv.esm.js # ESM bundle
|
|
344
|
+
│ └── jtcsv.cjs.js # CJS bundle
|
|
345
|
+
├── demo/ # Demo application
|
|
346
|
+
├── __tests__/ # Test files
|
|
347
|
+
├── rollup.config.mjs # Build configuration
|
|
348
|
+
└── package.json # Project configuration
|
|
373
349
|
```
|
|
374
350
|
|
|
375
351
|
## 🤝 Contributing
|
|
376
352
|
|
|
353
|
+
Contributions are welcome! Please feel free to submit a Pull Request.
|
|
354
|
+
|
|
377
355
|
1. Fork the repository
|
|
378
|
-
2. Create
|
|
379
|
-
3. Add
|
|
380
|
-
4.
|
|
381
|
-
5.
|
|
356
|
+
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
|
|
357
|
+
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
|
|
358
|
+
4. Push to the branch (`git push origin feature/amazing-feature`)
|
|
359
|
+
5. Open a Pull Request
|
|
382
360
|
|
|
383
361
|
## 📄 License
|
|
384
362
|
|
|
385
|
-
MIT
|
|
363
|
+
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
386
364
|
|
|
387
|
-
##
|
|
365
|
+
## 🙏 Acknowledgments
|
|
388
366
|
|
|
389
|
-
-
|
|
390
|
-
-
|
|
391
|
-
-
|
|
367
|
+
- Inspired by the need for secure, efficient CSV processing in browsers
|
|
368
|
+
- Thanks to all contributors who have helped improve this library
|
|
369
|
+
- Special thanks to the open source community for invaluable tools and libraries
|
|
392
370
|
|
|
393
|
-
|
|
371
|
+
## 📞 Support
|
|
394
372
|
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
- Simple JSON↔CSV conversion needs
|
|
399
|
-
- Security-conscious applications
|
|
400
|
-
- Large file processing (via streaming)
|
|
401
|
-
- Embedding in other packages (zero deps)
|
|
402
|
-
- TypeScript projects
|
|
403
|
-
- Enterprise applications requiring RFC compliance
|
|
404
|
-
|
|
405
|
-
### ⚠️ **Consider Alternatives For:**
|
|
406
|
-
- Browser-only applications (use PapaParse)
|
|
407
|
-
- Extremely complex CSV formats
|
|
408
|
-
- Real-time streaming in browsers
|
|
409
|
-
|
|
410
|
-
## 📊 Comparison with Alternatives
|
|
411
|
-
|
|
412
|
-
| Feature | jtcsv | json2csv | PapaParse | csv-parser |
|
|
413
|
-
|---------|-------|----------|-----------|------------|
|
|
414
|
-
| **Size** | 8KB | 45KB | 35KB | 1.5KB |
|
|
415
|
-
| **Dependencies** | 0 | 4 | 0 | 0 |
|
|
416
|
-
| **JSON→CSV** | ✅ | ✅ | ✅ | ❌ |
|
|
417
|
-
| **CSV→JSON** | ✅ | ✅ | ✅ | ✅ |
|
|
418
|
-
| **Streaming** | ✅ | ❌ | ✅ | ✅ |
|
|
419
|
-
| **Auto-detect Delimiter** | ✅ | ❌ | ✅ | ❌ |
|
|
420
|
-
| **CSV Injection Protection** | ✅ | ❌ | ⚠️ | ❌ |
|
|
421
|
-
| **TypeScript** | ✅ | ✅ | ✅ | ❌ |
|
|
422
|
-
| **RFC 4180** | ✅ | ✅ | ✅ | ✅ |
|
|
423
|
-
|
|
424
|
-
## 🆕 What's New in v1.0.0
|
|
425
|
-
|
|
426
|
-
- **Complete bidirectional conversion** (JSON↔CSV)
|
|
427
|
-
- **Streaming API** for large files (>100MB)
|
|
428
|
-
- **Enhanced security** with CSV injection protection
|
|
429
|
-
- **TypeScript definitions** for all functions
|
|
430
|
-
- **100% test coverage** (108 passing tests)
|
|
431
|
-
- **CI/CD pipeline** with GitHub Actions
|
|
432
|
-
- **Comprehensive documentation**
|
|
373
|
+
- **GitHub Issues**: [Report bugs or request features](https://github.com/Linol-Hamelton/jtcsv/issues)
|
|
374
|
+
- **Documentation**: [Full API documentation](https://github.com/Linol-Hamelton/jtcsv#readme)
|
|
375
|
+
- **Examples**: [Example code and demos](https://github.com/Linol-Hamelton/jtcsv/tree/main/demo)
|
|
433
376
|
|
|
434
377
|
---
|
|
435
378
|
|
|
436
|
-
**
|
|
379
|
+
**Happy coding!** If you find this library useful, please consider giving it a star on GitHub ⭐
|