convert-csv-to-json 3.28.0 → 4.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,4 +1,5 @@
1
- # CSVtoJSON
1
+ # CSVtoJSON
2
+
2
3
  [![Node CI](https://github.com/iuccio/csvToJson/actions/workflows/ci-cd.yml/badge.svg?branch=master)](https://github.com/iuccio/csvToJson/actions/workflows/ci-cd.yml)
3
4
  ![CodeQL](https://github.com/iuccio/csvToJson/actions/workflows/codeql-analysis.yml/badge.svg)
4
5
  [![Known Vulnerabilities](https://snyk.io/test/github/iuccio/csvToJson/badge.svg)](https://snyk.io/test/github/iuccio/csvToJson)
@@ -7,952 +8,291 @@
7
8
  ![NodeJS Version](https://img.shields.io/badge/nodeJS-%3E=18.x-brightgreen.svg)
8
9
  [![Downloads](https://img.shields.io/npm/dm/convert-csv-to-json.svg)](https://npmjs.org/package/convert-csv-to-json)
9
10
  [![NPM total downloads](https://img.shields.io/npm/dt/convert-csv-to-json.svg?style=flat)](https://npmjs.org/package/convert-csv-to-json)
11
+ [![Socket Badge](https://badge.socket.dev/npm/package/convert-csv-to-json/3.20.0)](https://socket.dev/npm/package/convert-csv-to-json)
10
12
 
11
13
 
12
14
  ![NodeJS](https://img.shields.io/badge/node.js-6DA55F?style=for-the-badge&logo=node.js&logoColor=white)
13
- ![Browser Support](https://img.shields.io/badge/browser-supported-brightgreen.svg?style=for-the-badge&logo=google-chrome&logoColor=white)
14
- ![JavaScript](https://img.shields.io/badge/javascript-%23323330.svg?style=for-the-badge&logo=javascript&logoColor=%23F7DF1E)
15
- ![TypeScript](https://img.shields.io/badge/typescript-%23007ACC.svg?style=for-the-badge&logo=typescript&logoColor=white)
16
-
17
- **This project is not dependent on others packages or libraries, and supports both synchronous and Promise-based asynchronous APIs.**
18
-
19
- This repository uses [![GitHub Action - iuccio/npm-semantic-publish-action@latest](https://img.shields.io/badge/GitHub_Action_-iuccio%2Fnpm--semantic--publish--action%40latest-2ea44f)](https://github.com/marketplace/actions/npm-semver-publish)
20
-
21
- Follow [me](https://github.com/iuccio), and consider starring the project to
22
- show your :heart: and support.
23
-
24
-
25
- ## Table of Contents
26
-
27
- <!-- toc -->
28
-
29
- - [Description](#description)
30
- - [Support for NodeJS, Browser, JS, TS](#support-for-nodejs-browser-js-ts)
31
- - [Prerequisites](#prerequisites)
32
- - [Install npm *convert-csv-to-json package*](#install-npm-convert-csv-to-json-package)
33
- - [Sync API Usage](#sync-api-usage)
34
- * [Generate JSON file](#generate-json-file)
35
- * [Generate Array of Object in JSON format](#generate-array-of-object-in-json-format)
36
- * [Generate Object with sub array](#generate-object-with-sub-array)
37
- * [Define field delimiter](#define-field-delimiter)
38
- * [Trim header field](#trim-header-field)
39
- * [Trim header field with whitespaces](#trim-header-field-with-whitespaces)
40
- * [Support Quoted Fields](#support-quoted-fields)
41
- * [Index header](#index-header)
42
- * [Empty rows](#empty-rows)
43
- * [Format property value by type](#format-property-value-by-type)
44
- + [Numbers](#numbers)
45
- + [Boolean](#boolean)
46
- + [Complete Example](#complete-example)
47
- * [Encoding](#encoding)
48
- * [Working with CSV strings directly](#working-with-csv-strings-directly)
49
- * [Sync API (TypeScript)](#sync-api-typescript)
50
- - [Browser API Usage](#browser-api-usage)
51
- * [Basic Browser Operations](#basic-browser-operations)
52
- * [Parsing File/Blob](#parsing-fileblob)
53
- * [Browser API Notes](#browser-api-notes)
54
- * [Browser API (TypeScript)](#browser-api-typescript)
55
- - [Async API Usage](#async-api-usage)
56
- * [Async API (TypeScript)](#async-api-typescript)
57
- * [Basic Async Operations](#basic-async-operations)
58
- * [Working with Raw CSV Data](#working-with-raw-csv-data)
59
- * [Processing Large Files](#processing-large-files)
60
- * [Error Handling and Retries](#error-handling-and-retries)
61
- * [Batch Processing](#batch-processing)
62
- - [Chaining Pattern](#chaining-pattern)
63
- * [Synchronous Chaining](#synchronous-chaining)
64
- * [Asynchronous Chaining](#asynchronous-chaining)
65
- - [Common Use Cases](#common-use-cases)
66
- * [1. Processing CSV from HTTP Response](#1-processing-csv-from-http-response)
67
- * [2. Batch Processing Multiple Files](#2-batch-processing-multiple-files)
68
- * [3. Data Transformation Pipeline](#3-data-transformation-pipeline)
69
- * [4. Error Recovery and Logging](#4-error-recovery-and-logging)
70
- - [Troubleshooting](#troubleshooting)
71
- * [Memory Issues with Large Files](#memory-issues-with-large-files)
72
- * [Handling Different CSV Formats](#handling-different-csv-formats)
73
- * [Common Error Solutions](#common-error-solutions)
74
- * [Performance Optimization](#performance-optimization)
75
- * [TypeScript Support](#typescript-support)
76
- - [Development](#development)
77
- - [CI CD github action](#ci-cd-github-action)
78
- - [License](#license)
79
- - [Buy me a Coffee](#buy-me-a-coffee)
80
-
81
- <!-- tocstop -->
82
-
83
- ## Description
84
- Converts *csv* files to *JSON* files with Node.js. Supports both synchronous operations and Promise-based asynchronous operations, allowing integration with modern async/await patterns.
85
-
86
- Give an input file like:
87
-
88
- |first_name|last_name|email|gender|age|zip|registered|
89
- |:----------:|:-------:|:---:|:----:|:---:|:---:|:---:|
90
- |Constantin|Langsdon|clangsdon0@hc360.com|Male|96|123|true|
91
- |Norah|Raison|nraison1@wired.com|Female|32| |false|
92
-
93
- e.g. :
94
- ~~~
95
- first_name;last_name;email;gender;age;zip;registered
96
- Constantin;Langsdon;clangsdon0@hc360.com;Male;96;123;true
97
- Norah;Raison;nraison1@wired.com;Female;32;;false
98
- ~~~
99
-
100
- will generate:
101
-
102
-
103
- ```json
104
- [
105
- {
106
- "first_name": "Constantin",
107
- "last_name": "Langsdon",
108
- "email": "clangsdon0@hc360.com",
109
- "gender": "Male",
110
- "age": "96",
111
- "zip": "123",
112
- "registered": "true"
113
- },
114
- {
115
- "first_name": "Norah",
116
- "last_name": "Raison",
117
- "email": "nraison1@wired.com",
118
- "gender": "Female",
119
- "age": "32",
120
- "zip": "",
121
- "registered": "false"
122
- }
123
- ]
124
- ```
125
- ## Support for NodeJS, Browser, JS, TS
126
-
127
- This package is compatible with:
128
-
129
- ![NodeJS](https://img.shields.io/badge/node.js-6DA55F?style=for-the-badge&logo=node.js&logoColor=white)
130
- ![Browser Support](https://img.shields.io/badge/browser-supported-brightgreen.svg?style=for-the-badge&logo=google-chrome&logoColor=white)
15
+ ![Browser Support](https://img.shields.io/badge/browser-supported-brightgreen.svg?style=for-the-badge&logo=google-chrome&logoColor=white)
131
16
  ![JavaScript](https://img.shields.io/badge/javascript-%23323330.svg?style=for-the-badge&logo=javascript&logoColor=%23F7DF1E)
132
- ![TypeScript](https://img.shields.io/badge/typescript-%23007ACC.svg?style=for-the-badge&logo=typescript&logoColor=white)
17
+ ![TypeScript](https://img.shields.io/badge/typescript-%23007ACC.svg?style=for-the-badge&logo=typescript&logoColor=white)
133
18
 
134
- ## Prerequisites
135
- **NPM** (see [Installing Npm](https://docs.npmjs.com/getting-started/installing-node)).
19
+ >
20
+ Convert CSV files to JSON with **no dependencies**. Supports Node.js (Sync & Async), and Browser environments with full RFC 4180 compliance.
136
21
 
137
- ## Install npm *convert-csv-to-json package*
138
- Go to NPM package [convert-csv-to-json](https://www.npmjs.com/package/convert-csv-to-json).
22
+ ## Overview
139
23
 
140
- Install package in your *package.json*
141
- ```bash
142
- $ npm install convert-csv-to-json --save
143
- ```
144
- Install package on your machine
145
- ```bash
146
- $ npm install -g convert-csv-to-json
147
- ```
148
-
149
- ## Sync API Usage
24
+ Transform CSV data into JSON with a simple, chainable API. Choose your implementation style:
150
25
 
151
- ### Generate JSON file
152
- ```js
153
- let csvToJson = require('convert-csv-to-json');
26
+ - **[Synchronous API](docs/SYNC.md)** - Blocking operations for simple workflows
27
+ - **[Asynchronous API](docs/ASYNC.md)** - Promise-based for modern async/await patterns
28
+ - **[Browser API](docs/BROWSER.md)** - Client-side CSV parsing for web applications
154
29
 
155
- let fileInputName = 'myInputFile.csv';
156
- let fileOutputName = 'myOutputFile.json';
30
+ ## Features
157
31
 
158
- csvToJson.generateJsonFileFromCsv(fileInputName,fileOutputName);
159
- ```
160
- ### Generate Array of Object in JSON format
161
- ```js
162
- let csvToJson = require('convert-csv-to-json');
32
+ ✅ **RFC 4180 Compliant** - Proper handling of quoted fields, delimiters, newlines, and escape sequences
33
+ ✅ **Zero Dependencies** - No external packages required
34
+ **Full TypeScript Support** - Included type definitions for all APIs
35
+ ✅ **Flexible Configuration** - Custom delimiters, encoding, trimming, and more
36
+ **Method Chaining** - Fluent API for readable code
37
+ ✅ **Large File Support** - Stream processing for memory-efficient handling
38
+ ✅ **Comprehensive Error Handling** - Detailed, actionable error messages with solutions (see [ERROR_HANDLING.md](docs/ERROR_HANDLING.md))
163
39
 
164
- let json = csvToJson.getJsonFromCsv("myInputFile.csv");
165
- for(let i=0; i<json.length;i++){
166
- console.log(json[i]);
167
- }
168
- ```
40
+ ## RFC 4180 Standard
169
41
 
170
- ### Generate Object with sub array
171
- ```
172
- firstName;lastName;email;gender;age;birth;sons
173
- Constantin;Langsdon;clangsdon0@hc360.com;Male;96;10.02.1965;*diego,marek,dries*
174
- ```
175
- Given the above CSV example, to generate a JSON Object with properties that contains sub Array, like the property **sons**
176
- with the values <b>*diego,marek,dries*</b> you have to call the function ```parseSubArray(delimiter, separator)``` .
177
- To generate the JSON Object with sub array from the above CSV example:
178
- ```js
179
- csvToJson.parseSubArray('*',',')
180
- .getJsonFromCsv('myInputFile.csv');
181
- ```
182
- The result will be:
183
- ```json
184
- [
185
- {
186
- "firstName": "Constantin",
187
- "lastName": "Langsdon",
188
- "email": "clangsdon0@hc360.com",
189
- "gender": "Male",
190
- "age": "96",
191
- "birth": "10.02.1965",
192
- "sons": ["diego","marek","dries"]
193
- }
194
- ]
195
- ```
42
+ **[RFC 4180](https://datatracker.ietf.org/doc/html/rfc4180)** is the IETF standard specification for CSV (Comma-Separated Values) files. This library is fully compliant with RFC 4180, ensuring proper handling of:
196
43
 
197
- ### Define field delimiter
198
- A field delimiter is needed to split the parsed values. As default the field delimiter is the **semicolon** (**;**), this means that during the parsing when a **semicolon (;)** is matched a new JSON entry is created.
199
- In case your CSV file has defined another field delimiter you have to call the function ```fieldDelimiter(myDelimiter)``` and pass it as parameter the field delimiter.
44
+ | Aspect | RFC 4180 Specification |
45
+ |--------|------------------------|
46
+ | **Default Delimiter** | Comma (`,`) |
47
+ | **Record Delimiter** | CRLF (`\r\n`) or LF (`\n`) |
48
+ | **Quote Character** | Double-quote (`"`) |
49
+ | **Quote Escaping** | Double quotes (`""`) |
200
50
 
201
- E.g. if your field delimiter is the comma **,** then:
51
+ ### RFC 4180 Example
202
52
 
203
- ```js
204
- csvToJson.fieldDelimiter(',')
205
- .getJsonFromCsv(fileInputName);
53
+ ```csv
54
+ firstName,lastName,email
55
+ "Smith, John",Smith,john@example.com
56
+ Jane,Doe,jane@example.com
57
+ "Cooper, Andy",Cooper,andy@company.com
206
58
  ```
207
59
 
208
- ### Trim header field
209
-
210
- The content of the field header is cut off at the beginning and end of the string. E.g. " Last Name " -> "Last Name".
60
+ Note the quoted fields containing commas are properly handled. See [RFC4180_MIGRATION_GUIDE.md](RFC4180_MIGRATION_GUIDE.md) for breaking changes and migration details.
211
61
 
212
- ### Trim header field with whitespaces
62
+ ## Quick Start
213
63
 
214
- Use the method *trimHeaderFieldWhiteSpace(true)* to remove the whitespaces in an header field (E.g. " Last Name " -> "LastName"):
64
+ ### Installation
215
65
 
216
- ```js
217
- csvToJson.trimHeaderFieldWhiteSpace(true)
218
- .getJsonFromCsv(fileInputName);
66
+ ```bash
67
+ npm install convert-csv-to-json
219
68
  ```
220
69
 
221
- ### Support Quoted Fields
222
- To be able to parse correctly fields wrapped in quote, like the **last_name** in the first row in the following example:
223
-
224
- |first_name| last_name |email|
225
- |:----------:|:--------------------------:|:---:|
226
- |Constantin| "Langsdon,Nandson,Gangson" |clangsdon0@hc360.com|
227
-
228
- you need to activate the support quoted fields feature:
70
+ ### Synchronous (Simple)
229
71
 
230
72
  ```js
231
- csvToJson.supportQuotedField(true)
232
- .getJsonFromCsv(fileInputName);
73
+ const csvToJson = require('convert-csv-to-json');
74
+ const json = csvToJson.getJsonFromCsv('input.csv');
233
75
  ```
234
76
 
235
- The result will be:
236
- ```json
237
- [
238
- {
239
- "firstName": "Constantin",
240
- "lastName": "Langsdon,Nandson,Gangson",
241
- "email": "clangsdon0@hc360.com"
242
- }
243
- ]
244
- ```
245
-
246
- ### Index header
247
- If the header is not on the first line you can define the header index like:
77
+ ### Asynchronous (Modern)
248
78
 
249
79
  ```js
250
- csvToJson.indexHeader(3)
251
- .getJsonFromCsv(fileInputName);
80
+ const csvToJson = require('convert-csv-to-json');
81
+ const json = await csvToJson.getJsonFromCsvAsync('input.csv');
252
82
  ```
253
83
 
254
- ### Empty rows
255
- Empty rows are ignored and not parsed.
256
-
257
- ### Format property value by type
258
- The `formatValueByType()` function intelligently converts string values to their appropriate types while preserving data integrity. To enable automatic type conversion:
84
+ ### Browser
259
85
 
260
86
  ```js
261
- csvToJson.formatValueByType()
262
- .getJsonFromCsv(fileInputName);
263
- ```
264
-
265
- This conversion follows these rules:
266
-
267
- #### Numbers
268
- - Regular integers and decimals are converted to Number type
269
- - Numbers with leading zeros are preserved as strings (e.g., "0012" stays "0012")
270
- - Large integers outside JavaScript's safe range are preserved as strings
271
- - Valid decimal numbers are converted to Number type
272
-
273
- For example:
274
- ```json
275
- {
276
- "normalInteger": 42, // Converted to number
277
- "decimal": 3.14, // Converted to number
278
- "leadingZeros": "0012345", // Kept as string to preserve leading zeros
279
- "largeNumber": "9007199254740992" // Kept as string to preserve precision
280
- }
281
- ```
282
-
283
- #### Boolean
284
- Case-insensitive "true" or "false" strings are converted to boolean values:
285
- ```json
286
- {
287
- "registered": true, // From "true" or "TRUE" or "True"
288
- "active": false // From "false" or "FALSE" or "False"
289
- }
87
+ const convert = require('convert-csv-to-json');
88
+ const json = await convert.browser.parseFile(file);
290
89
  ```
291
90
 
292
- #### Complete Example
293
- Input CSV:
294
- ```csv
295
- first_name;last_name;email;gender;age;id;zip;registered
296
- Constantin;Langsdon;clangsdon0@hc360.com;Male;96;00123;123;true
297
- Norah;Raison;nraison1@wired.com;Female;32;987;00456;FALSE
298
- ```
91
+ ## Documentation
299
92
 
300
- Output JSON:
301
- ```json
302
- [
303
- {
304
- "first_name": "Constantin",
305
- "last_name": "Langsdon",
306
- "email": "clangsdon0@hc360.com",
307
- "gender": "Male",
308
- "age": 96,
309
- "id": "00123", // Preserved leading zeros
310
- "zip": 123, // Converted to number
311
- "registered": true // Converted to boolean
312
- },
313
- {
314
- "first_name": "Norah",
315
- "last_name": "Raison",
316
- "email": "nraison1@wired.com",
317
- "gender": "Female",
318
- "age": 32,
319
- "id": "987",
320
- "zip": "00456", // Preserved leading zeros
321
- "registered": false // Case-insensitive boolean conversion
322
- }
323
- ]
324
- ```
93
+ | Implementation | Use Case | Learn More |
94
+ |---|---|---|
95
+ | **Sync API** | Simple, blocking operations | [Read SYNC.md](docs/SYNC.md) |
96
+ | **Async API** | Concurrent operations, large files | [Read ASYNC.md](docs/ASYNC.md) |
97
+ | **Browser API** | Client-side file parsing | [Read BROWSER.md](docs/BROWSER.md) |
325
98
 
326
- ### Encoding
327
- You can read and decode files with the following encoding:
328
- * utf8:
329
- ```js
330
- csvToJson.utf8Encoding()
331
- .getJsonFromCsv(fileInputName);
332
- ```
333
- * ucs2:
334
- ```js
335
- csvToJson.ucs2Encoding()
336
- .getJsonFromCsv(fileInputName);
337
- ```
338
- * utf16le:
339
- ```js
340
- csvToJson.utf16leEncoding()
341
- .getJsonFromCsv(fileInputName);
342
- ```
343
- * latin1:
344
- ```js
345
- csvToJson.latin1Encoding()
346
- .getJsonFromCsv(fileInputName);
347
- ```
348
- * ascii:
349
- ```js
350
- csvToJson.asciiEncoding()
351
- .getJsonFromCsv(fileInputName);
352
- ```
353
- * base64:
354
- ```js
355
- csvToJson.base64Encoding()
356
- .getJsonFromCsv(fileInputName);
357
- ```
358
- * hex:
359
- ```js
360
- csvToJson.hexEncoding()
361
- .getJsonFromCsv(fileInputName);
362
- ```
363
-
364
- ### Working with CSV strings directly
365
- If you have CSV content as a string (for example, from an API response or test data), you can parse it directly without writing to a file:
99
+ ## Common Tasks
366
100
 
101
+ ### Parse CSV String
367
102
  ```js
368
-
369
- // Parse CSV string to array of objects
370
- let csvString = 'firstName;lastName\nJohn;Doe\nJane;Smith';
371
- let jsonArray = csvToJson.csvStringToJson(csvString);
372
- // Output: [{"firstName":"John","lastName":"Doe"},{"firstName":"Jane","lastName":"Smith"}]
373
-
374
- // Parse CSV string to JSON string (validated)
375
- let jsonString = csvToJson.csvStringToJsonStringified(csvString);
376
- // Output: "[\n {\n \"firstName\": \"John\",\n \"lastName\": \"Doe\"\n },\n {\n \"firstName\": \"Jane\",\n \"lastName\": \"Smith\"\n }\n]"
103
+ const json = csvToJson.csvStringToJson('name,age\nAlice,30');
377
104
  ```
378
105
 
379
- Both methods support all configuration options through the chaining pattern:
380
-
106
+ ### Custom Delimiter
381
107
  ```js
382
- let jsonArray = csvToJson
383
- .fieldDelimiter(',')
384
- .formatValueByType()
385
- .csvStringToJson(csvString);
108
+ const json = csvToJson
109
+ .fieldDelimiter(';')
110
+ .getJsonFromCsv('input.csv');
386
111
  ```
387
112
 
388
- ### Sync API (TypeScript)
389
-
390
- TypeScript typings are available via the included `index.d.ts`. You can import the default converter or use named imports. Below are common patterns when using the synchronous API from TypeScript.
391
-
392
- ```ts
393
- // Named import (recommended when using ES modules)
394
- import converter, { /* or */ } from 'convert-csv-to-json';
395
- // Access the default converter
396
- const csvToJson = require('convert-csv-to-json');
397
-
398
- // Define a type for your CSV records
399
- interface Person {
400
- name: string;
401
- age: number;
402
- }
403
-
404
- // Parse CSV string synchronously and assert the returned type
405
- const csv = 'name,age\nAlice,30';
406
- const parsed = csvToJson.csvStringToJson(csv) as Person[];
407
-
408
- // Chain configuration and call sync methods
409
- const result = csvToJson
410
- .fieldDelimiter(',')
113
+ ### Format Values
114
+ ```js
115
+ const json = csvToJson
411
116
  .formatValueByType()
412
- .csvStringToJson('name,age\nBob,25') as Person[];
117
+ .getJsonFromCsv('input.csv');
118
+ // Converts "30" → 30, "true" → true, etc.
413
119
  ```
414
120
 
415
- ## Browser API Usage
416
-
417
- The package exposes a `browser` helper that reuses the library's parsing logic but provides browser-friendly helpers for parsing CSV strings and `File`/`Blob` objects. The API mirrors the synchronous and asynchronous Node APIs and supports method chaining for configuration.
418
-
419
- ### Basic Browser Operations
420
-
121
+ ### Handle Quoted Fields
421
122
  ```js
422
- const convert = require('convert-csv-to-json');
423
-
424
- // Parse CSV string synchronously
425
- const arr = convert.browser
123
+ const json = csvToJson
426
124
  .supportQuotedField(true)
427
- .fieldDelimiter(',')
428
- .csvStringToJson('name,age\nAlice,30');
429
-
430
- // Parse CSV string asynchronously (returns Promise)
431
- const arrAsync = await convert.browser.csvStringToJsonAsync('name;age\nBob;25');
432
-
433
- // Get stringified JSON synchronously
434
- const jsonString = convert.browser.csvStringToJsonStringified('name;age\nEve;40');
125
+ .getJsonFromCsv('input.csv');
435
126
  ```
436
127
 
437
- ### Parsing File/Blob
438
-
439
- `parseFile(file, options)` reads a `File` or `Blob` and returns a Promise that resolves with the parsed array of objects.
440
-
128
+ ### Batch Process Files (Async)
441
129
  ```js
442
- // In a browser environment with an <input type="file">
443
- const file = document.querySelector('input[type=file]').files[0];
444
- convert.browser
445
- .fieldDelimiter(',')
446
- .formatValueByType()
447
- .parseFile(file)
448
- .then(json => console.log(json))
449
- .catch(err => console.error(err));
130
+ const files = ['file1.csv', 'file2.csv', 'file3.csv'];
131
+ const results = await Promise.all(
132
+ files.map(f => csvToJson.getJsonFromCsvAsync(f))
133
+ );
450
134
  ```
451
135
 
452
- `parseFile` accepts an optional `options` object with `encoding` (passed to `FileReader.readAsText`). If `FileReader` is not available, `parseFile` will reject.
453
-
454
- ### Browser API Notes
455
-
456
- - The `browser` API proxies the same configuration methods as the Node API and follows the same behavior for quoted fields, sub-array parsing, trimming, and value formatting.
457
- - `parseFile` depends on the browser `FileReader` API; calling it in Node.js will reject with an informative error.
458
-
459
- ### Browser API (TypeScript)
460
-
461
- TypeScript typings are provided via the included `index.d.ts`. You can import the default converter and access the `browser` helper, or import `browser` directly. Below are common usage patterns.
462
-
463
- ```ts
464
- // Named import (recommended for direct use)
465
- import { browser } from 'convert-csv-to-json';
136
+ ## Configuration Options
466
137
 
467
- // Or default import and access the browser helper
468
- import converter from 'convert-csv-to-json';
469
- const browserApi = converter.browser;
470
-
471
- // Define a type for your CSV records
472
- interface Person {
473
- name: string;
474
- age: number;
475
- }
476
-
477
- // Synchronous parse (assert the returned type)
478
- const csv = 'name,age\nAlice,30';
479
- const parsed = browser.csvStringToJson(csv) as Person[];
480
-
481
- // Async parse
482
- const parsedAsync = await browser.csvStringToJsonAsync(csv) as Person[];
483
-
484
- // Parse a File in the browser
485
- const inputEl = document.querySelector('input[type=file]') as HTMLInputElement;
486
- const file = inputEl.files![0];
487
- const data = await browser.parseFile(file) as Person[];
488
- ```
138
+ All APIs (Sync, Async and Browser) support the same configuration methods:
489
139
 
490
- The `BrowserApi` interface in `index.d.ts` exposes typed method signatures for IDE autocompletion and compile-time checks.
140
+ - `fieldDelimiter(char)` - Set field delimiter (default: `,`)
141
+ - `formatValueByType()` - Auto-convert numbers, booleans
142
+ - `supportQuotedField(bool)` - Handle quoted fields with embedded delimiters
143
+ - `indexHeader(num)` - Specify header row (default: 0)
144
+ - `trimHeaderFieldWhiteSpace(bool)` - Remove spaces from headers
145
+ - `parseSubArray(delim, sep)` - Parse delimited arrays
146
+ - `mapRows(fn)` - Transform, filter, or enrich each row
147
+ - `utf8Encoding()`, `latin1Encoding()`, etc. - Set file encoding
491
148
 
492
- ## Async API Usage
149
+ ### Examples
493
150
 
494
- This library provides a Promise-based async API that's perfect for modern Node.js applications. For a detailed migration guide from sync to async API, see [MIGRATION.md](MIGRATION.md).
495
-
496
- ### Async API (TypeScript)
497
-
498
- The async API also has TypeScript typings. Typical usage in TypeScript looks like this:
499
-
500
- ```ts
501
- import csvToJson from 'convert-csv-to-json';
151
+ #### `fieldDelimiter(char)` - Set field delimiter (default: `,`)
152
+ ```js
153
+ // Semicolon-delimited
154
+ csvToJson.fieldDelimiter(';').getJsonFromCsv('data.csv');
502
155
 
503
- interface Person {
504
- name: string;
505
- age: number;
506
- }
156
+ // Tab-delimited
157
+ csvToJson.fieldDelimiter('\t').getJsonFromCsv('data.tsv');
507
158
 
508
- // Using async/await
509
- async function load(): Promise<Person[]> {
510
- const csv = 'name,age\nAlice,30';
511
- const parsed = await csvToJson.getJsonFromCsvAsync(csv, { raw: true }) as Person[];
512
- return parsed;
513
- }
514
-
515
- // Using the async helper that parses CSV strings
516
- const parsedDirect = await csvToJson.csvStringToJsonAsync('name;age\nBob;25') as Person[];
159
+ // Pipe-delimited
160
+ csvToJson.fieldDelimiter('|').getJsonFromCsv('data.psv');
517
161
  ```
518
162
 
519
-
520
- ### Basic Async Operations
521
-
522
- 1. Convert CSV file to JSON:
163
+ #### `formatValueByType()` - Auto-convert numbers, booleans
523
164
  ```js
524
- const csvToJson = require('convert-csv-to-json');
525
-
526
- // Using Promises
527
- csvToJson.getJsonFromCsvAsync('input.csv')
528
- .then(json => console.log(json))
529
- .catch(err => console.error('Error:', err));
530
-
531
- // Using async/await
532
- async function convertCsv() {
533
- try {
534
- const json = await csvToJson.getJsonFromCsvAsync('input.csv');
535
- console.log(json);
536
- } catch (err) {
537
- console.error('Error:', err);
538
- }
539
- }
165
+ // Input: name,age,active
166
+ // John,30,true
167
+ csvToJson.formatValueByType().getJsonFromCsv('data.csv');
168
+ // Output: { name: 'John', age: 30, active: true }
540
169
  ```
541
170
 
542
- 2. Generate JSON file from CSV:
171
+ #### `supportQuotedField(bool)` - Handle quoted fields with embedded delimiters
543
172
  ```js
544
- // Using async/await with chain configuration
545
- async function convertAndSave() {
546
- await csvToJson
547
- .fieldDelimiter(',')
548
- .formatValueByType()
549
- .generateJsonFileFromCsvAsync('input.csv', 'output.json');
550
- }
173
+ // Input: name,description
174
+ // "Smith, John","He said ""Hello"""
175
+ csvToJson.supportQuotedField(true).getJsonFromCsv('data.csv');
176
+ // Output: { name: 'Smith, John', description: 'He said "Hello"' }
551
177
  ```
552
178
 
553
- ### Working with Raw CSV Data
554
-
555
- Process CSV data from memory or network sources:
556
-
179
+ #### `indexHeader(num)` - Specify header row (default: 0)
557
180
  ```js
558
- // Example: Processing CSV from an API
559
- async function processCsvFromApi() {
560
- const response = await fetch('https://api.example.com/data.csv');
561
- const csvText = await response.text();
562
-
563
- const json = await csvToJson
564
- .formatValueByType()
565
- .getJsonFromCsvAsync(csvText, { raw: true });
566
-
567
- return json;
568
- }
181
+ // If headers are in row 2 (3rd line):
182
+ csvToJson.indexHeader(2).getJsonFromCsv('data.csv');
569
183
  ```
570
184
 
571
- ### Processing Large Files
572
-
573
- For large files, use streaming to manage memory efficiently:
574
-
185
+ #### `trimHeaderFieldWhiteSpace(bool)` - Remove spaces from headers
575
186
  ```js
576
- const { createReadStream } = require('fs');
577
- const { createInterface } = require('readline');
578
-
579
- async function* processLargeFile(filePath) {
580
- const fileStream = createReadStream(filePath);
581
- const rl = createInterface({
582
- input: fileStream,
583
- crlfDelay: Infinity
584
- });
585
-
586
- for await (const line of rl) {
587
- yield await csvToJson.getJsonFromCsvAsync(line, { raw: true });
588
- }
589
- }
590
-
591
- // Usage
592
- async function processData() {
593
- for await (const record of processLargeFile('large.csv')) {
594
- await saveToDatabase(record);
595
- }
596
- }
187
+ // Input: " First Name ", " Last Name "
188
+ csvToJson.trimHeaderFieldWhiteSpace(true).getJsonFromCsv('data.csv');
189
+ // Output: { FirstName: 'John', LastName: 'Doe' }
597
190
  ```
598
191
 
599
- ### Error Handling and Retries
600
-
601
- Implement robust error handling with retries:
602
-
192
+ #### `parseSubArray(delim, sep)` - Parse delimited arrays
603
193
  ```js
604
- async function processWithRetry(filePath, maxRetries = 3) {
605
- for (let i = 0; i < maxRetries; i++) {
606
- try {
607
- const json = await csvToJson
608
- .formatValueByType()
609
- .getJsonFromCsvAsync(filePath);
610
-
611
- return json;
612
- } catch (err) {
613
- if (i === maxRetries - 1) throw err;
614
- // Exponential backoff
615
- await new Promise(resolve =>
616
- setTimeout(resolve, Math.pow(2, i) * 1000)
617
- );
618
- }
619
- }
620
- }
194
+ // Input: name,tags
195
+ // John,*javascript,nodejs,typescript*
196
+ csvToJson.parseSubArray('*', ',').getJsonFromCsv('data.csv');
197
+ // Output: { name: 'John', tags: ['javascript', 'nodejs', 'typescript'] }
621
198
  ```
622
199
 
623
- ### Batch Processing
624
-
625
- Process multiple files efficiently:
200
+ #### `mapRows(fn)` - Transform, filter, or enrich each row
626
201
 
627
202
  ```js
628
- async function batchProcess(files, batchSize = 3) {
629
- const results = new Map();
630
-
631
- for (let i = 0; i < files.length; i += batchSize) {
632
- const batch = files.slice(i, i + batchSize);
633
- const processed = await Promise.all(
634
- batch.map(async file => {
635
- const json = await csvToJson.getJsonFromCsvAsync(file);
636
- return [file, json];
637
- })
638
- );
639
-
640
- processed.forEach(([file, json]) => results.set(file, json));
641
- }
642
-
643
- return results;
644
- }
645
-
646
- // Usage
647
- const files = ['data1.csv', 'data2.csv', 'data3.csv', 'data4.csv'];
648
- const results = await batchProcess(files, 2);
203
+ // Filter out rows that don't match a condition
204
+ const result = csvToJson
205
+ .fieldDelimiter(',')
206
+ .mapRows((row) => {
207
+ // Only keep rows where age >= 30
208
+ if (parseInt(row.age) >= 30) {
209
+ return row;
210
+ }
211
+ return null; // Filters out this row
212
+ })
213
+ .getJsonFromCsv('input.csv');
649
214
  ```
650
215
 
651
- ## Chaining Pattern
652
-
653
- The exposed API is implemented with the [Method Chaining Pattern](https://en.wikipedia.org/wiki/Method_chaining), which means that multiple methods can be chained. This pattern works with both synchronous and asynchronous methods:
654
-
655
- ### Synchronous Chaining
216
+ See [mapRows Feature - Usage Guide](docs/MAPROWS.md).
656
217
 
218
+ #### `utf8Encoding()`, `latin1Encoding()`, etc. - Set file encoding
657
219
  ```js
658
- const csvToJson = require('convert-csv-to-json');
659
-
660
- // Chain configuration methods with sync operation
661
- const json = csvToJson
662
- .fieldDelimiter(',')
663
- .formatValueByType()
664
- .parseSubArray("*", ',')
665
- .supportQuotedField(true)
666
- .getJsonFromCsv('myInputFile.csv');
220
+ // UTF-8 encoding
221
+ csvToJson.utf8Encoding().getJsonFromCsv('data.csv');
667
222
 
668
- // Chain with file generation
669
- csvToJson
670
- .fieldDelimiter(';')
671
- .utf8Encoding()
672
- .formatValueByType()
673
- .generateJsonFileFromCsv('input.csv', 'output.json');
223
+ // Latin-1 encoding
224
+ csvToJson.latin1Encoding().getJsonFromCsv('data.csv');
674
225
 
675
- // Chain with string parsing
676
- const jsonArray = csvToJson
677
- .fieldDelimiter(',')
678
- .trimHeaderFieldWhiteSpace(true)
679
- .csvStringToJson('name,age\nJohn,30\nJane,25');
226
+ // Custom encoding
227
+ csvToJson.customEncoding('ucs2').getJsonFromCsv('data.csv');
680
228
  ```
681
229
 
682
- ### Asynchronous Chaining
230
+ See [SYNC.md](docs/SYNC.md), [ASYNC.md](docs/ASYNC.md) or [BROWSER.md](docs/BROWSER.md) for complete configuration details.
231
+
232
+ ## Example: Complete Workflow
683
233
 
684
234
  ```js
685
235
  const csvToJson = require('convert-csv-to-json');
686
236
 
687
- // Using async/await
688
237
  async function processCSV() {
689
- // Chain configuration methods with async operation
690
- const json = await csvToJson
691
- .fieldDelimiter(',')
692
- .formatValueByType()
693
- .parseSubArray("*", ',')
694
- .supportQuotedField(true)
695
- .getJsonFromCsvAsync('myInputFile.csv');
696
-
697
- // Chain with async file generation
698
- await csvToJson
699
- .fieldDelimiter(';')
700
- .utf8Encoding()
701
- .formatValueByType()
702
- .generateJsonFileFromCsvAsync('input.csv', 'output.json');
703
- }
704
-
705
- // Using Promises
706
- csvToJson
238
+ const data = await csvToJson
707
239
  .fieldDelimiter(',')
708
240
  .formatValueByType()
709
- .getJsonFromCsvAsync('input.csv')
710
- .then(json => console.log(json))
711
- .catch(err => console.error('Error:', err));
712
- ```
713
-
714
- All configuration methods can be chained in any order before calling the final operation method (like `getJsonFromCsv`, `getJsonFromCsvAsync`, etc.). The configuration will be applied in the order it is chained.
715
-
716
- ## Common Use Cases
717
-
718
- Here are some common use cases and how to implement them:
719
-
720
- ### 1. Processing CSV from HTTP Response
721
- ```js
722
- const https = require('https');
723
-
724
- async function processRemoteCsv(url) {
725
- const csvData = await new Promise((resolve, reject) => {
726
- https.get(url, (res) => {
727
- let data = '';
728
- res.on('data', chunk => data += chunk);
729
- res.on('end', () => resolve(data));
730
- res.on('error', reject);
731
- });
732
- });
733
-
734
- return csvToJson.getJsonFromCsvAsync(csvData, { raw: true });
735
- }
736
- ```
737
-
738
- ### 2. Batch Processing Multiple Files
739
- ```js
740
- async function batchProcess(files) {
741
- const results = new Map();
742
-
743
- // Process in chunks of 3 files at a time
744
- for (let i = 0; i < files.length; i += 3) {
745
- const chunk = files.slice(i, i + 3);
746
- const processed = await Promise.all(
747
- chunk.map(async file => {
748
- const json = await csvToJson.getJsonFromCsvAsync(file);
749
- return [file, json];
750
- })
751
- );
752
-
753
- processed.forEach(([file, json]) => results.set(file, json));
754
- }
241
+ .supportQuotedField(true)
242
+ .getJsonFromCsvAsync('data.csv');
755
243
 
756
- return results;
244
+ console.log(`Parsed ${data.length} records`);
245
+ return data;
757
246
  }
758
247
  ```
759
248
 
760
- ### 3. Data Transformation Pipeline
761
- ```js
762
- async function transformData(csvFile) {
763
- // Step 1: Parse CSV
764
- const json = await csvToJson
765
- .formatValueByType()
766
- .getJsonFromCsvAsync(csvFile);
767
-
768
- // Step 2: Transform data
769
- const transformed = json.map(record => ({
770
- id: record.id,
771
- fullName: `${record.firstName} ${record.lastName}`,
772
- age: Number(record.age),
773
- isAdult: Number(record.age) >= 18,
774
- email: record.email.toLowerCase()
775
- }));
776
-
777
- // Step 3: Filter invalid records
778
- return transformed.filter(record =>
779
- record.id &&
780
- record.fullName.length > 0 &&
781
- !isNaN(record.age)
782
- );
783
- }
784
- ```
249
+ ## Migration Guides
785
250
 
786
- ### 4. Error Recovery and Logging
787
- ```js
788
- async function processWithLogging(file) {
789
- const logger = {
790
- info: (msg) => console.log(`[INFO] ${msg}`),
791
- error: (msg, err) => console.error(`[ERROR] ${msg}`, err)
792
- };
793
-
794
- try {
795
- logger.info(`Starting processing ${file}`);
796
- const startTime = Date.now();
797
-
798
- const json = await csvToJson.getJsonFromCsvAsync(file);
799
-
800
- const duration = Date.now() - startTime;
801
- logger.info(`Processed ${file} in ${duration}ms`);
802
-
803
- return json;
804
- } catch (err) {
805
- logger.error(`Failed to process ${file}`, err);
806
- throw err;
807
- }
808
- }
809
- ```
251
+ - **RFC 4180 Breaking Changes** - See [RFC4180_MIGRATION_GUIDE.md](migration/RFC4180_MIGRATION_GUIDE.md)
252
+ - **Sync to Async Migration** - See [MIGRATION.md](migration/MIGRATION_TO_ASYNC.md)
253
+ - **Error Handling** - See [docs/ERROR_HANDLING.md](docs/ERROR_HANDLING.md) for comprehensive error documentation
810
254
 
811
- ## Troubleshooting
812
-
813
- Here are solutions to common issues you might encounter:
814
-
815
- ### Memory Issues with Large Files
816
-
817
- If you're processing large CSV files and encountering memory issues:
818
-
819
- ```js
820
- // Instead of loading the entire file
821
- const json = await csvToJson.getJsonFromCsvAsync('large.csv'); // ❌
822
-
823
- // Use streaming with async iteration
824
- for await (const record of processLargeCsv('large.csv')) { // ✅
825
- // Process one record at a time
826
- await processRecord(record);
827
- }
828
- ```
829
-
830
- ### Handling Different CSV Formats
255
+ ## Development
831
256
 
832
- 1. **Mixed Quote Types**:
833
- ```js
834
- csvToJson
835
- .supportQuotedField(true) // Enable quoted field support
836
- .getJsonFromCsvAsync(file);
257
+ Install dependencies:
258
+ ```bash
259
+ npm install
837
260
  ```
838
261
 
839
- 2. **Custom Delimiters**:
840
- ```js
841
- csvToJson
842
- .fieldDelimiter(';') // Change delimiter
843
- .getJsonFromCsvAsync(file);
262
+ Run tests:
263
+ ```bash
264
+ npm test
844
265
  ```
845
266
 
846
- 3. **UTF-8 with BOM**:
847
- ```js
848
- csvToJson
849
- .encoding('utf8') // Specify encoding
850
- .getJsonFromCsvAsync(file);
267
+ Debug tests:
268
+ ```bash
269
+ npm run test-debug
851
270
  ```
852
271
 
853
- ### Common Error Solutions
854
-
855
- 1. **ENOENT: no such file or directory**
856
- - Check if the file path is correct and absolute
857
- - Verify file permissions
858
- - Ensure the file exists in the specified location
272
+ ## CI/CD GitHub Action
859
273
 
860
- 2. **Invalid CSV Structure**
861
- - Verify CSV format matches expected structure
862
- - Check for missing or extra delimiters
863
- - Validate header row exists if expected
274
+ See [CI/CD GitHub Action](.github/workflows/ci-cd.yml).
864
275
 
865
- 3. **Memory Leaks**
866
- - Use streaming for large files
867
- - Process files in smaller chunks
868
- - Implement proper cleanup in try/finally blocks
276
+ ### Release
277
+ When pushing to the `master` branch:
278
+ - Include `[MAJOR]` in commit message for major release (e.g., v1.0.0 → v2.0.0)
279
+ - Include `[PATCH]` in commit message for patch release (e.g., v1.0.0 → v1.0.1)
280
+ - Minor release is applied by default (e.g., v1.0.0 → v1.1.0)
869
281
 
870
- 4. **Encoding Issues**
871
- - Specify correct encoding using .encoding()
872
- - Check for BOM markers
873
- - Verify source file encoding
874
-
875
- ### Performance Optimization
876
-
877
- 1. **Parallel Processing**:
878
- ```js
879
- // Instead of sequential processing
880
- for (const file of files) {
881
- await process(file); // ❌
882
- }
883
-
884
- // Use parallel processing with limits
885
- async function processWithLimit(files, limit = 3) {
886
- const results = [];
887
- for (let i = 0; i < files.length; i += limit) {
888
- const chunk = files.slice(i, i + limit);
889
- const chunkResults = await Promise.all(
890
- chunk.map(file => csvToJson.getJsonFromCsvAsync(file))
891
- );
892
- results.push(...chunkResults);
893
- }
894
- return results;
895
- } // ✅
896
- ```
282
+ ## License
897
283
 
898
- 2. **Memory Usage**:
899
- ```js
900
- // Clear references when done
901
- async function processWithCleanup(file) {
902
- let json;
903
- try {
904
- json = await csvToJson.getJsonFromCsvAsync(file);
905
- return await processData(json);
906
- } finally {
907
- json = null; // Clear reference
908
- }
909
- }
910
- ```
284
+ CSVtoJSON is licensed under the MIT [License](LICENSE).
911
285
 
912
- ### TypeScript Support
286
+ ---
913
287
 
914
- If you're using TypeScript and encounter type issues:
288
+ ## Support
915
289
 
916
- ```typescript
917
- // Define custom types for your CSV structure
918
- interface MyCsvRecord {
919
- id: number;
920
- name: string;
921
- age?: number;
922
- }
290
+ Found a bug or need a feature? Open an issue on [GitHub](https://github.com/iuccio/csvToJson/issues).
923
291
 
924
- // Use type assertion
925
- const json = await csvToJson.getJsonFromCsvAsync<MyCsvRecord>('data.csv');
926
- ```
927
-
928
- ## Development
929
- * Download all csvToJson dependencies:
930
- ~~~
931
- npm install
932
- ~~~
933
- * Run Tests
934
- ~~~
935
- npm test
936
- ~~~
937
- * Debug Tests
938
- ~~~
939
- npm run test-debug
940
- ~~~
941
-
942
- ## CI CD github action
943
-
944
- This repository uses the [GitHub Action iuccio/npm-semantic-publish-action@latest](https://github.com/marketplace/actions/npm-semver-publish) to publish the npm packeges.
945
- Pushing on the master branch, depending on the git message, an new version will always be released.
946
- If the commit message contains the keyword:
947
- * **[MAJOR]**: new major relase, e.g. v1.0.0 -> v2.0.0
948
- * **[PATCH]**: new patch relase, e.g. v1.0.0 -> v1.0.1
949
- * without any of the above keywords a new minor relase will be applied, e.g. v1.0.0 -> v1.1.0
292
+ Follow [me](https://github.com/iuccio) and consider starring the project to show your support ⭐
950
293
 
294
+ ### Buy Me a Coffee
951
295
 
952
- ## License
953
- CSVtoJSON is licensed under the MIT [License](LICENSE).
296
+ If you find this project helpful and would like to support its development:
954
297
 
955
- ## Buy me a Coffee
956
- Just if you want to support this repository:
957
- * **BTC** tip address:
958
- 37vdjQhbaR7k7XzhMKWzMcnqUxfw1njBNk
298
+ **BTC**: `37vdjQhbaR7k7XzhMKWzMcnqUxfw1njBNk`