readline-pager 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -33
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,19 +1,19 @@
|
|
|
1
|
-
# readline-pager
|
|
1
|
+
# 📄 readline-pager
|
|
2
2
|
|
|
3
|
-
Memory-efficient, paginated file reader for Node.js with async iteration, prefetching, backward reading and optional worker support.
|
|
3
|
+
Memory-efficient, paginated file reader for Node.js with async iteration, prefetching, backward reading, and optional worker support.
|
|
4
4
|
|
|
5
5
|
`readline-pager` reads large text files page-by-page without loading the entire file into memory.
|
|
6
6
|
|
|
7
7
|
- ✅ Zero dependencies
|
|
8
8
|
- ✅ Async iterator support (`for await...of`)
|
|
9
|
-
- ✅ Forward & backward reading (
|
|
9
|
+
- ✅ Forward & backward reading (EOF → BOF)
|
|
10
10
|
- ✅ Optional worker thread mode (forward only)
|
|
11
|
-
- ✅ Up to ~3× faster than
|
|
12
|
-
- ✅ ~97% test coverage &
|
|
11
|
+
- ✅ Up to ~3× faster than Node.js `readline`
|
|
12
|
+
- ✅ ~97% test coverage & fully typed (TypeScript)
|
|
13
13
|
|
|
14
14
|
---
|
|
15
15
|
|
|
16
|
-
## Installation
|
|
16
|
+
## 📦 Installation
|
|
17
17
|
|
|
18
18
|
```bash
|
|
19
19
|
npm install readline-pager
|
|
@@ -21,7 +21,7 @@ npm install readline-pager
|
|
|
21
21
|
|
|
22
22
|
---
|
|
23
23
|
|
|
24
|
-
## Quick Start
|
|
24
|
+
## 🚀 Quick Start
|
|
25
25
|
|
|
26
26
|
```ts
|
|
27
27
|
import { createPager } from "readline-pager";
|
|
@@ -35,7 +35,7 @@ for await (const page of pager) {
|
|
|
35
35
|
|
|
36
36
|
---
|
|
37
37
|
|
|
38
|
-
## Manual iteration (recommended for
|
|
38
|
+
## ⚡ Manual iteration (recommended for maximum throughput)
|
|
39
39
|
|
|
40
40
|
```ts
|
|
41
41
|
const pager = createPager("./bigfile.txt");
|
|
@@ -56,7 +56,7 @@ while ((page = await pager.next()) !== null) {
|
|
|
56
56
|
|
|
57
57
|
---
|
|
58
58
|
|
|
59
|
-
## Options
|
|
59
|
+
## ⚙️ Options
|
|
60
60
|
|
|
61
61
|
```ts
|
|
62
62
|
createPager(filepath, {
|
|
@@ -70,56 +70,56 @@ createPager(filepath, {
|
|
|
70
70
|
|
|
71
71
|
- `pageSize` — number of lines per page.
|
|
72
72
|
- `delimiter` — line separator.
|
|
73
|
-
- `prefetch` — max pages buffered internally. Higher
|
|
73
|
+
- `prefetch` — max number of pages buffered internally. Higher values increase throughput but use more memory.
|
|
74
74
|
- `backward` — read file from end → start (not supported with `useWorker`).
|
|
75
|
-
- `useWorker` —
|
|
75
|
+
- `useWorker` — offload parsing to a worker thread (forward mode only).
|
|
76
76
|
|
|
77
77
|
---
|
|
78
78
|
|
|
79
|
-
## API
|
|
79
|
+
## 📚 API
|
|
80
80
|
|
|
81
81
|
### `pager.next(): Promise<string[] | null>`
|
|
82
82
|
|
|
83
83
|
Returns the next page or `null` when finished. Empty lines are preserved.
|
|
84
84
|
|
|
85
|
-
**Note:** Unlike Node.js `readline`, which
|
|
85
|
+
**Note:** Unlike Node.js `readline`, which may skip empty files or leading empty lines, `readline-pager` always returns all lines.
|
|
86
86
|
|
|
87
87
|
- A completely empty file (`0` bytes) produces `[""]` on the first read.
|
|
88
|
-
- A file with multiple empty lines returns each line as an empty string (e.g., `["", ""]` for two empty lines). Node.js `readline`
|
|
88
|
+
- A file with multiple empty lines returns each line as an empty string (e.g., `["", ""]` for two empty lines). Node.js `readline` may emit fewer or no `line` events in these cases.
|
|
89
89
|
|
|
90
90
|
✅ Key points:
|
|
91
91
|
|
|
92
92
|
- A 0-byte file → `[""]`
|
|
93
93
|
- Consecutive `\n\n` → `["", ""]`
|
|
94
|
-
- Node.js `readline`
|
|
94
|
+
- Node.js `readline` may skip initial empty line(s) and emit nothing for empty files.
|
|
95
95
|
|
|
96
96
|
### `pager.close(): void`
|
|
97
97
|
|
|
98
|
-
Stops reading and releases resources immediately. Safe to call any time.
|
|
98
|
+
Stops reading and releases resources immediately. Safe to call at any time.
|
|
99
99
|
|
|
100
100
|
### Properties
|
|
101
101
|
|
|
102
102
|
```ts
|
|
103
103
|
pager.lineCount; // total lines emitted so far
|
|
104
104
|
pager.firstLine; // first line emitted (available after first read)
|
|
105
|
-
pager.lastLine; // last line emitted (updated
|
|
105
|
+
pager.lastLine; // last line emitted (updated per page)
|
|
106
106
|
```
|
|
107
107
|
|
|
108
108
|
---
|
|
109
109
|
|
|
110
|
-
## Benchmark
|
|
110
|
+
## 📊 Benchmark
|
|
111
111
|
|
|
112
112
|
Run the included benchmark:
|
|
113
113
|
|
|
114
114
|
```bash
|
|
115
115
|
# default run
|
|
116
|
-
node test/
|
|
116
|
+
node test/_benchmark.ts
|
|
117
117
|
|
|
118
|
-
#
|
|
119
|
-
node test/
|
|
118
|
+
# customize
|
|
119
|
+
node test/_benchmark.ts --lines=20000 --page-size=500 --prefetch=4 --backward
|
|
120
120
|
```
|
|
121
121
|
|
|
122
|
-
Benchmarks were executed on a high-end
|
|
122
|
+
Benchmarks were executed on a high-end Linux machine (SSD + fast CPU) using generated files.
|
|
123
123
|
|
|
124
124
|
### Summary (averages)
|
|
125
125
|
|
|
@@ -135,38 +135,41 @@ Benchmarks were executed on a high-end consumer Linux machine (SSD + fast CPU) u
|
|
|
135
135
|
**Key takeaways**
|
|
136
136
|
|
|
137
137
|
- `readline-pager` is consistently **~2.3×–2.8× faster** than Node.js `readline`.
|
|
138
|
-
- Relative
|
|
138
|
+
- Relative performance gains increase with file size.
|
|
139
139
|
- Sustained throughput exceeds **1.2 GB/s** on large files (machine-dependent).
|
|
140
140
|
|
|
141
141
|
---
|
|
142
142
|
|
|
143
|
-
## Iteration
|
|
143
|
+
## 🧠 Iteration Performance Notes
|
|
144
144
|
|
|
145
|
-
- **Fastest**: manual
|
|
145
|
+
- **Fastest**: manual
|
|
146
|
+
`while ((page = await pager.next()) !== null) { ... }`
|
|
146
147
|
Avoids async-iterator protocol overhead and microtask churn.
|
|
147
148
|
|
|
148
|
-
- **More ergonomic**:
|
|
149
|
-
|
|
149
|
+
- **More ergonomic**:
|
|
150
|
+
`for await (const page of pager) { ... }`
|
|
151
|
+
Cleaner, but slightly slower in hot paths.
|
|
150
152
|
|
|
151
|
-
**Recommendation:** use the explicit `next()` loop for benchmarks and performance-critical
|
|
153
|
+
**Recommendation:** use the explicit `next()` loop for benchmarks and performance-critical workloads; use `for await...of` for clarity elsewhere.
|
|
152
154
|
|
|
153
155
|
---
|
|
154
156
|
|
|
155
|
-
## Development & Contributing
|
|
157
|
+
## 🛠 Development & Contributing
|
|
156
158
|
|
|
157
159
|
- Minimum supported Node.js: **18.12+** (LTS).
|
|
158
|
-
- Development/test environment
|
|
159
|
-
|
|
160
|
+
- Development/test environment: **Node v25.6.1**, TypeScript `~5.9.x`.
|
|
161
|
+
|
|
162
|
+
Run tests:
|
|
160
163
|
|
|
161
164
|
```bash
|
|
162
165
|
npm ci
|
|
163
166
|
npm test
|
|
164
167
|
```
|
|
165
168
|
|
|
166
|
-
|
|
169
|
+
Contributions are welcome — feel free to open an issue or PR.
|
|
167
170
|
|
|
168
171
|
---
|
|
169
172
|
|
|
170
|
-
## License
|
|
173
|
+
## 📜 License
|
|
171
174
|
|
|
172
175
|
MIT — © Morteza Jamshidi
|