js-dev-tool 1.2.11 → 1.2.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +286 -120
- package/extras/cc-map.d.ts +4 -4
- package/extras/cc-map.js +3 -3
- package/extras/json-minify.d.ts +29 -0
- package/extras/json-minify.js +287 -0
- package/extras/jsonl.d.ts +19 -2
- package/extras/jsonl.js +5 -230
- package/package.json +5 -3
- package/tools.js +7 -7
package/README.md
CHANGED
|
@@ -1,164 +1,330 @@
|
|
|
1
1
|
# js-dev-tool
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
`js-dev-tool` is a mixed package of `jstool` CLI commands and small Node.js helpers for JavaScript / TypeScript project maintenance.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
The published modules are CommonJS. The examples below use `require(...)` so the runtime behavior matches the package as published.
|
|
6
6
|
|
|
7
|
-
##
|
|
7
|
+
## Install
|
|
8
8
|
|
|
9
|
-
|
|
9
|
+
```bash
|
|
10
|
+
npm i js-dev-tool
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
## CLI Usage
|
|
14
|
+
|
|
15
|
+
`jstool` is the package binary and points to `tools.js`.
|
|
10
16
|
|
|
11
17
|
```bash
|
|
12
|
-
|
|
18
|
+
jstool -cmd <command_name> [options]
|
|
13
19
|
node tools.js -cmd <command_name> [options]
|
|
20
|
+
node tools.js -help <command_name>
|
|
21
|
+
```
|
|
14
22
|
|
|
15
|
-
|
|
16
|
-
|
|
23
|
+
## Published Package Layout
|
|
24
|
+
|
|
25
|
+
This section reflects the current `package.json#exports` and `package.json#files`.
|
|
26
|
+
|
|
27
|
+
### Stable Entry Points (`exports`)
|
|
28
|
+
|
|
29
|
+
| Path | Purpose |
|
|
30
|
+
| --- | --- |
|
|
31
|
+
| `js-dev-tool` | Root namespace object with `{ progress, common, utils }` |
|
|
32
|
+
| `js-dev-tool/utils` | File, JSON, CLI, TSV/CSV, and small utility helpers |
|
|
33
|
+
| `js-dev-tool/common` | File-system and terminal rendering primitives |
|
|
34
|
+
| `js-dev-tool/progress` | Progress renderer, spinner helpers, webpack / browserify progress hooks |
|
|
35
|
+
| `js-dev-tool/progress/progress-extras` | Environment and webpack-version helpers |
|
|
36
|
+
| `js-dev-tool/extras/algorithms` | Binary-search helper |
|
|
37
|
+
| `js-dev-tool/extras/cc-map` | Character-code map |
|
|
38
|
+
| `js-dev-tool/extras/jsonl` | JSONL readers, JSONL-to-array/map helpers, and `jsonMinify` |
|
|
39
|
+
| `js-dev-tool/extras/json-minify` | `jsonMinify` |
|
|
40
|
+
| `js-dev-tool/extras/progress-light` | Small fixed-frame spinner |
|
|
41
|
+
| `js-dev-tool/extras/tiny-progress` | Random-frame spinner wrapper |
|
|
42
|
+
| `js-dev-tool/basic-types` | Types-only reference entry |
|
|
43
|
+
|
|
44
|
+
Notes:
|
|
45
|
+
|
|
46
|
+
- `js-dev-tool/extras/list-deps-of` is bundled and works at runtime, but it does not ship a `.d.ts`. If you want types, use `listDependenciesOf` from `js-dev-tool/utils`.
|
|
47
|
+
- `package.json` exposes `./progress/*`, but the currently documented typed JS subpath is `js-dev-tool/progress/progress-extras`. Other bundled files under `progress/` should be treated as internal support assets unless you have verified the exact runtime file.
|
|
48
|
+
|
|
49
|
+
### Additional Bundled Files (`files`)
|
|
50
|
+
|
|
51
|
+
The publish list is broader than the stable import surface above.
|
|
52
|
+
|
|
53
|
+
- Root files: `index.js`, `index.d.ts`, `utils.js`, `utils.d.ts`, `tools.js`, `tools.d.ts`, `basic-types.d.ts`, `tsconfig.json`
|
|
54
|
+
- Bundled directories: `common/`, `extras/`, `lib/`, `progress/`, `scripts/`, `tool-lib/`
|
|
55
|
+
|
|
56
|
+
## Root Namespace
|
|
57
|
+
|
|
58
|
+
```js
|
|
59
|
+
const { utils, common, progress } = require("js-dev-tool");
|
|
17
60
|
```
|
|
18
61
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
62
|
+
For clearer dependency boundaries, importing a direct subpath is usually better than pulling from the root namespace.
|
|
63
|
+
|
|
64
|
+
## `utils`
|
|
65
|
+
|
|
66
|
+
`js-dev-tool/utils` is the main grab-bag module. A few useful exports:
|
|
67
|
+
|
|
68
|
+
- `extractVersion(versionString?)`
|
|
69
|
+
- `removeJsonComments(source)`
|
|
70
|
+
- `readText(path)` / `writeText(content, path)`
|
|
71
|
+
- `readJson(path)`
|
|
72
|
+
- `copyText(content, message?)`
|
|
73
|
+
- `indexByCol(tsvSource, options)` and `parseDelimitedToIndex`
|
|
74
|
+
- `listDependenciesOf(packageName)` for Yarn v1 lockfiles
|
|
75
|
+
|
|
76
|
+
Example:
|
|
77
|
+
|
|
78
|
+
```js
|
|
79
|
+
const { indexByCol, listDependenciesOf } = require("js-dev-tool/utils");
|
|
80
|
+
|
|
81
|
+
const map = indexByCol("id\tname\n1\talpha\n2\tbeta\n", {
|
|
82
|
+
mapKeyCol: 0,
|
|
83
|
+
});
|
|
84
|
+
|
|
85
|
+
const deps = listDependenciesOf("webpack");
|
|
86
|
+
console.log(map["1"], deps.slice(0, 5));
|
|
22
87
|
```
|
|
23
88
|
|
|
24
|
-
##
|
|
89
|
+
## `common`
|
|
25
90
|
|
|
26
|
-
|
|
27
|
-
> You probably don’t need this — but if you do, it’s handy.
|
|
91
|
+
`js-dev-tool/common` contains the low-level pieces used by the progress helpers.
|
|
28
92
|
|
|
29
|
-
|
|
30
|
-
|
|
93
|
+
- `checkParentDirectory(dest)`
|
|
94
|
+
- `createLogStreamAndResolvePath(logPath)`
|
|
95
|
+
- `renderLine(msg?, row?)`
|
|
96
|
+
- `cursor(enabled, output?)`
|
|
97
|
+
|
|
98
|
+
## `progress`
|
|
99
|
+
|
|
100
|
+
`js-dev-tool/progress` is the main progress API.
|
|
101
|
+
|
|
102
|
+
- `createProgress(timeSpanMS, frames)`
|
|
103
|
+
- `createProgressSync(frames, formatOpt?)`
|
|
104
|
+
- `createProgressObject(frames, formatOpt, messageEmitter)`
|
|
105
|
+
- `createWebpackProgressPluginHandler(logFilePath?, disableRenderLine?)`
|
|
106
|
+
- `createBrowserifyFileEventLogger(logFilePath)`
|
|
107
|
+
|
|
108
|
+
`js-dev-tool/progress/progress-extras` adds:
|
|
109
|
+
|
|
110
|
+
- `checkENV()`
|
|
111
|
+
- `wppHandlerV4`
|
|
112
|
+
- `wppHandlerV5`
|
|
113
|
+
- `isWebpackV5later()`
|
|
114
|
+
|
|
115
|
+
## Extras Modules
|
|
116
|
+
|
|
117
|
+
The `extras/` directory is where the smaller standalone modules live. Some are typed and ready to document as public helpers; some are runtime-only and better treated as advanced or legacy entry points.
|
|
118
|
+
|
|
119
|
+
### `js-dev-tool/extras/algorithms`
|
|
120
|
+
|
|
121
|
+
Exports:
|
|
122
|
+
|
|
123
|
+
- `bnSearch(src, value, comparator)`
|
|
124
|
+
|
|
125
|
+
Use this when you already have a sorted array and want a simple binary search helper.
|
|
126
|
+
|
|
127
|
+
```js
|
|
128
|
+
const { bnSearch } = require("js-dev-tool/extras/algorithms");
|
|
129
|
+
|
|
130
|
+
const values = [1, 4, 7, 9];
|
|
131
|
+
const index = bnSearch(values, 7, (left, right) => left - right);
|
|
132
|
+
|
|
133
|
+
console.log(index); // 2
|
|
31
134
|
```
|
|
32
135
|
|
|
33
|
-
|
|
136
|
+
### `js-dev-tool/extras/cc-map`
|
|
34
137
|
|
|
35
|
-
|
|
138
|
+
Exports a character-code map object. It is most useful when you want named constants for ASCII / control-character comparisons.
|
|
36
139
|
|
|
37
|
-
|
|
38
|
-
|
|
140
|
+
```js
|
|
141
|
+
const CC_MAP = require("js-dev-tool/extras/cc-map");
|
|
39
142
|
|
|
40
|
-
|
|
143
|
+
console.log(CC_MAP.LF); // 10
|
|
144
|
+
console.log(CC_MAP.DOUBLE_QUOTE); // 34
|
|
145
|
+
```
|
|
41
146
|
|
|
42
|
-
|
|
147
|
+
### `js-dev-tool/extras/jsonl`
|
|
148
|
+
|
|
149
|
+
Exports:
|
|
150
|
+
|
|
151
|
+
- `resolveJsonlPath(fileName, options?)`
|
|
152
|
+
- `readJsonlLines(fileName, onLine, options?)`
|
|
153
|
+
- `readJsonl(fileName, onRow, options?)`
|
|
154
|
+
- `readJsonlArray(fileName, options?)`
|
|
155
|
+
- `readJsonlMapByKey(fileName, options?)`
|
|
156
|
+
- `fastGetIntFieldCheap(line, key)`
|
|
157
|
+
- `jsonMinify(source)`
|
|
158
|
+
|
|
159
|
+
This is the richest `extras` entry and the main one worth documenting. It covers three related jobs:
|
|
160
|
+
|
|
161
|
+
- streaming JSONL line reads
|
|
162
|
+
- JSON parse + row transformation
|
|
163
|
+
- whitespace minification for JSON and JSONL
|
|
164
|
+
|
|
165
|
+
Key behaviors:
|
|
166
|
+
|
|
167
|
+
- `readJsonlLines` splits by `"\n"` and trims a trailing `"\r"` from each physical line
|
|
168
|
+
- empty lines are skipped by default
|
|
169
|
+
- `readJsonl` reports parse failures through `onParseError` when provided, otherwise it logs to `stderr`
|
|
170
|
+
- `readJsonlMapByKey` defaults to `"_key"` and lets later rows overwrite earlier rows
|
|
171
|
+
- `fastGetIntFieldCheap` is intentionally narrow: it is meant for top-level integer fields on hot paths
|
|
172
|
+
- `jsonMinify` accepts either a single JSON value or multiple top-level JSON values and returns minified JSONL in the multi-value case
|
|
173
|
+
|
|
174
|
+
Example:
|
|
175
|
+
|
|
176
|
+
```js
|
|
177
|
+
const {
|
|
178
|
+
readJsonlArray,
|
|
179
|
+
readJsonlMapByKey,
|
|
180
|
+
fastGetIntFieldCheap,
|
|
181
|
+
jsonMinify,
|
|
182
|
+
} = require("js-dev-tool/extras/jsonl");
|
|
183
|
+
|
|
184
|
+
async function main() {
|
|
185
|
+
const rows = await readJsonlArray("logs/app.jsonl", {
|
|
186
|
+
filter: (row) => row.level !== "debug",
|
|
187
|
+
map: (row) => ({
|
|
188
|
+
id: row.id,
|
|
189
|
+
level: row.level,
|
|
190
|
+
}),
|
|
191
|
+
});
|
|
192
|
+
|
|
193
|
+
const byId = await readJsonlMapByKey("logs/app.jsonl", {
|
|
194
|
+
keyField: "id",
|
|
195
|
+
});
|
|
196
|
+
|
|
197
|
+
console.log(rows.length, Object.keys(byId).length);
|
|
198
|
+
console.log(fastGetIntFieldCheap("{\"count\":42}", "count"));
|
|
199
|
+
console.log(jsonMinify("{ \"a\": 1 }\n{ \"a\": 2 }"));
|
|
200
|
+
}
|
|
201
|
+
|
|
202
|
+
main().catch(console.error);
|
|
203
|
+
```
|
|
43
204
|
|
|
44
|
-
|
|
205
|
+
### `js-dev-tool/extras/progress-light`
|
|
45
206
|
|
|
46
|
-
|
|
47
|
-
Records the size of webpack bundles or other files. The name stands for (r)ecord(W)ebpack(S)ize.
|
|
48
|
-
* **Usage:** `jstool -cmd rws -webpack lib/webpack.js -dest "./dev-extras/webpack-size.json"`
|
|
207
|
+
Exports:
|
|
49
208
|
|
|
50
|
-
|
|
209
|
+
- `create(fps?, messageEmitter?)`
|
|
51
210
|
|
|
52
|
-
|
|
53
|
-
Converts JavaScript files to browser-compatible modules. The name stands for (C)onvert (J)S to (B)rowser (M)odule.
|
|
54
|
-
* **Usage:** `jstool -cmd cjbm -basePath <source_dir> -targets "['file1.js', 'file2.js']"`
|
|
211
|
+
`progress-light` is the simpler spinner. It uses a fixed dot-style frame set and returns a progress object with:
|
|
55
212
|
|
|
56
|
-
|
|
213
|
+
- `run()`
|
|
214
|
+
- `stop()`
|
|
215
|
+
- `renderSync()`
|
|
216
|
+
- `setFPS(fps)`
|
|
217
|
+
- `updateOptions(newFrames?, newOpt?)`
|
|
218
|
+
- `deadline()`
|
|
219
|
+
- `newLine()`
|
|
220
|
+
- `isRunning()`
|
|
57
221
|
|
|
58
|
-
|
|
59
|
-
Toggles "comment tricks" in the code, useful for conditional code execution during development.
|
|
60
|
-
* **Usage:** `jstool -cmd cmtTrick -basePath <source_dir> -targets "['file1.js', 'file2.js']"`
|
|
222
|
+
Example:
|
|
61
223
|
|
|
62
|
-
|
|
224
|
+
```js
|
|
225
|
+
const { create } = require("js-dev-tool/extras/progress-light");
|
|
63
226
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
* **Usage:** `jstool -cmd replace [-after <replacement>] -regex "<regex>" [-targets "<path1>,<path2>" | <file1> <file2> ...]`
|
|
67
|
-
* **Note:** It's often better to pass target files as direct arguments instead of using the `-targets` option.
|
|
227
|
+
const progress = create(12, () => "building...");
|
|
228
|
+
progress.run();
|
|
68
229
|
|
|
69
|
-
|
|
230
|
+
setTimeout(() => {
|
|
231
|
+
progress.deadline();
|
|
232
|
+
progress.stop();
|
|
233
|
+
progress.newLine();
|
|
234
|
+
}, 800);
|
|
235
|
+
```
|
|
70
236
|
|
|
71
|
-
### `
|
|
72
|
-
Bumps the version number in `package.json` and optionally in other files.
|
|
73
|
-
* **Usage:** `jstool -cmd version [-major | -minor] [-pkgJsons "./package.json,../other/package.json"] [-extras "file.html"]`
|
|
74
|
-
* **Note:** Bumps the patch version by default. Use `-major` or `-minor` to bump those instead.
|
|
237
|
+
### `js-dev-tool/extras/tiny-progress`
|
|
75
238
|
|
|
76
|
-
|
|
239
|
+
Exports:
|
|
77
240
|
|
|
78
|
-
|
|
79
|
-
Minifies JavaScript files using Terser.
|
|
80
|
-
* **Usage:** `jstool -cmd minify -basePath <source_dir> [-test "<regex>"] [-suffix ".min"]`
|
|
81
|
-
* **Note:** `basePath` can be a comma-separated list. The default suffix is `.mini`.
|
|
241
|
+
- `create(fps?, messageEmitter?)`
|
|
82
242
|
|
|
83
|
-
|
|
243
|
+
`tiny-progress` is similar to `progress-light`, but it delegates to `js-dev-tool/progress` and uses a random spinner frame set from the bundled progress resources.
|
|
84
244
|
|
|
85
|
-
|
|
86
|
-
Removes C-style comments from files.
|
|
87
|
-
* **Usage:** `jstool -cmd rmc -basePath "./dist/cjs,./dist/other" -test "/\\.(js|d\\.ts)$/"`
|
|
88
|
-
* **Note:** Use the `-rmc4ts` flag for special handling of TypeScript comments (e.g., `/// <reference>`).
|
|
245
|
+
Use this when you want a little more visual variety and do not care about choosing the exact frame list yourself.
|
|
89
246
|
|
|
90
|
-
|
|
247
|
+
```js
|
|
248
|
+
const { create } = require("js-dev-tool/extras/tiny-progress");
|
|
91
249
|
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
* **Usage:** `jstool -cmd zip [-comment "the comment"] file1.js file2.js ...`
|
|
250
|
+
const progress = create(20, () => "waiting for tasks...");
|
|
251
|
+
progress.run();
|
|
95
252
|
|
|
96
|
-
|
|
253
|
+
setTimeout(() => {
|
|
254
|
+
progress.stop();
|
|
255
|
+
progress.newLine();
|
|
256
|
+
}, 800);
|
|
257
|
+
```
|
|
97
258
|
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
cmtTrick help: Do comment trick toggle
|
|
122
|
-
It recognizes three markers: `ctt | comment-toggle-trick | https://coderwall`
|
|
123
|
-
jstool -cmd cmtTrick[:clean] (-base <source dir> | -bases "<source dir>,<source dir>,...") [-test re/\\.js$/]
|
|
124
|
-
note:
|
|
125
|
-
:clean - remove comment token etc leaving the currently enabled code
|
|
126
|
-
base (or root) - scan single source folder
|
|
127
|
-
bases - must be array type arg, "['<path>', '<path>',...]" or "<path>,<path>,..."
|
|
128
|
-
test - terget extension, default is /\\.js$/
|
|
129
|
-
|
|
130
|
-
replace help: jstool -cmd replace [-after <replacement>] -regex "/^\s+<!--[\s\S]+?-->(?:\r?\n)?/gm" [-targets "<path>,<path>,..." | <args: file, file file...>]
|
|
131
|
-
note:
|
|
132
|
-
targets - must be array type arg, "['<path>', '<path>',...]" or "<path>,<path>,..."
|
|
133
|
-
note: targets - can use args parameter instead
|
|
134
|
-
It is better to use the <args: file, file file...>
|
|
135
|
-
e.g - jstool -cmd replace -after ".." -regex "re/(?<=reference path=")(\.)(?=\/index.d.ts")/" build/**/*.js
|
|
136
|
-
^^^^^^^^^^^^^
|
|
137
|
-
|
|
138
|
-
version help: jstool -cmd version [-major | -minor] [-pkgJsons "./package.json,../package.json"] [-extras "test/web/index.html"]
|
|
139
|
-
bump top level package.json version(can specify "package.json" paths by `pkgJsons` option if need), specify patch version is unnecessary.
|
|
140
|
-
note:
|
|
141
|
-
extras - can be "<path>,<path>,..." (array type arg)
|
|
142
|
-
|
|
143
|
-
minify help: jstool -cmd minify -basePath extra-tests/web/mini-semaphore [-test "re/\.js$/"] [-suffix ".mini"]
|
|
144
|
-
note:
|
|
145
|
-
basePath - can be "<path>,<path>,..." (array type arg)
|
|
146
|
-
test - can be omit, default is `/\.js$/`
|
|
147
|
-
suffix - can be omit, default is ".mini"
|
|
148
|
-
|
|
149
|
-
rmc help: $ jstool -cmd rmc [-rmc4ts[:keepBangLine]] -basePath "./dist/cjs,./dist/cjs/gulp" -test "/\.(js|d\.ts)$/"
|
|
150
|
-
note: basePath - can be "<path>,<path>,..." (array type arg)
|
|
151
|
-
test - can be omit, defulat `/.js$/`
|
|
152
|
-
rmc4ts- for typescript source.
|
|
153
|
-
keep comment that start with "/*" when "*/" end mark appears in same line.
|
|
154
|
-
if start with "/*-" remove it
|
|
155
|
-
rmc4ts=keepBangLine (2025/12/24)
|
|
156
|
-
- In addition to the "rmc4ts" processing, it also preserves line comments that start with "//!".
|
|
259
|
+
### Runtime-Only Extras
|
|
260
|
+
|
|
261
|
+
These modules are currently bundled, but they are not typed as standalone subpaths.
|
|
262
|
+
|
|
263
|
+
#### `js-dev-tool/extras/json-minify`
|
|
264
|
+
|
|
265
|
+
Exports:
|
|
266
|
+
|
|
267
|
+
- `jsonMinify(source)`
|
|
268
|
+
|
|
269
|
+
This is the direct runtime entry for the JSON / JSONL whitespace remover. Prefer `js-dev-tool/extras/jsonl` if you want the same function with TypeScript support.
|
|
270
|
+
|
|
271
|
+
```js
|
|
272
|
+
const { jsonMinify } = require("js-dev-tool/extras/json-minify");
|
|
273
|
+
|
|
274
|
+
console.log(jsonMinify("{ \"name\": \"alpha\" }"));
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
#### `js-dev-tool/extras/list-deps-of`
|
|
278
|
+
|
|
279
|
+
Exports:
|
|
280
|
+
|
|
281
|
+
- `listDependenciesOf(packageName)`
|
|
157
282
|
|
|
283
|
+
This helper reads `yarn.lock` from `process.cwd()` and collects transitive dependencies for the named package.
|
|
284
|
+
|
|
285
|
+
Constraints:
|
|
286
|
+
|
|
287
|
+
- Yarn v1 lockfile only
|
|
288
|
+
- current working directory must contain the target `yarn.lock`
|
|
289
|
+
- if you want types, import `listDependenciesOf` from `js-dev-tool/utils` instead
|
|
290
|
+
|
|
291
|
+
```js
|
|
292
|
+
const { listDependenciesOf } = require("js-dev-tool/utils");
|
|
293
|
+
|
|
294
|
+
console.log(listDependenciesOf("typescript"));
|
|
295
|
+
```
|
|
296
|
+
|
|
297
|
+
## Basic Types
|
|
298
|
+
|
|
299
|
+
`js-dev-tool/basic-types` is a types-only entry point.
|
|
300
|
+
|
|
301
|
+
```ts
|
|
302
|
+
/// <reference types="js-dev-tool/basic-types"/>
|
|
158
303
|
```
|
|
159
304
|
|
|
160
|
-
|
|
305
|
+
It is a small bag of utility types kept mainly for compatibility with older code.
|
|
306
|
+
|
|
307
|
+
## Commands
|
|
308
|
+
|
|
309
|
+
Available `jstool` commands:
|
|
310
|
+
|
|
311
|
+
- `rws`: record webpack or other bundle sizes
|
|
312
|
+
- `cjbm`: convert JavaScript files to browser-compatible modules
|
|
313
|
+
- `cmtTrick`: toggle comment-trick markers in source
|
|
314
|
+
- `replace`: regex-based multi-file replacement
|
|
315
|
+
- `version`: bump `package.json` versions and optional extra files
|
|
316
|
+
- `minify`: minify JavaScript files with Terser
|
|
317
|
+
- `rmc`: remove C-style comments
|
|
318
|
+
- `zip`: create zip archives with an optional comment
|
|
319
|
+
|
|
320
|
+
Use `jstool -help <command_name>` for the full option list of each command.
|
|
321
|
+
|
|
322
|
+
## Related Libraries
|
|
323
|
+
|
|
324
|
+
### [`literate-regex`](https://www.npmjs.com/package/literate-regex)
|
|
325
|
+
|
|
326
|
+
`literate-regex` was originally developed as part of this project, but it is now a separate package.
|
|
161
327
|
|
|
162
|
-
|
|
163
|
-
See [LICENSE](./LICENSE) for details.
|
|
328
|
+
## License
|
|
164
329
|
|
|
330
|
+
Released under the MIT License. See [LICENSE](./LICENSE) for details.
|
package/extras/cc-map.d.ts
CHANGED
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
* @file extras/cc-map.d.ts
|
|
10
10
|
*/
|
|
11
11
|
export declare type TCC_MAP = {
|
|
12
|
-
/** `\
|
|
12
|
+
/** `\u0000` */ NULL: 0;
|
|
13
13
|
/** `\u0001` */ SOH: 1;
|
|
14
14
|
/** `\u0002` */ STX: 2;
|
|
15
15
|
/** `\u0003` */ ETX: 3;
|
|
@@ -41,7 +41,7 @@ export declare type TCC_MAP = {
|
|
|
41
41
|
/** `\u001d` */ GS: 29;
|
|
42
42
|
/** `\u001e` */ RS: 30;
|
|
43
43
|
/** `\u001f` */ US: 31;
|
|
44
|
-
/**
|
|
44
|
+
/** ` ` */ SPACE: 32;
|
|
45
45
|
/** `!` */ EXCLAMATION: 33;
|
|
46
46
|
/** `"` */ DOUBLE_QUOTE: 34;
|
|
47
47
|
/** `#` */ HASH: 35;
|
|
@@ -103,8 +103,8 @@ export declare type TCC_MAP = {
|
|
|
103
103
|
/** `[` */ LEFT_BRACKET: 91;
|
|
104
104
|
/** `\` */ BACKSLASH: 92;
|
|
105
105
|
/** `]` */ RIGHT_BRACKET: 93;
|
|
106
|
-
/**
|
|
107
|
-
/**
|
|
106
|
+
/** `^` */ CARET: 94;
|
|
107
|
+
/** `_` */ UNDERSCORE: 95;
|
|
108
108
|
/** __`__ */ GRAVE: 96;
|
|
109
109
|
/** `a` */ a: 97;
|
|
110
110
|
/** `b` */ b: 98;
|
package/extras/cc-map.js
CHANGED
|
@@ -10,7 +10,7 @@
|
|
|
10
10
|
* @import { TCC_MAP } from "./cc-map";
|
|
11
11
|
*/
|
|
12
12
|
module.exports = /** @type {TCC_MAP} */({
|
|
13
|
-
/** `\
|
|
13
|
+
/** `\u0000` */ NULL: 0,
|
|
14
14
|
/** `\u0001` */ SOH: 1,
|
|
15
15
|
/** `\u0002` */ STX: 2,
|
|
16
16
|
/** `\u0003` */ ETX: 3,
|
|
@@ -104,8 +104,8 @@ module.exports = /** @type {TCC_MAP} */({
|
|
|
104
104
|
/** `[` */ LEFT_BRACKET: 91,
|
|
105
105
|
/** `\` */ BACKSLASH: 92,
|
|
106
106
|
/** `]` */ RIGHT_BRACKET: 93,
|
|
107
|
-
/**
|
|
108
|
-
/**
|
|
107
|
+
/** `^` */ CARET: 94,
|
|
108
|
+
/** `_` */ UNDERSCORE: 95,
|
|
109
109
|
/** __`__ */ GRAVE: 96,
|
|
110
110
|
/** `a` */ a: 97,
|
|
111
111
|
/** `b` */ b: 98,
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
export type TJsonSourceEnvelop = {
|
|
2
|
+
src: string;
|
|
3
|
+
};
|
|
4
|
+
export type TBlacketPairCode = [open: number, close: number];
|
|
5
|
+
export type TScannerFunction = (source: string, offset: number, out: TJsonSourceEnvelop) => number;
|
|
6
|
+
/**
|
|
7
|
+
* Minify JSON or JSONL text by removing JSON whitespace outside double-quoted strings.
|
|
8
|
+
*
|
|
9
|
+
* - A single top-level value is returned as minified JSON.
|
|
10
|
+
* - Multiple top-level values are returned as minified JSONL separated by `"\n"`.
|
|
11
|
+
*
|
|
12
|
+
* @param {string} source
|
|
13
|
+
* @returns {string}
|
|
14
|
+
* @version 2.0
|
|
15
|
+
*/
|
|
16
|
+
export function jsonMinify(source: string): string;
|
|
17
|
+
/**
|
|
18
|
+
* Parse JSON or JSONL text into an array of top-level JSON values.
|
|
19
|
+
*
|
|
20
|
+
* - A single top-level value is returned as a one-element array.
|
|
21
|
+
* - Multiple top-level values separated by whitespace are returned in order.
|
|
22
|
+
* - Whitespace-only input returns an empty array.
|
|
23
|
+
*
|
|
24
|
+
* @template {unknown} T
|
|
25
|
+
* @param {string} source
|
|
26
|
+
* @returns {T[]}
|
|
27
|
+
* @since 2026/04/10 19:51:10
|
|
28
|
+
*/
|
|
29
|
+
export function parseJsonl<T extends unknown>(source: string): T[];
|
|
@@ -0,0 +1,287 @@
|
|
|
1
|
+
/*!
|
|
2
|
+
// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
3
|
+
// Copyright (C) 2026 jeffy-g <hirotom1107@gmail.com>
|
|
4
|
+
// Released under the MIT license
|
|
5
|
+
// https://opensource.org/licenses/mit-license.php
|
|
6
|
+
// - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
|
|
7
|
+
*/
|
|
8
|
+
"use strict";
|
|
9
|
+
/**
|
|
10
|
+
* @file extras/json-minify.js
|
|
11
|
+
* @version 2.3 2026/04/06 16:11:36
|
|
12
|
+
* @version 2.4 2026/04/08 08:31:11 - expand `sjv` to v25x
|
|
13
|
+
*/
|
|
14
|
+
const ccMap = require("./cc-map");
|
|
15
|
+
/**
|
|
16
|
+
* @typedef {{ src: string }} TJsonSourceEnvelop
|
|
17
|
+
* @typedef {[ open: number, close: number ]} TBlacketPairCode
|
|
18
|
+
*/
|
|
19
|
+
/** @type {TBlacketPairCode} */
|
|
20
|
+
const array = [ccMap.LEFT_BRACKET, ccMap.RIGHT_BRACKET];
|
|
21
|
+
/** @type {TBlacketPairCode} */
|
|
22
|
+
const object = [ccMap.LEFT_BRACE, ccMap.RIGHT_BRACE];
|
|
23
|
+
/**
|
|
24
|
+
* ### (O)thers (M)ap
|
|
25
|
+
*
|
|
26
|
+
* @type {(0 | 1)[]}
|
|
27
|
+
*/
|
|
28
|
+
const om = Array(127).fill(0);
|
|
29
|
+
om[ccMap.DOUBLE_QUOTE] = 1;
|
|
30
|
+
om[ccMap.COMMA] = 1;
|
|
31
|
+
om[ccMap.COLON] = 1;
|
|
32
|
+
om[ccMap.LEFT_BRACKET] = 1;
|
|
33
|
+
om[ccMap.RIGHT_BRACKET] = 1;
|
|
34
|
+
om[ccMap.LEFT_BRACE] = 1;
|
|
35
|
+
om[ccMap.RIGHT_BRACE] = 1;
|
|
36
|
+
/**
|
|
37
|
+
* @param {string} message
|
|
38
|
+
* @param {number} offset
|
|
39
|
+
* @returns {never}
|
|
40
|
+
* @throws {SyntaxError}
|
|
41
|
+
*/
|
|
42
|
+
const throwJsonSyntaxError = (message, offset) => {
|
|
43
|
+
throw new SyntaxError(`${message}, offset=${offset}`);
|
|
44
|
+
};
|
|
45
|
+
/**
|
|
46
|
+
* Minify JSON or JSONL text by removing JSON whitespace outside double-quoted strings.
|
|
47
|
+
*
|
|
48
|
+
* - A single top-level value is returned as minified JSON.
|
|
49
|
+
* - Multiple top-level values are returned as minified JSONL separated by `"\n"`.
|
|
50
|
+
*
|
|
51
|
+
* @param {string} source
|
|
52
|
+
* @returns {string}
|
|
53
|
+
* @version 2.0
|
|
54
|
+
*/
|
|
55
|
+
function v25x(source) {
|
|
56
|
+
if (typeof source !== "string") {
|
|
57
|
+
throw new TypeError("jsonMinify(source): source must be a string");
|
|
58
|
+
}
|
|
59
|
+
/** @type {TJsonSourceEnvelop} */
|
|
60
|
+
const out = { src: "" };
|
|
61
|
+
const sourceLen = source.length;
|
|
62
|
+
let offset = 0;
|
|
63
|
+
let count = 0;
|
|
64
|
+
while (true) {
|
|
65
|
+
const next = skipWhitespaces(source, offset);
|
|
66
|
+
if (next >= sourceLen) {
|
|
67
|
+
break;
|
|
68
|
+
}
|
|
69
|
+
if (count > 0) {
|
|
70
|
+
if (next === offset) {
|
|
71
|
+
throwJsonSyntaxError("Missing whitespace between top-level JSON values", next);
|
|
72
|
+
}
|
|
73
|
+
out.src += "\n";
|
|
74
|
+
}
|
|
75
|
+
switch (source.charCodeAt(next)) {
|
|
76
|
+
case ccMap.DOUBLE_QUOTE: {
|
|
77
|
+
const skiped = sqs(source, next);
|
|
78
|
+
out.src += source.slice(next, skiped);
|
|
79
|
+
offset = skiped;
|
|
80
|
+
break;
|
|
81
|
+
}
|
|
82
|
+
case ccMap.LEFT_BRACKET:
|
|
83
|
+
case ccMap.LEFT_BRACE:
|
|
84
|
+
offset = sjoCode(source, next, out);
|
|
85
|
+
break;
|
|
86
|
+
default: {
|
|
87
|
+
const skiped = skipOthers(source, next);
|
|
88
|
+
out.src += source.slice(next, skiped);
|
|
89
|
+
offset = skiped;
|
|
90
|
+
break;
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
count++;
|
|
94
|
+
}
|
|
95
|
+
return out.src;
|
|
96
|
+
}
|
|
97
|
+
/**
|
|
98
|
+
* @param {string} source
|
|
99
|
+
* @param {number} offset
|
|
100
|
+
* @param {TJsonSourceEnvelop} out
|
|
101
|
+
* @returns {number}
|
|
102
|
+
* @remarks for node v25.x
|
|
103
|
+
*/
|
|
104
|
+
function sjv(source, offset, out) {
|
|
105
|
+
switch (source.charCodeAt(offset)) {
|
|
106
|
+
case ccMap.LEFT_BRACE:
|
|
107
|
+
case ccMap.LEFT_BRACKET: return sjoCode(source, offset, out);
|
|
108
|
+
case ccMap.DOUBLE_QUOTE: {
|
|
109
|
+
const next = sqs(source, offset);
|
|
110
|
+
out.src += source.slice(offset, next);
|
|
111
|
+
return next;
|
|
112
|
+
}
|
|
113
|
+
default: {
|
|
114
|
+
const next = skipOthers(source, offset);
|
|
115
|
+
out.src += source.slice(offset, next);
|
|
116
|
+
return next;
|
|
117
|
+
}
|
|
118
|
+
}
|
|
119
|
+
}
|
|
120
|
+
void sjv;
|
|
121
|
+
/**
|
|
122
|
+
* (S)can (J)son (O)bject
|
|
123
|
+
*
|
|
124
|
+
* @param {string} source
|
|
125
|
+
* @param {number} offset
|
|
126
|
+
* @param {TJsonSourceEnvelop} out
|
|
127
|
+
* @returns {number}
|
|
128
|
+
* @throws {SyntaxError}
|
|
129
|
+
* @version 2.2 `charCodeAt` version
|
|
130
|
+
*/
|
|
131
|
+
function sjoCode(source, offset, out) {
|
|
132
|
+
let sourceRef = out.src, nest = 0;
|
|
133
|
+
const bracket = source[offset] === "[" ? array : object;
|
|
134
|
+
const open = bracket[0];
|
|
135
|
+
const close = bracket[1];
|
|
136
|
+
const sourceLen = source.length;
|
|
137
|
+
while (offset < sourceLen) {
|
|
138
|
+
const code = source.charCodeAt(offset++);
|
|
139
|
+
if (code <= 32 /* ccMap.SPACE */) continue;
|
|
140
|
+
if (code === 34 /* ccMap.DOUBLE_QUOTE */) {
|
|
141
|
+
const next = sqs(source, offset - 1);
|
|
142
|
+
sourceRef += source.slice(offset - 1, next);
|
|
143
|
+
offset = next;
|
|
144
|
+
continue;
|
|
145
|
+
}
|
|
146
|
+
sourceRef += source[offset - 1];
|
|
147
|
+
if (code === open) {
|
|
148
|
+
nest++;
|
|
149
|
+
} else if (code === close) {
|
|
150
|
+
if (--nest === 0) break;
|
|
151
|
+
}
|
|
152
|
+
}
|
|
153
|
+
out.src = sourceRef;
|
|
154
|
+
return offset;
|
|
155
|
+
}
|
|
156
|
+
/**
|
|
157
|
+
* @param {string} source
|
|
158
|
+
* @param {number} offset
|
|
159
|
+
* @returns {number}
|
|
160
|
+
*/
|
|
161
|
+
function skipOthers(source, offset) {
|
|
162
|
+
const sourceLen = source.length;
|
|
163
|
+
let next = offset;
|
|
164
|
+
while (next < sourceLen) {
|
|
165
|
+
const code = source.charCodeAt(next++);
|
|
166
|
+
if (
|
|
167
|
+
code <= 32 || om[code]
|
|
168
|
+
) {
|
|
169
|
+
next -= 1;
|
|
170
|
+
break;
|
|
171
|
+
}
|
|
172
|
+
}
|
|
173
|
+
if (next === offset) throwJsonSyntaxError("Invalid JSON primitive", offset);
|
|
174
|
+
return next;
|
|
175
|
+
}
|
|
176
|
+
/**
|
|
177
|
+
* ### (S)can (Q)uoted (S)tring
|
|
178
|
+
*
|
|
179
|
+
* Scan a JSON double-quoted string and return the next offset.
|
|
180
|
+
*
|
|
181
|
+
* @param {string} source
|
|
182
|
+
* @param {number} offset
|
|
183
|
+
* @returns {number} `"` next offset
|
|
184
|
+
*/
|
|
185
|
+
function sqs(source, offset) {
|
|
186
|
+
const srcLen = source.length;
|
|
187
|
+
let next = offset + 1;
|
|
188
|
+
let inEscape = false;
|
|
189
|
+
while (next < srcLen) {
|
|
190
|
+
const code = source.charCodeAt(next++);
|
|
191
|
+
if (code === 92 /* ccMap.BACKSLASH */) {
|
|
192
|
+
inEscape = !inEscape;
|
|
193
|
+
} else if (inEscape) {
|
|
194
|
+
inEscape = false;
|
|
195
|
+
} else if (code === 34 /* ccMap.DOUBLE_QUOTE */) {
|
|
196
|
+
return next;
|
|
197
|
+
}
|
|
198
|
+
}
|
|
199
|
+
throwJsonSyntaxError("Incomplete JSON string", offset);
|
|
200
|
+
}
|
|
201
|
+
/**
|
|
202
|
+
* @param {number} code
|
|
203
|
+
* @returns {boolean}
|
|
204
|
+
*/
|
|
205
|
+
const isWhitespaceCode = (code) => code <= 32;
|
|
206
|
+
/**
|
|
207
|
+
* @param {string} source
|
|
208
|
+
* @param {number} offset
|
|
209
|
+
* @returns {number}
|
|
210
|
+
*/
|
|
211
|
+
const skipWhitespaces = (source, offset) => {
|
|
212
|
+
const srcLen = source.length;
|
|
213
|
+
while (offset < srcLen) {
|
|
214
|
+
if (!isWhitespaceCode(source.charCodeAt(offset++))) {
|
|
215
|
+
offset -= 1;
|
|
216
|
+
break;
|
|
217
|
+
}
|
|
218
|
+
}
|
|
219
|
+
return offset;
|
|
220
|
+
};
|
|
221
|
+
const jsonMinify = v25x;
|
|
222
|
+
const jparse = JSON.parse;
|
|
223
|
+
/**
|
|
224
|
+
* Parse JSON or JSONL text into an array of top-level JSON values.
|
|
225
|
+
*
|
|
226
|
+
* - A single top-level value is returned as a one-element array.
|
|
227
|
+
* - Multiple top-level values separated by whitespace are returned in order.
|
|
228
|
+
* - Whitespace-only input returns an empty array.
|
|
229
|
+
*
|
|
230
|
+
* @template {unknown} T
|
|
231
|
+
* @param {string} source
|
|
232
|
+
* @returns {T[]}
|
|
233
|
+
* @since 2026/04/10 19:51:10
|
|
234
|
+
*/
|
|
235
|
+
function parseJsonl(source) {
|
|
236
|
+
if (typeof source !== "string") {
|
|
237
|
+
throw new TypeError("parseJsonl(source): source must be a string");
|
|
238
|
+
}
|
|
239
|
+
/** @type {TJsonSourceEnvelop} */
|
|
240
|
+
const out = { src: "" };
|
|
241
|
+
const sourceLen = source.length;
|
|
242
|
+
let offset = 0;
|
|
243
|
+
let count = 0;
|
|
244
|
+
/** @type {T[]} */
|
|
245
|
+
const jsons = [];
|
|
246
|
+
while (true) {
|
|
247
|
+
const next = skipWhitespaces(source, offset);
|
|
248
|
+
if (next >= sourceLen) {
|
|
249
|
+
break;
|
|
250
|
+
}
|
|
251
|
+
if (count > 0) {
|
|
252
|
+
if (next === offset) {
|
|
253
|
+
throwJsonSyntaxError("Missing whitespace between top-level JSON values", next);
|
|
254
|
+
}
|
|
255
|
+
jsons.push(
|
|
256
|
+
jparse(out.src)
|
|
257
|
+
);
|
|
258
|
+
out.src = "";
|
|
259
|
+
}
|
|
260
|
+
switch (source.charCodeAt(next)) {
|
|
261
|
+
case ccMap.DOUBLE_QUOTE: {
|
|
262
|
+
const skiped = sqs(source, next);
|
|
263
|
+
out.src += source.slice(next, skiped);
|
|
264
|
+
offset = skiped;
|
|
265
|
+
break;
|
|
266
|
+
}
|
|
267
|
+
case ccMap.LEFT_BRACKET:
|
|
268
|
+
case ccMap.LEFT_BRACE:
|
|
269
|
+
offset = sjoCode(source, next, out);
|
|
270
|
+
break;
|
|
271
|
+
default: {
|
|
272
|
+
const skiped = skipOthers(source, next);
|
|
273
|
+
out.src += source.slice(next, skiped);
|
|
274
|
+
offset = skiped;
|
|
275
|
+
break;
|
|
276
|
+
}
|
|
277
|
+
}
|
|
278
|
+
count++;
|
|
279
|
+
}
|
|
280
|
+
count && jsons.push(
|
|
281
|
+
jparse(out.src)
|
|
282
|
+
);
|
|
283
|
+
return jsons;
|
|
284
|
+
}
|
|
285
|
+
module.exports = {
|
|
286
|
+
jsonMinify, parseJsonl
|
|
287
|
+
};
|
package/extras/jsonl.d.ts
CHANGED
|
@@ -115,9 +115,26 @@ export function readJsonlMapByKey<TRow = any, TOut = TRow>(fileName: string, opt
|
|
|
115
115
|
*/
|
|
116
116
|
export function fastGetIntFieldCheap(line: string, key: string): number | undefined;
|
|
117
117
|
/**
|
|
118
|
-
* Minify JSON or JSONL text by removing JSON whitespace outside strings.
|
|
118
|
+
* Minify JSON or JSONL text by removing JSON whitespace outside double-quoted strings.
|
|
119
119
|
*
|
|
120
120
|
* - A single top-level value is returned as minified JSON.
|
|
121
121
|
* - Multiple top-level values are returned as minified JSONL separated by `"\n"`.
|
|
122
|
+
*
|
|
123
|
+
* @param {string} source
|
|
124
|
+
* @returns {string}
|
|
125
|
+
* @version 2.0
|
|
126
|
+
*/
|
|
127
|
+
export function jsonMinify(source: string): string;
|
|
128
|
+
/**
|
|
129
|
+
* Parse JSON or JSONL text into an array of top-level JSON values.
|
|
130
|
+
*
|
|
131
|
+
* - A single top-level value is returned as a one-element array.
|
|
132
|
+
* - Multiple top-level values separated by whitespace are returned in order.
|
|
133
|
+
* - Whitespace-only input returns an empty array.
|
|
134
|
+
*
|
|
135
|
+
* @template {unknown} T
|
|
136
|
+
* @param {string} source
|
|
137
|
+
* @returns {T[]}
|
|
138
|
+
* @since 2026/04/10 19:51:10
|
|
122
139
|
*/
|
|
123
|
-
export function
|
|
140
|
+
export function parseJsonl<T extends unknown>(source: string): T[];
|
package/extras/jsonl.js
CHANGED
|
@@ -11,7 +11,7 @@
|
|
|
11
11
|
*/
|
|
12
12
|
const fs = require("fs");
|
|
13
13
|
const path = require("path");
|
|
14
|
-
const
|
|
14
|
+
const { jsonMinify, parseJsonl } = require("./json-minify.js");
|
|
15
15
|
/**
|
|
16
16
|
* @import { TJsonlReadOption, TJsonlReadBaseOption } from "./jsonl.js";
|
|
17
17
|
*/
|
|
@@ -72,231 +72,6 @@ function fastGetIntFieldCheap(line, key) {
|
|
|
72
72
|
}
|
|
73
73
|
return ok ? sign * n : undefined;
|
|
74
74
|
}
|
|
75
|
-
/**
|
|
76
|
-
* @typedef {{ src: string }} TJsonSourceEnvelop
|
|
77
|
-
*/
|
|
78
|
-
/**
|
|
79
|
-
* Minify JSON or JSONL text by removing JSON whitespace outside double-quoted strings.
|
|
80
|
-
*
|
|
81
|
-
* - A single top-level value is returned as minified JSON.
|
|
82
|
-
* - Multiple top-level values are returned as minified JSONL separated by `"\n"`.
|
|
83
|
-
*
|
|
84
|
-
* @param {string} source
|
|
85
|
-
* @returns {string}
|
|
86
|
-
*/
|
|
87
|
-
function jsonMinify(source) {
|
|
88
|
-
if (typeof source !== "string") {
|
|
89
|
-
throw new TypeError("jsonMinify(source): source must be a string");
|
|
90
|
-
}
|
|
91
|
-
/** @type {TJsonSourceEnvelop} */
|
|
92
|
-
const out = { src: "" };
|
|
93
|
-
const sourceLen = source.length;
|
|
94
|
-
let offset = 0;
|
|
95
|
-
let count = 0;
|
|
96
|
-
while (true) {
|
|
97
|
-
const next = skipWhitespaces(source, offset);
|
|
98
|
-
if (next >= sourceLen) {
|
|
99
|
-
break;
|
|
100
|
-
}
|
|
101
|
-
if (count > 0) {
|
|
102
|
-
if (next === offset) {
|
|
103
|
-
throwJsonSyntaxError("Missing whitespace between top-level JSON values", next);
|
|
104
|
-
}
|
|
105
|
-
out.src += "\n";
|
|
106
|
-
}
|
|
107
|
-
offset = scanJsonValue(source, next, out);
|
|
108
|
-
count++;
|
|
109
|
-
}
|
|
110
|
-
return out.src;
|
|
111
|
-
}
|
|
112
|
-
/**
|
|
113
|
-
* @param {string} source
|
|
114
|
-
* @param {number} offset
|
|
115
|
-
* @param {TJsonSourceEnvelop} out
|
|
116
|
-
* @returns {number}
|
|
117
|
-
*/
|
|
118
|
-
function scanJsonValue(source, offset, out) {
|
|
119
|
-
offset = skipWhitespaces(source, offset);
|
|
120
|
-
if (offset >= source.length) {
|
|
121
|
-
throwJsonSyntaxError("Unexpected end of JSON input", offset);
|
|
122
|
-
}
|
|
123
|
-
switch (source[offset]) {
|
|
124
|
-
case "\"": {
|
|
125
|
-
const next = skipQuotedString(source, offset);
|
|
126
|
-
out.src += source.slice(offset, next);
|
|
127
|
-
return next;
|
|
128
|
-
}
|
|
129
|
-
case "{": return scanJsonObject(source, offset, out);
|
|
130
|
-
case "[": return scanJsonArray(source, offset, out);
|
|
131
|
-
default: {
|
|
132
|
-
const next = skipOthers(source, offset);
|
|
133
|
-
out.src += source.slice(offset, next);
|
|
134
|
-
return next;
|
|
135
|
-
}
|
|
136
|
-
}
|
|
137
|
-
}
|
|
138
|
-
/**
|
|
139
|
-
* @param {string} source
|
|
140
|
-
* @param {number} offset
|
|
141
|
-
* @param {TJsonSourceEnvelop} out
|
|
142
|
-
* @returns {number}
|
|
143
|
-
* @throws {SyntaxError}
|
|
144
|
-
*/
|
|
145
|
-
function scanJsonObject(source, offset, out) {
|
|
146
|
-
out.src += "{";
|
|
147
|
-
offset = skipWhitespaces(source, offset + 1);
|
|
148
|
-
if (offset >= source.length) {
|
|
149
|
-
throwJsonSyntaxError("Unterminated JSON object", offset);
|
|
150
|
-
}
|
|
151
|
-
if (source[offset] === "}") {
|
|
152
|
-
out.src += "}";
|
|
153
|
-
return offset + 1;
|
|
154
|
-
}
|
|
155
|
-
while (offset < source.length) {
|
|
156
|
-
if (source.charCodeAt(offset) !== ccMap.DOUBLE_QUOTE) {
|
|
157
|
-
throwJsonSyntaxError("JSON object key must start with a double quote", offset);
|
|
158
|
-
}
|
|
159
|
-
const keyEnd = skipQuotedString(source, offset);
|
|
160
|
-
out.src += source.slice(offset, keyEnd);
|
|
161
|
-
offset = skipWhitespaces(source, keyEnd);
|
|
162
|
-
if (source[offset] !== ":") {
|
|
163
|
-
throwJsonSyntaxError("Missing colon after JSON object key", offset);
|
|
164
|
-
}
|
|
165
|
-
out.src += ":";
|
|
166
|
-
offset = scanJsonValue(source, offset + 1, out);
|
|
167
|
-
offset = skipWhitespaces(source, offset);
|
|
168
|
-
if (offset >= source.length) {
|
|
169
|
-
break;
|
|
170
|
-
}
|
|
171
|
-
const ch = source[offset];
|
|
172
|
-
if (ch === ",") {
|
|
173
|
-
out.src += ",";
|
|
174
|
-
offset = skipWhitespaces(source, offset + 1);
|
|
175
|
-
continue;
|
|
176
|
-
}
|
|
177
|
-
if (ch === "}") {
|
|
178
|
-
out.src += "}";
|
|
179
|
-
return offset + 1;
|
|
180
|
-
}
|
|
181
|
-
throwJsonSyntaxError("Unexpected token in JSON object", offset);
|
|
182
|
-
}
|
|
183
|
-
throwJsonSyntaxError("Unterminated JSON object", offset);
|
|
184
|
-
}
|
|
185
|
-
/**
|
|
186
|
-
* @param {string} source
|
|
187
|
-
* @param {number} offset
|
|
188
|
-
* @param {TJsonSourceEnvelop} out
|
|
189
|
-
* @returns {number}
|
|
190
|
-
*/
|
|
191
|
-
function scanJsonArray(source, offset, out) {
|
|
192
|
-
out.src += "[";
|
|
193
|
-
offset = skipWhitespaces(source, offset + 1);
|
|
194
|
-
if (offset >= source.length) {
|
|
195
|
-
throwJsonSyntaxError("Unterminated JSON array", offset);
|
|
196
|
-
}
|
|
197
|
-
if (source[offset] === "]") {
|
|
198
|
-
out.src += "]";
|
|
199
|
-
return offset + 1;
|
|
200
|
-
}
|
|
201
|
-
while (offset < source.length) {
|
|
202
|
-
offset = scanJsonValue(source, offset, out);
|
|
203
|
-
offset = skipWhitespaces(source, offset);
|
|
204
|
-
if (offset >= source.length) {
|
|
205
|
-
break;
|
|
206
|
-
}
|
|
207
|
-
const ch = source[offset];
|
|
208
|
-
if (ch === ",") {
|
|
209
|
-
out.src += ",";
|
|
210
|
-
offset = skipWhitespaces(source, offset + 1);
|
|
211
|
-
continue;
|
|
212
|
-
}
|
|
213
|
-
if (ch === "]") {
|
|
214
|
-
out.src += "]";
|
|
215
|
-
return offset + 1;
|
|
216
|
-
}
|
|
217
|
-
throwJsonSyntaxError("Unexpected token in JSON array", offset);
|
|
218
|
-
}
|
|
219
|
-
throwJsonSyntaxError("Unterminated JSON array", offset);
|
|
220
|
-
}
|
|
221
|
-
/**
|
|
222
|
-
* @param {string} source
|
|
223
|
-
* @param {number} offset
|
|
224
|
-
* @returns {number}
|
|
225
|
-
*/
|
|
226
|
-
function skipOthers(source, offset) {
|
|
227
|
-
const sourceLen = source.length;
|
|
228
|
-
let next = offset;
|
|
229
|
-
while (next < sourceLen) {
|
|
230
|
-
const code = source.charCodeAt(next++);
|
|
231
|
-
if (
|
|
232
|
-
isWhitespaceCode(code) ||
|
|
233
|
-
code === ccMap.COMMA ||
|
|
234
|
-
code === ccMap.COLON ||
|
|
235
|
-
code === ccMap.LEFT_BRACKET ||
|
|
236
|
-
code === ccMap.RIGHT_BRACKET ||
|
|
237
|
-
code === ccMap.LEFT_BRACE ||
|
|
238
|
-
code === ccMap.RIGHT_BRACE ||
|
|
239
|
-
code === ccMap.DOUBLE_QUOTE
|
|
240
|
-
) {
|
|
241
|
-
next -= 1;
|
|
242
|
-
break;
|
|
243
|
-
}
|
|
244
|
-
}
|
|
245
|
-
if (next === offset) throwJsonSyntaxError("Invalid JSON primitive", offset);
|
|
246
|
-
return next;
|
|
247
|
-
}
|
|
248
|
-
/**
|
|
249
|
-
* Scan a JSON double-quoted string and return the next offset.
|
|
250
|
-
*
|
|
251
|
-
* @param {string} source
|
|
252
|
-
* @param {number} offset
|
|
253
|
-
* @returns {number}
|
|
254
|
-
*/
|
|
255
|
-
function skipQuotedString(source, offset) {
|
|
256
|
-
const srcLen = source.length;
|
|
257
|
-
let next = offset + 1;
|
|
258
|
-
let inEscape = false;
|
|
259
|
-
while (next < srcLen) {
|
|
260
|
-
const ch = source[next++];
|
|
261
|
-
if (ch === "\\") {
|
|
262
|
-
inEscape = !inEscape;
|
|
263
|
-
} else if (inEscape) {
|
|
264
|
-
inEscape = false;
|
|
265
|
-
} else if (ch === "\"") {
|
|
266
|
-
return next;
|
|
267
|
-
}
|
|
268
|
-
}
|
|
269
|
-
throwJsonSyntaxError("Incomplete JSON string", offset);
|
|
270
|
-
}
|
|
271
|
-
/**
|
|
272
|
-
* @param {number} code
|
|
273
|
-
* @returns {boolean}
|
|
274
|
-
*/
|
|
275
|
-
const isWhitespaceCode = (code) => code === ccMap.SPACE || code === ccMap.TAB || code === ccMap.LF || code === ccMap.CR;
|
|
276
|
-
/**
|
|
277
|
-
* @param {string} source
|
|
278
|
-
* @param {number} offset
|
|
279
|
-
* @returns {number}
|
|
280
|
-
*/
|
|
281
|
-
const skipWhitespaces = (source, offset) => {
|
|
282
|
-
const srcLen = source.length;
|
|
283
|
-
while (offset < srcLen) {
|
|
284
|
-
if (!isWhitespaceCode(source.charCodeAt(offset++))) {
|
|
285
|
-
offset -= 1;
|
|
286
|
-
break;
|
|
287
|
-
}
|
|
288
|
-
}
|
|
289
|
-
return offset;
|
|
290
|
-
};
|
|
291
|
-
/**
|
|
292
|
-
* @param {string} message
|
|
293
|
-
* @param {number} offset
|
|
294
|
-
* @returns {never}
|
|
295
|
-
* @throws {SyntaxError}
|
|
296
|
-
*/
|
|
297
|
-
const throwJsonSyntaxError = (message, offset) => {
|
|
298
|
-
throw new SyntaxError(`${message}, offset=${offset}`);
|
|
299
|
-
};
|
|
300
75
|
const HIGHWATERMARK_MIN = 128;
|
|
301
76
|
/**
|
|
302
77
|
* @param {unknown} n
|
|
@@ -366,12 +141,12 @@ async function readJsonl(fileName, onRow, options = {}) {
|
|
|
366
141
|
try {
|
|
367
142
|
const row = jsonParse(line);
|
|
368
143
|
onRow(row, lineNo);
|
|
369
|
-
} catch (
|
|
144
|
+
} catch (e) {
|
|
370
145
|
if (onParseError) {
|
|
371
|
-
onParseError(
|
|
146
|
+
onParseError(e, line, lineNo);
|
|
372
147
|
return;
|
|
373
148
|
}
|
|
374
|
-
console.error("jsonl parse error:", fileName, `line=${lineNo}`,
|
|
149
|
+
console.error("jsonl parse error:", fileName, `line=${lineNo}`, e);
|
|
375
150
|
}
|
|
376
151
|
},
|
|
377
152
|
options
|
|
@@ -427,5 +202,5 @@ module.exports = {
|
|
|
427
202
|
readJsonlArray,
|
|
428
203
|
readJsonlMapByKey,
|
|
429
204
|
fastGetIntFieldCheap,
|
|
430
|
-
jsonMinify,
|
|
205
|
+
jsonMinify, parseJsonl
|
|
431
206
|
};
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "js-dev-tool",
|
|
3
|
-
"version": "1.2.
|
|
3
|
+
"version": "1.2.13",
|
|
4
4
|
"bin": {
|
|
5
5
|
"jstool": "tools.js"
|
|
6
6
|
},
|
|
@@ -76,6 +76,8 @@
|
|
|
76
76
|
"!regex-test.ts",
|
|
77
77
|
"!extras/npm",
|
|
78
78
|
"!/**/*.png",
|
|
79
|
+
"!*@*.*",
|
|
80
|
+
"!*fragments*",
|
|
79
81
|
"!tool.sh"
|
|
80
82
|
],
|
|
81
83
|
"peerDependencies": {
|
|
@@ -89,7 +91,7 @@
|
|
|
89
91
|
"mini-semaphore": "^1.5.4",
|
|
90
92
|
"replace": "^1.2.2",
|
|
91
93
|
"rm-cstyle-cmts": "^3.4.4",
|
|
92
|
-
"terser": "^5.46.
|
|
93
|
-
"tin-args": "^0.1.
|
|
94
|
+
"terser": "^5.46.1",
|
|
95
|
+
"tin-args": "^0.1.5"
|
|
94
96
|
}
|
|
95
97
|
}
|
package/tools.js
CHANGED
|
@@ -93,14 +93,14 @@ const ToolFunctions = {
|
|
|
93
93
|
get help() {
|
|
94
94
|
return `jstool -cmd replace [-after <replacement>] -regex \"/^\\s+<!--[\\s\\S]+?-->(?:\\r?\\n)?/gm\" [-targets "<path>,<path>,..." | <args: file, file file...>]
|
|
95
95
|
|
|
96
|
-
${"Options:".red}
|
|
97
|
-
${"after".cyan} ${`: If omitted, "" (empty string) is used, meaning that the matched string is deleted.`.gray(16)}
|
|
98
|
-
${"targets".cyan}${`: Single path string or Array type arg, "['<path>', '<path>',...]" or "<path>,<path>,..."`.gray(16)}
|
|
96
|
+
${" Options:".red}
|
|
97
|
+
${" after".cyan} ${`: If omitted, "" (empty string) is used, meaning that the matched string is deleted.`.gray(16)}
|
|
98
|
+
${" targets".cyan}${`: Single path string or Array type arg, "['<path>', '<path>',...]" or "<path>,<path>,..."`.gray(16)}
|
|
99
99
|
|
|
100
|
-
${"remarks".magenta}${": targets - can use args parameter instead".gray(16)}${`
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
100
|
+
${" remarks".magenta}${": targets - can use args parameter instead".gray(16)}${`
|
|
101
|
+
It is better to use the <args: file, file file...>
|
|
102
|
+
e.g - ${"jstool".yellow} ${"-cmd".green} ${"replace".cyan} ${"-after".green} ${`".."`.red} ${"-regex".green} ${String.raw`'re/(?<=reference path=")(\.)(?=\/index.d.ts")/'`.magenta} ${"build/**/*.js".gray(16)}
|
|
103
|
+
^^^^^^^^^^^^^
|
|
104
104
|
`.gray(16)}`;
|
|
105
105
|
},
|
|
106
106
|
},
|