oxlint-plugin-pngdeer 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -7
- package/bin/pngdeer.js +63 -75
- package/package.json +1 -1
- package/src/batch.js +85 -25
- package/src/cache.js +31 -3
- package/src/index.d.ts +3 -1
- package/src/rule.js +61 -51
- package/src/scan.js +14 -7
- package/src/shrink.js +63 -0
- package/src/summary.js +120 -26
- package/src/sync.js +7 -7
- package/src/tsconfig.js +132 -0
- package/src/worker.js +22 -12
package/README.md
CHANGED
|
@@ -65,14 +65,16 @@ pngdeer: 4 flagged images could be 718 kB smaller in all (1.0 MB → 310 kB, 70%
|
|
|
65
65
|
src/assets/diagram.png 248 kB → 63 kB 185 kB 90
|
|
66
66
|
public/photo.jpg 235 kB → 85 kB 150 kB 76
|
|
67
67
|
src/assets/crystal-ball.png 35 kB → 11 kB 24 kB 87
|
|
68
|
-
pngdeer: 1
|
|
69
|
-
public/unused-fire.png 13 kB → 7.5 kB 5.1 kB 90
|
|
68
|
+
pngdeer: also 1 image that no linted JS or TS names could be 5.1 kB smaller:
|
|
69
|
+
public/unused-fire.png 13 kB → 7.5 kB 5.1 kB 90 mentioned in README.md
|
|
70
70
|
pngdeer: quality is SSIMULACRA2, out of 100: 90+ looks identical, 80+ the same side by side, 70+ close, with differences you might spot side by side
|
|
71
71
|
pngdeer: to shrink the 4 flagged images in place (lossy, so commit first), run: npx oxlint-plugin-pngdeer --flagged
|
|
72
|
+
pngdeer: to shrink it too, run: npx oxlint-plugin-pngdeer public/unused-fire.png
|
|
72
73
|
```
|
|
73
74
|
|
|
74
|
-
- **Each image warns once per file,** on the line that names it, and says how close to the original it would look.
|
|
75
|
-
- **The table at the end** counts each image once, biggest saving first.
|
|
75
|
+
- **Each image warns once per file,** on the line that names it, and says how close to the original it would look. An image named in several files gets the full warning once per run, and a short one after that.
|
|
76
|
+
- **The table at the end** counts each image once, biggest saving first.
|
|
77
|
+
- **Images no JS or TS names** get their own list, since no warning can point at them. Each row says where the image is used instead: `used in src/App.css`, `mentioned in README.md` (a Markdown file that names it), or `nothing names it`, which may mean the image is unused and could be deleted. These aren't in `--flagged`; the list ends with the command for them.
|
|
76
78
|
- **The table is left out** with `--quiet` and in formats other programs read (`json`, `checkstyle`, `junit`, `sarif`, `gitlab`).
|
|
77
79
|
|
|
78
80
|
Then shrink what it found. There's no autofix (lint fixes can only edit code), so run the command: `--flagged` takes every image the last lint in that folder warned about, or name images yourself.
|
|
@@ -89,7 +91,7 @@ pngdeer: 4 images, 1.0 MB → 310 kB (718 kB smaller). Lint again to check.
|
|
|
89
91
|
|
|
90
92
|
- **It's lossy, and it writes over the file.** Commit first so git is your undo, or write a copy to compare with `-o hero.smaller.png`.
|
|
91
93
|
- **A warning you turn off with a comment** stays out of the table and out of `--flagged`.
|
|
92
|
-
- **
|
|
94
|
+
- **Several images run in parallel,** one engine per core, within a quarter of the machine's memory, like the lint's first run.
|
|
93
95
|
- **`--flagged` uses the `target` each image was measured at.** For images you name, pass it yourself (`--target 90`), as the command in each warning does. `--help` lists the options.
|
|
94
96
|
- **JPEGs keep their orientation and color profile;** other camera metadata (EXIF, GPS) is dropped.
|
|
95
97
|
- **It marks each file it writes** with a short text tag (`pngdeer <size>`, in a PNG `tEXt` chunk or a JPEG comment), so the rule doesn't flag it again, on any machine.
|
|
@@ -98,7 +100,7 @@ Already have oxlint 1.90 or later? Install only `oxlint-plugin-pngdeer`. If your
|
|
|
98
100
|
|
|
99
101
|
## Using ESLint
|
|
100
102
|
|
|
101
|
-
The same rule runs in ESLint 9 and 10, and shares the cache with oxlint:
|
|
103
|
+
The same rule runs in ESLint 9 and 10, and shares the cache with oxlint. With `--concurrency`, ESLint's threads take turns measuring and print one summary:
|
|
102
104
|
|
|
103
105
|
```bash
|
|
104
106
|
npm install --save-dev oxlint-plugin-pngdeer
|
|
@@ -157,7 +159,15 @@ Measured on a 16-core M-series Mac ([details](https://github.com/lndgalante/oxli
|
|
|
157
159
|
|
|
158
160
|
## In CI
|
|
159
161
|
|
|
160
|
-
|
|
162
|
+
The simplest way to never start cold: keep the results in a folder you commit, so CI and every clone start with them.
|
|
163
|
+
|
|
164
|
+
```json
|
|
165
|
+
"settings": { "pngdeer": { "cacheDir": ".pngdeer" } }
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
Results are small JSON files, one per image version, formatted the way Prettier formats JSON (Biome users: add `!.pngdeer` to `files.includes`). After you add or change images, run `npx oxlint` and commit `.pngdeer` with them. Images pngdeer fails on are remembered on each machine only, never in the committed folder. Upgrading the plugin to a new engine measures everything again; delete the folder first to drop the old results. In a monorepo, packages can share one folder (`"cacheDir": "../../.pngdeer"`).
|
|
169
|
+
|
|
170
|
+
Or let CI cache it. `npm ci` deletes `node_modules`, and the default cache with it, so restore the cache after installing, and save it even when the lint fails:
|
|
161
171
|
|
|
162
172
|
```yaml
|
|
163
173
|
- run: npm ci
|
package/bin/pngdeer.js
CHANGED
|
@@ -2,13 +2,11 @@
|
|
|
2
2
|
// oxlint-plugin-pngdeer: makes the images the lint rule flags smaller, with the engine the rule
|
|
3
3
|
// measures with, so each file it writes is the one the warning promised. Files it writes are marked
|
|
4
4
|
// as pngdeer's (src/marker.js), so the rule never measures them again, on any machine.
|
|
5
|
-
import {
|
|
6
|
-
import {
|
|
7
|
-
import {
|
|
8
|
-
import { unsupported } from "../src/images.js";
|
|
9
|
-
import { mark, marked } from "../src/marker.js";
|
|
5
|
+
import { readFileSync, realpathSync } from "node:fs";
|
|
6
|
+
import { peakOf, pool } from "../src/batch.js";
|
|
7
|
+
import { plain, shrink } from "../src/shrink.js";
|
|
10
8
|
import { readFlagged } from "../src/summary.js";
|
|
11
|
-
import { count,
|
|
9
|
+
import { count, pathFrom, size } from "../src/text.js";
|
|
12
10
|
|
|
13
11
|
const HELP = `usage: oxlint-plugin-pngdeer IMAGE... [--target auto|T]
|
|
14
12
|
oxlint-plugin-pngdeer --flagged [--target auto|T]
|
|
@@ -27,7 +25,7 @@ pngdeer is lossy: the picture looks the same, the bytes don't, so commit first.
|
|
|
27
25
|
|
|
28
26
|
Files it writes are marked as pngdeer's, so the lint rule doesn't flag them again.`;
|
|
29
27
|
|
|
30
|
-
function main(args) {
|
|
28
|
+
async function main(args) {
|
|
31
29
|
const inputs = [];
|
|
32
30
|
let output;
|
|
33
31
|
let target;
|
|
@@ -70,90 +68,80 @@ function main(args) {
|
|
|
70
68
|
return 0;
|
|
71
69
|
}
|
|
72
70
|
// each image, the path to show for it, and its target: a --target given here wins over the lint's
|
|
73
|
-
const jobs =
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
: inputs.map((input) => ({ input, target }));
|
|
71
|
+
const jobs = once(
|
|
72
|
+
fromLint
|
|
73
|
+
? flagged.map((image) => ({ input: pathFrom(process.cwd(), image.file), target: targeted ? target : image.target }))
|
|
74
|
+
: inputs.map((input) => ({ input, target })),
|
|
75
|
+
);
|
|
79
76
|
|
|
80
|
-
// several
|
|
81
|
-
|
|
82
|
-
|
|
77
|
+
// one image runs here; several run on a pool of engines, one per core, each line printed as it lands
|
|
78
|
+
if (jobs.length === 1) {
|
|
79
|
+
const [{ input, target }] = jobs;
|
|
80
|
+
try {
|
|
81
|
+
console.log(`${input}: ${shrink(input, output ?? input, target).text}`);
|
|
82
|
+
return 0;
|
|
83
|
+
} catch (error) {
|
|
84
|
+
console.error(`${input}: ${plain(error)}`);
|
|
85
|
+
return 1;
|
|
86
|
+
}
|
|
87
|
+
}
|
|
88
|
+
console.error(`pngdeer: shrinking ${count(jobs.length, "image")} in place (lossy; commit first so you can undo)`);
|
|
83
89
|
let status = 0;
|
|
90
|
+
let finished = 0;
|
|
84
91
|
let original = 0;
|
|
85
92
|
let compressed = 0;
|
|
86
93
|
let shrunk = 0;
|
|
87
|
-
|
|
88
|
-
const step =
|
|
94
|
+
const report = (input, result) => {
|
|
95
|
+
const step = `[${++finished}/${jobs.length}] ${input}`;
|
|
96
|
+
if (result.error) {
|
|
97
|
+
console.error(`${step}: ${result.error}`);
|
|
98
|
+
status = 1;
|
|
99
|
+
return;
|
|
100
|
+
}
|
|
101
|
+
console.log(`${step}: ${result.text}`);
|
|
102
|
+
if (result.written) {
|
|
103
|
+
shrunk++;
|
|
104
|
+
original += result.original;
|
|
105
|
+
compressed += result.written;
|
|
106
|
+
}
|
|
107
|
+
};
|
|
108
|
+
const work = [];
|
|
109
|
+
for (const { input, target } of jobs) {
|
|
110
|
+
let bytes;
|
|
89
111
|
try {
|
|
90
|
-
|
|
91
|
-
console.log(`${step}${input}: ${result.text}`);
|
|
92
|
-
if (result.written) {
|
|
93
|
-
shrunk++;
|
|
94
|
-
original += result.original;
|
|
95
|
-
compressed += result.written;
|
|
96
|
-
}
|
|
112
|
+
bytes = readFileSync(input);
|
|
97
113
|
} catch (error) {
|
|
98
|
-
|
|
99
|
-
|
|
114
|
+
report(input, { error: plain(error) });
|
|
115
|
+
continue;
|
|
100
116
|
}
|
|
117
|
+
work.push({ input, peak: peakOf(bytes), message: { file: input, target, out: input } });
|
|
101
118
|
}
|
|
102
|
-
|
|
119
|
+
try {
|
|
120
|
+
await pool(work, { done: (job, result) => report(job.input, result) });
|
|
121
|
+
} catch (error) {
|
|
122
|
+
console.error(`oxlint-plugin-pngdeer: ${plain(error)}`);
|
|
123
|
+
return 1;
|
|
124
|
+
}
|
|
125
|
+
if (shrunk) {
|
|
103
126
|
console.error(`pngdeer: ${count(shrunk, "image")}, ${size(original)} → ${size(compressed)} (${size(original - compressed)} smaller). Lint again to check.`);
|
|
104
127
|
}
|
|
105
128
|
return status;
|
|
106
129
|
}
|
|
107
130
|
|
|
108
|
-
/**
|
|
109
|
-
function
|
|
110
|
-
const
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
return { text: `${note}; left as is` };
|
|
114
|
-
};
|
|
115
|
-
if (marked(bytes)) return unchanged("already pngdeer's");
|
|
116
|
-
const why = unsupported(bytes);
|
|
117
|
-
if (why) throw new Error(why);
|
|
118
|
-
|
|
119
|
-
const { output: compressed, outcome, score } = compress(bytes, target);
|
|
120
|
-
if (outcome === "miss") {
|
|
121
|
-
const wanted = target === undefined ? "the automatic quality" : `--target ${target}`;
|
|
122
|
-
return unchanged(`no smaller file reaches ${wanted} (the best reaches quality ${Math.round(score)}/100)`);
|
|
123
|
-
}
|
|
124
|
-
// the mark costs a few dozen bytes, more than pngdeer saves on some tiny files
|
|
125
|
-
const result = outcome === "original" ? null : mark(compressed);
|
|
126
|
-
if (!result || result.length >= bytes.length) return unchanged("pngdeer can't make it smaller");
|
|
127
|
-
save(output, result);
|
|
128
|
-
const quality = outcome === "hit" ? `quality ${Math.round(score)}/100: looks ${looks(score)}` : "lossless: looks identical";
|
|
129
|
-
const text = `${size(bytes.length)} → ${size(result.length)} (${smaller(bytes.length, result.length)}% smaller, ${quality})`;
|
|
130
|
-
return { text, original: bytes.length, written: result.length };
|
|
131
|
-
}
|
|
132
|
-
|
|
133
|
-
/** Writes a whole new file or nothing: the image is often written over itself. */
|
|
134
|
-
function save(path, bytes) {
|
|
135
|
-
const tmp = join(dirname(path), `.${basename(path)}.${process.pid}.tmp`);
|
|
136
|
-
try {
|
|
137
|
-
writeFileSync(tmp, bytes);
|
|
131
|
+
/** Each image once: overlapping globs or ./ forms would shrink it twice, the second time lossily. */
|
|
132
|
+
function once(jobs) {
|
|
133
|
+
const seen = new Set();
|
|
134
|
+
return jobs.filter(({ input }) => {
|
|
135
|
+
let path = input;
|
|
138
136
|
try {
|
|
139
|
-
|
|
137
|
+
path = realpathSync(input);
|
|
140
138
|
} catch {
|
|
141
|
-
// a
|
|
139
|
+
// a missing file is reported when it's read
|
|
142
140
|
}
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
throw error;
|
|
148
|
-
}
|
|
149
|
-
}
|
|
150
|
-
|
|
151
|
-
/** An error as one line a person can act on. */
|
|
152
|
-
function plain(error) {
|
|
153
|
-
if (error.code === "ENOENT") return "no such file";
|
|
154
|
-
if (error.code === "EACCES" || error.code === "EPERM") return "permission denied";
|
|
155
|
-
if (error.code === "EISDIR") return "a folder, not an image";
|
|
156
|
-
return error.message;
|
|
141
|
+
if (seen.has(path)) return false;
|
|
142
|
+
seen.add(path);
|
|
143
|
+
return true;
|
|
144
|
+
});
|
|
157
145
|
}
|
|
158
146
|
|
|
159
147
|
function fail(message) {
|
|
@@ -161,4 +149,4 @@ function fail(message) {
|
|
|
161
149
|
return 2;
|
|
162
150
|
}
|
|
163
151
|
|
|
164
|
-
process.exitCode = main(process.argv.slice(2));
|
|
152
|
+
process.exitCode = await main(process.argv.slice(2));
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "oxlint-plugin-pngdeer",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.3.0",
|
|
4
4
|
"description": "Lint rule for oxlint and ESLint: flags PNGs and JPEGs your code uses that could be much smaller at about the same visual quality, with the command that shrinks them. Local WebAssembly, no upload.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"oxlint",
|
package/src/batch.js
CHANGED
|
@@ -1,11 +1,12 @@
|
|
|
1
1
|
// Fills the cache for a list of images, on a pool of worker threads (one wasm engine each, so all
|
|
2
|
-
// cores work even though the engine itself is single-threaded), within a memory budget.
|
|
3
|
-
|
|
2
|
+
// cores work even though the engine itself is single-threaded), within a memory budget. The command
|
|
3
|
+
// shrinks several images on the same pool.
|
|
4
|
+
import { mkdirSync, readFileSync, rmSync, statSync } from "node:fs";
|
|
4
5
|
import { availableParallelism, totalmem } from "node:os";
|
|
5
6
|
import { join } from "node:path";
|
|
6
7
|
import { pathToFileURL } from "node:url";
|
|
7
8
|
import { Worker } from "node:worker_threads";
|
|
8
|
-
import {
|
|
9
|
+
import { key, read, readFirst, write } from "./cache.js";
|
|
9
10
|
import { dimensions, kind, unsupported } from "./images.js";
|
|
10
11
|
import { marked } from "./marker.js";
|
|
11
12
|
|
|
@@ -13,18 +14,32 @@ const MAX_WORKERS = 8;
|
|
|
13
14
|
// Wasm memory never shrinks, so a worker that measured an image expected to peak above this is
|
|
14
15
|
// replaced rather than kept.
|
|
15
16
|
const BIG = 300e6;
|
|
17
|
+
// A batch holding the lock longer than this died with it (sync.js gives up waiting at 15 minutes too).
|
|
18
|
+
const STALE_MS = 15 * 60 * 1000;
|
|
16
19
|
|
|
17
20
|
/**
|
|
18
21
|
* Computes every file not cached yet; { computed, cached, skipped, failed, ms }. `progress(done,
|
|
19
22
|
* total)` hears about each image the engine finishes.
|
|
20
23
|
*/
|
|
21
|
-
export async function fill({
|
|
24
|
+
export async function fill({ dir, failures = dir, target, files }, progress = () => {}) {
|
|
25
|
+
// a result in either folder counts: `failures` keeps what pngdeer failed on, on this machine only
|
|
26
|
+
const cached = (k) => readFirst(k, dir, failures);
|
|
22
27
|
const start = performance.now();
|
|
23
|
-
const dir = cacheDir(cwd);
|
|
24
28
|
// a cache that can't be written fails here, before any image is compressed for nothing
|
|
25
|
-
|
|
26
|
-
|
|
29
|
+
for (const folder of new Set([dir, failures])) {
|
|
30
|
+
write(folder, "probe", {});
|
|
31
|
+
rmSync(join(folder, "probe.json"), { force: true });
|
|
32
|
+
}
|
|
33
|
+
const release = await lock(failures);
|
|
34
|
+
try {
|
|
35
|
+
return await measure({ dir, failures, target, files, cached }, progress, start);
|
|
36
|
+
} finally {
|
|
37
|
+
release();
|
|
38
|
+
}
|
|
39
|
+
}
|
|
27
40
|
|
|
41
|
+
/** fill() once it holds the lock: what's cached by then is skipped, the rest goes to the pool. */
|
|
42
|
+
async function measure({ dir, failures, target, files, cached }, progress, start) {
|
|
28
43
|
const todo = [];
|
|
29
44
|
const counts = { computed: 0, cached: 0, skipped: 0, failed: 0 };
|
|
30
45
|
for (const file of new Set(files)) {
|
|
@@ -35,12 +50,12 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
|
|
|
35
50
|
continue;
|
|
36
51
|
}
|
|
37
52
|
const k = key(bytes, target);
|
|
38
|
-
if (
|
|
53
|
+
if (cached(k)) {
|
|
39
54
|
counts.cached++;
|
|
40
55
|
continue;
|
|
41
56
|
}
|
|
57
|
+
// pngdeer's own output needs no engine, and no entry: the rule knows it by its mark
|
|
42
58
|
if (marked(bytes)) {
|
|
43
|
-
write(dir, k, { original: bytes.length, outcome: "output" });
|
|
44
59
|
counts.skipped++;
|
|
45
60
|
continue;
|
|
46
61
|
}
|
|
@@ -50,12 +65,8 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
|
|
|
50
65
|
counts.skipped++;
|
|
51
66
|
continue;
|
|
52
67
|
}
|
|
53
|
-
|
|
54
|
-
todo.push({ file, k, peak: peak(width * height * frames) });
|
|
68
|
+
todo.push({ file, k, peak: peakOf(bytes), message: { file, target } });
|
|
55
69
|
}
|
|
56
|
-
// biggest first, so the slowest image doesn't start last
|
|
57
|
-
todo.sort((a, b) => b.peak - a.peak);
|
|
58
|
-
const total = todo.length;
|
|
59
70
|
|
|
60
71
|
const record = (job, result) => {
|
|
61
72
|
// the file changed or went away after it was queued: cache nothing, the next lint measures it again
|
|
@@ -64,7 +75,7 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
|
|
|
64
75
|
return;
|
|
65
76
|
}
|
|
66
77
|
if (result.error) {
|
|
67
|
-
write(
|
|
78
|
+
write(failures, job.k, { skipped: result.error, failed: true });
|
|
68
79
|
counts.failed++;
|
|
69
80
|
return;
|
|
70
81
|
}
|
|
@@ -76,6 +87,30 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
|
|
|
76
87
|
if (output !== input) write(dir, output, { original: entry.compressed, outcome: "output" });
|
|
77
88
|
};
|
|
78
89
|
|
|
90
|
+
const total = todo.length;
|
|
91
|
+
await pool(todo, {
|
|
92
|
+
// the same bytes may have been marked as pngdeer's output since the batch started
|
|
93
|
+
skip: (job) => {
|
|
94
|
+
if (!cached(job.k)) return false;
|
|
95
|
+
counts.cached++;
|
|
96
|
+
return true;
|
|
97
|
+
},
|
|
98
|
+
done: (job, result) => {
|
|
99
|
+
record(job, result);
|
|
100
|
+
progress(counts.computed + counts.failed, total);
|
|
101
|
+
},
|
|
102
|
+
});
|
|
103
|
+
return { ...counts, ms: Math.round(performance.now() - start) };
|
|
104
|
+
}
|
|
105
|
+
|
|
106
|
+
/**
|
|
107
|
+
* Runs `jobs` ({ peak, message }) on a pool of engines (worker.js), biggest first, within the memory
|
|
108
|
+
* budget. `done(job, reply)` gets each reply as it comes; a job `skip(job)` turns down when its turn
|
|
109
|
+
* comes isn't sent. Rejects, with every engine stopped, if one fails or dies.
|
|
110
|
+
*/
|
|
111
|
+
export async function pool(jobs, { done, skip = () => false }) {
|
|
112
|
+
// biggest first, so the slowest image doesn't start last
|
|
113
|
+
const todo = [...jobs].sort((a, b) => b.peak - a.peak);
|
|
79
114
|
const limit = budget();
|
|
80
115
|
const lanes = Array.from({ length: Math.min(todo.length, MAX_WORKERS, Math.max(1, availableParallelism() - 1)) }, () => ({ worker: null, held: 0, busy: false }));
|
|
81
116
|
const waiting = [];
|
|
@@ -93,16 +128,12 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
|
|
|
93
128
|
continue;
|
|
94
129
|
}
|
|
95
130
|
const [job] = todo.splice(index, 1);
|
|
96
|
-
|
|
97
|
-
if (read(dir, job.k)) {
|
|
98
|
-
counts.cached++;
|
|
99
|
-
continue;
|
|
100
|
-
}
|
|
131
|
+
if (skip(job)) continue;
|
|
101
132
|
self.busy = true;
|
|
102
133
|
self.held = Math.max(self.held, job.peak);
|
|
103
|
-
self.worker ??= new Worker(new URL("./worker.js", import.meta.url)
|
|
134
|
+
self.worker ??= new Worker(new URL("./worker.js", import.meta.url));
|
|
104
135
|
try {
|
|
105
|
-
|
|
136
|
+
done(job, await ask(self.worker, job.message));
|
|
106
137
|
if (job.peak > BIG) {
|
|
107
138
|
await self.worker.terminate();
|
|
108
139
|
self.worker = null;
|
|
@@ -114,7 +145,6 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
|
|
|
114
145
|
for (const l of lanes) l.worker?.terminate();
|
|
115
146
|
} finally {
|
|
116
147
|
self.busy = false;
|
|
117
|
-
progress(counts.computed + counts.failed, total);
|
|
118
148
|
wake();
|
|
119
149
|
}
|
|
120
150
|
}
|
|
@@ -125,7 +155,37 @@ export async function fill({ cwd, target, files }, progress = () => {}) {
|
|
|
125
155
|
await Promise.all(lanes.map((l) => l.worker?.terminate()));
|
|
126
156
|
}
|
|
127
157
|
if (failure) throw failure;
|
|
128
|
-
|
|
158
|
+
}
|
|
159
|
+
|
|
160
|
+
/**
|
|
161
|
+
* Waits until no other batch measures for this project, then holds that until `release()`: ESLint's
|
|
162
|
+
* --concurrency lints in several threads, and a command-line lint may start while an editor measures.
|
|
163
|
+
* Whoever waits then finds most images cached.
|
|
164
|
+
*/
|
|
165
|
+
async function lock(folder) {
|
|
166
|
+
const path = join(folder, "measuring.lock");
|
|
167
|
+
for (;;) {
|
|
168
|
+
try {
|
|
169
|
+
mkdirSync(path);
|
|
170
|
+
return () => rmSync(path, { recursive: true, force: true });
|
|
171
|
+
} catch (error) {
|
|
172
|
+
// a folder where nothing can be locked: measure anyway
|
|
173
|
+
if (error.code !== "EEXIST") return () => {};
|
|
174
|
+
}
|
|
175
|
+
try {
|
|
176
|
+
if (Date.now() - statSync(path).mtimeMs > STALE_MS) rmSync(path, { recursive: true, force: true });
|
|
177
|
+
} catch {
|
|
178
|
+
// released meanwhile
|
|
179
|
+
}
|
|
180
|
+
await new Promise((resolve) => setTimeout(resolve, 250));
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
|
|
184
|
+
/** An engine's expected peak memory for an image (by its header), for the pool's budget. */
|
|
185
|
+
export function peakOf(bytes) {
|
|
186
|
+
const type = kind(bytes);
|
|
187
|
+
const [width, height, frames] = type ? dimensions(bytes, type) : [0, 0, 1];
|
|
188
|
+
return peak(width * height * frames);
|
|
129
189
|
}
|
|
130
190
|
|
|
131
191
|
/** An engine's expected peak memory for an image of this many pixels (over all its frames). */
|
|
@@ -157,7 +217,7 @@ function ask(worker, message) {
|
|
|
157
217
|
});
|
|
158
218
|
}
|
|
159
219
|
|
|
160
|
-
// `node src/batch.js` with {
|
|
220
|
+
// `node src/batch.js` with { dir, target, files } as JSON on stdin (dir: the cache folder), for trying it outside a linter
|
|
161
221
|
if (import.meta.url === pathToFileURL(process.argv[1] ?? "").href) {
|
|
162
222
|
const request = JSON.parse(readFileSync(0, "utf8"));
|
|
163
223
|
process.stdout.write(JSON.stringify(await fill(request)));
|
package/src/cache.js
CHANGED
|
@@ -2,10 +2,13 @@
|
|
|
2
2
|
// saving only changes when its bytes, the engine or the target do. One small JSON file per entry,
|
|
3
3
|
// so parallel lint threads and batch workers never write the same file twice over.
|
|
4
4
|
import { createHash } from "node:crypto";
|
|
5
|
-
import { mkdirSync, readFileSync, renameSync, writeFileSync } from "node:fs";
|
|
5
|
+
import { mkdirSync, readFileSync, renameSync, statSync, writeFileSync } from "node:fs";
|
|
6
6
|
import { join } from "node:path";
|
|
7
|
+
import { marked } from "./marker.js";
|
|
7
8
|
import { smaller } from "./text.js";
|
|
8
9
|
|
|
10
|
+
const keys = new Map(); // image path -> { size, mtimeMs, target, key, marked }, so images aren't re-read on every keystroke
|
|
11
|
+
|
|
9
12
|
// The engine's identity: the first 16 hex digits of the wasm's SHA-256 (a few ms for 2 MB). A new
|
|
10
13
|
// engine invalidates every entry.
|
|
11
14
|
export const ENGINE = createHash("sha256")
|
|
@@ -38,6 +41,15 @@ export function read(dir, k) {
|
|
|
38
41
|
}
|
|
39
42
|
}
|
|
40
43
|
|
|
44
|
+
/** The entry for `k` in the first of `dirs` that has one, or null. */
|
|
45
|
+
export function readFirst(k, ...dirs) {
|
|
46
|
+
for (const dir of new Set(dirs)) {
|
|
47
|
+
const entry = read(dir, k);
|
|
48
|
+
if (entry) return entry;
|
|
49
|
+
}
|
|
50
|
+
return null;
|
|
51
|
+
}
|
|
52
|
+
|
|
41
53
|
/** Whether an entry is a saving the rule reports: smaller at the target, by both thresholds. */
|
|
42
54
|
export function reportable(entry, { minSavedBytes, minSavedPercent }) {
|
|
43
55
|
if (!entry || entry.skipped || !["hit", "exact", "lossless"].includes(entry.outcome)) return false;
|
|
@@ -45,10 +57,26 @@ export function reportable(entry, { minSavedBytes, minSavedPercent }) {
|
|
|
45
57
|
return saved >= minSavedBytes && smaller(entry.original, entry.compressed) >= minSavedPercent;
|
|
46
58
|
}
|
|
47
59
|
|
|
60
|
+
/** An image's size, the cache key of its current bytes and whether pngdeer wrote it; null if it can't be read. */
|
|
61
|
+
export function fileKey(file, target) {
|
|
62
|
+
try {
|
|
63
|
+
const { size, mtimeMs } = statSync(file);
|
|
64
|
+
const memo = keys.get(file);
|
|
65
|
+
if (memo && memo.size === size && memo.mtimeMs === mtimeMs && memo.target === target) return memo;
|
|
66
|
+
const bytes = readFileSync(file);
|
|
67
|
+
const image = { size, mtimeMs, target, key: key(bytes, target), marked: marked(bytes) };
|
|
68
|
+
keys.set(file, image);
|
|
69
|
+
return image;
|
|
70
|
+
} catch {
|
|
71
|
+
return null;
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
|
|
48
75
|
export function write(dir, k, entry) {
|
|
49
76
|
mkdirSync(dir, { recursive: true });
|
|
50
|
-
// written whole and renamed, so a reader never sees half an entry
|
|
77
|
+
// written whole and renamed, so a reader never sees half an entry; formatted the way Prettier
|
|
78
|
+
// would, so a committed cache folder passes `prettier --check`
|
|
51
79
|
const tmp = join(dir, `${k}.${process.pid}.${Math.random().toString(36).slice(2)}.tmp`);
|
|
52
|
-
writeFileSync(tmp, JSON.stringify(entry));
|
|
80
|
+
writeFileSync(tmp, `${JSON.stringify(entry, null, 2)}\n`);
|
|
53
81
|
renameSync(tmp, join(dir, `${k}.json`));
|
|
54
82
|
}
|
package/src/index.d.ts
CHANGED
|
@@ -14,8 +14,10 @@ export interface CompressibleImageOptions {
|
|
|
14
14
|
export interface PngdeerSettings {
|
|
15
15
|
/** Folders where "/x.png" paths are looked up. Default ["public", "static"]. */
|
|
16
16
|
publicDirs?: string[];
|
|
17
|
-
/** Path prefixes and the folders they stand for. Default { "@/": "src/", "~/": "src/" }. */
|
|
17
|
+
/** Path prefixes and the folders they stand for. Default: the project's tsconfig/jsconfig paths, else { "@/": "src/", "~/": "src/" }. */
|
|
18
18
|
aliases?: Record<string, string>;
|
|
19
|
+
/** Where results are kept, relative to the project root: a folder you can commit. Default node_modules/.cache/pngdeer. */
|
|
20
|
+
cacheDir?: string;
|
|
19
21
|
}
|
|
20
22
|
|
|
21
23
|
// Shaped like an ESLint plugin, so `plugins: { pngdeer }` type-checks in eslint.config.ts and
|
package/src/rule.js
CHANGED
|
@@ -1,12 +1,12 @@
|
|
|
1
1
|
// pngdeer/compressible-image: on each string in the code that names a local PNG or JPEG (imports,
|
|
2
2
|
// require, new URL, JSX src and srcSet, plain strings), says how much smaller pngdeer would make it.
|
|
3
3
|
// oxlint never lints image files themselves, so the reference is where the warning goes.
|
|
4
|
-
import {
|
|
4
|
+
import { statSync } from "node:fs";
|
|
5
5
|
import { basename, dirname, join, resolve } from "node:path";
|
|
6
|
-
import { cacheDir,
|
|
7
|
-
import { marked } from "./marker.js";
|
|
6
|
+
import { cacheDir, fileKey, readFirst, reportable } from "./cache.js";
|
|
8
7
|
import { projectImages } from "./scan.js";
|
|
9
|
-
import { flag, name, summarize, watch } from "./summary.js";
|
|
8
|
+
import { flag, name, share, summarize, watch } from "./summary.js";
|
|
9
|
+
import { configAliases } from "./tsconfig.js";
|
|
10
10
|
import { fillInBackground, fillSync } from "./sync.js";
|
|
11
11
|
import { count, looks, pathFrom, quote, size, smaller } from "./text.js";
|
|
12
12
|
|
|
@@ -14,23 +14,27 @@ const RULE = "pngdeer/compressible-image";
|
|
|
14
14
|
const IMAGE = /\.(png|apng|jpe?g)$/i;
|
|
15
15
|
const REMOTE = /^[a-z][a-z0-9+.-]*:|^\/\//i;
|
|
16
16
|
const DEFAULTS = { minSavedBytes: 4096, minSavedPercent: 10, target: "auto", measure: "auto" };
|
|
17
|
+
// when neither the settings nor the project's tsconfig/jsconfig define aliases
|
|
18
|
+
const ALIASES = { "@/": "src/", "~/": "src/" };
|
|
17
19
|
|
|
18
20
|
// Editors lint on every keystroke, so there the rule measures in the background and reports on a
|
|
19
21
|
// later lint. Everywhere else it waits: a one-off run (the oxlint or eslint command, or a tool calling
|
|
20
22
|
// ESLint's Node API, like Nx or xo) would exit before a background batch finished. Neither linter
|
|
21
|
-
// has an API for this: it's how oxlint's language server (--lsp)
|
|
22
|
-
// and Neovim use, an Electron child in VS Code) start.
|
|
23
|
+
// has an API for this: it's how oxlint's language server (--lsp), ESLint's (the one VS Code, Zed
|
|
24
|
+
// and Neovim use, an Electron child in VS Code) and the eslint_d daemon that editors call start.
|
|
23
25
|
const EDITOR =
|
|
24
26
|
process.argv.includes("--lsp") ||
|
|
25
|
-
/eslintServer|language-?server|languageService/i.test(process.argv[1] ?? "") ||
|
|
27
|
+
/eslintServer|language-?server|languageService|eslint_d/i.test(process.argv[1] ?? "") ||
|
|
26
28
|
process.env.ELECTRON_RUN_AS_NODE === "1";
|
|
27
29
|
|
|
28
30
|
// Per process (the language server's lasts the whole editor session):
|
|
29
|
-
const keys = new Map(); // image path -> { size, mtimeMs, target, key, marked }, so images aren't re-read on every keystroke
|
|
30
31
|
const roots = new Map(); // directory -> its project root
|
|
31
|
-
const scanned = new Set(); // `${root}\0${target}` whose images were all measured, or queued, once
|
|
32
|
+
const scanned = new Set(); // `${root}\0${dir}\0${target}` whose project images were all measured, or queued, once
|
|
32
33
|
const failed = new Set(); // outside editors: roots whose batch failed, not tried again this run
|
|
33
34
|
const told = new Set(); // outside editors: images already named on stderr
|
|
35
|
+
// outside editors: `${image}\0${target}` -> the linted file that got the image's full warning; other
|
|
36
|
+
// files get a short one. A file linted again (eslint --fix passes, watch modes) keeps its full one.
|
|
37
|
+
const warned = new Map();
|
|
34
38
|
|
|
35
39
|
export default {
|
|
36
40
|
meta: {
|
|
@@ -42,6 +46,8 @@ export default {
|
|
|
42
46
|
},
|
|
43
47
|
messages: {
|
|
44
48
|
compressible: "{{name}} could be {{percent}}% smaller ({{before}} → {{after}}) and look {{looks}} (quality {{quality}}/100). Run: {{command}}",
|
|
49
|
+
// the same image again, in another file of the same run: its first warning said the rest
|
|
50
|
+
again: "{{name}} could be {{percent}}% smaller ({{before}} → {{after}}), as warned where it's first used. Run: {{command}}",
|
|
45
51
|
lossless: "{{name}} could be {{percent}}% smaller ({{before}} → {{after}}), losslessly. Run: {{command}}",
|
|
46
52
|
},
|
|
47
53
|
schema: [
|
|
@@ -64,11 +70,18 @@ export default {
|
|
|
64
70
|
create(context) {
|
|
65
71
|
const { minSavedBytes, minSavedPercent, target: option, measure } = { ...DEFAULTS, ...context.options[0] };
|
|
66
72
|
const target = option === "auto" ? undefined : option;
|
|
67
|
-
const
|
|
73
|
+
const settings = context.settings.pngdeer ?? {};
|
|
68
74
|
const from = context.physicalFilename ?? context.filename;
|
|
69
75
|
// public dirs, aliases and the cache belong to the package, wherever the linter was started
|
|
70
76
|
const root = projectRoot(dirname(from), context.cwd);
|
|
71
|
-
const
|
|
77
|
+
const { publicDirs = ["public", "static"] } = settings;
|
|
78
|
+
const aliases = settings.aliases ?? configAliases(root) ?? ALIASES;
|
|
79
|
+
// a folder the team commits (settings.pngdeer.cacheDir) keeps CI and fresh clones warm
|
|
80
|
+
const local = cacheDir(root);
|
|
81
|
+
const dir = settings.cacheDir ? resolve(root, settings.cacheDir) : local;
|
|
82
|
+
// pngdeer failing on an image is kept in `local` only: the failure may be this machine's alone
|
|
83
|
+
// (out of memory), and in a committed folder it would stick for the whole team
|
|
84
|
+
const lookup = (k) => readFirst(k, dir, local);
|
|
72
85
|
const find = (path) => locate(path, from, root, publicDirs, aliases);
|
|
73
86
|
// a path as the person running the linter would type it
|
|
74
87
|
const shown = (file) => pathFrom(context.cwd, file);
|
|
@@ -77,7 +90,7 @@ export default {
|
|
|
77
90
|
|
|
78
91
|
const reported = new Set(); // images warned about in this file: once each, at their first mention
|
|
79
92
|
let disabled; // (line) => whether a comment turns this rule off there, read on first use
|
|
80
|
-
if (!EDITOR) watch(root, { minSavedBytes, minSavedPercent, target });
|
|
93
|
+
if (!EDITOR) watch(root, { minSavedBytes, minSavedPercent, target, dir });
|
|
81
94
|
|
|
82
95
|
function check(node, text) {
|
|
83
96
|
for (const { file, part } of imagesIn(text, find)) {
|
|
@@ -102,17 +115,20 @@ export default {
|
|
|
102
115
|
if (!reportable(result, { minSavedBytes, minSavedPercent }) || reported.has(file)) return;
|
|
103
116
|
// The linter hides a warning a comment turns off by itself, but the image mustn't count as
|
|
104
117
|
// flagged either, or --flagged would rewrite an image someone chose to keep.
|
|
105
|
-
disabled ??= disabledLines(context.sourceCode);
|
|
118
|
+
disabled ??= disabledLines(context.sourceCode, context.id ?? RULE);
|
|
106
119
|
if (disabled((loc?.start ?? node.loc.start).line)) return;
|
|
107
120
|
reported.add(file);
|
|
108
|
-
if (!EDITOR) flag(file, { original: result.original, compressed: result.compressed, score: result.score, target });
|
|
121
|
+
if (!EDITOR) flag(file, from, { original: result.original, compressed: result.compressed, score: result.score, target });
|
|
109
122
|
// this package's own command (bin/pngdeer.js), at the target measured, so it writes exactly the
|
|
110
123
|
// file this promises; named by the package, so npx can't fetch someone else's
|
|
111
124
|
const command = `npx oxlint-plugin-pngdeer ${quote(shown(file))}${target === undefined ? "" : ` --target ${target}`}`;
|
|
125
|
+
const seen = `${file}\0${target}`;
|
|
126
|
+
if (!EDITOR && !warned.has(seen)) warned.set(seen, from);
|
|
127
|
+
const repeat = !EDITOR && warned.get(seen) !== from;
|
|
112
128
|
context.report({
|
|
113
129
|
node,
|
|
114
130
|
loc: loc ?? undefined,
|
|
115
|
-
messageId: result.outcome === "hit" ? "compressible" : "lossless",
|
|
131
|
+
messageId: repeat ? "again" : result.outcome === "hit" ? "compressible" : "lossless",
|
|
116
132
|
data: {
|
|
117
133
|
name: basename(file),
|
|
118
134
|
percent: smaller(result.original, result.compressed),
|
|
@@ -139,37 +155,45 @@ export default {
|
|
|
139
155
|
TemplateLiteral(node) {
|
|
140
156
|
if (node.expressions.length === 0) check(node, node.quasis[0].value.cooked ?? "");
|
|
141
157
|
},
|
|
142
|
-
// What this file names is measured, then reported. The first
|
|
143
|
-
//
|
|
158
|
+
// What this file names is measured, then reported. The first file of a project in a run also
|
|
159
|
+
// takes every image in it, named or not, in the same parallel batch: the files linted next find
|
|
160
|
+
// theirs cached, and the summary can tell about images only CSS or HTML use.
|
|
144
161
|
"Program:exit"() {
|
|
145
|
-
|
|
146
|
-
const
|
|
147
|
-
if (
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
162
|
+
const group = `${root}\0${dir}\0${target}`;
|
|
163
|
+
const first = measure !== "never" && !scanned.has(group);
|
|
164
|
+
if (found.length === 0 && !first) {
|
|
165
|
+
if (!EDITOR) share();
|
|
166
|
+
return;
|
|
167
|
+
}
|
|
168
|
+
scanned.add(group);
|
|
169
|
+
// what the engine has yet to measure: pngdeer's own output (marked) needs none, and gets no entry,
|
|
170
|
+
// so the editor and the command line leave the same files in a cache folder someone commits
|
|
171
|
+
const unmeasured = (file) => {
|
|
172
|
+
const image = fileKey(file, target);
|
|
173
|
+
return image && !image.marked && !lookup(image.key);
|
|
174
|
+
};
|
|
175
|
+
const named = measure === "never" ? [] : found.map(({ file }) => file);
|
|
176
|
+
const files = [...new Set(first ? [...named, ...projectImages(root, minSavedBytes)] : named)].filter(unmeasured);
|
|
177
|
+
if (files.length) {
|
|
178
|
+
if (!wait) fillInBackground({ dir, failures: local, target, files });
|
|
152
179
|
else if (!failed.has(root)) measureNow(files);
|
|
153
180
|
}
|
|
154
181
|
for (const ref of found) {
|
|
155
|
-
const result =
|
|
182
|
+
const result = lookup(ref.k);
|
|
156
183
|
tell(ref.file, result);
|
|
157
184
|
report(ref, result);
|
|
158
185
|
}
|
|
186
|
+
if (!EDITOR) share();
|
|
159
187
|
},
|
|
160
188
|
};
|
|
161
189
|
|
|
190
|
+
/** Measures `files` (none cached yet) while the lint waits, saying what the wait is for. */
|
|
162
191
|
function measureNow(files) {
|
|
163
|
-
// how many need the engine, to say what the wait is for (the hashes are kept for later lookups)
|
|
164
|
-
const fresh = files.filter((file) => {
|
|
165
|
-
const image = fileKey(file, target);
|
|
166
|
-
return image && !image.marked && !read(dir, image.key);
|
|
167
|
-
}).length;
|
|
168
192
|
// on a terminal the batch rewrites this line as a count (fill-worker.js), and it's cleared after
|
|
169
|
-
const tty = Boolean(
|
|
170
|
-
|
|
193
|
+
const tty = Boolean(process.stderr.isTTY);
|
|
194
|
+
process.stderr.write(`pngdeer: measuring ${count(files.length, "image")} not in the cache yet...${tty ? "" : "\n"}`);
|
|
171
195
|
try {
|
|
172
|
-
const { computed, ms } = fillSync({
|
|
196
|
+
const { computed, ms } = fillSync({ dir, failures: local, target, files, progress: tty });
|
|
173
197
|
if (tty) process.stderr.write("\r\x1b[K");
|
|
174
198
|
if (computed) process.stderr.write(`pngdeer: measured ${count(computed, "image")} in ${(ms / 1000).toFixed(1)} s; later runs read the cache\n`);
|
|
175
199
|
} catch (error) {
|
|
@@ -206,9 +230,10 @@ function imagesIn(text, find) {
|
|
|
206
230
|
|
|
207
231
|
/**
|
|
208
232
|
* Whether a comment turns this rule off at a line: `oxlint-` or `eslint-disable-next-line` above it,
|
|
209
|
-
* `-disable-line` on it, or a `-disable` block around it, for this rule
|
|
233
|
+
* `-disable-line` on it, or a `-disable` block around it, for this rule (`id`, as the config names
|
|
234
|
+
* the plugin) or for all rules.
|
|
210
235
|
*/
|
|
211
|
-
function disabledLines(source) {
|
|
236
|
+
function disabledLines(source, id) {
|
|
212
237
|
const lines = new Set();
|
|
213
238
|
const blocks = [];
|
|
214
239
|
let open = null;
|
|
@@ -217,7 +242,7 @@ function disabledLines(source) {
|
|
|
217
242
|
if (!directive) continue;
|
|
218
243
|
const [, kind, rest] = directive;
|
|
219
244
|
const rules = rest.split("--")[0].split(",").map((rule) => rule.trim()).filter(Boolean);
|
|
220
|
-
if (rules.length && !rules.includes(RULE)) continue;
|
|
245
|
+
if (rules.length && !rules.includes(id) && !rules.includes(RULE)) continue;
|
|
221
246
|
if (kind === "disable-next-line") lines.add(comment.loc.end.line + 1);
|
|
222
247
|
else if (kind === "disable-line") lines.add(comment.loc.start.line);
|
|
223
248
|
else if (kind === "disable") open ??= comment.loc.start.line;
|
|
@@ -252,21 +277,6 @@ function projectRoot(dir, fallback) {
|
|
|
252
277
|
return roots.get(dir);
|
|
253
278
|
}
|
|
254
279
|
|
|
255
|
-
/** An image's size, the cache key of its current bytes and whether pngdeer wrote it; null if it can't be read. */
|
|
256
|
-
function fileKey(file, target) {
|
|
257
|
-
try {
|
|
258
|
-
const { size, mtimeMs } = statSync(file);
|
|
259
|
-
const memo = keys.get(file);
|
|
260
|
-
if (memo && memo.size === size && memo.mtimeMs === mtimeMs && memo.target === target) return memo;
|
|
261
|
-
const bytes = readFileSync(file);
|
|
262
|
-
const image = { size, mtimeMs, target, key: key(bytes, target), marked: marked(bytes) };
|
|
263
|
-
keys.set(file, image);
|
|
264
|
-
return image;
|
|
265
|
-
} catch {
|
|
266
|
-
return null;
|
|
267
|
-
}
|
|
268
|
-
}
|
|
269
|
-
|
|
270
280
|
function isFile(path) {
|
|
271
281
|
try {
|
|
272
282
|
return statSync(path).isFile();
|
package/src/scan.js
CHANGED
|
@@ -1,8 +1,9 @@
|
|
|
1
|
-
// Every PNG and JPEG in a project, to measure them all in one parallel batch
|
|
2
|
-
//
|
|
3
|
-
// by name, not by reading .gitignore: dot folders,
|
|
4
|
-
// as visual-regression snapshots, which are images
|
|
5
|
-
// package.json is another project, with its own cache,
|
|
1
|
+
// Every PNG and JPEG in a project, to measure them all in one parallel batch on a run's first file
|
|
2
|
+
// (later references then find them cached), and the other files the summary searches for images no
|
|
3
|
+
// linted code names. Folders are skipped by name, not by reading .gitignore: dot folders,
|
|
4
|
+
// dependencies, build output, and test output such as visual-regression snapshots, which are images
|
|
5
|
+
// too but never shipped. A folder with its own package.json is another project, with its own cache,
|
|
6
|
+
// and is left to it.
|
|
6
7
|
import { readdirSync, statSync } from "node:fs";
|
|
7
8
|
import { extname, join } from "node:path";
|
|
8
9
|
import { EXTENSIONS } from "./images.js";
|
|
@@ -15,6 +16,11 @@ const SKIP = new Set([
|
|
|
15
16
|
|
|
16
17
|
/** The project's images of at least `minSize` bytes. */
|
|
17
18
|
export function projectImages(root, minSize = 0) {
|
|
19
|
+
return projectFiles(root, EXTENSIONS, minSize);
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
/** The project's files with one of `extensions`, from `minSize` to `maxSize` bytes. */
|
|
23
|
+
export function projectFiles(root, extensions, minSize = 0, maxSize = Infinity) {
|
|
18
24
|
const found = [];
|
|
19
25
|
const walk = (dir) => {
|
|
20
26
|
let entries;
|
|
@@ -30,8 +36,9 @@ export function projectImages(root, minSize = 0) {
|
|
|
30
36
|
if (entry.isDirectory()) {
|
|
31
37
|
// Playwright keeps a test's snapshots in "<test file>-snapshots"
|
|
32
38
|
if (!SKIP.has(entry.name) && !entry.name.endsWith("-snapshots")) walk(path);
|
|
33
|
-
} else if (entry.isFile() &&
|
|
34
|
-
|
|
39
|
+
} else if (entry.isFile() && extensions.includes(extname(entry.name).toLowerCase())) {
|
|
40
|
+
const bytes = size(path);
|
|
41
|
+
if (bytes >= minSize && bytes <= maxSize) found.push(path);
|
|
35
42
|
}
|
|
36
43
|
}
|
|
37
44
|
};
|
package/src/shrink.js
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
// Making one image smaller on disk: what the command does to each image it's given, in its own
|
|
2
|
+
// thread for one image or on the pool's engines for several (worker.js).
|
|
3
|
+
import { chmodSync, readFileSync, renameSync, rmSync, statSync, writeFileSync } from "node:fs";
|
|
4
|
+
import { basename, dirname, join } from "node:path";
|
|
5
|
+
import { compress } from "./engine.js";
|
|
6
|
+
import { unsupported } from "./images.js";
|
|
7
|
+
import { mark, marked } from "./marker.js";
|
|
8
|
+
import { looks, size, smaller } from "./text.js";
|
|
9
|
+
|
|
10
|
+
/**
|
|
11
|
+
* Writes `input` made smaller to `output`, marked as pngdeer's: { text } saying what happened, plus
|
|
12
|
+
* { original, written } sizes when it wrote a smaller file. Throws when it can't read or compress.
|
|
13
|
+
*/
|
|
14
|
+
export function shrink(input, output, target) {
|
|
15
|
+
const bytes = readFileSync(input);
|
|
16
|
+
const unchanged = (note) => {
|
|
17
|
+
if (output !== input) save(output, bytes);
|
|
18
|
+
return { text: `${note}; left as is` };
|
|
19
|
+
};
|
|
20
|
+
if (marked(bytes)) return unchanged("already pngdeer's");
|
|
21
|
+
const why = unsupported(bytes);
|
|
22
|
+
if (why) throw new Error(why);
|
|
23
|
+
|
|
24
|
+
const { output: compressed, outcome, score } = compress(bytes, target);
|
|
25
|
+
if (outcome === "miss") {
|
|
26
|
+
const wanted = target === undefined ? "the automatic quality" : `--target ${target}`;
|
|
27
|
+
return unchanged(`no smaller file reaches ${wanted} (the best reaches quality ${Math.round(score)}/100)`);
|
|
28
|
+
}
|
|
29
|
+
// the mark costs a few dozen bytes, more than pngdeer saves on some tiny files
|
|
30
|
+
const result = outcome === "original" ? null : mark(compressed);
|
|
31
|
+
if (!result || result.length >= bytes.length) return unchanged("pngdeer can't make it smaller");
|
|
32
|
+
save(output, result);
|
|
33
|
+
const quality = outcome === "hit" ? `quality ${Math.round(score)}/100: looks ${looks(score)}` : "lossless: looks identical";
|
|
34
|
+
const text = `${size(bytes.length)} → ${size(result.length)} (${smaller(bytes.length, result.length)}% smaller, ${quality})`;
|
|
35
|
+
return { text, original: bytes.length, written: result.length };
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
/** An error as one line a person can act on. */
|
|
39
|
+
export function plain(error) {
|
|
40
|
+
if (error?.code === "ENOENT") return "no such file";
|
|
41
|
+
if (error?.code === "EACCES" || error?.code === "EPERM") return "permission denied";
|
|
42
|
+
if (error?.code === "EISDIR") return "a folder, not an image";
|
|
43
|
+
return String(error?.message ?? error);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/** Writes a whole new file or nothing: the image is often written over itself. */
|
|
47
|
+
function save(path, bytes) {
|
|
48
|
+
// unique per writer: the pool's engines share the process, and one image may come twice
|
|
49
|
+
const tmp = join(dirname(path), `.${basename(path)}.${process.pid}.${Math.random().toString(36).slice(2)}.tmp`);
|
|
50
|
+
try {
|
|
51
|
+
writeFileSync(tmp, bytes);
|
|
52
|
+
try {
|
|
53
|
+
chmodSync(tmp, statSync(path).mode & 0o7777);
|
|
54
|
+
} catch {
|
|
55
|
+
// a new file keeps the default mode
|
|
56
|
+
}
|
|
57
|
+
renameSync(tmp, path);
|
|
58
|
+
} catch (error) {
|
|
59
|
+
rmSync(tmp, { force: true });
|
|
60
|
+
if (error.code === "ENOENT") throw new Error(`can't write ${path}: its folder doesn't exist`);
|
|
61
|
+
throw error;
|
|
62
|
+
}
|
|
63
|
+
}
|
package/src/summary.js
CHANGED
|
@@ -1,26 +1,35 @@
|
|
|
1
1
|
// What a lint flagged, in all: a table at the end of the run, biggest saving first, and the list that
|
|
2
2
|
// `npx oxlint-plugin-pngdeer --flagged` shrinks, so nobody copies paths from warnings one by one.
|
|
3
|
-
// The table also
|
|
4
|
-
//
|
|
5
|
-
import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
|
|
6
|
-
import { dirname, join } from "node:path";
|
|
7
|
-
import {
|
|
8
|
-
import {
|
|
9
|
-
import {
|
|
3
|
+
// The table also lists images the run measured that no linted JS or TS names, which no warning can
|
|
4
|
+
// point at, each with where it is used instead (a stylesheet, a page, a template), if anywhere.
|
|
5
|
+
import { existsSync, mkdirSync, readdirSync, readFileSync, rmSync, statSync, writeFileSync } from "node:fs";
|
|
6
|
+
import { basename, dirname, join } from "node:path";
|
|
7
|
+
import { isMainThread, threadId } from "node:worker_threads";
|
|
8
|
+
import { cacheDir, fileKey, read, reportable } from "./cache.js";
|
|
9
|
+
import { projectFiles, projectImages } from "./scan.js";
|
|
10
|
+
import { count, pathFrom, quote, size, smaller } from "./text.js";
|
|
10
11
|
|
|
11
12
|
// Rows shown per table; the rest are counted, and --flagged still takes them all.
|
|
12
13
|
const ROWS = 10;
|
|
13
14
|
const UNNAMED_ROWS = 5;
|
|
14
15
|
// Formats other programs read: the table on stderr would only get in the way.
|
|
15
16
|
const MACHINE = new Set(["json", "json-with-metadata", "checkstyle", "junit", "sarif", "gitlab"]);
|
|
17
|
+
// Files the linter doesn't read but that use images, most telling first: styles, pages and
|
|
18
|
+
// templates, then content and data. Markdown comes last, and only "mentions" an image: a README
|
|
19
|
+
// that names one may just be describing it.
|
|
20
|
+
const PAGES = [".css", ".scss", ".sass", ".less", ".styl", ".html", ".htm", ".vue", ".svelte", ".astro", ".mdx", ".json", ".yml", ".yaml"];
|
|
21
|
+
const DOCS = [".md"];
|
|
22
|
+
const PAGE_BYTES = 1e6; // bigger ones are build output or lockfiles, not where people name images
|
|
16
23
|
|
|
17
|
-
const flagged = new Map(); // image path -> { original, compressed, score, target }
|
|
24
|
+
const flagged = new Map(); // image path -> { original, compressed, score, target, files: the linted files warning about it }
|
|
18
25
|
const named = new Set(); // every image linted code names, flagged or not
|
|
19
26
|
const roots = new Map(); // project root -> the rule's options there
|
|
27
|
+
const SHARE_HOURS = 1; // a thread's share older than this was left by a run that died
|
|
20
28
|
|
|
21
|
-
/** Notes an image a warning was reported for. */
|
|
22
|
-
export function flag(file, entry) {
|
|
23
|
-
flagged.
|
|
29
|
+
/** Notes an image a warning in linted file `from` was reported for: one warning per file, however often it's linted. */
|
|
30
|
+
export function flag(file, from, entry) {
|
|
31
|
+
const files = flagged.get(file)?.files ?? new Set();
|
|
32
|
+
flagged.set(file, { ...entry, files: files.add(from) });
|
|
24
33
|
}
|
|
25
34
|
|
|
26
35
|
/** Notes an image linted code names. */
|
|
@@ -47,8 +56,58 @@ export function readFlagged(cwd) {
|
|
|
47
56
|
}
|
|
48
57
|
}
|
|
49
58
|
|
|
50
|
-
/**
|
|
59
|
+
/**
|
|
60
|
+
* ESLint's --concurrency lints in worker threads, each with its own copy of this module: each leaves
|
|
61
|
+
* what it saw for the main thread, after every file it lints (a thread may be stopped without warning),
|
|
62
|
+
* and only the main thread prints. oxlint runs every rule on its main thread.
|
|
63
|
+
*/
|
|
64
|
+
export function share(cwd = process.cwd()) {
|
|
65
|
+
if (isMainThread || roots.size === 0) return;
|
|
66
|
+
try {
|
|
67
|
+
const dir = join(cacheDir(cwd), "threads");
|
|
68
|
+
mkdirSync(dir, { recursive: true });
|
|
69
|
+
const state = { flagged: [...flagged].map(([file, entry]) => [file, { ...entry, files: [...entry.files] }]), named: [...named], roots: [...roots] };
|
|
70
|
+
writeFileSync(join(dir, `${process.pid}-${threadId}.json`), JSON.stringify(state));
|
|
71
|
+
} catch {
|
|
72
|
+
// that thread's images go untold
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
|
|
76
|
+
/** Takes in what this process's lint threads left, and drops shares a dead run left behind. */
|
|
77
|
+
function gather(cwd) {
|
|
78
|
+
const dir = join(cacheDir(cwd), "threads");
|
|
79
|
+
let names;
|
|
80
|
+
try {
|
|
81
|
+
names = readdirSync(dir);
|
|
82
|
+
} catch {
|
|
83
|
+
return;
|
|
84
|
+
}
|
|
85
|
+
for (const name of names) {
|
|
86
|
+
const path = join(dir, name);
|
|
87
|
+
try {
|
|
88
|
+
if (name.startsWith(`${process.pid}-`)) {
|
|
89
|
+
const state = JSON.parse(readFileSync(path, "utf8"));
|
|
90
|
+
for (const [file, entry] of state.flagged) {
|
|
91
|
+
const files = new Set([...(flagged.get(file)?.files ?? []), ...entry.files]);
|
|
92
|
+
flagged.set(file, { ...entry, files });
|
|
93
|
+
}
|
|
94
|
+
for (const file of state.named) named.add(file);
|
|
95
|
+
for (const [root, options] of state.roots) roots.set(root, options);
|
|
96
|
+
} else if (Date.now() - statSync(path).mtimeMs < SHARE_HOURS * 3600e3) continue;
|
|
97
|
+
rmSync(path, { force: true });
|
|
98
|
+
} catch {
|
|
99
|
+
// a share half written or already gone
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
/** Prints the table on stderr and leaves the list in `cwd`; once, as the lint exits, on the main thread. */
|
|
51
105
|
export function summarize(cwd = process.cwd()) {
|
|
106
|
+
if (!isMainThread) return;
|
|
107
|
+
gather(cwd);
|
|
108
|
+
// a process that linted nothing (ESLint's main thread, when worker threads did the linting and
|
|
109
|
+
// left nothing) has nothing to say, and mustn't empty the list the last lint left
|
|
110
|
+
if (roots.size === 0) return;
|
|
52
111
|
const list = flaggedList(cwd);
|
|
53
112
|
const images = [...flagged].sort(([, a], [, b]) => b.original - b.compressed - (a.original - a.compressed));
|
|
54
113
|
// a clean run replaces an older list, but doesn't create one
|
|
@@ -64,11 +123,19 @@ export function summarize(cwd = process.cwd()) {
|
|
|
64
123
|
|
|
65
124
|
const unnamed = unnamedImages();
|
|
66
125
|
if (images.length === 0 && unnamed.length === 0) return;
|
|
126
|
+
const shownUnnamed = unnamed.slice(0, UNNAMED_ROWS);
|
|
67
127
|
const row = ([file, { original, compressed, score }]) => [pathFrom(cwd, file), size(original), size(compressed), size(original - compressed), String(Math.round(score))];
|
|
68
|
-
const
|
|
69
|
-
|
|
70
|
-
const
|
|
71
|
-
|
|
128
|
+
const header = ["image", "now", "after", "saves", "quality"];
|
|
129
|
+
// the unnamed images say where they're named, as no warning can point there
|
|
130
|
+
const where = ([file, , root]) => {
|
|
131
|
+
const page = namedIn(root, file);
|
|
132
|
+
if (!page) return "nothing names it";
|
|
133
|
+
return `${page.endsWith(".md") ? "mentioned" : "used"} in ${pathFrom(cwd, page)}`;
|
|
134
|
+
};
|
|
135
|
+
const rows = [header, ...images.slice(0, ROWS).map(row), ...shownUnnamed.map((image) => [...row(image), where(image)])];
|
|
136
|
+
const width = (column) => Math.max(...rows.map((cells) => (cells[column] ?? "").length));
|
|
137
|
+
const line = ([path, before, after, saved, score, place], arrow = "→") =>
|
|
138
|
+
` ${path.padEnd(width(0))} ${before.padStart(width(1))} ${arrow} ${after.padStart(width(2))} ${saved.padStart(width(3))} ${score.padStart(width(4))}${place === undefined ? "" : ` ${place}`}`.trimEnd();
|
|
72
139
|
const total = (list) => {
|
|
73
140
|
const original = list.reduce((sum, [, image]) => sum + image.original, 0);
|
|
74
141
|
const compressed = list.reduce((sum, [, image]) => sum + image.compressed, 0);
|
|
@@ -77,14 +144,19 @@ export function summarize(cwd = process.cwd()) {
|
|
|
77
144
|
const lines = [];
|
|
78
145
|
if (images.length) {
|
|
79
146
|
const { saved, detail } = total(images);
|
|
80
|
-
|
|
147
|
+
// an image named in many files warns in each: say so, or the counts look wrong
|
|
148
|
+
const warnings = images.reduce((sum, [, image]) => sum + image.files.size, 0);
|
|
149
|
+
const often = warnings > images.length ? ` (${count(warnings, "warning")})` : "";
|
|
150
|
+
lines.push(`pngdeer: ${count(images.length, "flagged image")}${often} could be ${saved} smaller in all (${detail}):`, line(header, " "));
|
|
81
151
|
lines.push(...rows.slice(1, 1 + Math.min(images.length, ROWS)).map((cells) => line(cells)));
|
|
82
152
|
if (images.length > ROWS) lines.push(` …and ${count(images.length - ROWS, "more image")}, ${total(images.slice(ROWS)).saved} in all`);
|
|
83
153
|
}
|
|
84
154
|
if (unnamed.length) {
|
|
155
|
+
// no warning can point at these: their only mention is in CSS, HTML and the like, or nowhere
|
|
85
156
|
const { saved } = total(unnamed);
|
|
86
|
-
|
|
87
|
-
|
|
157
|
+
const lead = images.length ? "also" : "no warnings, but";
|
|
158
|
+
lines.push(`pngdeer: ${lead} ${count(unnamed.length, "image")} that no linted JS or TS names could be ${saved} smaller:`);
|
|
159
|
+
if (!images.length) lines.push(line(header, " "));
|
|
88
160
|
lines.push(...rows.slice(1 + Math.min(images.length, ROWS)).map((cells) => line(cells)));
|
|
89
161
|
if (unnamed.length > UNNAMED_ROWS) lines.push(` …and ${count(unnamed.length - UNNAMED_ROWS, "more image")}`);
|
|
90
162
|
}
|
|
@@ -92,28 +164,50 @@ export function summarize(cwd = process.cwd()) {
|
|
|
92
164
|
if (images.length) {
|
|
93
165
|
lines.push(`pngdeer: to shrink the ${images.length === 1 ? "flagged image" : `${images.length} flagged images`} in place (lossy, so commit first), run: npx oxlint-plugin-pngdeer --flagged`);
|
|
94
166
|
}
|
|
167
|
+
if (unnamed.length) {
|
|
168
|
+
const paths = shownUnnamed.map(([file]) => quote(pathFrom(cwd, file))).join(" ");
|
|
169
|
+
const which = unnamed.length === 1 ? "it" : "them";
|
|
170
|
+
lines.push(`pngdeer: to shrink ${images.length ? `${which} too` : `${which} in place (lossy, so commit first)`}, run: npx oxlint-plugin-pngdeer ${paths}`);
|
|
171
|
+
if (shownUnnamed.some((image) => !namedIn(image[2], image[0]))) {
|
|
172
|
+
lines.push("pngdeer: an image nothing names may be unused (or its path is built at runtime): deleting it saves it all");
|
|
173
|
+
}
|
|
174
|
+
}
|
|
95
175
|
process.stderr.write(`${lines.join("\n")}\n`);
|
|
96
176
|
}
|
|
97
177
|
|
|
98
178
|
/**
|
|
99
179
|
* Measured images in the linted projects that no linted code names, and that would clear the rule's
|
|
100
|
-
* thresholds: [[path, entry]], biggest saving first. Reads only results already cached.
|
|
180
|
+
* thresholds: [[path, entry, root]], biggest saving first. Reads only results already cached.
|
|
101
181
|
*/
|
|
102
182
|
function unnamedImages() {
|
|
103
183
|
const found = [];
|
|
104
184
|
for (const [root, options] of roots) {
|
|
105
185
|
for (const file of projectImages(root, options.minSavedBytes)) {
|
|
106
186
|
if (named.has(file) || flagged.has(file)) continue;
|
|
107
|
-
|
|
187
|
+
const image = fileKey(file, options.target);
|
|
188
|
+
const entry = image && read(options.dir, image.key);
|
|
189
|
+
if (reportable(entry, options)) found.push([file, entry, root]);
|
|
190
|
+
}
|
|
191
|
+
}
|
|
192
|
+
return found.sort(([, a], [, b]) => b.original - b.compressed - (a.original - a.compressed));
|
|
193
|
+
}
|
|
194
|
+
|
|
195
|
+
const pages = new Map(); // project root -> [{ path, text }] of its PAGES, then its DOCS, read once
|
|
196
|
+
|
|
197
|
+
/** The most telling file the linter doesn't read (a stylesheet, a template, a doc) that names `image`; or null. */
|
|
198
|
+
function namedIn(root, image) {
|
|
199
|
+
if (!pages.has(root)) {
|
|
200
|
+
const files = [...PAGES, ...DOCS].flatMap((extension) => projectFiles(root, [extension], 0, PAGE_BYTES));
|
|
201
|
+
pages.set(root, files.flatMap((path) => {
|
|
108
202
|
try {
|
|
109
|
-
|
|
203
|
+
return [{ path, text: readFileSync(path, "utf8") }];
|
|
110
204
|
} catch {
|
|
111
|
-
|
|
205
|
+
return [];
|
|
112
206
|
}
|
|
113
|
-
|
|
114
|
-
}
|
|
207
|
+
}));
|
|
115
208
|
}
|
|
116
|
-
|
|
209
|
+
const name = basename(image);
|
|
210
|
+
return pages.get(root).find(({ text }) => text.includes(name))?.path ?? null;
|
|
117
211
|
}
|
|
118
212
|
|
|
119
213
|
/** The linter's output format, from its command line: `-f x`, `--format x` or `--format=x`. */
|
package/src/sync.js
CHANGED
|
@@ -31,10 +31,10 @@ export function fillSync(request) {
|
|
|
31
31
|
|
|
32
32
|
// The editor's background batches: one at a time per process, whatever arrives meanwhile goes in
|
|
33
33
|
// the next one, so fill()'s pool and memory budget hold for the whole language server. Files are
|
|
34
|
-
// keyed by
|
|
35
|
-
const queued = new Map(); // `${
|
|
36
|
-
const pending = new Set(); // `${
|
|
37
|
-
const failedAt = new Map(); // `${
|
|
34
|
+
// keyed by cache folder and target, as a batch has one of each.
|
|
35
|
+
const queued = new Map(); // `${dir}\0${target}` -> { dir, target, files: Set }
|
|
36
|
+
const pending = new Set(); // `${dir}\0${target}\0${file}`, queued or being measured
|
|
37
|
+
const failedAt = new Map(); // `${dir}\0${target}` -> when its last batch failed
|
|
38
38
|
const RETRY_MS = 5 * 60 * 1000;
|
|
39
39
|
let running = false;
|
|
40
40
|
|
|
@@ -57,10 +57,10 @@ function drain() {
|
|
|
57
57
|
}
|
|
58
58
|
|
|
59
59
|
/** fill(request) in the background, for the editor: nothing waits, nothing is reported late. */
|
|
60
|
-
export function fillInBackground({
|
|
61
|
-
const group = `${
|
|
60
|
+
export function fillInBackground({ dir, failures, target, files }) {
|
|
61
|
+
const group = `${dir}\0${target}`;
|
|
62
62
|
if (Date.now() - (failedAt.get(group) ?? -Infinity) < RETRY_MS) return;
|
|
63
|
-
const batch = queued.get(group) ?? {
|
|
63
|
+
const batch = queued.get(group) ?? { dir, failures, target, files: new Set() };
|
|
64
64
|
for (const file of files) {
|
|
65
65
|
if (pending.has(`${group}\0${file}`)) continue;
|
|
66
66
|
pending.add(`${group}\0${file}`);
|
package/src/tsconfig.js
ADDED
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
// Path aliases a project defines in tsconfig.json or jsconfig.json (compilerOptions.paths), resolved
|
|
2
|
+
// the way TypeScript does, so "@/assets/hero.png" finds the file the build would: `extends` (later
|
|
3
|
+
// entries win, ".json" optional), `paths` and `baseUrl` each from the nearest config that sets them
|
|
4
|
+
// (`null` unsets), targets relative to `baseUrl` or else to the config that set `paths`, and
|
|
5
|
+
// `${configDir}` as the folder of the config being resolved. A config without paths falls back to
|
|
6
|
+
// its `references` (Vite keeps them in tsconfig.app.json). Configs from packages are skipped: they
|
|
7
|
+
// don't point into the project.
|
|
8
|
+
import { readFileSync, statSync } from "node:fs";
|
|
9
|
+
import { dirname, join, relative, resolve, sep } from "node:path";
|
|
10
|
+
|
|
11
|
+
const found = new Map(); // project root -> { aliases, seen: Map(config path -> mtime, or -1 if missing) }
|
|
12
|
+
const told = new Set(); // configs that couldn't be read, already said so
|
|
13
|
+
|
|
14
|
+
/** { "@/": "src/" }-style aliases from the project's config, as folders relative to `root`; null if none. */
|
|
15
|
+
export function configAliases(root) {
|
|
16
|
+
const memo = found.get(root);
|
|
17
|
+
// a config edited since (an editor runs for hours; .svelte-kit and .nuxt regenerate theirs)
|
|
18
|
+
if (memo && [...memo.seen].every(([file, mtime]) => mtimeOf(file) === mtime)) return memo.aliases;
|
|
19
|
+
const seen = new Map();
|
|
20
|
+
let aliases = null;
|
|
21
|
+
try {
|
|
22
|
+
aliases = projectAliases(root, seen);
|
|
23
|
+
} catch {
|
|
24
|
+
// whatever a config holds, the rule goes on with the default aliases
|
|
25
|
+
}
|
|
26
|
+
found.set(root, { aliases, seen });
|
|
27
|
+
return aliases;
|
|
28
|
+
}
|
|
29
|
+
|
|
30
|
+
/** The aliases of the project's tsconfig.json (or jsconfig.json, without one), or of its references. */
|
|
31
|
+
function projectAliases(root, seen) {
|
|
32
|
+
const tsconfig = join(root, "tsconfig.json");
|
|
33
|
+
const leaf = mtimeOf(tsconfig) >= 0 ? tsconfig : join(root, "jsconfig.json");
|
|
34
|
+
const options = effective(leaf, dirname(leaf), seen, new Set());
|
|
35
|
+
if (options?.paths !== undefined) return aliasesOf(options, root, dirname(leaf));
|
|
36
|
+
for (const reference of load(leaf, seen)?.references ?? []) {
|
|
37
|
+
if (typeof reference?.path !== "string") continue;
|
|
38
|
+
const file = configFile(resolve(dirname(leaf), reference.path));
|
|
39
|
+
if (!file) continue;
|
|
40
|
+
const referenced = effective(file, dirname(file), seen, new Set());
|
|
41
|
+
if (referenced?.paths !== undefined) return aliasesOf(referenced, root, dirname(file));
|
|
42
|
+
}
|
|
43
|
+
return null;
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
/**
|
|
47
|
+
* A config's `paths` (with `pathsDir`, the folder of the config that set them) and `baseUrl` (a
|
|
48
|
+
* folder), after its `extends`; a key is present only if some config in the chain set it.
|
|
49
|
+
*/
|
|
50
|
+
function effective(file, configDir, seen, chain) {
|
|
51
|
+
if (chain.has(file)) return {};
|
|
52
|
+
chain.add(file);
|
|
53
|
+
const config = load(file, seen);
|
|
54
|
+
if (!config) return null;
|
|
55
|
+
let options = {};
|
|
56
|
+
const extended = typeof config.extends === "string" ? [config.extends] : Array.isArray(config.extends) ? config.extends : [];
|
|
57
|
+
for (const spec of extended) {
|
|
58
|
+
const base = typeof spec === "string" && (spec.startsWith(".") || spec.startsWith("/") || spec.startsWith("${configDir}")) ? configFile(resolve(dirname(file), expand(spec, configDir))) : null;
|
|
59
|
+
if (base) options = { ...options, ...effective(base, configDir, seen, chain) };
|
|
60
|
+
}
|
|
61
|
+
const own = config.compilerOptions ?? {};
|
|
62
|
+
if ("baseUrl" in own) options.baseUrl = typeof own.baseUrl === "string" ? resolve(dirname(file), expand(own.baseUrl, configDir)) : undefined;
|
|
63
|
+
if ("paths" in own) {
|
|
64
|
+
options.paths = own.paths && typeof own.paths === "object" ? own.paths : null;
|
|
65
|
+
options.pathsDir = dirname(file);
|
|
66
|
+
}
|
|
67
|
+
return options;
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
/** Folder aliases from effective options; null when none is a folder pattern ("@/*": ["./src/*"]). */
|
|
71
|
+
function aliasesOf({ paths, pathsDir, baseUrl }, root, configDir) {
|
|
72
|
+
const aliases = {};
|
|
73
|
+
for (const [pattern, targets] of Object.entries(paths ?? {})) {
|
|
74
|
+
const target = Array.isArray(targets) ? targets.find((t) => typeof t === "string") : undefined;
|
|
75
|
+
// a bare "*" would match every path
|
|
76
|
+
if (pattern === "*" || !pattern.endsWith("*") || !target?.endsWith("*")) continue;
|
|
77
|
+
const folder = relative(root, resolve(baseUrl ?? pathsDir, expand(target, configDir).slice(0, -1))).split(sep).join("/");
|
|
78
|
+
aliases[pattern.slice(0, -1)] = folder ? `${folder}/` : "";
|
|
79
|
+
}
|
|
80
|
+
return Object.keys(aliases).length ? aliases : null;
|
|
81
|
+
}
|
|
82
|
+
|
|
83
|
+
/** A config's contents, or null if it doesn't exist; a config that can't be parsed is named on stderr once. */
|
|
84
|
+
function load(file, seen) {
|
|
85
|
+
seen.set(file, mtimeOf(file));
|
|
86
|
+
let text;
|
|
87
|
+
try {
|
|
88
|
+
text = readFileSync(file, "utf8");
|
|
89
|
+
} catch {
|
|
90
|
+
return null;
|
|
91
|
+
}
|
|
92
|
+
try {
|
|
93
|
+
return parse(text);
|
|
94
|
+
} catch (error) {
|
|
95
|
+
if (!told.has(file)) {
|
|
96
|
+
told.add(file);
|
|
97
|
+
process.stderr.write(`pngdeer: couldn't read path aliases from ${file}: ${error.message}\n`);
|
|
98
|
+
}
|
|
99
|
+
return null;
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
/** The config file a path in `extends` or `references` names: as is, with ".json", or a folder's tsconfig.json. */
|
|
104
|
+
function configFile(path) {
|
|
105
|
+
for (const file of [path, `${path}.json`, join(path, "tsconfig.json")]) {
|
|
106
|
+
try {
|
|
107
|
+
if (statSync(file).isFile()) return file;
|
|
108
|
+
} catch {
|
|
109
|
+
// not this one
|
|
110
|
+
}
|
|
111
|
+
}
|
|
112
|
+
return null;
|
|
113
|
+
}
|
|
114
|
+
|
|
115
|
+
/** JSON with a BOM, comments and trailing commas, as tsconfig files are written. */
|
|
116
|
+
function parse(text) {
|
|
117
|
+
const bare = text.replace(/^\uFEFF/, "").replace(/"(?:\\.|[^"\\])*"|\/\/[^\n]*|\/\*[\s\S]*?\*\//g, (match) => (match.startsWith('"') ? match : ""));
|
|
118
|
+
return JSON.parse(bare.replace(/,(\s*[}\]])/g, "$1"));
|
|
119
|
+
}
|
|
120
|
+
|
|
121
|
+
/** TypeScript 5.5's ${configDir}: the folder of the config being resolved, not of the one that says it. */
|
|
122
|
+
function expand(path, configDir) {
|
|
123
|
+
return path.replace(/^\$\{configDir\}/, configDir);
|
|
124
|
+
}
|
|
125
|
+
|
|
126
|
+
function mtimeOf(file) {
|
|
127
|
+
try {
|
|
128
|
+
return statSync(file).mtimeMs;
|
|
129
|
+
} catch {
|
|
130
|
+
return -1;
|
|
131
|
+
}
|
|
132
|
+
}
|
package/src/worker.js
CHANGED
|
@@ -1,31 +1,41 @@
|
|
|
1
|
-
// One pngdeer engine per worker thread
|
|
2
|
-
// key of the bytes it actually read (the file may have changed since it was
|
|
3
|
-
// its own output, so that file is recognized later if someone writes it to
|
|
1
|
+
// One pngdeer engine per worker thread, doing what it's sent: { file, target } measures a file and
|
|
2
|
+
// reports sizes, the cache key of the bytes it actually read (the file may have changed since it was
|
|
3
|
+
// queued), and the key of its own output, so that file is recognized later if someone writes it to
|
|
4
|
+
// disk; { file, target, out } makes the file smaller on disk, as the command does (shrink.js).
|
|
4
5
|
import { readFileSync } from "node:fs";
|
|
5
|
-
import { parentPort
|
|
6
|
+
import { parentPort } from "node:worker_threads";
|
|
6
7
|
import { key } from "./cache.js";
|
|
7
8
|
import { compress } from "./engine.js";
|
|
8
9
|
import { mark } from "./marker.js";
|
|
10
|
+
import { plain, shrink } from "./shrink.js";
|
|
9
11
|
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
12
|
+
parentPort.on("message", ({ file, target, out }) => {
|
|
13
|
+
if (out !== undefined) {
|
|
14
|
+
let reply;
|
|
15
|
+
try {
|
|
16
|
+
reply = shrink(file, out, target);
|
|
17
|
+
} catch (error) {
|
|
18
|
+
reply = { error: plain(error) };
|
|
19
|
+
}
|
|
20
|
+
parentPort.postMessage(reply);
|
|
21
|
+
return;
|
|
22
|
+
}
|
|
13
23
|
const start = performance.now();
|
|
14
24
|
let bytes;
|
|
15
25
|
try {
|
|
16
26
|
bytes = readFileSync(file);
|
|
17
27
|
} catch (error) {
|
|
18
|
-
parentPort.postMessage({ input: null, error:
|
|
28
|
+
parentPort.postMessage({ input: null, error: plain(error) });
|
|
19
29
|
return;
|
|
20
30
|
}
|
|
21
|
-
const input = key(bytes,
|
|
31
|
+
const input = key(bytes, target);
|
|
22
32
|
try {
|
|
23
|
-
const { output, outcome, score } = compress(bytes,
|
|
33
|
+
const { output, outcome, score } = compress(bytes, target);
|
|
24
34
|
const ms = Math.round(performance.now() - start);
|
|
25
35
|
// the size the command will write, mark included, so the warning and the command agree
|
|
26
36
|
const compressed = outcome === "original" ? bytes.length : mark(output).length;
|
|
27
|
-
parentPort.postMessage({ input, original: bytes.length, compressed, outcome, score, ms, output: key(output,
|
|
37
|
+
parentPort.postMessage({ input, original: bytes.length, compressed, outcome, score, ms, output: key(output, target) });
|
|
28
38
|
} catch (error) {
|
|
29
|
-
parentPort.postMessage({ input, error:
|
|
39
|
+
parentPort.postMessage({ input, error: plain(error) });
|
|
30
40
|
}
|
|
31
41
|
});
|