@lad-sapienza/scms-core 0.1.0-alpha.11 → 0.1.0-alpha.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -89,6 +89,96 @@ A consuming site typically wires these up as `npm run add-collection` / `npm run
89
89
 
90
90
  Both commands are multilingual-aware. `scms-add-collection` optionally takes a list of locale codes and scaffolds one sample per `src/content/<name>/<locale>/` folder; `scms-add-content` detects those language folders (every sub-folder is a locale code, nothing loose at the collection root) and asks which language the new file belongs in — a single locale, a comma-separated subset, or `all`.
91
91
 
92
+ ## Image optimization
93
+
94
+ `scms-optimize-images` is a maintenance command (also a `bin`, run from the site's project root) that keeps the images in your content light: it converts every JPG/JPEG/PNG under the content folder to WebP, scales them down to a maximum size, deletes the originals and rewrites every reference to them in your `.md`/`.mdx` files. It uses [sharp](https://sharp.pixelplumbing.com/), so nothing else needs to be installed on the machine.
95
+
96
+ ```bash
97
+ npx scms-optimize-images # convert, delete the originals, rewrite references
98
+ npx scms-optimize-images --dry-run # print the estimated savings, touch nothing
99
+ npx scms-optimize-images --check # touch nothing; exit 1 if something is left to do
100
+ npx scms-optimize-images --only a.jpg b.png # limit the work to these files
101
+ ```
102
+
103
+ Wire it up in the site's `package.json`:
104
+
105
+ ```json
106
+ {
107
+ "scripts": {
108
+ "images": "scms-optimize-images",
109
+ "prebuild": "scms-optimize-images --check"
110
+ }
111
+ }
112
+ ```
113
+
114
+ `--check` never changes anything — the build must not modify or delete source files. It exits with a non-zero code (and lists the offenders) if it finds a JPG/PNG that could be converted, a WebP larger than the maximum size, or a stale reference from code (see below), so a forgotten image fails the build or the CI instead of being published at 5 MB. Fix it locally with `npm run images`, review the diff, commit.
115
+
116
+ ### What it does
117
+
118
+ - **Resize**: the long side is limited to `maxSize` (2000 px); images are only ever scaled down. EXIF orientation is applied and metadata stripped.
119
+ - **JPG → lossy WebP** at quality 82. **PNG → lossless WebP**; if the result is still larger than 50% of the original *and* the original is over 500 KB, it falls back to lossy WebP (quality 85, alpha quality 100).
120
+ - **Not lighter, not converted**: when the WebP would not be smaller than the original, the original is left as it is. The verdict is stored (file hash + settings) in `.scms-optimize-images.json` at the project root — **commit it** — so such images are neither re-encoded on every run nor reported again by `--check`. Replace the file, or change a setting, and it is evaluated again.
121
+ - **References** in `.md`/`.mdx` files (frontmatter such as `img:`, `![](…)`, relative and absolute paths) are rewritten. Absolute paths are rooted at the content folder, as in `contentAssetsIntegration`. `%20` is decoded and names are compared case-insensitively.
122
+ - **Languages**: pages in a non-default language usually have no assets of their own and reuse those of the default-language twin (`blog/en/post/index.md` citing `a.jpg` that lives in `blog/it/post/`). Such references are resolved by swapping the language folder for the default one — using the languages declared in `userConfig.i18n` (below), not fixed codes.
123
+ - **Name conflicts**: `bdus.jpg` + `bdus.png` would both become `bdus.webp`. The one cited by a `.md`/`.mdx` in the same folder (or its twin in another language) is kept, the other is deleted; if that does not single one out, both are skipped and reported.
124
+ - **WebP already larger than `maxSize`** are scaled down in place (lossless stays lossless, lossy is re-encoded at `quality`), and reported by `--check`.
125
+ - **Idempotent**: a second run changes nothing (and takes a fraction of a second).
126
+ - The report lists converted files, skipped ones (with the reason), deleted duplicates and converted WebP that nothing cites (images in `gallery/` folders are not expected to be cited).
127
+
128
+ ### References in code
129
+
130
+ Only `.md`/`.mdx` files are rewritten. Images can also be referenced from `.astro`, `.jsx`, `.tsx`, `.ts`, `.js` and `.mjs` files under `src/` (`<img src="/didattica/cover.jpg">`, `import x from '@content/…'`): those are **never rewritten** — every one that points at an image that is, or was, converted is reported with file and line, and the command exits with an error:
131
+
132
+ ```
133
+ src/pages/[locale]/didattica/index.astro:22 /didattica/cover.jpg is now broken: the image was converted → use /didattica/cover.webp
134
+ ```
135
+
136
+ The report keeps coming back (also from `--check`) until the code is fixed, so a broken link cannot slip through silently. Paths are resolved from the content folder (`/…`), from the file (`./…`, `../…`) and through the `@content/` and `@user/` aliases.
137
+
138
+ ### Configuration
139
+
140
+ All optional; defaults shown. In `src/user.config.mjs`:
141
+
142
+ ```js
143
+ export const userConfig = {
144
+ // Languages of the site. Pages in a language other than defaultLocale may reuse
145
+ // the images of the defaultLocale twin. Omit for a monolingual site.
146
+ i18n: {
147
+ defaultLocale: 'it',
148
+ locales: ['it', 'en'],
149
+ },
150
+
151
+ images: {
152
+ contentDir: 'src/content', // scanned for images and .md/.mdx files
153
+ srcDir: 'src', // scanned for references from code
154
+ maxSize: 2000, // px, long side
155
+ quality: 82, // WebP quality for JPGs (and for re-encoded lossy WebP)
156
+ pngLossyRatio: 0.5, // PNG: go lossy if lossless WebP > this fraction of the original...
157
+ pngLossyMinSize: 500, // ...and the original is larger than this many KB
158
+ pngLossyQuality: 85, // quality of that lossy fallback
159
+ exclude: [], // folders left alone: paths relative to contentDir, or plain folder names
160
+ },
161
+ };
162
+ ```
163
+
164
+ Keep `i18n` in sync with the site's own locale list (e.g. `LOCALES`/`DEFAULT_LOCALE` in `src/utils/i18n.ts`). Images outside the content folder (e.g. `public/`) are never touched.
165
+
166
+ ### Optional: pre-commit hook
167
+
168
+ Nothing is installed automatically. If you want images checked at commit time, add `.git/hooks/pre-commit` (or the equivalent in husky/lefthook) and make it executable. It only looks at the images staged in the commit:
169
+
170
+ ```sh
171
+ #!/bin/sh
172
+ # Block the commit if a staged image is not optimized
173
+ git diff --cached --name-only --diff-filter=AM -z -- '*.jpg' '*.jpeg' '*.png' '*.JPG' '*.JPEG' '*.PNG' '*.webp' \
174
+ | xargs -0 npx scms-optimize-images --check --only || {
175
+ echo "Run 'npm run images', review the changes and stage them." >&2
176
+ exit 1
177
+ }
178
+ ```
179
+
180
+ An empty list is fine: `--only` with no files does nothing and exits 0. To convert instead of just blocking, run `npx scms-optimize-images --only <files>` and then `git add` the converted `.webp` files and the rewritten `.md`/`.mdx` — the command never stages anything itself.
181
+
92
182
  ## Development
93
183
 
94
184
  ```bash
@@ -0,0 +1,440 @@
1
+ // optimize-images engine — shared by bin/optimize-images.mjs and the tests.
2
+ //
3
+ // Converts the JPG/PNG images found under the content directory to WebP (max
4
+ // `maxSize` px on the long side), deletes the originals and rewrites every
5
+ // reference to them in the .md/.mdx files. Pure Node + sharp: no side effects
6
+ // besides the file system, no console output (the CLI prints the report).
7
+ //
8
+ // See the "Image optimization" section of the package README for the rules.
9
+
10
+ import crypto from 'node:crypto';
11
+ import fs from 'node:fs';
12
+ import path from 'node:path';
13
+ import sharp from 'sharp';
14
+
15
+ export const DEFAULTS = {
16
+ /** Folder (relative to the project root) scanned for images and .md/.mdx files. */
17
+ contentDir: 'src/content',
18
+ /** Folder (relative to the project root) scanned for code files referencing images. */
19
+ srcDir: 'src',
20
+ /** Max size, in px, of the long side. Images are only ever scaled down. */
21
+ maxSize: 2000,
22
+ /** WebP quality for JPGs. */
23
+ quality: 82,
24
+ /** PNG: fall back to lossy when the lossless WebP is larger than this fraction of the original... */
25
+ pngLossyRatio: 0.5,
26
+ /** ...and the original is larger than this many KB. */
27
+ pngLossyMinSize: 500,
28
+ /** Quality of the lossy PNG fallback. */
29
+ pngLossyQuality: 85,
30
+ /** Folders (relative to contentDir, or plain folder names) left untouched. */
31
+ exclude: [],
32
+ };
33
+
34
+ export const MANIFEST_NAME = '.scms-optimize-images.json';
35
+
36
+ const IMAGE_RE = /\.(jpe?g|png)$/i;
37
+ const TEXT_RE = /\.mdx?$/i;
38
+ const CODE_RE = /\.(astro|jsx|tsx|mjs|ts|js)$/i;
39
+ // A reference token inside a markdown/frontmatter line (no whitespace, quotes or brackets)
40
+ const MD_REF_RE = /[^\s"'()<>[\]=]+\.(?:jpe?g|png)(?![\w.])/gi;
41
+ // Same for code, where the token can also be delimited by template-literal / JSX / object punctuation
42
+ const CODE_REF_RE = /[^\s"'`()<>[\]={},;]+\.(?:jpe?g|png)(?![\w.])/gi;
43
+
44
+ const posix = (p) => p.split(path.sep).join('/');
45
+ const kb = (bytes) => (bytes / 1024).toFixed(0);
46
+
47
+ /** Merges `userConfig.images` / `userConfig.i18n` (either may be undefined) over the defaults. */
48
+ export function resolveConfig(root, userConfig = {}) {
49
+ const images = { ...DEFAULTS, ...(userConfig.images ?? {}) };
50
+ const i18n = userConfig.i18n ?? {};
51
+ const locales = Array.isArray(i18n.locales) ? i18n.locales : [];
52
+ const defaultLocale = i18n.defaultLocale ?? locales[0];
53
+ return {
54
+ root,
55
+ contentRoot: path.resolve(root, images.contentDir),
56
+ srcRoot: path.resolve(root, images.srcDir),
57
+ maxSize: images.maxSize,
58
+ quality: images.quality,
59
+ pngLossyRatio: images.pngLossyRatio,
60
+ pngLossyMinSize: images.pngLossyMinSize,
61
+ pngLossyQuality: images.pngLossyQuality,
62
+ exclude: (images.exclude ?? []).map((e) => posix(e).replace(/^\.?\/+|\/+$/g, '')),
63
+ locales,
64
+ defaultLocale,
65
+ };
66
+ }
67
+
68
+ function walk(dir) {
69
+ if (!fs.existsSync(dir)) return [];
70
+ return fs
71
+ .readdirSync(dir, { withFileTypes: true })
72
+ .sort((a, b) => a.name.localeCompare(b.name))
73
+ .flatMap((e) => {
74
+ if (e.name === 'node_modules' || e.name.startsWith('.')) return [];
75
+ const p = path.join(dir, e.name);
76
+ return e.isDirectory() ? walk(p) : [p];
77
+ });
78
+ }
79
+
80
+ function isExcluded(c, rel) {
81
+ const segments = rel.split('/');
82
+ return c.exclude.some((e) => rel === e || rel.startsWith(`${e}/`) || (!e.includes('/') && segments.slice(0, -1).includes(e)));
83
+ }
84
+
85
+ // ─── Locale twins ────────────────────────────────────────────────────────────
86
+
87
+ /**
88
+ * Alternative locations of a path: pages in a non-default locale have no assets
89
+ * of their own and reuse the files of their default-locale twin, so the first
90
+ * locale segment of a non-default language is swapped for the default one.
91
+ * `relPath` is relative to the content root, posix-style.
92
+ */
93
+ function defaultLocaleTwin(c, relPath) {
94
+ if (!c.defaultLocale) return null;
95
+ const segments = relPath.split('/');
96
+ const i = segments.findIndex((s) => s !== c.defaultLocale && c.locales.includes(s));
97
+ if (i === -1) return null;
98
+ segments[i] = c.defaultLocale;
99
+ return segments.join('/');
100
+ }
101
+
102
+ /** The folder itself plus its twins in the other languages (used to find which images a page cites). */
103
+ function twinDirs(c, dir) {
104
+ const rel = posix(path.relative(c.contentRoot, dir));
105
+ const segments = rel.split('/');
106
+ const out = [dir];
107
+ const i = c.defaultLocale ? segments.indexOf(c.defaultLocale) : -1;
108
+ if (i !== -1) {
109
+ for (const l of c.locales.filter((l) => l !== c.defaultLocale)) {
110
+ const s = [...segments];
111
+ s[i] = l;
112
+ out.push(path.join(c.contentRoot, ...s));
113
+ }
114
+ }
115
+ return out;
116
+ }
117
+
118
+ // ─── Manifest (images known not to benefit) ──────────────────────────────────
119
+
120
+ const sha1 = (file) => crypto.createHash('sha1').update(fs.readFileSync(file)).digest('hex');
121
+
122
+ function settingsHash(c) {
123
+ const s = [c.maxSize, c.quality, c.pngLossyRatio, c.pngLossyMinSize, c.pngLossyQuality].join('|');
124
+ return crypto.createHash('sha1').update(s).digest('hex').slice(0, 10);
125
+ }
126
+
127
+ function loadManifest(c) {
128
+ const file = path.join(c.root, MANIFEST_NAME);
129
+ try {
130
+ const m = JSON.parse(fs.readFileSync(file, 'utf8'));
131
+ return m && m.settings === settingsHash(c) && m.kept ? m.kept : {};
132
+ } catch {
133
+ return {};
134
+ }
135
+ }
136
+
137
+ function saveManifest(c, kept) {
138
+ const file = path.join(c.root, MANIFEST_NAME);
139
+ const keys = Object.keys(kept).sort();
140
+ if (!keys.length) {
141
+ fs.rmSync(file, { force: true });
142
+ return;
143
+ }
144
+ const sorted = Object.fromEntries(keys.map((k) => [k, kept[k]]));
145
+ fs.writeFileSync(file, `${JSON.stringify({ settings: settingsHash(c), kept: sorted }, null, 2)}\n`);
146
+ }
147
+
148
+ // ─── sharp ───────────────────────────────────────────────────────────────────
149
+
150
+ // Inputs are always passed to sharp as buffers, never as paths: libvips caches by file path
151
+ // and would serve stale metadata for a file rewritten earlier in the same process.
152
+ function pipeline(input, c) {
153
+ return sharp(Buffer.isBuffer(input) ? input : fs.readFileSync(input), { failOn: 'error' })
154
+ .rotate()
155
+ .resize({ width: c.maxSize, height: c.maxSize, fit: 'inside', withoutEnlargement: true });
156
+ }
157
+
158
+ /** Encodes a JPG/PNG to WebP. Same lossless/lossy logic as the original ImageMagick script. */
159
+ async function encode(src, c) {
160
+ if (!/\.png$/i.test(src)) return pipeline(src, c).webp({ quality: c.quality }).toBuffer();
161
+ // PNGs are mostly screenshots and graphics: lossless first, lossy if that is still heavy
162
+ const original = fs.statSync(src).size;
163
+ let out = await pipeline(src, c).webp({ lossless: true, effort: 6 }).toBuffer();
164
+ if (out.length > original * c.pngLossyRatio && original > c.pngLossyMinSize * 1024) {
165
+ out = await pipeline(src, c).webp({ quality: c.pngLossyQuality, alphaQuality: 100 }).toBuffer();
166
+ }
167
+ return out;
168
+ }
169
+
170
+ /** True for a lossless WebP (VP8L chunk), false for lossy (VP8 chunk). */
171
+ function webpIsLossless(buf) {
172
+ let offset = 12;
173
+ while (offset + 8 <= buf.length) {
174
+ const fourcc = buf.toString('ascii', offset, offset + 4);
175
+ if (fourcc === 'VP8L') return true;
176
+ if (fourcc === 'VP8 ') return false;
177
+ const size = buf.readUInt32LE(offset + 4);
178
+ offset += 8 + size + (size & 1);
179
+ }
180
+ return false;
181
+ }
182
+
183
+ async function resizeWebp(file, c) {
184
+ const buf = fs.readFileSync(file);
185
+ const opts = webpIsLossless(buf) ? { lossless: true, effort: 6 } : { quality: c.quality, alphaQuality: 100 };
186
+ return { before: buf.length, out: await pipeline(buf, c).webp(opts).toBuffer() };
187
+ }
188
+
189
+ async function pixelSize(file) {
190
+ const m = await sharp(fs.readFileSync(file), { failOn: 'error' }).metadata();
191
+ return { max: Math.max(m.width ?? 0, m.height ?? 0), animated: (m.pages ?? 1) > 1 };
192
+ }
193
+
194
+ // ─── Main ────────────────────────────────────────────────────────────────────
195
+
196
+ /**
197
+ * @param {ReturnType<typeof resolveConfig>} c
198
+ * @param {{ dryRun?: boolean, check?: boolean, only?: string[] | null }} [mode]
199
+ * `check`: touch nothing and convert nothing, just list what is left to do.
200
+ * `only`: restrict the images that get converted/checked to these files.
201
+ */
202
+ export async function optimizeImages(c, { dryRun = false, check = false, only = null } = {}) {
203
+ const readOnly = dryRun || check;
204
+ const rel = (f) => posix(path.relative(c.contentRoot, f));
205
+ const onlySet = only ? new Set(only.map((f) => path.resolve(f).toLowerCase())) : null;
206
+ const inScope = (f) => !onlySet || onlySet.has(f.toLowerCase());
207
+
208
+ const report = {
209
+ converted: [], // { rel, before, after }
210
+ skipped: [], // { rel, reason }
211
+ dropped: [], // unused duplicates (rel)
212
+ resized: [], // oversized WebPs scaled down: { rel, before, after }
213
+ pending: [], // --check: JPG/PNG still to convert (rel)
214
+ oversized: [], // --check: WebP larger than maxSize (rel)
215
+ unreferenced: [],
216
+ codeRefs: [], // { file, line, ref, kind, replacement }
217
+ keptCount: 0, // already known not to benefit, left alone
218
+ rewritten: 0,
219
+ before: 0,
220
+ after: 0,
221
+ problems: 0,
222
+ };
223
+
224
+ const all = walk(c.contentRoot).filter((f) => !isExcluded(c, rel(f)));
225
+ const textFiles = all.filter((f) => TEXT_RE.test(f));
226
+ const allImages = all.filter((f) => IMAGE_RE.test(f));
227
+ const webps = all.filter((f) => /\.webp$/i.test(f));
228
+
229
+ const kept = loadManifest(c);
230
+ const keptKey = (f) => rel(f);
231
+ const isKept = (f) => {
232
+ const e = kept[keptKey(f)];
233
+ return Boolean(e) && e.sha1 === sha1(f);
234
+ };
235
+ const markKept = (f, reason) => {
236
+ kept[keptKey(f)] = { sha1: sha1(f), reason };
237
+ };
238
+
239
+ const converted = new Map(); // old absolute path -> new absolute path
240
+
241
+ if (check) {
242
+ for (const f of allImages) {
243
+ if (!inScope(f)) continue;
244
+ if (isKept(f)) report.keptCount++;
245
+ else report.pending.push(rel(f));
246
+ }
247
+ for (const f of webps) {
248
+ if (!inScope(f) || isKept(f)) continue;
249
+ const { max, animated } = await pixelSize(f).catch(() => ({ max: 0, animated: true }));
250
+ if (!animated && max > c.maxSize) report.oversized.push(rel(f));
251
+ }
252
+ for (const f of allImages.filter((f) => inScope(f) && !isKept(f))) converted.set(f, f.replace(IMAGE_RE, '.webp'));
253
+ } else {
254
+ // Same name, different extension in one folder (e.g. bdus.jpg + bdus.png): they would both
255
+ // become bdus.webp. Keep only the one a sibling .md/.mdx (or its locale twin) points to.
256
+ let images = allImages;
257
+ const byTarget = new Map();
258
+ for (const f of images) {
259
+ const k = f.replace(IMAGE_RE, '');
260
+ byTarget.set(k, [...(byTarget.get(k) ?? []), f]);
261
+ }
262
+ for (const group of [...byTarget.values()].filter((g) => g.length > 1)) {
263
+ const dirs = twinDirs(c, path.dirname(group[0]));
264
+ const text = dirs
265
+ .flatMap((d) => textFiles.filter((t) => path.dirname(t) === d))
266
+ .map((t) => fs.readFileSync(t, 'utf8'))
267
+ .join('\n');
268
+ const used = group.filter((g) => text.includes(path.basename(g)));
269
+ if (used.length !== 1) continue; // ambiguous: reported as skipped below
270
+ for (const unused of group.filter((g) => g !== used[0])) {
271
+ if (!inScope(unused)) continue;
272
+ report.dropped.push(rel(unused));
273
+ if (!readOnly) fs.rmSync(unused);
274
+ }
275
+ images = images.filter((i) => !group.includes(i) || i === used[0]);
276
+ }
277
+
278
+ for (const src of images) {
279
+ if (!inScope(src)) continue;
280
+ const dest = src.replace(IMAGE_RE, '.webp');
281
+ const name = rel(src);
282
+
283
+ if (isKept(src)) {
284
+ report.keptCount++;
285
+ continue;
286
+ }
287
+ if (fs.existsSync(dest) || images.some((o) => o !== src && o.replace(IMAGE_RE, '.webp') === dest)) {
288
+ report.skipped.push({ rel: name, reason: 'target .webp already exists or name clashes with another image' });
289
+ continue;
290
+ }
291
+
292
+ const origSize = fs.statSync(src).size;
293
+ let out;
294
+ try {
295
+ const { animated } = await pixelSize(src);
296
+ if (animated) {
297
+ report.skipped.push({ rel: name, reason: 'animated image, left as is' });
298
+ continue;
299
+ }
300
+ out = await encode(src, c);
301
+ } catch (err) {
302
+ report.skipped.push({ rel: name, reason: `conversion failed (${err.message.split('\n')[0]})` });
303
+ continue;
304
+ }
305
+ if (out.length >= origSize) {
306
+ report.skipped.push({ rel: name, reason: `WebP is not smaller (${kb(origSize)} -> ${kb(out.length)} KB), kept as is` });
307
+ if (!readOnly) markKept(src, 'webp-not-smaller');
308
+ continue;
309
+ }
310
+
311
+ report.before += origSize;
312
+ report.after += out.length;
313
+ converted.set(src, dest);
314
+ report.converted.push({ rel: name, before: origSize, after: out.length });
315
+ if (!readOnly) {
316
+ const tmp = `${dest}.tmp`;
317
+ fs.writeFileSync(tmp, out);
318
+ fs.renameSync(tmp, dest);
319
+ fs.rmSync(src);
320
+ }
321
+ }
322
+
323
+ // WebPs that were already there but exceed the max size are scaled down in place
324
+ // (same mode as the source: lossless stays lossless, lossy is re-encoded at `quality`).
325
+ for (const f of webps) {
326
+ if (!inScope(f)) continue;
327
+ if (isKept(f)) {
328
+ report.keptCount++;
329
+ continue;
330
+ }
331
+ try {
332
+ const { max, animated } = await pixelSize(f);
333
+ if (animated || max <= c.maxSize) continue;
334
+ const { before, out } = await resizeWebp(f, c);
335
+ if (out.length >= before) {
336
+ report.skipped.push({ rel: rel(f), reason: 'oversized WebP, but scaling it down does not make it smaller, kept as is' });
337
+ if (!readOnly) markKept(f, 'resize-not-smaller');
338
+ continue;
339
+ }
340
+ report.resized.push({ rel: rel(f), before, after: out.length });
341
+ if (!readOnly) fs.writeFileSync(f, out);
342
+ } catch (err) {
343
+ report.skipped.push({ rel: rel(f), reason: `could not inspect WebP (${err.message.split('\n')[0]})` });
344
+ }
345
+ }
346
+
347
+ if (!readOnly) {
348
+ // prune entries whose file is gone (or no longer the one that was evaluated)
349
+ for (const k of Object.keys(kept)) {
350
+ const abs = path.join(c.contentRoot, ...k.split('/'));
351
+ if (!fs.existsSync(abs) || sha1(abs) !== kept[k].sha1) delete kept[k];
352
+ }
353
+ saveManifest(c, kept);
354
+ }
355
+
356
+ // Rewrite references in .md/.mdx
357
+ const lookup = new Map([...converted].map(([o, n]) => [o.toLowerCase(), n]));
358
+ for (const file of textFiles) {
359
+ const dir = path.dirname(file);
360
+ const text = fs.readFileSync(file, 'utf8');
361
+ const next = text.replace(MD_REF_RE, (ref) => {
362
+ if (/^(https?:)?\/\//i.test(ref)) return ref;
363
+ let decoded = ref;
364
+ try {
365
+ decoded = decodeURIComponent(ref);
366
+ } catch {
367
+ /* keep raw */
368
+ }
369
+ // Relative to the file, or absolute from the content root (see contentAssetsIntegration)
370
+ const target = decoded.startsWith('/') ? path.join(c.contentRoot, decoded) : path.resolve(dir, decoded);
371
+ // Pages in a non-default language have no assets of their own: they reuse the twin's files
372
+ const twinRel = defaultLocaleTwin(c, rel(target));
373
+ const twin = twinRel ? path.join(c.contentRoot, ...twinRel.split('/')) : null;
374
+ const hit = lookup.get(target.toLowerCase()) ?? (twin ? lookup.get(twin.toLowerCase()) : undefined);
375
+ return hit ? ref.replace(IMAGE_RE, '.webp') : ref;
376
+ });
377
+ if (next !== text) {
378
+ report.rewritten++;
379
+ if (!readOnly) fs.writeFileSync(file, next);
380
+ }
381
+ }
382
+ }
383
+
384
+ // ─── References from code (reported, never rewritten) ──────────────────────
385
+ const codeFiles = walk(c.srcRoot).filter((f) => CODE_RE.test(f) && !f.startsWith(c.contentRoot + path.sep));
386
+ const existing = new Set(all.map((f) => f.toLowerCase()));
387
+ const convertedLower = new Map([...converted].map(([o, n]) => [o.toLowerCase(), n]));
388
+ const kind = check || dryRun ? 'to-convert' : 'converted';
389
+
390
+ for (const file of codeFiles) {
391
+ const dir = path.dirname(file);
392
+ const lines = fs.readFileSync(file, 'utf8').split('\n');
393
+ lines.forEach((line, i) => {
394
+ for (const [ref] of line.matchAll(CODE_REF_RE)) {
395
+ if (/^(https?:)?\/\//i.test(ref)) continue;
396
+ let decoded = ref;
397
+ try {
398
+ decoded = decodeURIComponent(ref);
399
+ } catch {
400
+ /* keep raw */
401
+ }
402
+ const candidates = decoded.startsWith('@content/')
403
+ ? [path.join(c.contentRoot, decoded.slice(9))]
404
+ : decoded.startsWith('@user/')
405
+ ? [path.join(c.srcRoot, decoded.slice(6))]
406
+ : decoded.startsWith('/')
407
+ ? [path.join(c.contentRoot, decoded)]
408
+ : [path.resolve(dir, decoded), path.join(c.contentRoot, decoded)];
409
+ let found = null;
410
+ for (const cand of candidates) {
411
+ const lower = cand.toLowerCase();
412
+ if (convertedLower.has(lower)) {
413
+ found = kind;
414
+ break;
415
+ }
416
+ // a JPG/PNG that no longer exists although its .webp does: left over from an earlier conversion
417
+ if (!existing.has(lower) && existing.has(lower.replace(IMAGE_RE, '.webp'))) {
418
+ found = 'stale';
419
+ break;
420
+ }
421
+ }
422
+ if (found) {
423
+ report.codeRefs.push({ file: posix(path.relative(c.root, file)), line: i + 1, ref, kind: found, replacement: ref.replace(IMAGE_RE, '.webp') });
424
+ }
425
+ }
426
+ });
427
+ }
428
+
429
+ // Sanity check: every converted file must be referenced or live in a gallery/ folder
430
+ if (!readOnly) {
431
+ const corpus = [...textFiles, ...codeFiles].map((f) => fs.readFileSync(f, 'utf8')).join('\n');
432
+ report.unreferenced = [...converted.values()]
433
+ .filter((d) => !rel(d).split('/').includes('gallery'))
434
+ .filter((d) => !corpus.includes(path.basename(d)))
435
+ .map(rel);
436
+ }
437
+
438
+ report.problems = check ? report.pending.length + report.oversized.length + report.codeRefs.length : report.codeRefs.length;
439
+ return report;
440
+ }
@@ -0,0 +1,95 @@
1
+ #!/usr/bin/env node
2
+ // optimize-images.mjs — Convert content images to WebP and rewrite their references.
3
+ // Usage: npx scms-optimize-images [--dry-run | --check] [--only <file>...]
4
+ // or: npm run images (from a consuming site's package.json)
5
+
6
+ import fs from 'node:fs';
7
+ import path from 'node:path';
8
+ import { pathToFileURL } from 'node:url';
9
+ import { optimizeImages, resolveConfig } from './lib/optimize-images.mjs';
10
+
11
+ const G = '\x1b[32m', Y = '\x1b[33m', R = '\x1b[31m';
12
+ const B = '\x1b[1m', X = '\x1b[0m';
13
+
14
+ const HELP = `scms-optimize-images — convert content JPG/PNG images to WebP
15
+
16
+ scms-optimize-images convert, delete the originals, rewrite references
17
+ scms-optimize-images --dry-run report what would happen, touch nothing
18
+ scms-optimize-images --check touch nothing; exit 1 if anything is left to convert
19
+ (JPG/PNG, WebP larger than the max size, stale code references).
20
+ Meant for "prebuild" scripts and CI.
21
+ scms-optimize-images --only <files> limit conversion/checks to these image files
22
+ (e.g. the ones staged in a pre-commit hook)
23
+
24
+ Settings live in src/user.config.mjs (userConfig.images, userConfig.i18n); see the scms-core README.`;
25
+
26
+ const args = process.argv.slice(2);
27
+ if (args.includes('--help') || args.includes('-h')) {
28
+ console.log(HELP);
29
+ process.exit(0);
30
+ }
31
+ const dryRun = args.includes('--dry-run');
32
+ const check = args.includes('--check');
33
+ const onlyIdx = args.indexOf('--only');
34
+ const only = onlyIdx === -1 ? null : args.slice(onlyIdx + 1).filter((a) => !a.startsWith('--'));
35
+ const unknown = args.filter((a) => a.startsWith('-') && !['--dry-run', '--check', '--only'].includes(a));
36
+ if (unknown.length) {
37
+ console.error(`${R}✖${X} Unknown option: ${unknown.join(' ')}\n\n${HELP}`);
38
+ process.exit(2);
39
+ }
40
+ if (only && only.length === 0) {
41
+ console.log('No files given to --only, nothing to do.');
42
+ process.exit(0);
43
+ }
44
+
45
+ // This file ships inside the @lad-sapienza/scms-core package (bin/): the project is wherever it was run from.
46
+ const ROOT = process.cwd();
47
+ const configFile = path.join(ROOT, 'src', 'user.config.mjs');
48
+ let userConfig = {};
49
+ if (fs.existsSync(configFile)) {
50
+ try {
51
+ userConfig = (await import(pathToFileURL(configFile).href)).userConfig ?? {};
52
+ } catch (err) {
53
+ console.error(`${R}✖${X} Cannot load src/user.config.mjs: ${err.message}`);
54
+ process.exit(2);
55
+ }
56
+ }
57
+
58
+ const config = resolveConfig(ROOT, userConfig);
59
+ if (!fs.existsSync(config.contentRoot)) {
60
+ console.error(`${R}✖${X} Content folder not found: ${config.contentRoot}`);
61
+ process.exit(2);
62
+ }
63
+
64
+ const r = await optimizeImages(config, { dryRun, check, only });
65
+
66
+ const kb = (n) => (n / 1024).toFixed(0);
67
+ const mb = (n) => (n / 1048576).toFixed(1);
68
+ const list = (title, items) => items.length && console.log(`\n${title} (${items.length}):\n ${items.join('\n ')}`);
69
+ const tag = dryRun ? '[dry run] ' : '';
70
+
71
+ if (check) {
72
+ list(`${R}JPG/PNG images to convert${X}`, r.pending);
73
+ list(`${R}WebP images larger than ${config.maxSize}px${X}`, r.oversized);
74
+ } else {
75
+ for (const c of r.converted) console.log(`${c.rel}: ${kb(c.before)} -> ${kb(c.after)} KB`);
76
+ for (const c of r.resized) console.log(`${c.rel}: scaled to max ${config.maxSize}px, ${kb(c.before)} -> ${kb(c.after)} KB`);
77
+ console.log(`\n${tag}${r.converted.length} images converted, ${r.resized.length} resized, ${r.rewritten} text files updated`);
78
+ console.log(`Size: ${mb(r.before)} MB -> ${mb(r.after)} MB${r.before ? ` (-${(100 - (r.after / r.before) * 100).toFixed(0)}%)` : ''}`);
79
+ list(`${B}Unused duplicates ${dryRun ? 'to delete' : 'deleted'}${X}`, r.dropped);
80
+ list(`${Y}Skipped${X}`, r.skipped.map((s) => `${s.rel}: ${s.reason}`));
81
+ list(`${Y}Converted but not referenced anywhere (check)${X}`, r.unreferenced);
82
+ if (r.keptCount) console.log(`\n${r.keptCount} image(s) already evaluated and kept as they are (see .scms-optimize-images.json)`);
83
+ }
84
+
85
+ if (r.codeRefs.length) {
86
+ const verb = { 'to-convert': 'will break once the image is converted', converted: 'is now broken: the image was converted', stale: 'points to an image that was converted earlier' };
87
+ console.error(`\n${R}✖${X} ${r.codeRefs.length} reference(s) in code must be updated by hand (code is never rewritten):`);
88
+ for (const c of r.codeRefs) console.error(` ${c.file}:${c.line} ${c.ref} ${verb[c.kind]} → use ${c.replacement}`);
89
+ }
90
+
91
+ if (r.problems) {
92
+ console.error(`\n${R}✖${X} ${r.problems} problem(s) found.${check ? ' Run `npx scms-optimize-images` to fix the images.' : ''}`);
93
+ process.exit(1);
94
+ }
95
+ if (check) console.log(`${G}✔${X} Images are optimized.`);
@@ -78,6 +78,8 @@ export function Map({
78
78
  csv,
79
79
  json,
80
80
  directus,
81
+ markers,
82
+ markersZoom,
81
83
  }: MapProps) {
82
84
  const t = useTranslation();
83
85
  const [lng, lat, zoom] = center.split(',').map(Number);
@@ -323,8 +325,29 @@ export function Map({
323
325
 
324
326
  layers.push(layerConfig);
325
327
  }
328
+
329
+ if (markers) {
330
+ const list = Array.isArray(markers) ? markers : [markers];
331
+ const features = list
332
+ .filter(m => Number.isFinite(Number(m.lng)) && Number.isFinite(Number(m.lat)))
333
+ .map(m => ({
334
+ type: 'Feature' as const,
335
+ geometry: { type: 'Point' as const, coordinates: [Number(m.lng), Number(m.lat)] },
336
+ properties: { label: m.label ?? '' }
337
+ }));
338
+ if (features.length > 0) {
339
+ layers.push({
340
+ name: t('map.markers'),
341
+ source: { type: 'geojson', data: { type: 'FeatureCollection', features } },
342
+ popupTemplate: list.some(m => m.label) ? '<b>${label}</b>' : undefined,
343
+ fitToContent: true,
344
+ fitZoom: markersZoom,
345
+ visible: true
346
+ } as OverlayLayerConfig);
347
+ }
348
+ }
326
349
  return layers;
327
- }, [overlayLayers, implicitSource, geojson, csv, mapStyle, parsedStyleLayers.overlayLayers]);
350
+ }, [overlayLayers, implicitSource, geojson, csv, markers, markersZoom, mapStyle, parsedStyleLayers.overlayLayers]);
328
351
 
329
352
  // Initialize and update visibility when layers change
330
353
  useEffect(() => {
@@ -479,7 +502,15 @@ export function Map({
479
502
  [Math.min(...lons), Math.min(...lats)],
480
503
  [Math.max(...lons), Math.max(...lats)]
481
504
  ];
482
- map.fitBounds(bounds, { padding: 50, duration: 1000 });
505
+ const [[w, s], [e, n]] = bounds;
506
+ if (layerToFit.fitZoom !== undefined) {
507
+ map.easeTo({ center: [(w + e) / 2, (s + n) / 2], zoom: layerToFit.fitZoom, duration: 1000 });
508
+ } else if (w === e && s === n) {
509
+ // A lone point has no extent: fitBounds would zoom to maxZoom.
510
+ map.easeTo({ center: [w, s], zoom: Math.max(map.getZoom(), 12), duration: 1000 });
511
+ } else {
512
+ map.fitBounds(bounds, { padding: 50, duration: 1000 });
513
+ }
483
514
  }
484
515
  }, 100);
485
516
  }, [mapLoaded, layersData, allOverlayLayers]);
@@ -22,6 +22,7 @@ export function Map(props: MapProps) {
22
22
  const stableGeojson = useMemo(() => props?.geojson, [JSON.stringify(props?.geojson || null)]);
23
23
  const stableCsv = useMemo(() => props?.csv, [JSON.stringify(props?.csv || null)]);
24
24
  const stableJson = useMemo(() => props?.json, [JSON.stringify(props?.json || null)]);
25
+ const stableMarkers = useMemo(() => props?.markers, [JSON.stringify(props?.markers || null)]);
25
26
  // `directus` feeds the same `implicitSource` → `allOverlayLayers` memo chain
26
27
  // as `geojson`/`csv`/`json`; without stabilising it here an inline
27
28
  // `directus={{ table: '…' }}` object from MDX gets a new identity on every
@@ -38,6 +39,7 @@ export function Map(props: MapProps) {
38
39
  csv={stableCsv}
39
40
  json={stableJson}
40
41
  directus={stableDirectus}
42
+ markers={stableMarkers}
41
43
  />
42
44
  );
43
45
  }
@@ -31,6 +31,30 @@ touching `overlayLayers` directly:
31
31
  <Map directus={{ table: 'sites', geoField: 'location' }} />
32
32
  ```
33
33
 
34
+ For one or a few hand-placed points (e.g. the location of a single site) use
35
+ `markers` — no data file needed. The map fits to them automatically; a lone
36
+ marker is centred at a regional zoom (≥ 12) rather than zoomed to the max:
37
+
38
+ ```mdx
39
+ <Map markers={{ lng: 12.4964, lat: 41.9028, label: 'Roma' }} />
40
+
41
+ <Map markers={[
42
+ { lng: 12.49, lat: 41.90, label: 'Site A' },
43
+ { lng: 12.51, lat: 41.88, label: 'Site B' },
44
+ ]} />
45
+ ```
46
+
47
+ `label` is optional (shown in a popup as plain text). Markers are an extra
48
+ overlay layer and combine with any of the shorthands below.
49
+
50
+ `markersZoom` fixes the zoom: the map is centred on the markers at that level
51
+ instead of being fitted to their extent — useful to give a single site the
52
+ same geographic context on every page:
53
+
54
+ ```mdx
55
+ <Map markers={{ lng: 12.4964, lat: 41.9028, label: 'Roma' }} markersZoom={9} />
56
+ ```
57
+
34
58
  `geojson`/`csv`/`json`/`directus` each accept either a bare string (path/URL)
35
59
  or an object with `name`, `style`, `popup`, `fitToContent`, `filter` — see
36
60
  `MapProps` in [`types.ts`](./types.ts) for the exact shape of each.
@@ -124,6 +124,12 @@ export interface OverlayLayerConfig {
124
124
  /** Fit map bounds to layer content */
125
125
  fitToContent?: boolean;
126
126
 
127
+ /**
128
+ * With `fitToContent`: centre on the content's bounds at this fixed zoom
129
+ * instead of fitting them (used by the `markers` shorthand).
130
+ */
131
+ fitZoom?: number;
132
+
127
133
  /** Optional client-side filter for features */
128
134
  filter?: FilterObject | ((feature: any) => boolean);
129
135
 
@@ -131,6 +137,17 @@ export interface OverlayLayerConfig {
131
137
  searchInFields?: SearchInFields;
132
138
  }
133
139
 
140
+ /**
141
+ * A single point marker for the `markers` shorthand — for maps with one or a
142
+ * few hand-placed points, no data file needed.
143
+ */
144
+ export interface MarkerConfig {
145
+ lng: number;
146
+ lat: number;
147
+ /** Text shown in the popup on hover/click (plain text, HTML-escaped) */
148
+ label?: string;
149
+ }
150
+
134
151
  export interface MapProps {
135
152
  /** Map height (CSS string) */
136
153
  height?: string;
@@ -197,6 +214,19 @@ export interface MapProps {
197
214
  filter?: FilterObject | ((row: any) => boolean);
198
215
  };
199
216
 
217
+ /**
218
+ * Inline markers - one or a few hand-placed points, e.g. to give the
219
+ * geographic context of a single site. Rendered as an extra overlay layer
220
+ * and, by default, the map is fitted to them.
221
+ */
222
+ markers?: MarkerConfig | MarkerConfig[];
223
+
224
+ /**
225
+ * Fixed zoom for the `markers`: the map is centred on them at this zoom
226
+ * instead of being fitted to their extent. Handy for a single site.
227
+ */
228
+ markersZoom?: number;
229
+
200
230
  /** JSON shorthand - for simple single-layer maps */
201
231
  json?: any[] | string;
202
232
 
@@ -28,6 +28,7 @@ export const BUILTIN_DICTIONARY: Dictionary = {
28
28
  },
29
29
  map: {
30
30
  defaultLayerName: 'Default Layer',
31
+ markers: 'Markers',
31
32
  baseMaps: 'Base Maps',
32
33
  searchLayer: 'Search {name}',
33
34
  active: '(Active)',
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@lad-sapienza/scms-core",
3
- "version": "0.1.0-alpha.11",
3
+ "version": "0.1.0-alpha.13",
4
4
  "description": "Framework components and Astro integrations for s:CMS — DataTb, Gallery, Map, ZoteroGeoViewer, Record.",
5
5
  "type": "module",
6
6
  "license": "BSD-0-Clause",
@@ -13,7 +13,8 @@
13
13
  "bin": {
14
14
  "scms-add-collection": "./bin/add-collection.mjs",
15
15
  "scms-add-content": "./bin/add-content.mjs",
16
- "scms-create": "./bin/create.mjs"
16
+ "scms-create": "./bin/create.mjs",
17
+ "scms-optimize-images": "./bin/optimize-images.mjs"
17
18
  },
18
19
  "files": [
19
20
  "index.ts",
@@ -54,6 +55,7 @@
54
55
  "papaparse": "^5.5.3",
55
56
  "photoswipe": "^5.4.4",
56
57
  "react-photoswipe-gallery": "^4.0.0",
58
+ "sharp": "^0.35.3",
57
59
  "zod": "^4.3.6"
58
60
  },
59
61
  "devDependencies": {
@@ -14,6 +14,19 @@ Built with [s:CMS](https://github.com/lad-sapienza/scms-core), on top of [Astro]
14
14
  | `npm run preview` | Preview the production build locally |
15
15
  | `npm run add-collection` | Scaffold a new content collection |
16
16
  | `npm run add-content` | Add a new content file to an existing collection |
17
+ | `npm run images` | Convert content images to WebP (max 2000 px) and rewrite their references |
18
+
19
+ ## Optimizing images
20
+
21
+ Drop JPG/PNG images next to your content as usual, then run `npm run images`: they are converted to WebP, scaled down to 2000 px, the originals are deleted and the `.md`/`.mdx` references are rewritten (`--dry-run` shows the savings without touching anything). References from `.astro`/`.tsx` files are only reported — update them by hand.
22
+
23
+ To make the build fail when an unoptimized image slips in, add a check to `package.json`; it never modifies any file:
24
+
25
+ ```json
26
+ "prebuild": "scms-optimize-images --check"
27
+ ```
28
+
29
+ Limits, quality and the site languages are configured in `src/user.config.mjs` (`userConfig.images`, `userConfig.i18n`); see the [scms-core README](https://github.com/lad-sapienza/scms-core#image-optimization) for all the options, including an optional pre-commit hook. Commit the `.scms-optimize-images.json` file it may create.
17
30
 
18
31
  ## Updating the framework
19
32
 
@@ -10,7 +10,8 @@
10
10
  "preview": "astro preview",
11
11
  "astro": "astro",
12
12
  "add-collection": "scms-add-collection",
13
- "add-content": "scms-add-content"
13
+ "add-content": "scms-add-content",
14
+ "images": "scms-optimize-images"
14
15
  },
15
16
  "dependencies": {
16
17
  "@astrojs/check": "^0.9.10",