ultimate-jekyll-manager 1.5.0 → 1.6.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -14,6 +14,50 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
14
14
  - `Fixed` for any bug fixes.
15
15
  - `Security` in case of vulnerabilities.
16
16
 
17
+ ---
18
+ ## [1.6.1] - 2026-06-02
19
+
20
+ ### Changed
21
+
22
+ - **Translation system overhauled: JSON array format, gpt-5.4-nano model, .env API key.** Replaced tagged text `[0]...[/0]` format with JSON arrays (eliminates tag corruption). Upgraded from `gpt-4.1-mini` to `gpt-5.4-nano` (3.5x cheaper). API key now read from `BACKEND_MANAGER_OPENAI_API_KEY` in `.env` instead of remote `fetchOpenAIKey()`. Uses GPT-5+ Responses API (`developer` role, `reasoning: { effort: 'low' }`).
23
+ - **Translation concurrency: node-powertools `queue()` replaces batched delays.** Removed 25-page batches with 10s sleeps; now uses `queue({ concurrency: 5 })` for steady throughput.
24
+ - **Translation retries simplified.** Removed recursive subdivision system. On JSON array length mismatch, retries up to `MAX_RETRIES` (2) with file/language context in logs.
25
+
26
+ ### Added
27
+
28
+ - **`data-uj-no-translate` HTML attribute** — marks DOM elements (and children) to skip during translation. Applied to country and phone code `<select>` dropdowns on the account page, reducing account.html from ~1241 to ~454 translatable strings.
29
+ - **`BACKEND_MANAGER_OPENAI_API_KEY`** added to default `.env` template.
30
+ - **Prompt cache HIT/MISS logging** with entry counts per language.
31
+ - **`IGNORE_EXCEPTIONS`** — allows specific pages through folder-level exclusions (e.g. `test/translation.html` survives the `test/` folder exclusion).
32
+
33
+ ### Removed
34
+
35
+ - **`fetchOpenAIKey()`** — remote API key fetch from `api.itwcreativeworks.com` removed.
36
+ - **Subdivision retry system** (`subdivideAndTranslate`) — replaced by simple length-mismatch retry.
37
+ - **`test/`, `team/`, `updates/` folders** excluded from translation by default.
38
+
39
+ ---
40
+ ## [1.6.0] - 2026-06-02
41
+
42
+ ### Changed
43
+
44
+ - **`getEnvironment()` is now the single source of truth for environment detection** (and `testing` is a first-class environment). [src/utils/mode-helpers.js](src/utils/mode-helpers.js) — `getEnvironment()` is the ONLY function that reads the raw signals (`UJ_TEST_MODE` / `window.Configuration.uj.environment` / `UJ_BUILD_MODE` / `UJ_IS_SERVER` / `NODE_ENV`) and resolves them to exactly one of `'development' | 'testing' | 'production'` (mutually exclusive; **testing wins**). `isDevelopment()` / `isProduction()` / `isTesting()` now **derive** from it, so they can never disagree and exactly one is always true. `isProduction()` is a real positive check, not `!isDevelopment()`. [src/build.js](src/build.js) drops its own `getEnvironment()` (now mixed in via `attachTo`). Doc renamed `cross-context-helpers.md` → [docs/environment-detection.md](docs/environment-detection.md).
45
+ - **Redirect delay now keys off non-production (dev OR testing), not dev-only.** [src/assets/js/modules/redirect.js](src/assets/js/modules/redirect.js) delays the redirect whenever `environment !== 'production'` so it's observable in tests too.
46
+ - **Footer language dropdown gates the extra-languages list on `translation.enabled`** (+ vertical-alignment fix). [footer.html](src/defaults/dist/_includes/themes/classy/frontend/sections/footer.html)
47
+
48
+ ### Added
49
+
50
+ - **`test/_init.js` pre-test lifecycle hook (setup-only).** [src/test/runner.js](src/test/runner.js) loads an optional `test/_init.js` from BOTH the framework and consumer test roots and runs its `setup()` once before any suite (the `_`-prefix keeps it out of discovery). Mirrors BEM/EM/BXM. Default scaffold at [src/defaults/test/_init.js](src/defaults/test/_init.js). [docs/test-framework.md](docs/test-framework.md) adds a "NEVER mock — test against the real harness" section + `_init.js` reference, and tests now assert the env invariant across every signal scenario.
51
+ - **`translation.exclude` config** — folder or page paths excluded from AI translation (added to both the files and folders match sets). Default [_config.yml](src/defaults/src/_config.yml) excludes `blog`. [src/gulp/tasks/translation.js](src/gulp/tasks/translation.js)
52
+ - **`setup` prunes legacy first-name-only default team members** (e.g. `team/alex`) and their image dirs via `removeLegacyTeamMembers()`. [src/commands/setup.js](src/commands/setup.js)
53
+ - **`npx mgr install live`** accepted as an alias for `prod`/`production`. [src/commands/install.js](src/commands/install.js); docs use `install dev` / `install live` consistently.
54
+
55
+ ### Fixed
56
+
57
+ - **Download buttons use a `[data-download]` hook** instead of the brittle `.btn-primary:not([type="submit"])` selector. [download.html](src/defaults/dist/_layouts/themes/classy/frontend/pages/download.html) + [download/index.js](src/assets/js/pages/download/index.js)
58
+ - **Feedback review modal:** $100 gift-card incentive + an explicit "you must actually post your review" warning, and copy normalized "Write a review" → "Post your review". [feedback.html](src/defaults/dist/_layouts/themes/classy/frontend/pages/feedback.html) + [feedback/index.js](src/assets/js/pages/feedback/index.js)
59
+ - Removed a leftover `package copy.json` from the repo root.
60
+
17
61
  ---
18
62
  ## [1.5.0] - 2026-06-02
19
63
 
@@ -17,12 +17,6 @@ export default () => {
17
17
  setupForm();
18
18
  setupRatingButtons();
19
19
 
20
- // TEMP: expose for manual modal testing — remove before commit
21
- window.__testReviewModal = () => showReviewModal('https://www.trustpilot.com/review/example.com', {
22
- positive: 'This product is amazing and saved me hours!',
23
- comments: 'Highly recommend to anyone.',
24
- });
25
-
26
20
  // Resolve after initialization
27
21
  return resolve();
28
22
  });
File without changes
File without changes
File without changes
@@ -3,6 +3,9 @@
3
3
  # Get token at: https://github.com/settings/tokens
4
4
  GH_TOKEN=""
5
5
 
6
+ # Backend Manager
7
+ BACKEND_MANAGER_OPENAI_API_KEY=""
8
+
6
9
  # ========== Custom Values ==========
7
10
  # Add your custom environment variables below this line
8
11
  # ...
@@ -340,7 +340,7 @@ badges:
340
340
  <div class="row g-3 mb-3">
341
341
  <div class="col-md-6">
342
342
  <label for="country-select" class="form-label small text-muted">Country</label>
343
- <select class="form-select" id="country-select" name="personal.location.country" autocomplete="country">
343
+ <select class="form-select" id="country-select" name="personal.location.country" autocomplete="country" data-uj-no-translate>
344
344
  <option value="">Select country</option>
345
345
  <option value="AF">Afghanistan</option>
346
346
  <option value="AL">Albania</option>
@@ -551,7 +551,7 @@ badges:
551
551
  <div class="col-md-6">
552
552
  <label for="phone-input" class="form-label small text-muted">Phone</label>
553
553
  <div class="input-group">
554
- <select class="form-select flex-shrink-1" id="phone-country-code" name="personal.telephone.countryCode" autocomplete="tel-country-code" style="max-width: 140px;">
554
+ <select class="form-select flex-shrink-1" id="phone-country-code" name="personal.telephone.countryCode" autocomplete="tel-country-code" style="max-width: 140px;" data-uj-no-translate>
555
555
  <option value="+1">+1 (USA/CAN)</option>
556
556
  <option value="+7">+7 (RUS/KAZ)</option>
557
557
  <option value="+20">+20 (EGY)</option>
@@ -9,7 +9,7 @@ const jetpack = require('fs-jetpack');
9
9
  const cheerio = require('cheerio');
10
10
  const crypto = require('crypto');
11
11
  const yaml = require('js-yaml');
12
- const { execute, wait, template } = require('node-powertools');
12
+ const { template, queue } = require('node-powertools');
13
13
 
14
14
  // Utils
15
15
  const GitHubCache = require('./utils/github-cache');
@@ -33,9 +33,12 @@ const rootPathProject = Manager.getRootPath('project');
33
33
 
34
34
  // Settings
35
35
  const AI = {
36
- model: 'gpt-4.1-mini',
37
- inputCost: 0.40, // $0.40 per 1M tokens
38
- outputCost: 1.60, // $1.60 per 1M tokens
36
+ // model: 'gpt-5.4-mini',
37
+ // inputCost: 0.75, // $0.75 per 1M tokens
38
+ // outputCost: 4.50, // $4.50 per 1M tokens
39
+ model: 'gpt-5.4-nano',
40
+ inputCost: 0.20, // $0.20 per 1M tokens
41
+ outputCost: 1.25, // $1.25 per 1M tokens
39
42
  }
40
43
  const CACHE_DIR = '.temp/cache/translation';
41
44
  const CACHE_BRANCH = 'cache-uj-translation';
@@ -45,33 +48,41 @@ const LOUD = process.env.UJ_LOUD_LOGS === 'true';
45
48
  const CONTROL = 'UJ-TRANSLATION-CONTROL';
46
49
  const RTL_LANGUAGES = ['ar', 'he', 'fa', 'ur', 'ps', 'sd', 'ku', 'yi', 'ji', 'ckb', 'dv', 'arc', 'aii', 'syr'];
47
50
 
48
- const TRANSLATION_DELAY_MS = 500; // wait between each translation
49
- const TRANSLATION_BATCH_SIZE = 25; // wait longer every N translations
50
- const TRANSLATION_BATCH_DELAY_MS = 10000; // longer wait after batch
51
- const AI_BATCH_SIZE = 50;
51
+ const CONCURRENCY = 5; // max simultaneous API-calling pages
52
+ const STRINGS_PER_BATCH = 25;
53
+ const MAX_RETRIES = 2;
52
54
 
53
55
  // Prompt
54
56
  const SYSTEM_PROMPT = `
55
- You are a professional translator.
56
- Translate the provided content, preserving all original formatting, HTML structure, metadata, and links.
57
- Do not explain anything — just return the translated content.
58
-
59
- Maintain the tags:
60
- - Each line is TAGGED like "[0]...[/0]" to mark the text.
61
- - You MUST KEEP THESE TAGS IN PLACE IN YOUR RESPONSE
62
- - You MUST OPEN ([0]) and CLOSE ([/0]) each tag PROPERLY.
63
- - DO NOT change the order of the tags.
64
- - DO NOT COMBINE multiple tags into one (e.g. [0]A, B, [/0] and [1]C.[/1] SHOULD BE KEPT SEPARATE).
65
- - DO NOT OMIT any tags.
66
- - You SHOULD CONSIDER adjacent tags for context.
67
-
68
- DO NOT translate the following elements (but still keep them in place):
69
- - URLs or other non-text elements.
70
- - The brand name ({ brand }).
71
- - The control tag (${CONTROL}).
72
-
73
- Output Tags: { tags }
74
- Output Language: { lang }
57
+ <role>
58
+ Professional translator. Return ONLY a valid JSON array no commentary, no markdown fences, no wrapping.
59
+ </role>
60
+
61
+ <task>
62
+ Translate the input JSON array of strings into the target language.
63
+ The output array MUST have the EXACT same length as the input array.
64
+ </task>
65
+
66
+ <rules>
67
+ <format>
68
+ - Input: a JSON array of strings.
69
+ - Output: a JSON array of translated strings of the SAME length.
70
+ - DO NOT add, remove, merge, or reorder elements.
71
+ - Consider adjacent strings for context — they come from the same page.
72
+ - Preserve leading and trailing whitespace in each string.
73
+ </format>
74
+
75
+ <preserve>
76
+ - All HTML tags, attributes, and URLs (keep verbatim, do not translate).
77
+ - The brand name "{ brand }" (never translate).
78
+ - The control string "${CONTROL}" (return it unchanged at its exact position).
79
+ </preserve>
80
+ </rules>
81
+
82
+ <example lang="es">
83
+ Input: ["Welcome to { brand }", "Get started today", "${CONTROL}"]
84
+ Output: ["Bienvenido a { brand }", "Comienza hoy", "${CONTROL}"]
85
+ </example>
75
86
  `;
76
87
 
77
88
  // Variables
@@ -80,13 +91,7 @@ let index = -1;
80
91
 
81
92
  // Glob
82
93
  const input = [
83
- // Files to include
84
94
  '_site/**/*.html',
85
-
86
- // Files to exclude
87
- // Test pages (except translation.html)
88
- '!_site/**/test/**',
89
- '_site/test/translation.html',
90
95
  ];
91
96
  const output = '';
92
97
  const delay = 250;
@@ -180,16 +185,15 @@ async function processTranslation() {
180
185
  return logger.warn('🚫 No target languages configured.');
181
186
  }
182
187
 
183
- // For testing purposes
184
- const openAIKey = await fetchOpenAIKey();
188
+ const openAIKey = process.env.BACKEND_MANAGER_OPENAI_API_KEY;
185
189
  const ujOnly = process.env.UJ_TRANSLATION_ONLY;
186
190
 
187
191
  if (!openAIKey) {
188
- return logger.error('❌ openAIKey not set. Translation requires OpenAI API key.');
192
+ return logger.error('❌ BACKEND_MANAGER_OPENAI_API_KEY not set in .env. Translation requires an OpenAI API key.');
189
193
  }
190
194
 
191
195
  // Get files
192
- const allFiles = glob(input, getGlobOptions());
196
+ const allFiles = getFiles();
193
197
 
194
198
  // Log
195
199
  logger.log(`Translating ${allFiles.length} files for ${languages.length} supported languages: ${languages.join(', ')}`);
@@ -216,8 +220,11 @@ async function processTranslation() {
216
220
 
217
221
  // Check if the promptHash matches; if not, invalidate the cache
218
222
  if (meta.prompt?.hash !== promptHash) {
219
- logger.warn(`⚠️ Meta: [${lang}] Prompt hash mismatch - invalidating cache.`);
223
+ logger.warn(`⚠️ Prompt cache MISS [${lang}]: hash mismatch invalidating ${Object.keys(meta).length - 1} cached entries.`);
220
224
  meta = {};
225
+ } else {
226
+ const entries = Object.keys(meta).filter(k => k !== 'prompt').length;
227
+ logger.log(`✅ Prompt cache HIT [${lang}]: ${entries} cached entries available.`);
221
228
  }
222
229
 
223
230
  // Store the current promptHash in the meta file
@@ -227,7 +234,7 @@ async function processTranslation() {
227
234
 
228
235
  // Track token usage and statistics
229
236
  const tokens = { input: 0, output: 0 };
230
- const queue = [];
237
+ const tasks = [];
231
238
  const stats = {
232
239
  fromCache: 0,
233
240
  newlyProcessed: 0,
@@ -254,14 +261,14 @@ async function processTranslation() {
254
261
  // Reset originalHtml
255
262
  originalHtml = $.html();
256
263
 
257
- // Collect text nodes with tags
258
- const textNodes = collectTextNodes($, { tag: true });
264
+ // Collect text nodes
265
+ const textNodes = collectTextNodes($, { tag: false });
259
266
 
260
- // Build body text from tagged nodes
261
- const bodyText = textNodes.map(n => n.tagged).join('\n');
267
+ // Build strings array for translation
268
+ const strings = textNodes.map(n => n.text);
262
269
 
263
- // Compute hash of the body text only
264
- const hash = crypto.createHash('sha256').update(bodyText).digest('hex');
270
+ // Compute hash from the strings
271
+ const hash = crypto.createHash('sha256').update(JSON.stringify(strings)).digest('hex');
265
272
 
266
273
  // Skip all except the specified HTML file
267
274
  if (ujOnly && relativePath !== ujOnly) {
@@ -271,10 +278,8 @@ async function processTranslation() {
271
278
  }
272
279
 
273
280
  // Log the page being processed
274
- logger.log(`🔍 Processing: ${relativePath} (hash: ${hash})`);
275
- // console.log('---textNodes', textNodes);
276
- // console.log('---bodyText---', bodyText);
277
- if (LOUD) logger.log(`🔍 Body text: \n${bodyText}`)
281
+ logger.log(`🔍 Processing: ${relativePath} (${strings.length} strings, hash: ${hash})`);
282
+ if (LOUD) logger.log(`🔍 Strings: \n${JSON.stringify(strings, null, 2)}`)
278
283
 
279
284
  // Translate this file for all languages in parallel
280
285
  for (const lang of languages) {
@@ -326,13 +331,13 @@ async function processTranslation() {
326
331
  (useCached || process.env.UJ_TRANSLATION_CACHE === 'true')
327
332
  && jetpack.exists(cachePath)
328
333
  ) {
329
- translated = jetpack.read(cachePath);
334
+ translated = jetpack.read(cachePath, 'json');
330
335
  logger.log(`📦 Success [${progress} - ${percentage}%]: ${logTag} - Using cache`);
331
336
  stats.fromCache++;
332
337
  stats.cachedFiles.push(logTag);
333
338
  } else {
334
339
  try {
335
- const { result, usage } = await translateWithAPI(openAIKey, bodyText, lang);
340
+ const { result, usage } = await translateWithAPI(openAIKey, strings, lang, logTag);
336
341
 
337
342
  // Log
338
343
  const elapsedTime = ((Date.now() - startTime) / 1000).toFixed(2);
@@ -345,7 +350,7 @@ async function processTranslation() {
345
350
  tokens.input += usage.input_tokens || 0;
346
351
  tokens.output += usage.output_tokens || 0;
347
352
 
348
- // Save to cache
353
+ // Save to cache (JSON array)
349
354
  jetpack.write(cachePath, translated);
350
355
 
351
356
  // Set result
@@ -356,8 +361,8 @@ async function processTranslation() {
356
361
  const elapsedTime = ((Date.now() - startTime) / 1000).toFixed(2);
357
362
  logger.error(`❌ Failed [${progress} - ${percentage}%]: ${logTag} — ${e.message} (Elapsed time: ${elapsedTime}s)`);
358
363
 
359
- // Set translated result
360
- translated = bodyText;
364
+ // Fallback to original strings
365
+ translated = strings;
361
366
 
362
367
  // Save failure to cache
363
368
  setResult(false);
@@ -365,44 +370,33 @@ async function processTranslation() {
365
370
  }
366
371
  }
367
372
 
368
- // Fix translation
369
- translated = translated.trim();
370
-
371
- // Log result
372
- // console.log('---translated---', translated);
373
-
374
373
  // Reset the DOM to avoid conflicts between languages
375
374
  const $ = cheerio.load(originalHtml);
376
- // Collect text nodes with tags
377
- const textNodes = collectTextNodes($, { tag: true });
375
+ const textNodes = collectTextNodes($, { tag: false });
378
376
 
379
377
  // Replace original text nodes with translated versions
380
378
  textNodes.forEach((n, i) => {
381
- const regex = new RegExp(`\\[${i}\\](.*?)\\[/${i}\\]`, 's');
382
- const match = translated.match(regex);
383
- const translation = match?.[1];
379
+ const translation = translated[i];
384
380
 
385
- if (!translation) {
386
- return logger.warn(`⚠️ Warning: ${logTag} - Could not find translated tag for index ${i}`);
381
+ if (translation === undefined) {
382
+ return logger.warn(`⚠️ Warning: ${logTag} - Missing translation at index ${i}`);
387
383
  }
388
384
 
389
- // Extract original leading and trailing whitespace
390
- const originalText = n.text;
391
- const leadingWhitespace = originalText.match(/^\s*/)?.[0] || '';
392
- const trailingWhitespace = originalText.match(/\s*$/)?.[0] || '';
393
-
394
- // Reapply the original whitespace to the translation
395
- const adjustedTranslation = `${leadingWhitespace}${translation.trim()}${trailingWhitespace}`;
396
-
397
- // Its possible for a control tag mismatch, so we need to check that
385
+ // Validate control tag alignment
398
386
  if (
399
- adjustedTranslation.includes(CONTROL)
387
+ translation.includes(CONTROL)
400
388
  && n.node.attr('id') !== CONTROL
401
389
  ) {
402
- logger.error(`❌ Failed: ${logTag} — Control tag mismatch in translation for index ${i}`);
390
+ logger.error(`❌ Failed: ${logTag} — Control tag mismatch at index ${i}`);
403
391
  return setResult(false);
404
392
  }
405
393
 
394
+ // Preserve original leading/trailing whitespace
395
+ const originalText = n.text;
396
+ const leadingWhitespace = originalText.match(/^\s*/)?.[0] || '';
397
+ const trailingWhitespace = originalText.match(/\s*$/)?.[0] || '';
398
+ const adjustedTranslation = `${leadingWhitespace}${translation.trim()}${trailingWhitespace}`;
399
+
406
400
  // Replace the text in the node
407
401
  if (n.type === 'data') {
408
402
  n.reference.data = adjustedTranslation;
@@ -412,21 +406,19 @@ async function processTranslation() {
412
406
  n.node.attr(n.attr, adjustedTranslation);
413
407
  }
414
408
 
415
- // Log
416
409
  if (LOUD) logger.log(`${i}: "${n.text.trim()}" → "${adjustedTranslation.trim()}"`);
417
410
  });
418
411
 
419
412
  // Rewrite links
420
413
  rewriteLinks($, lang);
421
414
 
422
- // Check that the control tag matches the expected value
415
+ // Verify control tag survived translation intact
423
416
  const controlTag = $(`#${CONTROL}`);
424
417
  if (
425
418
  controlTag.length === 0
426
419
  || controlTag.text() !== CONTROL
427
420
  ) {
428
421
  logger.error(`❌ Failed: ${logTag} — Control tag mismatch or missing`);
429
-
430
422
  return setResult(false);
431
423
  }
432
424
 
@@ -482,23 +474,14 @@ async function processTranslation() {
482
474
  stats.totalProcessed++;
483
475
  };
484
476
 
485
- // Add to queue
486
- queue.push(task);
477
+ // Add to tasks
478
+ tasks.push(task);
487
479
  }
488
480
  }
489
481
 
490
- // Process queue in batches with delay
491
- for (let i = 0; i < queue.length; i += TRANSLATION_BATCH_SIZE) {
492
- const batch = queue.slice(i, i + TRANSLATION_BATCH_SIZE).map(fn => fn());
493
-
494
- // Wait for all tasks in this batch to finish
495
- await Promise.all(batch);
496
-
497
- // Delay between batches
498
- if (i + TRANSLATION_BATCH_SIZE < queue.length) {
499
- await wait(TRANSLATION_BATCH_DELAY_MS);
500
- }
501
- }
482
+ // Process tasks with concurrency limit
483
+ const q = queue({ concurrency: CONCURRENCY });
484
+ await Promise.all(tasks.map(task => q.add(task)));
502
485
 
503
486
  // Log skipped files
504
487
  logger.warn('🚫 Skipped files:');
@@ -619,155 +602,35 @@ async function processTranslation() {
619
602
  }
620
603
  }
621
604
 
622
- async function translateWithAPI(openAIKey, content, lang) {
623
- const lines = content.trim().split('\n');
624
-
625
- // If content is small enough, use single batch
626
- if (lines.length <= AI_BATCH_SIZE) {
627
- return await translateBatch(openAIKey, content, lang, 0);
605
+ async function translateWithAPI(openAIKey, strings, lang, logTag) {
606
+ // If content is small enough, translate in one call
607
+ if (strings.length <= STRINGS_PER_BATCH) {
608
+ return await translateBatch(openAIKey, strings, lang, logTag);
628
609
  }
629
610
 
630
- // Split into batches
631
- const batches = [];
632
- for (let i = 0; i < lines.length; i += AI_BATCH_SIZE) {
633
- const batchLines = lines.slice(i, i + AI_BATCH_SIZE);
634
- batches.push({
635
- content: batchLines.join('\n'),
636
- startIndex: i,
637
- endIndex: i + batchLines.length - 1,
638
- batchNumber: Math.floor(i / AI_BATCH_SIZE)
639
- });
640
- }
641
-
642
- // Process batches in parallel
643
- const batchPromises = batches.map(batch =>
644
- translateBatch(openAIKey, batch.content, lang, batch.batchNumber)
645
- .then(result => ({ ...result, ...batch }))
646
- .catch(error => ({ error, ...batch }))
647
- );
648
-
649
- const batchResults = await Promise.all(batchPromises);
650
-
651
- // Reconstruct translation maintaining order
652
- const translatedLines = new Array(lines.length);
611
+ // Split into batches for large pages
612
+ const translated = [];
653
613
  let totalUsage = { input_tokens: 0, output_tokens: 0 };
654
- let hasErrors = false;
655
-
656
- for (const batchResult of batchResults) {
657
- if (batchResult.error) {
658
- // Log
659
- logger.error(`❌ Batch ${batchResult.batchNumber} failed: ${batchResult.error.message}`);
660
-
661
- // Try to subdivide the failed batch
662
- const failedBatchLines = lines.slice(batchResult.startIndex, batchResult.endIndex + 1);
663
- try {
664
- logger.log(`🔄 Attempting to subdivide failed batch ${batchResult.batchNumber} (${failedBatchLines.length} lines)`);
665
- const subdivisionResult = await subdivideAndTranslate(openAIKey, failedBatchLines, lang, batchResult.batchNumber);
666
-
667
- // Place subdivided results in correct positions
668
- for (let i = 0; i < subdivisionResult.length; i++) {
669
- translatedLines[batchResult.startIndex + i] = subdivisionResult[i];
670
- }
671
614
 
672
- logger.log(`✅ Successfully subdivided batch ${batchResult.batchNumber}`);
673
- } catch (subdivisionError) {
674
- logger.error(`❌ Subdivision failed for batch ${batchResult.batchNumber}: ${subdivisionError.message}`);
615
+ for (let i = 0; i < strings.length; i += STRINGS_PER_BATCH) {
616
+ const batch = strings.slice(i, i + STRINGS_PER_BATCH);
617
+ const { result, usage } = await translateBatch(openAIKey, batch, lang, logTag);
675
618
 
676
- // Final fallback: use original content
677
- for (let i = 0; i < failedBatchLines.length; i++) {
678
- translatedLines[batchResult.startIndex + i] = failedBatchLines[i];
679
- }
680
- hasErrors = true;
681
- }
682
- } else {
683
- const resultLines = batchResult.result.split('\n');
684
-
685
- // Log
686
- logger.log(`✅ Batch ${batchResult.batchNumber} translated successfully`);
687
-
688
- // Place translated lines in correct positions
689
- for (let i = 0; i < resultLines.length; i++) {
690
- translatedLines[batchResult.startIndex + i] = resultLines[i];
691
- }
692
- totalUsage.input_tokens += batchResult.usage.input_tokens || 0;
693
- totalUsage.output_tokens += batchResult.usage.output_tokens || 0;
694
- }
695
- }
696
-
697
- const finalResult = translatedLines.join('\n');
698
-
699
- if (hasErrors) {
700
- logger.warn(`⚠️ Some batches failed, partial translation completed`);
619
+ translated.push(...result);
620
+ totalUsage.input_tokens += usage.input_tokens || 0;
621
+ totalUsage.output_tokens += usage.output_tokens || 0;
701
622
  }
702
623
 
703
624
  return {
704
- result: finalResult,
625
+ result: translated,
705
626
  usage: totalUsage,
706
627
  };
707
628
  }
708
629
 
709
- async function subdivideAndTranslate(openAIKey, lines, lang, originalBatchNumber, depth = 0) {
710
- const maxDepth = 10; // Prevent infinite recursion
711
-
712
- if (depth > maxDepth) {
713
- throw new Error(`Maximum subdivision depth reached for batch ${originalBatchNumber}`);
714
- }
715
-
716
- // If only one line, try to translate it directly
717
- if (lines.length === 1) {
718
- try {
719
- const singleLineResult = await translateBatch(openAIKey, lines[0], lang, `${originalBatchNumber}.${depth}`, true);
720
- return [singleLineResult.result];
721
- } catch (error) {
722
- logger.warn(`⚠️ Single line translation failed, using original: ${error.message}`);
723
- return lines; // Return original line if single line fails
724
- }
725
- }
726
-
727
- // Divide the batch into smaller sub-batches (half the size, minimum 1)
728
- const subBatchSize = Math.max(1, Math.floor(lines.length / 2));
729
- const subBatches = [];
730
-
731
- for (let i = 0; i < lines.length; i += subBatchSize) {
732
- subBatches.push(lines.slice(i, i + subBatchSize));
733
- }
734
-
735
- logger.log(`🔀 Subdividing into ${subBatches.length} sub-batches of ~${subBatchSize} lines each`);
736
-
737
- const results = [];
738
-
739
- // Process sub-batches sequentially to avoid overwhelming the API
740
- for (let i = 0; i < subBatches.length; i++) {
741
- const subBatch = subBatches[i];
742
- const subBatchContent = subBatch.join('\n');
743
-
744
- try {
745
- const subResult = await translateBatch(openAIKey, subBatchContent, lang, `${originalBatchNumber}.${depth}.${i}`);
746
- results.push(...subResult.result.split('\n'));
747
- } catch (error) {
748
- logger.warn(`⚠️ Sub-batch ${i} failed, attempting further subdivision: ${error.message}`);
749
-
750
- // Recursively subdivide this failed sub-batch
751
- const recursiveResult = await subdivideAndTranslate(openAIKey, subBatch, lang, originalBatchNumber, depth + 1);
752
- results.push(...recursiveResult);
753
- }
754
- }
755
-
756
- return results;
757
- }
758
-
759
- async function translateBatch(openAIKey, content, lang, batchNumber, isSingleLine = false) {
630
+ async function translateBatch(openAIKey, strings, lang, logTag, attempt = 0) {
760
631
  const brand = config?.brand?.name || 'Unknown Brand';
761
- const inputLines = content.split('\n');
762
-
763
- const systemPrompt = template(SYSTEM_PROMPT, {
764
- lang,
765
- brand,
766
- tags: inputLines.length - 1,
767
- });
768
-
769
- // Format content
770
- content = content.trim();
632
+ const systemPrompt = template(SYSTEM_PROMPT, { brand });
633
+ const userMessage = `Language: ${lang}\nArray length: ${strings.length}\n\n${JSON.stringify(strings)}`;
771
634
 
772
635
  // Request
773
636
  const res = await fetch('https://api.openai.com/v1/responses', {
@@ -782,49 +645,48 @@ async function translateBatch(openAIKey, content, lang, batchNumber, isSingleLin
782
645
  body: {
783
646
  model: AI.model,
784
647
  input: [
785
- { role: 'system', content: systemPrompt },
786
- { role: 'user', content: content }
648
+ { role: 'developer', content: systemPrompt },
649
+ { role: 'user', content: userMessage }
787
650
  ],
788
- temperature: 0.2,
651
+ reasoning: { effort: 'low' },
789
652
  },
790
653
  });
791
654
 
792
- // Get result
793
- const result = res?.output?.[0]?.content?.[0]?.text;
655
+ // Get result (reasoning models put a reasoning item first — find the message)
656
+ const message = res?.output?.find(o => o.type === 'message');
657
+ const text = message?.content?.[0]?.text;
794
658
  const usage = res?.usage || {};
795
- const trimmed = result?.trim();
796
659
 
797
- // Check for errors
798
- if (!result || trimmed === '') {
799
- throw new Error(`Translation result was empty for batch ${batchNumber}`);
660
+ // Check for empty response
661
+ if (!text || text.trim() === '') {
662
+ const types = (res?.output || []).map(o => o.type).join(', ');
663
+ throw new Error(`Translation result was empty (output types: [${types}])`);
800
664
  }
801
665
 
802
- // Get content line count
803
- const outputLines = trimmed.split('\n');
804
-
805
- // console.log(`----Batch ${batchNumber} inputLines`, inputLines.length);
806
- // console.log(`----Batch ${batchNumber} outputLines`, outputLines.length);
807
-
808
- if (LOUD) {
809
- // console.log(`-----Batch ${batchNumber} content\n`, content);
810
- // console.log(`-----Batch ${batchNumber} trimmed\n`, trimmed);
666
+ // Parse JSON array from response (strip markdown fences if model wraps it)
667
+ let parsed;
668
+ try {
669
+ const cleaned = text.trim().replace(/^```json?\n?|\n?```$/g, '');
670
+ parsed = JSON.parse(cleaned);
671
+ } catch (e) {
672
+ throw new Error(`Failed to parse translation JSON: ${e.message}`);
673
+ }
811
674
 
812
- // Loop thru and log the translated lines
813
- outputLines.forEach((line, index) => {
814
- // Log "original line -> translated line"
815
- const prefix = isSingleLine ? '🔸' : '🔤';
816
- logger.log(`${prefix} Translated line [batch=[${batchNumber}], line=${index + 1}]: "${inputLines[index]}" → "${line}"`);
817
- });
675
+ // Validate array and length
676
+ if (!Array.isArray(parsed)) {
677
+ throw new Error(`Translation result is not an array (got ${typeof parsed})`);
818
678
  }
819
679
 
820
- // Throw error if there is a mismatch in line count for this batch
821
- if (inputLines.length !== outputLines.length) {
822
- throw new Error(`Translation line count mismatch in batch ${batchNumber}: ${inputLines.length} ${outputLines.length}`);
680
+ if (parsed.length !== strings.length) {
681
+ if (attempt < MAX_RETRIES) {
682
+ logger.warn(`⚠️ ${logTag} Length mismatch (expected ${strings.length}, got ${parsed.length}), retry ${attempt + 1}/${MAX_RETRIES}...`);
683
+ return translateBatch(openAIKey, strings, lang, logTag, attempt + 1);
684
+ }
685
+ throw new Error(`Translation length mismatch: expected ${strings.length}, got ${parsed.length}`);
823
686
  }
824
687
 
825
- // Return
826
688
  return {
827
- result: trimmed,
689
+ result: parsed,
828
690
  usage,
829
691
  };
830
692
  }
@@ -1061,6 +923,15 @@ function getIgnoredPages() {
1061
923
  // Admin
1062
924
  'admin',
1063
925
 
926
+ // Test pages
927
+ 'test',
928
+
929
+ // Team pages
930
+ 'team',
931
+
932
+ // Updates/changelog pages
933
+ 'updates',
934
+
1064
935
  // Firestore auth pages
1065
936
  '__/auth',
1066
937
 
@@ -1070,6 +941,11 @@ function getIgnoredPages() {
1070
941
  };
1071
942
  }
1072
943
 
944
+ // Pages allowed even when their parent folder is excluded
945
+ const IGNORE_EXCEPTIONS = [
946
+ '_site/test/translation.html',
947
+ ];
948
+
1073
949
  function getGlobOptions() {
1074
950
  const ignoredPages = getIgnoredPages();
1075
951
  return {
@@ -1081,6 +957,13 @@ function getGlobOptions() {
1081
957
  }
1082
958
  }
1083
959
 
960
+ function getFiles() {
961
+ const files = glob(input, getGlobOptions());
962
+ const extras = IGNORE_EXCEPTIONS.filter(f => jetpack.exists(f));
963
+
964
+ return [...new Set([...files, ...extras])];
965
+ }
966
+
1084
967
  // Initialize or get cache
1085
968
  async function initializeCache() {
1086
969
  const useCache = process.env.UJ_TRANSLATION_CACHE !== 'false';
@@ -1108,32 +991,6 @@ async function initializeCache() {
1108
991
  return cache;
1109
992
  }
1110
993
 
1111
- async function fetchOpenAIKey() {
1112
- const url = 'https://api.itwcreativeworks.com/get-api-keys';
1113
-
1114
- try {
1115
- const response = await fetch(url, {
1116
- method: 'GET',
1117
- response: 'json',
1118
- tries: 2,
1119
- headers: {
1120
- 'Authorization': `Bearer ${process.env.GH_TOKEN}`,
1121
- },
1122
- query: {
1123
- authorizationKeyName: 'github',
1124
- }
1125
- });
1126
-
1127
- // Log
1128
- // logger.log('OpenAI API response:', response);
1129
-
1130
- // Return
1131
- return response.openai.ultimate_jekyll.translation;
1132
- } catch (error) {
1133
- logger.error('Error:', error);
1134
- }
1135
- }
1136
-
1137
994
  function getCanonicalUrl(lang, relativePath) {
1138
995
  const baseUrl = Manager.getWorkingUrl();
1139
996
 
@@ -16,6 +16,11 @@ const collectTextNodes = ($, options) => {
16
16
  return;
17
17
  }
18
18
 
19
+ // Skip elements (and their children) marked with data-uj-no-translate
20
+ if (node.closest('[data-uj-no-translate]').length) {
21
+ return;
22
+ }
23
+
19
24
  // Handle <title>
20
25
  if (node.is('title')) {
21
26
  const i = textNodes.length;
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "ultimate-jekyll-manager",
3
- "version": "1.5.0",
3
+ "version": "1.6.1",
4
4
  "description": "Ultimate Jekyll dependency manager",
5
5
  "main": "dist/index.js",
6
6
  "exports": {
@@ -1,75 +0,0 @@
1
- {
2
- "name": "ultimate-jekyll-manager",
3
- "version": "1.0.0",
4
- "description": "Ultimate Jekyll dependency manager",
5
- "main": "index.js",
6
- "scripts": {
7
-
8
- },
9
- "engines": {
10
- "node": ">=18.x"
11
- },
12
- "repository": {
13
- "type": "git",
14
- "url": "https://github.com/itw-creative-works/ultimate-jekyll.git"
15
- },
16
- "keywords": [
17
- "Autoprefixer",
18
- "Browsersync",
19
- "gulp",
20
- "imagemin",
21
- "Jekyll",
22
- "PostCSS",
23
- "Sass",
24
- "Webpack"
25
- ],
26
- "author": "ITW Creative Works",
27
- "license": "MIT",
28
- "bugs": {
29
- "url": "https://github.com/itw-creative-works/ultimate-jekyll/issues"
30
- },
31
- "homepage": "https://template.itwcreativeworks.com",
32
- "noteDeps": {
33
- "browser-sync": "2.23.7 (6-22-2023): Hard lock because every version after uses socket.io@4.7.0 which uses engine.io@6.5.0 which is incompatible with node 10.15.1 due to TextDecoder() in build process",
34
- "sharp": "0.23.1 (sometime before 2021ish): Hard lock because later versions had issues. Possibly solved in higher node versions"
35
- },
36
- "dependencies": {
37
- "@babel/core": "7.24.7",
38
- "@babel/preset-env": "7.24.7",
39
- "autoprefixer": "9.8.8",
40
- "browser-sync": "2.23.7",
41
- "del": "6.1.1",
42
- "eslint-config-google": "0.14.0",
43
- "eslint-webpack-plugin": "3.2.0",
44
- "event-stream": "4.0.1",
45
- "fs-jetpack": "4.3.1",
46
- "glob": "7.2.3",
47
- "gulp": "3.9.1",
48
- "gulp-babel": "8.0.0",
49
- "gulp-cached": "1.1.1",
50
- "gulp-eslint": "6.0.0",
51
- "gulp-newer": "1.4.0",
52
- "gulp-plumber": "1.2.1",
53
- "gulp-postcss": "9.0.1",
54
- "gulp-responsive": "3.0.1",
55
- "gulp-sass": "4.1.1",
56
- "gulp-watch": "5.0.1",
57
- "js-yaml": "4.1.0",
58
- "json5": "2.2.3",
59
- "mocha": "8.4.0",
60
- "node-fetch": "2.6.12",
61
- "require-dir": "1.2.0",
62
- "sharp": "0.23.1",
63
- "source-map-loader": "2.0.2",
64
- "terser-webpack-plugin": "5.3.10",
65
- "through2": "4.0.2",
66
- "ultimate-jekyll-poster": "1.0.1",
67
- "vinyl-named": "1.1.0",
68
- "web-manager": "3.2.60",
69
- "webpack": "5.89.0",
70
- "webpack-stream": "6.1.2",
71
- "wonderful-fetch": "1.1.12",
72
- "yargs": "16.2.0",
73
- "zzzzzzzzzz": "9.9.9"
74
- }
75
- }