jskelet 0.1.2 → 0.1.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -4,8 +4,9 @@ This document explains JSkelet's ISR substitute in full detail: the HTML TTL
4
4
  cache and its stale-while-revalidate behaviour, where `revalidate` comes from,
5
5
  how the cache key is built, the values of the `X-JSkelet-Cache` header, why the
6
6
  compressed body is kept in the cache, per-request memoization
7
- (`withRequestCache` / `cache()`), how upstream failures affect the cache
8
- (`reportUpstreamFailure`) and the prewarm round at server startup. The
7
+ (`withRequestCache` / `cache()`), the data cache (`withDataCache`), how upstream
8
+ failures affect the cache (`reportUpstreamFailure`) and the prewarm round at
9
+ server startup. The
9
10
  measurement rationale behind the decisions is in
10
11
  [02-architecture.md](./02-architecture.md), and the full reference of config
11
12
  fields is in [07-configuration.md](./07-configuration.md).
@@ -18,6 +19,7 @@ route(controller, { revalidate })
18
19
  └─ withUpstreamTracking(...) ← missing data detection
19
20
  └─ withRequestCache(...) ← per-request memoization
20
21
  └─ produce() → controller + renderPage
22
+ └─ withDataCache(...) ← upstream data cache
21
23
  ```
22
24
 
23
25
  The order matters: the **per-request cache must be innermost** so that two
@@ -25,6 +27,22 @@ calls in the same render collapse into a single upstream request; **upstream
25
27
  tracking must be inside the HTML cache** so that output produced with missing
26
28
  data is not written to the cache.
27
29
 
30
+ How the two caches divide the work:
31
+
32
+ | | HTML cache | Data cache |
33
+ | --- | --- | --- |
34
+ | What it holds | The whole page (+ its compressed body) | The JSON that came from upstream |
35
+ | Entry size | ~100-200 kB | ~1-20 kB |
36
+ | Entry limit | 500 (`cache().maxEntries`) | 10,000 (`cache().data.maxEntries`) |
37
+ | Who benefits | Pages with traffic: not even rendered | The long tail: rendered, but without going to the API |
38
+
39
+ In practice this distinction means: on a site with tens of thousands of paths it
40
+ is impossible to keep every page hot as HTML — a warm-up that goes past 500
41
+ entries deletes what it just warmed. For the long tail the goal is not "have the
42
+ HTML ready" but **"have the data that produces the page available without going
43
+ to the API"**. Then a page that was never warmed is also produced within
44
+ milliseconds on the first visit, and spends no quota.
45
+
28
46
  ## Public versus per-visitor
29
47
 
30
48
  Everything in this document applies to HTML that **can go to everyone
@@ -109,8 +127,8 @@ The practical consequence: a page that does not depend on the query string
109
127
  produces a separate entry for every combination when it is called with
110
128
  different campaign parameters (`?utm_source=…`). Stripping such parameters at
111
129
  the reverse proxy layer, or turning off the cache (by not supplying
112
- `revalidate`), is a reasonable precaution; the store holds at most 500 entries
113
- and evicts the oldest with LRU.
130
+ `revalidate`), is a reasonable precaution; by default the store holds at most
131
+ 500 entries and evicts the oldest with LRU.
114
132
 
115
133
  ## Stale-while-revalidate
116
134
 
@@ -141,8 +159,8 @@ data in the HTML can be at most `revalidate + one refresh round` behind. That
141
159
  price is acceptable, because live fields such as prices are updated on the
142
160
  client over WebSocket.
143
161
 
144
- The store is an LRU: an accessed entry is moved to the end, and once
145
- `MAX_ENTRIES = 500` is exceeded the oldest is evicted.
162
+ The store is an LRU: an accessed entry is moved to the end, and once the limit
163
+ (`cache().maxEntries`, 500 by default) is exceeded the oldest is evicted.
146
164
 
147
165
  ## What gets written to the cache
148
166
 
@@ -223,6 +241,83 @@ Details:
223
241
  - `withRequestCache(run)` is exported; it can be used to set up the same scope
224
242
  outside `route()` (for example in an Express handler you wrote yourself).
225
243
 
244
+ ## Cross-request data cache: `withDataCache`
245
+
246
+ `cache()` only lives for the duration of **a single request**. What it takes to
247
+ protect the long tail from the API quota is a data layer that lives across
248
+ requests, has a TTL and refreshes itself:
249
+
250
+ ```js
251
+ // lib/api/articles.js
252
+ import { withDataCache, reportUpstreamFailure } from "jskelet";
253
+
254
+ export async function getArticle(slug) {
255
+ return withDataCache(`news:${slug}`, 600, async () => {
256
+ const response = await fetch(`${process.env.API_ORIGIN}/articles/${slug}`);
257
+
258
+ if (!response.ok) {
259
+ reportUpstreamFailure({ status: response.status, path: `/articles/${slug}` });
260
+ return null;
261
+ }
262
+
263
+ return response.json();
264
+ });
265
+ }
266
+ ```
267
+
268
+ The wrapper form of the same pattern — the key is derived from the arguments:
269
+
270
+ ```js
271
+ import { dataCache } from "jskelet";
272
+
273
+ export const getArticle = dataCache(
274
+ async (slug) => apiGet(`/articles/${slug}`),
275
+ { key: "news", revalidate: 600 },
276
+ );
277
+ ```
278
+
279
+ Behaviour:
280
+
281
+ | State | Result |
282
+ | --- | --- |
283
+ | Fresh entry | Returns immediately, the `producer` does not run |
284
+ | TTL expired, still inside the stale window | The stale value returns **immediately**, the refresh runs in the background |
285
+ | No entry | The `producer` is awaited |
286
+ | The `producer` failed, a stale entry exists | The stale value returns, warning: `[data-cache] producer failed, serving stale value: …` |
287
+ | The `producer` failed, there is no entry | The error goes to the caller |
288
+
289
+ Details:
290
+
291
+ - **Concurrent calls for the same key collapse into one upstream request.** This
292
+ is the behaviour that saves the most quota during warm-up rounds: if 50 pages
293
+ want the same index data, the API is called once.
294
+ - **`null` and `undefined` are not stored.** An application's HTTP client
295
+ usually returns `null` on failure; storing that would freeze a transient 429
296
+ into "no data" for the whole TTL. Pass `{ storeEmpty: true }` if you want the
297
+ empty answer stored deliberately.
298
+ - **The stale window is longer than the HTML one**: `staleFactor` defaults to 10,
299
+ so an entry stays as an emergency fallback for 11 times its TTL. Stale data is
300
+ better than an incomplete page. It can be turned off per key with
301
+ `{ staleFactor: 0 }`.
302
+ - The key belongs entirely to the application: distinctions such as language,
303
+ version or page number go into the key (`news:en:v2:${slug}`).
304
+ - When the TTL is `0` the cache is disabled and the `producer` runs on every
305
+ call — enough to switch a setting off temporarily.
306
+
307
+ The management surface:
308
+
309
+ | Function | What it does |
310
+ | --- | --- |
311
+ | `withDataCache(key, ttlSeconds, producer, options?)` | The main entry point |
312
+ | `dataCache(fn, { key, revalidate, … })` | The function wrapper |
313
+ | `clearDataCache(prefix?)` | Drops the entries matching the prefix (or all of them), returns how many were removed |
314
+ | `getDataCacheSize()` | The number of entries |
315
+ | `getDataCacheEntries()` | A dump: `{ key, stale, expiresIn }`. The value itself is not returned. |
316
+
317
+ `clearDataCache("news:")` is the counterpart of a "this content was updated"
318
+ webhook: it drops one section's data so the next HTML refresh picks up the new
319
+ content.
320
+
226
321
  ## Degraded render: `reportUpstreamFailure`
227
322
 
228
323
  If upstream went down during the render, the output contains missing data.
@@ -267,6 +362,32 @@ do not get better by retrying. Turning the cache off because of them would mean
267
362
  rendering the page from scratch on every visit — the content comes back just as
268
363
  incomplete, and the visitor only pays the render time.
269
364
 
365
+ Output produced with missing data is **not offered to shared caches** either: a
366
+ `degraded` response gets `private, no-store` instead of `public, s-maxage=…`.
367
+ Taking back the "do not store" decision at the CDN would repeat the same mistake
368
+ one layer up. The diagnostic header (`X-JSkelet-Cache: MISS`) is still written.
369
+
370
+ ### When `notFound()` coincides with a transient failure
371
+
372
+ A controller that calls `notFound()` because no data arrived can turn the whole
373
+ site into 404s when upstream is rate limited — and because those 404s enter the
374
+ cache, a temporary quota problem becomes a "this page does not exist" answer for
375
+ the whole TTL. For a search engine that is a permanent loss.
376
+
377
+ The framework separates the two cases: if a **transient** upstream failure was
378
+ reported during the render, `notFound()` is not served as a 404.
379
+
380
+ | During the render | Result of `notFound()` |
381
+ | --- | --- |
382
+ | A transient failure exists (`429`, `5xx`, network error) | `503`, `Retry-After: 30`, `no-store` — **not** written to the cache, the next request produces the real content |
383
+ | A permanent failure (`404`, `403`…) or no failure | A normal `404` |
384
+
385
+ The log line:
386
+ `[render] /news/x returned notFound() while upstream is failing (429 /api/...), serving an uncached 503 instead`
387
+
388
+ So when upstream runs out of quota the page is produced dynamically, without
389
+ being written to the cache; nothing is frozen as "missing".
390
+
270
391
  ## Managing the cache
271
392
 
272
393
  `jskelet` exports these functions:
@@ -345,26 +466,78 @@ Rules:
345
466
  - Ones starting with one of the `prewarmSkip` prefixes are skipped. The default
346
467
  list: `/api/`, `/_fragment/`, `/__jskelet/`. Session-dependent pages should
347
468
  not be warmed.
348
- - Deduplication **preserves order**: since the list is trimmed with
349
- `PREWARM_MAX`, the priority order the application gives is meaningful — put
350
- the most important pages first.
469
+ - Deduplication **preserves order**: when no `priority` is given, the order the
470
+ application provides is meaningful — put the most important pages first.
351
471
  - If this hook is not defined the warm-up is never set up; not even the timer
352
472
  is started.
353
473
 
354
474
  ### Round logic
355
475
 
356
- 1. The list is collected and trimmed with `PREWARM_MAX` (400 by default).
357
- 2. `PREWARM_CONCURRENCY` workers send requests in parallel (4 in prod, 2 in
358
- dev). Less parallelism in dev: so the scan does not compete for CPU with the
359
- render of the page you currently have open in the browser.
360
- 3. **A single serial retry round** is performed for the failed paths
361
- (`concurrency: 1`). The errors are mostly upstream rate limiting (429): the
362
- first round strains the API while fetching hundreds of pages at once. The
363
- retry round gets those pages into the cache; otherwise the visitor pays for
364
- the cold render.
365
- 4. A summary is logged:
476
+ 1. The list is collected. If it is longer than `max` (400 by default) a slice is
477
+ selected: the paths matching `priority` are taken first **on every round**,
478
+ and the remaining slots are filled from the queue.
479
+ 2. `concurrency` workers send requests in parallel (4 in prod, 2 in dev). Less
480
+ parallelism in dev: so the scan does not compete for CPU with the render of
481
+ the page you currently have open in the browser.
482
+ 3. If `rps` is given, the round never goes above that rate — no matter the
483
+ parallelism.
484
+ 4. **A single serial retry round** is performed for the failed paths after
485
+ waiting `retryDelayMs` (`concurrency: 1`). The wait is deliberate: rate limit
486
+ windows are on the order of seconds, so retrying immediately just earns the
487
+ same 429.
488
+ 5. A summary is logged:
366
489
  `[prewarm] warmed 128/130 pages, 2 failed, 5 recovered on the retry pass (12.4s)`
367
490
 
491
+ ### Warm-up order: `priority`
492
+
493
+ ```js
494
+ // jskelet.config.mjs
495
+ cache: () => ({
496
+ prewarm: {
497
+ priority: [
498
+ "/",
499
+ "/markets/:path*",
500
+ /-comments$/,
501
+ ],
502
+ },
503
+ }),
504
+ ```
505
+
506
+ The pattern syntax (`/news/:slug`) and a plain `RegExp` can be used together;
507
+ the latter is for rules the pattern syntax does not cover, such as "everything
508
+ ending in `-comments`". Whatever is written first is warmed first; paths that
509
+ match nothing go to the queue and keep their relative order.
510
+
511
+ ### Drip warm-up: `rotate` + `rps` + `intervalSeconds`
512
+
513
+ On a site with 10,000 paths, warming everything in a single round is neither
514
+ possible (the HTML cache holds 500 entries) nor right (the API quota runs out).
515
+ The correct behaviour is to spread the list over time:
516
+
517
+ ```js
518
+ prewarm: {
519
+ max: 300, // 300 pages per round
520
+ rps: 4, // at most 4 requests per second
521
+ intervalSeconds: 300, // a round every 5 minutes
522
+ rotate: true, // the queue continues where it left off
523
+ priority: ["/", "/markets/:path*"],
524
+ }
525
+ ```
526
+
527
+ In this setup the priority pages are refreshed on every round, the rest of the
528
+ queue is walked end to end across rounds, and upstream never sees more than four
529
+ requests per second. Used together with the data cache, the warm-up barely
530
+ reaches the API after the second round: it reads from the data layer.
531
+
532
+ With rotation on, the paths left outside the limit are not lost, they are left
533
+ for the next round; the log distinguishes this:
534
+ `… , 700 deferred to the next pass`. With `rotate: false` you get the classic
535
+ behaviour — every round warms the same first slice of the list and the rest is
536
+ never warmed (`… , 700 over the limit`).
537
+
538
+ If a round takes longer than `intervalSeconds`, a new round is not started;
539
+ overlapping rounds would put twice the load on upstream.
540
+
368
541
  The requests go out with the headers `user-agent: jskelet-prewarm`
369
542
  (`brand.prewarmUserAgent`) and `accept-encoding: br, gzip`; the second one so
370
543
  that the compressed body enters the cache too.
@@ -397,10 +570,14 @@ comes first so that one-off experiments can be done without editing the config.
397
570
  | Setting | Env | `cache().prewarm` | Default |
398
571
  | --- | --- | --- | --- |
399
572
  | On/off | `PREWARM=0` disables it, `PREWARM=1` overrides the config and enables it | `enabled` | `true` |
400
- | Maximum paths | `PREWARM_MAX` | `max` | `400` |
573
+ | Maximum paths per round | `PREWARM_MAX` | `max` | `400` |
401
574
  | Parallelism | `PREWARM_CONCURRENCY` | `concurrency` | prod 4, dev 2 |
575
+ | Requests per second | `PREWARM_RPS` | `rps` | `0` (unlimited) |
402
576
  | Startup delay (ms) | `PREWARM_DELAY_MS` | `delayMs` | prod 500, dev 3000 |
577
+ | Retry round delay (ms) | `PREWARM_RETRY_DELAY_MS` | `retryDelayMs` | `2000` |
403
578
  | Period (seconds) | `PREWARM_INTERVAL_SECONDS` | `intervalSeconds` | `0` (off) |
579
+ | Queue rotation | — | `rotate` | `true` |
580
+ | Warm-up order | — | `priority` | `[]` |
404
581
 
405
582
  Numeric settings only accept **positive and finite** values; an invalid value
406
583
  silently falls through to the next layer.
@@ -445,6 +622,15 @@ filled the cache.
445
622
  parameters may be multiplying entries.
446
623
  - **The warm-up never runs.** `hooks.prewarmPaths` is not defined, `PREWARM=0`
447
624
  is set, or `cache().prewarm.enabled === false`.
625
+ - **The warm-up round pushes the API into 429.** No `rps` was given. Lowering
626
+ `concurrency` is not enough; the setting that protects the quota is the total
627
+ rate. The lasting fix is the data cache: after the second round the warm-up
628
+ does not reach upstream.
629
+ - **The warm-up list is longer than `max` and its tail never warms.** `rotate`
630
+ may be `false`; the `over the limit` phrase in the log shows this.
631
+ - **A whole section returns 404.** Upstream may be down. In that case a 503 that
632
+ does not enter the cache is now returned instead of a 404; look for the
633
+ `returned notFound() while upstream is failing` line in the log.
448
634
 
449
635
  ## What's next
450
636
 
@@ -119,7 +119,17 @@ export default {
119
119
  async cache() {
120
120
  return {
121
121
  html: { "/": 60, "/news/:slug": 300 },
122
- prewarm: { enabled: true, max: 400, concurrency: 4, intervalSeconds: 0 },
122
+ maxEntries: 500,
123
+ data: { maxEntries: 10000, staleFactor: 10 },
124
+ prewarm: {
125
+ enabled: true,
126
+ max: 400,
127
+ concurrency: 4,
128
+ rps: 0,
129
+ intervalSeconds: 0,
130
+ rotate: true,
131
+ priority: ["/", "/news/:slug"],
132
+ },
123
133
  };
124
134
  },
125
135
 
@@ -561,8 +571,10 @@ Details: [03-routing.md](./03-routing.md).
561
571
 
562
572
  ## `cache()`
563
573
 
564
- **Type:** `() => { html?: Record<string, number>, prewarm?: object }` —
565
- **Default:** `{ html: {}, prewarm: { enabled: true, max: 400, intervalSeconds: 0 } }`
574
+ **Type:**
575
+ `() => { html?: Record<string, number>, maxEntries?: number, data?: object, prewarm?: object }` —
576
+ **Default:**
577
+ `{ html: {}, maxEntries: 500, data: { maxEntries: 10000, staleFactor: 10 }, prewarm: { enabled: true, max: 400, intervalSeconds: 0 } }`
566
578
 
567
579
  ### `cache().html`
568
580
 
@@ -582,18 +594,57 @@ html: {
582
594
  }
583
595
  ```
584
596
 
597
+ ### `cache().maxEntries`
598
+
599
+ **Type:** `number` — **Default:** `500`
600
+
601
+ The entry limit of the HTML cache. Because an entry costs a hundred kilobytes,
602
+ raising this number burns through memory quickly; trying to solve a site with
603
+ tens of thousands of paths from here is the wrong layer — the right place is
604
+ `cache().data`.
605
+
606
+ ### `cache().data`
607
+
608
+ The upstream data cache (`withDataCache`). Details:
609
+ [06-caching.md](./06-caching.md).
610
+
611
+ | Field | Type | Default | Meaning |
612
+ | --- | --- | --- | --- |
613
+ | `maxEntries` | `number` | `10000` | The LRU entry limit. The limit is high because JSON is tens of times smaller than HTML. |
614
+ | `staleFactor` | `number` | `10` | For how many TTLs an entry stays usable after the TTL expired. `0` → no stale serving. |
615
+
585
616
  ### `cache().prewarm`
586
617
 
587
618
  | Field | Type | Default | Meaning |
588
619
  | --- | --- | --- | --- |
589
620
  | `enabled` | `boolean` | `true` | If `false`, no prewarming happens (can be overridden with `PREWARM=1`) |
590
- | `max` | `number` | `400` | At most how many paths are prewarmed |
621
+ | `max` | `number` | `400` | At most how many paths are prewarmed per pass |
591
622
  | `concurrency` | `number` | prod 4, dev 2 | Number of parallel workers |
623
+ | `rps` | `number` | `0` | At most how many prewarm requests per second; `0` is unlimited. This is the setting that protects the upstream quota. |
592
624
  | `delayMs` | `number` | prod 500, dev 3000 | Delay of the first pass after startup |
625
+ | `retryDelayMs` | `number` | `2000` | How long to wait before the retry pass |
593
626
  | `intervalSeconds` | `number` | `0` | If greater than 0, the pass repeats periodically |
627
+ | `rotate` | `boolean` | `true` | If the list is longer than `max`, periodic passes continue where they left off |
628
+ | `priority` | `(string \| RegExp)[]` | `[]` | Warm-up order; matching paths are taken first on every pass |
629
+
630
+ `priority` accepts two forms: the pattern syntax used everywhere in the config,
631
+ and a plain `RegExp`. Whatever is written first is warmed first.
632
+
633
+ ```js
634
+ prewarm: {
635
+ max: 500,
636
+ rps: 4,
637
+ intervalSeconds: 300,
638
+ priority: [
639
+ "/", // the home page
640
+ "/markets/:path*", // the whole markets section
641
+ /-comments$/, // a rule the pattern syntax does not cover
642
+ ],
643
+ }
644
+ ```
594
645
 
595
- Each one can be overridden by an environment variable of the same name; env
596
- takes precedence. Details: [06-caching.md](./06-caching.md).
646
+ Each numeric field can be overridden by an environment variable of the same
647
+ name; env takes precedence. Details: [06-caching.md](./06-caching.md).
597
648
 
598
649
  ## `hooks`
599
650
 
@@ -689,7 +740,9 @@ and no warning is printed.
689
740
  | `PREWARM` | `startPrewarm` | — | `0` turns prewarming off; `1` overrides `enabled: false` in the config and turns it on |
690
741
  | `PREWARM_MAX` | `prewarm` | `400` | At most how many paths are prewarmed |
691
742
  | `PREWARM_CONCURRENCY` | `prewarm` | prod 4, dev 2 | Number of parallel workers |
743
+ | `PREWARM_RPS` | `prewarm` | `0` | At most how many prewarm requests per second; `0` is unlimited |
692
744
  | `PREWARM_DELAY_MS` | `startPrewarm` | prod 500, dev 3000 | Delay of the first pass |
745
+ | `PREWARM_RETRY_DELAY_MS` | `prewarm` | `2000` | The wait before the retry pass |
693
746
  | `PREWARM_INTERVAL_SECONDS` | `startPrewarm` | `0` | If greater than 0, a periodic pass |
694
747
  | `JSKELET_VERBOSE` | `jskelet dev` | — | If `1`, all of the changed files are listed on restart |
695
748
  | `JSKELET_COLOR` | `jskelet/log` | — | If `1`, colour is forced. Because child processes write to a pipe, colour detection turns off; `jskelet dev` sets this itself. |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "jskelet",
3
- "version": "0.1.2",
3
+ "version": "0.1.3",
4
4
  "description": "A framework that feels like no framework: Express 5 + EJS server rendering, vanilla JS islands, Tailwind v4 and an in-process HTML TTL cache.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -55,6 +55,55 @@ export const DEFAULT_PREWARM = {
55
55
  concurrency: 4,
56
56
  /** İki tur arasında beklenen süre: upstream'e ani yük binmesin. */
57
57
  delayMs: 0,
58
+ /**
59
+ * Saniyedeki en fazla ısıtma isteği. 0 → sınırsız (yalnızca `concurrency`
60
+ * frenler). Upstream'i kota sınırının altında tutmanın en doğrudan yolu bu:
61
+ * paralellik ne kadar yükselse de tur bu hızın üstüne çıkmaz.
62
+ */
63
+ rps: 0,
64
+ /**
65
+ * Tekrar turundan önce beklenen süre. Rate limit pencereleri saniye
66
+ * mertebesinde; hemen tekrar denemek aynı 429'u almak demek.
67
+ */
68
+ retryDelayMs: 2000,
69
+ /**
70
+ * Liste `max`'tan uzunsa periyodik turlar kaldığı yerden devam eder.
71
+ * Böylece 10.000 yolluk bir site tek turda değil, turlar boyunca ısınır.
72
+ * `priority` eşleşen yollar her turda ısıtıldığı için rotasyon yalnızca
73
+ * kuyruğu dolaşır.
74
+ */
75
+ rotate: true,
76
+ /**
77
+ * Isıtma sırasını belirleyen desenler. String (`/haber/:slug`) ya da
78
+ * `RegExp` kabul eder; önce yazılan önce ısınır.
79
+ * @type {(string | RegExp)[]}
80
+ */
81
+ priority: [],
82
+ };
83
+
84
+ /**
85
+ * HTML önbelleğinin girdi sınırı. 500 girdi ortalama bir sayfa boyutunda
86
+ * yaklaşık 100-200 MB tutar; uzun kuyruklu siteler bunu yükseltmek yerine
87
+ * veri önbelleğine yaslanmalı (bkz. `DEFAULT_DATA_CACHE`).
88
+ */
89
+ export const DEFAULT_HTML_CACHE_MAX_ENTRIES = 500;
90
+
91
+ /**
92
+ * Upstream veri önbelleği.
93
+ *
94
+ * HTML önbelleğinden bilinçli olarak çok daha büyük: JSON, aynı sayfanın
95
+ * HTML'ine göre onlarca kat küçük. Uzun kuyruğu (on binlerce haber/etiket)
96
+ * HTML olarak tutmak imkânsız, verisini tutmak ise ucuz — ve API kotasını
97
+ * koruyan katman burası.
98
+ */
99
+ export const DEFAULT_DATA_CACHE = {
100
+ maxEntries: 10000,
101
+ /**
102
+ * TTL dolduktan sonra girdinin kaç TTL boyunca daha kullanılabileceği.
103
+ * HTML'deki 1 katsayısından yüksek: bayat veri, eksik sayfadan iyidir ve
104
+ * upstream düştüğünde tek elde kalan şey budur.
105
+ */
106
+ staleFactor: 10,
58
107
  };
59
108
 
60
109
  /** Oturuma bağlı sayfalar ısıtılmaz; uygulama kendi listesini verebilir. */
@@ -15,7 +15,8 @@
15
15
  * headers() → [{ source, headers: [{ key, value }] }]
16
16
  * redirects() → [{ source, destination, permanent?, statusCode? }]
17
17
  * rewrites() → [{ source, destination }] | { beforeFiles?, afterFiles? }
18
- * cache() → { html?: { [source]: saniye }, prewarm?: {...} }
18
+ * cache() → { html?: { [source]: saniye }, maxEntries?: number,
19
+ * data?: {...}, prewarm?: {...} }
19
20
  *
20
21
  * Fonksiyon olmayan bölümler (`brand`, `security`, `static`, `navigation`…)
21
22
  * düz nesne olarak okunur.
@@ -24,11 +25,13 @@ import fs from "node:fs";
24
25
  import path from "node:path";
25
26
  import process from "node:process";
26
27
  import { pathToFileURL } from "node:url";
27
- import { compilePattern } from "./pattern.js";
28
+ import { compilePattern, matchPattern } from "./pattern.js";
28
29
  import {
29
30
  DEFAULT_BRAND,
31
+ DEFAULT_DATA_CACHE,
30
32
  DEFAULT_DEV_GATE_BYPASS,
31
33
  DEFAULT_DIRS,
34
+ DEFAULT_HTML_CACHE_MAX_ENTRIES,
32
35
  DEFAULT_NAVIGATION,
33
36
  DEFAULT_NAVIGATION_EXCLUDE,
34
37
  DEFAULT_PREWARM,
@@ -63,7 +66,10 @@ const CONFIG_FILE = "jskelet.config.mjs";
63
66
  * @property {{ pattern: CompiledPattern, destination: string, statusCode: number }[]} redirects
64
67
  * @property {{ phase: "beforeFiles" | "afterFiles", pattern: CompiledPattern, destination: string }[]} rewrites
65
68
  * @property {{ pattern: CompiledPattern, seconds: number }[]} html
69
+ * @property {number} htmlMaxEntries HTML önbelleğinin girdi sınırı.
70
+ * @property {Record<string, unknown>} data Upstream veri önbelleği ayarları.
66
71
  * @property {Record<string, unknown>} prewarm
72
+ * @property {{ source: string, test: (pathname: string) => boolean }[]} prewarmPriority
67
73
  * @property {Record<string, unknown>} brand
68
74
  * @property {Record<string, Function>} hooks
69
75
  * @property {string} layout Layout `.ejs` dosyasının mutlak yolu.
@@ -168,8 +174,40 @@ function normalizeRewrites(raw) {
168
174
  }
169
175
 
170
176
  /**
177
+ * Isıtma sırası desenleri. İki biçim kabul edilir: config'in her yerinde
178
+ * geçerli olan `/haber/:slug` sözdizimi ve doğrudan `RegExp` — ikincisi
179
+ * "sonu `-yorumlar` ile bitenler" gibi desen sözdiziminin karşılamadığı
180
+ * kuralları yazabilmek için.
181
+ *
171
182
  * @param {unknown} raw
172
- * @returns {{ html: ResolvedConfig["html"], prewarm: Record<string, unknown> }}
183
+ * @returns {ResolvedConfig["prewarmPriority"]}
184
+ */
185
+ function normalizePriority(raw) {
186
+ /** @type {ResolvedConfig["prewarmPriority"]} */
187
+ const out = [];
188
+
189
+ for (const entry of asArray(raw, "cache().prewarm.priority")) {
190
+ if (entry instanceof RegExp) {
191
+ out.push({ source: String(entry), test: (pathname) => entry.test(pathname) });
192
+ continue;
193
+ }
194
+
195
+ const pattern = compilePattern(entry);
196
+ if (!pattern) continue;
197
+ out.push({
198
+ source: pattern.source,
199
+ test: (pathname) => matchPattern(pattern, pathname) !== null,
200
+ });
201
+ }
202
+
203
+ return out;
204
+ }
205
+
206
+ /**
207
+ * @param {unknown} raw
208
+ * @returns {{ html: ResolvedConfig["html"], htmlMaxEntries: number,
209
+ * data: Record<string, unknown>, prewarm: Record<string, unknown>,
210
+ * prewarmPriority: ResolvedConfig["prewarmPriority"] }}
173
211
  */
174
212
  function normalizeCache(raw) {
175
213
  /** @type {ResolvedConfig["html"]} */
@@ -182,7 +220,21 @@ function normalizeCache(raw) {
182
220
  html.push({ pattern, seconds: value });
183
221
  }
184
222
 
185
- return { html, prewarm: { ...DEFAULT_PREWARM, ...(raw?.prewarm ?? {}) } };
223
+ const prewarm = { ...DEFAULT_PREWARM, ...(raw?.prewarm ?? {}) };
224
+ const maxEntries = Number(raw?.maxEntries);
225
+
226
+ return {
227
+ html,
228
+ htmlMaxEntries:
229
+ Number.isFinite(maxEntries) && maxEntries > 0
230
+ ? Math.floor(maxEntries)
231
+ : DEFAULT_HTML_CACHE_MAX_ENTRIES,
232
+ data: { ...DEFAULT_DATA_CACHE, ...(raw?.data ?? {}) },
233
+ // Desenler derlenmiş hâlde ayrı alanda tutulur: `prewarm` sayısal
234
+ // ayarların düz torbası olarak kalsın, her turda yeniden derlenmesin.
235
+ prewarm,
236
+ prewarmPriority: normalizePriority(prewarm.priority),
237
+ };
186
238
  }
187
239
 
188
240
  /** Speculation Rules'un tanıdığı eagerness değerleri. */
@@ -388,7 +440,8 @@ export async function loadConfig(options = {}) {
388
440
  section("cache"),
389
441
  ]);
390
442
 
391
- const { html, prewarm } = normalizeCache(cache);
443
+ const { html, htmlMaxEntries, data, prewarm, prewarmPriority } =
444
+ normalizeCache(cache);
392
445
  const dirs = resolveDirs(root, source.paths);
393
446
  const brand = { ...DEFAULT_BRAND, ...(source.brand ?? {}) };
394
447
 
@@ -400,7 +453,10 @@ export async function loadConfig(options = {}) {
400
453
  redirects: normalizeRedirects(redirects),
401
454
  rewrites: normalizeRewrites(rewrites),
402
455
  html,
456
+ htmlMaxEntries,
457
+ data,
403
458
  prewarm,
459
+ prewarmPriority,
404
460
  brand,
405
461
  hooks: source.hooks ?? {},
406
462
  layout: resolveLayout(dirs, source.layout),
package/src/index.js CHANGED
@@ -45,6 +45,13 @@ export {
45
45
  getHtmlCacheSize,
46
46
  withHtmlCache,
47
47
  } from "./server/html-cache.js";
48
+ export {
49
+ clearDataCache,
50
+ dataCache,
51
+ getDataCacheEntries,
52
+ getDataCacheSize,
53
+ withDataCache,
54
+ } from "./server/data-cache.js";
48
55
  export { prewarm, prewarmProgress } from "./server/prewarm.js";
49
56
  export { createProxy } from "./server/middleware/upstream-proxy.js";
50
57
  export { getConfig, loadConfig } from "./config/index.js";