jskelet 0.6.2 → 0.6.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (155) hide show
  1. package/AGENTS.md +136 -136
  2. package/CHANGELOG.md +620 -596
  3. package/LICENSE +21 -21
  4. package/bin/jskelet.mjs +130 -130
  5. package/docs/01-baslangic.md +291 -291
  6. package/docs/02-mimari.md +310 -309
  7. package/docs/03-routing.md +515 -515
  8. package/docs/04-render-ve-sablonlar.md +661 -661
  9. package/docs/05-islands.md +486 -486
  10. package/docs/06-cache.md +1443 -1423
  11. package/docs/07-yapilandirma.md +12 -6
  12. package/docs/08-build.md +429 -428
  13. package/docs/09-dev-araclari.md +364 -364
  14. package/docs/10-dagitim.md +338 -338
  15. package/docs/12-panel-ve-oturum.md +478 -478
  16. package/docs/README.md +83 -83
  17. package/docs/en/01-getting-started.md +298 -298
  18. package/docs/en/02-architecture.md +329 -328
  19. package/docs/en/03-routing.md +531 -531
  20. package/docs/en/04-rendering.md +669 -669
  21. package/docs/en/05-islands.md +497 -497
  22. package/docs/en/06-caching.md +1453 -1431
  23. package/docs/en/07-configuration.md +1219 -1214
  24. package/docs/en/08-build.md +447 -446
  25. package/docs/en/09-dev-tools.md +373 -373
  26. package/docs/en/10-deployment.md +340 -340
  27. package/docs/en/11-migration.md +398 -398
  28. package/docs/en/12-dashboards-and-sessions.md +488 -488
  29. package/docs/en/README.md +87 -87
  30. package/package.json +137 -137
  31. package/src/build/ensure-build.mjs +19 -19
  32. package/src/build/paths.mjs +153 -153
  33. package/src/build/resolve-peer.mjs +36 -36
  34. package/src/build/tasks/client.mjs +349 -349
  35. package/src/build/tasks/css.mjs +235 -235
  36. package/src/build/tasks/fonts.mjs +146 -146
  37. package/src/build/tasks/icons.mjs +357 -357
  38. package/src/build/tasks/images.mjs +244 -244
  39. package/src/build/tasks/precompress.mjs +78 -78
  40. package/src/build/tasks/templates.mjs +20 -20
  41. package/src/client/admin/i18n.js +764 -764
  42. package/src/client/admin/login.html +74 -74
  43. package/src/client/admin/panel.css +809 -809
  44. package/src/client/admin/panel.html +495 -495
  45. package/src/client/admin/panel.js +1251 -1251
  46. package/src/client/devtools/report.html +185 -185
  47. package/src/client/devtools/report.js +745 -745
  48. package/src/client/devtools/seo.js +628 -628
  49. package/src/client/dom.js +95 -95
  50. package/src/client/form.js +192 -192
  51. package/src/client/index.js +45 -45
  52. package/src/client/registry.js +305 -305
  53. package/src/client/safe-image.js +91 -91
  54. package/src/client/shared-cookie.js +225 -225
  55. package/src/client/store.js +36 -36
  56. package/src/client/swap.js +188 -188
  57. package/src/compile/codegen.js +336 -336
  58. package/src/compile/compile-all.js +149 -149
  59. package/src/compile/errors.js +66 -66
  60. package/src/compile/expr.js +409 -409
  61. package/src/compile/index.js +17 -17
  62. package/src/compile/parse.js +541 -541
  63. package/src/compile/resolve.js +211 -211
  64. package/src/compile/scan-exports.js +51 -51
  65. package/src/config/defaults.js +17 -1
  66. package/src/config/index.js +13 -0
  67. package/src/config/pattern.js +107 -107
  68. package/src/generate.mjs +163 -163
  69. package/src/http/control-flow.js +71 -71
  70. package/src/http/cookies-entry.js +21 -21
  71. package/src/http/cookies.js +277 -277
  72. package/src/http/request-cache.js +46 -46
  73. package/src/http/request-context.js +165 -165
  74. package/src/http/shared-cookie.js +178 -178
  75. package/src/index.js +101 -101
  76. package/src/init.mjs +230 -230
  77. package/src/migrate/apply.mjs +262 -262
  78. package/src/migrate/babel.mjs +79 -79
  79. package/src/migrate/classify.mjs +155 -155
  80. package/src/migrate/config.mjs +126 -126
  81. package/src/migrate/fs-walk.mjs +191 -191
  82. package/src/migrate/parse.mjs +26 -26
  83. package/src/migrate/scan.mjs +177 -177
  84. package/src/migrate/transform/expr-source.mjs +168 -168
  85. package/src/migrate/transform/island.mjs +67 -67
  86. package/src/migrate/transform/jsx-to-component.mjs +302 -302
  87. package/src/migrate/transform/jsx-to-jsk.mjs +330 -330
  88. package/src/migrate/transform/page-split.mjs +435 -435
  89. package/src/migrate/write.mjs +81 -81
  90. package/src/migrate.mjs +171 -171
  91. package/src/runtime/alias-hooks.mjs +119 -119
  92. package/src/runtime/register.mjs +4 -4
  93. package/src/server/admin/actions.js +229 -229
  94. package/src/server/admin/auth.js +125 -125
  95. package/src/server/admin/event-log.js +151 -151
  96. package/src/server/admin/gate.js +209 -209
  97. package/src/server/admin/inventory.js +188 -188
  98. package/src/server/admin/mount.js +56 -56
  99. package/src/server/admin/router.js +216 -216
  100. package/src/server/admin/snapshot.js +241 -241
  101. package/src/server/assets.js +147 -147
  102. package/src/server/auth/handoff.js +309 -309
  103. package/src/server/cache-blob.js +70 -0
  104. package/src/server/cache-deps.js +42 -42
  105. package/src/server/cache-vary.js +113 -113
  106. package/src/server/cloudflare.js +607 -607
  107. package/src/server/create-app.js +366 -366
  108. package/src/server/data-cache.js +553 -462
  109. package/src/server/dev/report.js +485 -485
  110. package/src/server/dev/socket.js +170 -170
  111. package/src/server/dev/version-check.mjs +139 -139
  112. package/src/server/disk-cache.js +233 -0
  113. package/src/server/ejs-adapter.js +59 -59
  114. package/src/server/html-cache.js +1196 -1122
  115. package/src/server/image-optimizer.js +500 -407
  116. package/src/server/logs/access-middleware.js +66 -66
  117. package/src/server/logs/file-sink.js +193 -66
  118. package/src/server/logs/pipeline.js +165 -158
  119. package/src/server/logs/s3-put.js +214 -214
  120. package/src/server/logs/s3-sink.js +112 -112
  121. package/src/server/metadata.js +102 -102
  122. package/src/server/middleware/compression.js +205 -205
  123. package/src/server/middleware/csrf.js +134 -134
  124. package/src/server/middleware/dev-gate.js +75 -75
  125. package/src/server/middleware/headers.js +37 -37
  126. package/src/server/middleware/redirects.js +32 -32
  127. package/src/server/middleware/robots-txt.js +341 -341
  128. package/src/server/middleware/static-precompressed.js +121 -100
  129. package/src/server/middleware/trailing-slash.js +53 -53
  130. package/src/server/middleware/upstream-proxy.js +141 -141
  131. package/src/server/og-image.js +356 -356
  132. package/src/server/port-guard.js +255 -255
  133. package/src/server/prewarm.js +1082 -1058
  134. package/src/server/redis.js +588 -569
  135. package/src/server/render.js +4 -4
  136. package/src/server/router.js +157 -157
  137. package/src/server/status-page.js +265 -265
  138. package/src/server/upstream-limiter.js +376 -376
  139. package/src/server/upstream-tracking.js +166 -166
  140. package/src/shared/cookie-domain.js +66 -66
  141. package/src/start.mjs +22 -22
  142. package/src/templates/layout.ejs +30 -30
  143. package/src/templates/layout.jsk +30 -30
  144. package/src/version.mjs +31 -31
  145. package/src/views/components/loader.js +101 -101
  146. package/src/views/helpers/html.js +102 -102
  147. package/src/views/helpers/tags.js +375 -375
  148. package/types/config/defaults.d.ts +15 -1
  149. package/types/config/index.d.ts +8 -0
  150. package/types/server/cache-blob.d.ts +13 -0
  151. package/types/server/data-cache.d.ts +9 -0
  152. package/types/server/disk-cache.d.ts +36 -0
  153. package/types/server/html-cache.d.ts +26 -3
  154. package/types/server/logs/file-sink.d.ts +16 -5
  155. package/types/server/redis.d.ts +2 -1
@@ -1,373 +1,373 @@
1
- # 09 — Development tools
2
-
3
- This document explains what `jskelet dev` does and why it does it that way: how
4
- the two child processes are managed, the shape of the terminal output, the
5
- watched directories and the reason a custom watcher was written instead of using
6
- `node --watch`, the distinction between CSS hot-swap and a full reload, the
7
- devtools overlay opened with `Alt+D`, the detailed report page, and the dev gate
8
- built on `DEV_TOKEN`. The build steps themselves are in
9
- [08-build.md](./08-build.md).
10
-
11
- ## The `jskelet dev` flow
12
-
13
- The command manages two long-lived child processes in a single terminal:
14
-
15
- ```
16
- jskelet dev
17
- ├─ build watch node src/build/build.mjs --watch
18
- └─ server node [--env-file=.env] --import <register.mjs> src/start.mjs
19
- ```
20
-
21
- `NODE_ENV=development` is assigned here in a platform-independent way — no
22
- `cross-env` needed. The child processes also receive `JSKELET_CHILD=1`
23
- (suppresses the build banner) and, if a TTY is present, `JSKELET_COLOR=1`
24
- (forces color on piped output).
25
-
26
- If the listen port (`PORT`, default `3000`) is already taken, the server
27
- **does not start**; the error line includes the PID and a `--murder` hint.
28
- `jskelet dev --murder` kills the listener and binds the same port (for a
29
- process left running in another terminal).
30
-
31
- Startup order: banner → build steps → server ready → `Ready` summary. The
32
- summary is printed once both the build and the server are ready; otherwise it
33
- got buried among the build lines arriving afterwards.
34
-
35
- The server signals readiness with a single line inside `startServer`:
36
-
37
- ```
38
- jskelet → http://localhost:3000 (development)
39
- ```
40
-
41
- The shape of this line is a contract; the dev script parses it and prints the
42
- summary line accordingly.
43
-
44
- ### The shape of the terminal output
45
-
46
- Child process output does not stream through as-is. There are two regions and
47
- they never mix:
48
-
49
- 1. **Startup:** banner, aligned build lines, `Ready` summary.
50
- 2. **Runtime:** timestamped, single-line events (HTTP requests, CSS rebuild,
51
- server restart).
52
-
53
- Error stacks are turned into a framed box: because stack lines arrive in
54
- fragments, they are collected after a short silence (60 ms), the error name and
55
- message are parsed, the first three frames are shown, and the project root is
56
- shortened to `.`. When you are developing your own framework, not losing the
57
- error in the stream is the detail that genuinely makes a difference.
58
-
59
- Color carries meaning: `✓` green, `✖` red, `⚠` yellow, `↻` cyan; durations and
60
- paths gray. No decorative color is used. If `NO_COLOR` is set, no color is used
61
- at all.
62
-
63
- `Ctrl+C` (SIGINT/SIGTERM) shuts down all child processes. If a child exits with
64
- a non-zero code (other than an expected restart), an error is printed and the
65
- dev process exits too.
66
-
67
- ## Watch directories
68
-
69
- Server restarts are managed by the framework's own watcher.
70
-
71
- ```js
72
- WATCH_DIRS = [
73
- config.dirs.routes,
74
- config.dirs.views,
75
- <root>/lib,
76
- ...config.watch, // jskelet.config.mjs → watch
77
- ]
78
- ```
79
-
80
- The `jskelet.config.mjs` file itself is watched as well: when the config
81
- changes, both the server and the build must come up with the new settings.
82
-
83
- Watched extensions: `.js`, `.mjs`, `.json`, `.jsk`, `.ejs`.
84
-
85
- `views` is watched too, because most components live in
86
- `views/components/**.js` and, since those modules are imported into the server
87
- once, changes made without a restart never reached the browser (that "I edited
88
- the template and nothing changed" feeling comes from here).
89
-
90
- `client/` and `styles/` are **not** in this list: esbuild and the CSS watchers
91
- handle them on their own ([08-build.md](./08-build.md)).
92
-
93
- If a directory cannot be watched, a warning is printed and no automatic restart
94
- happens for that directory; everything else keeps working.
95
-
96
- ### Why `node --watch` was not used
97
-
98
- Even when given `--watch-path`, Node watched the project root in this setup.
99
- Whenever build output (`public/assets`, `manifest.json`) or the dev tooling log
100
- was written, the server restarted for nothing — in fact a self-feeding loop was
101
- set up: restart → startup warning → write → restart.
102
-
103
- The custom watcher does three things:
104
-
105
- 1. **Watches only server sources.**
106
- 2. **Coalesces changes** (250 ms) and reports which files changed.
107
- 3. **Filters out phantom events.** On Windows, `fs.watch` can emit events for a
108
- file's neighbors when it is written; without comparing `mtime`, a single save
109
- turned into two restarts. Current timestamps are read up front at startup, so
110
- the first phantom event is filtered out as well.
111
-
112
- The restart line shows the changed file, or how many changed:
113
-
114
- ```
115
- 21:04:12 ↻ server restarting… routes/10-pages.mjs
116
- 21:04:12 server restarted 412ms
117
- ```
118
-
119
- If `JSKELET_VERBOSE=1` is set, all files are listed when more than one changed.
120
-
121
- ## CSS hot-swap and full reload
122
-
123
- The dev server watches `.jskelet/manifest.json` and broadcasts events to the
124
- browser over the live channel (`<devBasePath>/ws`). Since the manifest is
125
- rewritten on every build round, change detection is done through the manifest.
126
-
127
- | What changed | Behavior |
128
- | --- | --- |
129
- | Only `.css` keys | **CSS hot-swap:** each changed stylesheet is swapped, the page is not reloaded. State and scroll position are preserved. |
130
- | `main.js`, the sprite, another asset, or more than one key | **Full reload** |
131
-
132
- In both cases the HTML cache is cleared first: the stored HTML carries the old
133
- hashed asset URLs and, if not cleared, the page keeps asking for a deleted file.
134
-
135
- Manifest events are coalesced over 120 ms. If watching is not supported, live
136
- reload is disabled and everything else keeps working.
137
-
138
- When the server restarts, the overlay figures it out from the **boot id**: every
139
- process broadcasts a unique `boot` value, the overlay sees the change, shows the
140
- "restarted" note and does not reset its own state.
141
-
142
- ## The live channel
143
-
144
- Everything the overlay shows — statistics, live reload and CSS hot-swap events —
145
- arrives over a single WebSocket (`<devBasePath>/ws`). The panel used to poll for
146
- statistics every two seconds, so every open tab kept hitting the server even
147
- while the panel was closed. Now the server pushes as things change: when a
148
- request or an error is recorded (coalesced over 120 ms), and every two seconds so
149
- the time-based fields (uptime, memory, the prewarm counter) stay fresh. The
150
- heartbeat is deliberately independent of prewarming: if the channel's tempo
151
- followed a background job, the panel would be tied to that job's rhythm. Nothing
152
- is computed when no panel is connected.
153
-
154
- The handshake happens on the HTTP `upgrade` event, and that event never reaches
155
- the middleware chain, so the channel is attached straight to the server after
156
- `listen` (`attachDevSocket`). The server side pulls in no dependency such as
157
- `ws`: all it needs is to write server-to-client text frames and to answer the
158
- client's ping/close frames.
159
-
160
- If the socket opens and later drops — that is, the server is restarting — it
161
- reconnects every half second and the indicator reads "server restarting…" in the
162
- meantime. If it never opens, it is retried four times with a widening gap: the
163
- page may have been loaded during the server's restart window, and a single
164
- failure does not mean the channel is unavailable. If those attempts fail too (a
165
- proxy in between may not pass WebSocket through), the overlay falls back to the
166
- old path: the `/events` SSE stream plus polling `/stats`.
167
-
168
- ## Devtools overlay
169
-
170
- A floating bubble in the bottom right; opened with `Alt+D`, closed with `Esc` or
171
- by clicking the backdrop. It is only emitted by the layout when
172
- `NODE_ENV=development`:
173
-
174
- ```ejs
175
- <% if (devtools) { %>
176
- <script type="module" src="<%= devBasePath %>/overlay.js"></script>
177
- <% } %>
178
- ```
179
-
180
- The overlay file is **not part of the build.** The server serves it raw from the
181
- framework package; that is why there is no bundler involved. It loads sibling
182
- modules such as `seo.js` with native ESM imports, and it adds nothing to the
183
- production output. The entire UI lives inside a shadow DOM and does not mix with
184
- the page's CSS.
185
-
186
- What it shows:
187
-
188
- - **Errors:** browser-side JS errors, resource loading errors
189
- (`img`/`script`/`link`), the server's `console.error` / `console.warn`
190
- output, and failed SSR / browser `fetch` calls (4xx/5xx or network). Each
191
- record carries a **page path**, **API URL**, and (on the client) an **island
192
- name** when known; the response body opens under **show details** as JSON
193
- instead of `[object Object]`. On the server side `console` is wrapped so
194
- warnings do not get lost in the terminal; upstream failures still land in the
195
- overlay even when the app's own logger writes them only to stderr.
196
- - **SEO:** a client-side scan of the current page — title and meta description
197
- length, `html lang`, viewport, canonical, robots/`noindex`, Open Graph and
198
- Twitter tags, H1/outline, image `alt`, empty links, and JSON-LD parse errors.
199
- Findings appear in the panel; turning on **Highlight issues on the page**
200
- draws red (error) or yellow (warning) boxes on the elements, with a short
201
- title on the border. Clicking the label (or a row in the panel) opens the
202
- full explanation. The scan lives in `/seo.js` next to the overlay and is not
203
- part of the production bundle.
204
- - **Requests:** the method, path, status, duration and `X-JSkelet-Cache` value
205
- of every HTML request. The same lines are printed to the terminal too.
206
- - **Web Vitals:** TTFB, FCP, LCP, CLS, INP, DCL, load, long task count and
207
- blocking time.
208
- - **Prewarm:** the progress of the warming round; it can be triggered manually
209
- from the panel, and individual paths can be retried.
210
- - **Process:** pid, Node version, uptime, RSS and heap usage.
211
- - **Version:** the installed JSkelet version compared against the `latest` tag
212
- on npm. When a newer release exists, the **Server** tab grows an `update` chip
213
- and a line that copies the upgrade command. The lookup runs once, 1.5 seconds
214
- after boot, is cached for six hours in `os.tmpdir()` and is skipped silently
215
- when the registry is unreachable. Set `JSKELET_VERSION_CHECK=0` to disable it.
216
-
217
- Warming requests (`user-agent: jskelet-prewarm`) are filtered out of both the
218
- terminal and the request list: hundreds of requests should not flood the view.
219
- Progress appears in the badge next to the bubble.
220
-
221
- ### Why state is written to `os.tmpdir()`
222
-
223
- If request and error records lived in process memory, history would be erased on
224
- every restart and the overlay would come up empty. So the records are carried
225
- across restarts in a file.
226
-
227
- The file is **not written into the project tree**: every write triggered the
228
- watcher and restarted the server, which set up a self-feeding loop (restart →
229
- startup warning → write → restart). Instead, the file is written to
230
- `os.tmpdir()/jskelet-devtools-<hash of the project root>.json`; thanks to the
231
- hash, multiple JSkelet projects on the same machine do not overwrite each
232
- other's records.
233
-
234
- The write happens after 300 ms of silence rather than on every request, and its
235
- failure does not stop the dev flow. At most 50 requests and 50 errors are kept.
236
-
237
- The panel's open/closed state, its active tab and the browser error log are kept
238
- in tab memory (`sessionStorage`), so after a reload the panel comes back with
239
- the same tab.
240
-
241
- ## Report page
242
-
243
- The bubble shows the current state; the report page produces a view of the whole
244
- site. The address:
245
-
246
- ```
247
- http://localhost:3000/__jskelet/dev/report
248
- ```
249
-
250
- (Or wherever `brand.devBasePath` points, if you changed it.)
251
-
252
- Its contents:
253
-
254
- - **Pages:** the Web Vitals measurements of every visited page, resource count
255
- and total bytes (broken down by type), island status (how many are ready, and
256
- their names), API calls made in the browser, and the size/duration/cache
257
- status of the SSR output. Pages that were never visited but were warmed are
258
- listed too: the SSR side is known, the client measurements stay empty.
259
- - **Server API calls:** outbound `fetch` calls made during SSR — URL, host,
260
- method, status, duration, bytes, which page was rendering, and a body summary
261
- on failures. `globalThis.fetch` is only wrapped in development; the production
262
- path is left untouched. Requests to our own server (warming, health check) do
263
- not count as API calls. Failures also appear on the overlay Errors tab.
264
- - **Build output:** the raw/gzip/brotli size of every asset in the manifest, and
265
- chunk analysis from esbuild's metafile — the size of each output, which
266
- sources it is made of, which chunks it imports. Sources are reduced to
267
- readable groups (package name or parent folder), so the question "which
268
- library accounts for 40 kB of this chunk" can be answered.
269
- - **HTML cache:** entry count and a dump (key, bytes, status, whether it is
270
- stale, how many seconds until it expires, which encodings are stored).
271
- - **Prewarm:** the full result of the last round.
272
- - **Request and error logs.**
273
-
274
- Measurements live on the server, not in the browser tab; resetting is done from
275
- the server as well. Size calculations are not repeated unless the file changed.
276
-
277
- The report layer is only loaded in development and never enters the production
278
- output.
279
-
280
- ## Dev endpoints
281
-
282
- Under `brand.devBasePath` (default `/__jskelet/dev`):
283
-
284
- | Path | Method | Job |
285
- | --- | --- | --- |
286
- | `/overlay.js` | GET | The overlay script |
287
- | `/seo.js` | GET | SEO scan + page highlight helper (imported by the overlay) |
288
- | `/logo.png` | GET | The overlay logo |
289
- | `/ws` | GET (upgrade) | Live channel: statistics, live reload and CSS hot-swap events |
290
- | `/events` | GET | SSE: the fallback event stream, used only when WebSocket cannot be established |
291
- | `/stats` | GET | Current statistics; the data endpoint of that same fallback |
292
- | `/report` | GET | The report page (HTML) |
293
- | `/report.js` | GET | The report page's script |
294
- | `/report/data` | GET | The report's single data source (JSON) |
295
- | `/vitals` | POST | The measurement bundle sent by the overlay |
296
- | `/report/clear` | POST | Resets page measurements and server API records |
297
- | `/prewarm` | POST | Triggers warming manually. If the body has `paths`, only those paths; 409 if warming is already running. |
298
- | `/clear` | POST | Resets the request and error logs |
299
-
300
- All of these endpoints are mounted by `mountDevtools()` only when
301
- `NODE_ENV=development`; thanks to the dynamic import, nothing is loaded into the
302
- production process.
303
-
304
- ## Dev gate — `DEV_TOKEN`
305
-
306
- To hide an environment that is not public yet. **The framework does not require
307
- the token:** a `DEV_TOKEN` sitting in the environment does not lock the site.
308
- You turn the gate on.
309
-
310
- ```bash
311
- DEV_GATE=1 DEV_TOKEN=a-long-random-string npm start
312
- ```
313
-
314
- The same thing from config:
315
-
316
- ```js
317
- export default {
318
- devGate: true,
319
- };
320
- ```
321
-
322
- Access:
323
-
324
- ```
325
- https://staging.example.com/?dev_token=a-long-random-string
326
- ```
327
-
328
- Behavior:
329
-
330
- - **404, not 403.** A 403 confirms the environment exists; a 404 acts as if it
331
- never did.
332
- - Once the token arrives as a query parameter, it is written to a cookie
333
- (`Path=/`, `SameSite=Lax`, 14 days), so sharing the link is enough. The cookie
334
- and parameter name is `brand.devTokenCookie` (default `dev_token`).
335
- - The **exact** paths in the `devGateBypass` list are open under all conditions.
336
- Default: `/api/healthcheck`, `/robots.txt`, `/sitemap.xml`,
337
- `/site.webmanifest`, `/favicon.ico`. If your health check lives at a different
338
- path, remember to add it to this list, otherwise your orchestrator will see a
339
- 404.
340
- - While `devGate` is off (the default) or `DEV_TOKEN` is empty, the middleware
341
- passes the request through. A `DEV_TOKEN` that leaked into a production task
342
- does not ask visitors for a token; startup prints a warning.
343
- - `DEV_GATE=0` turns the gate off even when config says `devGate: true`.
344
- - While the gate is on, warming carries the token as a cookie; without it every
345
- page gets a 404 and the cache never fills up
346
- ([06-caching.md](./06-caching.md)).
347
-
348
- In the middleware chain the gate sits after `headers` and **before**
349
- `redirects`: an environment that is not public yet should not leak even its
350
- redirect rules.
351
-
352
- ## Differences between development and production
353
-
354
- | Topic | Development | Production |
355
- | --- | --- | --- |
356
- | EJS template cache | Off | On |
357
- | Manifest reading | On every request | Once |
358
- | Image manifest | On every call | Once |
359
- | Broken route module | Warn + skip | Throw |
360
- | Devtools and report | Mounted | Never loaded |
361
- | `globalThis.fetch` | Wrapped (measurement) | Untouched |
362
- | Prewarm concurrency | 1 | 4 |
363
- | Prewarm rate limit | 4 requests/second | Unlimited |
364
- | Prewarm delay | 3000 ms | 500 ms |
365
- | Missing icon warning | Emitted | Not emitted |
366
- | Precompress | Does not run in watch | Runs |
367
- | Image optimization | Does not run in watch | Runs |
368
-
369
- ## What's next
370
-
371
- - Details of the build steps: [08-build.md](./08-build.md)
372
- - Going to production: [10-deployment.md](./10-deployment.md)
373
- - Reading and clearing the cache: [06-caching.md](./06-caching.md)
1
+ # 09 — Development tools
2
+
3
+ This document explains what `jskelet dev` does and why it does it that way: how
4
+ the two child processes are managed, the shape of the terminal output, the
5
+ watched directories and the reason a custom watcher was written instead of using
6
+ `node --watch`, the distinction between CSS hot-swap and a full reload, the
7
+ devtools overlay opened with `Alt+D`, the detailed report page, and the dev gate
8
+ built on `DEV_TOKEN`. The build steps themselves are in
9
+ [08-build.md](./08-build.md).
10
+
11
+ ## The `jskelet dev` flow
12
+
13
+ The command manages two long-lived child processes in a single terminal:
14
+
15
+ ```
16
+ jskelet dev
17
+ ├─ build watch node src/build/build.mjs --watch
18
+ └─ server node [--env-file=.env] --import <register.mjs> src/start.mjs
19
+ ```
20
+
21
+ `NODE_ENV=development` is assigned here in a platform-independent way — no
22
+ `cross-env` needed. The child processes also receive `JSKELET_CHILD=1`
23
+ (suppresses the build banner) and, if a TTY is present, `JSKELET_COLOR=1`
24
+ (forces color on piped output).
25
+
26
+ If the listen port (`PORT`, default `3000`) is already taken, the server
27
+ **does not start**; the error line includes the PID and a `--murder` hint.
28
+ `jskelet dev --murder` kills the listener and binds the same port (for a
29
+ process left running in another terminal).
30
+
31
+ Startup order: banner → build steps → server ready → `Ready` summary. The
32
+ summary is printed once both the build and the server are ready; otherwise it
33
+ got buried among the build lines arriving afterwards.
34
+
35
+ The server signals readiness with a single line inside `startServer`:
36
+
37
+ ```
38
+ jskelet → http://localhost:3000 (development)
39
+ ```
40
+
41
+ The shape of this line is a contract; the dev script parses it and prints the
42
+ summary line accordingly.
43
+
44
+ ### The shape of the terminal output
45
+
46
+ Child process output does not stream through as-is. There are two regions and
47
+ they never mix:
48
+
49
+ 1. **Startup:** banner, aligned build lines, `Ready` summary.
50
+ 2. **Runtime:** timestamped, single-line events (HTTP requests, CSS rebuild,
51
+ server restart).
52
+
53
+ Error stacks are turned into a framed box: because stack lines arrive in
54
+ fragments, they are collected after a short silence (60 ms), the error name and
55
+ message are parsed, the first three frames are shown, and the project root is
56
+ shortened to `.`. When you are developing your own framework, not losing the
57
+ error in the stream is the detail that genuinely makes a difference.
58
+
59
+ Color carries meaning: `✓` green, `✖` red, `⚠` yellow, `↻` cyan; durations and
60
+ paths gray. No decorative color is used. If `NO_COLOR` is set, no color is used
61
+ at all.
62
+
63
+ `Ctrl+C` (SIGINT/SIGTERM) shuts down all child processes. If a child exits with
64
+ a non-zero code (other than an expected restart), an error is printed and the
65
+ dev process exits too.
66
+
67
+ ## Watch directories
68
+
69
+ Server restarts are managed by the framework's own watcher.
70
+
71
+ ```js
72
+ WATCH_DIRS = [
73
+ config.dirs.routes,
74
+ config.dirs.views,
75
+ <root>/lib,
76
+ ...config.watch, // jskelet.config.mjs → watch
77
+ ]
78
+ ```
79
+
80
+ The `jskelet.config.mjs` file itself is watched as well: when the config
81
+ changes, both the server and the build must come up with the new settings.
82
+
83
+ Watched extensions: `.js`, `.mjs`, `.json`, `.jsk`, `.ejs`.
84
+
85
+ `views` is watched too, because most components live in
86
+ `views/components/**.js` and, since those modules are imported into the server
87
+ once, changes made without a restart never reached the browser (that "I edited
88
+ the template and nothing changed" feeling comes from here).
89
+
90
+ `client/` and `styles/` are **not** in this list: esbuild and the CSS watchers
91
+ handle them on their own ([08-build.md](./08-build.md)).
92
+
93
+ If a directory cannot be watched, a warning is printed and no automatic restart
94
+ happens for that directory; everything else keeps working.
95
+
96
+ ### Why `node --watch` was not used
97
+
98
+ Even when given `--watch-path`, Node watched the project root in this setup.
99
+ Whenever build output (`public/assets`, `manifest.json`) or the dev tooling log
100
+ was written, the server restarted for nothing — in fact a self-feeding loop was
101
+ set up: restart → startup warning → write → restart.
102
+
103
+ The custom watcher does three things:
104
+
105
+ 1. **Watches only server sources.**
106
+ 2. **Coalesces changes** (250 ms) and reports which files changed.
107
+ 3. **Filters out phantom events.** On Windows, `fs.watch` can emit events for a
108
+ file's neighbors when it is written; without comparing `mtime`, a single save
109
+ turned into two restarts. Current timestamps are read up front at startup, so
110
+ the first phantom event is filtered out as well.
111
+
112
+ The restart line shows the changed file, or how many changed:
113
+
114
+ ```
115
+ 21:04:12 ↻ server restarting… routes/10-pages.mjs
116
+ 21:04:12 server restarted 412ms
117
+ ```
118
+
119
+ If `JSKELET_VERBOSE=1` is set, all files are listed when more than one changed.
120
+
121
+ ## CSS hot-swap and full reload
122
+
123
+ The dev server watches `.jskelet/manifest.json` and broadcasts events to the
124
+ browser over the live channel (`<devBasePath>/ws`). Since the manifest is
125
+ rewritten on every build round, change detection is done through the manifest.
126
+
127
+ | What changed | Behavior |
128
+ | --- | --- |
129
+ | Only `.css` keys | **CSS hot-swap:** each changed stylesheet is swapped, the page is not reloaded. State and scroll position are preserved. |
130
+ | `main.js`, the sprite, another asset, or more than one key | **Full reload** |
131
+
132
+ In both cases the HTML cache is cleared first: the stored HTML carries the old
133
+ hashed asset URLs and, if not cleared, the page keeps asking for a deleted file.
134
+
135
+ Manifest events are coalesced over 120 ms. If watching is not supported, live
136
+ reload is disabled and everything else keeps working.
137
+
138
+ When the server restarts, the overlay figures it out from the **boot id**: every
139
+ process broadcasts a unique `boot` value, the overlay sees the change, shows the
140
+ "restarted" note and does not reset its own state.
141
+
142
+ ## The live channel
143
+
144
+ Everything the overlay shows — statistics, live reload and CSS hot-swap events —
145
+ arrives over a single WebSocket (`<devBasePath>/ws`). The panel used to poll for
146
+ statistics every two seconds, so every open tab kept hitting the server even
147
+ while the panel was closed. Now the server pushes as things change: when a
148
+ request or an error is recorded (coalesced over 120 ms), and every two seconds so
149
+ the time-based fields (uptime, memory, the prewarm counter) stay fresh. The
150
+ heartbeat is deliberately independent of prewarming: if the channel's tempo
151
+ followed a background job, the panel would be tied to that job's rhythm. Nothing
152
+ is computed when no panel is connected.
153
+
154
+ The handshake happens on the HTTP `upgrade` event, and that event never reaches
155
+ the middleware chain, so the channel is attached straight to the server after
156
+ `listen` (`attachDevSocket`). The server side pulls in no dependency such as
157
+ `ws`: all it needs is to write server-to-client text frames and to answer the
158
+ client's ping/close frames.
159
+
160
+ If the socket opens and later drops — that is, the server is restarting — it
161
+ reconnects every half second and the indicator reads "server restarting…" in the
162
+ meantime. If it never opens, it is retried four times with a widening gap: the
163
+ page may have been loaded during the server's restart window, and a single
164
+ failure does not mean the channel is unavailable. If those attempts fail too (a
165
+ proxy in between may not pass WebSocket through), the overlay falls back to the
166
+ old path: the `/events` SSE stream plus polling `/stats`.
167
+
168
+ ## Devtools overlay
169
+
170
+ A floating bubble in the bottom right; opened with `Alt+D`, closed with `Esc` or
171
+ by clicking the backdrop. It is only emitted by the layout when
172
+ `NODE_ENV=development`:
173
+
174
+ ```ejs
175
+ <% if (devtools) { %>
176
+ <script type="module" src="<%= devBasePath %>/overlay.js"></script>
177
+ <% } %>
178
+ ```
179
+
180
+ The overlay file is **not part of the build.** The server serves it raw from the
181
+ framework package; that is why there is no bundler involved. It loads sibling
182
+ modules such as `seo.js` with native ESM imports, and it adds nothing to the
183
+ production output. The entire UI lives inside a shadow DOM and does not mix with
184
+ the page's CSS.
185
+
186
+ What it shows:
187
+
188
+ - **Errors:** browser-side JS errors, resource loading errors
189
+ (`img`/`script`/`link`), the server's `console.error` / `console.warn`
190
+ output, and failed SSR / browser `fetch` calls (4xx/5xx or network). Each
191
+ record carries a **page path**, **API URL**, and (on the client) an **island
192
+ name** when known; the response body opens under **show details** as JSON
193
+ instead of `[object Object]`. On the server side `console` is wrapped so
194
+ warnings do not get lost in the terminal; upstream failures still land in the
195
+ overlay even when the app's own logger writes them only to stderr.
196
+ - **SEO:** a client-side scan of the current page — title and meta description
197
+ length, `html lang`, viewport, canonical, robots/`noindex`, Open Graph and
198
+ Twitter tags, H1/outline, image `alt`, empty links, and JSON-LD parse errors.
199
+ Findings appear in the panel; turning on **Highlight issues on the page**
200
+ draws red (error) or yellow (warning) boxes on the elements, with a short
201
+ title on the border. Clicking the label (or a row in the panel) opens the
202
+ full explanation. The scan lives in `/seo.js` next to the overlay and is not
203
+ part of the production bundle.
204
+ - **Requests:** the method, path, status, duration and `X-JSkelet-Cache` value
205
+ of every HTML request. The same lines are printed to the terminal too.
206
+ - **Web Vitals:** TTFB, FCP, LCP, CLS, INP, DCL, load, long task count and
207
+ blocking time.
208
+ - **Prewarm:** the progress of the warming round; it can be triggered manually
209
+ from the panel, and individual paths can be retried.
210
+ - **Process:** pid, Node version, uptime, RSS and heap usage.
211
+ - **Version:** the installed JSkelet version compared against the `latest` tag
212
+ on npm. When a newer release exists, the **Server** tab grows an `update` chip
213
+ and a line that copies the upgrade command. The lookup runs once, 1.5 seconds
214
+ after boot, is cached for six hours in `os.tmpdir()` and is skipped silently
215
+ when the registry is unreachable. Set `JSKELET_VERSION_CHECK=0` to disable it.
216
+
217
+ Warming requests (`user-agent: jskelet-prewarm`) are filtered out of both the
218
+ terminal and the request list: hundreds of requests should not flood the view.
219
+ Progress appears in the badge next to the bubble.
220
+
221
+ ### Why state is written to `os.tmpdir()`
222
+
223
+ If request and error records lived in process memory, history would be erased on
224
+ every restart and the overlay would come up empty. So the records are carried
225
+ across restarts in a file.
226
+
227
+ The file is **not written into the project tree**: every write triggered the
228
+ watcher and restarted the server, which set up a self-feeding loop (restart →
229
+ startup warning → write → restart). Instead, the file is written to
230
+ `os.tmpdir()/jskelet-devtools-<hash of the project root>.json`; thanks to the
231
+ hash, multiple JSkelet projects on the same machine do not overwrite each
232
+ other's records.
233
+
234
+ The write happens after 300 ms of silence rather than on every request, and its
235
+ failure does not stop the dev flow. At most 50 requests and 50 errors are kept.
236
+
237
+ The panel's open/closed state, its active tab and the browser error log are kept
238
+ in tab memory (`sessionStorage`), so after a reload the panel comes back with
239
+ the same tab.
240
+
241
+ ## Report page
242
+
243
+ The bubble shows the current state; the report page produces a view of the whole
244
+ site. The address:
245
+
246
+ ```
247
+ http://localhost:3000/__jskelet/dev/report
248
+ ```
249
+
250
+ (Or wherever `brand.devBasePath` points, if you changed it.)
251
+
252
+ Its contents:
253
+
254
+ - **Pages:** the Web Vitals measurements of every visited page, resource count
255
+ and total bytes (broken down by type), island status (how many are ready, and
256
+ their names), API calls made in the browser, and the size/duration/cache
257
+ status of the SSR output. Pages that were never visited but were warmed are
258
+ listed too: the SSR side is known, the client measurements stay empty.
259
+ - **Server API calls:** outbound `fetch` calls made during SSR — URL, host,
260
+ method, status, duration, bytes, which page was rendering, and a body summary
261
+ on failures. `globalThis.fetch` is only wrapped in development; the production
262
+ path is left untouched. Requests to our own server (warming, health check) do
263
+ not count as API calls. Failures also appear on the overlay Errors tab.
264
+ - **Build output:** the raw/gzip/brotli size of every asset in the manifest, and
265
+ chunk analysis from esbuild's metafile — the size of each output, which
266
+ sources it is made of, which chunks it imports. Sources are reduced to
267
+ readable groups (package name or parent folder), so the question "which
268
+ library accounts for 40 kB of this chunk" can be answered.
269
+ - **HTML cache:** entry count and a dump (key, bytes, status, whether it is
270
+ stale, how many seconds until it expires, which encodings are stored).
271
+ - **Prewarm:** the full result of the last round.
272
+ - **Request and error logs.**
273
+
274
+ Measurements live on the server, not in the browser tab; resetting is done from
275
+ the server as well. Size calculations are not repeated unless the file changed.
276
+
277
+ The report layer is only loaded in development and never enters the production
278
+ output.
279
+
280
+ ## Dev endpoints
281
+
282
+ Under `brand.devBasePath` (default `/__jskelet/dev`):
283
+
284
+ | Path | Method | Job |
285
+ | --- | --- | --- |
286
+ | `/overlay.js` | GET | The overlay script |
287
+ | `/seo.js` | GET | SEO scan + page highlight helper (imported by the overlay) |
288
+ | `/logo.png` | GET | The overlay logo |
289
+ | `/ws` | GET (upgrade) | Live channel: statistics, live reload and CSS hot-swap events |
290
+ | `/events` | GET | SSE: the fallback event stream, used only when WebSocket cannot be established |
291
+ | `/stats` | GET | Current statistics; the data endpoint of that same fallback |
292
+ | `/report` | GET | The report page (HTML) |
293
+ | `/report.js` | GET | The report page's script |
294
+ | `/report/data` | GET | The report's single data source (JSON) |
295
+ | `/vitals` | POST | The measurement bundle sent by the overlay |
296
+ | `/report/clear` | POST | Resets page measurements and server API records |
297
+ | `/prewarm` | POST | Triggers warming manually. If the body has `paths`, only those paths; 409 if warming is already running. |
298
+ | `/clear` | POST | Resets the request and error logs |
299
+
300
+ All of these endpoints are mounted by `mountDevtools()` only when
301
+ `NODE_ENV=development`; thanks to the dynamic import, nothing is loaded into the
302
+ production process.
303
+
304
+ ## Dev gate — `DEV_TOKEN`
305
+
306
+ To hide an environment that is not public yet. **The framework does not require
307
+ the token:** a `DEV_TOKEN` sitting in the environment does not lock the site.
308
+ You turn the gate on.
309
+
310
+ ```bash
311
+ DEV_GATE=1 DEV_TOKEN=a-long-random-string npm start
312
+ ```
313
+
314
+ The same thing from config:
315
+
316
+ ```js
317
+ export default {
318
+ devGate: true,
319
+ };
320
+ ```
321
+
322
+ Access:
323
+
324
+ ```
325
+ https://staging.example.com/?dev_token=a-long-random-string
326
+ ```
327
+
328
+ Behavior:
329
+
330
+ - **404, not 403.** A 403 confirms the environment exists; a 404 acts as if it
331
+ never did.
332
+ - Once the token arrives as a query parameter, it is written to a cookie
333
+ (`Path=/`, `SameSite=Lax`, 14 days), so sharing the link is enough. The cookie
334
+ and parameter name is `brand.devTokenCookie` (default `dev_token`).
335
+ - The **exact** paths in the `devGateBypass` list are open under all conditions.
336
+ Default: `/api/healthcheck`, `/robots.txt`, `/sitemap.xml`,
337
+ `/site.webmanifest`, `/favicon.ico`. If your health check lives at a different
338
+ path, remember to add it to this list, otherwise your orchestrator will see a
339
+ 404.
340
+ - While `devGate` is off (the default) or `DEV_TOKEN` is empty, the middleware
341
+ passes the request through. A `DEV_TOKEN` that leaked into a production task
342
+ does not ask visitors for a token; startup prints a warning.
343
+ - `DEV_GATE=0` turns the gate off even when config says `devGate: true`.
344
+ - While the gate is on, warming carries the token as a cookie; without it every
345
+ page gets a 404 and the cache never fills up
346
+ ([06-caching.md](./06-caching.md)).
347
+
348
+ In the middleware chain the gate sits after `headers` and **before**
349
+ `redirects`: an environment that is not public yet should not leak even its
350
+ redirect rules.
351
+
352
+ ## Differences between development and production
353
+
354
+ | Topic | Development | Production |
355
+ | --- | --- | --- |
356
+ | EJS template cache | Off | On |
357
+ | Manifest reading | On every request | Once |
358
+ | Image manifest | On every call | Once |
359
+ | Broken route module | Warn + skip | Throw |
360
+ | Devtools and report | Mounted | Never loaded |
361
+ | `globalThis.fetch` | Wrapped (measurement) | Untouched |
362
+ | Prewarm concurrency | 1 | 4 |
363
+ | Prewarm rate limit | 4 requests/second | Unlimited |
364
+ | Prewarm delay | 3000 ms | 500 ms |
365
+ | Missing icon warning | Emitted | Not emitted |
366
+ | Precompress | Does not run in watch | Runs |
367
+ | Image optimization | Does not run in watch | Runs |
368
+
369
+ ## What's next
370
+
371
+ - Details of the build steps: [08-build.md](./08-build.md)
372
+ - Going to production: [10-deployment.md](./10-deployment.md)
373
+ - Reading and clearing the cache: [06-caching.md](./06-caching.md)