jskelet 0.6.3 → 0.6.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (153) hide show
  1. package/AGENTS.md +136 -136
  2. package/CHANGELOG.md +628 -620
  3. package/LICENSE +21 -21
  4. package/README.md +2 -0
  5. package/bin/jskelet.mjs +130 -130
  6. package/docs/01-baslangic.md +291 -291
  7. package/docs/02-mimari.md +310 -310
  8. package/docs/03-routing.md +515 -515
  9. package/docs/04-render-ve-sablonlar.md +667 -661
  10. package/docs/05-islands.md +486 -486
  11. package/docs/06-cache.md +1467 -1443
  12. package/docs/07-yapilandirma.md +1208 -1197
  13. package/docs/08-build.md +429 -429
  14. package/docs/09-dev-araclari.md +364 -364
  15. package/docs/10-dagitim.md +348 -338
  16. package/docs/12-panel-ve-oturum.md +479 -478
  17. package/docs/README.md +83 -83
  18. package/docs/en/01-getting-started.md +298 -298
  19. package/docs/en/02-architecture.md +329 -329
  20. package/docs/en/03-routing.md +531 -531
  21. package/docs/en/04-rendering.md +675 -669
  22. package/docs/en/05-islands.md +497 -497
  23. package/docs/en/06-caching.md +1476 -1453
  24. package/docs/en/07-configuration.md +1229 -1219
  25. package/docs/en/08-build.md +447 -447
  26. package/docs/en/09-dev-tools.md +373 -373
  27. package/docs/en/10-deployment.md +351 -340
  28. package/docs/en/11-migration.md +398 -398
  29. package/docs/en/12-dashboards-and-sessions.md +489 -488
  30. package/docs/en/README.md +87 -87
  31. package/package.json +137 -137
  32. package/src/build/ensure-build.mjs +19 -19
  33. package/src/build/paths.mjs +153 -153
  34. package/src/build/resolve-peer.mjs +36 -36
  35. package/src/build/tasks/client.mjs +349 -349
  36. package/src/build/tasks/css.mjs +235 -235
  37. package/src/build/tasks/fonts.mjs +146 -146
  38. package/src/build/tasks/icons.mjs +357 -357
  39. package/src/build/tasks/images.mjs +244 -244
  40. package/src/build/tasks/precompress.mjs +78 -78
  41. package/src/build/tasks/templates.mjs +20 -20
  42. package/src/client/admin/i18n.js +764 -764
  43. package/src/client/admin/login.html +74 -74
  44. package/src/client/admin/panel.css +809 -809
  45. package/src/client/admin/panel.html +495 -495
  46. package/src/client/admin/panel.js +1251 -1251
  47. package/src/client/devtools/report.html +185 -185
  48. package/src/client/devtools/report.js +745 -745
  49. package/src/client/devtools/seo.js +628 -628
  50. package/src/client/dom.js +95 -95
  51. package/src/client/form.js +192 -192
  52. package/src/client/index.js +45 -45
  53. package/src/client/registry.js +305 -305
  54. package/src/client/safe-image.js +91 -91
  55. package/src/client/shared-cookie.js +225 -225
  56. package/src/client/store.js +36 -36
  57. package/src/client/swap.js +188 -188
  58. package/src/compile/codegen.js +336 -336
  59. package/src/compile/compile-all.js +149 -149
  60. package/src/compile/errors.js +66 -66
  61. package/src/compile/expr.js +409 -409
  62. package/src/compile/index.js +17 -17
  63. package/src/compile/parse.js +541 -541
  64. package/src/compile/resolve.js +211 -211
  65. package/src/compile/scan-exports.js +51 -51
  66. package/src/config/defaults.js +541 -534
  67. package/src/config/index.js +1500 -1469
  68. package/src/config/pattern.js +107 -107
  69. package/src/generate.mjs +163 -163
  70. package/src/http/control-flow.js +71 -71
  71. package/src/http/cookies-entry.js +21 -21
  72. package/src/http/cookies.js +277 -277
  73. package/src/http/request-cache.js +46 -46
  74. package/src/http/request-context.js +165 -165
  75. package/src/http/shared-cookie.js +178 -178
  76. package/src/index.js +101 -101
  77. package/src/init.mjs +232 -230
  78. package/src/migrate/apply.mjs +262 -262
  79. package/src/migrate/babel.mjs +79 -79
  80. package/src/migrate/classify.mjs +155 -155
  81. package/src/migrate/config.mjs +126 -126
  82. package/src/migrate/fs-walk.mjs +191 -191
  83. package/src/migrate/parse.mjs +26 -26
  84. package/src/migrate/scan.mjs +177 -177
  85. package/src/migrate/transform/expr-source.mjs +168 -168
  86. package/src/migrate/transform/island.mjs +67 -67
  87. package/src/migrate/transform/jsx-to-component.mjs +302 -302
  88. package/src/migrate/transform/jsx-to-jsk.mjs +330 -330
  89. package/src/migrate/transform/page-split.mjs +435 -435
  90. package/src/migrate/write.mjs +81 -81
  91. package/src/migrate.mjs +171 -171
  92. package/src/runtime/alias-hooks.mjs +119 -119
  93. package/src/runtime/register.mjs +4 -4
  94. package/src/server/admin/actions.js +229 -229
  95. package/src/server/admin/auth.js +125 -125
  96. package/src/server/admin/event-log.js +151 -151
  97. package/src/server/admin/gate.js +209 -209
  98. package/src/server/admin/inventory.js +188 -188
  99. package/src/server/admin/mount.js +56 -56
  100. package/src/server/admin/router.js +216 -216
  101. package/src/server/admin/snapshot.js +241 -241
  102. package/src/server/assets.js +147 -147
  103. package/src/server/auth/handoff.js +309 -309
  104. package/src/server/cache-blob.js +70 -70
  105. package/src/server/cache-control.js +45 -0
  106. package/src/server/cache-deps.js +42 -42
  107. package/src/server/cache-vary.js +113 -113
  108. package/src/server/cloudflare.js +607 -607
  109. package/src/server/create-app.js +366 -366
  110. package/src/server/data-cache.js +553 -553
  111. package/src/server/dev/report.js +485 -485
  112. package/src/server/dev/socket.js +170 -170
  113. package/src/server/dev/version-check.mjs +139 -139
  114. package/src/server/disk-cache.js +233 -233
  115. package/src/server/ejs-adapter.js +59 -59
  116. package/src/server/html-cache.js +1196 -1196
  117. package/src/server/image-optimizer.js +500 -500
  118. package/src/server/logs/access-middleware.js +66 -66
  119. package/src/server/logs/file-sink.js +193 -193
  120. package/src/server/logs/pipeline.js +165 -165
  121. package/src/server/logs/s3-put.js +214 -214
  122. package/src/server/logs/s3-sink.js +112 -112
  123. package/src/server/metadata.js +102 -102
  124. package/src/server/middleware/compression.js +205 -205
  125. package/src/server/middleware/csrf.js +134 -134
  126. package/src/server/middleware/dev-gate.js +75 -75
  127. package/src/server/middleware/headers.js +37 -37
  128. package/src/server/middleware/redirects.js +32 -32
  129. package/src/server/middleware/robots-txt.js +341 -341
  130. package/src/server/middleware/static-precompressed.js +121 -121
  131. package/src/server/middleware/trailing-slash.js +53 -53
  132. package/src/server/middleware/upstream-proxy.js +141 -141
  133. package/src/server/og-image.js +369 -356
  134. package/src/server/port-guard.js +255 -255
  135. package/src/server/prewarm.js +1082 -1082
  136. package/src/server/redis.js +588 -588
  137. package/src/server/render.js +910 -910
  138. package/src/server/router.js +157 -157
  139. package/src/server/status-page.js +265 -265
  140. package/src/server/upstream-limiter.js +376 -376
  141. package/src/server/upstream-tracking.js +166 -166
  142. package/src/shared/cookie-domain.js +66 -66
  143. package/src/start.mjs +22 -22
  144. package/src/templates/layout.ejs +30 -30
  145. package/src/templates/layout.jsk +30 -30
  146. package/src/version.mjs +31 -31
  147. package/src/views/components/loader.js +101 -101
  148. package/src/views/helpers/html.js +102 -102
  149. package/src/views/helpers/tags.js +375 -375
  150. package/types/config/defaults.d.ts +6 -0
  151. package/types/config/index.d.ts +6 -0
  152. package/types/server/cache-control.d.ts +28 -0
  153. package/types/server/og-image.d.ts +5 -0
@@ -1,373 +1,373 @@
1
- # 09 — Development tools
2
-
3
- This document explains what `jskelet dev` does and why it does it that way: how
4
- the two child processes are managed, the shape of the terminal output, the
5
- watched directories and the reason a custom watcher was written instead of using
6
- `node --watch`, the distinction between CSS hot-swap and a full reload, the
7
- devtools overlay opened with `Alt+D`, the detailed report page, and the dev gate
8
- built on `DEV_TOKEN`. The build steps themselves are in
9
- [08-build.md](./08-build.md).
10
-
11
- ## The `jskelet dev` flow
12
-
13
- The command manages two long-lived child processes in a single terminal:
14
-
15
- ```
16
- jskelet dev
17
- ├─ build watch node src/build/build.mjs --watch
18
- └─ server node [--env-file=.env] --import <register.mjs> src/start.mjs
19
- ```
20
-
21
- `NODE_ENV=development` is assigned here in a platform-independent way — no
22
- `cross-env` needed. The child processes also receive `JSKELET_CHILD=1`
23
- (suppresses the build banner) and, if a TTY is present, `JSKELET_COLOR=1`
24
- (forces color on piped output).
25
-
26
- If the listen port (`PORT`, default `3000`) is already taken, the server
27
- **does not start**; the error line includes the PID and a `--murder` hint.
28
- `jskelet dev --murder` kills the listener and binds the same port (for a
29
- process left running in another terminal).
30
-
31
- Startup order: banner → build steps → server ready → `Ready` summary. The
32
- summary is printed once both the build and the server are ready; otherwise it
33
- got buried among the build lines arriving afterwards.
34
-
35
- The server signals readiness with a single line inside `startServer`:
36
-
37
- ```
38
- jskelet → http://localhost:3000 (development)
39
- ```
40
-
41
- The shape of this line is a contract; the dev script parses it and prints the
42
- summary line accordingly.
43
-
44
- ### The shape of the terminal output
45
-
46
- Child process output does not stream through as-is. There are two regions and
47
- they never mix:
48
-
49
- 1. **Startup:** banner, aligned build lines, `Ready` summary.
50
- 2. **Runtime:** timestamped, single-line events (HTTP requests, CSS rebuild,
51
- server restart).
52
-
53
- Error stacks are turned into a framed box: because stack lines arrive in
54
- fragments, they are collected after a short silence (60 ms), the error name and
55
- message are parsed, the first three frames are shown, and the project root is
56
- shortened to `.`. When you are developing your own framework, not losing the
57
- error in the stream is the detail that genuinely makes a difference.
58
-
59
- Color carries meaning: `✓` green, `✖` red, `⚠` yellow, `↻` cyan; durations and
60
- paths gray. No decorative color is used. If `NO_COLOR` is set, no color is used
61
- at all.
62
-
63
- `Ctrl+C` (SIGINT/SIGTERM) shuts down all child processes. If a child exits with
64
- a non-zero code (other than an expected restart), an error is printed and the
65
- dev process exits too.
66
-
67
- ## Watch directories
68
-
69
- Server restarts are managed by the framework's own watcher.
70
-
71
- ```js
72
- WATCH_DIRS = [
73
- config.dirs.routes,
74
- config.dirs.views,
75
- <root>/lib,
76
- ...config.watch, // jskelet.config.mjs → watch
77
- ]
78
- ```
79
-
80
- The `jskelet.config.mjs` file itself is watched as well: when the config
81
- changes, both the server and the build must come up with the new settings.
82
-
83
- Watched extensions: `.js`, `.mjs`, `.json`, `.jsk`, `.ejs`.
84
-
85
- `views` is watched too, because most components live in
86
- `views/components/**.js` and, since those modules are imported into the server
87
- once, changes made without a restart never reached the browser (that "I edited
88
- the template and nothing changed" feeling comes from here).
89
-
90
- `client/` and `styles/` are **not** in this list: esbuild and the CSS watchers
91
- handle them on their own ([08-build.md](./08-build.md)).
92
-
93
- If a directory cannot be watched, a warning is printed and no automatic restart
94
- happens for that directory; everything else keeps working.
95
-
96
- ### Why `node --watch` was not used
97
-
98
- Even when given `--watch-path`, Node watched the project root in this setup.
99
- Whenever build output (`public/assets`, `manifest.json`) or the dev tooling log
100
- was written, the server restarted for nothing — in fact a self-feeding loop was
101
- set up: restart → startup warning → write → restart.
102
-
103
- The custom watcher does three things:
104
-
105
- 1. **Watches only server sources.**
106
- 2. **Coalesces changes** (250 ms) and reports which files changed.
107
- 3. **Filters out phantom events.** On Windows, `fs.watch` can emit events for a
108
- file's neighbors when it is written; without comparing `mtime`, a single save
109
- turned into two restarts. Current timestamps are read up front at startup, so
110
- the first phantom event is filtered out as well.
111
-
112
- The restart line shows the changed file, or how many changed:
113
-
114
- ```
115
- 21:04:12 ↻ server restarting… routes/10-pages.mjs
116
- 21:04:12 server restarted 412ms
117
- ```
118
-
119
- If `JSKELET_VERBOSE=1` is set, all files are listed when more than one changed.
120
-
121
- ## CSS hot-swap and full reload
122
-
123
- The dev server watches `.jskelet/manifest.json` and broadcasts events to the
124
- browser over the live channel (`<devBasePath>/ws`). Since the manifest is
125
- rewritten on every build round, change detection is done through the manifest.
126
-
127
- | What changed | Behavior |
128
- | --- | --- |
129
- | Only `.css` keys | **CSS hot-swap:** each changed stylesheet is swapped, the page is not reloaded. State and scroll position are preserved. |
130
- | `main.js`, the sprite, another asset, or more than one key | **Full reload** |
131
-
132
- In both cases the HTML cache is cleared first: the stored HTML carries the old
133
- hashed asset URLs and, if not cleared, the page keeps asking for a deleted file.
134
-
135
- Manifest events are coalesced over 120 ms. If watching is not supported, live
136
- reload is disabled and everything else keeps working.
137
-
138
- When the server restarts, the overlay figures it out from the **boot id**: every
139
- process broadcasts a unique `boot` value, the overlay sees the change, shows the
140
- "restarted" note and does not reset its own state.
141
-
142
- ## The live channel
143
-
144
- Everything the overlay shows — statistics, live reload and CSS hot-swap events —
145
- arrives over a single WebSocket (`<devBasePath>/ws`). The panel used to poll for
146
- statistics every two seconds, so every open tab kept hitting the server even
147
- while the panel was closed. Now the server pushes as things change: when a
148
- request or an error is recorded (coalesced over 120 ms), and every two seconds so
149
- the time-based fields (uptime, memory, the prewarm counter) stay fresh. The
150
- heartbeat is deliberately independent of prewarming: if the channel's tempo
151
- followed a background job, the panel would be tied to that job's rhythm. Nothing
152
- is computed when no panel is connected.
153
-
154
- The handshake happens on the HTTP `upgrade` event, and that event never reaches
155
- the middleware chain, so the channel is attached straight to the server after
156
- `listen` (`attachDevSocket`). The server side pulls in no dependency such as
157
- `ws`: all it needs is to write server-to-client text frames and to answer the
158
- client's ping/close frames.
159
-
160
- If the socket opens and later drops — that is, the server is restarting — it
161
- reconnects every half second and the indicator reads "server restarting…" in the
162
- meantime. If it never opens, it is retried four times with a widening gap: the
163
- page may have been loaded during the server's restart window, and a single
164
- failure does not mean the channel is unavailable. If those attempts fail too (a
165
- proxy in between may not pass WebSocket through), the overlay falls back to the
166
- old path: the `/events` SSE stream plus polling `/stats`.
167
-
168
- ## Devtools overlay
169
-
170
- A floating bubble in the bottom right; opened with `Alt+D`, closed with `Esc` or
171
- by clicking the backdrop. It is only emitted by the layout when
172
- `NODE_ENV=development`:
173
-
174
- ```ejs
175
- <% if (devtools) { %>
176
- <script type="module" src="<%= devBasePath %>/overlay.js"></script>
177
- <% } %>
178
- ```
179
-
180
- The overlay file is **not part of the build.** The server serves it raw from the
181
- framework package; that is why there is no bundler involved. It loads sibling
182
- modules such as `seo.js` with native ESM imports, and it adds nothing to the
183
- production output. The entire UI lives inside a shadow DOM and does not mix with
184
- the page's CSS.
185
-
186
- What it shows:
187
-
188
- - **Errors:** browser-side JS errors, resource loading errors
189
- (`img`/`script`/`link`), the server's `console.error` / `console.warn`
190
- output, and failed SSR / browser `fetch` calls (4xx/5xx or network). Each
191
- record carries a **page path**, **API URL**, and (on the client) an **island
192
- name** when known; the response body opens under **show details** as JSON
193
- instead of `[object Object]`. On the server side `console` is wrapped so
194
- warnings do not get lost in the terminal; upstream failures still land in the
195
- overlay even when the app's own logger writes them only to stderr.
196
- - **SEO:** a client-side scan of the current page — title and meta description
197
- length, `html lang`, viewport, canonical, robots/`noindex`, Open Graph and
198
- Twitter tags, H1/outline, image `alt`, empty links, and JSON-LD parse errors.
199
- Findings appear in the panel; turning on **Highlight issues on the page**
200
- draws red (error) or yellow (warning) boxes on the elements, with a short
201
- title on the border. Clicking the label (or a row in the panel) opens the
202
- full explanation. The scan lives in `/seo.js` next to the overlay and is not
203
- part of the production bundle.
204
- - **Requests:** the method, path, status, duration and `X-JSkelet-Cache` value
205
- of every HTML request. The same lines are printed to the terminal too.
206
- - **Web Vitals:** TTFB, FCP, LCP, CLS, INP, DCL, load, long task count and
207
- blocking time.
208
- - **Prewarm:** the progress of the warming round; it can be triggered manually
209
- from the panel, and individual paths can be retried.
210
- - **Process:** pid, Node version, uptime, RSS and heap usage.
211
- - **Version:** the installed JSkelet version compared against the `latest` tag
212
- on npm. When a newer release exists, the **Server** tab grows an `update` chip
213
- and a line that copies the upgrade command. The lookup runs once, 1.5 seconds
214
- after boot, is cached for six hours in `os.tmpdir()` and is skipped silently
215
- when the registry is unreachable. Set `JSKELET_VERSION_CHECK=0` to disable it.
216
-
217
- Warming requests (`user-agent: jskelet-prewarm`) are filtered out of both the
218
- terminal and the request list: hundreds of requests should not flood the view.
219
- Progress appears in the badge next to the bubble.
220
-
221
- ### Why state is written to `os.tmpdir()`
222
-
223
- If request and error records lived in process memory, history would be erased on
224
- every restart and the overlay would come up empty. So the records are carried
225
- across restarts in a file.
226
-
227
- The file is **not written into the project tree**: every write triggered the
228
- watcher and restarted the server, which set up a self-feeding loop (restart →
229
- startup warning → write → restart). Instead, the file is written to
230
- `os.tmpdir()/jskelet-devtools-<hash of the project root>.json`; thanks to the
231
- hash, multiple JSkelet projects on the same machine do not overwrite each
232
- other's records.
233
-
234
- The write happens after 300 ms of silence rather than on every request, and its
235
- failure does not stop the dev flow. At most 50 requests and 50 errors are kept.
236
-
237
- The panel's open/closed state, its active tab and the browser error log are kept
238
- in tab memory (`sessionStorage`), so after a reload the panel comes back with
239
- the same tab.
240
-
241
- ## Report page
242
-
243
- The bubble shows the current state; the report page produces a view of the whole
244
- site. The address:
245
-
246
- ```
247
- http://localhost:3000/__jskelet/dev/report
248
- ```
249
-
250
- (Or wherever `brand.devBasePath` points, if you changed it.)
251
-
252
- Its contents:
253
-
254
- - **Pages:** the Web Vitals measurements of every visited page, resource count
255
- and total bytes (broken down by type), island status (how many are ready, and
256
- their names), API calls made in the browser, and the size/duration/cache
257
- status of the SSR output. Pages that were never visited but were warmed are
258
- listed too: the SSR side is known, the client measurements stay empty.
259
- - **Server API calls:** outbound `fetch` calls made during SSR — URL, host,
260
- method, status, duration, bytes, which page was rendering, and a body summary
261
- on failures. `globalThis.fetch` is only wrapped in development; the production
262
- path is left untouched. Requests to our own server (warming, health check) do
263
- not count as API calls. Failures also appear on the overlay Errors tab.
264
- - **Build output:** the raw/gzip/brotli size of every asset in the manifest, and
265
- chunk analysis from esbuild's metafile — the size of each output, which
266
- sources it is made of, which chunks it imports. Sources are reduced to
267
- readable groups (package name or parent folder), so the question "which
268
- library accounts for 40 kB of this chunk" can be answered.
269
- - **HTML cache:** entry count and a dump (key, bytes, status, whether it is
270
- stale, how many seconds until it expires, which encodings are stored).
271
- - **Prewarm:** the full result of the last round.
272
- - **Request and error logs.**
273
-
274
- Measurements live on the server, not in the browser tab; resetting is done from
275
- the server as well. Size calculations are not repeated unless the file changed.
276
-
277
- The report layer is only loaded in development and never enters the production
278
- output.
279
-
280
- ## Dev endpoints
281
-
282
- Under `brand.devBasePath` (default `/__jskelet/dev`):
283
-
284
- | Path | Method | Job |
285
- | --- | --- | --- |
286
- | `/overlay.js` | GET | The overlay script |
287
- | `/seo.js` | GET | SEO scan + page highlight helper (imported by the overlay) |
288
- | `/logo.png` | GET | The overlay logo |
289
- | `/ws` | GET (upgrade) | Live channel: statistics, live reload and CSS hot-swap events |
290
- | `/events` | GET | SSE: the fallback event stream, used only when WebSocket cannot be established |
291
- | `/stats` | GET | Current statistics; the data endpoint of that same fallback |
292
- | `/report` | GET | The report page (HTML) |
293
- | `/report.js` | GET | The report page's script |
294
- | `/report/data` | GET | The report's single data source (JSON) |
295
- | `/vitals` | POST | The measurement bundle sent by the overlay |
296
- | `/report/clear` | POST | Resets page measurements and server API records |
297
- | `/prewarm` | POST | Triggers warming manually. If the body has `paths`, only those paths; 409 if warming is already running. |
298
- | `/clear` | POST | Resets the request and error logs |
299
-
300
- All of these endpoints are mounted by `mountDevtools()` only when
301
- `NODE_ENV=development`; thanks to the dynamic import, nothing is loaded into the
302
- production process.
303
-
304
- ## Dev gate — `DEV_TOKEN`
305
-
306
- To hide an environment that is not public yet. **The framework does not require
307
- the token:** a `DEV_TOKEN` sitting in the environment does not lock the site.
308
- You turn the gate on.
309
-
310
- ```bash
311
- DEV_GATE=1 DEV_TOKEN=a-long-random-string npm start
312
- ```
313
-
314
- The same thing from config:
315
-
316
- ```js
317
- export default {
318
- devGate: true,
319
- };
320
- ```
321
-
322
- Access:
323
-
324
- ```
325
- https://staging.example.com/?dev_token=a-long-random-string
326
- ```
327
-
328
- Behavior:
329
-
330
- - **404, not 403.** A 403 confirms the environment exists; a 404 acts as if it
331
- never did.
332
- - Once the token arrives as a query parameter, it is written to a cookie
333
- (`Path=/`, `SameSite=Lax`, 14 days), so sharing the link is enough. The cookie
334
- and parameter name is `brand.devTokenCookie` (default `dev_token`).
335
- - The **exact** paths in the `devGateBypass` list are open under all conditions.
336
- Default: `/api/healthcheck`, `/robots.txt`, `/sitemap.xml`,
337
- `/site.webmanifest`, `/favicon.ico`. If your health check lives at a different
338
- path, remember to add it to this list, otherwise your orchestrator will see a
339
- 404.
340
- - While `devGate` is off (the default) or `DEV_TOKEN` is empty, the middleware
341
- passes the request through. A `DEV_TOKEN` that leaked into a production task
342
- does not ask visitors for a token; startup prints a warning.
343
- - `DEV_GATE=0` turns the gate off even when config says `devGate: true`.
344
- - While the gate is on, warming carries the token as a cookie; without it every
345
- page gets a 404 and the cache never fills up
346
- ([06-caching.md](./06-caching.md)).
347
-
348
- In the middleware chain the gate sits after `headers` and **before**
349
- `redirects`: an environment that is not public yet should not leak even its
350
- redirect rules.
351
-
352
- ## Differences between development and production
353
-
354
- | Topic | Development | Production |
355
- | --- | --- | --- |
356
- | EJS template cache | Off | On |
357
- | Manifest reading | On every request | Once |
358
- | Image manifest | On every call | Once |
359
- | Broken route module | Warn + skip | Throw |
360
- | Devtools and report | Mounted | Never loaded |
361
- | `globalThis.fetch` | Wrapped (measurement) | Untouched |
362
- | Prewarm concurrency | 1 | 4 |
363
- | Prewarm rate limit | 4 requests/second | Unlimited |
364
- | Prewarm delay | 3000 ms | 500 ms |
365
- | Missing icon warning | Emitted | Not emitted |
366
- | Precompress | Does not run in watch | Runs |
367
- | Image optimization | Does not run in watch | Runs |
368
-
369
- ## What's next
370
-
371
- - Details of the build steps: [08-build.md](./08-build.md)
372
- - Going to production: [10-deployment.md](./10-deployment.md)
373
- - Reading and clearing the cache: [06-caching.md](./06-caching.md)
1
+ # 09 — Development tools
2
+
3
+ This document explains what `jskelet dev` does and why it does it that way: how
4
+ the two child processes are managed, the shape of the terminal output, the
5
+ watched directories and the reason a custom watcher was written instead of using
6
+ `node --watch`, the distinction between CSS hot-swap and a full reload, the
7
+ devtools overlay opened with `Alt+D`, the detailed report page, and the dev gate
8
+ built on `DEV_TOKEN`. The build steps themselves are in
9
+ [08-build.md](./08-build.md).
10
+
11
+ ## The `jskelet dev` flow
12
+
13
+ The command manages two long-lived child processes in a single terminal:
14
+
15
+ ```
16
+ jskelet dev
17
+ ├─ build watch node src/build/build.mjs --watch
18
+ └─ server node [--env-file=.env] --import <register.mjs> src/start.mjs
19
+ ```
20
+
21
+ `NODE_ENV=development` is assigned here in a platform-independent way — no
22
+ `cross-env` needed. The child processes also receive `JSKELET_CHILD=1`
23
+ (suppresses the build banner) and, if a TTY is present, `JSKELET_COLOR=1`
24
+ (forces color on piped output).
25
+
26
+ If the listen port (`PORT`, default `3000`) is already taken, the server
27
+ **does not start**; the error line includes the PID and a `--murder` hint.
28
+ `jskelet dev --murder` kills the listener and binds the same port (for a
29
+ process left running in another terminal).
30
+
31
+ Startup order: banner → build steps → server ready → `Ready` summary. The
32
+ summary is printed once both the build and the server are ready; otherwise it
33
+ got buried among the build lines arriving afterwards.
34
+
35
+ The server signals readiness with a single line inside `startServer`:
36
+
37
+ ```
38
+ jskelet → http://localhost:3000 (development)
39
+ ```
40
+
41
+ The shape of this line is a contract; the dev script parses it and prints the
42
+ summary line accordingly.
43
+
44
+ ### The shape of the terminal output
45
+
46
+ Child process output does not stream through as-is. There are two regions and
47
+ they never mix:
48
+
49
+ 1. **Startup:** banner, aligned build lines, `Ready` summary.
50
+ 2. **Runtime:** timestamped, single-line events (HTTP requests, CSS rebuild,
51
+ server restart).
52
+
53
+ Error stacks are turned into a framed box: because stack lines arrive in
54
+ fragments, they are collected after a short silence (60 ms), the error name and
55
+ message are parsed, the first three frames are shown, and the project root is
56
+ shortened to `.`. When you are developing your own framework, not losing the
57
+ error in the stream is the detail that genuinely makes a difference.
58
+
59
+ Color carries meaning: `✓` green, `✖` red, `⚠` yellow, `↻` cyan; durations and
60
+ paths gray. No decorative color is used. If `NO_COLOR` is set, no color is used
61
+ at all.
62
+
63
+ `Ctrl+C` (SIGINT/SIGTERM) shuts down all child processes. If a child exits with
64
+ a non-zero code (other than an expected restart), an error is printed and the
65
+ dev process exits too.
66
+
67
+ ## Watch directories
68
+
69
+ Server restarts are managed by the framework's own watcher.
70
+
71
+ ```js
72
+ WATCH_DIRS = [
73
+ config.dirs.routes,
74
+ config.dirs.views,
75
+ <root>/lib,
76
+ ...config.watch, // jskelet.config.mjs → watch
77
+ ]
78
+ ```
79
+
80
+ The `jskelet.config.mjs` file itself is watched as well: when the config
81
+ changes, both the server and the build must come up with the new settings.
82
+
83
+ Watched extensions: `.js`, `.mjs`, `.json`, `.jsk`, `.ejs`.
84
+
85
+ `views` is watched too, because most components live in
86
+ `views/components/**.js` and, since those modules are imported into the server
87
+ once, changes made without a restart never reached the browser (that "I edited
88
+ the template and nothing changed" feeling comes from here).
89
+
90
+ `client/` and `styles/` are **not** in this list: esbuild and the CSS watchers
91
+ handle them on their own ([08-build.md](./08-build.md)).
92
+
93
+ If a directory cannot be watched, a warning is printed and no automatic restart
94
+ happens for that directory; everything else keeps working.
95
+
96
+ ### Why `node --watch` was not used
97
+
98
+ Even when given `--watch-path`, Node watched the project root in this setup.
99
+ Whenever build output (`public/assets`, `manifest.json`) or the dev tooling log
100
+ was written, the server restarted for nothing — in fact a self-feeding loop was
101
+ set up: restart → startup warning → write → restart.
102
+
103
+ The custom watcher does three things:
104
+
105
+ 1. **Watches only server sources.**
106
+ 2. **Coalesces changes** (250 ms) and reports which files changed.
107
+ 3. **Filters out phantom events.** On Windows, `fs.watch` can emit events for a
108
+ file's neighbors when it is written; without comparing `mtime`, a single save
109
+ turned into two restarts. Current timestamps are read up front at startup, so
110
+ the first phantom event is filtered out as well.
111
+
112
+ The restart line shows the changed file, or how many changed:
113
+
114
+ ```
115
+ 21:04:12 ↻ server restarting… routes/10-pages.mjs
116
+ 21:04:12 server restarted 412ms
117
+ ```
118
+
119
+ If `JSKELET_VERBOSE=1` is set, all files are listed when more than one changed.
120
+
121
+ ## CSS hot-swap and full reload
122
+
123
+ The dev server watches `.jskelet/manifest.json` and broadcasts events to the
124
+ browser over the live channel (`<devBasePath>/ws`). Since the manifest is
125
+ rewritten on every build round, change detection is done through the manifest.
126
+
127
+ | What changed | Behavior |
128
+ | --- | --- |
129
+ | Only `.css` keys | **CSS hot-swap:** each changed stylesheet is swapped, the page is not reloaded. State and scroll position are preserved. |
130
+ | `main.js`, the sprite, another asset, or more than one key | **Full reload** |
131
+
132
+ In both cases the HTML cache is cleared first: the stored HTML carries the old
133
+ hashed asset URLs and, if not cleared, the page keeps asking for a deleted file.
134
+
135
+ Manifest events are coalesced over 120 ms. If watching is not supported, live
136
+ reload is disabled and everything else keeps working.
137
+
138
+ When the server restarts, the overlay figures it out from the **boot id**: every
139
+ process broadcasts a unique `boot` value, the overlay sees the change, shows the
140
+ "restarted" note and does not reset its own state.
141
+
142
+ ## The live channel
143
+
144
+ Everything the overlay shows — statistics, live reload and CSS hot-swap events —
145
+ arrives over a single WebSocket (`<devBasePath>/ws`). The panel used to poll for
146
+ statistics every two seconds, so every open tab kept hitting the server even
147
+ while the panel was closed. Now the server pushes as things change: when a
148
+ request or an error is recorded (coalesced over 120 ms), and every two seconds so
149
+ the time-based fields (uptime, memory, the prewarm counter) stay fresh. The
150
+ heartbeat is deliberately independent of prewarming: if the channel's tempo
151
+ followed a background job, the panel would be tied to that job's rhythm. Nothing
152
+ is computed when no panel is connected.
153
+
154
+ The handshake happens on the HTTP `upgrade` event, and that event never reaches
155
+ the middleware chain, so the channel is attached straight to the server after
156
+ `listen` (`attachDevSocket`). The server side pulls in no dependency such as
157
+ `ws`: all it needs is to write server-to-client text frames and to answer the
158
+ client's ping/close frames.
159
+
160
+ If the socket opens and later drops — that is, the server is restarting — it
161
+ reconnects every half second and the indicator reads "server restarting…" in the
162
+ meantime. If it never opens, it is retried four times with a widening gap: the
163
+ page may have been loaded during the server's restart window, and a single
164
+ failure does not mean the channel is unavailable. If those attempts fail too (a
165
+ proxy in between may not pass WebSocket through), the overlay falls back to the
166
+ old path: the `/events` SSE stream plus polling `/stats`.
167
+
168
+ ## Devtools overlay
169
+
170
+ A floating bubble in the bottom right; opened with `Alt+D`, closed with `Esc` or
171
+ by clicking the backdrop. It is only emitted by the layout when
172
+ `NODE_ENV=development`:
173
+
174
+ ```ejs
175
+ <% if (devtools) { %>
176
+ <script type="module" src="<%= devBasePath %>/overlay.js"></script>
177
+ <% } %>
178
+ ```
179
+
180
+ The overlay file is **not part of the build.** The server serves it raw from the
181
+ framework package; that is why there is no bundler involved. It loads sibling
182
+ modules such as `seo.js` with native ESM imports, and it adds nothing to the
183
+ production output. The entire UI lives inside a shadow DOM and does not mix with
184
+ the page's CSS.
185
+
186
+ What it shows:
187
+
188
+ - **Errors:** browser-side JS errors, resource loading errors
189
+ (`img`/`script`/`link`), the server's `console.error` / `console.warn`
190
+ output, and failed SSR / browser `fetch` calls (4xx/5xx or network). Each
191
+ record carries a **page path**, **API URL**, and (on the client) an **island
192
+ name** when known; the response body opens under **show details** as JSON
193
+ instead of `[object Object]`. On the server side `console` is wrapped so
194
+ warnings do not get lost in the terminal; upstream failures still land in the
195
+ overlay even when the app's own logger writes them only to stderr.
196
+ - **SEO:** a client-side scan of the current page — title and meta description
197
+ length, `html lang`, viewport, canonical, robots/`noindex`, Open Graph and
198
+ Twitter tags, H1/outline, image `alt`, empty links, and JSON-LD parse errors.
199
+ Findings appear in the panel; turning on **Highlight issues on the page**
200
+ draws red (error) or yellow (warning) boxes on the elements, with a short
201
+ title on the border. Clicking the label (or a row in the panel) opens the
202
+ full explanation. The scan lives in `/seo.js` next to the overlay and is not
203
+ part of the production bundle.
204
+ - **Requests:** the method, path, status, duration and `X-JSkelet-Cache` value
205
+ of every HTML request. The same lines are printed to the terminal too.
206
+ - **Web Vitals:** TTFB, FCP, LCP, CLS, INP, DCL, load, long task count and
207
+ blocking time.
208
+ - **Prewarm:** the progress of the warming round; it can be triggered manually
209
+ from the panel, and individual paths can be retried.
210
+ - **Process:** pid, Node version, uptime, RSS and heap usage.
211
+ - **Version:** the installed JSkelet version compared against the `latest` tag
212
+ on npm. When a newer release exists, the **Server** tab grows an `update` chip
213
+ and a line that copies the upgrade command. The lookup runs once, 1.5 seconds
214
+ after boot, is cached for six hours in `os.tmpdir()` and is skipped silently
215
+ when the registry is unreachable. Set `JSKELET_VERSION_CHECK=0` to disable it.
216
+
217
+ Warming requests (`user-agent: jskelet-prewarm`) are filtered out of both the
218
+ terminal and the request list: hundreds of requests should not flood the view.
219
+ Progress appears in the badge next to the bubble.
220
+
221
+ ### Why state is written to `os.tmpdir()`
222
+
223
+ If request and error records lived in process memory, history would be erased on
224
+ every restart and the overlay would come up empty. So the records are carried
225
+ across restarts in a file.
226
+
227
+ The file is **not written into the project tree**: every write triggered the
228
+ watcher and restarted the server, which set up a self-feeding loop (restart →
229
+ startup warning → write → restart). Instead, the file is written to
230
+ `os.tmpdir()/jskelet-devtools-<hash of the project root>.json`; thanks to the
231
+ hash, multiple JSkelet projects on the same machine do not overwrite each
232
+ other's records.
233
+
234
+ The write happens after 300 ms of silence rather than on every request, and its
235
+ failure does not stop the dev flow. At most 50 requests and 50 errors are kept.
236
+
237
+ The panel's open/closed state, its active tab and the browser error log are kept
238
+ in tab memory (`sessionStorage`), so after a reload the panel comes back with
239
+ the same tab.
240
+
241
+ ## Report page
242
+
243
+ The bubble shows the current state; the report page produces a view of the whole
244
+ site. The address:
245
+
246
+ ```
247
+ http://localhost:3000/__jskelet/dev/report
248
+ ```
249
+
250
+ (Or wherever `brand.devBasePath` points, if you changed it.)
251
+
252
+ Its contents:
253
+
254
+ - **Pages:** the Web Vitals measurements of every visited page, resource count
255
+ and total bytes (broken down by type), island status (how many are ready, and
256
+ their names), API calls made in the browser, and the size/duration/cache
257
+ status of the SSR output. Pages that were never visited but were warmed are
258
+ listed too: the SSR side is known, the client measurements stay empty.
259
+ - **Server API calls:** outbound `fetch` calls made during SSR — URL, host,
260
+ method, status, duration, bytes, which page was rendering, and a body summary
261
+ on failures. `globalThis.fetch` is only wrapped in development; the production
262
+ path is left untouched. Requests to our own server (warming, health check) do
263
+ not count as API calls. Failures also appear on the overlay Errors tab.
264
+ - **Build output:** the raw/gzip/brotli size of every asset in the manifest, and
265
+ chunk analysis from esbuild's metafile — the size of each output, which
266
+ sources it is made of, which chunks it imports. Sources are reduced to
267
+ readable groups (package name or parent folder), so the question "which
268
+ library accounts for 40 kB of this chunk" can be answered.
269
+ - **HTML cache:** entry count and a dump (key, bytes, status, whether it is
270
+ stale, how many seconds until it expires, which encodings are stored).
271
+ - **Prewarm:** the full result of the last round.
272
+ - **Request and error logs.**
273
+
274
+ Measurements live on the server, not in the browser tab; resetting is done from
275
+ the server as well. Size calculations are not repeated unless the file changed.
276
+
277
+ The report layer is only loaded in development and never enters the production
278
+ output.
279
+
280
+ ## Dev endpoints
281
+
282
+ Under `brand.devBasePath` (default `/__jskelet/dev`):
283
+
284
+ | Path | Method | Job |
285
+ | --- | --- | --- |
286
+ | `/overlay.js` | GET | The overlay script |
287
+ | `/seo.js` | GET | SEO scan + page highlight helper (imported by the overlay) |
288
+ | `/logo.png` | GET | The overlay logo |
289
+ | `/ws` | GET (upgrade) | Live channel: statistics, live reload and CSS hot-swap events |
290
+ | `/events` | GET | SSE: the fallback event stream, used only when WebSocket cannot be established |
291
+ | `/stats` | GET | Current statistics; the data endpoint of that same fallback |
292
+ | `/report` | GET | The report page (HTML) |
293
+ | `/report.js` | GET | The report page's script |
294
+ | `/report/data` | GET | The report's single data source (JSON) |
295
+ | `/vitals` | POST | The measurement bundle sent by the overlay |
296
+ | `/report/clear` | POST | Resets page measurements and server API records |
297
+ | `/prewarm` | POST | Triggers warming manually. If the body has `paths`, only those paths; 409 if warming is already running. |
298
+ | `/clear` | POST | Resets the request and error logs |
299
+
300
+ All of these endpoints are mounted by `mountDevtools()` only when
301
+ `NODE_ENV=development`; thanks to the dynamic import, nothing is loaded into the
302
+ production process.
303
+
304
+ ## Dev gate — `DEV_TOKEN`
305
+
306
+ To hide an environment that is not public yet. **The framework does not require
307
+ the token:** a `DEV_TOKEN` sitting in the environment does not lock the site.
308
+ You turn the gate on.
309
+
310
+ ```bash
311
+ DEV_GATE=1 DEV_TOKEN=a-long-random-string npm start
312
+ ```
313
+
314
+ The same thing from config:
315
+
316
+ ```js
317
+ export default {
318
+ devGate: true,
319
+ };
320
+ ```
321
+
322
+ Access:
323
+
324
+ ```
325
+ https://staging.example.com/?dev_token=a-long-random-string
326
+ ```
327
+
328
+ Behavior:
329
+
330
+ - **404, not 403.** A 403 confirms the environment exists; a 404 acts as if it
331
+ never did.
332
+ - Once the token arrives as a query parameter, it is written to a cookie
333
+ (`Path=/`, `SameSite=Lax`, 14 days), so sharing the link is enough. The cookie
334
+ and parameter name is `brand.devTokenCookie` (default `dev_token`).
335
+ - The **exact** paths in the `devGateBypass` list are open under all conditions.
336
+ Default: `/api/healthcheck`, `/robots.txt`, `/sitemap.xml`,
337
+ `/site.webmanifest`, `/favicon.ico`. If your health check lives at a different
338
+ path, remember to add it to this list, otherwise your orchestrator will see a
339
+ 404.
340
+ - While `devGate` is off (the default) or `DEV_TOKEN` is empty, the middleware
341
+ passes the request through. A `DEV_TOKEN` that leaked into a production task
342
+ does not ask visitors for a token; startup prints a warning.
343
+ - `DEV_GATE=0` turns the gate off even when config says `devGate: true`.
344
+ - While the gate is on, warming carries the token as a cookie; without it every
345
+ page gets a 404 and the cache never fills up
346
+ ([06-caching.md](./06-caching.md)).
347
+
348
+ In the middleware chain the gate sits after `headers` and **before**
349
+ `redirects`: an environment that is not public yet should not leak even its
350
+ redirect rules.
351
+
352
+ ## Differences between development and production
353
+
354
+ | Topic | Development | Production |
355
+ | --- | --- | --- |
356
+ | EJS template cache | Off | On |
357
+ | Manifest reading | On every request | Once |
358
+ | Image manifest | On every call | Once |
359
+ | Broken route module | Warn + skip | Throw |
360
+ | Devtools and report | Mounted | Never loaded |
361
+ | `globalThis.fetch` | Wrapped (measurement) | Untouched |
362
+ | Prewarm concurrency | 1 | 4 |
363
+ | Prewarm rate limit | 4 requests/second | Unlimited |
364
+ | Prewarm delay | 3000 ms | 500 ms |
365
+ | Missing icon warning | Emitted | Not emitted |
366
+ | Precompress | Does not run in watch | Runs |
367
+ | Image optimization | Does not run in watch | Runs |
368
+
369
+ ## What's next
370
+
371
+ - Details of the build steps: [08-build.md](./08-build.md)
372
+ - Going to production: [10-deployment.md](./10-deployment.md)
373
+ - Reading and clearing the cache: [06-caching.md](./06-caching.md)