@plurnk/plurnk-schemes-http 2.0.4 → 2.0.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (64) hide show
  1. package/README.md +4 -4
  2. package/SPEC.md +122 -606
  3. package/dist/Guard.d.ts +4 -0
  4. package/dist/Guard.d.ts.map +1 -1
  5. package/dist/Guard.js +43 -5
  6. package/dist/Guard.js.map +1 -1
  7. package/dist/Http.d.ts +1 -1
  8. package/dist/Http.d.ts.map +1 -1
  9. package/dist/Http.js +22 -619
  10. package/dist/Http.js.map +1 -1
  11. package/dist/HttpCache.d.ts +10 -0
  12. package/dist/HttpCache.d.ts.map +1 -0
  13. package/dist/HttpCache.js +384 -0
  14. package/dist/HttpCache.js.map +1 -0
  15. package/dist/HttpGet.d.ts +5 -26
  16. package/dist/HttpGet.d.ts.map +1 -1
  17. package/dist/HttpGet.js +30 -63
  18. package/dist/HttpGet.js.map +1 -1
  19. package/dist/HttpRequester.d.ts +4 -15
  20. package/dist/HttpRequester.d.ts.map +1 -1
  21. package/dist/HttpRequester.js +31 -49
  22. package/dist/HttpRequester.js.map +1 -1
  23. package/dist/HttpShared.d.ts +42 -0
  24. package/dist/HttpShared.d.ts.map +1 -0
  25. package/dist/HttpShared.js +244 -0
  26. package/dist/HttpShared.js.map +1 -0
  27. package/dist/WebFetcher.d.ts +2 -7
  28. package/dist/WebFetcher.d.ts.map +1 -1
  29. package/dist/WebFetcher.js +14 -26
  30. package/dist/WebFetcher.js.map +1 -1
  31. package/dist/Ws.d.ts.map +1 -1
  32. package/dist/Ws.js +11 -7
  33. package/dist/Ws.js.map +1 -1
  34. package/dist/index.d.ts +0 -4
  35. package/dist/index.d.ts.map +1 -1
  36. package/dist/index.js +2 -5
  37. package/dist/index.js.map +1 -1
  38. package/docs/https.md +2 -2
  39. package/docs/wss.md +1 -1
  40. package/package.json +5 -5
  41. package/dist/Config.d.ts +0 -5
  42. package/dist/Config.d.ts.map +0 -1
  43. package/dist/Config.js +0 -13
  44. package/dist/Config.js.map +0 -1
  45. package/dist/ContentType.d.ts +0 -2
  46. package/dist/ContentType.d.ts.map +0 -1
  47. package/dist/ContentType.js +0 -15
  48. package/dist/ContentType.js.map +0 -1
  49. package/dist/ErrorDetail.d.ts +0 -5
  50. package/dist/ErrorDetail.d.ts.map +0 -1
  51. package/dist/ErrorDetail.js +0 -5
  52. package/dist/ErrorDetail.js.map +0 -1
  53. package/dist/HostPolicy.d.ts +0 -11
  54. package/dist/HostPolicy.d.ts.map +0 -1
  55. package/dist/HostPolicy.js +0 -58
  56. package/dist/HostPolicy.js.map +0 -1
  57. package/dist/LiveAcquisitions.d.ts +0 -8
  58. package/dist/LiveAcquisitions.d.ts.map +0 -1
  59. package/dist/LiveAcquisitions.js +0 -30
  60. package/dist/LiveAcquisitions.js.map +0 -1
  61. package/dist/http-names.d.ts +0 -5
  62. package/dist/http-names.d.ts.map +0 -1
  63. package/dist/http-names.js +0 -6
  64. package/dist/http-names.js.map +0 -1
package/SPEC.md CHANGED
@@ -1,640 +1,156 @@
1
- # plurnk-schemes-http — Specification
1
+ # `@plurnk/plurnk-schemes-http` Specification
2
2
 
3
- This package owns the `http(s)://` request/response scheme, the `ws(s)://`
4
- full-duplex scheme, plus automatic entry-acquisition, pluggable page
5
- materializers, and the local HTML projection foundation. Both handlers implement
6
- the DB-free `SchemeCtx` author contract.
3
+ Owns the `http(s)://` request/response scheme (`Http`), the `ws(s)://` full-duplex WebSocket scheme (`Ws`), automatic web acquisition (`WebFetcher`), URL/host policy (`Guard`, `HostPolicy`), and pluggable HTML materializers (`MaterializerRegistry`).
7
4
 
8
- ## §http-manifest §1 HTTP manifest
5
+ ## §http-manifest HTTP manifest and operation surface
9
6
 
10
- | Field | Value |
11
- | --------------- | ----------------------------- |
12
- | Registered name | `https` (plain `http` routes to it — supported, never advertised as a peer endpoint) |
13
- | Category | `data` |
14
- | Writers | `model`, `client` |
15
- | Volatile | `true` |
16
- | Model-visible | `true` |
17
- | Requires web | `true` |
18
- | Metadata modifier | `true` |
19
- | Default channel | `body` |
7
+ `Http` registers as `https` (plain `http` routes to the same handler; `package.json#plurnk.schemes` registers `http` via default export and `wss` via `Ws`) with `category: "data"`, `writableBy: ["model", "client"]`, `volatile: true`, `modelVisible: true`, `metadataModifier: true`, `traits: ["web"]`, and channels:
20
8
 
21
- | Channel | Seed type | Meaning |
22
- | -------- | --------------------------- | ------------------------------------------------------------------------------ |
23
- | `body` | `application/octet-stream` | Source text or original binary bytes; SSE data for a live response |
24
- | `header` | `text/plain` | Origin, acquisition, materializer, projection, provider, and usage evidence |
25
- | `readable` | `text/markdown` | Optional derived text/facts: an HTML materializer or installed mimetype projection ({§readable-channel}) |
9
+ | Channel | Seed type | Meaning |
10
+ |---|---|---|
11
+ | `body` (default) | `application/octet-stream` | Source text or original binary bytes; SSE `data` lines for a live response |
12
+ | `header` | `text/plain` | Origin status/headers, acquisition stamps, cache variant, materializer, and projection evidence |
13
+ | `readable` | `text/markdown` | Optional derived text/facts from an HTML materializer or installed mimetype projection ({§channel-selection}) |
26
14
 
27
- `package.json#plurnk.schemes` registers `http` through the default export and
28
- `wss` through `Ws`.
15
+ Identity and transport URLs normalize through `NetworkAddress` ({§network-address}): authority is `<host>[:<non-default-port>]`, pathname is `<path>[?<serialized-query>]` (`%28`/`%29` in path canonicalize to literal `(` / `)`, query order/duplicates/empty `?` preserved), `#fragment` selects the Plurnk channel, and URL userinfo is rejected (`400`).
29
16
 
30
- Routing is not identity. `NetworkAddress` {§network-address} retains the exact
31
- addressed protocol and stores authority `<host>[:<non-default-port>]` separately
32
- from pathname `<path>[?<serialized-query>]`. Query ordering,
33
- duplicate names, and an explicit empty `?` remain significant. The fragment is
34
- a Plurnk channel selector and never enters network identity or transport.
35
- Request metadata affects transport but not identity. URL userinfo is rejected;
36
- neither credentials nor metadata are reconstructed from `raw`. Within the path
37
- component, `%28` and `%29` canonicalize to literal parentheses; the query is
38
- preserved exactly.
17
+ | Operation | Remote action | Contract |
18
+ |---|---|---|
19
+ | Exact `READ (url) <scope?>` | `GET` (unless a fresh cached `GET` is usable) | Scope-blind `prepareRepresentation`, then Core selects channel and scope |
20
+ | Exact `FIND (url)` | `GET` only when acquisition is required | Same preparation, then universal query, matcher, weighting, and pagination |
21
+ | Pattern `FIND (pattern-url)` | None (`PathSyntax.hasGlob`, {§path-syntax}) | Query already-materialized web entries in the workspace without remote discovery |
22
+ | `SEND (url)` with body | `POST` | Proposes (`effect: "host"`, `202`); on apply, streams and persists response under `url` |
23
+ | `EDIT (url)` with body | `PUT` | Proposes (`effect: "host"`, `202`); replaces whole remote resource (line markers fail `400`: `Remove the line range and submit the complete replacement body.`) |
24
+ | `KILL (url)` | None | Cancels all live workspace acquisitions of `url` (owners settle `499`), else deletes local stored entry |
25
+ | `KILL (url) [{"remote": true}]` | `DELETE` | Proposes (`effect: "host"`, `202`); remaining options in the metadata block become request headers |
39
26
 
40
- ## §op-surface §2 HTTP operation surface
27
+ - **Headers and proposals (`http-outbound-proposes`):** Request headers are passed in one `[{"Key": "value"}]` metadata block ({§slot-order}). `POST`, `PUT`, remote `DELETE`, and WebSocket `EDIT`/`SEND` frames declare `effect: "host"` and return `202` ({§proposal}, {§exec-lifetime}), governed by `PLURNK_SERVICE_EFFECT_HOST`. `GET` and WebSocket `READ` connection opens are observational and ungated; a WebSocket frame checks for an open socket both before proposing and again at `applyResolution`, and the resolver may replace the proposed body before it is sent.
41
28
 
42
- | Operation | Remote action | Contract |
43
- | ------------------------------------------ | ------------------------------------- | --------------------------------------------------------------------------------------- |
44
- | Exact ```` ```READ (url) <scope?> ```` | GET unless a fresh GET copy is usable | Prepare one complete canonical representation, then let core select and project it |
45
- | Exact ```` ```FIND (url) ```` | GET only when acquisition is required | Use the same preparation, then universal query, matcher, weighting, and pagination |
46
- | ```` ```FIND (pattern-url) ```` | None | Query already-materialized web entries; a path pattern does not discover the remote web |
47
- | ```` ```SEND (url) ```` with body | POST | Stream and persist the response under the addressed URL |
48
- | ```` ```EDIT (url) ```` with body | PUT | Replace the whole remote resource; a line marker is invalid |
49
- | ```` ```KILL (url) ```` | None | {§http-kill}: cancel the workspace's live acquisitions of the URL, else delete the local stored entry |
50
- | ```` ```KILL (url) [{"remote": true}] ```` | DELETE | Delete the remote resource and stream its response; other options in the same metadata block are its headers |
51
-
52
- Finite GET uses scope-blind representation preparation; POST, PUT, DELETE,
53
- and genuinely live GET responses retain the subscription path. Request
54
- headers are options in one `[{"Key": "value", ...}]` metadata block under
55
- {§scheme-metadata-modifier}. HTTP interprets the merged options; the resource
56
- target remains pure. Remote status and headers are persisted in `header`, not
57
- interpreted as Plurnk operation or loop lifecycle signals.
58
- Exact-versus-pattern FIND preparation uses the shared
59
- `PathSyntax.hasGlob` classifier {§path-glob}; HTTP owns no reduced
60
- path-pattern grammar.
61
-
62
- §http-outbound-proposes **A request that changes a remote resource proposes.** POST, PUT, the remote DELETE, and a WebSocket frame (EDIT or SEND on an open `ws(s)://` socket) declare `effect: "host"` and return `202` under {§proposal}; the panel's `PLURNK_SERVICE_EFFECT_HOST` decides whether that settles by consent or runs unattended, exactly as it does for a subprocess ({§exec-host-proposes}). This is the same reason a local `EDIT` proposes: Plurnk does not claim containment, so consent — not a boundary — is what stands between the model and the world.
63
-
64
- | Fact | Rule |
65
- | --- | --- |
66
- | GET, and the socket open a `ws(s)://` READ performs | observation, ungated: an acquisition leaks only what a URL can carry, and that is the same for both |
67
- | A frame | needs an open socket, so that refusal arrives before any proposal — nobody is asked to approve a message to a connection that is not there — and the admission runs again at apply, because the socket is live and may have closed while the question was open |
68
- | The proposal | carries the method, target, and body, and the resolver may replace the body before it is sent; nothing leaves the process until the settlement accepts |
69
-
70
- ## §http-lifecycle §3 Acquisition, materialization, and query lifecycle
71
-
72
- <!-- map -->
73
- | Tag | Promise |
74
- | --- | --- |
75
- | {§http-text-decoding} | Text response decoding |
76
- | {§http-binary-source} | Finite binary responses retain bounded original bytes in `body`, with optional derived facts/text in… |
77
- | {§http-json-presentation} | Finite `application/json` and `+json` responses use {§json-document-presentation} in their canonical body… |
78
- | {§html-materialization} | Readable materialization |
79
- | {§http-materializer-extensions} | Materializer extensions |
80
- | {§http-channel-outcomes} | Channel outcomes |
81
- | {§http-llms-txt} | Origin llms.txt companions. |
82
- <!-- /map -->
29
+ ## §http-lifecycle Acquisition, decoding, and materialization
83
30
 
84
31
  ```mermaid
85
32
  flowchart TD
86
33
  get["GET acquisition"] --> live{"text/event-stream?"}
87
- live -->|yes| subscription["Seed canonical channels<br/>retain subscription; return 102"]
34
+ live -->|yes| sse["Seed canonical channels, retain subscription, return 102"]
88
35
  live -->|no| origin{"Finite origin outcome"}
89
- origin -->|text, including Markdown| text["Source text in body"]
90
- origin -->|binary| binary["Bounded original bytes in body<br/>optional projection in readable"]
91
- origin -->|server HTML| source["Retain exact server HTML in body"]
92
- origin -->|transport unavailable<br/>after public admission| noSource["Preserve source failure in body"]
93
- source --> eligible{"Generic public request<br/>with a materializer selected?"}
94
- noSource --> eligible
95
- eligible -->|no| local["Installed HTML projection"]
96
- eligible -->|yes| mat["Materializer Markdown"]
97
- mat -->|success| providerText["Materializer readable"]
98
- mat -->|recoverable failure<br/>and source exists| recovery["Local readable; status 203"]
99
- mat -->|hard failure or<br/>no recovery source| readableError["Readable errored"]
100
- local -->|present, including empty| localText["Local readable"]
101
- local -->|absent or no source| readableError
102
- text --> settle["Write complete canonical channels<br/>and durable producer results"]
103
- binary --> settle
104
- providerText --> settle
105
- recovery --> settle
106
- localText --> settle
107
- readableError --> settle
108
- settle --> core["Core selects channel and scope<br/>native media follows ordinary READ"]
36
+ origin -->|text / negotiated Markdown| text["Fetch UTF-8 text in body (JSON formatted via contract-representations)"]
37
+ origin -->|binary| bin["Bounded original bytes in body + optional projection in readable"]
38
+ origin -->|server HTML or public transport loss| html{"Generic public request with materializer?"}
39
+ html -->|yes| mat["Materializer extract -> readable (203 local fallback on recoverable error)"]
40
+ html -->|no| local["Installed HTML projection -> readable"]
41
+ text & bin & mat & local --> settle["Write complete canonical channels & durable producerResults atomically"]
109
42
  ```
110
43
 
111
- The same page producer serves exact GET preparation, exact FIND, and executor
112
- entry acquisition. Their outer policies differ: authored GET accepts explicit
113
- targets and HTTP error responses; automatic acquisition first admits a public
114
- credential-free target and treats non-2xx responses as unavailable. READ scope
115
- and channel selection never enter the producer. Freshness may avoid transport,
116
- but cold and warm representations pass through the same core projection.
117
-
118
- | Response | `body` | Auxiliary materialization | Completion |
119
- | --------------------------------------- | -------------------------------------------------------- | --------------------------------------------------------- | ------------------------------------------- |
120
- | Negotiated origin Markdown | Exact origin Markdown | None: readable text is the source itself; one request | Static representation, then core projection |
121
- | HTML-page production | Exact server source when the origin supplied it | Materializer Markdown, local Markdown, or independent error in `readable` | Static representation, then core projection |
122
- | `text/event-stream` | One `data` value plus newline per `text/plain` chunk | Initial response in `header` | `102`; origin close settles subscription |
123
- | No response body | Present empty text | Response and package metadata in `header` | Static representation; body READ is `204` |
124
- | Origin HTTP `4xx`/`5xx` | Preserve available origin text or bytes | Exact response evidence; origin-backed channels errored | Selected channel's durable outcome |
125
- | Configured textual type | Complete Fetch-decoded Unicode | Response and package metadata in `header` | Static representation |
126
- | Readable binary type | Complete original bytes | Derived Unicode in `readable`; projection identity in `header` | Static representation, then core projection |
127
- | Unreadable binary or unknown bytes | Complete original bytes | No readable projection or native media claim | Static representation; ordinary hex READ |
128
- | Binary input exceeds configured bound | No representation is fabricated | Exact size evidence in the Problem | `413 projection-input-limit` |
129
-
130
- The producer persists every available channel; core publishes only the channel
131
- selected by the fragment, or `body` by default. Broad FIND returns standard
132
- resource metadata; exact matcher FIND returns flat match locations. Exact READ
133
- returns the selected channel's requested text/hex projection.
134
-
135
- ### §http-text-decoding Text response decoding
136
-
137
- | Surface | Contract |
138
- | ----------------------------- | ------------------------------------------------------------------------------------------------ |
139
- | Response media type | WHATWG `MIMEType` essence; absent or unparseable metadata becomes `application/octet-stream` |
140
- | Finite GET text | Fetch `Response.text()` UTF-8 decoding; buffering does not select a different character encoding |
141
- | Streamed mutation response | Incremental replacement-mode UTF-8 through `TextDecoder`; finite JSON documents publish after completion, other text preserves incremental backpressure |
142
- | `charset` parameter | Preserve in `header` as origin evidence; it does not replace Fetch text decoding |
143
- | JSON and XML textual families | Use the same HTTP byte-to-string rule; format projections consume the resulting Unicode string |
144
- | Direct HTML GET | Fetch UTF-8 decoding of origin server HTML |
145
- | Server-sent events | UTF-8 only under the HTML event-stream standard |
146
- | Malformed UTF-8 | Preserve the Encoding Standard's replacement-character behavior |
147
-
148
- This decoder boundary remains text normalization, not a media-format processor.
149
- A configured binary type bypasses it and enters the mimetype framework's bounded
150
- source acquisition {§mimetype-binary-input}; original bytes remain in `body`.
151
-
152
- §http-binary-source Finite binary responses retain bounded original bytes in `body`,
153
- with optional derived facts/text in `readable`. GET, exact FIND acquisition, executor
154
- acquisition, and mutation responses use the same source/projection contract.
155
- Core READ of `body`, `#bytes`, or `#readable` may attach that complete native source
156
- under {§packet-attachment-parts}; scope selects the text/hex view, never native bytes.
157
- Headers carry transport evidence, not native media. Invalid or unsupported formats
158
- remain byte-readable without invented facts or native parts. Incomplete transfers
159
- and over-limit input publish no partial binary payload.
160
-
161
- §http-json-presentation Finite `application/json` and `+json` responses use
162
- {§json-document-presentation} in their canonical body channel before indexing,
163
- scope selection, or previews. Both GET and mutation responses follow this rule.
164
- Malformed JSON remains unchanged; interrupted mutation responses preserve the
165
- received partial text and the acquisition failure. Request bodies, authored
166
- entries, non-JSON text, SSE, and JSONL retain their original formatting.
167
-
168
- ### §html-materialization Readable materialization
169
-
170
- `WebFetcher.materialize` is the shared source/projection seam for exact
171
- GET/FIND preparation and executor entry acquisition; `Http` publishes its
172
- `WebFetcher` as the framework's {§web-materialization-contract}. It returns complete
173
- source and derived channels for one atomic canonical entry write; SSE and
174
- textual mutation responses retain incremental streaming.
175
-
176
- | Input or event | Action | Result |
177
- | ------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------ |
178
- | Configured non-HTML text | Decode with Fetch UTF-8 | Original representation in `body` |
179
- | Configured binary with reader | Retain bounded bytes and apply projection | Original `body`, derived `readable`, and projection identity |
180
- | Configured binary without reader | Retain bounded bytes | Original `body` and projection identity; no `readable` |
181
- | Binary input exceeds the common bound | Cancel and preserve the typed cause | `WebMaterializationError` caused by `ProjectionInputLimitError` |
182
- | Negotiated origin `text/markdown` | Accept it without the materializer | Exact Markdown in `body`; no `readable`, no second request |
183
- | Eligible HTML with a materializer selected | Use the materializer as the readable producer | Origin server source in `body`; materializer Markdown in `readable` |
184
- | Ineligible HTML or no materializer selected | Use the installed HTML projection | Origin server source in `body`; local Markdown in `readable` |
185
- | Recoverable materializer failure with HTML | Use the installed projection as recovery | Source in `body`; local Markdown in `readable` with terminal `203` |
186
- | Hard materializer failure | Preserve evidence; do not run local recovery | `readable` errored; independently successful `body`/`header` survive |
187
- | Materializer success after origin loss | Preserve provider text and origin failure | `readable` static; `body` errored with the origin's own failure |
188
- | Projection implementation throws | Stop and preserve the cause | `WebMaterializationError` with stage `projection` |
189
-
190
- A projection object is present even when its content is `""`; only `null`
191
- denotes absence. A materialization exception retains its original `cause`; it
192
- never enters the absence channel.
193
-
194
- #### §http-materializer-extensions Materializer extensions
195
-
196
- `PLURNK_SCHEMES_HTTP_MATERIALIZER` selects one discovered `http-materializer`
197
- extension by id; unset means the installed HTML projection is the only readable
198
- producer. A materializer package declares
199
- `plurnk: { kind: "http-materializer", materializers: [{ id, module }] }` and
200
- exports one `HttpMaterializer` per entry under the exec framework's
201
- discovery, trust, and one-flat-id-namespace rules ({§extension-discovery}). The
202
- selected materializer is consulted only for a credential-free generic request
203
- whose target has been admitted as public. Authored request metadata—including
204
- an authored `Accept` field—makes the request ineligible. The package-generated
205
- Markdown negotiation, browser-compatible `User-Agent`, and conditional cache
206
- fields are transport mechanics, not authored metadata. No authored request headers or
207
- origin credentials cross the materializer boundary.
208
-
209
- An npm-installed Agent Plugin puts the same declaration under
210
- `plugin.json#extensions.ai.plurnk` ({§extension-manifest-read}); file entries live in
211
- `ai.plurnk/`. Discovery and implementations are cached process-wide and loaded
212
- lazily. A daemon instance neither registers nor removes materializers.
213
-
214
- When no `Accept` is authored, origin acquisition offers
215
- `text/markdown, text/html;q=0.9, */*;q=0.1`. An origin `text/markdown`
216
- representation wins and the materializer is skipped; no second HTML request is
217
- made. An authored `Accept` value is
218
- sent unchanged and makes any returned HTML use the local projection route.
219
-
220
- The materializer's `eligible(url)` returns its identity string when it will
221
- serve the URL (its credentials and supported targets are its own); `extract`
222
- returns `success` (its sanitized Markdown body plus its own evidence headers),
223
- `recoverable` (local projection fallback with the materializer's evidence and
224
- terminal `203`), or `hard` (an exact materializer-owned Problem; no local
225
- recovery). A materializer `extract` throw is the materialization failure, never
226
- a silent fallback. A selected id with no installed provider fails hard rather
227
- than degrading silently.
228
-
229
- Without server HTML, no local recovery input exists. A materializer success can
230
- still close `readable`, but every materializer failure remains the readable
231
- failure, and `body` closes errored with the origin's own failure.
232
-
233
- #### §http-channel-outcomes Channel outcomes
234
-
235
- For finite HTML-page production, `body`, `header`, and `readable` have independent
236
- durable outcomes and are written atomically. The selected channel alone
237
- determines the projected operation result: successful `#readable` or `#header` can
238
- be read when `body` failed, and a successful default `body` is not invalidated
239
- by an errored `readable`. Unselected failures remain `errored` channels carrying
240
- their exact `producerResult`. A recoverable local projection remains a static
241
- `readable` channel with producer status `203`, so cold and warm READ both return
242
- its content with that same status. Every READ names the page's other channels
243
- with their tokens ({§channel-selection-visibility}), so a READ of the source
244
- shows that `#readable` exists before the model has ever listed the page.
245
-
246
- Exact READ, exact FIND, and executor materialization preserve the same channel
247
- representation. A missing source variant is an explicit empty `errored`
248
- channel, never an absent fact that later cache use can reinterpret as successful
249
- empty content. A finite origin `2xx` or `3xx` response completes ordinary local
250
- production, independently of the origin's status meaning; `202` does not leave
251
- an acquisition pending, and origin `203` is not a materializer recovery. Origin
252
- `4xx`/`5xx` content remains available as evidence while origin-backed channels
253
- carry the exact `http-response-status` producer Problem. A final status outside
254
- `200`–`599` instead produces `502 invalid-response-status` with `originStatus`,
255
- preserving the received content and header. A materializer-produced `readable`
256
- and the acquisition `header` remain independent of an unavailable origin source.
257
- Conditional `304` correspondence is checked before production under {§revalidation}.
258
-
259
- §http-llms-txt **Origin llms.txt companions.** After a successful generic GET
260
- materialization, the scheme opportunistically acquires `<origin>/llms.txt`
261
- once per origin per TTL window and materializes it as its own origin entry
262
- (`https://host/llms.txt`), surfaceable by FIND. A missing companion, any
263
- non-2xx, binary content, or a transport failure is quiet: the companion is
264
- never fabricated, the failed probe is remembered (not retried per READ), and
265
- the piggyback never fails the READ that carried it. The companion itself
266
- never recurses. This is lazy by construction — one extra request only when
267
- an origin is first being read — and it is the operator's to allow:
268
- `PLURNK_SCHEMES_HTTP_LLMS_TXT=0` makes only the requests a READ asks for.
269
-
270
- ## §http-status §4 HTTP status mapping
271
-
272
- | Outcome | Operation status |
273
- | ------------------------------------------------------------- | -------------------------------------------------------- |
274
- | Finite origin `2xx`/`3xx` exact READ after preparation | Universal selected-channel result (`200` or `204`); origin status remains in `header` |
275
- | Exact FIND after preparation | Exact universal query result |
276
- | Exact acquisition returns no WebFetcher value | `404` (`not-materialized`) |
277
- | Selected origin-backed channel received HTTP `4xx`/`5xx` | Exact durable `http-response-status` Problem |
278
- | Selected origin-backed channel received an invalid final status | `502` (`invalid-response-status`) with `originStatus` |
279
- | Selected HTML-page channel fails | That channel's exact durable producer Problem |
280
- | Local HTML projection is absent | `422` (`no-readable-projection`) |
281
- | Recoverable materializer failure uses local projection | `readable` with durable status `203` |
282
- | Finite textual or binary response | Universal READ result |
283
- | Finite empty text response | `204` |
284
- | Binary projection input exceeds the configured byte bound | `413` (`projection-input-limit`) |
285
- | `KILL` with no live acquisition | Exact entry-delete result |
286
- | `KILL` of a live acquisition | `200`; the owner settles `499` through its aborted signal |
287
- | Client-cancelled finite acquisition | `499` (`cancelled`) |
288
- | SSE after acquisition | `102` initial; terminal selected-channel result |
289
- | SSE cancellation after acquisition | `102` initial; terminal `499` |
290
- | SSE parser/transfer failure after acquisition | `102` initial; terminal `502` |
291
- | Multi-statement HTTP edit batch | `409` (`non-atomic-edit-batch`) |
292
- | Invalid target, line edit, or URL userinfo | `400` with the corresponding stable Problem kind |
293
- | Missing response channel | `404 channel-not-found` under {§channel-selection-missing}; no remote mutation |
294
- | Non-corresponding 304 | `502` (`fetch-failed`) |
295
- | Acquisition failure without successful materializer output | `502` (`fetch-failed`) |
296
- | Projection exception | `500` (`projection-failed`) |
297
- | Uninterpreted SEND status | `501` (`send-status-unsupported`) |
298
-
299
- An HTTP error status is still a successfully acquired direct response: its
300
- content and response evidence are preserved, and origin-backed channels carry
301
- that exact status as durable producer evidence. HTML still enters page
302
- production, so a successful materializer `readable` can remain readable while the
303
- origin `body` records the HTTP error. Automatic WebFetcher acquisition instead treats
304
- a non-2xx response as unavailable. Handler failures use RFC 9457 Problem Details.
305
- Caught direct-acquisition diagnostics are bounded by
306
- `PLURNK_SCHEMES_HTTP_ERROR_DETAIL_LIMIT` in model-facing detail while complete
307
- errors remain in daemon diagnostics.
308
-
309
- §http-replay A failed GET, PUT, or DELETE acquisition may recommend automatic
310
- identical replay because those methods are idempotent. POST may already have
311
- applied its effect when acquisition fails and is therefore non-retryable. Once
312
- an SSE response has been acquired, a parser or transfer failure is likewise
313
- non-retryable because replay could duplicate an already-consumed prefix.
314
-
315
- Binary responses follow {§http-binary-source}, appending authoritative
316
- `x-plurnk-projection-id` evidence after the origin and acquisition fields.
317
- Exceeding the input ceiling returns `413` with configured and observed sizes;
318
- a mutation response retains its lifecycle/header evidence and settles errored
319
- without publishing partial bytes. This is a materialization failure, not the
320
- remote HTTP outcome—a POST, PUT, or DELETE might already have changed the resource.
321
- Missing or malformed `Content-Type` becomes `application/octet-stream`; the
322
- handler does not sniff or guess unknown bytes.
323
-
324
- ## §5 Dependencies and configuration
325
-
326
- <!-- map -->
327
- | Tag | Promise |
328
- | --- | --- |
329
- | {§http-config} | Operator configuration |
330
- | {§http-host-policy} | Operator web host policy |
331
- | {§automatic-fetch-check} | Automatic acquisition URL check |
332
- <!-- /map -->
333
-
334
- | Surface | Runtime contract |
335
- | ----------------- | --------------------------------------------------------------------------------------------------------- |
336
- | Platform | Node ≥26 native fetch, streams, abort signals, decoding, DNS, and `WebSocket` |
337
- | SSE | `eventsource-parser` for bounded WHATWG event-stream framing |
338
- | Materializer extensions | Discovered `http-materializer` packages selected by `PLURNK_SCHEMES_HTTP_MATERIALIZER` {§http-materializer-extensions} |
339
-
340
- ### §http-config Operator configuration
341
-
342
- | Concern | Contract |
343
- | --------------------- | -------------------------------------------------------------------------------------------------------- |
344
- | Canonical registry | Shipped `.env.defaults`; the daemon assembles it as a set-if-unset floor {§operator-config-env-defaults} |
345
- | Required values | Missing or invalid required values fail at readiness/owning read; code carries no hidden fallback |
346
- | Materializer selection | `PLURNK_SCHEMES_HTTP_MATERIALIZER=` names a discovered materializer id; unset = local projection only |
347
-
348
- ### §http-host-policy Operator web host policy
349
-
350
- `PLURNK_SCHEMES_HTTP_HOSTS` confines every web acquisition to named hosts. Unset
351
- or empty admits every host, as before. A JSON array admits exactly its members:
352
- `"example.com"` names that host, `"*.example.com"` any subdomain of it, and `[]`
353
- names none. A value that is not a JSON array of host names fails at first use.
354
- Direct HTTP operations and WebSocket connections are refused with 403
355
- `host-not-permitted` before any I/O, and a followed redirect that leaves the
356
- admitted hosts is cancelled with the same refusal. Automatic acquisition checks
357
- the target and every redirect hop, and a final URL outside the policy is the
358
- ordinary unavailable `null`. The policy is operator configuration, alias-free and
359
- not model teaching; an isolated benchmark sets `[]` as its network boundary and denies the
360
- `web` trait ({§capability-admission}) as the model's.
361
-
362
- ### §automatic-fetch-check Automatic acquisition URL check
363
-
364
- `WebFetcher` is the sole caller of `Guard.fetch`. Before automatic byte
365
- acquisition, it accepts credential-free HTTP(S) targets only when every
366
- resolved address is ordinary globally reachable unicast, and repeats the check
367
- before every manually followed redirect. A refused or unresolvable target is
368
- the ordinary unavailable `null` result. A transport failure after public
369
- admission may still enter materializer page production.
370
-
371
- Direct HTTP operations and WebSocket connections do not use this check. Loopback,
372
- private, and link-local destinations are therefore valid explicit targets.
373
-
374
- Redirect transitions follow WHATWG Fetch: 301/302 rewrite POST to GET; 303
375
- rewrites methods other than GET/HEAD to GET; 307/308 preserve method and body.
376
- A body rewrite removes body headers, cross-origin redirects remove
377
- `Authorization`, and followed redirect bodies are cancelled. The configured
378
- hop limit returns the last redirect rather than following beyond the limit.
379
- Validation-time DNS answers are not pinned to connection-time resolution. This
380
- check therefore makes no DNS-rebinding or total-egress claim.
381
-
382
- ## §materialization-lifecycle §6 Materialization lifecycle
383
-
384
- <!-- map -->
385
- | Tag | Promise |
386
- | --- | --- |
387
- | {§host-rewrite} | Acquisition target rewrite |
388
- | {§revalidation} | GET representation freshness |
389
- | {§http-kill} | KILL follows the entry rule; the remote DELETE is its own spelling. |
390
- | {§sse} | Server-sent events |
391
- <!-- /map -->
392
-
393
- | Concern | Contract |
394
- | -------------- | ---------------------------------------------------------------------------------------------------------- |
395
- | Direct gate | Explicit targets use native origin transport; only a generic public request grants materializer authority |
396
- | Automatic gate | Target and redirects require public admission; accepted generic HTML follows the same page producer |
397
- | Readable owner | Origin Markdown needs no projection; HTML uses the eligible materializer or installed local reader, binary uses its mimetype handler |
398
- | Source owner | `body` preserves origin text or bytes; it is never replaced by a derived projection |
399
- | Projection | A present projection, including `""`, becomes `readable`; `null` alone means absence |
400
- | Cancellation | One caller signal spans origin, auxiliary origin, projection, and materializer work |
401
-
402
- ### §host-rewrite Acquisition target rewrite
403
-
404
- Only acquisition GETs are eligible for host rewriting. A GitHub
405
- `…/blob/…` address uses the corresponding `raw.githubusercontent.com` source for
406
- byte transport. Direct READ, exact FIND, and
407
- WebFetcher prefetch share that rule. The originally addressed GitHub URL remains
408
- entry identity. POST, PUT, and DELETE are never retargeted. Rewritten targets
409
- are checked under {§automatic-fetch-check} only when `WebFetcher` acquires them.
410
-
411
- ### §revalidation GET representation freshness
412
-
413
- Acquired responses append package-owned `x-plurnk-request-method`,
414
- `x-plurnk-fetched-at`, and `x-plurnk-cache-variant` fields after origin headers.
415
- Page bodies append `x-plurnk-materializer-id`; local projections append
416
- `x-plurnk-projection-id`; The materializer appends its own evidence headers;
417
- and bounded failure evidence. Only package metadata after the acquisition stamp
418
- is authoritative.
419
- Plurnk stores one representation per canonical URL rather than a variant set:
420
-
421
- | Acquisition context | Package variant | Later cache use |
422
- | -------------------------------------------- | --------------- | --------------- |
423
- | No explicit request metadata; no `Vary` | `default` | Eligible |
424
- | Any explicit request metadata | `bypass` | Ineligible |
425
- | No explicit request metadata; any `Vary` | `bypass` | Ineligible |
426
-
427
- This conservative selection avoids persisting request values or fabricating a
428
- multi-variant store. Exact FIND passes request metadata through acquisition but
429
- does not reuse that response later. A `304` that introduces `Vary` changes the
430
- restored representation's package marker to `bypass`.
431
-
432
- Only an eligible GET representation can supply a direct READ's body, TTL stamp,
433
- conditional validators, or exact-FIND preparation. Completion comes from
434
- channel lifecycle, not body length: `body` and `header` must be successful
435
- (`static` or `closed`), and every channel must be terminal. An auxiliary channel
436
- may be `errored`; `active` or unknown state is ineligible. Durable operation
437
- evidence and HTTP reuse eligibility remain distinct: an acquired response stays
438
- in the entry even when its origin policy prevents later cache use.
439
- Workspace reuse follows shared-cache rules (RFC 9111); it does not change
440
- retained evidence or workspace access. Reuse also requires a heuristically
441
- cacheable origin status (RFC 9110 §15.1) or explicit cache permission (`public`,
442
- `s-maxage`, `max-age`, or `Expires`).
443
- Partial `206` responses remain evidence only: this cache does not combine or
444
- select byte ranges. Ineligible responses supply neither cached content nor
445
- validators to a later READ or exact FIND.
446
-
447
- | Stored origin policy | Direct READ after acquisition | Exact FIND after acquisition |
448
- | ------------------------------------- | ------------------------------------------------------ | ---------------------------------------- |
449
- | `no-store` or `private` (qualified or unqualified) | Full acquisition without stored validators | Full acquisition |
450
- | `no-cache` (qualified or unqualified) | Validate when a stored validator exists; else acquire | Full acquisition |
451
- | Valid `s-maxage` | Serve only inside origin lifetime and operator ceiling | Reuse only while fresh under both limits |
452
- | Valid `max-age`, without `s-maxage` | Same, using `max-age` | Same |
453
- | Valid `Expires`, without either age directive | Same, using the origin expiration lifetime | Same |
454
- | Invalid or ambiguous explicit expiry | Treat as stale; validate or acquire | Full acquisition |
455
- | Eligible response without explicit lifetime | Use the operator TTL as Plurnk's heuristic | Reuse inside the operator TTL |
456
-
457
- The operator ceiling is `PLURNK_SCHEMES_HTTP_TTL_MS`; `0` disables every
458
- validation-free reuse. Origin age is the greater of the response's `Age` value
459
- and apparent age from `Date`, plus residence since the authoritative package
460
- stamp. A representation is fresh only while both origin lifetime and operator
461
- ceiling permit it. `s-maxage` takes precedence even when invalid or duplicated:
462
- such a value is stale, not a reason to fall back to `max-age` or `Expires`.
463
- `private` and `no-store` prohibit reuse regardless of other permission directives.
464
- Unknown cache extensions are inert. Stale content is never served, so
465
- `must-revalidate` and `proxy-revalidate` require no separate path.
466
-
467
- Outside the fresh window, a representation without a materializer identity may
468
- send a stored ETag or Last-Modified only when that field is singular and
469
- syntactically valid. A 304 can restore those stored channels only when its
470
- validator corresponds to the nominated stored representation:
471
-
472
- | 304 validator | Correspondence requirement |
473
- | ----------------------------------------------------- | ------------------------------------------- |
474
- | Strong ETag | Same stored strong ETag |
475
- | Weak ETag | Same opaque tag under weak comparison |
476
- | No ETag; Last-Modified | Same valid stored Last-Modified instant |
477
- | Missing, malformed, or non-corresponding validator | Do not update the stored representation |
478
-
479
- Response ETags use RFC 9111 §4.3.4 cache-update correspondence, not the origin's
480
- `If-None-Match` precondition comparison. A strong response tag cannot promote a
481
- stored weak tag into byte identity. A non-corresponding 304 triggers one GET
482
- without conditional headers; a complete response is acquired normally. A
483
- second 304, or a 304 without a reusable stored representation, returns `502`
484
- (`fetch-failed`) without serving or replacing the stored body. No validators
485
- are invented for an unsolicited 304.
486
-
487
- A corresponding 304 restores the stored channels without rematerializing and
488
- updates the header under the following ownership rule; any other response
489
- replaces the channels:
490
-
491
- | 304 metadata class | Stored-header action |
492
- | --------------------------------------------------------------------- | ------------------------------------------------------- |
493
- | Present end-to-end origin fields, including cache and validators | Replace prior fields of the same case-insensitive name |
494
- | Origin fields absent from the 304 | Preserve |
495
- | `Content-Type`, `Content-Encoding`, `Content-Range`, `Content-Length` | Preserve metadata describing the already-processed body |
496
- | Package method, acquisition stamp, and variant | Rebuild authoritatively; refresh stamp and variant |
497
- | Projection evidence | Preserve |
498
-
499
- A 304-provided `Vary`, `private`, `no-store`, `no-cache`, expiry, or validator
500
- therefore governs the next operation without relabeling derived Unicode as a
501
- different source representation.
502
-
503
- A stored derived representation requires the current projection identity. A
504
- stored page body also requires its current route: negotiated origin Markdown,
505
- local projection while no materializer is selected, metadata-ineligible local
506
- projection, the configured materializer id, or the corresponding recoverable
507
- local-fallback route. Selecting or changing the materializer invalidates the
508
- affected route.
509
-
510
- Once a page representation leaves its fresh window, Plurnk performs complete
511
- reacquisition without old origin validators. Origin `304` can certify origin
512
- bytes, but it cannot certify a composite that may also contain a fresh
513
- materializer extraction or a newly negotiated Markdown representation.
514
- Projection or materializer mismatch likewise invalidates
515
- body, TTL, and validators together.
516
-
517
- POST, PUT, and DELETE responses retain their method marker but cannot satisfy a
518
- later GET or exact-FIND acquisition. An unmarked authored entry and an eligible
519
- stored GET remain visible to universal FIND as durable evidence; exact HTTP
520
- preparation applies the policy above. A metadata-less `KILL` deletes the stored entry ({§http-kill}).
44
+ | Response / surface | `body` | Auxiliary materialization (`readable` / `header`) | Completion |
45
+ |---|---|---|---|
46
+ | Negotiated origin `text/markdown` | Exact origin Markdown | None (single request; no `readable` needed) | Static representation |
47
+ | HTML page production | Exact server HTML when origin supplied it | Materializer Markdown, local Markdown (`200` or `203` fallback), or independent error in `readable` | Static representation; channels carry independent `producerResult`s |
48
+ | `text/event-stream` (SSE) | One joined `data` value + `\n` per `text/plain` chunk | Initial response metadata in `header` | `102`; origin close settles subscription |
49
+ | Empty response body | Present empty text `""` | Response and package metadata in `header` | Static representation (`READ` of `body` returns `204`) |
50
+ | Origin HTTP `4xx`/`5xx` | Available origin text or bytes preserved | Origin-backed channels carry durable `http-response-status` Problem | Selected channel's durable outcome |
51
+ | Configured textual / JSON type | Fetch `Response.text()` UTF-8 (streamed mutations use incremental `TextDecoder`; finite JSON formats via {§contract-representations}) | Response and package metadata in `header` (`charset` preserved as evidence) | Static representation |
52
+ | Configured binary type | Complete original bytes via {§mimetype-binary-input} (native attachment eligible on `READ` of `body`, `#bytes`, or `#readable`, {§log-wire-format}) | Derived Unicode in `readable` (when reader exists); `x-plurnk-projection-id` in `header` | Static representation, or `413 projection-input-limit` if over byte ceiling |
53
+ | Unreadable binary or unknown bytes | Complete original bytes (`application/octet-stream`) | No `readable` projection or native media claim | Static representation; ordinary hex `READ` |
54
+
55
+ `WebFetcher.materialize` outcomes across input and materializer states:
56
+
57
+ | Input or event | Action | Result |
58
+ |---|---|---|
59
+ | Configured non-HTML text | Decode with Fetch UTF-8 | Original representation in `body` |
60
+ | Configured binary with / without reader | Retain bounded bytes (and apply projection when present) | Original `body`, optional `readable`, and `projectionIdentity` (or `WebMaterializationError` from `ProjectionInputLimitError` when over limit) |
61
+ | Negotiated origin `text/markdown` | Accept without materializer | Exact Markdown in `body`; no `readable`, no second request |
62
+ | Eligible HTML with materializer selected | Run materializer `eligible(url)` + `extract` | Origin HTML in `body`; materializer Markdown in `readable` |
63
+ | Ineligible HTML or no materializer selected | Run installed HTML projection | Origin HTML in `body`; local Markdown in `readable` |
64
+ | Recoverable materializer failure with HTML (`502 materializer-recoverable`) | Fall back to installed HTML projection | Source in `body`; local Markdown in `readable` with durable status `203` |
65
+ | Hard materializer failure | Preserve evidence; no local recovery | `readable` errored; independently successful `body`/`header` survive |
66
+ | Materializer success after origin loss | Preserve provider text and origin failure | `readable` static; `body` errored with origin's own failure |
67
+ | Projection implementation throws | Stop and preserve cause | `WebMaterializationError` with stage `projection` |
68
+
69
+ - **Materializer extensions (`PLURNK_SCHEMES_HTTP_MATERIALIZER`):** `Http` publishes `WebFetcher` as `webMaterializer` ({§scheme-operation-dispatch}). When no `Accept` header is authored, `GET` sends `Accept: text/markdown, text/html;q=0.9, */*;q=0.1`. For a credential-free, metadata-free generic public request, `PLURNK_SCHEMES_HTTP_MATERIALIZER` selects a discovered `http-materializer` package (`plurnk: { kind: "http-materializer", materializers: [{ id, module }] }` or `plugin.json#extensions.ai.plurnk`, {§extension-discovery}). If `eligible(url)` returns an identity string, `extract` returns `success`, `recoverable` (`502 materializer-recoverable`: `The materializer could not extract <url> (<reason>); the failure is retryable.`), or `hard`.
70
+ - **Origin `llms.txt` companions (`PLURNK_SCHEMES_HTTP_LLMS_TXT`):** When `PLURNK_SCHEMES_HTTP_LLMS_TXT=1` (default), a successful generic `GET` opportunistically probes `<origin>/llms.txt` once per origin TTL window and materializes `https://<host>/llms.txt` as a discoverable workspace entry; missing/non-`2xx`/binary/failed probes are remembered quietly and never fail the initiating `READ`.
71
+
72
+ ## §http-status Status mapping and replay
73
+
74
+ | Outcome | Operation status |
75
+ |---|---|
76
+ | Finite origin `2xx`/`3xx` exact `READ` / empty text response | `200` / `204` (origin status preserved in `header`) |
77
+ | Exact `FIND` after preparation / `KILL` with no live acquisition | Universal query result / exact entry-delete result |
78
+ | Automatic `WebFetcher` returns `null` / missing response channel | `404 not-materialized` / `404 channel-not-found` |
79
+ | Selected origin-backed channel received HTTP `4xx`/`5xx` | Origin status with durable `http-response-status` Problem |
80
+ | Final status outside `200..599` | `502 invalid-response-status` with `originStatus` |
81
+ | Recoverable materializer fallback / absent local HTML projection | `203` on `#readable` / `422 no-readable-projection` |
82
+ | Binary input exceeds byte bound | `413 projection-input-limit` |
83
+ | `KILL` of live acquisition / client-cancelled finite acquisition | `200` (owner settles `499`) / `499 cancelled` |
84
+ | `SSE` after acquisition: normal / cancelled / parser or transfer failure | `102` initial; settles channel result, `499`, or `502` |
85
+ | Multi-statement HTTP edit batch / invalid target, line edit, or userinfo | `409 non-atomic-edit-batch` / `400` |
86
+ | Non-corresponding `304` or transport failure without materializer output | `502 fetch-failed` (detail bounded by `PLURNK_SCHEMES_HTTP_ERROR_DETAIL_LIMIT`) |
87
+ | Projection exception / uninterpreted `SEND` status | `500 projection-failed` / `501 send-status-unsupported` |
88
+
89
+ - **Replay:** Failed `GET`, `PUT`, and `DELETE` acquisitions are retryable (idempotent); `POST` and post-open `SSE` stream failures are non-retryable.
90
+
91
+ ## §http-host-policy Host policy and URL guard
92
+
93
+ | Surface | Contract |
94
+ |---|---|
95
+ | Configuration floor | `plurnk-schemes-http/.env.defaults` assembled via {§operator-config-precedence}; missing or invalid required values fail at readiness or owning read. |
96
+ | `PLURNK_SCHEMES_HTTP_HOSTS` | Unset or empty admits all hosts. A JSON array admits exact hostnames (`"example.com"`), wildcard subdomains (`"*.example.com"`), or none (`[]`). Disallowed direct `HTTP`/`WS` targets and redirect hops fail `403 host-not-permitted` before I/O; disallowed automatic acquisitions return `null` ({§capability-admission}). |
97
+ | `Guard.fetch` (automatic acquisition) | Solely used by `WebFetcher` for automatic byte acquisition: requires credential-free `http(s):` targets whose resolved DNS addresses are all globally reachable unicast, re-checked on every redirect hop (`301`/`302` rewrite `POST` $\to$ `GET`; `303` rewrites non-`GET`/`HEAD` $\to$ `GET`; `307`/`308` preserve method/body; cross-origin hops strip `Authorization`). Direct `HTTP` and `WebSocket` operations do not run `Guard.fetch` and permit loopback/private/link-local targets. |
521
98
 
522
- §http-kill **KILL follows the entry rule; the remote DELETE is its own spelling.** ```` ```KILL (url) ```` cancels all live acquisitions of that exact URL within the workspace — GET, SSE, or mutations — by aborting their registered controllers; each initiating operation settles itself as `499` cancelled; with nothing in flight it deletes the local stored entry so the next READ must acquire again. Only ```` ```KILL (url) [{"remote": true}] ```` sends the HTTP DELETE, and the other options in that metadata block are the request's headers. A KILL never reaches the remote by accident.
99
+ ## §revalidation Revalidation, target rewrite, and SSE
523
100
 
524
- ### §sse Server-sent events
101
+ - **GitHub raw rewrite (`host-rewrite`):** Acquisition `GET`s (direct `READ`, exact `FIND`, `WebFetcher`) rewrite `https://github.com/<owner>/<repo>/blob/<ref>/<path>` to `raw.githubusercontent.com` for byte transport while preserving the addressed GitHub URL as entry identity. `POST`, `PUT`, and `DELETE` are never retargeted.
102
+ - **Cache variants and freshness (`PLURNK_SCHEMES_HTTP_TTL_MS`):** Acquired responses append authoritative `x-plurnk-request-method`, `x-plurnk-fetched-at`, `x-plurnk-cache-variant` (`default` when no explicit request metadata and no `Vary`, else `bypass`), `x-plurnk-materializer-id`, and `x-plurnk-projection-id`. Only terminal `default`-variant `GET` responses with successful `body` and `header` and cacheable status/directives (excluding `no-store`, `private`, and `206`) are eligible for reuse:
525
103
 
526
- A direct GET whose response is `text/event-stream` feeds the bounded parser.
527
- Each event's joined `data` value plus a newline becomes one `text/plain` body
528
- chunk. Comments and `event`, `id`, and `retry` metadata are not projected.
529
- The response plus persisted header is the acquisition boundary: READ returns
530
- `102`, and parsing continues through the retained `StreamSubscription` without
531
- retaining `SchemeCtx`. Buffer exhaustion and post-acquisition transport failure
532
- settle that subscription at `502`; cancellation settles it at `499`. The
533
- handler does not reconnect; events accumulate until the origin closes or the
534
- operation is cancelled.
104
+ | Stored origin policy | Direct `READ` after acquisition | Exact `FIND` after acquisition |
105
+ |---|---|---|
106
+ | `no-store` or `private` (qualified or unqualified) | Full acquisition without stored validators | Full acquisition |
107
+ | `no-cache` (qualified or unqualified) | Validate when a stored validator exists; else acquire | Full acquisition |
108
+ | Valid `s-maxage` > `max-age` > `Expires` | Serve while fresh under both origin lifetime and `PLURNK_SCHEMES_HTTP_TTL_MS` (`s-maxage` overrides `max-age`/`Expires` even if invalid) | Reuse while fresh under both limits |
109
+ | Invalid or ambiguous explicit expiry | Treat as stale; validate or acquire | Full acquisition |
110
+ | Eligible response without explicit lifetime | Heuristic reuse within `PLURNK_SCHEMES_HTTP_TTL_MS` (`0` disables validation-free reuse) | Reuse within `PLURNK_SCHEMES_HTTP_TTL_MS` |
535
111
 
536
- ## §prefetch §7 WebFetcher
112
+ - **Conditional `304` revalidation:** Outside the fresh window, a non-materializer representation with a singular valid `ETag` or `Last-Modified` sends conditional headers. A `304` restores stored channels only when its validator corresponds per RFC 9111 §4.3.4:
537
113
 
538
- | Result | Meaning |
539
- | ------------------------------------ | ---------------------------------------------------------------------------------------- |
540
- | Origin HTML | Exact source text, MIME type, package-stamped evidence, and page-producer eligibility |
541
- | Negotiated origin Markdown | Exact body without a second HTML acquisition |
542
- | Other accepted body | One unconsumed byte stream, MIME type, package-stamped headers, and cancellation owner |
543
- | Admitted origin transport failure | Bounded origin failure plus eligibility for provider-only page production |
544
- | Materialized configured text | Complete Fetch-decoded UTF-8 body |
545
- | Materialized binary | Original bounded `body` bytes, optional `readable`, and projection identity in `header` |
546
- | Materialized HTML page | Independent source/readable outcomes and complete route/provider evidence |
547
- | Automatic top-level `null` | Refused target, non-2xx response, or unavailable response with no provider route |
548
- | Caller-cancelled acquisition | Rejects with the caller signal's exact reason |
114
+ | `304` validator / metadata class | Correspondence & stored-header action |
115
+ |---|---|
116
+ | Strong `ETag` / Weak `ETag` / `Last-Modified` (no `ETag`) | Must match stored strong `ETag`, weak `ETag` (weak comparison; strong `304` tag cannot promote stored weak tag), or exact `Last-Modified` instant; non-corresponding `304` triggers one unconditional `GET`, and a second or unsolicited `304` fails `502 fetch-failed`. |
117
+ | Present end-to-end origin fields vs absent origin fields | Present fields replace prior case-insensitive matches; absent origin fields are preserved (`Vary` on `304` downgrades variant to `bypass`). |
118
+ | `Content-Type`, `Content-Encoding`, `Content-Range`, `Content-Length`, projection evidence | Preserved from the stored representation; package method, acquisition stamp, and cache variant are rebuilt authoritatively. |
549
119
 
550
- Top-level `null` is an automatic-acquisition liveness value rather than a thrown
551
- failure. Caller cancellation is not liveness: a pre-aborted caller fails before
552
- acquisition and a caller abort wins with its exact reason. Package-owned origin
553
- timeouts are ordinary classified acquisition outcomes; materializer timeouts
554
- are the materializer's own classified outcomes. WebFetcher
555
- owns no entry identity, registry selection, selected-channel policy, or query
556
- policy; its consumers supply those boundaries and the projection capability.
120
+ - **Server-sent events (`sse`):** Direct `GET` with `text/event-stream` persists `header`, returns `102`, and streams each event's joined `data` + `\n` into `body` (`text/plain`) via `eventsource-parser` and the retained `StreamSubscription` (no reconnect; buffer/transport failure settles `502`, cancellation settles `499`).
557
121
 
558
- ## §ws §8 WebSocket
122
+ ## §prefetch WebFetcher
559
123
 
560
- <!-- map -->
561
- | Tag | Promise |
562
- | --- | --- |
563
- | {§ws-lifecycle} | Socket ownership and settlement |
564
- | {§problems-http} | Web materialization Problems. |
565
- | {§pinned-wording-http} | Pinned wording. |
566
- <!-- /map -->
124
+ `WebFetcher` implements `webMaterializer` ({§scheme-operation-dispatch}) over `Guard.fetch`, `HostPolicy`, and `MaterializerRegistry`:
567
125
 
568
- `wss` is a first-class data scheme; `ws` routes to the same handler. WebSocket
569
- is bidirectional and stateful, not an HTTP content type. It uses `messages` as a
570
- `text/plain` default channel and the same canonical network address contract
571
- {§network-address}. A frame written by EDIT or SEND is data leaving the
572
- process and proposes ({§http-outbound-proposes}); the socket open itself is an
573
- acquisition and does not.
126
+ | Result | Meaning |
127
+ |---|---|
128
+ | Origin HTML / Negotiated origin Markdown | Exact source text (negotiated Markdown skips a second HTML request), MIME type, package-stamped headers, and page-producer eligibility |
129
+ | Other accepted body | Unconsumed byte stream, MIME type, package-stamped headers, and cancellation owner |
130
+ | Admitted origin transport failure | Bounded origin failure plus eligibility for materializer-only page production |
131
+ | `materialize`: text / binary / HTML page | Complete UTF-8 `body`; bounded `body` bytes + optional `readable` + `projectionIdentity`; or independent HTML `body`/`readable` outcomes |
132
+ | Automatic top-level `null` vs caller cancellation | Refused target, non-`2xx` automatic response, or unavailable target with no materializer returns `null`; caller `AbortSignal` aborts reject with the caller's exact reason |
574
133
 
575
- ### §ws-lifecycle Socket ownership and settlement
134
+ ## §ws-lifecycle WebSocket scheme and lifecycle
576
135
 
577
136
  ```mermaid
578
137
  stateDiagram-v2
579
138
  [*] --> Idle
580
- Idle --> Claimed: claim canonical workspace address
581
- Claimed --> Connecting: seed entry, open subscription, construct socket
139
+ Idle --> Claimed: READ claims workspace + canonical URL
140
+ Claimed --> Connecting: seed entry, open StreamSubscription, construct WebSocket
582
141
  Claimed --> Idle: setup or construction failure
583
- Connecting --> Open: native open; messages active; READ 102
584
- Connecting --> Settling: pre-open close/error, KILL, cancel, activation failure, or shutdown
585
- Open --> Open: ordered inbound frame, EDIT, or SEND 200
586
- Open --> Settling: closing state, close/error, KILL, cancel, binary frame, persistence failure, or shutdown
587
- Settling --> Idle: await retained work, close subscription, release claim
142
+ Connecting --> Open: native open -> messages active -> READ returns 102
143
+ Connecting --> Settling: pre-open close/error (502 connection-failed), KILL, cancel, or shutdown
144
+ Open --> Open: ordered inbound text frame, single EDIT, or SEND 200
145
+ Open --> Settling: close/error, KILL, cancel, binary frame (415 + close 4003), persistence failure (500), or shutdown
146
+ Settling --> Idle: drain accepted frame prefix, close transport & subscription, release claim
588
147
  ```
589
148
 
590
- | Operation or event | Contract |
591
- | --------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
592
- | ```` ```READ (ws(s)://…) ```` | Claim, seed/subscribe, construct `CONNECTING`, then return `102` after native `open` plus durable activation |
593
- | Concurrent duplicate READ | Join the pending acquisition; once acquired, observe the shared connection with `200` and its connection state. Never create a second subscription or socket. |
594
- | ```` ```EDIT (ws(s)://…) ```` | Send one whole text frame only for an open owner; line ranges and multi-statement batches are rejected |
595
- | ```` ```SEND (ws(s)://…) ```` with body | Send only for owner `open` plus native `readyState=OPEN`; absent or non-open owner is `409`; send throw is `502` |
596
- | ```` ```KILL (ws(s)://…) ```` | Close/cancel the claimed owner; no owner is `404`; an attempted close throw is `502` |
597
- | Inbound frame after native `open` | Join the owner's persistence chain; the next write begins only after the preceding write succeeds |
598
- | Binary frame after native `open` | Retain the text prefix, prune the binary and later frames, settle `415 binary-frame-unsupported`, close with private-use code `4003` |
599
- | First inbound persistence failure | Retain the successful prefix, prune queued and later frames, and settle with `500 message-persistence-failed` |
600
- | Socket closes before `open` | Close subscription with `502 connection-failed`; the pending READ returns that exact failure |
601
- | Socket closes after `open` | Drain the accepted frame prefix; initial READ remains `102`; settle with `200` unless a drained write fails |
602
- | Failure after `open` | Initial READ remains `102`; persist the exact terminal failure and wake through subscription settlement |
603
-
604
- Markerless single-statement EDIT and SEND with signal `200` converge on one
605
- outbound text-frame path and have the same connection and transport outcomes.
606
- Authored dispatch order ({§op-execution-order}) lets either write follow the
607
- opening READ in the same turn, after acquisition establishes the live owner.
608
-
609
- The in-instance registry is keyed by workspace and complete canonical network
610
- URL, matching the commons entry. Workers share the connection and may SEND or
611
- KILL it; acquisition attribution and settlement remain with its initiating
612
- operation ({§runtime-resource-binding}). The claim remains registered through terminal
613
- cleanup, so a new READ cannot overlap an owner's subscription settlement. Every
614
- terminal path drains retained owner work, closes the transport when necessary,
615
- closes the durable subscription, then releases the claim. A persistence failure
616
- found while draining supersedes a graceful terminal result; cancellation and
617
- transport failures retain their exact result. Handler shutdown requests
618
- settlement for every remainder, awaits every owner, and aggregates
619
- transport-close failures under {§handler-lifecycle}.
620
-
621
- | Transport limit | Current contract |
622
- | ------------------ | ------------------------------------------------------------------------------------------------------- |
623
- | Payload projection | String event data only into `text/plain`; binary data terminates with Plurnk `415` and private-use WebSocket code `4003` |
624
- | Reconnection | None; READ again after terminal cleanup |
625
- | Handshake metadata | `[metadata]` is unsupported; the default global-WebSocket identity is used |
626
- | Runtime | Node ≥26 |
627
-
628
- §problems-http **Web materialization Problems.** Every code minted here, its status, and the sentence that is its contract (placeholders in *italics* are filled at emission; a fixed recovery follows its detail).
629
-
630
- | code | status | contract |
631
- |---|---:|---|
632
- | `materializer-recoverable` | 502 | The materializer could not extract *url* (*reason*); the failure is retryable. |
633
-
634
- §pinned-wording-http **Pinned wording.** Verbatim sentences tests pin: each is contract, and a change here is a change of contract.
149
+ `Ws` registers as `wss` (`ws` routes to it) with default `text/plain` channel `messages` and canonical `NetworkAddress` identity (no `[metadata]` support; Node $\ge 26$):
635
150
 
636
- | sentence | arises when |
151
+ | Operation / event | Contract |
637
152
  |---|---|
638
- | Remove the line range and submit the complete replacement body. | an HTTP EDIT that carries a line range |
639
- | READ the WebSocket URL before sending a message. | a SEND to a WebSocket that was never opened |
640
- | The WebSocket message could not be sent. / The WebSocket connection could not be closed. | socket send or close failures |
153
+ | `READ (ws(s)://...)` | Claims workspace/URL coordinate, seeds entry and `StreamSubscription`, opens native `WebSocket`, and returns `102` on `open`. Concurrent duplicate `READ`s join the pending open and return `200` with connection state without opening a second socket. |
154
+ | `EDIT (ws(s)://...)` / `SEND (ws(s)://...)` | Single-statement markerless `EDIT` and `SEND` (`200`) propose (`202`, `effect: "host"`) and send one text frame when the owner is `open` (`readyState === OPEN`). Absent/non-open owner fails `409` (`READ the WebSocket URL before sending a message.`); send throw fails `502` (`The WebSocket message could not be sent.`); line ranges and multi-statement batches fail `400`/`409`. |
155
+ | `KILL (ws(s)://...)` | Closes/cancels the claimed owner (`404` when no live owner exists; close throw fails `502`: `The WebSocket connection could not be closed.`). |
156
+ | Inbound frames & settlement | Text frames chain sequentially onto `messages` after `open`. A binary frame retains the text prefix, prunes later frames, closes with WebSocket code `4003`, and settles `415 binary-frame-unsupported`. First persistence failure prunes later frames and settles `500 message-persistence-failed`. Pre-open close settles `502 connection-failed`; post-open close drains accepted frames and settles `200` ({§execution-output-identity}, {§scheme-operation-dispatch}). |