@plurnk/plurnk-schemes-http 1.18.0 → 1.19.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.defaults +10 -0
- package/README.md +31 -12
- package/SPEC.md +135 -72
- package/dist/Config.d.ts +2 -0
- package/dist/Config.d.ts.map +1 -1
- package/dist/Config.js +15 -0
- package/dist/Config.js.map +1 -1
- package/dist/Guard.d.ts.map +1 -1
- package/dist/Guard.js +3 -1
- package/dist/Guard.js.map +1 -1
- package/dist/HostPolicy.d.ts +11 -0
- package/dist/HostPolicy.d.ts.map +1 -0
- package/dist/HostPolicy.js +58 -0
- package/dist/HostPolicy.js.map +1 -0
- package/dist/Http.d.ts +6 -4
- package/dist/Http.d.ts.map +1 -1
- package/dist/Http.js +74 -26
- package/dist/Http.js.map +1 -1
- package/dist/HttpGet.d.ts.map +1 -1
- package/dist/HttpGet.js +81 -113
- package/dist/HttpGet.js.map +1 -1
- package/dist/HttpRequester.d.ts +2 -2
- package/dist/HttpRequester.d.ts.map +1 -1
- package/dist/HttpRequester.js +33 -27
- package/dist/HttpRequester.js.map +1 -1
- package/dist/WebFetcher.d.ts +7 -10
- package/dist/WebFetcher.d.ts.map +1 -1
- package/dist/WebFetcher.js +68 -91
- package/dist/WebFetcher.js.map +1 -1
- package/dist/Ws.d.ts +4 -3
- package/dist/Ws.d.ts.map +1 -1
- package/dist/Ws.js +66 -19
- package/dist/Ws.js.map +1 -1
- package/docs/https.md +27 -22
- package/docs/wss.md +3 -3
- package/package.json +4 -4
package/.env.defaults
CHANGED
|
@@ -2,15 +2,25 @@
|
|
|
2
2
|
|
|
3
3
|
# Request deadline, ms.
|
|
4
4
|
PLURNK_SCHEMES_HTTP_FETCH_TIMEOUT=30000
|
|
5
|
+
# User-Agent sent when a request names none. Plurnk says what it is: an agent, not a browser.
|
|
6
|
+
# An origin that serves only browsers may refuse it — state your own identity here if you need one,
|
|
7
|
+
# and own that choice.
|
|
8
|
+
PLURNK_SCHEMES_HTTP_USER_AGENT="PlurnkBot/1.0 (+https://plurnk.xyz/bot)"
|
|
5
9
|
# Stored URL freshness, ms; within the TTL, READ performs no request.
|
|
6
10
|
# 0 = revalidate on every READ (ETag/304 when available).
|
|
7
11
|
PLURNK_SCHEMES_HTTP_TTL_MS=300000
|
|
12
|
+
# After a successful GET, also fetch the origin's /llms.txt once and keep it as its own entry:
|
|
13
|
+
# 1 = yes; 0 = make only the requests a READ asks for.
|
|
14
|
+
PLURNK_SCHEMES_HTTP_LLMS_TXT=1
|
|
8
15
|
# Maximum automatic redirect hops; 0 = none.
|
|
9
16
|
PLURNK_SCHEMES_HTTP_REDIRECTS=5
|
|
10
17
|
# Maximum buffered characters per incomplete SSE line/event; overflow terminates the stream.
|
|
11
18
|
PLURNK_SCHEMES_HTTP_SSE_MAX_BUFFER_CHARS=1048576
|
|
12
19
|
# Maximum diagnostic characters in model-facing errors; daemon diagnostics remain complete.
|
|
13
20
|
PLURNK_SCHEMES_HTTP_ERROR_DETAIL_LIMIT=512
|
|
21
|
+
# Web host policy: unset/empty = every host; a JSON array admits only its hosts
|
|
22
|
+
# ("example.com", "*.example.com"); [] = no web access at all (e.g. an isolated benchmark).
|
|
23
|
+
# PLURNK_SCHEMES_HTTP_HOSTS=[]
|
|
14
24
|
|
|
15
25
|
# Optional HTML materializer; unset = installed local HTML projection.
|
|
16
26
|
# @plurnk/plurnk-schemes-http-tavily ships with the service; it needs TAVILY_API_KEY in the
|
package/README.md
CHANGED
|
@@ -10,9 +10,9 @@ authored against the DB-free
|
|
|
10
10
|
|
|
11
11
|
| Operation | Behavior |
|
|
12
12
|
| ---------------------------------------------- | ----------------------------------------------------------------------------------------- |
|
|
13
|
-
| ```` ```READ (http(s)://…) ```` | Fetch or reuse/revalidate a GET representation, then
|
|
14
|
-
| ```` ```READ (http(s)://…) <scope> ```` | Apply
|
|
15
|
-
| ```` ```FIND (http(s)://…)
|
|
13
|
+
| ```` ```READ (http(s)://…) ```` | Fetch or reuse/revalidate a GET representation, then project the selected response channel |
|
|
14
|
+
| ```` ```READ (http(s)://…) <scope> ```` | Apply standard entry scope after the same acquisition/revalidation; scope does not bypass it |
|
|
15
|
+
| ```` ```FIND (http(s)://…) [{"pattern": "…"}] ```` | Materialize an exact URL when required, then use the universal entry query and matcher |
|
|
16
16
|
| ```` ```SEND (http(s)://…) ```` with body | POST the body and stream the response |
|
|
17
17
|
| ```` ```EDIT (http(s)://…) ```` with body | PUT a whole-resource replacement; line-scoped HTTP edits are invalid |
|
|
18
18
|
| ```` ```KILL (http(s)://…) ```` | Cancel a live acquisition of the address, or forget its stored response |
|
|
@@ -22,22 +22,28 @@ A path-pattern FIND surveys already-materialized web entries; it does not crawl
|
|
|
22
22
|
or discover the remote web. Exact matcher FIND shares the standard flat
|
|
23
23
|
location, weighting, pagination, and status contract.
|
|
24
24
|
|
|
25
|
+
The verbs that change a remote resource — POST, PUT, the remote DELETE, and a
|
|
26
|
+
WebSocket frame (EDIT or SEND on an open `ws(s)://` socket) — propose before
|
|
27
|
+
anything leaves the process, the same way a subprocess does;
|
|
28
|
+
`PLURNK_SERVICE_EFFECT_HOST` decides whether that settles by consent or runs
|
|
29
|
+
unattended ({§http-outbound-proposes}). GET and the socket open are observation
|
|
30
|
+
and are ungated.
|
|
31
|
+
|
|
25
32
|
## HTTP channels
|
|
26
33
|
|
|
27
34
|
| Channel | Content |
|
|
28
35
|
| -------- | -------------------------------------------------------------------------- |
|
|
29
|
-
| `body` |
|
|
36
|
+
| `body` | Original text or binary response; SSE data |
|
|
30
37
|
| `header` | Origin, acquisition, materializer, projection, provider, and usage evidence |
|
|
31
|
-
| `
|
|
38
|
+
| `readable` | Curated page Markdown or binary facts/text when a reader supplies them |
|
|
32
39
|
|
|
33
40
|
A fragmentless operation publishes only `body`; auxiliary channels remain
|
|
34
41
|
durable and can be addressed explicitly. Remote HTTP status is stored in
|
|
35
|
-
`header`; the PLURNK operation result reports the
|
|
36
|
-
Binary responses
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
projection ceiling returns `413`.
|
|
42
|
+
`header`; the PLURNK operation result reports the selected channel's outcome.
|
|
43
|
+
Binary responses retain their bounded original bytes. Ordinary READ returns
|
|
44
|
+
hex and, on a supporting model, attaches native media; `#readable` selects
|
|
45
|
+
derived facts/text with the same native source. Unknown formats remain
|
|
46
|
+
byte-readable. Input above the common binary ceiling returns `413`.
|
|
41
47
|
|
|
42
48
|
## Design
|
|
43
49
|
|
|
@@ -45,7 +51,7 @@ projection ceiling returns `413`.
|
|
|
45
51
|
WebSocket retain their explicit-target authority.
|
|
46
52
|
- Generic GETs negotiate origin Markdown first. When the origin returns HTML,
|
|
47
53
|
a selected materializer plugin ({§http-materializer-plugins}) produces
|
|
48
|
-
`
|
|
54
|
+
`readable`; otherwise the installed HTML reader is the local route. Recoverable
|
|
49
55
|
materializer failures use that same reader as a `203` recovery floor. Hard
|
|
50
56
|
provider failures do not silently change producers.
|
|
51
57
|
- `body`, `header`, and `readable` settle independently. The selected channel
|
|
@@ -90,3 +96,16 @@ Without a selection, `@plurnk/plurnk-mimetypes-text-html` supplies the local
|
|
|
90
96
|
```sh
|
|
91
97
|
npm test
|
|
92
98
|
```
|
|
99
|
+
|
|
100
|
+
The optional HTTP cache survey
|
|
101
|
+
uses the unmodified [http-tests corpus](https://github.com/http-tests/cache-tests).
|
|
102
|
+
It is a diagnostic, not a conformance score or part of the test gate.
|
|
103
|
+
|
|
104
|
+
| Boundary | Current behavior |
|
|
105
|
+
| --- | --- |
|
|
106
|
+
| Origin lifetime | `s-maxage` overrides `max-age`, then `Expires`; origin age and operator TTL also constrain reuse. |
|
|
107
|
+
| Heuristic reuse | Requires an eligible status or explicit permission; errors and partial `206` responses are reacquired. |
|
|
108
|
+
| Request headers / `Vary` | Bypass reuse; there is no variant cache. |
|
|
109
|
+
| Stale data | Never substituted for failed acquisition; no stale-while-revalidate behavior. |
|
|
110
|
+
| Response evidence | Stored headers are not a forwarded HTTP response; no generated `Age` or removed hop-by-hop evidence. |
|
|
111
|
+
| Shared-cache directives | `private` and `no-store` prohibit reuse, not retention of acquisition evidence or workspace access. |
|
package/SPEC.md
CHANGED
|
@@ -20,9 +20,9 @@ the DB-free `SchemeCtx` author contract.
|
|
|
20
20
|
|
|
21
21
|
| Channel | Seed type | Meaning |
|
|
22
22
|
| -------- | --------------------------- | ------------------------------------------------------------------------------ |
|
|
23
|
-
| `body` | `application/octet-stream` |
|
|
23
|
+
| `body` | `application/octet-stream` | Source text or original binary bytes; SSE data for a live response |
|
|
24
24
|
| `header` | `text/plain` | Origin, acquisition, materializer, projection, provider, and usage evidence |
|
|
25
|
-
| `readable` | `text/markdown` |
|
|
25
|
+
| `readable` | `text/markdown` | Optional derived text/facts: an HTML materializer or installed mimetype projection ({§readable-channel}) |
|
|
26
26
|
|
|
27
27
|
`package.json#plurnk.schemes` registers `http` through the default export and
|
|
28
28
|
`wss` through `Ws`.
|
|
@@ -47,18 +47,35 @@ preserved exactly.
|
|
|
47
47
|
| ```` ```SEND (url) ```` with body | POST | Stream and persist the response under the addressed URL |
|
|
48
48
|
| ```` ```EDIT (url) ```` with body | PUT | Replace the whole remote resource; a line marker is invalid |
|
|
49
49
|
| ```` ```KILL (url) ```` | None | {§http-kill}: cancel the workspace's live acquisitions of the URL, else delete the local stored entry |
|
|
50
|
-
| ```` ```KILL (url) [{"remote": true}] ```` | DELETE | Delete the remote resource and stream its response; other metadata
|
|
50
|
+
| ```` ```KILL (url) [{"remote": true}] ```` | DELETE | Delete the remote resource and stream its response; other options in the same metadata block are its headers |
|
|
51
51
|
|
|
52
52
|
Finite GET uses scope-blind representation preparation; POST, PUT, DELETE,
|
|
53
53
|
and genuinely live GET responses retain the subscription path. Request
|
|
54
|
-
headers are
|
|
55
|
-
|
|
56
|
-
pure.
|
|
57
|
-
|
|
54
|
+
headers are options in one `[{"Key": "value", ...}]` metadata block under
|
|
55
|
+
{§scheme-metadata-modifier}. HTTP interprets the merged options; the resource
|
|
56
|
+
target remains pure. Remote status and headers are persisted in `header`, not
|
|
57
|
+
interpreted as Plurnk operation or loop lifecycle signals.
|
|
58
58
|
Exact-versus-pattern FIND preparation uses the shared
|
|
59
59
|
`PathSyntax.hasGlob` classifier {§path-glob}; HTTP owns no reduced
|
|
60
60
|
path-pattern grammar.
|
|
61
61
|
|
|
62
|
+
§http-outbound-proposes **A request that changes a remote resource proposes.**
|
|
63
|
+
POST, PUT, the remote DELETE, and a WebSocket frame (EDIT or SEND on an open
|
|
64
|
+
`ws(s)://` socket) declare `effect: "host"` and return `202` under
|
|
65
|
+
{§proposal}; the panel's `PLURNK_SERVICE_EFFECT_HOST` decides whether that
|
|
66
|
+
settles by consent or runs unattended, exactly as it does for a subprocess
|
|
67
|
+
({§exec-host-proposes}). GET is observation and stays ungated, and so is the
|
|
68
|
+
socket open a `ws(s)://` READ performs: an acquisition leaks only what a URL
|
|
69
|
+
can carry, and that is the same for both. A frame needs an open socket, so that
|
|
70
|
+
refusal arrives before any proposal — nobody is asked to approve a message to a
|
|
71
|
+
connection that is not there — and the admission runs again at apply, because
|
|
72
|
+
the socket is live and may have closed while the question was open. The
|
|
73
|
+
proposal carries the method, target, and body, and the resolver may replace
|
|
74
|
+
the body before it is sent; nothing leaves the process until the settlement
|
|
75
|
+
accepts. This is the same reason a local `EDIT` proposes: Plurnk does not
|
|
76
|
+
claim containment, so consent — not a boundary — is what stands between the
|
|
77
|
+
model and the world.
|
|
78
|
+
|
|
62
79
|
## §http-lifecycle §3 Acquisition, materialization, and query lifecycle
|
|
63
80
|
|
|
64
81
|
```mermaid
|
|
@@ -66,24 +83,26 @@ flowchart TD
|
|
|
66
83
|
get["GET acquisition"] --> live{"text/event-stream?"}
|
|
67
84
|
live -->|yes| subscription["Seed canonical channels<br/>retain subscription; return 102"]
|
|
68
85
|
live -->|no| origin{"Finite origin outcome"}
|
|
69
|
-
origin -->|text
|
|
70
|
-
origin -->|
|
|
71
|
-
origin -->|
|
|
86
|
+
origin -->|text, including Markdown| text["Source text in body"]
|
|
87
|
+
origin -->|binary| binary["Bounded original bytes in body<br/>optional projection in readable"]
|
|
88
|
+
origin -->|server HTML| source["Retain exact server HTML in body"]
|
|
89
|
+
origin -->|transport unavailable<br/>after public admission| noSource["Preserve source failure in body"]
|
|
72
90
|
source --> eligible{"Generic public request<br/>with a materializer selected?"}
|
|
73
91
|
noSource --> eligible
|
|
74
92
|
eligible -->|no| local["Installed HTML projection"]
|
|
75
93
|
eligible -->|yes| mat["Materializer Markdown"]
|
|
76
|
-
mat -->|success|
|
|
77
|
-
mat -->|recoverable failure<br/>and source exists| recovery["Local
|
|
78
|
-
mat -->|hard failure or<br/>no recovery source|
|
|
79
|
-
local -->|present, including empty|
|
|
80
|
-
local -->|absent or no source|
|
|
81
|
-
|
|
82
|
-
|
|
94
|
+
mat -->|success| providerText["Materializer readable"]
|
|
95
|
+
mat -->|recoverable failure<br/>and source exists| recovery["Local readable; status 203"]
|
|
96
|
+
mat -->|hard failure or<br/>no recovery source| readableError["Readable errored"]
|
|
97
|
+
local -->|present, including empty| localText["Local readable"]
|
|
98
|
+
local -->|absent or no source| readableError
|
|
99
|
+
text --> settle["Write complete canonical channels<br/>and durable producer results"]
|
|
100
|
+
binary --> settle
|
|
101
|
+
providerText --> settle
|
|
83
102
|
recovery --> settle
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
settle --> core["Core selects channel<br/>
|
|
103
|
+
localText --> settle
|
|
104
|
+
readableError --> settle
|
|
105
|
+
settle --> core["Core selects channel and scope<br/>native media follows ordinary READ"]
|
|
87
106
|
```
|
|
88
107
|
|
|
89
108
|
The same page producer serves exact GET preparation, exact FIND, and executor
|
|
@@ -99,16 +118,16 @@ but cold and warm representations pass through the same core projection.
|
|
|
99
118
|
| HTML-page production | Exact server source when the origin supplied it | Materializer Markdown, local Markdown, or independent error in `readable` | Static representation, then core projection |
|
|
100
119
|
| `text/event-stream` | One `data` value plus newline per `text/plain` chunk | Initial response in `header` | `102`; origin close settles subscription |
|
|
101
120
|
| No response body | Present empty text | Response and package metadata in `header` | Static representation; body READ is `204` |
|
|
102
|
-
| Origin HTTP `4xx`/`5xx` | Preserve available origin or
|
|
121
|
+
| Origin HTTP `4xx`/`5xx` | Preserve available origin text or bytes | Exact response evidence; origin-backed channels errored | Selected channel's durable outcome |
|
|
103
122
|
| Configured textual type | Complete Fetch-decoded Unicode | Response and package metadata in `header` | Static representation |
|
|
104
|
-
| Readable binary type |
|
|
105
|
-
| Unreadable binary or unknown bytes | No
|
|
123
|
+
| Readable binary type | Complete original bytes | Derived Unicode in `readable`; projection identity in `header` | Static representation, then core projection |
|
|
124
|
+
| Unreadable binary or unknown bytes | Complete original bytes | No readable projection or native media claim | Static representation; ordinary hex READ |
|
|
106
125
|
| Binary input exceeds configured bound | No representation is fabricated | Exact size evidence in the Problem | `413 projection-input-limit` |
|
|
107
126
|
|
|
108
127
|
The producer persists every available channel; core publishes only the channel
|
|
109
128
|
selected by the fragment, or `body` by default. Broad FIND returns standard
|
|
110
129
|
resource metadata; exact matcher FIND returns flat match locations. Exact READ
|
|
111
|
-
returns the selected channel's requested text projection.
|
|
130
|
+
returns the selected channel's requested text/hex projection.
|
|
112
131
|
|
|
113
132
|
### §http-text-decoding Text response decoding
|
|
114
133
|
|
|
@@ -125,8 +144,16 @@ returns the selected channel's requested text projection.
|
|
|
125
144
|
|
|
126
145
|
This decoder boundary remains text normalization, not a media-format processor.
|
|
127
146
|
A configured binary type bypasses it and enters the mimetype family's bounded
|
|
128
|
-
|
|
129
|
-
|
|
147
|
+
source acquisition {§mimetype-binary-input}; original bytes remain in `body`.
|
|
148
|
+
|
|
149
|
+
§http-binary-source Finite binary responses retain bounded original bytes in `body`,
|
|
150
|
+
with optional derived facts/text in `readable`. GET, exact FIND acquisition, executor
|
|
151
|
+
acquisition, and mutation responses use the same source/projection contract.
|
|
152
|
+
Core READ of `body`, `#bytes`, or `#readable` may attach that complete native source
|
|
153
|
+
under {§packet-attachment-parts}; scope selects the text/hex view, never native bytes.
|
|
154
|
+
Headers carry transport evidence, not native media. Invalid or unsupported formats
|
|
155
|
+
remain byte-readable without invented facts or native parts. Incomplete transfers
|
|
156
|
+
and over-limit input publish no partial binary payload.
|
|
130
157
|
|
|
131
158
|
§http-json-presentation Finite `application/json` and `+json` responses use
|
|
132
159
|
{§json-document-presentation} in their canonical body channel before indexing,
|
|
@@ -137,16 +164,16 @@ entries, non-JSON text, SSE, and JSONL retain their original formatting.
|
|
|
137
164
|
|
|
138
165
|
### §html-materialization Readable materialization
|
|
139
166
|
|
|
140
|
-
`WebFetcher.materialize` is the shared
|
|
167
|
+
`WebFetcher.materialize` is the shared source/projection seam for exact
|
|
141
168
|
GET/FIND preparation and executor entry acquisition. It returns complete
|
|
142
|
-
|
|
143
|
-
mutation responses retain incremental streaming.
|
|
169
|
+
source and derived channels for one atomic canonical entry write; SSE and
|
|
170
|
+
textual mutation responses retain incremental streaming.
|
|
144
171
|
|
|
145
172
|
| Input or event | Action | Result |
|
|
146
173
|
| ------------------------------------------- | ------------------------------------------- | ------------------------------------------------------------------ |
|
|
147
174
|
| Configured non-HTML text | Decode with Fetch UTF-8 | Original representation in `body` |
|
|
148
|
-
| Configured binary with reader |
|
|
149
|
-
| Configured binary without reader |
|
|
175
|
+
| Configured binary with reader | Retain bounded bytes and apply projection | Original `body`, derived `readable`, and projection identity |
|
|
176
|
+
| Configured binary without reader | Retain bounded bytes | Original `body` and projection identity; no `readable` |
|
|
150
177
|
| Binary input exceeds the common bound | Cancel and preserve the typed cause | `WebMaterializationError` caused by `ProjectionInputLimitError` |
|
|
151
178
|
| Negotiated origin `text/markdown` | Accept it without the materializer | Exact Markdown in `body`; no `readable`, no second request |
|
|
152
179
|
| Eligible HTML with a materializer selected | Use the materializer as the readable producer | Origin server source in `body`; materializer Markdown in `readable` |
|
|
@@ -163,7 +190,7 @@ never enters the absence channel.
|
|
|
163
190
|
#### §http-materializer-plugins Materializer plugins
|
|
164
191
|
|
|
165
192
|
`PLURNK_SCHEMES_HTTP_MATERIALIZER` selects one discovered `http-materializer`
|
|
166
|
-
plugin by id; unset means the installed HTML projection is the only
|
|
193
|
+
plugin by id; unset means the installed HTML projection is the only readable
|
|
167
194
|
producer. A materializer package declares
|
|
168
195
|
`plurnk: { kind: "http-materializer", materializers: [{ id, module }] }` and
|
|
169
196
|
exports one `HttpMaterializer` per entry under the executor family's
|
|
@@ -177,8 +204,8 @@ origin credentials cross the materializer boundary.
|
|
|
177
204
|
|
|
178
205
|
When no `Accept` is authored, origin acquisition offers
|
|
179
206
|
`text/markdown, text/html;q=0.9, */*;q=0.1`. An origin `text/markdown`
|
|
180
|
-
representation wins and the materializer is skipped;
|
|
181
|
-
|
|
207
|
+
representation wins and the materializer is skipped; no second HTML request is
|
|
208
|
+
made. An authored `Accept` value is
|
|
182
209
|
sent unchanged and makes any returned HTML use the local projection route.
|
|
183
210
|
|
|
184
211
|
The materializer's `eligible(url)` returns its identity string when it will
|
|
@@ -210,10 +237,15 @@ shows that `#readable` exists before the model has ever listed the page.
|
|
|
210
237
|
Exact READ, exact FIND, and executor materialization preserve the same channel
|
|
211
238
|
representation. A missing source variant is an explicit empty `errored`
|
|
212
239
|
channel, never an absent fact that later cache use can reinterpret as successful
|
|
213
|
-
empty content. A
|
|
214
|
-
|
|
215
|
-
`
|
|
216
|
-
|
|
240
|
+
empty content. A finite origin `2xx` or `3xx` response completes ordinary local
|
|
241
|
+
production, independently of the origin's status meaning; `202` does not leave
|
|
242
|
+
an acquisition pending, and origin `203` is not a materializer recovery. Origin
|
|
243
|
+
`4xx`/`5xx` content remains available as evidence while origin-backed channels
|
|
244
|
+
carry the exact `http-response-status` producer Problem. A final status outside
|
|
245
|
+
`200`–`599` instead produces `502 invalid-response-status` with `originStatus`,
|
|
246
|
+
preserving the received content and header. A materializer-produced `readable`
|
|
247
|
+
and the acquisition `header` remain independent of an unavailable origin source.
|
|
248
|
+
Conditional `304` correspondence is checked before production under {§revalidation}.
|
|
217
249
|
|
|
218
250
|
§http-llms-txt **Origin llms.txt companions.** After a successful generic GET
|
|
219
251
|
materialization, the scheme opportunistically acquires `<origin>/llms.txt`
|
|
@@ -223,22 +255,23 @@ non-2xx, binary content, or a transport failure is quiet: the companion is
|
|
|
223
255
|
never fabricated, the failed probe is remembered (not retried per READ), and
|
|
224
256
|
the piggyback never fails the READ that carried it. The companion itself
|
|
225
257
|
never recurses. This is lazy by construction — one extra request only when
|
|
226
|
-
an origin is first being read
|
|
258
|
+
an origin is first being read — and it is the operator's to allow:
|
|
259
|
+
`PLURNK_SCHEMES_HTTP_LLMS_TXT=0` makes only the requests a READ asks for.
|
|
227
260
|
|
|
228
261
|
## §http-status §4 HTTP status mapping
|
|
229
262
|
|
|
230
263
|
| Outcome | Operation status |
|
|
231
264
|
| ------------------------------------------------------------- | -------------------------------------------------------- |
|
|
232
|
-
| Finite exact READ after preparation
|
|
265
|
+
| Finite origin `2xx`/`3xx` exact READ after preparation | Universal selected-channel result (`200` or `204`); origin status remains in `header` |
|
|
233
266
|
| Exact FIND after preparation | Exact universal query result |
|
|
234
267
|
| Exact acquisition returns no WebFetcher value | `404` (`not-materialized`) |
|
|
235
268
|
| Selected origin-backed channel received HTTP `4xx`/`5xx` | Exact durable `http-response-status` Problem |
|
|
269
|
+
| Selected origin-backed channel received an invalid final status | `502` (`invalid-response-status`) with `originStatus` |
|
|
236
270
|
| Selected HTML-page channel fails | That channel's exact durable producer Problem |
|
|
237
271
|
| Local HTML projection is absent | `422` (`no-readable-projection`) |
|
|
238
|
-
| Recoverable materializer failure uses local projection |
|
|
239
|
-
| Finite textual or
|
|
272
|
+
| Recoverable materializer failure uses local projection | `readable` with durable status `203` |
|
|
273
|
+
| Finite textual or binary response | Universal READ result |
|
|
240
274
|
| Finite empty text response | `204` |
|
|
241
|
-
| Finite binary response has no readable projection | `415` (`binary-response-unsupported`) |
|
|
242
275
|
| Binary projection input exceeds the configured byte bound | `413` (`projection-input-limit`) |
|
|
243
276
|
| `KILL` with no live acquisition | Exact entry-delete result |
|
|
244
277
|
| `KILL` of a live acquisition | `200`; the owner settles `499` through its aborted signal |
|
|
@@ -250,7 +283,7 @@ an origin is first being read.
|
|
|
250
283
|
| Invalid target, line edit, or URL userinfo | `400` with the corresponding stable Problem kind |
|
|
251
284
|
| Missing response channel | `404 channel-not-found` under {§channel-selection-missing}; no remote mutation |
|
|
252
285
|
| Non-corresponding 304 | `502` (`fetch-failed`) |
|
|
253
|
-
| Acquisition failure without
|
|
286
|
+
| Acquisition failure without successful materializer output | `502` (`fetch-failed`) |
|
|
254
287
|
| Projection exception | `500` (`projection-failed`) |
|
|
255
288
|
| Uninterpreted SEND status | `501` (`send-status-unsupported`) |
|
|
256
289
|
|
|
@@ -270,15 +303,12 @@ applied its effect when acquisition fails and is therefore non-retryable. Once
|
|
|
270
303
|
an SSE response has been acquired, a parser or transfer failure is likewise
|
|
271
304
|
non-retryable because replay could duplicate an already-consumed prefix.
|
|
272
305
|
|
|
273
|
-
|
|
274
|
-
|
|
275
|
-
|
|
276
|
-
|
|
277
|
-
without
|
|
278
|
-
|
|
279
|
-
owns a seeded lifecycle entry and settles its typed channel as errored instead.
|
|
280
|
-
These statuses describe Plurnk's materialization boundary, not the remote HTTP
|
|
281
|
-
outcome—a POST, PUT, or DELETE might already have changed the remote resource.
|
|
306
|
+
Binary responses follow {§http-binary-source}, appending authoritative
|
|
307
|
+
`x-plurnk-projection-id` evidence after the origin and acquisition fields.
|
|
308
|
+
Exceeding the input ceiling returns `413` with configured and observed sizes;
|
|
309
|
+
a mutation response retains its lifecycle/header evidence and settles errored
|
|
310
|
+
without publishing partial bytes. This is a materialization failure, not the
|
|
311
|
+
remote HTTP outcome—a POST, PUT, or DELETE might already have changed the resource.
|
|
282
312
|
Missing or malformed `Content-Type` becomes `application/octet-stream`; the
|
|
283
313
|
handler does not sniff or guess unknown bytes.
|
|
284
314
|
|
|
@@ -298,6 +328,19 @@ handler does not sniff or guess unknown bytes.
|
|
|
298
328
|
| Required values | Missing or invalid required values fail at readiness/owning read; code carries no hidden fallback |
|
|
299
329
|
| Materializer selection | `PLURNK_SCHEMES_HTTP_MATERIALIZER=` names a discovered materializer id; unset = local projection only |
|
|
300
330
|
|
|
331
|
+
### §http-host-policy Operator web host policy
|
|
332
|
+
|
|
333
|
+
`PLURNK_SCHEMES_HTTP_HOSTS` confines every web acquisition to named hosts. Unset
|
|
334
|
+
or empty admits every host, as before. A JSON array admits exactly its members:
|
|
335
|
+
`"example.com"` names that host, `"*.example.com"` any subdomain of it, and `[]`
|
|
336
|
+
names none. A value that is not a JSON array of host names fails at first use.
|
|
337
|
+
Direct HTTP operations and WebSocket connections are refused with 403
|
|
338
|
+
`host-not-permitted` before any I/O, and a followed redirect that leaves the
|
|
339
|
+
admitted hosts is cancelled with the same refusal. Automatic acquisition checks
|
|
340
|
+
the target and every redirect hop, and a final URL outside the policy is the
|
|
341
|
+
ordinary unavailable `null`. The policy is operator configuration, alias-free and
|
|
342
|
+
not model teaching; an isolated benchmark sets `[]`.
|
|
343
|
+
|
|
301
344
|
### §automatic-fetch-check Automatic acquisition URL check
|
|
302
345
|
|
|
303
346
|
`WebFetcher` is the sole caller of `Guard.fetch`. Before automatic byte
|
|
@@ -324,9 +367,9 @@ check therefore makes no DNS-rebinding or total-egress claim.
|
|
|
324
367
|
| -------------- | ---------------------------------------------------------------------------------------------------------- |
|
|
325
368
|
| Direct gate | Explicit targets use native origin transport; only a generic public request grants materializer authority |
|
|
326
369
|
| Automatic gate | Target and redirects require public admission; accepted generic HTML follows the same page producer |
|
|
327
|
-
|
|
|
328
|
-
| Source owner | `
|
|
329
|
-
| Projection | A present projection, including `""`, becomes `
|
|
370
|
+
| Readable owner | Origin Markdown needs no projection; HTML uses the eligible materializer or installed local reader, binary uses its mimetype handler |
|
|
371
|
+
| Source owner | `body` preserves origin text or bytes; it is never replaced by a derived projection |
|
|
372
|
+
| Projection | A present projection, including `""`, becomes `readable`; `null` alone means absence |
|
|
330
373
|
| Cancellation | One caller signal spans origin, auxiliary origin, projection, and materializer work |
|
|
331
374
|
|
|
332
375
|
### §host-rewrite Acquisition target rewrite
|
|
@@ -367,22 +410,33 @@ channel lifecycle, not body length: `body` and `header` must be successful
|
|
|
367
410
|
may be `errored`; `active` or unknown state is ineligible. Durable operation
|
|
368
411
|
evidence and HTTP reuse eligibility remain distinct: an acquired response stays
|
|
369
412
|
in the entry even when its origin policy prevents later cache use.
|
|
413
|
+
Workspace reuse follows shared-cache rules (RFC 9111); it does not change
|
|
414
|
+
retained evidence or workspace access. Reuse also requires a heuristically
|
|
415
|
+
cacheable origin status (RFC 9110 §15.1) or explicit cache permission (`public`,
|
|
416
|
+
`s-maxage`, `max-age`, or `Expires`).
|
|
417
|
+
Partial `206` responses remain evidence only: this cache does not combine or
|
|
418
|
+
select byte ranges. Ineligible responses supply neither cached content nor
|
|
419
|
+
validators to a later READ or exact FIND.
|
|
370
420
|
|
|
371
421
|
| Stored origin policy | Direct READ after acquisition | Exact FIND after acquisition |
|
|
372
422
|
| ------------------------------------- | ------------------------------------------------------ | ---------------------------------------- |
|
|
373
|
-
| `no-store`
|
|
423
|
+
| `no-store` or `private` (qualified or unqualified) | Full acquisition without stored validators | Full acquisition |
|
|
374
424
|
| `no-cache` (qualified or unqualified) | Validate when a stored validator exists; else acquire | Full acquisition |
|
|
375
|
-
| Valid `
|
|
376
|
-
| Valid `
|
|
425
|
+
| Valid `s-maxage` | Serve only inside origin lifetime and operator ceiling | Reuse only while fresh under both limits |
|
|
426
|
+
| Valid `max-age`, without `s-maxage` | Same, using `max-age` | Same |
|
|
427
|
+
| Valid `Expires`, without either age directive | Same, using the origin expiration lifetime | Same |
|
|
377
428
|
| Invalid or ambiguous explicit expiry | Treat as stale; validate or acquire | Full acquisition |
|
|
378
|
-
|
|
|
429
|
+
| Eligible response without explicit lifetime | Use the operator TTL as Plurnk's heuristic | Reuse inside the operator TTL |
|
|
379
430
|
|
|
380
431
|
The operator ceiling is `PLURNK_SCHEMES_HTTP_TTL_MS`; `0` disables every
|
|
381
432
|
validation-free reuse. Origin age is the greater of the response's `Age` value
|
|
382
433
|
and apparent age from `Date`, plus residence since the authoritative package
|
|
383
434
|
stamp. A representation is fresh only while both origin lifetime and operator
|
|
384
|
-
ceiling permit it.
|
|
385
|
-
|
|
435
|
+
ceiling permit it. `s-maxage` takes precedence even when invalid or duplicated:
|
|
436
|
+
such a value is stale, not a reason to fall back to `max-age` or `Expires`.
|
|
437
|
+
`private` and `no-store` prohibit reuse regardless of other permission directives.
|
|
438
|
+
Unknown cache extensions are inert. Stale content is never served, so
|
|
439
|
+
`must-revalidate` and `proxy-revalidate` require no separate path.
|
|
386
440
|
|
|
387
441
|
Outside the fresh window, a representation without a materializer identity may
|
|
388
442
|
send a stored ETag or Last-Modified only when that field is singular and
|
|
@@ -394,7 +448,15 @@ validator corresponds to the nominated stored representation:
|
|
|
394
448
|
| Strong ETag | Same stored strong ETag |
|
|
395
449
|
| Weak ETag | Same opaque tag under weak comparison |
|
|
396
450
|
| No ETag; Last-Modified | Same valid stored Last-Modified instant |
|
|
397
|
-
| Missing, malformed,
|
|
451
|
+
| Missing, malformed, or non-corresponding validator | Do not update the stored representation |
|
|
452
|
+
|
|
453
|
+
Response ETags use RFC 9111 §4.3.4 cache-update correspondence, not the origin's
|
|
454
|
+
`If-None-Match` precondition comparison. A strong response tag cannot promote a
|
|
455
|
+
stored weak tag into byte identity. A non-corresponding 304 triggers one GET
|
|
456
|
+
without conditional headers; a complete response is acquired normally. A
|
|
457
|
+
second 304, or a 304 without a reusable stored representation, returns `502`
|
|
458
|
+
(`fetch-failed`) without serving or replacing the stored body. No validators
|
|
459
|
+
are invented for an unsolicited 304.
|
|
398
460
|
|
|
399
461
|
A corresponding 304 restores the stored channels without rematerializing and
|
|
400
462
|
updates the header under the following ownership rule; any other response
|
|
@@ -408,7 +470,7 @@ replaces the channels:
|
|
|
408
470
|
| Package method, acquisition stamp, and variant | Rebuild authoritatively; refresh stamp and variant |
|
|
409
471
|
| Projection evidence | Preserve |
|
|
410
472
|
|
|
411
|
-
A 304-provided `Vary`, `no-store`, `no-cache`, expiry, or validator
|
|
473
|
+
A 304-provided `Vary`, `private`, `no-store`, `no-cache`, expiry, or validator
|
|
412
474
|
therefore governs the next operation without relabeling derived Unicode as a
|
|
413
475
|
different source representation.
|
|
414
476
|
|
|
@@ -422,8 +484,8 @@ affected route.
|
|
|
422
484
|
Once a page representation leaves its fresh window, Plurnk performs complete
|
|
423
485
|
reacquisition without old origin validators. Origin `304` can certify origin
|
|
424
486
|
bytes, but it cannot certify a composite that may also contain a fresh
|
|
425
|
-
materializer extraction
|
|
426
|
-
|
|
487
|
+
materializer extraction or a newly negotiated Markdown representation.
|
|
488
|
+
Projection or materializer mismatch likewise invalidates
|
|
427
489
|
body, TTL, and validators together.
|
|
428
490
|
|
|
429
491
|
POST, PUT, and DELETE responses retain their method marker but cannot satisfy a
|
|
@@ -431,7 +493,7 @@ later GET or exact-FIND acquisition. An unmarked authored entry and an eligible
|
|
|
431
493
|
stored GET remain visible to universal FIND as durable evidence; exact HTTP
|
|
432
494
|
preparation applies the policy above. A metadata-less `KILL` deletes the stored entry ({§http-kill}).
|
|
433
495
|
|
|
434
|
-
§http-kill **KILL follows the entry rule; the remote DELETE is its own spelling.** ```` ```KILL (url) ```` cancels all live acquisitions of that exact URL within the workspace — GET, SSE, or mutations — by aborting their registered controllers; each initiating operation settles itself as `499` cancelled; with nothing in flight it deletes the local stored entry so the next READ must acquire again. Only ```` ```KILL (url) [{"remote": true}] ```` sends the HTTP DELETE, and
|
|
496
|
+
§http-kill **KILL follows the entry rule; the remote DELETE is its own spelling.** ```` ```KILL (url) ```` cancels all live acquisitions of that exact URL within the workspace — GET, SSE, or mutations — by aborting their registered controllers; each initiating operation settles itself as `499` cancelled; with nothing in flight it deletes the local stored entry so the next READ must acquire again. Only ```` ```KILL (url) [{"remote": true}] ```` sends the HTTP DELETE, and the other options in that metadata block are the request's headers. A KILL never reaches the remote by accident.
|
|
435
497
|
|
|
436
498
|
### §sse Server-sent events
|
|
437
499
|
|
|
@@ -450,14 +512,13 @@ operation is cancelled.
|
|
|
450
512
|
| Result | Meaning |
|
|
451
513
|
| ------------------------------------ | ---------------------------------------------------------------------------------------- |
|
|
452
514
|
| Origin HTML | Exact source text, MIME type, package-stamped evidence, and page-producer eligibility |
|
|
453
|
-
| Negotiated origin Markdown | Exact body
|
|
515
|
+
| Negotiated origin Markdown | Exact body without a second HTML acquisition |
|
|
454
516
|
| Other accepted body | One unconsumed byte stream, MIME type, package-stamped headers, and cancellation owner |
|
|
455
517
|
| Admitted origin transport failure | Bounded origin failure plus eligibility for provider-only page production |
|
|
456
518
|
| Materialized configured text | Complete Fetch-decoded UTF-8 body |
|
|
457
|
-
| Materialized
|
|
458
|
-
| Materialized HTML page | Independent
|
|
519
|
+
| Materialized binary | Original bounded `body` bytes, optional `readable`, and projection identity in `header` |
|
|
520
|
+
| Materialized HTML page | Independent source/readable outcomes and complete route/provider evidence |
|
|
459
521
|
| Automatic top-level `null` | Refused target, non-2xx response, or unavailable response with no provider route |
|
|
460
|
-
| Non-page materialization `null` | No final readable binary projection |
|
|
461
522
|
| Caller-cancelled acquisition | Rejects with the caller signal's exact reason |
|
|
462
523
|
|
|
463
524
|
Top-level `null` is an automatic-acquisition liveness value rather than a thrown
|
|
@@ -473,7 +534,9 @@ policy; its consumers supply those boundaries and the projection capability.
|
|
|
473
534
|
`wss` is a first-class data scheme; `ws` routes to the same handler. WebSocket
|
|
474
535
|
is bidirectional and stateful, not an HTTP content type. It uses `messages` as a
|
|
475
536
|
`text/plain` default channel and the same canonical network address contract
|
|
476
|
-
{§network-address}.
|
|
537
|
+
{§network-address}. A frame written by EDIT or SEND is data leaving the
|
|
538
|
+
process and proposes ({§http-outbound-proposes}); the socket open itself is an
|
|
539
|
+
acquisition and does not.
|
|
477
540
|
|
|
478
541
|
### §ws-lifecycle Socket ownership and settlement
|
|
479
542
|
|
package/dist/Config.d.ts
CHANGED
|
@@ -1,3 +1,5 @@
|
|
|
1
|
+
export declare const requireTextEnv: (key: string) => string;
|
|
2
|
+
export declare const requireFlagEnv: (key: string) => boolean;
|
|
1
3
|
export declare const requireNonNegativeIntegerEnv: (key: string) => number;
|
|
2
4
|
export declare const requirePositiveIntegerEnv: (key: string) => number;
|
|
3
5
|
//# sourceMappingURL=Config.d.ts.map
|
package/dist/Config.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"Config.d.ts","sourceRoot":"","sources":["../src/Config.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"Config.d.ts","sourceRoot":"","sources":["../src/Config.ts"],"names":[],"mappings":"AACA,eAAO,MAAM,cAAc,QAAS,MAAM,KAAG,MAM5C,CAAC;AAGF,eAAO,MAAM,cAAc,QAAS,MAAM,KAAG,OAI5C,CAAC;AAEF,eAAO,MAAM,4BAA4B,QAAS,MAAM,KAAG,MAU1D,CAAC;AAEF,eAAO,MAAM,yBAAyB,QAAS,MAAM,KAAG,MAIvD,CAAC"}
|
package/dist/Config.js
CHANGED
|
@@ -1,3 +1,18 @@
|
|
|
1
|
+
// {§operator-config-only-home} — the panel states the value; an unset key is a broken floor.
|
|
2
|
+
export const requireTextEnv = (key) => {
|
|
3
|
+
const raw = process.env[key];
|
|
4
|
+
if (raw === undefined || raw.trim().length === 0) {
|
|
5
|
+
throw new Error(`${key} is unset.`);
|
|
6
|
+
}
|
|
7
|
+
return raw;
|
|
8
|
+
};
|
|
9
|
+
// The house switch: exactly 0 or 1.
|
|
10
|
+
export const requireFlagEnv = (key) => {
|
|
11
|
+
const raw = process.env[key];
|
|
12
|
+
if (raw !== "0" && raw !== "1")
|
|
13
|
+
throw new Error(`${key} must be 0 or 1; got ${JSON.stringify(raw)}.`);
|
|
14
|
+
return raw === "1";
|
|
15
|
+
};
|
|
1
16
|
export const requireNonNegativeIntegerEnv = (key) => {
|
|
2
17
|
const raw = process.env[key];
|
|
3
18
|
if (raw === undefined || raw.trim().length === 0) {
|
package/dist/Config.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"Config.js","sourceRoot":"","sources":["../src/Config.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,4BAA4B,GAAG,CAAC,GAAW,EAAU,EAAE;IAChE,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAC7B,IAAI,GAAG,KAAK,SAAS,IAAI,GAAG,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC/C,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,YAAY,CAAC,CAAC;IACxC,CAAC;IACD,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC,EAAE,CAAC;QAC5C,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,kCAAkC,CAAC,CAAC;IAC9D,CAAC;IACD,OAAO,KAAK,CAAC;AACjB,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,yBAAyB,GAAG,CAAC,GAAW,EAAU,EAAE;IAC7D,MAAM,KAAK,GAAG,4BAA4B,CAAC,GAAG,CAAC,CAAC;IAChD,IAAI,KAAK,KAAK,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,8BAA8B,CAAC,CAAC;IACvE,OAAO,KAAK,CAAC;AACjB,CAAC,CAAC"}
|
|
1
|
+
{"version":3,"file":"Config.js","sourceRoot":"","sources":["../src/Config.ts"],"names":[],"mappings":"AAAA,6FAA6F;AAC7F,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,GAAW,EAAU,EAAE;IAClD,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAC7B,IAAI,GAAG,KAAK,SAAS,IAAI,GAAG,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC/C,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,YAAY,CAAC,CAAC;IACxC,CAAC;IACD,OAAO,GAAG,CAAC;AACf,CAAC,CAAC;AAEF,oCAAoC;AACpC,MAAM,CAAC,MAAM,cAAc,GAAG,CAAC,GAAW,EAAW,EAAE;IACnD,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAC7B,IAAI,GAAG,KAAK,GAAG,IAAI,GAAG,KAAK,GAAG;QAAE,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,wBAAwB,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IACtG,OAAO,GAAG,KAAK,GAAG,CAAC;AACvB,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,4BAA4B,GAAG,CAAC,GAAW,EAAU,EAAE;IAChE,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAC7B,IAAI,GAAG,KAAK,SAAS,IAAI,GAAG,CAAC,IAAI,EAAE,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC/C,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,YAAY,CAAC,CAAC;IACxC,CAAC;IACD,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,GAAG,CAAC,EAAE,CAAC;QAC5C,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,kCAAkC,CAAC,CAAC;IAC9D,CAAC;IACD,OAAO,KAAK,CAAC;AACjB,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,yBAAyB,GAAG,CAAC,GAAW,EAAU,EAAE;IAC7D,MAAM,KAAK,GAAG,4BAA4B,CAAC,GAAG,CAAC,CAAC;IAChD,IAAI,KAAK,KAAK,CAAC;QAAE,MAAM,IAAI,KAAK,CAAC,GAAG,GAAG,8BAA8B,CAAC,CAAC;IACvE,OAAO,KAAK,CAAC;AACjB,CAAC,CAAC"}
|
package/dist/Guard.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"Guard.d.ts","sourceRoot":"","sources":["../src/Guard.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"Guard.d.ts","sourceRoot":"","sources":["../src/Guard.ts"],"names":[],"mappings":"AA+DA,qBAAa,iBAAkB,SAAQ,KAAK;IACxC,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;IACrB,YAAY,GAAG,EAAE,MAAM,EAKtB;CACJ;AAED,MAAM,CAAC,OAAO,OAAO,KAAK;IAGtB,MAAM,CAAC,eAAe,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO,CAM1C;IAID,OAAa,WAAW,CAAC,GAAG,EAAE,MAAM,GAAG,OAAO,CAAC,OAAO,CAAC,CActD;IAOD,OAAa,KAAK,CACd,GAAG,EAAE,MAAM,EACX,IAAI,EAAE;QAAE,MAAM,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,GAAG,SAAS,CAAC;QAAC,OAAO,EAAE,aAAa,CAAC,SAAS,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC,CAAA;KAAE,EACrG,MAAM,EAAE,WAAW,GACpB,OAAO,CAAC,QAAQ,CAAC,CA6BnB;CACJ"}
|
package/dist/Guard.js
CHANGED
|
@@ -5,6 +5,7 @@
|
|
|
5
5
|
import dns from "node:dns/promises";
|
|
6
6
|
import net from "node:net";
|
|
7
7
|
import { requireNonNegativeIntegerEnv } from "./Config.js";
|
|
8
|
+
import HostPolicy from "./HostPolicy.js";
|
|
8
9
|
const BLOCKED = new net.BlockList();
|
|
9
10
|
for (const [network, prefix] of [
|
|
10
11
|
["0.0.0.0", 8],
|
|
@@ -121,7 +122,8 @@ export default class Guard {
|
|
|
121
122
|
catch {
|
|
122
123
|
throw new GuardBlockedError(target);
|
|
123
124
|
}
|
|
124
|
-
|
|
125
|
+
// {§http-host-policy} — every hop, not just the first, stays inside the operator's hosts.
|
|
126
|
+
if (!["http:", "https:"].includes(current.protocol) || !HostPolicy.permits(current.href) || !(await Guard.isPublicUrl(current.href))) {
|
|
125
127
|
throw new GuardBlockedError(current.href);
|
|
126
128
|
}
|
|
127
129
|
const response = await fetch(current.href, { method, body, headers, signal, redirect: "manual" });
|
package/dist/Guard.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"Guard.js","sourceRoot":"","sources":["../src/Guard.ts"],"names":[],"mappings":"AAAA,wEAAwE;AACxE,wEAAwE;AACxE,4EAA4E;AAC5E,4DAA4D;AAE5D,OAAO,GAAG,MAAM,mBAAmB,CAAC;AACpC,OAAO,GAAG,MAAM,UAAU,CAAC;AAC3B,OAAO,EAAE,4BAA4B,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"Guard.js","sourceRoot":"","sources":["../src/Guard.ts"],"names":[],"mappings":"AAAA,wEAAwE;AACxE,wEAAwE;AACxE,4EAA4E;AAC5E,4DAA4D;AAE5D,OAAO,GAAG,MAAM,mBAAmB,CAAC;AACpC,OAAO,GAAG,MAAM,UAAU,CAAC;AAC3B,OAAO,EAAE,4BAA4B,EAAE,MAAM,aAAa,CAAC;AAC3D,OAAO,UAAU,MAAM,iBAAiB,CAAC;AAEzC,MAAM,OAAO,GAAG,IAAI,GAAG,CAAC,SAAS,EAAE,CAAC;AACpC,KAAK,MAAM,CAAC,OAAO,EAAE,MAAM,CAAC,IAAI;IAC5B,CAAC,SAAS,EAAE,CAAC,CAAC;IACd,CAAC,UAAU,EAAE,CAAC,CAAC;IACf,CAAC,YAAY,EAAE,EAAE,CAAC;IAClB,CAAC,WAAW,EAAE,CAAC,CAAC;IAChB,CAAC,aAAa,EAAE,EAAE,CAAC;IACnB,CAAC,YAAY,EAAE,EAAE,CAAC;IAClB,CAAC,WAAW,EAAE,EAAE,CAAC;IACjB,CAAC,WAAW,EAAE,EAAE,CAAC;IACjB,CAAC,aAAa,EAAE,EAAE,CAAC;IACnB,CAAC,aAAa,EAAE,EAAE,CAAC;IACnB,CAAC,YAAY,EAAE,EAAE,CAAC;IAClB,CAAC,cAAc,EAAE,EAAE,CAAC;IACpB,CAAC,aAAa,EAAE,EAAE,CAAC;IACnB,CAAC,WAAW,EAAE,CAAC,CAAC;IAChB,CAAC,WAAW,EAAE,CAAC,CAAC;CACV;IAAE,OAAO,CAAC,SAAS,CAAC,OAAO,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;AACvD,KAAK,MAAM,CAAC,OAAO,EAAE,MAAM,CAAC,IAAI;IAC5B,CAAC,QAAQ,EAAE,EAAE,CAAC;IACd,CAAC,YAAY,EAAE,EAAE,CAAC;IAClB,CAAC,QAAQ,EAAE,EAAE,CAAC;IACd,CAAC,QAAQ,EAAE,EAAE,CAAC;CACR;IAAE,OAAO,CAAC,SAAS,CAAC,OAAO,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;AAEvD,MAAM,WAAW,GAAG,IAAI,GAAG,CAAC,SAAS,EAAE,CAAC;AACxC,WAAW,CAAC,SAAS,CAAC,SAAS,EAAE,CAAC,EAAE,MAAM,CAAC,CAAC;AAC5C,MAAM,WAAW,GAAG,IAAI,GAAG,CAAC,SAAS,EAAE,CAAC;AACxC,WAAW,CAAC,SAAS,CAAC,QAAQ,EAAE,CAAC,EAAE,MAAM,CAAC,CAAC;AAE3C,MAAM,iBAAiB,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,EAAE,GAAG,CAAC,CAAC,CAAC;AAC7D,MAAM,YAAY,GAAG,IAAI,GAAG,CAAC,CAAC,kBAAkB,EAAE,kBAAkB,EAAE,kBAAkB,EAAE,cAAc,CAAC,CAAC,CAAC;AAC3G,MAAM,qBAAqB,GAAG,IAAI,GAAG,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC;AAEzD,MAAM,cAAc,GAAG,CACnB,OAAiD,EACjD,KAA0B,EACH,EAAE,CAAC,OAAO;KAChC,MAAM,CAAC,CAAC,CAAC,IAAI,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,CAAC;KAClD,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,KAAK,CAAC,EAAE,EAAE,CAAC,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC;AAE3C,MAAM,OAAO,GAAG,CAAC,GAAW,EAAU,EAAE;IACpC,IAAI,CAAC;QACD,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;QACzB,GAAG,CAAC,QAAQ,GAAG,EAAE,CAAC;QAClB,GAAG,CAAC,QAAQ,GAAG,EAAE,CAAC;QAClB,OAAO,GAAG,CAAC,IAAI,CAAC;IACpB,CAAC;IAAC,MAAM,CAAC;QACL,OAAO,GAAG,CAAC;IACf,CAAC;AACL,CAAC,CAAC;AAEF,uEAAuE;AACvE,MAAM,OAAO,iBAAkB,SAAQ,KAAK;IAC/B,GAAG,CAAS;IACrB,YAAY,GAAW;QACnB,MAAM,SAAS,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC;QAC/B,KAAK,CAAC,0BAA0B,SAAS,iCAAiC,CAAC,CAAC;QAC5E,IAAI,CAAC,IAAI,GAAG,mBAAmB,CAAC;QAChC,IAAI,CAAC,GAAG,GAAG,SAAS,CAAC;IACzB,CAAC;CACJ;AAED,MAAM,CAAC,OAAO,OAAO,KAAK;IACtB,2EAA2E;IAC3E,oEAAoE;IACpE,MAAM,CAAC,eAAe,CAAC,EAAU;QAC7B,MAAM,MAAM,GAAG,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QAC5B,IAAI,MAAM,KAAK,CAAC;YAAE,OAAO,KAAK,CAAC;QAC/B,IAAI,MAAM,KAAK,CAAC;YAAE,OAAO,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC;QACpD,IAAI,WAAW,CAAC,KAAK,CAAC,EAAE,EAAE,MAAM,CAAC;YAAE,OAAO,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC;QACrE,OAAO,WAAW,CAAC,KAAK,CAAC,EAAE,EAAE,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC;IACvE,CAAC;IAED,uEAAuE;IACvE,0EAA0E;IAC1E,MAAM,CAAC,KAAK,CAAC,WAAW,CAAC,GAAW;QAChC,IAAI,GAAQ,CAAC;QACb,IAAI,CAAC;YAAC,GAAG,GAAG,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;QAAC,CAAC;QAAC,MAAM,CAAC;YAAC,OAAO,KAAK,CAAC;QAAC,CAAC;QACnD,IAAI,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC,QAAQ,CAAC;YAAE,OAAO,KAAK,CAAC;QAC9D,IAAI,GAAG,CAAC,QAAQ,KAAK,EAAE,IAAI,GAAG,CAAC,QAAQ,KAAK,EAAE;YAAE,OAAO,KAAK,CAAC;QAC7D,MAAM,IAAI,GAAG,GAAG,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC;QAChE,IAAI,IAAI,KAAK,WAAW,IAAI,IAAI,CAAC,QAAQ,CAAC,YAAY,CAAC;YAAE,OAAO,KAAK,CAAC;QACtE,IAAI,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC;YAAE,OAAO,KAAK,CAAC,eAAe,CAAC,IAAI,CAAC,CAAC;QACvD,IAAI,CAAC;YACD,MAAM,KAAK,GAAG,MAAM,GAAG,CAAC,MAAM,CAAC,IAAI,EAAE,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC,CAAC;YACpD,OAAO,KAAK,CAAC,MAAM,GAAG,CAAC,IAAI,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC,KAAK,CAAC,eAAe,CAAC,OAAO,CAAC,CAAC,CAAC;QAC5F,CAAC;QAAC,MAAM,CAAC;YACL,OAAO,KAAK,CAAC,CAAC,8BAA8B;QAChD,CAAC;IACL,CAAC;IAED,kFAAkF;IAClF,6EAA6E;IAC7E,uEAAuE;IACvE,uEAAuE;IACvE,+DAA+D;IAC/D,MAAM,CAAC,KAAK,CAAC,KAAK,CACd,GAAW,EACX,IAAqG,EACrG,MAAmB;QAEnB,IAAI,MAAM,GAAG,GAAG,CAAC;QACjB,IAAI,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;QACzB,IAAI,IAAI,GAAG,IAAI,CAAC,IAAI,CAAC;QACrB,IAAI,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,KAAK,CAAC,EAAoB,EAAE,CAAC,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC;QACnF,IAAI,IAAI,GAAG,4BAA4B,CAAC,+BAA+B,CAAC,CAAC;QACzE,OAAO,IAAI,EAAE,CAAC;YACV,IAAI,OAAY,CAAC;YACjB,IAAI,CAAC;gBAAC,OAAO,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,CAAC;YAAC,CAAC;YAAC,MAAM,CAAC;gBAAC,MAAM,IAAI,iBAAiB,CAAC,MAAM,CAAC,CAAC;YAAC,CAAC;YACjF,0FAA0F;YAC1F,IAAI,CAAC,CAAC,OAAO,EAAE,QAAQ,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,UAAU,CAAC,OAAO,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,WAAW,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,EAAE,CAAC;gBACnI,MAAM,IAAI,iBAAiB,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;YAC9C,CAAC;YACD,MAAM,QAAQ,GAAG,MAAM,KAAK,CAAC,OAAO,CAAC,IAAI,EAAE,EAAE,MAAM,EAAE,IAAI,EAAE,OAAO,EAAE,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,CAAC,CAAC;YAClG,IAAI,CAAC,iBAAiB,CAAC,GAAG,CAAC,QAAQ,CAAC,MAAM,CAAC;gBAAE,OAAO,QAAQ,CAAC;YAC7D,MAAM,QAAQ,GAAG,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC;YAClD,IAAI,QAAQ,KAAK,IAAI,IAAI,IAAI,IAAI,CAAC;gBAAE,OAAO,QAAQ,CAAC;YACpD,MAAM,QAAQ,CAAC,IAAI,EAAE,MAAM,EAAE,CAAC;YAC9B,IAAI,IAAI,CAAC,CAAC;YACV,MAAM,IAAI,GAAG,IAAI,GAAG,CAAC,QAAQ,EAAE,OAAO,CAAC,CAAC;YACxC,IAAI,CAAC,CAAC,QAAQ,CAAC,MAAM,KAAK,GAAG,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG,CAAC,IAAI,MAAM,KAAK,MAAM,CAAC;mBACxE,CAAC,QAAQ,CAAC,MAAM,KAAK,GAAG,IAAI,MAAM,KAAK,KAAK,IAAI,MAAM,KAAK,MAAM,CAAC,EAAE,CAAC;gBACxE,MAAM,GAAG,KAAK,CAAC;gBACf,IAAI,GAAG,SAAS,CAAC;gBACjB,OAAO,GAAG,cAAc,CAAC,OAAO,EAAE,YAAY,CAAC,CAAC;YACpD,CAAC;YACD,IAAI,OAAO,CAAC,MAAM,KAAK,IAAI,CAAC,MAAM;gBAAE,OAAO,GAAG,cAAc,CAAC,OAAO,EAAE,qBAAqB,CAAC,CAAC;YAC7F,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC;QACvB,CAAC;IACL,CAAC;CACJ"}
|