@plurnk/plurnk-schemes-http 1.7.0 → 1.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.defaults +6 -7
- package/README.md +9 -8
- package/SPEC.md +84 -73
- package/dist/Http.d.ts.map +1 -1
- package/dist/Http.js +130 -31
- package/dist/Http.js.map +1 -1
- package/dist/Materializer.d.ts +49 -0
- package/dist/Materializer.d.ts.map +1 -0
- package/dist/Materializer.js +112 -0
- package/dist/Materializer.js.map +1 -0
- package/dist/WebFetcher.d.ts +4 -11
- package/dist/WebFetcher.d.ts.map +1 -1
- package/dist/WebFetcher.js +70 -88
- package/dist/WebFetcher.js.map +1 -1
- package/dist/Ws.d.ts.map +1 -1
- package/dist/Ws.js +12 -7
- package/dist/Ws.js.map +1 -1
- package/dist/index.d.ts +1 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -0
- package/dist/index.js.map +1 -1
- package/docs/{http.md → https.md} +25 -18
- package/docs/wss.md +5 -1
- package/package.json +8 -3
- package/dist/TavilyExtract.d.ts +0 -35
- package/dist/TavilyExtract.d.ts.map +0 -1
- package/dist/TavilyExtract.js +0 -204
- package/dist/TavilyExtract.js.map +0 -1
|
@@ -1,4 +1,11 @@
|
|
|
1
|
-
#
|
|
1
|
+
# https://
|
|
2
|
+
|
|
3
|
+
Plain `http://` is also accepted for the endpoint that requires it — a local
|
|
4
|
+
service, an interior tool. Prefer `https://` everywhere else.
|
|
5
|
+
|
|
6
|
+
## Summary
|
|
7
|
+
|
|
8
|
+
Read and modify web resources through addressable HTTP(S) entries.
|
|
2
9
|
|
|
3
10
|
Use a web URL as an addressable entry. Every exact READ acquires or refreshes a
|
|
4
11
|
complete representation when needed, then core selects the channel and applies
|
|
@@ -8,11 +15,11 @@ are never presented by default.
|
|
|
8
15
|
|
|
9
16
|
| Operation | Remote action | Effect |
|
|
10
17
|
| ---------------------------------------------- | ------------- | ------------------------------------------------------------------------- |
|
|
11
|
-
| `## READ0 (
|
|
12
|
-
| `## FIND0 (
|
|
13
|
-
| `## SEND0 [200] (
|
|
14
|
-
| `## EDIT0 (
|
|
15
|
-
| `## KILL0 (
|
|
18
|
+
| `## READ0 (https://…) <scope?>` | GET if needed | Acquire/reuse the complete response, then return the selected scoped text |
|
|
19
|
+
| `## FIND0 (https://…)` with matcher body | GET if needed | Prepare an exact URL, then return flat match locations |
|
|
20
|
+
| `## SEND0 [200] (https://…)` with body | POST | Submit the body and stream the response |
|
|
21
|
+
| `## EDIT0 (https://…)` with body | PUT | Replace the whole remote resource; do not use a line scope |
|
|
22
|
+
| `## KILL0 (https://…)` | DELETE | Delete the remote resource and stream the response |
|
|
16
23
|
|
|
17
24
|
A path-pattern FIND searches only web entries already materialized in the
|
|
18
25
|
workspace; a pattern cannot discover the remote web. FIND returns navigation
|
|
@@ -23,19 +30,19 @@ Caller cancellation of an exact acquisition returns `499 cancelled`.
|
|
|
23
30
|
| Response | `body` | Other channel |
|
|
24
31
|
| ------------------------------------ | --------------------------------------------------- | ---------------------------------------------- |
|
|
25
32
|
| Negotiated origin Markdown | Exact origin Markdown | Independently requested server HTML in `#html` |
|
|
26
|
-
| GET HTML |
|
|
33
|
+
| GET HTML | Materializer Markdown or local HTML-reader projection | Original server HTML in `#html` |
|
|
27
34
|
| GET `text/event-stream` | Event `data` chunks after READ `102` | Initial response in `#header` |
|
|
28
35
|
| Configured textual response | Complete Fetch-decoded text under its declared type | Status and headers in `#header` |
|
|
29
36
|
| Origin HTTP `4xx`/`5xx` | Preserve available origin or independently produced text | Exact status on each origin-backed channel |
|
|
30
37
|
| Binary with a readable projection | Derived Unicode under the projection output type | Origin type and projection ID in `#header` |
|
|
31
38
|
| Binary without a readable projection | No fabricated text representation | Exact `415` Problem |
|
|
32
39
|
|
|
33
|
-
Generic public HTML uses
|
|
40
|
+
Generic public HTML uses the selected materializer as its body producer. A recoverable
|
|
34
41
|
timeout, transport error, `429`, `5xx`, or per-URL extraction failure uses the
|
|
35
42
|
local HTML reader and records terminal body status `203`. Authentication,
|
|
36
43
|
provider rejection, and malformed provider responses are hard failures and do
|
|
37
44
|
not silently switch producers. Any authored target metadata makes HTML use the
|
|
38
|
-
local reader directly. Origin Markdown always wins without
|
|
45
|
+
local reader directly. Origin Markdown always wins without the materializer.
|
|
39
46
|
|
|
40
47
|
Projection presence is structural: a returned projection is accepted even
|
|
41
48
|
when its content is empty. `422 no-readable-projection` means the local route
|
|
@@ -50,15 +57,15 @@ without fabricating an entry. Input above the configured byte ceiling returns
|
|
|
50
57
|
owns lifecycle evidence in `#header`; do not retry a mutation solely to
|
|
51
58
|
retrieve its binary body.
|
|
52
59
|
|
|
53
|
-
`#header` contains origin and package acquisition evidence. A
|
|
60
|
+
`#header` contains origin and package acquisition evidence. A materializer attempt
|
|
54
61
|
adds its route, status, timing, any reported request ID and credits, and bounded
|
|
55
62
|
failure evidence. `body`, `header`, and `html` carry independent durable
|
|
56
63
|
producer outcomes; the selected channel determines success. Thus `#html` or
|
|
57
|
-
`#header` can remain readable after a body failure, while a
|
|
64
|
+
`#header` can remain readable after a body failure, while a materializer body can
|
|
58
65
|
succeed without fabricating unavailable server HTML. A SEND signal is never the
|
|
59
66
|
remote HTTP status. A direct non-success origin response is still materialized:
|
|
60
67
|
origin-backed channels carry its exact durable `http-response-status` Problem,
|
|
61
|
-
while independently produced
|
|
68
|
+
while independently produced materializer content and acquisition headers retain
|
|
62
69
|
their own outcomes.
|
|
63
70
|
|
|
64
71
|
For SSE, the response and persisted `#header` establish acquisition. READ then
|
|
@@ -79,7 +86,7 @@ ETag or Last-Modified value identifies the nominated representation; otherwise
|
|
|
79
86
|
acquisition fails without serving the stored body. Responses to POST, PUT, and
|
|
80
87
|
DELETE are not reused as later GET representations. A projected GET is reused
|
|
81
88
|
only while the installed reader has the same projection identity. Page bodies
|
|
82
|
-
likewise require the same origin-Markdown, local,
|
|
89
|
+
likewise require the same origin-Markdown, local, materializer-id, or
|
|
83
90
|
local-fallback route. Once stale, a page composite is fully reacquired without
|
|
84
91
|
old validators; an origin 304 cannot certify provider or auxiliary material.
|
|
85
92
|
|
|
@@ -101,9 +108,9 @@ blocks, one header per block:
|
|
|
101
108
|
Percent-encode `)`, `<`, and `}` inside a header value.
|
|
102
109
|
An exact FIND forwards these headers when it must acquire the URL, but the
|
|
103
110
|
result remains intentionally ineligible for later cache reuse. Target headers
|
|
104
|
-
are never forwarded to
|
|
105
|
-
use the local HTML-reader projection. Executor-supplied HTML also
|
|
106
|
-
only generic web acquisition grants
|
|
111
|
+
are never forwarded to the materializer, so HTML requests carrying any explicit
|
|
112
|
+
metadata use the local HTML-reader projection. Executor-supplied HTML also
|
|
113
|
+
stays local; only generic web acquisition grants materializer authority.
|
|
107
114
|
|
|
108
115
|
GET acquisition of a GitHub `…/blob/…` URL uses its
|
|
109
116
|
`raw.githubusercontent.com` source. The addressed GitHub URL remains entry
|
|
@@ -111,7 +118,7 @@ identity. POST, PUT, and DELETE never use that rewrite.
|
|
|
111
118
|
|
|
112
119
|
| Control | Effect |
|
|
113
120
|
| ------------------------ | --------------------------------------------------------------------- |
|
|
114
|
-
| `## SEND0 [499] (
|
|
115
|
-
| `## SEND0 [410] (
|
|
121
|
+
| `## SEND0 [499] (https://…)` | Cancel the routed in-flight acquisition |
|
|
122
|
+
| `## SEND0 [410] (https://…)` | Delete the local stored response; the next READ must acquire it again |
|
|
116
123
|
|
|
117
124
|
For a persistent bidirectional connection, use `wss://`.
|
package/docs/wss.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# wss:// — WebSocket
|
|
2
2
|
|
|
3
|
+
## Summary
|
|
4
|
+
|
|
5
|
+
Maintain persistent, bidirectional WebSocket connections as addressable entries.
|
|
6
|
+
|
|
3
7
|
Use WebSocket for a persistent, bidirectional connection. `wss` is a
|
|
4
8
|
stateful scheme, not an HTTP content type: READ claims a workspace address and
|
|
5
9
|
owns its socket until terminal settlement, while EDIT, SEND, and KILL address
|
|
@@ -32,7 +36,7 @@ EDIT and SEND share the same outbound-frame behavior. Operation phase order is
|
|
|
32
36
|
the only scheduling distinction: SEND can follow READ in one turn, while EDIT
|
|
33
37
|
runs before READ and therefore requires a connection opened by an earlier turn.
|
|
34
38
|
|
|
35
|
-
Connection identity includes the
|
|
39
|
+
Connection identity includes the owning worker, exact `ws`/`wss` protocol, host,
|
|
36
40
|
non-default port, path, and ordered query. A fragment does not change socket
|
|
37
41
|
identity; `messages` is the only current channel.
|
|
38
42
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@plurnk/plurnk-schemes-http",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.9.0",
|
|
4
4
|
"description": "HTTP(S) request/response and WebSocket scheme handlers for the plurnk agent runtime.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"plurnk",
|
|
@@ -34,7 +34,7 @@
|
|
|
34
34
|
"kind": "scheme",
|
|
35
35
|
"schemes": [
|
|
36
36
|
{
|
|
37
|
-
"name": "
|
|
37
|
+
"name": "https",
|
|
38
38
|
"export": "default"
|
|
39
39
|
},
|
|
40
40
|
{
|
|
@@ -49,6 +49,11 @@
|
|
|
49
49
|
"types": "./dist/index.d.ts",
|
|
50
50
|
"default": "./dist/index.js"
|
|
51
51
|
},
|
|
52
|
+
"./materializer": {
|
|
53
|
+
"plurnk-dev": "./src/Materializer.ts",
|
|
54
|
+
"types": "./dist/Materializer.d.ts",
|
|
55
|
+
"default": "./dist/Materializer.js"
|
|
56
|
+
},
|
|
52
57
|
"./package.json": "./package.json"
|
|
53
58
|
},
|
|
54
59
|
"files": [
|
|
@@ -69,7 +74,7 @@
|
|
|
69
74
|
"prepack": "npm run build"
|
|
70
75
|
},
|
|
71
76
|
"peerDependencies": {
|
|
72
|
-
"@plurnk/plurnk-schemes": "^1.
|
|
77
|
+
"@plurnk/plurnk-schemes": "^1.9.0"
|
|
73
78
|
},
|
|
74
79
|
"dependencies": {
|
|
75
80
|
"eventsource-parser": "^4.0.0"
|
package/dist/TavilyExtract.d.ts
DELETED
|
@@ -1,35 +0,0 @@
|
|
|
1
|
-
export declare const TAVILY_DEPTH = "PLURNK_SCHEMES_HTTP_TAVILY_DEPTH";
|
|
2
|
-
export declare const TAVILY_TIMEOUT_MS = "PLURNK_SCHEMES_HTTP_TAVILY_TIMEOUT_MS";
|
|
3
|
-
export type TavilyDepth = "basic" | "advanced";
|
|
4
|
-
export type TavilyFailureReason = "authentication" | "provider-rejection" | "rate-limit" | "server" | "failed-result" | "timeout" | "network" | "malformed-response";
|
|
5
|
-
interface TavilyEvidence {
|
|
6
|
-
readonly status?: number;
|
|
7
|
-
readonly requestId?: string;
|
|
8
|
-
readonly credits?: number;
|
|
9
|
-
readonly retryAfter?: string;
|
|
10
|
-
readonly error?: string;
|
|
11
|
-
readonly elapsedMs: number;
|
|
12
|
-
}
|
|
13
|
-
export interface TavilyExtractSuccess extends TavilyEvidence {
|
|
14
|
-
readonly outcome: "success";
|
|
15
|
-
readonly markdown: string;
|
|
16
|
-
readonly sourceUrl?: string;
|
|
17
|
-
}
|
|
18
|
-
export interface TavilyExtractFailure extends TavilyEvidence {
|
|
19
|
-
readonly outcome: "recoverable" | "hard";
|
|
20
|
-
readonly reason: TavilyFailureReason;
|
|
21
|
-
}
|
|
22
|
-
export type TavilyExtractResult = TavilyExtractSuccess | TavilyExtractFailure;
|
|
23
|
-
export interface TavilyConfiguration {
|
|
24
|
-
readonly apiKey: string;
|
|
25
|
-
readonly depth: TavilyDepth;
|
|
26
|
-
readonly timeoutMs: number;
|
|
27
|
-
readonly identity: string;
|
|
28
|
-
}
|
|
29
|
-
export declare const tavilyConfiguration: () => TavilyConfiguration | null;
|
|
30
|
-
export declare const tavilyExtractIdentity: () => string | null;
|
|
31
|
-
export declare const tavilyExtract: (url: string, opts?: {
|
|
32
|
-
signal?: AbortSignal;
|
|
33
|
-
}) => Promise<TavilyExtractResult | null>;
|
|
34
|
-
export {};
|
|
35
|
-
//# sourceMappingURL=TavilyExtract.d.ts.map
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"TavilyExtract.d.ts","sourceRoot":"","sources":["../src/TavilyExtract.ts"],"names":[],"mappings":"AAIA,eAAO,MAAM,YAAY,qCAAqC,CAAC;AAC/D,eAAO,MAAM,iBAAiB,0CAA0C,CAAC;AAEzE,MAAM,MAAM,WAAW,GAAG,OAAO,GAAG,UAAU,CAAC;AAC/C,MAAM,MAAM,mBAAmB,GACzB,gBAAgB,GAChB,oBAAoB,GACpB,YAAY,GACZ,QAAQ,GACR,eAAe,GACf,SAAS,GACT,SAAS,GACT,oBAAoB,CAAC;AAE3B,UAAU,cAAc;IACpB,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;IAC5B,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;CAC9B;AAED,MAAM,WAAW,oBAAqB,SAAQ,cAAc;IACxD,QAAQ,CAAC,OAAO,EAAE,SAAS,CAAC;IAC5B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAC;CAC/B;AAED,MAAM,WAAW,oBAAqB,SAAQ,cAAc;IACxD,QAAQ,CAAC,OAAO,EAAE,aAAa,GAAG,MAAM,CAAC;IACzC,QAAQ,CAAC,MAAM,EAAE,mBAAmB,CAAC;CACxC;AAED,MAAM,MAAM,mBAAmB,GAAG,oBAAoB,GAAG,oBAAoB,CAAC;AAE9E,MAAM,WAAW,mBAAmB;IAChC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,KAAK,EAAE,WAAW,CAAC;IAC5B,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;CAC7B;AAUD,eAAO,MAAM,mBAAmB,QAAO,mBAAmB,GAAG,IAW5D,CAAC;AAEF,eAAO,MAAM,qBAAqB,QAAO,MAAM,GAAG,IAA+C,CAAC;AAoDlG,eAAO,MAAM,aAAa,QACjB,MAAM,SACJ;IAAE,MAAM,CAAC,EAAE,WAAW,CAAA;CAAE,KAChC,OAAO,CAAC,mBAAmB,GAAG,IAAI,CAuIpC,CAAC"}
|
package/dist/TavilyExtract.js
DELETED
|
@@ -1,204 +0,0 @@
|
|
|
1
|
-
import ErrorDetail from "./ErrorDetail.js";
|
|
2
|
-
import { requirePositiveIntegerEnv } from "./Config.js";
|
|
3
|
-
// Provider boundary and failure algebra {§tavily-extract}.
|
|
4
|
-
export const TAVILY_DEPTH = "PLURNK_SCHEMES_HTTP_TAVILY_DEPTH";
|
|
5
|
-
export const TAVILY_TIMEOUT_MS = "PLURNK_SCHEMES_HTTP_TAVILY_TIMEOUT_MS";
|
|
6
|
-
const depth = () => {
|
|
7
|
-
const value = process.env[TAVILY_DEPTH];
|
|
8
|
-
if (value !== "basic" && value !== "advanced") {
|
|
9
|
-
throw new Error(`${TAVILY_DEPTH} must be "basic" or "advanced", got ${JSON.stringify(value)}.`);
|
|
10
|
-
}
|
|
11
|
-
return value;
|
|
12
|
-
};
|
|
13
|
-
export const tavilyConfiguration = () => {
|
|
14
|
-
const configuredDepth = depth();
|
|
15
|
-
const timeoutMs = requirePositiveIntegerEnv(TAVILY_TIMEOUT_MS);
|
|
16
|
-
const apiKey = process.env.TAVILY_API_KEY?.trim() ?? "";
|
|
17
|
-
if (apiKey.length === 0)
|
|
18
|
-
return null;
|
|
19
|
-
return {
|
|
20
|
-
apiKey,
|
|
21
|
-
depth: configuredDepth,
|
|
22
|
-
timeoutMs,
|
|
23
|
-
identity: `tavily-extract:v1:${configuredDepth}`,
|
|
24
|
-
};
|
|
25
|
-
};
|
|
26
|
-
export const tavilyExtractIdentity = () => tavilyConfiguration()?.identity ?? null;
|
|
27
|
-
const elapsed = (started) => Math.max(0, Math.round(performance.now() - started));
|
|
28
|
-
const bounded = (value) => ErrorDetail.preview(value, ErrorDetail.configuredLimit());
|
|
29
|
-
const objectRecord = (value) => value !== null
|
|
30
|
-
&& typeof value === "object"
|
|
31
|
-
&& !Array.isArray(value)
|
|
32
|
-
? value
|
|
33
|
-
: null;
|
|
34
|
-
const optionalString = (value) => typeof value === "string" && value.length > 0
|
|
35
|
-
? value
|
|
36
|
-
: undefined;
|
|
37
|
-
const reportedCredits = (value) => {
|
|
38
|
-
const usage = objectRecord(value);
|
|
39
|
-
const credits = usage?.credits;
|
|
40
|
-
return typeof credits === "number" && Number.isFinite(credits) && credits >= 0
|
|
41
|
-
? credits
|
|
42
|
-
: undefined;
|
|
43
|
-
};
|
|
44
|
-
const responseError = async (response) => {
|
|
45
|
-
const text = await response.text();
|
|
46
|
-
let parsed = null;
|
|
47
|
-
try {
|
|
48
|
-
parsed = objectRecord(JSON.parse(text));
|
|
49
|
-
}
|
|
50
|
-
catch { }
|
|
51
|
-
const requestId = optionalString(parsed?.request_id)
|
|
52
|
-
?? optionalString(response.headers.get("x-request-id"));
|
|
53
|
-
const credits = reportedCredits(parsed?.usage);
|
|
54
|
-
const error = optionalString(parsed?.detail)
|
|
55
|
-
?? optionalString(parsed?.message)
|
|
56
|
-
?? optionalString(parsed?.error)
|
|
57
|
-
?? (text.length > 0 ? text : response.statusText);
|
|
58
|
-
return {
|
|
59
|
-
error: bounded(error),
|
|
60
|
-
...(requestId === undefined ? {} : { requestId }),
|
|
61
|
-
...(credits === undefined ? {} : { credits }),
|
|
62
|
-
};
|
|
63
|
-
};
|
|
64
|
-
const failed = (outcome, reason, evidence) => ({ outcome, reason, ...evidence });
|
|
65
|
-
export const tavilyExtract = async (url, opts) => {
|
|
66
|
-
const configured = tavilyConfiguration();
|
|
67
|
-
if (configured === null)
|
|
68
|
-
return null;
|
|
69
|
-
opts?.signal?.throwIfAborted();
|
|
70
|
-
const started = performance.now();
|
|
71
|
-
const timeout = AbortSignal.timeout(configured.timeoutMs);
|
|
72
|
-
const signal = opts?.signal === undefined
|
|
73
|
-
? timeout
|
|
74
|
-
: AbortSignal.any([opts.signal, timeout]);
|
|
75
|
-
const interrupted = (cause) => {
|
|
76
|
-
if (opts?.signal?.aborted === true)
|
|
77
|
-
throw opts.signal.reason;
|
|
78
|
-
return timeout.aborted
|
|
79
|
-
? failed("recoverable", "timeout", {
|
|
80
|
-
elapsedMs: elapsed(started),
|
|
81
|
-
error: bounded(timeout.reason),
|
|
82
|
-
})
|
|
83
|
-
: failed("recoverable", "network", {
|
|
84
|
-
elapsedMs: elapsed(started),
|
|
85
|
-
error: bounded(cause),
|
|
86
|
-
});
|
|
87
|
-
};
|
|
88
|
-
let response;
|
|
89
|
-
try {
|
|
90
|
-
response = await fetch("https://api.tavily.com/extract", {
|
|
91
|
-
method: "POST",
|
|
92
|
-
headers: {
|
|
93
|
-
"Content-Type": "application/json",
|
|
94
|
-
"Authorization": `Bearer ${configured.apiKey}`,
|
|
95
|
-
},
|
|
96
|
-
body: JSON.stringify({
|
|
97
|
-
urls: [url],
|
|
98
|
-
extract_depth: configured.depth,
|
|
99
|
-
format: "markdown",
|
|
100
|
-
include_usage: true,
|
|
101
|
-
}),
|
|
102
|
-
signal,
|
|
103
|
-
});
|
|
104
|
-
}
|
|
105
|
-
catch (cause) {
|
|
106
|
-
return interrupted(cause);
|
|
107
|
-
}
|
|
108
|
-
if (!response.ok) {
|
|
109
|
-
let evidence;
|
|
110
|
-
try {
|
|
111
|
-
evidence = await responseError(response);
|
|
112
|
-
}
|
|
113
|
-
catch (cause) {
|
|
114
|
-
if (opts?.signal?.aborted === true || timeout.aborted)
|
|
115
|
-
return interrupted(cause);
|
|
116
|
-
evidence = { error: bounded(cause) };
|
|
117
|
-
}
|
|
118
|
-
const common = {
|
|
119
|
-
status: response.status,
|
|
120
|
-
elapsedMs: elapsed(started),
|
|
121
|
-
...evidence,
|
|
122
|
-
...(response.headers.get("retry-after") === null
|
|
123
|
-
? {}
|
|
124
|
-
: { retryAfter: response.headers.get("retry-after") }),
|
|
125
|
-
};
|
|
126
|
-
if (response.status === 401 || response.status === 403) {
|
|
127
|
-
return failed("hard", "authentication", common);
|
|
128
|
-
}
|
|
129
|
-
if (response.status === 429)
|
|
130
|
-
return failed("recoverable", "rate-limit", common);
|
|
131
|
-
if (response.status >= 500)
|
|
132
|
-
return failed("recoverable", "server", common);
|
|
133
|
-
return failed("hard", "provider-rejection", common);
|
|
134
|
-
}
|
|
135
|
-
let text;
|
|
136
|
-
try {
|
|
137
|
-
text = await response.text();
|
|
138
|
-
}
|
|
139
|
-
catch (cause) {
|
|
140
|
-
return interrupted(cause);
|
|
141
|
-
}
|
|
142
|
-
let data;
|
|
143
|
-
try {
|
|
144
|
-
data = objectRecord(JSON.parse(text));
|
|
145
|
-
}
|
|
146
|
-
catch (cause) {
|
|
147
|
-
return failed("hard", "malformed-response", {
|
|
148
|
-
status: response.status,
|
|
149
|
-
elapsedMs: elapsed(started),
|
|
150
|
-
error: bounded(cause),
|
|
151
|
-
});
|
|
152
|
-
}
|
|
153
|
-
if (data === null) {
|
|
154
|
-
return failed("hard", "malformed-response", {
|
|
155
|
-
status: response.status,
|
|
156
|
-
elapsedMs: elapsed(started),
|
|
157
|
-
error: "Tavily Extract returned a non-object payload.",
|
|
158
|
-
});
|
|
159
|
-
}
|
|
160
|
-
const requestId = optionalString(data.request_id);
|
|
161
|
-
const credits = reportedCredits(data.usage);
|
|
162
|
-
const common = {
|
|
163
|
-
status: response.status,
|
|
164
|
-
elapsedMs: elapsed(started),
|
|
165
|
-
...(requestId === undefined ? {} : { requestId }),
|
|
166
|
-
...(credits === undefined ? {} : { credits }),
|
|
167
|
-
};
|
|
168
|
-
const results = Array.isArray(data.results) ? data.results : [];
|
|
169
|
-
const failedResults = Array.isArray(data.failed_results) ? data.failed_results : [];
|
|
170
|
-
if (requestId === undefined || credits === undefined
|
|
171
|
-
|| (!Array.isArray(data.results) && !Array.isArray(data.failed_results))) {
|
|
172
|
-
return failed("hard", "malformed-response", {
|
|
173
|
-
...common,
|
|
174
|
-
error: "Tavily Extract omitted required results, request_id, or usage.credits evidence.",
|
|
175
|
-
});
|
|
176
|
-
}
|
|
177
|
-
const result = objectRecord(results[0]);
|
|
178
|
-
const markdown = typeof result?.markdown === "string"
|
|
179
|
-
? result.markdown
|
|
180
|
-
: typeof result?.raw_content === "string"
|
|
181
|
-
? result.raw_content
|
|
182
|
-
: undefined;
|
|
183
|
-
if (markdown !== undefined) {
|
|
184
|
-
const sourceUrl = optionalString(result?.url);
|
|
185
|
-
return {
|
|
186
|
-
outcome: "success",
|
|
187
|
-
markdown,
|
|
188
|
-
...common,
|
|
189
|
-
...(sourceUrl === undefined ? {} : { sourceUrl }),
|
|
190
|
-
};
|
|
191
|
-
}
|
|
192
|
-
const failedResult = objectRecord(failedResults[0]);
|
|
193
|
-
if (failedResult !== null) {
|
|
194
|
-
return failed("recoverable", "failed-result", {
|
|
195
|
-
...common,
|
|
196
|
-
error: bounded(optionalString(failedResult.error) ?? "Tavily Extract could not extract the URL."),
|
|
197
|
-
});
|
|
198
|
-
}
|
|
199
|
-
return failed("hard", "malformed-response", {
|
|
200
|
-
...common,
|
|
201
|
-
error: "Tavily Extract returned neither Markdown nor a failed_results occurrence.",
|
|
202
|
-
});
|
|
203
|
-
};
|
|
204
|
-
//# sourceMappingURL=TavilyExtract.js.map
|
|
@@ -1 +0,0 @@
|
|
|
1
|
-
{"version":3,"file":"TavilyExtract.js","sourceRoot":"","sources":["../src/TavilyExtract.ts"],"names":[],"mappings":"AAAA,OAAO,WAAW,MAAM,kBAAkB,CAAC;AAC3C,OAAO,EAAE,yBAAyB,EAAE,MAAM,aAAa,CAAC;AAExD,2DAA2D;AAC3D,MAAM,CAAC,MAAM,YAAY,GAAG,kCAAkC,CAAC;AAC/D,MAAM,CAAC,MAAM,iBAAiB,GAAG,uCAAuC,CAAC;AA0CzE,MAAM,KAAK,GAAG,GAAgB,EAAE;IAC5B,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC;IACxC,IAAI,KAAK,KAAK,OAAO,IAAI,KAAK,KAAK,UAAU,EAAE,CAAC;QAC5C,MAAM,IAAI,KAAK,CAAC,GAAG,YAAY,uCAAuC,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IACpG,CAAC;IACD,OAAO,KAAK,CAAC;AACjB,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,mBAAmB,GAAG,GAA+B,EAAE;IAChE,MAAM,eAAe,GAAG,KAAK,EAAE,CAAC;IAChC,MAAM,SAAS,GAAG,yBAAyB,CAAC,iBAAiB,CAAC,CAAC;IAC/D,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,cAAc,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC;IACxD,IAAI,MAAM,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IACrC,OAAO;QACH,MAAM;QACN,KAAK,EAAE,eAAe;QACtB,SAAS;QACT,QAAQ,EAAE,qBAAqB,eAAe,EAAE;KACnD,CAAC;AACN,CAAC,CAAC;AAEF,MAAM,CAAC,MAAM,qBAAqB,GAAG,GAAkB,EAAE,CAAC,mBAAmB,EAAE,EAAE,QAAQ,IAAI,IAAI,CAAC;AAElG,MAAM,OAAO,GAAG,CAAC,OAAe,EAAU,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,KAAK,CAAC,WAAW,CAAC,GAAG,EAAE,GAAG,OAAO,CAAC,CAAC,CAAC;AAElG,MAAM,OAAO,GAAG,CAAC,KAAc,EAAU,EAAE,CAAC,WAAW,CAAC,OAAO,CAAC,KAAK,EAAE,WAAW,CAAC,eAAe,EAAE,CAAC,CAAC;AAEtG,MAAM,YAAY,GAAG,CAAC,KAAc,EAAkC,EAAE,CAAC,KAAK,KAAK,IAAI;OAChF,OAAO,KAAK,KAAK,QAAQ;OACzB,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC;IACxB,CAAC,CAAC,KAAgC;IAClC,CAAC,CAAC,IAAI,CAAC;AAEX,MAAM,cAAc,GAAG,CAAC,KAAc,EAAsB,EAAE,CAAC,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,CAAC,MAAM,GAAG,CAAC;IACxG,CAAC,CAAC,KAAK;IACP,CAAC,CAAC,SAAS,CAAC;AAEhB,MAAM,eAAe,GAAG,CAAC,KAAc,EAAsB,EAAE;IAC3D,MAAM,KAAK,GAAG,YAAY,CAAC,KAAK,CAAC,CAAC;IAClC,MAAM,OAAO,GAAG,KAAK,EAAE,OAAO,CAAC;IAC/B,OAAO,OAAO,OAAO,KAAK,QAAQ,IAAI,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,IAAI,OAAO,IAAI,CAAC;QAC1E,CAAC,CAAC,OAAO;QACT,CAAC,CAAC,SAAS,CAAC;AACpB,CAAC,CAAC;AAEF,MAAM,aAAa,GAAG,KAAK,EAAE,QAAkB,EAI5C,EAAE;IACD,MAAM,IAAI,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;IACnC,IAAI,MAAM,GAAmC,IAAI,CAAC;IAClD,IAAI,CAAC;QAAC,MAAM,GAAG,YAAY,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC;IAAC,CAAC;IAAC,MAAM,CAAC,CAAA,CAAC;IACzD,MAAM,SAAS,GAAG,cAAc,CAAC,MAAM,EAAE,UAAU,CAAC;WAC7C,cAAc,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,CAAC,CAAC,CAAC;IAC5D,MAAM,OAAO,GAAG,eAAe,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;IAC/C,MAAM,KAAK,GAAG,cAAc,CAAC,MAAM,EAAE,MAAM,CAAC;WACrC,cAAc,CAAC,MAAM,EAAE,OAAO,CAAC;WAC/B,cAAc,CAAC,MAAM,EAAE,KAAK,CAAC;WAC7B,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC;IACtD,OAAO;QACH,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC;QACrB,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC;QACjD,GAAG,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC;KAChD,CAAC;AACN,CAAC,CAAC;AAEF,MAAM,MAAM,GAAG,CACX,OAAwC,EACxC,MAA2B,EAC3B,QAA0D,EACtC,EAAE,CAAC,CAAC,EAAE,OAAO,EAAE,MAAM,EAAE,GAAG,QAAQ,EAAE,CAAC,CAAC;AAE9D,MAAM,CAAC,MAAM,aAAa,GAAG,KAAK,EAC9B,GAAW,EACX,IAA+B,EACI,EAAE;IACrC,MAAM,UAAU,GAAG,mBAAmB,EAAE,CAAC;IACzC,IAAI,UAAU,KAAK,IAAI;QAAE,OAAO,IAAI,CAAC;IACrC,IAAI,EAAE,MAAM,EAAE,cAAc,EAAE,CAAC;IAC/B,MAAM,OAAO,GAAG,WAAW,CAAC,GAAG,EAAE,CAAC;IAClC,MAAM,OAAO,GAAG,WAAW,CAAC,OAAO,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;IAC1D,MAAM,MAAM,GAAG,IAAI,EAAE,MAAM,KAAK,SAAS;QACrC,CAAC,CAAC,OAAO;QACT,CAAC,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,IAAI,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC,CAAC;IAC9C,MAAM,WAAW,GAAG,CAAC,KAAc,EAAwB,EAAE;QACzD,IAAI,IAAI,EAAE,MAAM,EAAE,OAAO,KAAK,IAAI;YAAE,MAAM,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC;QAC7D,OAAO,OAAO,CAAC,OAAO;YAClB,CAAC,CAAC,MAAM,CAAC,aAAa,EAAE,SAAS,EAAE;gBAC/B,SAAS,EAAE,OAAO,CAAC,OAAO,CAAC;gBAC3B,KAAK,EAAE,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC;aACjC,CAAC;YACF,CAAC,CAAC,MAAM,CAAC,aAAa,EAAE,SAAS,EAAE;gBAC/B,SAAS,EAAE,OAAO,CAAC,OAAO,CAAC;gBAC3B,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC;aACxB,CAAC,CAAC;IACX,CAAC,CAAC;IAEF,IAAI,QAAkB,CAAC;IACvB,IAAI,CAAC;QACD,QAAQ,GAAG,MAAM,KAAK,CAAC,gCAAgC,EAAE;YACrD,MAAM,EAAE,MAAM;YACd,OAAO,EAAE;gBACL,cAAc,EAAE,kBAAkB;gBAClC,eAAe,EAAE,UAAU,UAAU,CAAC,MAAM,EAAE;aACjD;YACD,IAAI,EAAE,IAAI,CAAC,SAAS,CAAC;gBACjB,IAAI,EAAE,CAAC,GAAG,CAAC;gBACX,aAAa,EAAE,UAAU,CAAC,KAAK;gBAC/B,MAAM,EAAE,UAAU;gBAClB,aAAa,EAAE,IAAI;aACtB,CAAC;YACF,MAAM;SACT,CAAC,CAAC;IACP,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACb,OAAO,WAAW,CAAC,KAAK,CAAC,CAAC;IAC9B,CAAC;IAED,IAAI,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAC;QACf,IAAI,QAAmD,CAAC;QACxD,IAAI,CAAC;YACD,QAAQ,GAAG,MAAM,aAAa,CAAC,QAAQ,CAAC,CAAC;QAC7C,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACb,IAAI,IAAI,EAAE,MAAM,EAAE,OAAO,KAAK,IAAI,IAAI,OAAO,CAAC,OAAO;gBAAE,OAAO,WAAW,CAAC,KAAK,CAAC,CAAC;YACjF,QAAQ,GAAG,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;QACzC,CAAC;QACD,MAAM,MAAM,GAAG;YACX,MAAM,EAAE,QAAQ,CAAC,MAAM;YACvB,SAAS,EAAE,OAAO,CAAC,OAAO,CAAC;YAC3B,GAAG,QAAQ;YACX,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,aAAa,CAAC,KAAK,IAAI;gBAC5C,CAAC,CAAC,EAAE;gBACJ,CAAC,CAAC,EAAE,UAAU,EAAE,QAAQ,CAAC,OAAO,CAAC,GAAG,CAAC,aAAa,CAAE,EAAE,CAAC;SAC9D,CAAC;QACF,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG,EAAE,CAAC;YACrD,OAAO,MAAM,CAAC,MAAM,EAAE,gBAAgB,EAAE,MAAM,CAAC,CAAC;QACpD,CAAC;QACD,IAAI,QAAQ,CAAC,MAAM,KAAK,GAAG;YAAE,OAAO,MAAM,CAAC,aAAa,EAAE,YAAY,EAAE,MAAM,CAAC,CAAC;QAChF,IAAI,QAAQ,CAAC,MAAM,IAAI,GAAG;YAAE,OAAO,MAAM,CAAC,aAAa,EAAE,QAAQ,EAAE,MAAM,CAAC,CAAC;QAC3E,OAAO,MAAM,CAAC,MAAM,EAAE,oBAAoB,EAAE,MAAM,CAAC,CAAC;IACxD,CAAC;IAED,IAAI,IAAY,CAAC;IACjB,IAAI,CAAC;QACD,IAAI,GAAG,MAAM,QAAQ,CAAC,IAAI,EAAE,CAAC;IACjC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACb,OAAO,WAAW,CAAC,KAAK,CAAC,CAAC;IAC9B,CAAC;IACD,IAAI,IAAoC,CAAC;IACzC,IAAI,CAAC;QACD,IAAI,GAAG,YAAY,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC;IAC1C,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACb,OAAO,MAAM,CAAC,MAAM,EAAE,oBAAoB,EAAE;YACxC,MAAM,EAAE,QAAQ,CAAC,MAAM;YACvB,SAAS,EAAE,OAAO,CAAC,OAAO,CAAC;YAC3B,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC;SACxB,CAAC,CAAC;IACP,CAAC;IACD,IAAI,IAAI,KAAK,IAAI,EAAE,CAAC;QAChB,OAAO,MAAM,CAAC,MAAM,EAAE,oBAAoB,EAAE;YACxC,MAAM,EAAE,QAAQ,CAAC,MAAM;YACvB,SAAS,EAAE,OAAO,CAAC,OAAO,CAAC;YAC3B,KAAK,EAAE,+CAA+C;SACzD,CAAC,CAAC;IACP,CAAC;IAED,MAAM,SAAS,GAAG,cAAc,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;IAClD,MAAM,OAAO,GAAG,eAAe,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IAC5C,MAAM,MAAM,GAAG;QACX,MAAM,EAAE,QAAQ,CAAC,MAAM;QACvB,SAAS,EAAE,OAAO,CAAC,OAAO,CAAC;QAC3B,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC;QACjD,GAAG,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC;KAChD,CAAC;IACF,MAAM,OAAO,GAAG,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IAChE,MAAM,aAAa,GAAG,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC,EAAE,CAAC;IACpF,IAAI,SAAS,KAAK,SAAS,IAAI,OAAO,KAAK,SAAS;WAC7C,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC,EAAE,CAAC;QAC3E,OAAO,MAAM,CAAC,MAAM,EAAE,oBAAoB,EAAE;YACxC,GAAG,MAAM;YACT,KAAK,EAAE,iFAAiF;SAC3F,CAAC,CAAC;IACP,CAAC;IAED,MAAM,MAAM,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;IACxC,MAAM,QAAQ,GAAG,OAAO,MAAM,EAAE,QAAQ,KAAK,QAAQ;QACjD,CAAC,CAAC,MAAM,CAAC,QAAQ;QACjB,CAAC,CAAC,OAAO,MAAM,EAAE,WAAW,KAAK,QAAQ;YACrC,CAAC,CAAC,MAAM,CAAC,WAAW;YACpB,CAAC,CAAC,SAAS,CAAC;IACpB,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QACzB,MAAM,SAAS,GAAG,cAAc,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC;QAC9C,OAAO;YACH,OAAO,EAAE,SAAS;YAClB,QAAQ;YACR,GAAG,MAAM;YACT,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,SAAS,EAAE,CAAC;SACpD,CAAC;IACN,CAAC;IAED,MAAM,YAAY,GAAG,YAAY,CAAC,aAAa,CAAC,CAAC,CAAC,CAAC,CAAC;IACpD,IAAI,YAAY,KAAK,IAAI,EAAE,CAAC;QACxB,OAAO,MAAM,CAAC,aAAa,EAAE,eAAe,EAAE;YAC1C,GAAG,MAAM;YACT,KAAK,EAAE,OAAO,CAAC,cAAc,CAAC,YAAY,CAAC,KAAK,CAAC,IAAI,2CAA2C,CAAC;SACpG,CAAC,CAAC;IACP,CAAC;IACD,OAAO,MAAM,CAAC,MAAM,EAAE,oBAAoB,EAAE;QACxC,GAAG,MAAM;QACT,KAAK,EAAE,2EAA2E;KACrF,CAAC,CAAC;AACP,CAAC,CAAC"}
|