flex-race 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +105 -0
- package/dist/index.d.ts +4 -0
- package/dist/index.d.ts.map +1 -0
- package/dist/index.js +3 -0
- package/dist/index.js.map +1 -0
- package/dist/race.d.ts +17 -0
- package/dist/race.d.ts.map +1 -0
- package/dist/race.js +66 -0
- package/dist/race.js.map +1 -0
- package/dist/start-by.d.ts +11 -0
- package/dist/start-by.d.ts.map +1 -0
- package/dist/start-by.js +68 -0
- package/dist/start-by.js.map +1 -0
- package/dist/with-flex.d.ts +28 -0
- package/dist/with-flex.d.ts.map +1 -0
- package/dist/with-flex.js +140 -0
- package/dist/with-flex.js.map +1 -0
- package/package.json +62 -0
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Aditya Perswal
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# flex-race
|
|
2
|
+
|
|
3
|
+
flex-race lets your app try OpenAI's flex tier with a set wait for it to start. If flex takes too long or fails before it starts, the wrapper sends the request to the default tier. This gives your app a way to use flex while limiting the wait for flex to accept the work. The [wrapper](src/with-flex.ts) adds start_by to responses.create on an OpenAI client.
|
|
4
|
+
|
|
5
|
+
## Install
|
|
6
|
+
|
|
7
|
+
The [package](package.json) needs Node.js 20 or later and openai 6 or later. It exports an ES module. Install it and the OpenAI SDK from npm:
|
|
8
|
+
|
|
9
|
+
```sh
|
|
10
|
+
npm install flex-race openai
|
|
11
|
+
npm install --save-dev tsx
|
|
12
|
+
```
|
|
13
|
+
|
|
14
|
+
## Usage
|
|
15
|
+
|
|
16
|
+
Save this TypeScript example as example.ts. It sets start_by to an ISO string with a timezone. The time is 120 seconds from the call.
|
|
17
|
+
|
|
18
|
+
```ts
|
|
19
|
+
import OpenAI from "openai";
|
|
20
|
+
import { withFlex } from "flex-race";
|
|
21
|
+
|
|
22
|
+
async function main() {
|
|
23
|
+
const client = withFlex(new OpenAI(), {
|
|
24
|
+
onOutcome(outcome) {
|
|
25
|
+
console.log(outcome);
|
|
26
|
+
},
|
|
27
|
+
});
|
|
28
|
+
|
|
29
|
+
const response = await client.responses.create({
|
|
30
|
+
model: "gpt-6-luna",
|
|
31
|
+
input: "Reply with the single word: ok",
|
|
32
|
+
start_by: new Date(Date.now() + 120_000).toISOString(),
|
|
33
|
+
});
|
|
34
|
+
|
|
35
|
+
console.log(response.output_text);
|
|
36
|
+
}
|
|
37
|
+
|
|
38
|
+
main().catch((error: unknown) => {
|
|
39
|
+
console.error(error);
|
|
40
|
+
process.exitCode = 1;
|
|
41
|
+
});
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
Set OPENAI_API_KEY to your key, then run the example:
|
|
45
|
+
|
|
46
|
+
```sh
|
|
47
|
+
export OPENAI_API_KEY="your-api-key"
|
|
48
|
+
npx tsx example.ts
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
For a stream, set stream to true in the request. Use for await to read the returned stream. The wrapper returns the flex events it read while waiting before it returns the rest of the events. Without start_by, the wrapper passes the request to the client and does not call onOutcome. Other client methods stay in place, including responses.retrieve. The [wrapper](src/with-flex.ts) returns a plain Promise from responses.create.
|
|
52
|
+
|
|
53
|
+
## How the wait works
|
|
54
|
+
|
|
55
|
+
start_by sets the time by which flex must admit the request. Admission means the wrapper has read an event after response.in_progress without a thrown error or a response.failed event. On gpt-4o, the live check saw response.in_progress followed by invalid_request_error. This is why [the race](src/race.ts) waits for the next event before it counts admission.
|
|
56
|
+
|
|
57
|
+
The wrapper uses a stream for every flex attempt, even when your app asks for a full response. It counts the time spent opening the stream as part of the wait. Once flex admits the request, the timer stops. For a full response, the wrapper then reads until response.completed or response.incomplete. start_by does not set a time by which the response must finish. The default request can also run past start_by. See [the race](src/race.ts) and [the wrapper](src/with-flex.ts).
|
|
58
|
+
|
|
59
|
+
The [time parser](src/start-by.ts) requires an ISO 8601 string with seconds and Z or a timezone offset such as +05:00. Bad dates, strings with no timezone, Date objects, and numbers cause StartByError before any request. Passing start_by with a non-null service_tier also causes StartByError.
|
|
60
|
+
|
|
61
|
+
The wrapper skips flex when less than 5 seconds remain, including when start_by has passed. It caps the wait at 10 minutes when start_by is farther away. These are local code limits set by MIN_LEAD_MS and MAX_WINDOW_MS in [the time parser](src/start-by.ts). The clock defaults to Date.now. The now option lets tests supply a clock.
|
|
62
|
+
|
|
63
|
+
## Fallback and errors
|
|
64
|
+
|
|
65
|
+
If the wait ends before admission, the wrapper aborts the flex attempt and starts a default request. It does the same if flex throws an error, sends response.failed, or ends the stream before admission. The wrapper tries any model on flex when enough time remains. It keeps no list of models with a flex tier. A model with no flex tier thus adds a flex attempt before the default request. See [the fallback code](src/with-flex.ts).
|
|
66
|
+
|
|
67
|
+
After admission, a full response and a stream have different failure rules. For a full response, the wrapper starts a default request if flex sends response.failed or ends without a final response. It also retries thrown errors on the default tier unless the error has type invalid_request_error or the caller has aborted. For a stream, it passes later events and errors to the caller without starting a default request. The [wrapper](src/with-flex.ts) passes errors from the default request to the caller.
|
|
68
|
+
|
|
69
|
+
Your app supplies the OpenAI client and its credentials. The wrapper adds no access checks or stored records. It removes start_by before sending the body to OpenAI. For flex, it sets service_tier to flex, stream to true, and maxRetries to 0. For fallback, it sets service_tier to default and keeps the caller's stream choice and request options. The caller's abort signal also aborts the flex attempt. An abort during the wait ends the call without fallback. See [the request and abort code](src/with-flex.ts).
|
|
70
|
+
|
|
71
|
+
## Outcomes
|
|
72
|
+
|
|
73
|
+
The onOutcome option receives a FlexOutcome with tier, reason, waitedMs, and capped. tier is flex or default. capped is true when the wrapper cut the wait to MAX_WINDOW_MS. waitedMs counts from the call to the report. For a flex stream, the report comes at admission. For a full flex response, it comes after the final response. A report for the default tier comes before that request finishes, so it does not prove that the request passed. See [the outcome code](src/with-flex.ts).
|
|
74
|
+
|
|
75
|
+
The reason is admitted when the wrapper uses flex. It is deadline_too_near when the wrapper skips flex, or deadline_elapsed when the wait runs out. It is flex_refused when flex fails before admission. It is flex_failed_after_admission when a full response fails after admission and the wrapper starts a default request. Calls without start_by, bad input, and errors that end the call before a tier report do not call onOutcome. The [public exports](src/index.ts) include the outcome types and StartByError.
|
|
76
|
+
|
|
77
|
+
## Tests and live results
|
|
78
|
+
|
|
79
|
+
Run the unit tests from this repo:
|
|
80
|
+
|
|
81
|
+
```sh
|
|
82
|
+
pnpm test
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The [time tests](src/start-by.test.ts) check valid times, bad input, and the wait cap. The [wrapper tests](src/with-flex.test.ts) check admission, event order, fallback, errors, and aborts. They use a simulated clock to check that the wrapper aborts a waiting flex attempt when start_by passes. The confirmed run on 2026-09-26 passed 57 unit tests and covered every source line.
|
|
86
|
+
|
|
87
|
+
The live tests read OPENAI_API_KEY from a local .env file through the [live test config](vitest.live.config.ts). Add your key to that file:
|
|
88
|
+
|
|
89
|
+
```dotenv
|
|
90
|
+
OPENAI_API_KEY=your-api-key
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
Then run:
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
pnpm test:live
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
The [live tests](test/openai.live.test.ts) skip when no key is set. They call the live OpenAI API. They check full responses, streams, fallback for a model with no flex tier, and a start_by too near to try flex. For full responses and streams, they check that the served tier matches onOutcome. The confirmed run on 2026-09-26 passed all 4 live tests.
|
|
100
|
+
|
|
101
|
+
Separate checks on 2026-09-26 installed the packed tarball in another project and called the live OpenAI API. With start_by 120 seconds ahead, gpt-6-luna returned a full response on flex with reason admitted and waitedMs 990. A gpt-6-sol stream used flex with reason admitted and waitedMs 647. That stream had 9 events. A gpt-4o request used the default tier with reason flex_refused and waitedMs 638.
|
|
102
|
+
|
|
103
|
+
Those checks also saw deadline_too_near on the default tier with start_by 2 seconds ahead. A call without start_by passed through with no onOutcome call. A start_by with no timezone threw StartByError before any request. responses.retrieve also worked through the wrapper.
|
|
104
|
+
|
|
105
|
+
The path where start_by passes while flex still waits has not been seen live. Live flex admissions took about one second. The simulated clock tests cover that path. These measured times do not set a speed target or a guarantee. A live check also saw gpt-6-astra served on flex. What OpenAI bills for that request is unknown because its pricing page listed no flex price.
|
package/dist/index.d.ts
ADDED
|
@@ -0,0 +1,4 @@
|
|
|
1
|
+
export { withFlex } from "./with-flex.js";
|
|
2
|
+
export type { FlexClient, FlexOutcome, FlexReason, FlexResponses, WithFlexOptions, WithStartBy, } from "./with-flex.js";
|
|
3
|
+
export { MAX_WINDOW_MS, MIN_LEAD_MS, StartByError } from "./start-by.js";
|
|
4
|
+
//# sourceMappingURL=index.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,gBAAgB,CAAC;AAC1C,YAAY,EACV,UAAU,EACV,WAAW,EACX,UAAU,EACV,aAAa,EACb,eAAe,EACf,WAAW,GACZ,MAAM,gBAAgB,CAAC;AACxB,OAAO,EAAE,aAAa,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC"}
|
package/dist/index.js
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,gBAAgB,CAAC;AAS1C,OAAO,EAAE,aAAa,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC"}
|
package/dist/race.d.ts
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
export declare const IN_PROGRESS_EVENT = "response.in_progress";
|
|
2
|
+
export type RaceResult<Event> = {
|
|
3
|
+
readonly kind: "admitted";
|
|
4
|
+
readonly buffered: readonly Event[];
|
|
5
|
+
readonly rest: AsyncIterator<Event>;
|
|
6
|
+
} | {
|
|
7
|
+
readonly kind: "elapsed";
|
|
8
|
+
} | {
|
|
9
|
+
readonly kind: "refused";
|
|
10
|
+
readonly cause: unknown;
|
|
11
|
+
};
|
|
12
|
+
interface TypedEvent {
|
|
13
|
+
readonly type: string;
|
|
14
|
+
}
|
|
15
|
+
export declare function raceForAdmission<Event extends TypedEvent>(open: () => Promise<AsyncIterable<Event>>, msLeft: number): Promise<RaceResult<Event>>;
|
|
16
|
+
export {};
|
|
17
|
+
//# sourceMappingURL=race.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"race.d.ts","sourceRoot":"","sources":["../src/race.ts"],"names":[],"mappings":"AAUA,eAAO,MAAM,iBAAiB,yBAAyB,CAAC;AAExD,MAAM,MAAM,UAAU,CAAC,KAAK,IAGxB;IAAE,QAAQ,CAAC,IAAI,EAAE,UAAU,CAAC;IAAC,QAAQ,CAAC,QAAQ,EAAE,SAAS,KAAK,EAAE,CAAC;IAAC,QAAQ,CAAC,IAAI,EAAE,aAAa,CAAC,KAAK,CAAC,CAAA;CAAE,GAEvG;IAAE,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAA;CAAE,GAI5B;IAAE,QAAQ,CAAC,IAAI,EAAE,SAAS,CAAC;IAAC,QAAQ,CAAC,KAAK,EAAE,OAAO,CAAA;CAAE,CAAC;AAE1D,UAAU,UAAU;IAClB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;CACvB;AAMD,wBAAsB,gBAAgB,CAAC,KAAK,SAAS,UAAU,EAC7D,IAAI,EAAE,MAAM,OAAO,CAAC,aAAa,CAAC,KAAK,CAAC,CAAC,EACzC,MAAM,EAAE,MAAM,GACb,OAAO,CAAC,UAAU,CAAC,KAAK,CAAC,CAAC,CAgC5B"}
|
package/dist/race.js
ADDED
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
// Wait for the flex attempt to be admitted, or give up on it.
|
|
2
|
+
//
|
|
3
|
+
// OpenAI streams `response.created` as soon as it accepts a flex request, then `response.in_progress`
|
|
4
|
+
// once flex capacity picks the request up. `response.in_progress` alone is not proof of admission: on a
|
|
5
|
+
// model with no flex tier (gpt-4o, seen live on 2026-09-26) OpenAI sends it and then an
|
|
6
|
+
// `invalid_request_error` straight after. So the request counts as admitted at the first event AFTER
|
|
7
|
+
// `response.in_progress` that is not a failure. On a real admission that event follows at once and
|
|
8
|
+
// still comes before the first output token, so the deadline measures time to admission, not time to
|
|
9
|
+
// first token.
|
|
10
|
+
export const IN_PROGRESS_EVENT = "response.in_progress";
|
|
11
|
+
const ELAPSED = Symbol("elapsed");
|
|
12
|
+
// `open` starts the flex request. It runs inside the deadline, so a slow connection counts against
|
|
13
|
+
// start_by the same way a slow queue does.
|
|
14
|
+
export async function raceForAdmission(open, msLeft) {
|
|
15
|
+
const timer = deadline(msLeft);
|
|
16
|
+
try {
|
|
17
|
+
const opened = open();
|
|
18
|
+
const stream = await Promise.race([opened, timer.elapsed]);
|
|
19
|
+
if (stream === ELAPSED) {
|
|
20
|
+
opened.catch(() => undefined); // the caller aborts it; swallow the rejection that follows
|
|
21
|
+
return { kind: "elapsed" };
|
|
22
|
+
}
|
|
23
|
+
const iterator = stream[Symbol.asyncIterator]();
|
|
24
|
+
const buffered = [];
|
|
25
|
+
let inProgress = false;
|
|
26
|
+
for (;;) {
|
|
27
|
+
const next = iterator.next();
|
|
28
|
+
const step = await Promise.race([next, timer.elapsed]);
|
|
29
|
+
if (step === ELAPSED) {
|
|
30
|
+
next.catch(() => undefined);
|
|
31
|
+
return { kind: "elapsed" };
|
|
32
|
+
}
|
|
33
|
+
if (step.done === true) {
|
|
34
|
+
return { kind: "refused", cause: new Error("flex stream ended before admission") };
|
|
35
|
+
}
|
|
36
|
+
buffered.push(step.value);
|
|
37
|
+
if (step.value.type === "response.failed")
|
|
38
|
+
return { kind: "refused", cause: step.value };
|
|
39
|
+
if (inProgress)
|
|
40
|
+
return { kind: "admitted", buffered, rest: iterator };
|
|
41
|
+
if (step.value.type === IN_PROGRESS_EVENT)
|
|
42
|
+
inProgress = true;
|
|
43
|
+
}
|
|
44
|
+
}
|
|
45
|
+
catch (cause) {
|
|
46
|
+
return { kind: "refused", cause };
|
|
47
|
+
}
|
|
48
|
+
finally {
|
|
49
|
+
timer.cancel();
|
|
50
|
+
}
|
|
51
|
+
}
|
|
52
|
+
function deadline(ms) {
|
|
53
|
+
let id;
|
|
54
|
+
const elapsed = new Promise((resolve) => {
|
|
55
|
+
id = setTimeout(() => {
|
|
56
|
+
resolve(ELAPSED);
|
|
57
|
+
}, Math.max(0, ms));
|
|
58
|
+
});
|
|
59
|
+
return {
|
|
60
|
+
elapsed,
|
|
61
|
+
cancel: () => {
|
|
62
|
+
clearTimeout(id);
|
|
63
|
+
},
|
|
64
|
+
};
|
|
65
|
+
}
|
|
66
|
+
//# sourceMappingURL=race.js.map
|
package/dist/race.js.map
ADDED
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"race.js","sourceRoot":"","sources":["../src/race.ts"],"names":[],"mappings":"AAAA,8DAA8D;AAC9D,EAAE;AACF,sGAAsG;AACtG,wGAAwG;AACxG,wFAAwF;AACxF,qGAAqG;AACrG,mGAAmG;AACnG,qGAAqG;AACrG,eAAe;AAEf,MAAM,CAAC,MAAM,iBAAiB,GAAG,sBAAsB,CAAC;AAiBxD,MAAM,OAAO,GAAG,MAAM,CAAC,SAAS,CAAC,CAAC;AAElC,mGAAmG;AACnG,2CAA2C;AAC3C,MAAM,CAAC,KAAK,UAAU,gBAAgB,CACpC,IAAyC,EACzC,MAAc;IAEd,MAAM,KAAK,GAAG,QAAQ,CAAC,MAAM,CAAC,CAAC;IAC/B,IAAI,CAAC;QACH,MAAM,MAAM,GAAG,IAAI,EAAE,CAAC;QACtB,MAAM,MAAM,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC;QAC3D,IAAI,MAAM,KAAK,OAAO,EAAE,CAAC;YACvB,MAAM,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC,CAAC,2DAA2D;YAC1F,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;QAC7B,CAAC;QACD,MAAM,QAAQ,GAAG,MAAM,CAAC,MAAM,CAAC,aAAa,CAAC,EAAE,CAAC;QAChD,MAAM,QAAQ,GAAY,EAAE,CAAC;QAC7B,IAAI,UAAU,GAAG,KAAK,CAAC;QACvB,SAAS,CAAC;YACR,MAAM,IAAI,GAAG,QAAQ,CAAC,IAAI,EAAE,CAAC;YAC7B,MAAM,IAAI,GAAG,MAAM,OAAO,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC;YACvD,IAAI,IAAI,KAAK,OAAO,EAAE,CAAC;gBACrB,IAAI,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;gBAC5B,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,CAAC;YAC7B,CAAC;YACD,IAAI,IAAI,CAAC,IAAI,KAAK,IAAI,EAAE,CAAC;gBACvB,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,IAAI,KAAK,CAAC,oCAAoC,CAAC,EAAE,CAAC;YACrF,CAAC;YACD,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;YAC1B,IAAI,IAAI,CAAC,KAAK,CAAC,IAAI,KAAK,iBAAiB;gBAAE,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC;YACzF,IAAI,UAAU;gBAAE,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,QAAQ,EAAE,IAAI,EAAE,QAAQ,EAAE,CAAC;YACtE,IAAI,IAAI,CAAC,KAAK,CAAC,IAAI,KAAK,iBAAiB;gBAAE,UAAU,GAAG,IAAI,CAAC;QAC/D,CAAC;IACH,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,OAAO,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,CAAC;IACpC,CAAC;YAAS,CAAC;QACT,KAAK,CAAC,MAAM,EAAE,CAAC;IACjB,CAAC;AACH,CAAC;AAED,SAAS,QAAQ,CAAC,EAAU;IAC1B,IAAI,EAA6C,CAAC;IAClD,MAAM,OAAO,GAAG,IAAI,OAAO,CAAiB,CAAC,OAAO,EAAE,EAAE;QACtD,EAAE,GAAG,UAAU,CACb,GAAG,EAAE;YACH,OAAO,CAAC,OAAO,CAAC,CAAC;QACnB,CAAC,EACD,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,CAAC,CAChB,CAAC;IACJ,CAAC,CAAC,CAAC;IACH,OAAO;QACL,OAAO;QACP,MAAM,EAAE,GAAG,EAAE;YACX,YAAY,CAAC,EAAE,CAAC,CAAC;QACnB,CAAC;KACF,CAAC;AACJ,CAAC"}
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
export interface Deadline {
|
|
2
|
+
readonly msLeft: number;
|
|
3
|
+
readonly capped: boolean;
|
|
4
|
+
}
|
|
5
|
+
export declare const MIN_LEAD_MS = 5000;
|
|
6
|
+
export declare const MAX_WINDOW_MS = 600000;
|
|
7
|
+
export declare class StartByError extends Error {
|
|
8
|
+
readonly name = "StartByError";
|
|
9
|
+
}
|
|
10
|
+
export declare function parseStartBy(value: unknown, nowMs: number): Deadline;
|
|
11
|
+
//# sourceMappingURL=start-by.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"start-by.d.ts","sourceRoot":"","sources":["../src/start-by.ts"],"names":[],"mappings":"AAMA,MAAM,WAAW,QAAQ;IAGvB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IAExB,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAC;CAC1B;AAGD,eAAO,MAAM,WAAW,OAAQ,CAAC;AAEjC,eAAO,MAAM,aAAa,SAAU,CAAC;AAErC,qBAAa,YAAa,SAAQ,KAAK;IACrC,SAAkB,IAAI,kBAAkB;CACzC;AAOD,wBAAgB,YAAY,CAAC,KAAK,EAAE,OAAO,EAAE,KAAK,EAAE,MAAM,GAAG,QAAQ,CAKpE"}
|
package/dist/start-by.js
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
// `start_by`: the instant by which OpenAI's flex tier must have admitted the request. It is the only
|
|
2
|
+
// input this library adds to a Responses call, and it accepts exactly one shape: an ISO 8601 date-time
|
|
3
|
+
// with seconds and an explicit timezone (`Z` or `+hh:mm` / `-hh:mm`). A string without a timezone means
|
|
4
|
+
// different instants on different servers, and a Date or a bare number invites "30 means 30 seconds",
|
|
5
|
+
// so both are refused rather than guessed at.
|
|
6
|
+
// Less time than this is not worth a flex attempt: the request goes straight to the default tier.
|
|
7
|
+
export const MIN_LEAD_MS = 5_000;
|
|
8
|
+
// The longest a flex attempt waits for admission, however far away start_by is.
|
|
9
|
+
export const MAX_WINDOW_MS = 600_000;
|
|
10
|
+
export class StartByError extends Error {
|
|
11
|
+
name = "StartByError";
|
|
12
|
+
}
|
|
13
|
+
const ISO_DATE_TIME = /^(\d{4})-(\d{2})-(\d{2})T(\d{2}):(\d{2}):(\d{2})(?:\.(\d{1,9}))?(?:(Z)|([+-])(\d{2}):(\d{2}))$/;
|
|
14
|
+
// Parse start_by and measure it against `nowMs`. Throws StartByError on anything that is not a valid,
|
|
15
|
+
// timezone-qualified ISO 8601 date-time.
|
|
16
|
+
export function parseStartBy(value, nowMs) {
|
|
17
|
+
const targetMs = instantOf(value);
|
|
18
|
+
const msLeft = targetMs - nowMs;
|
|
19
|
+
if (msLeft > MAX_WINDOW_MS)
|
|
20
|
+
return { msLeft: MAX_WINDOW_MS, capped: true };
|
|
21
|
+
return { msLeft, capped: false };
|
|
22
|
+
}
|
|
23
|
+
function instantOf(value) {
|
|
24
|
+
if (typeof value !== "string") {
|
|
25
|
+
throw new StartByError(`start_by must be an ISO 8601 string with a timezone, such as "2026-09-25T18:00:00Z"; got ${describe(value)}`);
|
|
26
|
+
}
|
|
27
|
+
const match = ISO_DATE_TIME.exec(value);
|
|
28
|
+
if (match === null) {
|
|
29
|
+
throw new StartByError(`start_by must be an ISO 8601 date-time with seconds and a timezone ("Z" or "+hh:mm"), such as ` +
|
|
30
|
+
`"2026-09-25T18:00:00Z"; got ${JSON.stringify(value)}`);
|
|
31
|
+
}
|
|
32
|
+
const [, y, mo, d, h, mi, s, fraction, zulu, sign, offH, offM] = match;
|
|
33
|
+
const year = Number(y);
|
|
34
|
+
const month = Number(mo);
|
|
35
|
+
const day = Number(d);
|
|
36
|
+
const hour = Number(h);
|
|
37
|
+
const minute = Number(mi);
|
|
38
|
+
const second = Number(s);
|
|
39
|
+
const millis = fraction === undefined ? 0 : Number(fraction.slice(0, 3).padEnd(3, "0"));
|
|
40
|
+
const utcOfFields = Date.UTC(year, month - 1, day, hour, minute, second, millis);
|
|
41
|
+
const roundTrip = new Date(utcOfFields);
|
|
42
|
+
const fieldsValid = hour < 24 &&
|
|
43
|
+
minute < 60 &&
|
|
44
|
+
second < 60 &&
|
|
45
|
+
roundTrip.getUTCFullYear() === year &&
|
|
46
|
+
roundTrip.getUTCMonth() === month - 1 &&
|
|
47
|
+
roundTrip.getUTCDate() === day;
|
|
48
|
+
if (!fieldsValid)
|
|
49
|
+
throw new StartByError(`start_by is not a real date and time: ${JSON.stringify(value)}`);
|
|
50
|
+
if (zulu !== undefined)
|
|
51
|
+
return utcOfFields;
|
|
52
|
+
const offsetHours = Number(offH);
|
|
53
|
+
const offsetMinutes = Number(offM);
|
|
54
|
+
if (offsetHours > 23 || offsetMinutes > 59) {
|
|
55
|
+
throw new StartByError(`start_by has an impossible timezone offset: ${JSON.stringify(value)}`);
|
|
56
|
+
}
|
|
57
|
+
const offsetMs = (offsetHours * 60 + offsetMinutes) * 60_000;
|
|
58
|
+
// A local time at +05:00 is five hours AHEAD of UTC, so the UTC instant is the fields minus the offset.
|
|
59
|
+
return sign === "+" ? utcOfFields - offsetMs : utcOfFields + offsetMs;
|
|
60
|
+
}
|
|
61
|
+
function describe(value) {
|
|
62
|
+
if (value instanceof Date)
|
|
63
|
+
return "a Date object";
|
|
64
|
+
if (value === null)
|
|
65
|
+
return "null";
|
|
66
|
+
return typeof value;
|
|
67
|
+
}
|
|
68
|
+
//# sourceMappingURL=start-by.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"start-by.js","sourceRoot":"","sources":["../src/start-by.ts"],"names":[],"mappings":"AAAA,qGAAqG;AACrG,uGAAuG;AACvG,wGAAwG;AACxG,sGAAsG;AACtG,8CAA8C;AAU9C,kGAAkG;AAClG,MAAM,CAAC,MAAM,WAAW,GAAG,KAAK,CAAC;AACjC,gFAAgF;AAChF,MAAM,CAAC,MAAM,aAAa,GAAG,OAAO,CAAC;AAErC,MAAM,OAAO,YAAa,SAAQ,KAAK;IACnB,IAAI,GAAG,cAAc,CAAC;CACzC;AAED,MAAM,aAAa,GACjB,gGAAgG,CAAC;AAEnG,sGAAsG;AACtG,yCAAyC;AACzC,MAAM,UAAU,YAAY,CAAC,KAAc,EAAE,KAAa;IACxD,MAAM,QAAQ,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC;IAClC,MAAM,MAAM,GAAG,QAAQ,GAAG,KAAK,CAAC;IAChC,IAAI,MAAM,GAAG,aAAa;QAAE,OAAO,EAAE,MAAM,EAAE,aAAa,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC;IAC3E,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,KAAK,EAAE,CAAC;AACnC,CAAC;AAED,SAAS,SAAS,CAAC,KAAc;IAC/B,IAAI,OAAO,KAAK,KAAK,QAAQ,EAAE,CAAC;QAC9B,MAAM,IAAI,YAAY,CACpB,4FAA4F,QAAQ,CAAC,KAAK,CAAC,EAAE,CAC9G,CAAC;IACJ,CAAC;IACD,MAAM,KAAK,GAAG,aAAa,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACxC,IAAI,KAAK,KAAK,IAAI,EAAE,CAAC;QACnB,MAAM,IAAI,YAAY,CACpB,gGAAgG;YAC9F,+BAA+B,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,EAAE,CACzD,CAAC;IACJ,CAAC;IACD,MAAM,CAAC,EAAE,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,QAAQ,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,IAAI,CAAC,GAAG,KAAK,CAAC;IACvE,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;IACvB,MAAM,KAAK,GAAG,MAAM,CAAC,EAAE,CAAC,CAAC;IACzB,MAAM,GAAG,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;IACtB,MAAM,IAAI,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;IACvB,MAAM,MAAM,GAAG,MAAM,CAAC,EAAE,CAAC,CAAC;IAC1B,MAAM,MAAM,GAAG,MAAM,CAAC,CAAC,CAAC,CAAC;IACzB,MAAM,MAAM,GAAG,QAAQ,KAAK,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC;IAExF,MAAM,WAAW,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,GAAG,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;IACjF,MAAM,SAAS,GAAG,IAAI,IAAI,CAAC,WAAW,CAAC,CAAC;IACxC,MAAM,WAAW,GACf,IAAI,GAAG,EAAE;QACT,MAAM,GAAG,EAAE;QACX,MAAM,GAAG,EAAE;QACX,SAAS,CAAC,cAAc,EAAE,KAAK,IAAI;QACnC,SAAS,CAAC,WAAW,EAAE,KAAK,KAAK,GAAG,CAAC;QACrC,SAAS,CAAC,UAAU,EAAE,KAAK,GAAG,CAAC;IACjC,IAAI,CAAC,WAAW;QAAE,MAAM,IAAI,YAAY,CAAC,yCAAyC,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;IAE3G,IAAI,IAAI,KAAK,SAAS;QAAE,OAAO,WAAW,CAAC;IAC3C,MAAM,WAAW,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC;IACjC,MAAM,aAAa,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC;IACnC,IAAI,WAAW,GAAG,EAAE,IAAI,aAAa,GAAG,EAAE,EAAE,CAAC;QAC3C,MAAM,IAAI,YAAY,CAAC,+CAA+C,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;IACjG,CAAC;IACD,MAAM,QAAQ,GAAG,CAAC,WAAW,GAAG,EAAE,GAAG,aAAa,CAAC,GAAG,MAAM,CAAC;IAC7D,wGAAwG;IACxG,OAAO,IAAI,KAAK,GAAG,CAAC,CAAC,CAAC,WAAW,GAAG,QAAQ,CAAC,CAAC,CAAC,WAAW,GAAG,QAAQ,CAAC;AACxE,CAAC;AAED,SAAS,QAAQ,CAAC,KAAc;IAC9B,IAAI,KAAK,YAAY,IAAI;QAAE,OAAO,eAAe,CAAC;IAClD,IAAI,KAAK,KAAK,IAAI;QAAE,OAAO,MAAM,CAAC;IAClC,OAAO,OAAO,KAAK,CAAC;AACtB,CAAC"}
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
import type OpenAI from "openai";
|
|
2
|
+
import { Stream } from "openai/streaming";
|
|
3
|
+
import type { Response, ResponseCreateParamsNonStreaming, ResponseCreateParamsStreaming, ResponseStreamEvent } from "openai/resources/responses/responses";
|
|
4
|
+
export type FlexReason = "admitted" | "deadline_too_near" | "deadline_elapsed" | "flex_refused" | "flex_failed_after_admission";
|
|
5
|
+
export interface FlexOutcome {
|
|
6
|
+
readonly tier: "flex" | "default";
|
|
7
|
+
readonly reason: FlexReason;
|
|
8
|
+
readonly waitedMs: number;
|
|
9
|
+
readonly capped: boolean;
|
|
10
|
+
}
|
|
11
|
+
export interface WithFlexOptions {
|
|
12
|
+
readonly onOutcome?: (outcome: FlexOutcome) => void;
|
|
13
|
+
readonly now?: () => number;
|
|
14
|
+
}
|
|
15
|
+
type RequestOptions = NonNullable<Parameters<OpenAI["responses"]["create"]>[1]>;
|
|
16
|
+
export type WithStartBy<Params> = Params & {
|
|
17
|
+
readonly start_by?: string;
|
|
18
|
+
};
|
|
19
|
+
export interface FlexResponses {
|
|
20
|
+
create(body: WithStartBy<ResponseCreateParamsNonStreaming>, options?: RequestOptions): Promise<Response>;
|
|
21
|
+
create(body: WithStartBy<ResponseCreateParamsStreaming>, options?: RequestOptions): Promise<Stream<ResponseStreamEvent>>;
|
|
22
|
+
}
|
|
23
|
+
export type FlexClient = Omit<OpenAI, "responses"> & {
|
|
24
|
+
readonly responses: Omit<OpenAI["responses"], "create"> & FlexResponses;
|
|
25
|
+
};
|
|
26
|
+
export declare function withFlex(client: OpenAI, options?: WithFlexOptions): FlexClient;
|
|
27
|
+
export {};
|
|
28
|
+
//# sourceMappingURL=with-flex.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"with-flex.d.ts","sourceRoot":"","sources":["../src/with-flex.ts"],"names":[],"mappings":"AASA,OAAO,KAAK,MAAM,MAAM,QAAQ,CAAC;AACjC,OAAO,EAAE,MAAM,EAAE,MAAM,kBAAkB,CAAC;AAC1C,OAAO,KAAK,EACV,QAAQ,EACR,gCAAgC,EAChC,6BAA6B,EAC7B,mBAAmB,EACpB,MAAM,sCAAsC,CAAC;AAK9C,MAAM,MAAM,UAAU,GAElB,UAAU,GAEV,mBAAmB,GAEnB,kBAAkB,GAGlB,cAAc,GAGd,6BAA6B,CAAC;AAElC,MAAM,WAAW,WAAW;IAC1B,QAAQ,CAAC,IAAI,EAAE,MAAM,GAAG,SAAS,CAAC;IAClC,QAAQ,CAAC,MAAM,EAAE,UAAU,CAAC;IAE5B,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAE1B,QAAQ,CAAC,MAAM,EAAE,OAAO,CAAC;CAC1B;AAED,MAAM,WAAW,eAAe;IAE9B,QAAQ,CAAC,SAAS,CAAC,EAAE,CAAC,OAAO,EAAE,WAAW,KAAK,IAAI,CAAC;IAEpD,QAAQ,CAAC,GAAG,CAAC,EAAE,MAAM,MAAM,CAAC;CAC7B;AAED,KAAK,cAAc,GAAG,WAAW,CAAC,UAAU,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAEhF,MAAM,MAAM,WAAW,CAAC,MAAM,IAAI,MAAM,GAAG;IAEzC,QAAQ,CAAC,QAAQ,CAAC,EAAE,MAAM,CAAC;CAC5B,CAAC;AAEF,MAAM,WAAW,aAAa;IAC5B,MAAM,CAAC,IAAI,EAAE,WAAW,CAAC,gCAAgC,CAAC,EAAE,OAAO,CAAC,EAAE,cAAc,GAAG,OAAO,CAAC,QAAQ,CAAC,CAAC;IACzG,MAAM,CACJ,IAAI,EAAE,WAAW,CAAC,6BAA6B,CAAC,EAChD,OAAO,CAAC,EAAE,cAAc,GACvB,OAAO,CAAC,MAAM,CAAC,mBAAmB,CAAC,CAAC,CAAC;CACzC;AAED,MAAM,MAAM,UAAU,GAAG,IAAI,CAAC,MAAM,EAAE,WAAW,CAAC,GAAG;IACnD,QAAQ,CAAC,SAAS,EAAE,IAAI,CAAC,MAAM,CAAC,WAAW,CAAC,EAAE,QAAQ,CAAC,GAAG,aAAa,CAAC;CACzE,CAAC;AAMF,wBAAgB,QAAQ,CAAC,MAAM,EAAE,MAAM,EAAE,OAAO,GAAE,eAAoB,GAAG,UAAU,CAgBlF"}
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
// withFlex wraps an OpenAI client so `responses.create` accepts one extra field, `start_by`. With it,
|
|
2
|
+
// the request first goes to OpenAI's flex tier (roughly half the price) and waits for flex to admit it.
|
|
3
|
+
// If flex admits it by start_by, the caller gets the flex response. If start_by passes first, or flex
|
|
4
|
+
// refuses or fails before admitting it, the flex attempt is aborted and the request runs on the default
|
|
5
|
+
// tier instead. Without start_by the call passes through untouched.
|
|
6
|
+
//
|
|
7
|
+
// There is no model list: any model is tried on flex, and a model OpenAI will not serve on flex simply
|
|
8
|
+
// falls back. The cost is one extra round trip on such models, and only when start_by is set.
|
|
9
|
+
import { Stream } from "openai/streaming";
|
|
10
|
+
import { raceForAdmission } from "./race.js";
|
|
11
|
+
import { MIN_LEAD_MS, StartByError, parseStartBy } from "./start-by.js";
|
|
12
|
+
export function withFlex(client, options = {}) {
|
|
13
|
+
const responses = client.responses;
|
|
14
|
+
const rawCreate = responses.create.bind(responses);
|
|
15
|
+
const create = makeCreate(rawCreate, options);
|
|
16
|
+
const flexResponses = new Proxy(responses, {
|
|
17
|
+
get(target, property, receiver) {
|
|
18
|
+
if (property === "create")
|
|
19
|
+
return create;
|
|
20
|
+
return Reflect.get(target, property, receiver);
|
|
21
|
+
},
|
|
22
|
+
});
|
|
23
|
+
return new Proxy(client, {
|
|
24
|
+
get(target, property, receiver) {
|
|
25
|
+
if (property === "responses")
|
|
26
|
+
return flexResponses;
|
|
27
|
+
return Reflect.get(target, property, receiver);
|
|
28
|
+
},
|
|
29
|
+
});
|
|
30
|
+
}
|
|
31
|
+
function makeCreate(rawCreate, options) {
|
|
32
|
+
const now = options.now ?? Date.now;
|
|
33
|
+
const report = (outcome) => options.onOutcome?.(outcome);
|
|
34
|
+
async function create(body, requestOptions) {
|
|
35
|
+
const { start_by: startBy, ...params } = body;
|
|
36
|
+
if (startBy === undefined)
|
|
37
|
+
return rawCreate(params, requestOptions);
|
|
38
|
+
if (params.service_tier !== undefined && params.service_tier !== null) {
|
|
39
|
+
throw new StartByError("start_by chooses the service tier; do not also pass service_tier");
|
|
40
|
+
}
|
|
41
|
+
const startedAt = now();
|
|
42
|
+
const { msLeft, capped } = parseStartBy(startBy, startedAt);
|
|
43
|
+
const decided = (tier, reason) => {
|
|
44
|
+
report({ tier, reason, waitedMs: now() - startedAt, capped });
|
|
45
|
+
};
|
|
46
|
+
const runDefault = () => rawCreate({ ...params, service_tier: "default" }, requestOptions);
|
|
47
|
+
if (msLeft < MIN_LEAD_MS) {
|
|
48
|
+
decided("default", "deadline_too_near");
|
|
49
|
+
return runDefault();
|
|
50
|
+
}
|
|
51
|
+
const callerSignal = requestOptions?.signal ?? undefined;
|
|
52
|
+
const flex = linkedController(callerSignal);
|
|
53
|
+
const flexOptions = { ...requestOptions, signal: flex.signal, maxRetries: 0 };
|
|
54
|
+
const result = await raceForAdmission(() => rawCreate({ ...params, stream: true, service_tier: "flex" }, flexOptions), msLeft);
|
|
55
|
+
if (result.kind !== "admitted") {
|
|
56
|
+
flex.abort();
|
|
57
|
+
if (callerSignal?.aborted === true)
|
|
58
|
+
throw callerAbort(callerSignal);
|
|
59
|
+
decided("default", result.kind === "elapsed" ? "deadline_elapsed" : "flex_refused");
|
|
60
|
+
return runDefault();
|
|
61
|
+
}
|
|
62
|
+
if (params.stream === true) {
|
|
63
|
+
decided("flex", "admitted");
|
|
64
|
+
return new Stream(() => replay(result.buffered, result.rest), flex);
|
|
65
|
+
}
|
|
66
|
+
// The admitting event can itself be the final one, so the buffered head is read too.
|
|
67
|
+
const final = await finalResponseOf(replay(result.buffered, result.rest), callerSignal);
|
|
68
|
+
if (final.kind === "response") {
|
|
69
|
+
decided("flex", "admitted");
|
|
70
|
+
return final.response;
|
|
71
|
+
}
|
|
72
|
+
flex.abort();
|
|
73
|
+
if (!final.retryable)
|
|
74
|
+
throw final.cause;
|
|
75
|
+
decided("default", "flex_failed_after_admission");
|
|
76
|
+
return runDefault();
|
|
77
|
+
}
|
|
78
|
+
return create;
|
|
79
|
+
}
|
|
80
|
+
// Drain an admitted flex stream for a non-streaming caller and keep only the finished response.
|
|
81
|
+
async function finalResponseOf(rest, callerSignal) {
|
|
82
|
+
try {
|
|
83
|
+
for (;;) {
|
|
84
|
+
const step = await rest.next();
|
|
85
|
+
if (step.done === true) {
|
|
86
|
+
return {
|
|
87
|
+
kind: "failed",
|
|
88
|
+
cause: new Error("flex stream ended without a final response"),
|
|
89
|
+
retryable: true,
|
|
90
|
+
};
|
|
91
|
+
}
|
|
92
|
+
const event = step.value;
|
|
93
|
+
if (event.type === "response.completed" || event.type === "response.incomplete") {
|
|
94
|
+
return { kind: "response", response: event.response };
|
|
95
|
+
}
|
|
96
|
+
// A failed flex response after admission is a capacity or server fault on OpenAI's side; the
|
|
97
|
+
// same request can still succeed on the default tier.
|
|
98
|
+
if (event.type === "response.failed")
|
|
99
|
+
return { kind: "failed", cause: event, retryable: true };
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
catch (cause) {
|
|
103
|
+
if (callerSignal?.aborted === true)
|
|
104
|
+
throw cause;
|
|
105
|
+
return { kind: "failed", cause, retryable: !isInvalidRequest(cause) };
|
|
106
|
+
}
|
|
107
|
+
}
|
|
108
|
+
// An invalid request fails the same way on every tier, so it is thrown rather than re-run.
|
|
109
|
+
function isInvalidRequest(error) {
|
|
110
|
+
if (typeof error !== "object" || error === null)
|
|
111
|
+
return false;
|
|
112
|
+
const type = error.type;
|
|
113
|
+
return type === "invalid_request_error";
|
|
114
|
+
}
|
|
115
|
+
async function* replay(buffered, rest) {
|
|
116
|
+
yield* buffered;
|
|
117
|
+
for (;;) {
|
|
118
|
+
const step = await rest.next();
|
|
119
|
+
if (step.done === true)
|
|
120
|
+
return;
|
|
121
|
+
yield step.value;
|
|
122
|
+
}
|
|
123
|
+
}
|
|
124
|
+
function linkedController(parent) {
|
|
125
|
+
const controller = new AbortController();
|
|
126
|
+
if (parent === undefined)
|
|
127
|
+
return controller;
|
|
128
|
+
if (parent.aborted)
|
|
129
|
+
controller.abort(parent.reason);
|
|
130
|
+
else {
|
|
131
|
+
parent.addEventListener("abort", () => {
|
|
132
|
+
controller.abort(parent.reason);
|
|
133
|
+
}, { once: true });
|
|
134
|
+
}
|
|
135
|
+
return controller;
|
|
136
|
+
}
|
|
137
|
+
function callerAbort(signal) {
|
|
138
|
+
return signal.reason ?? new Error("request aborted by caller");
|
|
139
|
+
}
|
|
140
|
+
//# sourceMappingURL=with-flex.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"with-flex.js","sourceRoot":"","sources":["../src/with-flex.ts"],"names":[],"mappings":"AAAA,sGAAsG;AACtG,wGAAwG;AACxG,sGAAsG;AACtG,wGAAwG;AACxG,oEAAoE;AACpE,EAAE;AACF,uGAAuG;AACvG,8FAA8F;AAG9F,OAAO,EAAE,MAAM,EAAE,MAAM,kBAAkB,CAAC;AAQ1C,OAAO,EAAE,gBAAgB,EAAE,MAAM,WAAW,CAAC;AAC7C,OAAO,EAAE,WAAW,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAuDxE,MAAM,UAAU,QAAQ,CAAC,MAAc,EAAE,UAA2B,EAAE;IACpE,MAAM,SAAS,GAAG,MAAM,CAAC,SAAS,CAAC;IACnC,MAAM,SAAS,GAAG,SAAS,CAAC,MAAM,CAAC,IAAI,CAAC,SAAS,CAAyB,CAAC;IAC3E,MAAM,MAAM,GAAG,UAAU,CAAC,SAAS,EAAE,OAAO,CAAC,CAAC;IAC9C,MAAM,aAAa,GAAG,IAAI,KAAK,CAAC,SAAS,EAAE;QACzC,GAAG,CAAC,MAAM,EAAE,QAAQ,EAAE,QAAQ;YAC5B,IAAI,QAAQ,KAAK,QAAQ;gBAAE,OAAO,MAAM,CAAC;YACzC,OAAO,OAAO,CAAC,GAAG,CAAC,MAAM,EAAE,QAAQ,EAAE,QAAQ,CAAY,CAAC;QAC5D,CAAC;KACF,CAAC,CAAC;IACH,OAAO,IAAI,KAAK,CAAC,MAAM,EAAE;QACvB,GAAG,CAAC,MAAM,EAAE,QAAQ,EAAE,QAAQ;YAC5B,IAAI,QAAQ,KAAK,WAAW;gBAAE,OAAO,aAAa,CAAC;YACnD,OAAO,OAAO,CAAC,GAAG,CAAC,MAAM,EAAE,QAAQ,EAAE,QAAQ,CAAY,CAAC;QAC5D,CAAC;KACF,CAAC,CAAC;AACL,CAAC;AAED,SAAS,UAAU,CAAC,SAAoB,EAAE,OAAwB;IAChE,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,IAAI,IAAI,CAAC,GAAG,CAAC;IACpC,MAAM,MAAM,GAAG,CAAC,OAAoB,EAAQ,EAAE,CAAC,OAAO,CAAC,SAAS,EAAE,CAAC,OAAO,CAAC,CAAC;IAE5E,KAAK,UAAU,MAAM,CAAC,IAAe,EAAE,cAA+B;QACpE,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,GAAG,MAAM,EAAE,GAAG,IAAI,CAAC;QAC9C,IAAI,OAAO,KAAK,SAAS;YAAE,OAAO,SAAS,CAAC,MAAM,EAAE,cAAc,CAAC,CAAC;QACpE,IAAI,MAAM,CAAC,YAAY,KAAK,SAAS,IAAI,MAAM,CAAC,YAAY,KAAK,IAAI,EAAE,CAAC;YACtE,MAAM,IAAI,YAAY,CAAC,kEAAkE,CAAC,CAAC;QAC7F,CAAC;QAED,MAAM,SAAS,GAAG,GAAG,EAAE,CAAC;QACxB,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,YAAY,CAAC,OAAO,EAAE,SAAS,CAAC,CAAC;QAC5D,MAAM,OAAO,GAAG,CAAC,IAAyB,EAAE,MAAkB,EAAQ,EAAE;YACtE,MAAM,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,QAAQ,EAAE,GAAG,EAAE,GAAG,SAAS,EAAE,MAAM,EAAE,CAAC,CAAC;QAChE,CAAC,CAAC;QACF,MAAM,UAAU,GAAG,GAA0B,EAAE,CAC7C,SAAS,CAAC,EAAE,GAAG,MAAM,EAAE,YAAY,EAAE,SAAS,EAAE,EAAE,cAAc,CAAC,CAAC;QAEpE,IAAI,MAAM,GAAG,WAAW,EAAE,CAAC;YACzB,OAAO,CAAC,SAAS,EAAE,mBAAmB,CAAC,CAAC;YACxC,OAAO,UAAU,EAAE,CAAC;QACtB,CAAC;QAED,MAAM,YAAY,GAAG,cAAc,EAAE,MAAM,IAAI,SAAS,CAAC;QACzD,MAAM,IAAI,GAAG,gBAAgB,CAAC,YAAY,CAAC,CAAC;QAC5C,MAAM,WAAW,GAAmB,EAAE,GAAG,cAAc,EAAE,MAAM,EAAE,IAAI,CAAC,MAAM,EAAE,UAAU,EAAE,CAAC,EAAE,CAAC;QAC9F,MAAM,MAAM,GAAG,MAAM,gBAAgB,CACnC,GAAG,EAAE,CACH,SAAS,CAAC,EAAE,GAAG,MAAM,EAAE,MAAM,EAAE,IAAI,EAAE,YAAY,EAAE,MAAM,EAAE,EAAE,WAAW,CAEvE,EACH,MAAM,CACP,CAAC;QAEF,IAAI,MAAM,CAAC,IAAI,KAAK,UAAU,EAAE,CAAC;YAC/B,IAAI,CAAC,KAAK,EAAE,CAAC;YACb,IAAI,YAAY,EAAE,OAAO,KAAK,IAAI;gBAAE,MAAM,WAAW,CAAC,YAAY,CAAC,CAAC;YACpE,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC,IAAI,KAAK,SAAS,CAAC,CAAC,CAAC,kBAAkB,CAAC,CAAC,CAAC,cAAc,CAAC,CAAC;YACpF,OAAO,UAAU,EAAE,CAAC;QACtB,CAAC;QAED,IAAI,MAAM,CAAC,MAAM,KAAK,IAAI,EAAE,CAAC;YAC3B,OAAO,CAAC,MAAM,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,IAAI,MAAM,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,EAAE,IAAI,CAAC,CAAC;QACtE,CAAC;QAED,qFAAqF;QACrF,MAAM,KAAK,GAAG,MAAM,eAAe,CAAC,MAAM,CAAC,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,IAAI,CAAC,EAAE,YAAY,CAAC,CAAC;QACxF,IAAI,KAAK,CAAC,IAAI,KAAK,UAAU,EAAE,CAAC;YAC9B,OAAO,CAAC,MAAM,EAAE,UAAU,CAAC,CAAC;YAC5B,OAAO,KAAK,CAAC,QAAQ,CAAC;QACxB,CAAC;QACD,IAAI,CAAC,KAAK,EAAE,CAAC;QACb,IAAI,CAAC,KAAK,CAAC,SAAS;YAAE,MAAM,KAAK,CAAC,KAAK,CAAC;QACxC,OAAO,CAAC,SAAS,EAAE,6BAA6B,CAAC,CAAC;QAClD,OAAO,UAAU,EAAE,CAAC;IACtB,CAAC;IAED,OAAO,MAAiC,CAAC;AAC3C,CAAC;AAMD,gGAAgG;AAChG,KAAK,UAAU,eAAe,CAC5B,IAAwC,EACxC,YAAqC;IAErC,IAAI,CAAC;QACH,SAAS,CAAC;YACR,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,IAAI,EAAE,CAAC;YAC/B,IAAI,IAAI,CAAC,IAAI,KAAK,IAAI,EAAE,CAAC;gBACvB,OAAO;oBACL,IAAI,EAAE,QAAQ;oBACd,KAAK,EAAE,IAAI,KAAK,CAAC,4CAA4C,CAAC;oBAC9D,SAAS,EAAE,IAAI;iBAChB,CAAC;YACJ,CAAC;YACD,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC;YACzB,IAAI,KAAK,CAAC,IAAI,KAAK,oBAAoB,IAAI,KAAK,CAAC,IAAI,KAAK,qBAAqB,EAAE,CAAC;gBAChF,OAAO,EAAE,IAAI,EAAE,UAAU,EAAE,QAAQ,EAAE,KAAK,CAAC,QAAQ,EAAE,CAAC;YACxD,CAAC;YACD,6FAA6F;YAC7F,sDAAsD;YACtD,IAAI,KAAK,CAAC,IAAI,KAAK,iBAAiB;gBAAE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,KAAK,EAAE,KAAK,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC;QACjG,CAAC;IACH,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,IAAI,YAAY,EAAE,OAAO,KAAK,IAAI;YAAE,MAAM,KAAK,CAAC;QAChD,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,KAAK,EAAE,SAAS,EAAE,CAAC,gBAAgB,CAAC,KAAK,CAAC,EAAE,CAAC;IACxE,CAAC;AACH,CAAC;AAED,2FAA2F;AAC3F,SAAS,gBAAgB,CAAC,KAAc;IACtC,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,KAAK,IAAI;QAAE,OAAO,KAAK,CAAC;IAC9D,MAAM,IAAI,GAAI,KAA4B,CAAC,IAAI,CAAC;IAChD,OAAO,IAAI,KAAK,uBAAuB,CAAC;AAC1C,CAAC;AAED,KAAK,SAAS,CAAC,CAAC,MAAM,CAAQ,QAA0B,EAAE,IAA0B;IAClF,KAAK,CAAC,CAAC,QAAQ,CAAC;IAChB,SAAS,CAAC;QACR,MAAM,IAAI,GAAG,MAAM,IAAI,CAAC,IAAI,EAAE,CAAC;QAC/B,IAAI,IAAI,CAAC,IAAI,KAAK,IAAI;YAAE,OAAO;QAC/B,MAAM,IAAI,CAAC,KAAK,CAAC;IACnB,CAAC;AACH,CAAC;AAED,SAAS,gBAAgB,CAAC,MAA+B;IACvD,MAAM,UAAU,GAAG,IAAI,eAAe,EAAE,CAAC;IACzC,IAAI,MAAM,KAAK,SAAS;QAAE,OAAO,UAAU,CAAC;IAC5C,IAAI,MAAM,CAAC,OAAO;QAAE,UAAU,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;SAC/C,CAAC;QACJ,MAAM,CAAC,gBAAgB,CACrB,OAAO,EACP,GAAG,EAAE;YACH,UAAU,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QAClC,CAAC,EACD,EAAE,IAAI,EAAE,IAAI,EAAE,CACf,CAAC;IACJ,CAAC;IACD,OAAO,UAAU,CAAC;AACpB,CAAC;AAED,SAAS,WAAW,CAAC,MAAmB;IACtC,OAAO,MAAM,CAAC,MAAM,IAAI,IAAI,KAAK,CAAC,2BAA2B,CAAC,CAAC;AACjE,CAAC"}
|
package/package.json
ADDED
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "flex-race",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "Race OpenAI's flex tier against a deadline on the Responses API, falling back to the default tier.",
|
|
5
|
+
"license": "MIT",
|
|
6
|
+
"author": "Aditya Perswal",
|
|
7
|
+
"repository": {
|
|
8
|
+
"type": "git",
|
|
9
|
+
"url": "git+https://github.com/aperswal/flex-race.git"
|
|
10
|
+
},
|
|
11
|
+
"homepage": "https://github.com/aperswal/flex-race#readme",
|
|
12
|
+
"bugs": {
|
|
13
|
+
"url": "https://github.com/aperswal/flex-race/issues"
|
|
14
|
+
},
|
|
15
|
+
"keywords": [
|
|
16
|
+
"openai",
|
|
17
|
+
"flex",
|
|
18
|
+
"service_tier",
|
|
19
|
+
"responses",
|
|
20
|
+
"deadline"
|
|
21
|
+
],
|
|
22
|
+
"type": "module",
|
|
23
|
+
"exports": {
|
|
24
|
+
".": {
|
|
25
|
+
"types": "./dist/index.d.ts",
|
|
26
|
+
"import": "./dist/index.js"
|
|
27
|
+
}
|
|
28
|
+
},
|
|
29
|
+
"files": [
|
|
30
|
+
"dist"
|
|
31
|
+
],
|
|
32
|
+
"engines": {
|
|
33
|
+
"node": ">=20"
|
|
34
|
+
},
|
|
35
|
+
"scripts": {
|
|
36
|
+
"build": "tsc -p tsconfig.build.json",
|
|
37
|
+
"typecheck": "tsc --noEmit",
|
|
38
|
+
"lint": "eslint --max-warnings=0 .",
|
|
39
|
+
"format": "prettier --write .",
|
|
40
|
+
"format:check": "prettier --check .",
|
|
41
|
+
"test": "vitest run",
|
|
42
|
+
"test:live": "vitest run --config vitest.live.config.ts",
|
|
43
|
+
"verify": "pnpm format:check && pnpm lint && pnpm typecheck && pnpm test && pnpm build",
|
|
44
|
+
"prepare": "husky"
|
|
45
|
+
},
|
|
46
|
+
"peerDependencies": {
|
|
47
|
+
"openai": ">=6"
|
|
48
|
+
},
|
|
49
|
+
"devDependencies": {
|
|
50
|
+
"@eslint/js": "^10.0.1",
|
|
51
|
+
"@types/node": "^26.6.3",
|
|
52
|
+
"@vitest/coverage-v8": "^5.0.2",
|
|
53
|
+
"eslint": "^10.11.0",
|
|
54
|
+
"husky": "^9.1.7",
|
|
55
|
+
"lint-staged": "^17.5.1",
|
|
56
|
+
"openai": "^7.23.0",
|
|
57
|
+
"prettier": "^3.9.9",
|
|
58
|
+
"typescript": "^6.0.3",
|
|
59
|
+
"typescript-eslint": "^8.70.1",
|
|
60
|
+
"vitest": "^5.0.2"
|
|
61
|
+
}
|
|
62
|
+
}
|