@milkenjoyer/toolchain 0.16.0 → 0.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/STARTS.md +12 -6
- package/dist/starts.d.mts +19 -6
- package/dist/starts.mjs +117 -44
- package/package.json +1 -1
- package/src/starts/budget.ts +98 -25
- package/src/starts/merges.ts +99 -0
- package/src/starts.ts +21 -4
package/README.md
CHANGED
|
@@ -268,4 +268,4 @@ Source checks skip tests/proofs. Runtime story coverage is tracked in #325.
|
|
|
268
268
|
|
|
269
269
|
# Boat start budgets
|
|
270
270
|
|
|
271
|
-
See [Boat start
|
|
271
|
+
See [Boat start admission](STARTS.md) for `@milkenjoyer/toolchain/starts` in 0.17.0.
|
package/STARTS.md
CHANGED
|
@@ -1,11 +1,17 @@
|
|
|
1
|
-
# Boat start
|
|
1
|
+
# Boat start admission
|
|
2
2
|
|
|
3
|
-
Pin `@milkenjoyer/toolchain` to `0.
|
|
3
|
+
Pin `@milkenjoyer/toolchain` to `0.17.0`. Import `admitStart`, `BoatStarts`, `Merges` and `Repo` from `@milkenjoyer/toolchain/starts`. Wrap each create, fork or resume in `admitStart({ repo, jobs: 1, priority: "ordinary" }, start, { deadline })`. Priority is required and accepts `"ordinary"` or `"p0"`. Red main is p0. `Repo` validates an `owner/name`. `deadline` is an absolute epoch time in milliseconds. `start` is a lazy Effect that runs once after admission.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
Provide `BoatStarts.layer(key, { fetch? })` and `Merges.layer(githubToken, { fetch? })` with redacted credentials. Boat reads the key's default billing wallet. GitHub must see all four repos. Each optional `fetch` supplies that SDK's transport. The pure `decide(request, counts, now)` takes the current epoch time explicitly.
|
|
6
6
|
|
|
7
|
-
|
|
7
|
+
Each request counts as `jobs` in every live minute, hour and day window. A paid or unavailable next-start quote waits. Below 50% account use, every priority passes, subject to the account ceiling. From 50% to 80%, ordinary requests must fit the repo's share of an 80% pool. From 80% to 95%, only p0 passes. No request may exceed its account ceiling, rounded down to 80% for ordinary or 95% for p0, including the requested jobs.
|
|
8
8
|
|
|
9
|
-
`
|
|
9
|
+
`startRepos` contains agent-kit, siglata/siglata, optionhouse-xyz/optionhouse and birbprophet/vibebattler. Each repo's share is `10% + 60% × weight / total weight`. Weight is `max(merges in 24h, 4 × merges in 6h, merges in 7 days / 7)` to main. All zero weights give each repo 25%. Each repo budget is `floor(share × 80% × window limit)`, computed with exact arithmetic. Merge and repo history are read only when an ordinary request faces at least 50% use in a window.
|
|
10
10
|
|
|
11
|
-
|
|
11
|
+
`StartHeld.line` names the refusal. `Unread` means a read failed, history is incomplete or input is invalid. Read failures and incomplete histories retry until the deadline. Invalid plans, repos, jobs and timestamps hold immediately. `Paid` waits for a free quote. `Account` means the request exceeds its account ceiling and waits on minute or hour, or holds on day. `Budget` means the repo exceeds its live share and waits for the kth oldest start to leave the window. A request larger than its repo budget holds. Every wait polls fresh readings until the deadline, then fails `StartHeld` with the waiting line.
|
|
12
|
+
|
|
13
|
+
Repo CI history counts a Boat job without `runStartedAt` as starting now unless Boat reports it settled, including states the SDK does not list, such as `paused`. A start stamped after Boat answers counts at the answer's time. That history omits direct sandbox starts. Admission reserves nothing, so concurrent callers can race.
|
|
14
|
+
|
|
15
|
+
Import `boatJobFindings` from `@milkenjoyer/toolchain/starts`. Add its findings to the existing workflow policy, with every workflow as a `[path, text]` pair. It bounds each closure to `BOAT_JOBS_PER_RUN = 2`, including literal matrices, repeated local reusable calls and `workflow_run` descendants. Each descendant counts once per distinct activity type, or three times when `types` is absent. Dynamic and unresolved graphs produce findings. Keep runner, timeout and concurrency checks.
|
|
16
|
+
|
|
17
|
+
PR and main workflow triggers stay unchanged until the request controller ships. Workflows start without admission, bounded only by `BOAT_JOBS_PER_RUN`. Product repos adopt the published pin instead of copying files. Boat reports no start spend, so nothing here proves zero paid starts.
|
package/dist/starts.d.mts
CHANGED
|
@@ -7,9 +7,12 @@ export declare const Repo: Schema.brand<Schema.String, "Repo">;
|
|
|
7
7
|
export type Repo = typeof Repo.Type;
|
|
8
8
|
type Window = "minute" | "hour" | "day";
|
|
9
9
|
type PerWindow<A> = { readonly [W in Window]: A; };
|
|
10
|
-
type
|
|
10
|
+
type Priority = "ordinary" | "p0";
|
|
11
|
+
type MergeCounts = {
|
|
11
12
|
readonly repo: Repo;
|
|
12
|
-
readonly
|
|
13
|
+
readonly h6: number;
|
|
14
|
+
readonly h24: number;
|
|
15
|
+
readonly d7: number;
|
|
13
16
|
};
|
|
14
17
|
type Plan = {
|
|
15
18
|
readonly limits: PerWindow<number>;
|
|
@@ -24,11 +27,13 @@ type RepoStarts = {
|
|
|
24
27
|
type Counts = {
|
|
25
28
|
readonly now: number;
|
|
26
29
|
readonly plan: Plan;
|
|
27
|
-
readonly repo
|
|
30
|
+
readonly repo?: RepoStarts | undefined;
|
|
31
|
+
readonly merges?: ReadonlyArray<MergeCounts> | undefined;
|
|
28
32
|
};
|
|
29
33
|
type StartRequest = {
|
|
30
34
|
readonly repo: Repo;
|
|
31
35
|
readonly jobs: number;
|
|
36
|
+
readonly priority: Priority;
|
|
32
37
|
};
|
|
33
38
|
declare const Line: Schema.Union<readonly [Schema.TaggedStruct<"Unread", {
|
|
34
39
|
readonly reason: Schema.String;
|
|
@@ -55,7 +60,7 @@ type Admission = {
|
|
|
55
60
|
readonly _tag: "Hold";
|
|
56
61
|
readonly line: Line;
|
|
57
62
|
};
|
|
58
|
-
export declare const
|
|
63
|
+
export declare const startRepos: ReadonlyArray<Repo>;
|
|
59
64
|
export declare const decide: (request: StartRequest, counts: Counts | undefined, now: number) => Admission;
|
|
60
65
|
declare const StartReadError_base: Schema.Class<StartReadError, Schema.TaggedStruct<"StartReadError", {
|
|
61
66
|
readonly reason: Schema.String;
|
|
@@ -72,6 +77,14 @@ export declare class BoatStarts extends BoatStarts_base {
|
|
|
72
77
|
readonly fetch?: (url: string | URL | Request, init?: RequestInit) => Promise<Response>;
|
|
73
78
|
}) => Layer.Layer<BoatStarts, never, never>;
|
|
74
79
|
}
|
|
80
|
+
declare const Merges_base: Context.ServiceClass<Merges, "toolchain/Merges", {
|
|
81
|
+
readonly counts: Effect.Effect<ReadonlyArray<MergeCounts>, StartReadError>;
|
|
82
|
+
}>;
|
|
83
|
+
export declare class Merges extends Merges_base {
|
|
84
|
+
static readonly layer: (token: Redacted.Redacted<string>, options?: {
|
|
85
|
+
readonly fetch?: (url: string | URL | Request, init?: RequestInit) => Promise<Response>;
|
|
86
|
+
}) => Layer.Layer<Merges, never, never>;
|
|
87
|
+
}
|
|
75
88
|
export declare const BOAT_JOBS_PER_RUN = 2;
|
|
76
89
|
type WorkflowFile = readonly [path: string, text: string];
|
|
77
90
|
export declare const boatJobFindings: (files: ReadonlyArray<WorkflowFile>) => ReadonlyArray<string>;
|
|
@@ -96,5 +109,5 @@ declare const StartHeld_base: Schema.Class<StartHeld, Schema.TaggedStruct<"Start
|
|
|
96
109
|
export declare class StartHeld extends StartHeld_base {}
|
|
97
110
|
export declare const admitStart: <A, E, R>(request: StartRequest, start: Effect.Effect<A, E, R>, args_2: {
|
|
98
111
|
readonly deadline: number;
|
|
99
|
-
}) => Effect.Effect<A, E | StartHeld, R | BoatStarts>;
|
|
100
|
-
export type { Admission, Counts, Line, PerWindow, Plan,
|
|
112
|
+
}) => Effect.Effect<A, E | StartHeld, R | BoatStarts | Merges>;
|
|
113
|
+
export type { Admission, Counts, Line, MergeCounts, PerWindow, Plan, Priority, RepoStarts, StartRequest, Window };
|
package/dist/starts.mjs
CHANGED
|
@@ -3,6 +3,7 @@ import * as Effect from "effect/Effect";
|
|
|
3
3
|
import * as Layer from "effect/Layer";
|
|
4
4
|
import * as Clock from "effect/Clock";
|
|
5
5
|
import * as Context from "effect/Context";
|
|
6
|
+
import { Octokit } from "@octokit/rest";
|
|
6
7
|
import * as Redacted from "effect/Redacted";
|
|
7
8
|
import { BoatApi, CiJobState, Configuration } from "@boatdev/sdk";
|
|
8
9
|
import { parse } from "yaml";
|
|
@@ -28,7 +29,7 @@ const Line = Schema.Union([
|
|
|
28
29
|
budget: Schema.Number
|
|
29
30
|
})
|
|
30
31
|
]);
|
|
31
|
-
const windows = [
|
|
32
|
+
const windows$1 = [
|
|
32
33
|
"minute",
|
|
33
34
|
"hour",
|
|
34
35
|
"day"
|
|
@@ -43,29 +44,39 @@ const pollMillis = {
|
|
|
43
44
|
hour: 6e4,
|
|
44
45
|
day: 3e5
|
|
45
46
|
};
|
|
46
|
-
const
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
{
|
|
52
|
-
repo: Repo.make("siglata/siglata"),
|
|
53
|
-
share: .2
|
|
54
|
-
},
|
|
55
|
-
{
|
|
56
|
-
repo: Repo.make("optionhouse-xyz/optionhouse"),
|
|
57
|
-
share: .25
|
|
58
|
-
},
|
|
59
|
-
{
|
|
60
|
-
repo: Repo.make("birbprophet/vibebattler"),
|
|
61
|
-
share: .25
|
|
62
|
-
},
|
|
63
|
-
{
|
|
64
|
-
repo: Repo.make("birbprophet/secrets"),
|
|
65
|
-
share: 0
|
|
66
|
-
}
|
|
47
|
+
const startRepos = [
|
|
48
|
+
Repo.make("birbprophet/agent-kit"),
|
|
49
|
+
Repo.make("siglata/siglata"),
|
|
50
|
+
Repo.make("optionhouse-xyz/optionhouse"),
|
|
51
|
+
Repo.make("birbprophet/vibebattler")
|
|
67
52
|
];
|
|
68
53
|
const count = (value) => Number.isSafeInteger(value) && value >= 0;
|
|
54
|
+
const ceiling = (request, plan, window) => Number(BigInt(plan.limits[window]) * (request.priority === "p0" ? 95n : 80n) / 100n);
|
|
55
|
+
const sharedWindows = (request, plan) => request.priority === "p0" || plan.nextStartDollars !== 0 || !windows$1.every((window) => count(plan.limits[window]) && count(plan.used[window])) || windows$1.some((window) => plan.used[window] + request.jobs > ceiling(request, plan, window)) ? [] : windows$1.filter((window) => 2 * plan.used[window] >= plan.limits[window]);
|
|
56
|
+
const liveShare = (repo, merges) => {
|
|
57
|
+
if (merges === void 0 || merges.length !== startRepos.length || !startRepos.every((repo) => merges.filter((row) => row.repo === repo).length === 1) || !merges.every((row) => [
|
|
58
|
+
row.h6,
|
|
59
|
+
row.h24,
|
|
60
|
+
row.d7
|
|
61
|
+
].every(count))) return void 0;
|
|
62
|
+
const weights = merges.map((row) => ({
|
|
63
|
+
repo: row.repo,
|
|
64
|
+
weight: [
|
|
65
|
+
7n * BigInt(row.h24),
|
|
66
|
+
28n * BigInt(row.h6),
|
|
67
|
+
BigInt(row.d7)
|
|
68
|
+
].reduce((largest, value) => value > largest ? value : largest, 0n)
|
|
69
|
+
}));
|
|
70
|
+
const total = weights.reduce((sum, row) => sum + row.weight, 0n);
|
|
71
|
+
const weight = weights.find((row) => row.repo === repo)?.weight;
|
|
72
|
+
return weight === void 0 ? void 0 : total === 0n ? {
|
|
73
|
+
numerator: 1n,
|
|
74
|
+
denominator: 4n
|
|
75
|
+
} : {
|
|
76
|
+
numerator: total + 6n * weight,
|
|
77
|
+
denominator: 10n * total
|
|
78
|
+
};
|
|
79
|
+
};
|
|
69
80
|
const unread = (reason) => ({
|
|
70
81
|
_tag: "Hold",
|
|
71
82
|
line: {
|
|
@@ -74,8 +85,7 @@ const unread = (reason) => ({
|
|
|
74
85
|
}
|
|
75
86
|
});
|
|
76
87
|
const decide = (request, counts, now) => {
|
|
77
|
-
|
|
78
|
-
if (share === void 0) return unread("repo has no start budget");
|
|
88
|
+
if (!startRepos.includes(request.repo)) return unread("repo has no start budget");
|
|
79
89
|
if (!count(request.jobs) || request.jobs === 0) return unread("jobs must be a positive safe integer");
|
|
80
90
|
if (!Number.isFinite(now)) return unread("start time unread");
|
|
81
91
|
if (counts === void 0) return {
|
|
@@ -86,16 +96,7 @@ const decide = (request, counts, now) => {
|
|
|
86
96
|
reason: "start plan unread"
|
|
87
97
|
}
|
|
88
98
|
};
|
|
89
|
-
if (!Number.isFinite(counts.now) || !windows.every((window) => count(counts.plan.limits[window]) && count(counts.plan.used[window]))) return unread("start plan unread");
|
|
90
|
-
if (counts.repo.repo !== request.repo || !counts.repo.starts.every((at) => Number.isFinite(at) && at <= counts.now)) return unread("repo start history incomplete");
|
|
91
|
-
if (!counts.repo.complete) return {
|
|
92
|
-
_tag: "Wait",
|
|
93
|
-
until: now + pollMillis.minute,
|
|
94
|
-
line: {
|
|
95
|
-
_tag: "Unread",
|
|
96
|
-
reason: "repo start history incomplete"
|
|
97
|
-
}
|
|
98
|
-
};
|
|
99
|
+
if (!Number.isFinite(counts.now) || counts.now > now || !windows$1.every((window) => count(counts.plan.limits[window]) && count(counts.plan.used[window]))) return unread("start plan unread");
|
|
99
100
|
const { plan } = counts;
|
|
100
101
|
if (plan.nextStartDollars !== 0) return {
|
|
101
102
|
_tag: "Wait",
|
|
@@ -105,9 +106,9 @@ const decide = (request, counts, now) => {
|
|
|
105
106
|
nextStartDollars: plan.nextStartDollars
|
|
106
107
|
}
|
|
107
108
|
};
|
|
108
|
-
for (const window of windows) {
|
|
109
|
+
for (const window of windows$1) {
|
|
109
110
|
const used = plan.used[window];
|
|
110
|
-
const line = plan
|
|
111
|
+
const line = ceiling(request, plan, window);
|
|
111
112
|
if (used + request.jobs > line) return window === "day" ? {
|
|
112
113
|
_tag: "Hold",
|
|
113
114
|
line: {
|
|
@@ -127,10 +128,30 @@ const decide = (request, counts, now) => {
|
|
|
127
128
|
}
|
|
128
129
|
};
|
|
129
130
|
}
|
|
130
|
-
|
|
131
|
+
const shared = sharedWindows(request, plan);
|
|
132
|
+
if (shared.length === 0) return { _tag: "Admit" };
|
|
133
|
+
if (counts.repo === void 0 || !counts.repo.complete) return {
|
|
134
|
+
_tag: "Wait",
|
|
135
|
+
until: now + pollMillis.minute,
|
|
136
|
+
line: {
|
|
137
|
+
_tag: "Unread",
|
|
138
|
+
reason: "repo start history incomplete"
|
|
139
|
+
}
|
|
140
|
+
};
|
|
141
|
+
if (counts.repo.repo !== request.repo || !counts.repo.starts.every((at) => Number.isFinite(at) && at <= counts.now)) return unread("repo start history incomplete");
|
|
142
|
+
const share = liveShare(request.repo, counts.merges);
|
|
143
|
+
if (share === void 0) return {
|
|
144
|
+
_tag: "Wait",
|
|
145
|
+
until: now + pollMillis.minute,
|
|
146
|
+
line: {
|
|
147
|
+
_tag: "Unread",
|
|
148
|
+
reason: "merge history incomplete"
|
|
149
|
+
}
|
|
150
|
+
};
|
|
151
|
+
for (const window of shared) {
|
|
131
152
|
const starts = counts.repo.starts.filter((at) => at > now - windowMillis[window]).toSorted((left, right) => left - right);
|
|
132
153
|
const used = starts.length;
|
|
133
|
-
const budget =
|
|
154
|
+
const budget = Number(share.numerator * 4n * BigInt(plan.limits[window]) / (share.denominator * 5n));
|
|
134
155
|
if (used + request.jobs <= budget) continue;
|
|
135
156
|
const line = {
|
|
136
157
|
_tag: "Budget",
|
|
@@ -153,10 +174,10 @@ const decide = (request, counts, now) => {
|
|
|
153
174
|
};
|
|
154
175
|
// src/starts/boat.ts
|
|
155
176
|
var StartReadError = class extends Schema.TaggedError()("StartReadError", { reason: Schema.String }) {};
|
|
156
|
-
const Count = Schema.Number.check(Schema.makeFilter((value) => Number.isSafeInteger(value) && value >= 0));
|
|
177
|
+
const Count$1 = Schema.Number.check(Schema.makeFilter((value) => Number.isSafeInteger(value) && value >= 0));
|
|
157
178
|
const Usage = Schema.Struct({
|
|
158
|
-
limit: Count,
|
|
159
|
-
used: Count
|
|
179
|
+
limit: Count$1,
|
|
180
|
+
used: Count$1
|
|
160
181
|
});
|
|
161
182
|
const Limits = Schema.Struct({
|
|
162
183
|
ok: Schema.Literal(true),
|
|
@@ -238,6 +259,51 @@ var BoatStarts = class BoatStarts extends Context.Service()("toolchain/BoatStart
|
|
|
238
259
|
});
|
|
239
260
|
});
|
|
240
261
|
};
|
|
262
|
+
// src/starts/merges.ts
|
|
263
|
+
const Count = Schema.Struct({ issueCount: Schema.Number.check(Schema.makeFilter((value) => Number.isSafeInteger(value) && value >= 0)) });
|
|
264
|
+
const windows = {
|
|
265
|
+
h6: 216e5,
|
|
266
|
+
h24: 864e5,
|
|
267
|
+
d7: 6048e5
|
|
268
|
+
};
|
|
269
|
+
var Merges = class Merges extends Context.Service()("toolchain/Merges") {
|
|
270
|
+
static layer = (token, options = {}) => Layer.sync(Merges, () => {
|
|
271
|
+
const api = new Octokit({
|
|
272
|
+
auth: Redacted.value(token),
|
|
273
|
+
...options.fetch === void 0 ? {} : { request: { fetch: options.fetch } }
|
|
274
|
+
});
|
|
275
|
+
const counts = Effect.gen(function* () {
|
|
276
|
+
const now = yield* Clock.currentTimeMillis;
|
|
277
|
+
const query = `query { ${startRepos.flatMap((repo, index) => {
|
|
278
|
+
const [owner, name] = repo.split("/");
|
|
279
|
+
return [`repo${index}: repository(owner: "${owner}", name: "${name}") { nameWithOwner }`, ...Object.entries(windows).map(([window, millis]) => {
|
|
280
|
+
return `${window}_${index}: search(type: ISSUE, first: 0, query: "repo:${repo} is:pr is:merged base:main merged:>=${new Date(now - millis).toISOString().replace(/\.\d{3}Z$/u, "Z")}") { issueCount }`;
|
|
281
|
+
})];
|
|
282
|
+
}).join(" ")} }`;
|
|
283
|
+
const response = yield* Effect.tryPromise({
|
|
284
|
+
try: () => api.graphql(query),
|
|
285
|
+
catch: () => new StartReadError({ reason: "GitHub merges request failed" })
|
|
286
|
+
});
|
|
287
|
+
return yield* Schema.decodeUnknownEffect(Schema.Record(Schema.String, Schema.Unknown))(response).pipe(Effect.flatMap((data) => Effect.forEach(startRepos, (repo, index) => Schema.decodeUnknownEffect(Schema.Struct({
|
|
288
|
+
repository: Schema.Struct({ nameWithOwner: Schema.Literal(repo) }),
|
|
289
|
+
h6: Count,
|
|
290
|
+
h24: Count,
|
|
291
|
+
d7: Count
|
|
292
|
+
}))({
|
|
293
|
+
repository: data[`repo${index}`],
|
|
294
|
+
h6: data[`h6_${index}`],
|
|
295
|
+
h24: data[`h24_${index}`],
|
|
296
|
+
d7: data[`d7_${index}`]
|
|
297
|
+
}).pipe(Effect.map((row) => ({
|
|
298
|
+
repo,
|
|
299
|
+
h6: row.h6.issueCount,
|
|
300
|
+
h24: row.h24.issueCount,
|
|
301
|
+
d7: row.d7.issueCount
|
|
302
|
+
}))), { concurrency: 1 })), Effect.mapError(() => new StartReadError({ reason: "GitHub merges unread" })));
|
|
303
|
+
});
|
|
304
|
+
return Merges.of({ counts });
|
|
305
|
+
});
|
|
306
|
+
};
|
|
241
307
|
// src/starts/policy.ts
|
|
242
308
|
const BOAT_JOBS_PER_RUN = 2;
|
|
243
309
|
const Matrix = Schema.Record(Schema.String, Schema.Array(Schema.Unknown));
|
|
@@ -350,6 +416,7 @@ const boatJobFindings = (files) => {
|
|
|
350
416
|
var StartHeld = class extends Schema.TaggedError()("StartHeld", { line: Line }) {};
|
|
351
417
|
const admitStart = Effect.fn("Starts.admitStart")(function* (request, start, { deadline }) {
|
|
352
418
|
const boat = yield* BoatStarts;
|
|
419
|
+
const merges = yield* Merges;
|
|
353
420
|
let last = {
|
|
354
421
|
_tag: "Unread",
|
|
355
422
|
reason: "start deadline reached"
|
|
@@ -359,11 +426,17 @@ const admitStart = Effect.fn("Starts.admitStart")(function* (request, start, { d
|
|
|
359
426
|
if (!Number.isFinite(remaining) || remaining <= 0) return yield* new StartHeld({ line: last });
|
|
360
427
|
const counts = yield* Effect.gen(function* () {
|
|
361
428
|
const plan = yield* boat.plan;
|
|
362
|
-
|
|
429
|
+
if (sharedWindows(request, plan).length === 0) return {
|
|
430
|
+
now: yield* Clock.currentTimeMillis,
|
|
431
|
+
plan
|
|
432
|
+
};
|
|
433
|
+
const repo = yield* boat.repoStarts(request.repo).pipe(Effect.catchTag("StartReadError", () => Effect.undefined));
|
|
434
|
+
const history = yield* merges.counts.pipe(Effect.catchTag("StartReadError", () => Effect.undefined));
|
|
363
435
|
return {
|
|
364
436
|
now: yield* Clock.currentTimeMillis,
|
|
365
437
|
plan,
|
|
366
|
-
repo
|
|
438
|
+
repo,
|
|
439
|
+
merges: history
|
|
367
440
|
};
|
|
368
441
|
}).pipe(Effect.catchTag("StartReadError", () => Effect.undefined), Effect.timeoutOrElse({
|
|
369
442
|
duration: remaining,
|
|
@@ -382,4 +455,4 @@ const admitStart = Effect.fn("Starts.admitStart")(function* (request, start, { d
|
|
|
382
455
|
yield* Effect.sleep(Math.max(0, Math.min(decision.until, now + pollMillis[window], deadline) - now));
|
|
383
456
|
}
|
|
384
457
|
});
|
|
385
|
-
export { BOAT_JOBS_PER_RUN, BoatStarts, Repo, StartHeld, StartReadError, admitStart, boatJobFindings, decide,
|
|
458
|
+
export { BOAT_JOBS_PER_RUN, BoatStarts, Merges, Repo, StartHeld, StartReadError, admitStart, boatJobFindings, decide, startRepos };
|
package/package.json
CHANGED
package/src/starts/budget.ts
CHANGED
|
@@ -6,7 +6,13 @@ export const Repo = Schema.String.check(
|
|
|
6
6
|
export type Repo = typeof Repo.Type;
|
|
7
7
|
export type Window = "minute" | "hour" | "day";
|
|
8
8
|
export type PerWindow<A> = { readonly [W in Window]: A };
|
|
9
|
-
export type
|
|
9
|
+
export type Priority = "ordinary" | "p0";
|
|
10
|
+
export type MergeCounts = {
|
|
11
|
+
readonly repo: Repo;
|
|
12
|
+
readonly h6: number;
|
|
13
|
+
readonly h24: number;
|
|
14
|
+
readonly d7: number;
|
|
15
|
+
};
|
|
10
16
|
export type Plan = {
|
|
11
17
|
readonly limits: PerWindow<number>;
|
|
12
18
|
readonly used: PerWindow<number>;
|
|
@@ -20,9 +26,14 @@ export type RepoStarts = {
|
|
|
20
26
|
export type Counts = {
|
|
21
27
|
readonly now: number;
|
|
22
28
|
readonly plan: Plan;
|
|
23
|
-
readonly repo
|
|
29
|
+
readonly repo?: RepoStarts | undefined;
|
|
30
|
+
readonly merges?: ReadonlyArray<MergeCounts> | undefined;
|
|
31
|
+
};
|
|
32
|
+
export type StartRequest = {
|
|
33
|
+
readonly repo: Repo;
|
|
34
|
+
readonly jobs: number;
|
|
35
|
+
readonly priority: Priority;
|
|
24
36
|
};
|
|
25
|
-
export type StartRequest = { readonly repo: Repo; readonly jobs: number };
|
|
26
37
|
|
|
27
38
|
const Window = Schema.Literals(["minute", "hour", "day"]);
|
|
28
39
|
export const Line = Schema.Union([
|
|
@@ -60,15 +71,64 @@ export const pollMillis: PerWindow<number> = {
|
|
|
60
71
|
day: 300_000,
|
|
61
72
|
};
|
|
62
73
|
|
|
63
|
-
export const
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
{ repo: Repo.make("birbprophet/secrets"), share: 0 },
|
|
74
|
+
export const startRepos: ReadonlyArray<Repo> = [
|
|
75
|
+
Repo.make("birbprophet/agent-kit"),
|
|
76
|
+
Repo.make("siglata/siglata"),
|
|
77
|
+
Repo.make("optionhouse-xyz/optionhouse"),
|
|
78
|
+
Repo.make("birbprophet/vibebattler"),
|
|
69
79
|
];
|
|
70
80
|
|
|
71
81
|
const count = (value: number) => Number.isSafeInteger(value) && value >= 0;
|
|
82
|
+
const ceiling = (request: StartRequest, plan: Plan, window: Window) =>
|
|
83
|
+
Number(
|
|
84
|
+
(BigInt(plan.limits[window]) * (request.priority === "p0" ? 95n : 80n)) /
|
|
85
|
+
100n
|
|
86
|
+
);
|
|
87
|
+
|
|
88
|
+
export const sharedWindows = (
|
|
89
|
+
request: StartRequest,
|
|
90
|
+
plan: Plan
|
|
91
|
+
): ReadonlyArray<Window> =>
|
|
92
|
+
request.priority === "p0" ||
|
|
93
|
+
plan.nextStartDollars !== 0 ||
|
|
94
|
+
!windows.every(
|
|
95
|
+
(window) => count(plan.limits[window]) && count(plan.used[window])
|
|
96
|
+
) ||
|
|
97
|
+
windows.some(
|
|
98
|
+
(window) =>
|
|
99
|
+
plan.used[window] + request.jobs > ceiling(request, plan, window)
|
|
100
|
+
)
|
|
101
|
+
? []
|
|
102
|
+
: windows.filter((window) => 2 * plan.used[window] >= plan.limits[window]);
|
|
103
|
+
|
|
104
|
+
const liveShare = (
|
|
105
|
+
repo: Repo,
|
|
106
|
+
merges: Counts["merges"]
|
|
107
|
+
): { readonly numerator: bigint; readonly denominator: bigint } | undefined => {
|
|
108
|
+
if (
|
|
109
|
+
merges === undefined ||
|
|
110
|
+
merges.length !== startRepos.length ||
|
|
111
|
+
!startRepos.every(
|
|
112
|
+
(repo) => merges.filter((row) => row.repo === repo).length === 1
|
|
113
|
+
) ||
|
|
114
|
+
!merges.every((row) => [row.h6, row.h24, row.d7].every(count))
|
|
115
|
+
)
|
|
116
|
+
return undefined;
|
|
117
|
+
const weights = merges.map((row) => ({
|
|
118
|
+
repo: row.repo,
|
|
119
|
+
weight: [7n * BigInt(row.h24), 28n * BigInt(row.h6), BigInt(row.d7)].reduce(
|
|
120
|
+
(largest, value) => (value > largest ? value : largest),
|
|
121
|
+
0n
|
|
122
|
+
),
|
|
123
|
+
}));
|
|
124
|
+
const total = weights.reduce((sum, row) => sum + row.weight, 0n);
|
|
125
|
+
const weight = weights.find((row) => row.repo === repo)?.weight;
|
|
126
|
+
return weight === undefined
|
|
127
|
+
? undefined
|
|
128
|
+
: total === 0n
|
|
129
|
+
? { numerator: 1n, denominator: 4n }
|
|
130
|
+
: { numerator: total + 6n * weight, denominator: 10n * total };
|
|
131
|
+
};
|
|
72
132
|
const unread = (reason: string): Admission => ({
|
|
73
133
|
_tag: "Hold",
|
|
74
134
|
line: { _tag: "Unread", reason },
|
|
@@ -79,8 +139,8 @@ export const decide = (
|
|
|
79
139
|
counts: Counts | undefined,
|
|
80
140
|
now: number
|
|
81
141
|
): Admission => {
|
|
82
|
-
|
|
83
|
-
|
|
142
|
+
if (!startRepos.includes(request.repo))
|
|
143
|
+
return unread("repo has no start budget");
|
|
84
144
|
if (!count(request.jobs) || request.jobs === 0)
|
|
85
145
|
return unread("jobs must be a positive safe integer");
|
|
86
146
|
if (!Number.isFinite(now)) return unread("start time unread");
|
|
@@ -92,23 +152,13 @@ export const decide = (
|
|
|
92
152
|
};
|
|
93
153
|
if (
|
|
94
154
|
!Number.isFinite(counts.now) ||
|
|
155
|
+
counts.now > now ||
|
|
95
156
|
!windows.every(
|
|
96
157
|
(window) =>
|
|
97
158
|
count(counts.plan.limits[window]) && count(counts.plan.used[window])
|
|
98
159
|
)
|
|
99
160
|
)
|
|
100
161
|
return unread("start plan unread");
|
|
101
|
-
if (
|
|
102
|
-
counts.repo.repo !== request.repo ||
|
|
103
|
-
!counts.repo.starts.every((at) => Number.isFinite(at) && at <= counts.now)
|
|
104
|
-
)
|
|
105
|
-
return unread("repo start history incomplete");
|
|
106
|
-
if (!counts.repo.complete)
|
|
107
|
-
return {
|
|
108
|
-
_tag: "Wait",
|
|
109
|
-
until: now + pollMillis.minute,
|
|
110
|
-
line: { _tag: "Unread", reason: "repo start history incomplete" },
|
|
111
|
-
};
|
|
112
162
|
const { plan } = counts;
|
|
113
163
|
if (plan.nextStartDollars !== 0)
|
|
114
164
|
return {
|
|
@@ -118,7 +168,7 @@ export const decide = (
|
|
|
118
168
|
};
|
|
119
169
|
for (const window of windows) {
|
|
120
170
|
const used = plan.used[window];
|
|
121
|
-
const line = plan
|
|
171
|
+
const line = ceiling(request, plan, window);
|
|
122
172
|
if (used + request.jobs > line)
|
|
123
173
|
return window === "day"
|
|
124
174
|
? { _tag: "Hold", line: { _tag: "Account", window, used, line } }
|
|
@@ -128,12 +178,35 @@ export const decide = (
|
|
|
128
178
|
line: { _tag: "Account", window, used, line },
|
|
129
179
|
};
|
|
130
180
|
}
|
|
131
|
-
|
|
181
|
+
const shared = sharedWindows(request, plan);
|
|
182
|
+
if (shared.length === 0) return { _tag: "Admit" };
|
|
183
|
+
if (counts.repo === undefined || !counts.repo.complete)
|
|
184
|
+
return {
|
|
185
|
+
_tag: "Wait",
|
|
186
|
+
until: now + pollMillis.minute,
|
|
187
|
+
line: { _tag: "Unread", reason: "repo start history incomplete" },
|
|
188
|
+
};
|
|
189
|
+
if (
|
|
190
|
+
counts.repo.repo !== request.repo ||
|
|
191
|
+
!counts.repo.starts.every((at) => Number.isFinite(at) && at <= counts.now)
|
|
192
|
+
)
|
|
193
|
+
return unread("repo start history incomplete");
|
|
194
|
+
const share = liveShare(request.repo, counts.merges);
|
|
195
|
+
if (share === undefined)
|
|
196
|
+
return {
|
|
197
|
+
_tag: "Wait",
|
|
198
|
+
until: now + pollMillis.minute,
|
|
199
|
+
line: { _tag: "Unread", reason: "merge history incomplete" },
|
|
200
|
+
};
|
|
201
|
+
for (const window of shared) {
|
|
132
202
|
const starts = counts.repo.starts
|
|
133
203
|
.filter((at) => at > now - windowMillis[window])
|
|
134
204
|
.toSorted((left, right) => left - right);
|
|
135
205
|
const used = starts.length;
|
|
136
|
-
const budget =
|
|
206
|
+
const budget = Number(
|
|
207
|
+
(share.numerator * 4n * BigInt(plan.limits[window])) /
|
|
208
|
+
(share.denominator * 5n)
|
|
209
|
+
);
|
|
137
210
|
if (used + request.jobs <= budget) continue;
|
|
138
211
|
const line: Line = {
|
|
139
212
|
_tag: "Budget",
|
|
@@ -0,0 +1,99 @@
|
|
|
1
|
+
import { Octokit } from "@octokit/rest";
|
|
2
|
+
import * as Clock from "effect/Clock";
|
|
3
|
+
import * as Context from "effect/Context";
|
|
4
|
+
import * as Effect from "effect/Effect";
|
|
5
|
+
import * as Layer from "effect/Layer";
|
|
6
|
+
import * as Redacted from "effect/Redacted";
|
|
7
|
+
import * as Schema from "effect/Schema";
|
|
8
|
+
|
|
9
|
+
import { StartReadError } from "./boat.ts";
|
|
10
|
+
import { type MergeCounts, startRepos } from "./budget.ts";
|
|
11
|
+
|
|
12
|
+
const Count = Schema.Struct({
|
|
13
|
+
issueCount: Schema.Number.check(
|
|
14
|
+
Schema.makeFilter((value) => Number.isSafeInteger(value) && value >= 0)
|
|
15
|
+
),
|
|
16
|
+
});
|
|
17
|
+
const windows = { h6: 21_600_000, h24: 86_400_000, d7: 604_800_000 };
|
|
18
|
+
|
|
19
|
+
export class Merges extends Context.Service<
|
|
20
|
+
Merges,
|
|
21
|
+
{ readonly counts: Effect.Effect<ReadonlyArray<MergeCounts>, StartReadError> }
|
|
22
|
+
>()("toolchain/Merges") {
|
|
23
|
+
static readonly layer = (
|
|
24
|
+
token: Redacted.Redacted<string>,
|
|
25
|
+
options: {
|
|
26
|
+
readonly fetch?: (
|
|
27
|
+
url: string | URL | Request,
|
|
28
|
+
init?: RequestInit
|
|
29
|
+
) => Promise<Response>;
|
|
30
|
+
} = {}
|
|
31
|
+
) =>
|
|
32
|
+
Layer.sync(Merges, () => {
|
|
33
|
+
const api = new Octokit({
|
|
34
|
+
auth: Redacted.value(token),
|
|
35
|
+
...(options.fetch === undefined
|
|
36
|
+
? {}
|
|
37
|
+
: { request: { fetch: options.fetch } }),
|
|
38
|
+
});
|
|
39
|
+
const counts = Effect.gen(function* () {
|
|
40
|
+
const now = yield* Clock.currentTimeMillis;
|
|
41
|
+
const query = `query { ${startRepos
|
|
42
|
+
.flatMap((repo, index) => {
|
|
43
|
+
const [owner, name] = repo.split("/");
|
|
44
|
+
return [
|
|
45
|
+
`repo${index}: repository(owner: "${owner}", name: "${name}") { nameWithOwner }`,
|
|
46
|
+
...Object.entries(windows).map(([window, millis]) => {
|
|
47
|
+
const since = new Date(now - millis)
|
|
48
|
+
.toISOString()
|
|
49
|
+
.replace(/\.\d{3}Z$/u, "Z");
|
|
50
|
+
return `${window}_${index}: search(type: ISSUE, first: 0, query: "repo:${repo} is:pr is:merged base:main merged:>=${since}") { issueCount }`;
|
|
51
|
+
}),
|
|
52
|
+
];
|
|
53
|
+
})
|
|
54
|
+
.join(" ")} }`;
|
|
55
|
+
const response = yield* Effect.tryPromise({
|
|
56
|
+
try: () => api.graphql<unknown>(query),
|
|
57
|
+
catch: () =>
|
|
58
|
+
new StartReadError({ reason: "GitHub merges request failed" }),
|
|
59
|
+
});
|
|
60
|
+
return yield* Schema.decodeUnknownEffect(
|
|
61
|
+
Schema.Record(Schema.String, Schema.Unknown)
|
|
62
|
+
)(response).pipe(
|
|
63
|
+
Effect.flatMap((data) =>
|
|
64
|
+
Effect.forEach(
|
|
65
|
+
startRepos,
|
|
66
|
+
(repo, index) =>
|
|
67
|
+
Schema.decodeUnknownEffect(
|
|
68
|
+
Schema.Struct({
|
|
69
|
+
repository: Schema.Struct({
|
|
70
|
+
nameWithOwner: Schema.Literal(repo),
|
|
71
|
+
}),
|
|
72
|
+
h6: Count,
|
|
73
|
+
h24: Count,
|
|
74
|
+
d7: Count,
|
|
75
|
+
})
|
|
76
|
+
)({
|
|
77
|
+
repository: data[`repo${index}`],
|
|
78
|
+
h6: data[`h6_${index}`],
|
|
79
|
+
h24: data[`h24_${index}`],
|
|
80
|
+
d7: data[`d7_${index}`],
|
|
81
|
+
}).pipe(
|
|
82
|
+
Effect.map((row) => ({
|
|
83
|
+
repo,
|
|
84
|
+
h6: row.h6.issueCount,
|
|
85
|
+
h24: row.h24.issueCount,
|
|
86
|
+
d7: row.d7.issueCount,
|
|
87
|
+
}))
|
|
88
|
+
),
|
|
89
|
+
{ concurrency: 1 }
|
|
90
|
+
)
|
|
91
|
+
),
|
|
92
|
+
Effect.mapError(
|
|
93
|
+
() => new StartReadError({ reason: "GitHub merges unread" })
|
|
94
|
+
)
|
|
95
|
+
);
|
|
96
|
+
});
|
|
97
|
+
return Merges.of({ counts });
|
|
98
|
+
});
|
|
99
|
+
}
|
package/src/starts.ts
CHANGED
|
@@ -7,18 +7,22 @@ import {
|
|
|
7
7
|
decide,
|
|
8
8
|
Line,
|
|
9
9
|
pollMillis,
|
|
10
|
+
sharedWindows,
|
|
10
11
|
type StartRequest,
|
|
11
12
|
} from "./starts/budget.ts";
|
|
13
|
+
import { Merges } from "./starts/merges.ts";
|
|
12
14
|
|
|
13
15
|
export { BoatStarts, StartReadError } from "./starts/boat.ts";
|
|
14
|
-
export { decide, Repo,
|
|
16
|
+
export { decide, Repo, startRepos } from "./starts/budget.ts";
|
|
17
|
+
export { Merges } from "./starts/merges.ts";
|
|
15
18
|
export type {
|
|
16
19
|
Admission,
|
|
17
20
|
Counts,
|
|
21
|
+
MergeCounts,
|
|
18
22
|
PerWindow,
|
|
19
23
|
Plan,
|
|
24
|
+
Priority,
|
|
20
25
|
RepoStarts,
|
|
21
|
-
Share,
|
|
22
26
|
StartRequest,
|
|
23
27
|
Window,
|
|
24
28
|
} from "./starts/budget.ts";
|
|
@@ -38,6 +42,7 @@ export const admitStart = Effect.fn("Starts.admitStart")(function* <A, E, R>(
|
|
|
38
42
|
{ deadline }: { readonly deadline: number }
|
|
39
43
|
) {
|
|
40
44
|
const boat = yield* BoatStarts;
|
|
45
|
+
const merges = yield* Merges;
|
|
41
46
|
let last: Line = {
|
|
42
47
|
_tag: "Unread",
|
|
43
48
|
reason: "start deadline reached",
|
|
@@ -48,8 +53,20 @@ export const admitStart = Effect.fn("Starts.admitStart")(function* <A, E, R>(
|
|
|
48
53
|
return yield* new StartHeld({ line: last });
|
|
49
54
|
const counts = yield* Effect.gen(function* () {
|
|
50
55
|
const plan = yield* boat.plan;
|
|
51
|
-
|
|
52
|
-
|
|
56
|
+
if (sharedWindows(request, plan).length === 0)
|
|
57
|
+
return { now: yield* Clock.currentTimeMillis, plan };
|
|
58
|
+
const repo = yield* boat
|
|
59
|
+
.repoStarts(request.repo)
|
|
60
|
+
.pipe(Effect.catchTag("StartReadError", () => Effect.undefined));
|
|
61
|
+
const history = yield* merges.counts.pipe(
|
|
62
|
+
Effect.catchTag("StartReadError", () => Effect.undefined)
|
|
63
|
+
);
|
|
64
|
+
return {
|
|
65
|
+
now: yield* Clock.currentTimeMillis,
|
|
66
|
+
plan,
|
|
67
|
+
repo,
|
|
68
|
+
merges: history,
|
|
69
|
+
};
|
|
53
70
|
}).pipe(
|
|
54
71
|
Effect.catchTag("StartReadError", () => Effect.undefined),
|
|
55
72
|
Effect.timeoutOrElse({
|