@davidvornholt/standards 0.17.2 → 0.17.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/package.json +3 -1
- package/src/poller-claim-reconciliation.ts +160 -0
- package/src/poller-claim.ts +92 -89
- package/src/poller-comment-reconciliation.ts +29 -0
- package/src/poller-fix-run.ts +13 -10
- package/src/poller-queue-marker.ts +10 -9
- package/src/poller-review-execution.ts +11 -9
- package/src/poller-review-run.ts +17 -21
- package/src/poller-schedule.ts +5 -0
- package/src/poller-status.ts +34 -19
package/README.md
CHANGED
|
@@ -35,7 +35,7 @@ standards poller --print-units --config <path>
|
|
|
35
35
|
|
|
36
36
|
`structure` enforces the canonical monorepo structure contract: workspace scripts (`check-types`, `lint`, `lint:fix`, `test`), root gate scripts, filtered Turbo convenience aliases, internal `0.0.0`/`workspace:*` versioning, package `exports`, shared tsconfig inheritance, and browser a11y wiring. Only a workspace containing an explicit `*.a11y.ts` suite must provide a `test:a11y` script and direct `@axe-core/playwright` and `@playwright/test` dependencies. `check` includes `structure`, so these rules gate every consumer PR.
|
|
37
37
|
|
|
38
|
-
`poller` runs one tick of the fix poller: host-level automation that turns maintainer-approved GitHub issues into verified draft PRs and maintainer-approved draft PRs into completed review-fix cycles, using headless Codex runs. All workflow state lives in GitHub labels, comments, and draft PRs (the labels are canonical declared settings — see `github --apply`), so ticks are stateless, re-runnable, and safe to schedule from any host. A tick claims work by label (`fix-in-progress` / `review-in-progress`), releases claims stale for longer than `staleClaimHours`, and fails loudly — a red tick means jobs stalled, never silence. A separate acknowledgement tick validates pending approvals and posts one deduplicated queued message per approval generation; it never claims or starts work, so it can keep giving maintainers prompt feedback while the worker is occupied. When a worker claims a request, it posts a concise start message. The exact approval and distributed claim records remain in hidden comment metadata, and losing claim contenders remove their duplicate comments. Trust is verified per event, not per configuration: the approval labels (`approved-for-fix`, `approved-for-review`) only count when the applying user holds the `admin` or `maintain` role, bind to the exact approved issue revision or PR head, and are revalidated before publication; only admin/maintain comments are treated as answers to agent questions. Approved Codex runs inherit the poller's GitHub token and use authenticated `gh` for relevant reads and comments; review runs also maintain the PR review ledger, apply authorized title or body edits, post fix threads, and file deferred findings directly. The run works in a throwaway git worktree, grants Codex write access to that worktree and its exact Git common directory so commits can update the linked repository metadata, and hands results back through a structured outcome file. The poller then verifies the approved head, commits, and protected paths before pushing or publishing; it replies to and resolves agent-posted fix threads only after the reviewed head is pushed, and it alone changes protocol labels, posts the final report, and flips a reviewed draft ready, while required CI remains the quality gate. Sealed review plans from the earlier protocol are rejected and their output branches are ignored, so the review reruns on a fresh protocol-versioned branch rather than silently dropping legacy deferred work. Host mutation is declarative-only: `--print-units` emits four correctly escaped systemd units for the polling host's infrastructure repository to adapt and deploy: the worker service and timer plus an independent lightweight acknowledgement service and timer. That repository owns the service identity, writable HOME, PATH, authenticated `gh`, lingering, and deployment. The worker service's tick budget (`TimeoutStartSec`) is derived from `maxJobsPerTick` × `runTimeoutMinutes` plus overhead, so re-render and redeploy the units after changing either value.
|
|
38
|
+
`poller` runs one tick of the fix poller: host-level automation that turns maintainer-approved GitHub issues into verified draft PRs and maintainer-approved draft PRs into completed review-fix cycles, using headless Codex runs. All workflow state lives in GitHub labels, comments, and draft PRs (the labels are canonical declared settings — see `github --apply`), so ticks are stateless, re-runnable, and safe to schedule from any host. A tick claims work by label (`fix-in-progress` / `review-in-progress`), releases claims stale for longer than `staleClaimHours`, and fails loudly — a red tick means jobs stalled, never silence. A separate acknowledgement tick validates pending approvals and posts one deduplicated queued message per approval generation; it never claims or starts work, so it can keep giving maintainers prompt feedback while the worker is occupied. When a worker claims a request, it posts a concise start message. The exact approval and distributed claim records remain in hidden comment metadata, and losing claim contenders remove their duplicate comments. Trust is verified per event, not per configuration: the approval labels (`approved-for-fix`, `approved-for-review`) only count when the applying user holds the `admin` or `maintain` role, bind to the exact approved issue revision or PR head, and are revalidated before publication; only admin/maintain comments are treated as answers to agent questions. Approved Codex runs inherit the poller's GitHub token and use authenticated `gh` for relevant reads and comments; review runs also maintain the PR review ledger, apply authorized title or body edits, post fix threads, and file deferred findings directly. The run works in a throwaway git worktree, grants Codex write access to that worktree and its exact Git common directory so commits can update the linked repository metadata, and hands results back through a structured outcome file. The poller then verifies the approved head, commits, and protected paths before pushing or publishing; it replies to and resolves agent-posted fix threads only after the reviewed head is pushed, and it alone changes protocol labels, posts the final report, and flips a reviewed draft ready, while required CI remains the quality gate. If publishing a sealed fix or review result fails, the poller releases only its owned in-progress claim and keeps the approval and sealed output, so the next worker tick retries publication without another Codex run. Sealed review plans from the earlier protocol are rejected and their output branches are ignored, so the review reruns on a fresh protocol-versioned branch rather than silently dropping legacy deferred work. Host mutation is declarative-only: `--print-units` emits four correctly escaped systemd units for the polling host's infrastructure repository to adapt and deploy: the worker service and timer plus an independent lightweight acknowledgement service and timer. That repository owns the service identity, writable HOME, PATH, authenticated `gh`, lingering, and deployment. The worker service's tick budget (`TimeoutStartSec`) is derived from `maxJobsPerTick` × `runTimeoutMinutes` plus overhead, so re-render and redeploy the units after changing either value.
|
|
39
39
|
|
|
40
40
|
`creds` is the credential broker: it mints scoped Cloudflare tokens, rotates and revokes those tokens, and places GitHub App credentials into SOPS-encrypted targets — secret values never touch stdout, argv, or an agent's context; commands print only metadata (token name, scopes, expiry, destination). Cloudflare reconciliation has exactly two sources of truth: the plaintext key structure of the SOPS files and the provider token list under the deterministic naming scheme `standards/<owner>/<repo>/<sops-target>/<dotted.key>`; there is no separate credential manifest. `plan` and `apply` reconcile Cloudflare only: each run uses the stored bootstrap credential to verify its current provider token ID, excludes that ID from reconciliation regardless of its current name, and aborts if the identity cannot be correlated with the complete account token list; a brokered token whose SOPS key was deleted is revoked, while one nearing expiry (30 days) is renewed by creating a fresh-expiry replacement with the live token's policy, durably writing and verifying the new value, and only then revoking the old token; secrets the broker did not mint are never touched. Without a manifest there is no desired-policy record, so live policy drift is not detected, and a repository rename or transfer changes the deterministic namespace and requires the visible old token to be re-minted under the new name and revoked. `add cloudflare` mints an account-owned API token (default TTL 90 days) scoped to the named permission groups (`permissions` lists the valid names) and writes it to `--dest <target>:<dotted.key>`, where `<target>` resolves like the canonical `just secrets` targets (`ci` → `secrets/ci.yaml`, `prod-1` → `infra/hosts/prod-1/secrets.yaml`); zone-scoped groups such as DNS Write are rejected because the current command supplies an account resource, not a zone resource. `--bucket <name>` switches the policy resource to a single R2 bucket; `--jurisdiction default|eu` selects the provider jurisdiction and defaults to `default`, producing `com.cloudflare.edge.r2.bucket.<account_id>_default_<bucket>` or `com.cloudflare.edge.r2.bucket.<account_id>_eu_<bucket>`. FedRAMP jurisdictions are not supported. Bucket destinations accept only bucket-scoped groups such as Workers R2 Storage Bucket Item Read/Write, and account-scoped groups are rejected. `--s3` requires `--bucket` and stores the S3-compatible credential pair Cloudflare derives from an API token — access key ID is the token ID, secret access key is the SHA-256 hex digest of the token value — as `<dotted.key>.access_key_id` and `<dotted.key>.secret_access_key` under the destination instead of the raw token value; the default endpoint is `https://<account_id>.r2.cloudflarestorage.com`, while EU uses `https://<account_id>.eu.r2.cloudflarestorage.com`. Reconciliation infers each destination's concrete write footprint from the plaintext SOPS key structure — a bearer owns the leaf at its key, while an S3 token owns both pair leaves — and aborts before mutation when managed footprints intersect. `plan` and `apply` renew and revoke S3 destinations like any other brokered token, but renewal first proves the stored access key ID matches the current provider token; an incomplete pair, unreadable value, or identity mismatch is surfaced as a finding and never mutated automatically. `add github` writes the broker GitHub App's `app_id` and `private_key` under the destination key; durable GitHub tokens cannot be created via API (PAT and OAuth-app creation are UI-only), so workflows mint short-lived installation tokens from the App at runtime (`actions/create-github-app-token`), scoped per repository and permission. The canonical weekly sync expects `bun standards creds add github --dest ci:ci.broker_app`, then mints a current-repository token with Contents read and Pull requests write; missing credentials or minting failures stop it without a fallback. Bootstrap is once per machine: `login github` runs the App manifest flow (a localhost page posts the pre-filled manifest; one "Create GitHub App" click; credentials return through the conversion endpoint — nothing is copied by hand) and opens the installation page, where the App must be bound to selected repositories rather than all repositories; `login cloudflare` guides the one dashboard paste Cloudflare requires for the first account token (named `standards-broker` to match the App, created from the "Create additional tokens" token template because its "Account API Tokens: Edit" permission is unavailable in the custom token builder, and holding that permission only), verifies that it is active, correlates its verified ID with the complete account token list and an active list record, and stores it; a bootstrap token named inside the minted `standards/` namespace is rejected before storage to keep its role unambiguous, while any other divergence from the recommended name only warns. `plan` and `apply` do not rotate GitHub App keys; create a replacement key on the App's settings page, atomically replace `github.private_key` in the protected machine-global broker store while preserving its `0600` mode, then update every intended SOPS destination, verify workflows can mint installation tokens, and revoke the old key manually. Root credentials live in the machine-global broker store (see Configuration); SOPS writes go through `sops edit` with a non-interactive editor, preserving comments and reusing the file's data key.
|
|
41
41
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@davidvornholt/standards",
|
|
3
|
-
"version": "0.17.
|
|
3
|
+
"version": "0.17.3",
|
|
4
4
|
"description": "Bootstrap, synchronize, and validate davidvornholt/standards consumers",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -84,6 +84,8 @@
|
|
|
84
84
|
"src/poller-codex-stderr.ts",
|
|
85
85
|
"src/poller-commands.ts",
|
|
86
86
|
"src/poller-comment-metadata.ts",
|
|
87
|
+
"src/poller-comment-reconciliation.ts",
|
|
88
|
+
"src/poller-claim-reconciliation.ts",
|
|
87
89
|
"src/poller-config.ts",
|
|
88
90
|
"src/poller-fix-outcome.ts",
|
|
89
91
|
"src/poller-fix-output.ts",
|
|
@@ -0,0 +1,160 @@
|
|
|
1
|
+
import { isRecord } from './github-settings-parse';
|
|
2
|
+
import type { ApprovalBinding } from './poller-approval';
|
|
3
|
+
import { parseHiddenCommentMetadata } from './poller-comment-metadata';
|
|
4
|
+
import { retainEarliestComment } from './poller-comment-reconciliation';
|
|
5
|
+
import {
|
|
6
|
+
collaboratorRole,
|
|
7
|
+
type IssueComment,
|
|
8
|
+
listIssueComments,
|
|
9
|
+
} from './poller-github';
|
|
10
|
+
import { CLAIM_METADATA_MARKER, isTrustedRole } from './poller-protocol';
|
|
11
|
+
|
|
12
|
+
export type ClaimMarkerBinding = {
|
|
13
|
+
readonly approval: ApprovalBinding;
|
|
14
|
+
readonly claimLabel: string;
|
|
15
|
+
readonly claimEpoch: string;
|
|
16
|
+
};
|
|
17
|
+
|
|
18
|
+
type ClaimMarker = ClaimMarkerBinding & {
|
|
19
|
+
readonly nonce: string;
|
|
20
|
+
};
|
|
21
|
+
|
|
22
|
+
type ClaimContext = {
|
|
23
|
+
readonly token: string | null;
|
|
24
|
+
readonly repo: string;
|
|
25
|
+
readonly issueNumber: number;
|
|
26
|
+
};
|
|
27
|
+
|
|
28
|
+
const isApprovalBinding = (value: unknown): value is ApprovalBinding =>
|
|
29
|
+
isRecord(value) &&
|
|
30
|
+
typeof value.id === 'string' &&
|
|
31
|
+
typeof value.repo === 'string' &&
|
|
32
|
+
typeof value.issueNumber === 'number' &&
|
|
33
|
+
typeof value.eventId === 'number' &&
|
|
34
|
+
typeof value.label === 'string' &&
|
|
35
|
+
typeof value.actorLogin === 'string' &&
|
|
36
|
+
typeof value.approvedAt === 'string' &&
|
|
37
|
+
typeof value.target === 'string';
|
|
38
|
+
|
|
39
|
+
const parseMarker = (comment: IssueComment): ClaimMarker | null => {
|
|
40
|
+
const payload = parseHiddenCommentMetadata(
|
|
41
|
+
comment.body,
|
|
42
|
+
CLAIM_METADATA_MARKER,
|
|
43
|
+
);
|
|
44
|
+
if (!isRecord(payload)) {
|
|
45
|
+
return null;
|
|
46
|
+
}
|
|
47
|
+
if (
|
|
48
|
+
typeof payload.claimLabel !== 'string' ||
|
|
49
|
+
typeof payload.claimEpoch !== 'string' ||
|
|
50
|
+
typeof payload.nonce !== 'string' ||
|
|
51
|
+
!isApprovalBinding(payload.approval)
|
|
52
|
+
) {
|
|
53
|
+
return null;
|
|
54
|
+
}
|
|
55
|
+
return {
|
|
56
|
+
approval: payload.approval,
|
|
57
|
+
claimLabel: payload.claimLabel,
|
|
58
|
+
claimEpoch: payload.claimEpoch,
|
|
59
|
+
nonce: payload.nonce,
|
|
60
|
+
};
|
|
61
|
+
};
|
|
62
|
+
|
|
63
|
+
const trustedMarkers = async (
|
|
64
|
+
context: ClaimContext,
|
|
65
|
+
matches: (marker: ClaimMarker) => boolean,
|
|
66
|
+
): Promise<
|
|
67
|
+
ReadonlyArray<{ readonly id: number; readonly marker: ClaimMarker }>
|
|
68
|
+
> => {
|
|
69
|
+
const comments = await listIssueComments(
|
|
70
|
+
context.token,
|
|
71
|
+
context.repo,
|
|
72
|
+
context.issueNumber,
|
|
73
|
+
);
|
|
74
|
+
const markers: Array<{ readonly id: number; readonly marker: ClaimMarker }> =
|
|
75
|
+
[];
|
|
76
|
+
for (const comment of comments) {
|
|
77
|
+
const marker = parseMarker(comment);
|
|
78
|
+
if (marker !== null && matches(marker)) {
|
|
79
|
+
// biome-ignore lint/performance/noAwaitInLoops: claim authors must be checked against their current role; the usually-one-item list is deliberately fail-closed.
|
|
80
|
+
const role = await collaboratorRole(
|
|
81
|
+
context.token,
|
|
82
|
+
context.repo,
|
|
83
|
+
comment.authorLogin,
|
|
84
|
+
);
|
|
85
|
+
if (isTrustedRole(role)) {
|
|
86
|
+
markers.push({ id: comment.id, marker });
|
|
87
|
+
}
|
|
88
|
+
}
|
|
89
|
+
}
|
|
90
|
+
return markers;
|
|
91
|
+
};
|
|
92
|
+
|
|
93
|
+
const claimGroupKey = (approvalId: string, claimEpoch: string): string =>
|
|
94
|
+
JSON.stringify([approvalId, claimEpoch]);
|
|
95
|
+
|
|
96
|
+
const reconcileTrustedClaimGroups = async (
|
|
97
|
+
context: ClaimContext,
|
|
98
|
+
claimLabel: string,
|
|
99
|
+
): Promise<
|
|
100
|
+
ReadonlyMap<
|
|
101
|
+
string,
|
|
102
|
+
{
|
|
103
|
+
readonly markerIds: ReadonlyArray<number>;
|
|
104
|
+
readonly retainedId: number | null;
|
|
105
|
+
}
|
|
106
|
+
>
|
|
107
|
+
> => {
|
|
108
|
+
const groups = new Map<string, Array<number>>();
|
|
109
|
+
const markers = await trustedMarkers(
|
|
110
|
+
context,
|
|
111
|
+
(marker) => marker.claimLabel === claimLabel,
|
|
112
|
+
);
|
|
113
|
+
for (const { id, marker } of markers) {
|
|
114
|
+
const key = claimGroupKey(marker.approval.id, marker.claimEpoch);
|
|
115
|
+
const ids = groups.get(key) ?? [];
|
|
116
|
+
ids.push(id);
|
|
117
|
+
groups.set(key, ids);
|
|
118
|
+
}
|
|
119
|
+
const reconciled = new Map<
|
|
120
|
+
string,
|
|
121
|
+
{
|
|
122
|
+
readonly markerIds: ReadonlyArray<number>;
|
|
123
|
+
readonly retainedId: number | null;
|
|
124
|
+
}
|
|
125
|
+
>();
|
|
126
|
+
for (const [key, ids] of groups) {
|
|
127
|
+
// biome-ignore lint/performance/noAwaitInLoops: each group contains GitHub mutations and groups are deliberately reconciled serially.
|
|
128
|
+
const retainedId = await retainEarliestComment(context, ids);
|
|
129
|
+
reconciled.set(key, { markerIds: ids, retainedId });
|
|
130
|
+
}
|
|
131
|
+
return reconciled;
|
|
132
|
+
};
|
|
133
|
+
|
|
134
|
+
export const reconcileTrustedClaimElection = async (
|
|
135
|
+
context: ClaimContext,
|
|
136
|
+
binding: ClaimMarkerBinding,
|
|
137
|
+
): Promise<{
|
|
138
|
+
readonly markerIds: ReadonlyArray<number>;
|
|
139
|
+
readonly retainedId: number | null;
|
|
140
|
+
}> => {
|
|
141
|
+
const markers = await trustedMarkers(
|
|
142
|
+
context,
|
|
143
|
+
(marker) =>
|
|
144
|
+
marker.approval.id === binding.approval.id &&
|
|
145
|
+
marker.claimLabel === binding.claimLabel &&
|
|
146
|
+
marker.claimEpoch === binding.claimEpoch,
|
|
147
|
+
);
|
|
148
|
+
const markerIds = markers.map(({ id }) => id);
|
|
149
|
+
return {
|
|
150
|
+
markerIds,
|
|
151
|
+
retainedId: await retainEarliestComment(context, markerIds),
|
|
152
|
+
};
|
|
153
|
+
};
|
|
154
|
+
|
|
155
|
+
export const reconcileTrustedClaimHistory = async (
|
|
156
|
+
context: ClaimContext,
|
|
157
|
+
claimLabel: string,
|
|
158
|
+
): Promise<void> => {
|
|
159
|
+
await reconcileTrustedClaimGroups(context, claimLabel);
|
|
160
|
+
};
|
package/src/poller-claim.ts
CHANGED
|
@@ -1,34 +1,33 @@
|
|
|
1
1
|
import { randomUUID } from 'node:crypto';
|
|
2
2
|
import { hasLabel } from './github-label-identity';
|
|
3
|
-
import { isRecord } from './github-settings-parse';
|
|
4
3
|
import { type ApprovalBinding, readApprovalBinding } from './poller-approval';
|
|
5
4
|
import {
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
5
|
+
type ClaimMarkerBinding,
|
|
6
|
+
reconcileTrustedClaimElection,
|
|
7
|
+
reconcileTrustedClaimHistory,
|
|
8
|
+
} from './poller-claim-reconciliation';
|
|
9
|
+
import { hiddenCommentMetadata } from './poller-comment-metadata';
|
|
10
|
+
import { getIssue, lastLabelEvent } from './poller-github';
|
|
9
11
|
import {
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
} from './poller-
|
|
16
|
-
import { createComment, deleteComment } from './poller-github-write';
|
|
12
|
+
addLabels,
|
|
13
|
+
createComment,
|
|
14
|
+
deleteComment,
|
|
15
|
+
removeLabel,
|
|
16
|
+
} from './poller-github-write';
|
|
17
|
+
import type { JobDeps } from './poller-job-shared';
|
|
17
18
|
import {
|
|
18
19
|
CLAIM_METADATA_MARKER,
|
|
19
20
|
FIX_IN_PROGRESS,
|
|
20
|
-
isTrustedRole,
|
|
21
21
|
REVIEW_IN_PROGRESS,
|
|
22
22
|
} from './poller-protocol';
|
|
23
|
+
import { inProgressLabelFor, type PollerJobKind } from './poller-queue-marker';
|
|
24
|
+
import { reconcileExistingQueuedJob } from './poller-status';
|
|
23
25
|
|
|
24
|
-
export type ClaimBinding = {
|
|
25
|
-
readonly approval: ApprovalBinding;
|
|
26
|
-
readonly claimLabel: string;
|
|
27
|
-
readonly claimEpoch: string;
|
|
26
|
+
export type ClaimBinding = ClaimMarkerBinding & {
|
|
28
27
|
readonly markerId: number;
|
|
29
28
|
};
|
|
30
29
|
|
|
31
|
-
type ClaimMarker =
|
|
30
|
+
type ClaimMarker = ClaimMarkerBinding & {
|
|
32
31
|
readonly nonce: string;
|
|
33
32
|
};
|
|
34
33
|
|
|
@@ -54,73 +53,6 @@ const markerBody = (marker: ClaimMarker): string =>
|
|
|
54
53
|
marker,
|
|
55
54
|
)}`;
|
|
56
55
|
|
|
57
|
-
const isApprovalBinding = (value: unknown): value is ApprovalBinding =>
|
|
58
|
-
isRecord(value) &&
|
|
59
|
-
typeof value.id === 'string' &&
|
|
60
|
-
typeof value.repo === 'string' &&
|
|
61
|
-
typeof value.issueNumber === 'number' &&
|
|
62
|
-
typeof value.eventId === 'number' &&
|
|
63
|
-
typeof value.label === 'string' &&
|
|
64
|
-
typeof value.actorLogin === 'string' &&
|
|
65
|
-
typeof value.approvedAt === 'string' &&
|
|
66
|
-
typeof value.target === 'string';
|
|
67
|
-
|
|
68
|
-
const parseMarker = (comment: IssueComment): ClaimMarker | null => {
|
|
69
|
-
const payload = parseHiddenCommentMetadata(
|
|
70
|
-
comment.body,
|
|
71
|
-
CLAIM_METADATA_MARKER,
|
|
72
|
-
);
|
|
73
|
-
if (!isRecord(payload)) {
|
|
74
|
-
return null;
|
|
75
|
-
}
|
|
76
|
-
const raw = payload;
|
|
77
|
-
if (
|
|
78
|
-
typeof raw.claimLabel !== 'string' ||
|
|
79
|
-
typeof raw.claimEpoch !== 'string' ||
|
|
80
|
-
typeof raw.nonce !== 'string' ||
|
|
81
|
-
!isApprovalBinding(raw.approval)
|
|
82
|
-
) {
|
|
83
|
-
return null;
|
|
84
|
-
}
|
|
85
|
-
return {
|
|
86
|
-
approval: raw.approval,
|
|
87
|
-
claimLabel: raw.claimLabel,
|
|
88
|
-
claimEpoch: raw.claimEpoch,
|
|
89
|
-
nonce: raw.nonce,
|
|
90
|
-
};
|
|
91
|
-
};
|
|
92
|
-
|
|
93
|
-
const winningMarkerId = async (
|
|
94
|
-
context: ClaimContext,
|
|
95
|
-
binding: Omit<ClaimBinding, 'markerId'>,
|
|
96
|
-
): Promise<number | null> => {
|
|
97
|
-
const comments = await listIssueComments(
|
|
98
|
-
context.token,
|
|
99
|
-
context.repo,
|
|
100
|
-
context.issueNumber,
|
|
101
|
-
);
|
|
102
|
-
let winner: number | null = null;
|
|
103
|
-
for (const comment of comments) {
|
|
104
|
-
const marker = parseMarker(comment);
|
|
105
|
-
const differs =
|
|
106
|
-
marker?.approval.id !== binding.approval.id ||
|
|
107
|
-
marker.claimLabel !== binding.claimLabel ||
|
|
108
|
-
marker.claimEpoch !== binding.claimEpoch;
|
|
109
|
-
if (!differs) {
|
|
110
|
-
// biome-ignore lint/performance/noAwaitInLoops: claim authors must be checked against their current role; the usually-one-item list is deliberately fail-closed.
|
|
111
|
-
const role = await collaboratorRole(
|
|
112
|
-
context.token,
|
|
113
|
-
context.repo,
|
|
114
|
-
comment.authorLogin,
|
|
115
|
-
);
|
|
116
|
-
if (isTrustedRole(role) && (winner === null || comment.id < winner)) {
|
|
117
|
-
winner = comment.id;
|
|
118
|
-
}
|
|
119
|
-
}
|
|
120
|
-
}
|
|
121
|
-
return winner;
|
|
122
|
-
};
|
|
123
|
-
|
|
124
56
|
export const acquireClaim = async (
|
|
125
57
|
context: ClaimContext,
|
|
126
58
|
approval: ApprovalBinding,
|
|
@@ -147,14 +79,36 @@ export const acquireClaim = async (
|
|
|
147
79
|
context.issueNumber,
|
|
148
80
|
markerBody({ ...provisional, nonce }),
|
|
149
81
|
);
|
|
150
|
-
const
|
|
151
|
-
if (
|
|
152
|
-
|
|
82
|
+
const election = await reconcileTrustedClaimElection(context, provisional);
|
|
83
|
+
if (election.retainedId !== markerId) {
|
|
84
|
+
if (!election.markerIds.includes(markerId)) {
|
|
85
|
+
await deleteComment(context.token, context.repo, markerId);
|
|
86
|
+
}
|
|
153
87
|
return null;
|
|
154
88
|
}
|
|
155
89
|
return { ...provisional, markerId };
|
|
156
90
|
};
|
|
157
91
|
|
|
92
|
+
export const startClaim = async (
|
|
93
|
+
deps: JobDeps,
|
|
94
|
+
issueNumber: number,
|
|
95
|
+
approval: ApprovalBinding,
|
|
96
|
+
kind: PollerJobKind,
|
|
97
|
+
): Promise<ClaimBinding | null> => {
|
|
98
|
+
await reconcileExistingQueuedJob(deps, issueNumber, approval, kind);
|
|
99
|
+
const claimLabel = inProgressLabelFor(kind);
|
|
100
|
+
await reconcileTrustedClaimHistory(
|
|
101
|
+
{ token: deps.token, repo: deps.repo, issueNumber },
|
|
102
|
+
claimLabel,
|
|
103
|
+
);
|
|
104
|
+
await addLabels(deps.token, deps.repo, issueNumber, [claimLabel]);
|
|
105
|
+
return acquireClaim(
|
|
106
|
+
{ token: deps.token, repo: deps.repo, issueNumber },
|
|
107
|
+
approval,
|
|
108
|
+
claimLabel,
|
|
109
|
+
);
|
|
110
|
+
};
|
|
111
|
+
|
|
158
112
|
export const validateClaim = async (
|
|
159
113
|
context: ClaimContext,
|
|
160
114
|
claim: ClaimBinding,
|
|
@@ -182,8 +136,57 @@ export const validateClaim = async (
|
|
|
182
136
|
if (!hasLabel(issue.labels, claim.claimLabel)) {
|
|
183
137
|
return `"${claim.claimLabel}" is no longer present`;
|
|
184
138
|
}
|
|
185
|
-
const
|
|
186
|
-
return
|
|
139
|
+
const election = await reconcileTrustedClaimElection(context, claim);
|
|
140
|
+
return election.retainedId === claim.markerId
|
|
187
141
|
? null
|
|
188
142
|
: 'claim ownership changed or could not be proven';
|
|
189
143
|
};
|
|
144
|
+
|
|
145
|
+
const errorMessage = (error: unknown): string =>
|
|
146
|
+
error instanceof Error ? error.message : String(error);
|
|
147
|
+
|
|
148
|
+
const releaseOwnedClaim = async (
|
|
149
|
+
deps: JobDeps,
|
|
150
|
+
issueNumber: number,
|
|
151
|
+
claim: ClaimBinding,
|
|
152
|
+
): Promise<void> => {
|
|
153
|
+
const context = { token: deps.token, repo: deps.repo, issueNumber };
|
|
154
|
+
const election = await reconcileTrustedClaimElection(context, claim);
|
|
155
|
+
const issue = await getIssue(deps.token, deps.repo, issueNumber);
|
|
156
|
+
const event = await lastLabelEvent(
|
|
157
|
+
deps.token,
|
|
158
|
+
deps.repo,
|
|
159
|
+
issueNumber,
|
|
160
|
+
claim.claimLabel,
|
|
161
|
+
);
|
|
162
|
+
if (
|
|
163
|
+
election.retainedId === claim.markerId &&
|
|
164
|
+
hasLabel(issue.labels, claim.claimLabel) &&
|
|
165
|
+
String(event?.id) === claim.claimEpoch
|
|
166
|
+
) {
|
|
167
|
+
await removeLabel(deps.token, deps.repo, issueNumber, claim.claimLabel);
|
|
168
|
+
}
|
|
169
|
+
};
|
|
170
|
+
|
|
171
|
+
export const withClaimReleasedOnFailure = async <Result>(
|
|
172
|
+
deps: JobDeps,
|
|
173
|
+
issueNumber: number,
|
|
174
|
+
claim: ClaimBinding,
|
|
175
|
+
operation: () => Promise<Result>,
|
|
176
|
+
): Promise<Result> => {
|
|
177
|
+
try {
|
|
178
|
+
return await operation();
|
|
179
|
+
} catch (operationError) {
|
|
180
|
+
try {
|
|
181
|
+
await releaseOwnedClaim(deps, issueNumber, claim);
|
|
182
|
+
} catch (releaseError) {
|
|
183
|
+
throw new Error(
|
|
184
|
+
`${errorMessage(operationError)}; releasing the claim for an automatic retry also failed: ${errorMessage(releaseError)}`,
|
|
185
|
+
{
|
|
186
|
+
cause: releaseError,
|
|
187
|
+
},
|
|
188
|
+
);
|
|
189
|
+
}
|
|
190
|
+
throw operationError;
|
|
191
|
+
}
|
|
192
|
+
};
|
|
@@ -0,0 +1,29 @@
|
|
|
1
|
+
import { deleteComment } from './poller-github-write';
|
|
2
|
+
|
|
3
|
+
type CommentContext = {
|
|
4
|
+
readonly token: string | null;
|
|
5
|
+
readonly repo: string;
|
|
6
|
+
};
|
|
7
|
+
|
|
8
|
+
export const earliestCommentId = (
|
|
9
|
+
commentIds: ReadonlyArray<number>,
|
|
10
|
+
): number | null =>
|
|
11
|
+
commentIds.reduce<number | null>(
|
|
12
|
+
(earliest, commentId) =>
|
|
13
|
+
earliest === null || commentId < earliest ? commentId : earliest,
|
|
14
|
+
null,
|
|
15
|
+
);
|
|
16
|
+
|
|
17
|
+
export const retainEarliestComment = async (
|
|
18
|
+
context: CommentContext,
|
|
19
|
+
commentIds: ReadonlyArray<number>,
|
|
20
|
+
): Promise<number | null> => {
|
|
21
|
+
const retainedId = earliestCommentId(commentIds);
|
|
22
|
+
for (const commentId of commentIds) {
|
|
23
|
+
if (commentId !== retainedId) {
|
|
24
|
+
// biome-ignore lint/performance/noAwaitInLoops: GitHub mutations are deliberately serialized to avoid secondary rate limits.
|
|
25
|
+
await deleteComment(context.token, context.repo, commentId);
|
|
26
|
+
}
|
|
27
|
+
}
|
|
28
|
+
return retainedId;
|
|
29
|
+
};
|
package/src/poller-fix-run.ts
CHANGED
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
import { join } from 'node:path';
|
|
4
4
|
import { hasLabel } from './github-label-identity';
|
|
5
5
|
import { approvalIsCurrent, issueRevision } from './poller-approval';
|
|
6
|
-
import {
|
|
6
|
+
import { startClaim, withClaimReleasedOnFailure } from './poller-claim';
|
|
7
7
|
import { runCodex } from './poller-codex';
|
|
8
8
|
import { handleNonFixedOutcome } from './poller-fix-outcome';
|
|
9
9
|
import { readSealedFixOutput } from './poller-fix-output';
|
|
@@ -14,7 +14,6 @@ import {
|
|
|
14
14
|
validateFixClaim,
|
|
15
15
|
} from './poller-fix-publication';
|
|
16
16
|
import { getIssue, type IssueItem } from './poller-github';
|
|
17
|
-
import { addLabels } from './poller-github-write';
|
|
18
17
|
import {
|
|
19
18
|
failJob,
|
|
20
19
|
type JobDeps,
|
|
@@ -118,12 +117,7 @@ export const runFixJob = async (
|
|
|
118
117
|
await acknowledgeQueuedJob(deps, issue.number, preamble.approval, 'fix');
|
|
119
118
|
return result(issue.number, 'waiting for run capacity');
|
|
120
119
|
}
|
|
121
|
-
await
|
|
122
|
-
const claim = await acquireClaim(
|
|
123
|
-
{ token, repo, issueNumber: issue.number },
|
|
124
|
-
preamble.approval,
|
|
125
|
-
FIX_IN_PROGRESS,
|
|
126
|
-
);
|
|
120
|
+
const claim = await startClaim(deps, issue.number, preamble.approval, 'fix');
|
|
127
121
|
if (claim === null) {
|
|
128
122
|
return result(issue.number, 'another poller owns the claim');
|
|
129
123
|
}
|
|
@@ -138,7 +132,11 @@ export const runFixJob = async (
|
|
|
138
132
|
if (sealed !== null) {
|
|
139
133
|
assertSealedOwnership(sealed, issue.number, claim.approval.id, branch);
|
|
140
134
|
return {
|
|
141
|
-
lines: [
|
|
135
|
+
lines: [
|
|
136
|
+
await withClaimReleasedOnFailure(deps, issue.number, claim, () =>
|
|
137
|
+
publishFixedOutput(resumableJob, FIX_LABELS, sealed, null),
|
|
138
|
+
),
|
|
139
|
+
],
|
|
142
140
|
ranCodex: false,
|
|
143
141
|
};
|
|
144
142
|
}
|
|
@@ -186,8 +184,13 @@ export const runFixJob = async (
|
|
|
186
184
|
};
|
|
187
185
|
}
|
|
188
186
|
const nonFixed = await handleNonFixedOutcome(job, FIX_LABELS, outcome);
|
|
187
|
+
const message =
|
|
188
|
+
nonFixed ??
|
|
189
|
+
(await withClaimReleasedOnFailure(deps, issue.number, claim, () =>
|
|
190
|
+
finishFixedJob(job, FIX_LABELS, outcome),
|
|
191
|
+
));
|
|
189
192
|
return {
|
|
190
|
-
lines: [
|
|
193
|
+
lines: [message],
|
|
191
194
|
ranCodex: true,
|
|
192
195
|
};
|
|
193
196
|
} finally {
|
|
@@ -182,13 +182,14 @@ export const inspectQueuedAcknowledgement = async (options: {
|
|
|
182
182
|
item.number,
|
|
183
183
|
expectedLabel,
|
|
184
184
|
);
|
|
185
|
-
|
|
186
|
-
|
|
187
|
-
|
|
188
|
-
|
|
189
|
-
|
|
190
|
-
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
);
|
|
185
|
+
if (event === null) {
|
|
186
|
+
return false;
|
|
187
|
+
}
|
|
188
|
+
const currentCheckpointCount = checkpoints.filter(
|
|
189
|
+
(checkpoint) =>
|
|
190
|
+
checkpoint.eventId === event.id &&
|
|
191
|
+
checkpoint.actorLogin === event.actorLogin &&
|
|
192
|
+
checkpoint.approvedAt === event.createdAt,
|
|
193
|
+
).length;
|
|
194
|
+
return currentCheckpointCount === 1;
|
|
194
195
|
};
|
|
@@ -1,5 +1,5 @@
|
|
|
1
1
|
import { join } from 'node:path';
|
|
2
|
-
import type
|
|
2
|
+
import { type ClaimBinding, withClaimReleasedOnFailure } from './poller-claim';
|
|
3
3
|
import type { runCodex } from './poller-codex';
|
|
4
4
|
import type { PullRequest } from './poller-github-pulls';
|
|
5
5
|
import {
|
|
@@ -95,14 +95,16 @@ export const executeReviewJob = async (options: {
|
|
|
95
95
|
}
|
|
96
96
|
return {
|
|
97
97
|
lines: [
|
|
98
|
-
await
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
98
|
+
await withClaimReleasedOnFailure(deps, pr.number, claim, () =>
|
|
99
|
+
finishReviewedJob({
|
|
100
|
+
deps,
|
|
101
|
+
labels,
|
|
102
|
+
pr,
|
|
103
|
+
claim,
|
|
104
|
+
workDir: workspace.dir,
|
|
105
|
+
outcome,
|
|
106
|
+
}),
|
|
107
|
+
),
|
|
106
108
|
],
|
|
107
109
|
ranCodex: true,
|
|
108
110
|
};
|
package/src/poller-review-run.ts
CHANGED
|
@@ -6,11 +6,10 @@
|
|
|
6
6
|
|
|
7
7
|
import { hasLabel } from './github-label-identity';
|
|
8
8
|
import { approvalIsCurrent, prRevision } from './poller-approval';
|
|
9
|
-
import {
|
|
9
|
+
import { startClaim, withClaimReleasedOnFailure } from './poller-claim';
|
|
10
10
|
import { runCodex } from './poller-codex';
|
|
11
11
|
import { getIssue, type IssueItem } from './poller-github';
|
|
12
12
|
import { getPullRequest } from './poller-github-pulls';
|
|
13
|
-
import { addLabels } from './poller-github-write';
|
|
14
13
|
import {
|
|
15
14
|
failJob,
|
|
16
15
|
type JobDeps,
|
|
@@ -86,7 +85,7 @@ export const runReviewJob = async (
|
|
|
86
85
|
};
|
|
87
86
|
}
|
|
88
87
|
const pr = await getPullRequest(token, repo, prItem.number);
|
|
89
|
-
|
|
88
|
+
const plan = await currentReviewPlan(deps, pr);
|
|
90
89
|
const readTarget = () => currentReviewTarget(token, repo, pr.number, plan);
|
|
91
90
|
const preamble = await jobPreamble(deps, currentItem, REVIEW_LABELS, {
|
|
92
91
|
approved: prRevision(
|
|
@@ -157,32 +156,29 @@ export const runReviewJob = async (
|
|
|
157
156
|
ranCodex: false,
|
|
158
157
|
};
|
|
159
158
|
}
|
|
160
|
-
await
|
|
161
|
-
const claim = await acquireClaim(
|
|
162
|
-
{ token, repo, issueNumber: pr.number },
|
|
163
|
-
preamble.approval,
|
|
164
|
-
REVIEW_IN_PROGRESS,
|
|
165
|
-
);
|
|
159
|
+
const claim = await startClaim(deps, pr.number, preamble.approval, 'review');
|
|
166
160
|
if (claim === null) {
|
|
167
161
|
return {
|
|
168
162
|
lines: [`PR #${pr.number}: another poller owns the claim`],
|
|
169
163
|
ranCodex: false,
|
|
170
164
|
};
|
|
171
165
|
}
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
plan = sealed;
|
|
175
|
-
}
|
|
176
|
-
if (plan !== null) {
|
|
166
|
+
const resumablePlan = plan ?? sealed;
|
|
167
|
+
if (resumablePlan !== null) {
|
|
177
168
|
return {
|
|
178
169
|
lines: [
|
|
179
|
-
await
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
185
|
-
|
|
170
|
+
await withClaimReleasedOnFailure(deps, pr.number, claim, async () => {
|
|
171
|
+
if (plan === null) {
|
|
172
|
+
await publishReviewPlan(deps, pr, claim, resumablePlan);
|
|
173
|
+
}
|
|
174
|
+
return resumeReviewedJob({
|
|
175
|
+
deps,
|
|
176
|
+
labels: REVIEW_LABELS,
|
|
177
|
+
pr,
|
|
178
|
+
claim,
|
|
179
|
+
plan: resumablePlan,
|
|
180
|
+
cloneDir: cacheClone,
|
|
181
|
+
});
|
|
186
182
|
}),
|
|
187
183
|
],
|
|
188
184
|
ranCodex: false,
|
package/src/poller-schedule.ts
CHANGED
|
@@ -1,4 +1,5 @@
|
|
|
1
1
|
import { hasLabel } from './github-label-identity';
|
|
2
|
+
import { reconcileTrustedClaimHistory } from './poller-claim-reconciliation';
|
|
2
3
|
import {
|
|
3
4
|
type IssueItem,
|
|
4
5
|
lastLabelEvent,
|
|
@@ -47,6 +48,10 @@ const sweepStaleClaims = async (
|
|
|
47
48
|
? Number.POSITIVE_INFINITY
|
|
48
49
|
: now - Date.parse(event.createdAt);
|
|
49
50
|
if (ageMs > config.staleClaimHours * MS_PER_HOUR) {
|
|
51
|
+
await reconcileTrustedClaimHistory(
|
|
52
|
+
{ token, repo, issueNumber: item.number },
|
|
53
|
+
claimLabel,
|
|
54
|
+
);
|
|
50
55
|
await removeLabel(token, repo, item.number, claimLabel);
|
|
51
56
|
await createComment(
|
|
52
57
|
token,
|
package/src/poller-status.ts
CHANGED
|
@@ -6,6 +6,7 @@ import {
|
|
|
6
6
|
readApprovalBinding,
|
|
7
7
|
} from './poller-approval';
|
|
8
8
|
import { hiddenCommentMetadata } from './poller-comment-metadata';
|
|
9
|
+
import { retainEarliestComment } from './poller-comment-reconciliation';
|
|
9
10
|
import { getIssue } from './poller-github';
|
|
10
11
|
import { getPullRequest } from './poller-github-pulls';
|
|
11
12
|
import { createComment, deleteComment } from './poller-github-write';
|
|
@@ -94,16 +95,14 @@ export const acknowledgeQueuedJob = async (
|
|
|
94
95
|
) {
|
|
95
96
|
return false;
|
|
96
97
|
}
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
).length > 0
|
|
106
|
-
) {
|
|
98
|
+
const existingMarkerIds = await matchingTrustedQueueCommentIds(
|
|
99
|
+
deps,
|
|
100
|
+
issueNumber,
|
|
101
|
+
expected.approvalId,
|
|
102
|
+
kind,
|
|
103
|
+
);
|
|
104
|
+
if (existingMarkerIds.length > 0) {
|
|
105
|
+
await retainEarliestComment(deps, existingMarkerIds);
|
|
107
106
|
return false;
|
|
108
107
|
}
|
|
109
108
|
const markerId = await createComment(
|
|
@@ -115,16 +114,17 @@ export const acknowledgeQueuedJob = async (
|
|
|
115
114
|
expected,
|
|
116
115
|
)}`,
|
|
117
116
|
);
|
|
118
|
-
const
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
].sort((left, right) => left - right);
|
|
117
|
+
const markerIds = await matchingTrustedQueueCommentIds(
|
|
118
|
+
deps,
|
|
119
|
+
issueNumber,
|
|
120
|
+
expected.approvalId,
|
|
121
|
+
kind,
|
|
122
|
+
);
|
|
123
|
+
const winner = await retainEarliestComment(deps, markerIds);
|
|
126
124
|
if (winner !== markerId) {
|
|
127
|
-
|
|
125
|
+
if (!markerIds.includes(markerId)) {
|
|
126
|
+
await deleteComment(deps.token, deps.repo, markerId);
|
|
127
|
+
}
|
|
128
128
|
return false;
|
|
129
129
|
}
|
|
130
130
|
if (!(await isStillQueueable(deps, issueNumber, approval, kind))) {
|
|
@@ -133,3 +133,18 @@ export const acknowledgeQueuedJob = async (
|
|
|
133
133
|
}
|
|
134
134
|
return true;
|
|
135
135
|
};
|
|
136
|
+
|
|
137
|
+
export const reconcileExistingQueuedJob = async (
|
|
138
|
+
deps: JobDeps,
|
|
139
|
+
issueNumber: number,
|
|
140
|
+
approval: ApprovalBinding,
|
|
141
|
+
kind: PollerJobKind,
|
|
142
|
+
): Promise<void> => {
|
|
143
|
+
const markerIds = await matchingTrustedQueueCommentIds(
|
|
144
|
+
deps,
|
|
145
|
+
issueNumber,
|
|
146
|
+
approval.id,
|
|
147
|
+
kind,
|
|
148
|
+
);
|
|
149
|
+
await retainEarliestComment(deps, markerIds);
|
|
150
|
+
};
|