pi-superagent 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ADAPTATION.md +29 -31
- package/README.md +69 -122
- package/docs/architecture.md +103 -124
- package/docs/design-decisions.md +128 -228
- package/docs/formal-verification.md +48 -52
- package/docs/operator-guide.md +101 -89
- package/formal/README.md +20 -20
- package/package.json +1 -1
package/docs/design-decisions.md
CHANGED
|
@@ -1,230 +1,130 @@
|
|
|
1
1
|
# Design decisions
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
##
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
the
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
it
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
the
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
##
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
exception: that pull request's merge or close ends it, as merged or as closed, not as a stop.
|
|
132
|
-
|
|
133
|
-
Closing the issue is the one stop that needs no tool and no knowledge of the system. Tickets on threads
|
|
134
|
-
from other sources are left alone: the router can only tell whether a GitHub thread is open.
|
|
135
|
-
|
|
136
|
-
## Work that changes a repository ends with its pull request
|
|
137
|
-
|
|
138
|
-
A worker whose work changes a repository opens a pull request and names it in `ticket_report`. The router
|
|
139
|
-
binds the pull request to the ticket before the engine reads the report, so *ready* then means awaiting
|
|
140
|
-
review rather than done. Comments and reviews on the pull request go to the worker. The merge ends the
|
|
141
|
-
ticket as merged, a close ends it as closed, and the router says which on the issue. A report that names
|
|
142
|
-
no pull request ends the ticket as before.
|
|
143
|
-
|
|
144
|
-
The work decides, not the request. A deployment, an answer or an investigation changes no repository, and a
|
|
145
|
-
ticket made to wait for a pull request would never end; the worker knows by the end of its run whether it
|
|
146
|
-
opened one. The worker opens it, not the router, because the worker knows which repository the work
|
|
147
|
-
belongs in: the watched repository is often only where people ask. Only a pull request URL counts, since
|
|
148
|
-
`#5` may be an issue, and a ticket bound to an issue would wait for a merge that never comes. The binding
|
|
149
|
-
comes first because the engine completes a ticket on a *ready* report unless it already follows an open
|
|
150
|
-
pull request. The lifecycle itself is the vendored engine's (`bindThread`, `threadClosed`, and upstream's
|
|
151
|
-
`docs/003-execution.md`); the router feeds it facts.
|
|
152
|
-
|
|
153
|
-
The watched repository's pull requests cost nothing extra: their comments and reviews arrive with every
|
|
154
|
-
poll. One in another repository is read every two minutes, three calls each time, so a dozen open reviews
|
|
155
|
-
stay well inside GitHub's hourly limit.
|
|
156
|
-
|
|
157
|
-
## A review is a request; opening a pull request is not
|
|
158
|
-
|
|
159
|
-
A submitted review, whether it requests changes or approves, is a person speaking about delivered work,
|
|
160
|
-
on a conversation the ticket already owns. It fires a fresh judgement, and the reply lands in the pull
|
|
161
|
-
request's own conversation (on GitHub a pull request is also issue N, so no second address is needed).
|
|
162
|
-
|
|
163
|
-
Opening a pull request is a delivery, not a request. Treating every opening as a trigger would make the
|
|
164
|
-
agent a mandatory reviewer of its own output, a loop that feeds itself.
|
|
165
|
-
|
|
166
|
-
## Only work holds a slot
|
|
167
|
-
|
|
168
|
-
The cap counts workers on a run. A worker between runs, waiting on a person's answer or simply done, is
|
|
169
|
-
parked: it keeps its session and its context, and its slot goes to the next ticket in the queue. A reply
|
|
170
|
-
puts it back on a run, even past the cap for a while.
|
|
171
|
-
|
|
172
|
-
A slot is the machine's capacity, and an idle session uses almost none of it. When every live worker held
|
|
173
|
-
a slot, a queue of 22 tickets waited while one of the four slots belonged to a worker that had finished its
|
|
174
|
-
run without saying so.
|
|
175
|
-
|
|
176
|
-
## A tab is closed only when nobody is in it
|
|
177
|
-
|
|
178
|
-
A finished worker's tab stays open for ten minutes of follow-up and then closes only if its pi sits idle;
|
|
179
|
-
while the agent is busy, the router looks again every ten minutes instead of giving up.
|
|
180
|
-
The startup sweep closes a worker tab only when no record owns it or herdr no longer knows its agent.
|
|
181
|
-
|
|
182
|
-
Worker tabs do outlive their purpose: a restart adopts a worker, a stop starts a newer one, and the older
|
|
183
|
-
tab stays behind, so a sweep has to exist. But herdr has no guarded close, and a tab is someone's
|
|
184
|
-
terminal. The liveness check is the single guard that keeps the sweep away from an agent that is still
|
|
185
|
-
working. (`Workers.tla` W2, W4, W5.)
|
|
186
|
-
|
|
187
|
-
## One router, one store, one state directory
|
|
188
|
-
|
|
189
|
-
One router runs at a time, and one process holds the engine. The store is synchronous SQLite, and
|
|
190
|
-
everything the system knows lives in one directory.
|
|
191
|
-
|
|
192
|
-
The engine's commands are atomic only if the store is synchronous: an asynchronous driver would turn that
|
|
193
|
-
atomicity into a comment. One directory makes the crash model concrete. A restart is "start again with
|
|
194
|
-
this directory", which is exactly what the specifications model. One router per process matters too: the
|
|
195
|
-
judge and every worker load this same extension, and each of them used to boot a router of its own.
|
|
196
|
-
|
|
197
|
-
## Health is an exit code
|
|
198
|
-
|
|
199
|
-
`superagent status` answers with a number: 0 healthy, 1 down, 2 a part is failing, 3 waiting for setup.
|
|
200
|
-
A router that fails three polls in a row reports its GitHub source as failing.
|
|
201
|
-
|
|
202
|
-
A scheduled check should act without reading prose, and each code has one remedy: a restart may fix 1
|
|
203
|
-
and fixes neither 2 nor 3. A router that cannot reach GitHub otherwise looks healthy forever, answering
|
|
204
|
-
nothing.
|
|
205
|
-
|
|
206
|
-
## The panel is local and cannot be steered from a web page
|
|
207
|
-
|
|
208
|
-
The panel binds 127.0.0.1, refuses any Host header but 127.0.0.1 and localhost, and serves its API only
|
|
209
|
-
to callers that send `x-superagent-request: 1`.
|
|
210
|
-
|
|
211
|
-
A browser will not let another site forge either one. Without them, any page you visit could read the
|
|
212
|
-
queue, or pause the engine, by rebinding a hostname to localhost.
|
|
213
|
-
|
|
214
|
-
## The engine stays byte-identical to upstream
|
|
215
|
-
|
|
216
|
-
`src/core/` and its test suite are vendored from upstream and never edited. Every adaptation, from the
|
|
217
|
-
judge's prompt to the GitHub shapes, lives in `src/pi/`.
|
|
218
|
-
|
|
219
|
-
The engine is the part the lifecycle specification models and upstream maintains. Untouched, an upstream
|
|
220
|
-
fix can be re-vendored by copying, and the specification keeps describing the code that runs. The
|
|
221
|
-
[adaptation ledger](../ADAPTATION.md) says how to prove the copy is exact.
|
|
222
|
-
|
|
223
|
-
## The specifications live beside the code
|
|
224
|
-
|
|
225
|
-
`formal/` holds three TLA+ specifications, and `scripts/check-formal` checks all of them exhaustively in
|
|
226
|
-
about five minutes. Every action names the function it mirrors; every header states what the model
|
|
227
|
-
leaves out.
|
|
228
|
-
|
|
229
|
-
A proof that cannot be re-run in one command rots. These are cheap to keep honest, and they have paid:
|
|
230
|
-
the intake specification found a request-losing bug in shipped code.
|
|
3
|
+
## Why work starts on GitHub
|
|
4
|
+
|
|
5
|
+
A reviewer who spots a problem in a pull request already has the diff and a place to leave a
|
|
6
|
+
comment. Asking them to copy that comment onto a separate task board would split the conversation
|
|
7
|
+
just when the worker needs its context. Issues, comments and reviews therefore enter through the
|
|
8
|
+
same source, `source-github.ts`. A new request can open a ticket, and feedback can reach the
|
|
9
|
+
worker that made the change.
|
|
10
|
+
|
|
11
|
+
**Check the author before routing the message.** An accepted request starts a worker with a shell
|
|
12
|
+
on the operator’s machine. A comment on an existing ticket goes straight to that worker, so a
|
|
13
|
+
permission check in the judge alone would miss follow-ups. The engine checks `scopeOf` first.
|
|
14
|
+
By default it admits the GitHub associations `OWNER`, `MEMBER` and `COLLABORATOR`. A missing
|
|
15
|
+
association fails the check. Setting `PI_SUPERAGENT_ALLOWED` replaces that rule with a login list.
|
|
16
|
+
|
|
17
|
+
**Establish a baseline on the first poll.** An early version treated an entire backlog as new work
|
|
18
|
+
when the operator connected a repository. The source now saves the initial set of open issues and
|
|
19
|
+
seeds the review cursor. Closing and reopening an issue makes it eligible again. Its updated
|
|
20
|
+
timestamp distinguishes the new trigger from the earlier one. See `Intake.tla` C1 and C2.
|
|
21
|
+
|
|
22
|
+
**Save later read positions after delivery.** A poll stages its read position for the next poll to
|
|
23
|
+
commit. If the router crashes after reading a comment but before the engine receives it, the old
|
|
24
|
+
position lets the router read it again. Stable trigger IDs make that replay safe. The initial
|
|
25
|
+
baseline commits immediately because it represents history already present at setup. TLC found
|
|
26
|
+
the comment-loss case in the original implementation. The regression test reproduces that event
|
|
27
|
+
order. See `Intake.tla` C3 and C6.
|
|
28
|
+
|
|
29
|
+
## Deciding what deserves a ticket
|
|
30
|
+
|
|
31
|
+
**Apply policy in code.** The router checks the author against current bot patterns even when a
|
|
32
|
+
stored trigger identifies the author as human. An older trigger once caused an agent’s request
|
|
33
|
+
for test approval to become a ticket to run the test. After the judge answers, the router also
|
|
34
|
+
checks whether an open ticket already owns the thread. The judge once accepted a duplicate despite
|
|
35
|
+
having the existing ticket in its prompt.
|
|
36
|
+
|
|
37
|
+
**Give the judge one request at a time.** Judging in the operator’s pane tied it up for an entire
|
|
38
|
+
model turn. Sending 41 candidates into one active turn produced answers out of order, and some
|
|
39
|
+
candidates expired before the judge got to them. The assessor now reuses one headless pi session
|
|
40
|
+
without extensions, skills or context files. It opens the session once because startup took 20 to
|
|
41
|
+
45 seconds and concurrent requests used to start separate judges.
|
|
42
|
+
|
|
43
|
+
## Running a worker
|
|
44
|
+
|
|
45
|
+
A worker can write “done” in its terminal while the person who opened the issue is still waiting
|
|
46
|
+
on GitHub. To finish a run, it calls `ticket_report` with an outcome. A *ready* report can carry
|
|
47
|
+
the link to a PR. A *blocked* report carries the question the requester needs to answer. The other
|
|
48
|
+
outcomes are *failed* and *cancelled*. Only the engine writes the result to the ticket.
|
|
49
|
+
See `Ticket.tla` I2.
|
|
50
|
+
|
|
51
|
+
If a run ends without a report, the router posts a failure on the thread. Launch errors and crashes
|
|
52
|
+
also get a reply there, so the person who asked knows what happened. The outbox recognizes repeat
|
|
53
|
+
failures. One broken setup command deserves one notice, even if the router tries it on every poll.
|
|
54
|
+
A quiet worker is a different case. After 15 minutes of silence it appears on the panel, where the
|
|
55
|
+
operator can see whether it needs help or is simply waiting for a build.
|
|
56
|
+
|
|
57
|
+
**Keep tool use within the assignment.** Worker tool hooks check file edits and writes against the
|
|
58
|
+
ticket workspace, block configured commands and disable delegation by default. The prepare hook
|
|
59
|
+
also gets a PATH with shims for blocked commands. Operators can use these controls to restrict
|
|
60
|
+
personal tools such as `gh`. The brief asks workers sharing the machine to run builds sequentially,
|
|
61
|
+
bound parallel jobs and clean up processes when they finish.
|
|
62
|
+
|
|
63
|
+
**Replace a crashed worker once.** The engine records the failed attempt, frees its slot and
|
|
64
|
+
requeues unread comments. The replacement gets the original workspace and a brief explaining the
|
|
65
|
+
crash. A second crash on the same ticket requires attention. The limit applies per router process
|
|
66
|
+
and prevents a persistent fault from spawning a new tab every few seconds.
|
|
67
|
+
|
|
68
|
+
**Count active work against the cap.** A parked worker retains its session for follow-ups and frees
|
|
69
|
+
a slot for queued work. A reply can briefly take the active count above the cap. Counting every
|
|
70
|
+
open session used to leave a queue of 22 tickets waiting while an idle worker occupied one of four
|
|
71
|
+
slots.
|
|
72
|
+
|
|
73
|
+
**Check the tab before closing it.** A finished worker gets ten minutes for follow-ups. The router
|
|
74
|
+
closes its tab once pi is idle and checks again every ten minutes while it is busy. Startup cleanup
|
|
75
|
+
handles tabs with no owning record or whose agent herdr no longer knows. Since herdr has no guarded
|
|
76
|
+
close operation, the liveness check matters whenever someone could still be working in a tab.
|
|
77
|
+
See `Workers.tla` W2, W4 and W5.
|
|
78
|
+
|
|
79
|
+
## Ending a ticket
|
|
80
|
+
|
|
81
|
+
**Respect an explicit stop.** Once a person stops a ticket, later messages, wake-ups and retries
|
|
82
|
+
cannot resume it. Asking again creates a new ticket with its own history. See `Ticket.tla` I1.
|
|
83
|
+
|
|
84
|
+
**Follow the GitHub conversation.** When an issue disappears from the open set, the router can
|
|
85
|
+
observe the closure even if it cannot distinguish deletion, manual closure and closure by merge.
|
|
86
|
+
It stops the ticket and queues a notice. For a ticket awaiting its own PR, the PR’s merge or close
|
|
87
|
+
determines the outcome. Reopening an issue lets its author ask again.
|
|
88
|
+
|
|
89
|
+
**Bind the PR before processing a ready report.** A worker that changes a repository opens a PR and
|
|
90
|
+
includes its URL in its report. A number such as `#5` could also name an issue. Binding the URL
|
|
91
|
+
first lets the engine hold the ticket open for review. Merging or closing the PR ends the ticket,
|
|
92
|
+
and the issue receives the outcome. An investigation or deployment can finish with a ready report
|
|
93
|
+
alone. The worker knows which repository it changed, even when the request originated elsewhere.
|
|
94
|
+
|
|
95
|
+
The router calls the vendored engine’s `bindThread` and `threadClosed` transitions. It gets review
|
|
96
|
+
activity on the watched repository during the regular poll. For a PR elsewhere it polls every two
|
|
97
|
+
minutes, using three API calls per check.
|
|
98
|
+
|
|
99
|
+
**Route review feedback to the worker.** A submitted review on a bound PR goes to its ticket.
|
|
100
|
+
A review on an unrelated PR can reach the judge as a new request. GitHub accepts replies in the
|
|
101
|
+
PR conversation through its issue number. Opening a PR alone produces no request, so a worker can
|
|
102
|
+
deliver a change without triggering another worker to review it.
|
|
103
|
+
|
|
104
|
+
## Operating the router
|
|
105
|
+
|
|
106
|
+
**Keep ticket state in one store.** One router owns the synchronous SQLite store in its state
|
|
107
|
+
directory. All writes from an engine command commit together. Recovery reopens the same directory.
|
|
108
|
+
The router role is tied to the appointed process because pi also loads the extension in worker
|
|
109
|
+
and judge sessions. Previously those sessions could each start a router.
|
|
110
|
+
|
|
111
|
+
**Report health through exit codes.** A scheduled check can act on `superagent status`. Code 0
|
|
112
|
+
means healthy or intentionally stopped. Code 1 indicates a router failure that a restart may fix.
|
|
113
|
+
Code 2 directs the operator to a failing component, including a GitHub source that failed three
|
|
114
|
+
polls in a row. Code 3 identifies a setup problem.
|
|
115
|
+
|
|
116
|
+
**Limit panel access to local callers.** The panel binds to loopback, validates the Host header
|
|
117
|
+
and requires `x-superagent-request: 1` for API calls. These checks prevent a page on another
|
|
118
|
+
site from reading the queue or pausing the router through DNS rebinding.
|
|
119
|
+
|
|
120
|
+
## Maintaining the engine
|
|
121
|
+
|
|
122
|
+
The engine and its test suite match the pinned upstream revision. Local adapters live in
|
|
123
|
+
`src/pi/`, so an upstream fix can be imported and compared directly with its source.
|
|
124
|
+
[Adaptation ledger](../ADAPTATION.md)
|
|
125
|
+
|
|
126
|
+
The three TLA+ models live beside the implementation. Their actions name the functions they mirror
|
|
127
|
+
and their headers record the assumptions. When a transition changes, update its model and run
|
|
128
|
+
`scripts/check-formal`. The intake model’s crash counterexample is one case where checking all
|
|
129
|
+
orderings within a small bound found a failure the tests had missed.
|
|
130
|
+
[Architecture](architecture.md) · [Formal verification](formal-verification.md)
|
|
@@ -1,7 +1,16 @@
|
|
|
1
1
|
# Formal verification
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
|
|
3
|
+
When is it safe to mark a GitHub comment as read? An early router saved its place as soon as it
|
|
4
|
+
read the comment. If it crashed before passing the message to the ticket engine, it restarted one
|
|
5
|
+
comment too far ahead. From the requester’s point of view, the worker had simply ignored them.
|
|
6
|
+
|
|
7
|
+
TLC found the precise sequence that lost the comment. It works by trying every ordering of events
|
|
8
|
+
allowed by a small model written in TLA+. That sequence became a regression test, and the router
|
|
9
|
+
now waits until the engine has received the comment before saving its place.
|
|
10
|
+
|
|
11
|
+
The project checks three such models. The ticket model asks who owns each conversation and how
|
|
12
|
+
work can end. The intake model follows messages across polls and crashes. The worker model follows
|
|
13
|
+
terminal tabs as agents start, finish and leave tabs behind.
|
|
5
14
|
|
|
6
15
|
| Specification | Models | States explored (distinct) | Claims |
|
|
7
16
|
| --- | --- | --- | --- |
|
|
@@ -9,12 +18,9 @@ specified in TLA+ and checked exhaustively by TLC, the TLA+ model checker:
|
|
|
9
18
|
| [`Intake.tla`](../formal/Intake.tla) | the GitHub source (`source-github.ts`) | 565,551 (113,281) | 6 |
|
|
10
19
|
| [`Workers.tla`](../formal/Workers.tla) | worker records, tabs and agents (`herdr-workers.ts`) | 347 (94) | 5 |
|
|
11
20
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
shipped with that interleaving as a regression test. Along the way TLC also caught two mistakes in the
|
|
16
|
-
models themselves (a dropped evaluation batch, and a stopped ticket brought back to life), which is
|
|
17
|
-
exactly what a checker is for.
|
|
21
|
+
The counts include repeat visits to a state. The numbers in parentheses count each distinct
|
|
22
|
+
state once. All listed properties hold within the bounds below. TLC also caught two errors in the
|
|
23
|
+
models themselves, one that dropped a batch of requests and another that revived a stopped ticket.
|
|
18
24
|
|
|
19
25
|
## Running the checks
|
|
20
26
|
|
|
@@ -23,17 +29,16 @@ scripts/check-formal # all three specifications, about five minutes
|
|
|
23
29
|
```
|
|
24
30
|
|
|
25
31
|
The script needs a JDK (`brew install openjdk`, or point `JAVA` at one) and downloads `tla2tools.jar` to
|
|
26
|
-
`~/.cache/tla2tools/` on first use. It exits 0 only when every specification is checked to completion.
|
|
27
|
-
|
|
28
|
-
|
|
32
|
+
`~/.cache/tla2tools/` on first use. It exits 0 only when every specification is checked to completion.
|
|
33
|
+
It exits 1 and prints the reason if Java is missing, a specification fails to parse, TLC finds a
|
|
34
|
+
violation or a run ends early.
|
|
29
35
|
|
|
30
36
|
## What each specification proves
|
|
31
37
|
|
|
32
38
|
### `Ticket.tla`: the ticket lifecycle
|
|
33
39
|
|
|
34
40
|
- **I1 StopAbsorbing.** A ticket a person stopped is never live again.
|
|
35
|
-
- **I2 DoneFromReady.**
|
|
36
|
-
unreachable, not merely untested.
|
|
41
|
+
- **I2 DoneFromReady.** Every completed ticket has a recorded *ready* report.
|
|
37
42
|
- **I3 OneRunning.** A ticket has at most one running assignment.
|
|
38
43
|
- **I4 NoSecondTicket.** No ticket is created for a thread that a live ticket already follows, in any
|
|
39
44
|
interleaving, including a follow-up that arrives in the middle of an evaluation.
|
|
@@ -65,42 +70,33 @@ that stays silent is treated as a failed one.
|
|
|
65
70
|
- **W5 ZombieTabsGone.** After any finite number of launches, every dead agent's tab is eventually closed:
|
|
66
71
|
the workspace cannot fill up with abandoned tabs.
|
|
67
72
|
|
|
68
|
-
##
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
A model proves things about the model. These cover the control flow described above, not the judge's
|
|
100
|
-
decisions, GitHub's behavior (status codes, rate limits, pagination), SQLite, herdr, or the network.
|
|
101
|
-
Newer behavior is tested but not modeled: reviews, the stop on a closed conversation, the outbox's check
|
|
102
|
-
before a retry, and how each way a worker's run ends reaches the thread.
|
|
103
|
-
|
|
104
|
-
What the specifications do guarantee: no implementation that follows these transitions can reach a
|
|
105
|
-
forbidden state, and the liveness claims hold under the stated fairness. When the code a specification
|
|
106
|
-
mirrors changes, upstream's ticket transitions included, the specification must change with it.
|
|
73
|
+
## Interpreting the results
|
|
74
|
+
|
|
75
|
+
Each action names the function whose guards it mirrors. A change to that function may change which
|
|
76
|
+
states the implementation can reach. Update the model and rerun the check when a transition changes.
|
|
77
|
+
For example, I1 depends on every path that might resume a ticket checking `ended.by === 'human'`.
|
|
78
|
+
|
|
79
|
+
The ticket model has two threads, two ticket slots and four messages. The intake model has two
|
|
80
|
+
issues, two comments and one crash. The worker model has two workers and three launches. These
|
|
81
|
+
bounds let TLC explore every ordering in minutes. Each model header records its exclusions.
|
|
82
|
+
|
|
83
|
+
The eventual-delivery and cleanup properties (LT_Delivered, C6 and W5) also depend on fairness.
|
|
84
|
+
An action that stays enabled must eventually run. The bounds on crashes and launches give the
|
|
85
|
+
system a chance to finish delivery or cleanup.
|
|
86
|
+
|
|
87
|
+
## Maintaining the models
|
|
88
|
+
|
|
89
|
+
- Put temporal formulas such as `[][A]_vars` under `PROPERTY`. Use `INVARIANT` for state predicates.
|
|
90
|
+
- Pass a tuple such as `<<x, y>>` to `UNCHANGED`. If an action assigns `x'`, omit `x` from that
|
|
91
|
+
tuple unless the assignment really must preserve its value. Use `-coverage 1` to spot disabled actions.
|
|
92
|
+
- Give a model with a valid terminal state an explicit stuttering step (`Terminate == UNCHANGED vars`).
|
|
93
|
+
This lets TLC distinguish normal termination from a deadlock. Liveness checks still detect stuck work.
|
|
94
|
+
- The intake model uses the state name `opened` because `open` caused a parse error.
|
|
95
|
+
|
|
96
|
+
## Scope
|
|
97
|
+
|
|
98
|
+
The models describe control flow. The judge’s decisions, GitHub API behavior, SQLite, herdr and network
|
|
99
|
+
failures have their own behavior outside these bounds. Integration tests cover review routing,
|
|
100
|
+
closing a conversation, checking for a previous post before retrying, and notifying the thread
|
|
101
|
+
when a worker’s run ends. Consult the test suite for those cases and the model headers for the
|
|
102
|
+
assumptions behind each property.
|