pi-superagent 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,230 +1,130 @@
1
1
  # Design decisions
2
2
 
3
- Each section states a rule the system keeps and the failure that taught it. Where each rule lives is
4
- mapped in [architecture.md](architecture.md); the proofs are in [formal-verification.md](formal-verification.md).
5
-
6
- ## GitHub is the only way in
7
-
8
- Every request arrives as an issue, a comment or a review on one repository. There is no command-line
9
- submission, no board and no label vocabulary.
10
-
11
- Each extra door grows its own answers to "is this a person?" and "is this addressed to me?", and the
12
- agent's behavior then depends on which door a request happened to use. With one door, the identity rules
13
- live in one file (`source-github.ts`), and who is heard and who may start work in two functions
14
- (`scopeOf` and `policyOf` in `router.ts`).
15
-
16
- ## Only people with write access are heard
17
-
18
- The router hears an author only when GitHub says they can write to the repository: the OWNER, MEMBER and
19
- COLLABORATOR author associations. Anyone else's issue, comment or review is stored as filtered and goes no
20
- further: no judgement, no ticket, no input to a running worker. `PI_SUPERAGENT_ALLOWED` replaces the rule
21
- with a list of logins.
22
-
23
- On a public repository anyone can open an issue, and an accepted issue starts a worker with a shell on
24
- the operator's machine. The judge cannot be that boundary, since a stranger's request reads like the
25
- owner's. The admission gate cannot either: a comment on an issue that already has a ticket goes straight
26
- to its worker and is never admitted again. So the check sits at the source scope, the engine's first step,
27
- where it covers every way in. An author whose association is missing counts as having none.
28
-
29
- ## The first look is silent
30
-
31
- The first time the source sees a repository, it records every open issue and fires none of them. Only
32
- issues opened afterward are requests.
33
-
34
- The first version, pointed at a repository with a backlog, fired every open issue at once. A backlog is
35
- history, not a queue of fresh requests. Closing and reopening an issue, though, is a person saying "go
36
- again": a closed issue leaves the seen set and fires when it returns, under a new trigger id, because the
37
- id includes the issue's last update. The review cursor seeds the same way. (`Intake.tla` C1, C2.)
38
-
39
- ## Read positions commit late; seeds commit at once
40
-
41
- A poll stages its read positions, and they commit at the start of the next poll, once the engine holds
42
- every trigger they produced.
43
-
44
- Committing at the moment of reading leaves a window: if the process dies between reading a comment and
45
- handing it to the engine, the saved position already points past a request the engine never saw, and
46
- that request is gone. A position that commits late can only cause a replay, and replays are free because
47
- the engine accepts each trigger id once. A seed is the mirror image: losing one would replay the backlog,
48
- and a seed can never swallow a request, so it commits immediately. The model checker found the early
49
- commit in shipped code; the fix ships with the counterexample as a regression test. (`Intake.tla` C3, C6.)
50
-
51
- ## Code admits; the model judges
52
-
53
- A policy gate in code admits a trigger only from a person who is not on the bot list. Only then does
54
- the judge see it. When the judge accepts a thread that already has an open ticket, the router overrules
55
- it and records an ignore.
56
-
57
- A model's verdicts wobble; the deterministic layer keeps a wobble from turning bot traffic, the agent's
58
- own comments or a duplicate into work. Both checks were learned. A trigger stored before bot names were
59
- matched as patterns still said "human", and an agent asking the operator to approve a test became a ticket
60
- to run it; the gate now checks the author again. And the judge once opened a second ticket on a thread
61
- whose first ticket was right there in its prompt.
62
-
63
- ## The judge works alone, one question at a time
64
-
65
- Judgements go to a headless pi session that loads no extensions, skills or context files, never to the
66
- operator's own pane, and the assessor hands it one candidate per turn.
67
-
68
- Judging in the operator's pane made the pane look frozen for every model turn. Sending candidates in a
69
- burst was worse: 41 of them steered into one running turn came back out of order, and the tail expired
70
- unanswered. The judge is also opened once and shared: opening a session takes 20 to 45 seconds, and
71
- before that rule every eval arriving in the window opened its own.
72
-
73
- ## Workers report through a tool, never in prose
74
-
75
- A worker ends its assignment by calling `ticket_report` with a state: ready, blocked, failed or
76
- cancelled. Only *ready* completes a ticket.
77
-
78
- A model that says "done" in prose while having done nothing must be unable to complete anything. In the
79
- lifecycle model, completing without a *ready* report is not merely untested but unreachable
80
- (`Ticket.tla` I2). A run that ends without a report completes nothing either: the worker files it as a
81
- failure, and the person hears about it on the thread.
82
-
83
- ## Every ending reaches the thread
84
-
85
- Whatever ends a worker's run is posted on the issue: its report, its question, a crash, a run that ended
86
- without a report, a launch that never got going. Each message says what happened and what the person can
87
- do next. The one exception is the 15-minute quiet flag, which stays on the panel: a long build is not news.
88
-
89
- The person who asked is on GitHub, not on the panel. An ending they cannot see is a request that died in
90
- silence, and a question they cannot see is a worker waiting forever. The same ending posts once per
91
- assignment, so a launch that fails on every poll says so once, not every 30 seconds.
92
-
93
- ## A worker is fenced in
94
-
95
- Inside a worker, a guard refuses delegation tools, refuses blocked commands, and refuses writes outside
96
- the ticket's workspace. The workspace's setup script, which runs before pi does, meets the blocked
97
- commands as refusing shims first on its PATH. The brief also tells the worker that the machine is shared.
98
-
99
- Delegated work leaves the mandate, the workspace and the report behind. Blocked commands are the
100
- operator's personal tools (`gh`, for instance), and the operator's identity is not the worker's to use.
101
- The shared-machine brief exists because workers run side by side with each other and with people:
102
- sequential builds, bounded parallel jobs, and no process left running after the report.
103
-
104
- ## A crash gets one fresh worker, not a loop
105
-
106
- When a worker crashes, the router retries the ticket once on its own, through the engine's retry. The
107
- crashed attempt ends on record, the comments it never read go to the new worker, and the new worker's brief
108
- says what happened, because the old one's changes are still in the workspace. A second crash on the same
109
- ticket waits for a person. Either way, the crashed worker's slot is freed at once.
110
-
111
- A lost tab or a killed process is usually bad luck, and one retry rides it out without waking anyone. A
112
- worker that crashes every time is a bug, and looping on it would open a new tab every few seconds while
113
- hiding the problem. One retry per ticket per router process is the same bound that startup recovery uses,
114
- and the lifecycle model already covers it: `Ticket.tla` lets a retry happen at any moment after a failed
115
- report.
116
-
117
- ## A person's stop is final
118
-
119
- Once a person stops a ticket, nothing revives it: no new message, no wake-up, no retry. A request made
120
- again becomes a new ticket.
121
-
122
- A stop is a person saying "not now". An agent that resumes what it was told to stop teaches everyone that
123
- their words do not count. (`Ticket.tla` I1.)
124
-
125
- ## A closed conversation stops its ticket
126
-
127
- GitHub ends a conversation in three ways: a merge closes the issue it names, a close says no, and a
128
- deletion says never mind. All three look the same from a poll, because the thread leaves the set of open
129
- issues and pull requests. The router then stops the ticket with a person's stop, posts one notice on the
130
- thread, and stays quiet until the issue is reopened. A ticket that follows its own pull request is the
131
- exception: that pull request's merge or close ends it, as merged or as closed, not as a stop.
132
-
133
- Closing the issue is the one stop that needs no tool and no knowledge of the system. Tickets on threads
134
- from other sources are left alone: the router can only tell whether a GitHub thread is open.
135
-
136
- ## Work that changes a repository ends with its pull request
137
-
138
- A worker whose work changes a repository opens a pull request and names it in `ticket_report`. The router
139
- binds the pull request to the ticket before the engine reads the report, so *ready* then means awaiting
140
- review rather than done. Comments and reviews on the pull request go to the worker. The merge ends the
141
- ticket as merged, a close ends it as closed, and the router says which on the issue. A report that names
142
- no pull request ends the ticket as before.
143
-
144
- The work decides, not the request. A deployment, an answer or an investigation changes no repository, and a
145
- ticket made to wait for a pull request would never end; the worker knows by the end of its run whether it
146
- opened one. The worker opens it, not the router, because the worker knows which repository the work
147
- belongs in: the watched repository is often only where people ask. Only a pull request URL counts, since
148
- `#5` may be an issue, and a ticket bound to an issue would wait for a merge that never comes. The binding
149
- comes first because the engine completes a ticket on a *ready* report unless it already follows an open
150
- pull request. The lifecycle itself is the vendored engine's (`bindThread`, `threadClosed`, and upstream's
151
- `docs/003-execution.md`); the router feeds it facts.
152
-
153
- The watched repository's pull requests cost nothing extra: their comments and reviews arrive with every
154
- poll. One in another repository is read every two minutes, three calls each time, so a dozen open reviews
155
- stay well inside GitHub's hourly limit.
156
-
157
- ## A review is a request; opening a pull request is not
158
-
159
- A submitted review, whether it requests changes or approves, is a person speaking about delivered work,
160
- on a conversation the ticket already owns. It fires a fresh judgement, and the reply lands in the pull
161
- request's own conversation (on GitHub a pull request is also issue N, so no second address is needed).
162
-
163
- Opening a pull request is a delivery, not a request. Treating every opening as a trigger would make the
164
- agent a mandatory reviewer of its own output, a loop that feeds itself.
165
-
166
- ## Only work holds a slot
167
-
168
- The cap counts workers on a run. A worker between runs, waiting on a person's answer or simply done, is
169
- parked: it keeps its session and its context, and its slot goes to the next ticket in the queue. A reply
170
- puts it back on a run, even past the cap for a while.
171
-
172
- A slot is the machine's capacity, and an idle session uses almost none of it. When every live worker held
173
- a slot, a queue of 22 tickets waited while one of the four slots belonged to a worker that had finished its
174
- run without saying so.
175
-
176
- ## A tab is closed only when nobody is in it
177
-
178
- A finished worker's tab stays open for ten minutes of follow-up and then closes only if its pi sits idle;
179
- while the agent is busy, the router looks again every ten minutes instead of giving up.
180
- The startup sweep closes a worker tab only when no record owns it or herdr no longer knows its agent.
181
-
182
- Worker tabs do outlive their purpose: a restart adopts a worker, a stop starts a newer one, and the older
183
- tab stays behind, so a sweep has to exist. But herdr has no guarded close, and a tab is someone's
184
- terminal. The liveness check is the single guard that keeps the sweep away from an agent that is still
185
- working. (`Workers.tla` W2, W4, W5.)
186
-
187
- ## One router, one store, one state directory
188
-
189
- One router runs at a time, and one process holds the engine. The store is synchronous SQLite, and
190
- everything the system knows lives in one directory.
191
-
192
- The engine's commands are atomic only if the store is synchronous: an asynchronous driver would turn that
193
- atomicity into a comment. One directory makes the crash model concrete. A restart is "start again with
194
- this directory", which is exactly what the specifications model. One router per process matters too: the
195
- judge and every worker load this same extension, and each of them used to boot a router of its own.
196
-
197
- ## Health is an exit code
198
-
199
- `superagent status` answers with a number: 0 healthy, 1 down, 2 a part is failing, 3 waiting for setup.
200
- A router that fails three polls in a row reports its GitHub source as failing.
201
-
202
- A scheduled check should act without reading prose, and each code has one remedy: a restart may fix 1
203
- and fixes neither 2 nor 3. A router that cannot reach GitHub otherwise looks healthy forever, answering
204
- nothing.
205
-
206
- ## The panel is local and cannot be steered from a web page
207
-
208
- The panel binds 127.0.0.1, refuses any Host header but 127.0.0.1 and localhost, and serves its API only
209
- to callers that send `x-superagent-request: 1`.
210
-
211
- A browser will not let another site forge either one. Without them, any page you visit could read the
212
- queue, or pause the engine, by rebinding a hostname to localhost.
213
-
214
- ## The engine stays byte-identical to upstream
215
-
216
- `src/core/` and its test suite are vendored from upstream and never edited. Every adaptation, from the
217
- judge's prompt to the GitHub shapes, lives in `src/pi/`.
218
-
219
- The engine is the part the lifecycle specification models and upstream maintains. Untouched, an upstream
220
- fix can be re-vendored by copying, and the specification keeps describing the code that runs. The
221
- [adaptation ledger](../ADAPTATION.md) says how to prove the copy is exact.
222
-
223
- ## The specifications live beside the code
224
-
225
- `formal/` holds three TLA+ specifications, and `scripts/check-formal` checks all of them exhaustively in
226
- about five minutes. Every action names the function it mirrors; every header states what the model
227
- leaves out.
228
-
229
- A proof that cannot be re-run in one command rots. These are cheap to keep honest, and they have paid:
230
- the intake specification found a request-losing bug in shipped code.
3
+ ## Why work starts on GitHub
4
+
5
+ A reviewer who spots a problem in a pull request already has the diff and a place to leave a
6
+ comment. Asking them to copy that comment onto a separate task board would split the conversation
7
+ just when the worker needs its context. Issues, comments and reviews therefore enter through the
8
+ same source, `source-github.ts`. A new request can open a ticket, and feedback can reach the
9
+ worker that made the change.
10
+
11
+ **Check the author before routing the message.** An accepted request starts a worker with a shell
12
+ on the operator’s machine. A comment on an existing ticket goes straight to that worker, so a
13
+ permission check in the judge alone would miss follow-ups. The engine checks `scopeOf` first.
14
+ By default it admits the GitHub associations `OWNER`, `MEMBER` and `COLLABORATOR`. A missing
15
+ association fails the check. Setting `PI_SUPERAGENT_ALLOWED` replaces that rule with a login list.
16
+
17
+ **Establish a baseline on the first poll.** An early version treated an entire backlog as new work
18
+ when the operator connected a repository. The source now saves the initial set of open issues and
19
+ seeds the review cursor. Closing and reopening an issue makes it eligible again. Its updated
20
+ timestamp distinguishes the new trigger from the earlier one. See `Intake.tla` C1 and C2.
21
+
22
+ **Save later read positions after delivery.** A poll stages its read position for the next poll to
23
+ commit. If the router crashes after reading a comment but before the engine receives it, the old
24
+ position lets the router read it again. Stable trigger IDs make that replay safe. The initial
25
+ baseline commits immediately because it represents history already present at setup. TLC found
26
+ the comment-loss case in the original implementation. The regression test reproduces that event
27
+ order. See `Intake.tla` C3 and C6.
28
+
29
+ ## Deciding what deserves a ticket
30
+
31
+ **Apply policy in code.** The router checks the author against current bot patterns even when a
32
+ stored trigger identifies the author as human. An older trigger once caused an agent’s request
33
+ for test approval to become a ticket to run the test. After the judge answers, the router also
34
+ checks whether an open ticket already owns the thread. The judge once accepted a duplicate despite
35
+ having the existing ticket in its prompt.
36
+
37
+ **Give the judge one request at a time.** Judging in the operator’s pane tied it up for an entire
38
+ model turn. Sending 41 candidates into one active turn produced answers out of order, and some
39
+ candidates expired before the judge got to them. The assessor now reuses one headless pi session
40
+ without extensions, skills or context files. It opens the session once because startup took 20 to
41
+ 45 seconds and concurrent requests used to start separate judges.
42
+
43
+ ## Running a worker
44
+
45
+ A worker can write “done” in its terminal while the person who opened the issue is still waiting
46
+ on GitHub. To finish a run, it calls `ticket_report` with an outcome. A *ready* report can carry
47
+ the link to a PR. A *blocked* report carries the question the requester needs to answer. The other
48
+ outcomes are *failed* and *cancelled*. Only the engine writes the result to the ticket.
49
+ See `Ticket.tla` I2.
50
+
51
+ If a run ends without a report, the router posts a failure on the thread. Launch errors and crashes
52
+ also get a reply there, so the person who asked knows what happened. The outbox recognizes repeat
53
+ failures. One broken setup command deserves one notice, even if the router tries it on every poll.
54
+ A quiet worker is a different case. After 15 minutes of silence it appears on the panel, where the
55
+ operator can see whether it needs help or is simply waiting for a build.
56
+
57
+ **Keep tool use within the assignment.** Worker tool hooks check file edits and writes against the
58
+ ticket workspace, block configured commands and disable delegation by default. The prepare hook
59
+ also gets a PATH with shims for blocked commands. Operators can use these controls to restrict
60
+ personal tools such as `gh`. The brief asks workers sharing the machine to run builds sequentially,
61
+ bound parallel jobs and clean up processes when they finish.
62
+
63
+ **Replace a crashed worker once.** The engine records the failed attempt, frees its slot and
64
+ requeues unread comments. The replacement gets the original workspace and a brief explaining the
65
+ crash. A second crash on the same ticket requires attention. The limit applies per router process
66
+ and prevents a persistent fault from spawning a new tab every few seconds.
67
+
68
+ **Count active work against the cap.** A parked worker retains its session for follow-ups and frees
69
+ a slot for queued work. A reply can briefly take the active count above the cap. Counting every
70
+ open session used to leave a queue of 22 tickets waiting while an idle worker occupied one of four
71
+ slots.
72
+
73
+ **Check the tab before closing it.** A finished worker gets ten minutes for follow-ups. The router
74
+ closes its tab once pi is idle and checks again every ten minutes while it is busy. Startup cleanup
75
+ handles tabs with no owning record or whose agent herdr no longer knows. Since herdr has no guarded
76
+ close operation, the liveness check matters whenever someone could still be working in a tab.
77
+ See `Workers.tla` W2, W4 and W5.
78
+
79
+ ## Ending a ticket
80
+
81
+ **Respect an explicit stop.** Once a person stops a ticket, later messages, wake-ups and retries
82
+ cannot resume it. Asking again creates a new ticket with its own history. See `Ticket.tla` I1.
83
+
84
+ **Follow the GitHub conversation.** When an issue disappears from the open set, the router can
85
+ observe the closure even if it cannot distinguish deletion, manual closure and closure by merge.
86
+ It stops the ticket and queues a notice. For a ticket awaiting its own PR, the PR’s merge or close
87
+ determines the outcome. Reopening an issue lets its author ask again.
88
+
89
+ **Bind the PR before processing a ready report.** A worker that changes a repository opens a PR and
90
+ includes its URL in its report. A number such as `#5` could also name an issue. Binding the URL
91
+ first lets the engine hold the ticket open for review. Merging or closing the PR ends the ticket,
92
+ and the issue receives the outcome. An investigation or deployment can finish with a ready report
93
+ alone. The worker knows which repository it changed, even when the request originated elsewhere.
94
+
95
+ The router calls the vendored engine’s `bindThread` and `threadClosed` transitions. It gets review
96
+ activity on the watched repository during the regular poll. For a PR elsewhere it polls every two
97
+ minutes, using three API calls per check.
98
+
99
+ **Route review feedback to the worker.** A submitted review on a bound PR goes to its ticket.
100
+ A review on an unrelated PR can reach the judge as a new request. GitHub accepts replies in the
101
+ PR conversation through its issue number. Opening a PR alone produces no request, so a worker can
102
+ deliver a change without triggering another worker to review it.
103
+
104
+ ## Operating the router
105
+
106
+ **Keep ticket state in one store.** One router owns the synchronous SQLite store in its state
107
+ directory. All writes from an engine command commit together. Recovery reopens the same directory.
108
+ The router role is tied to the appointed process because pi also loads the extension in worker
109
+ and judge sessions. Previously those sessions could each start a router.
110
+
111
+ **Report health through exit codes.** A scheduled check can act on `superagent status`. Code 0
112
+ means healthy or intentionally stopped. Code 1 indicates a router failure that a restart may fix.
113
+ Code 2 directs the operator to a failing component, including a GitHub source that failed three
114
+ polls in a row. Code 3 identifies a setup problem.
115
+
116
+ **Limit panel access to local callers.** The panel binds to loopback, validates the Host header
117
+ and requires `x-superagent-request: 1` for API calls. These checks prevent a page on another
118
+ site from reading the queue or pausing the router through DNS rebinding.
119
+
120
+ ## Maintaining the engine
121
+
122
+ The engine and its test suite match the pinned upstream revision. Local adapters live in
123
+ `src/pi/`, so an upstream fix can be imported and compared directly with its source.
124
+ [Adaptation ledger](../ADAPTATION.md)
125
+
126
+ The three TLA+ models live beside the implementation. Their actions name the functions they mirror
127
+ and their headers record the assumptions. When a transition changes, update its model and run
128
+ `scripts/check-formal`. The intake model’s crash counterexample is one case where checking all
129
+ orderings within a small bound found a failure the tests had missed.
130
+ [Architecture](architecture.md) · [Formal verification](formal-verification.md)
@@ -1,7 +1,16 @@
1
1
  # Formal verification
2
2
 
3
- Three parts of pi-superagent, the places where a bad ordering would quietly lose or duplicate work, are
4
- specified in TLA+ and checked exhaustively by TLC, the TLA+ model checker:
3
+ When is it safe to mark a GitHub comment as read? An early router saved its place as soon as it
4
+ read the comment. If it crashed before passing the message to the ticket engine, it restarted one
5
+ comment too far ahead. From the requester’s point of view, the worker had simply ignored them.
6
+
7
+ TLC found the precise sequence that lost the comment. It works by trying every ordering of events
8
+ allowed by a small model written in TLA+. That sequence became a regression test, and the router
9
+ now waits until the engine has received the comment before saving its place.
10
+
11
+ The project checks three such models. The ticket model asks who owns each conversation and how
12
+ work can end. The intake model follows messages across polls and crashes. The worker model follows
13
+ terminal tabs as agents start, finish and leave tabs behind.
5
14
 
6
15
  | Specification | Models | States explored (distinct) | Claims |
7
16
  | --- | --- | --- | --- |
@@ -9,12 +18,9 @@ specified in TLA+ and checked exhaustively by TLC, the TLA+ model checker:
9
18
  | [`Intake.tla`](../formal/Intake.tla) | the GitHub source (`source-github.ts`) | 565,551 (113,281) | 6 |
10
19
  | [`Workers.tla`](../formal/Workers.tla) | worker records, tabs and agents (`herdr-workers.ts`) | 347 (94) | 5 |
11
20
 
12
- Every claim holds in every reachable state. The specifications have already paid for themselves: the
13
- intake model found a bug in shipped code, a read position saved before the engine held its triggers, so
14
- that a crash at the wrong moment lost a request for good. TLC printed the exact interleaving, and the fix
15
- shipped with that interleaving as a regression test. Along the way TLC also caught two mistakes in the
16
- models themselves (a dropped evaluation batch, and a stopped ticket brought back to life), which is
17
- exactly what a checker is for.
21
+ The counts include repeat visits to a state. The numbers in parentheses count each distinct
22
+ state once. All listed properties hold within the bounds below. TLC also caught two errors in the
23
+ models themselves, one that dropped a batch of requests and another that revived a stopped ticket.
18
24
 
19
25
  ## Running the checks
20
26
 
@@ -23,17 +29,16 @@ scripts/check-formal # all three specifications, about five minutes
23
29
  ```
24
30
 
25
31
  The script needs a JDK (`brew install openjdk`, or point `JAVA` at one) and downloads `tla2tools.jar` to
26
- `~/.cache/tla2tools/` on first use. It exits 0 only when every specification is checked to completion. A
27
- missing JDK, a parse error, a violated claim or a truncated run each exits 1 with a short reason: a check
28
- that stays silent is treated as a failed one.
32
+ `~/.cache/tla2tools/` on first use. It exits 0 only when every specification is checked to completion.
33
+ It exits 1 and prints the reason if Java is missing, a specification fails to parse, TLC finds a
34
+ violation or a run ends early.
29
35
 
30
36
  ## What each specification proves
31
37
 
32
38
  ### `Ticket.tla`: the ticket lifecycle
33
39
 
34
40
  - **I1 StopAbsorbing.** A ticket a person stopped is never live again.
35
- - **I2 DoneFromReady.** A completed ticket always has a *ready* report, so completing without one is
36
- unreachable, not merely untested.
41
+ - **I2 DoneFromReady.** Every completed ticket has a recorded *ready* report.
37
42
  - **I3 OneRunning.** A ticket has at most one running assignment.
38
43
  - **I4 NoSecondTicket.** No ticket is created for a thread that a live ticket already follows, in any
39
44
  interleaving, including a follow-up that arrives in the middle of an evaluation.
@@ -65,42 +70,33 @@ that stays silent is treated as a failed one.
65
70
  - **W5 ZombieTabsGone.** After any finite number of launches, every dead agent's tab is eventually closed:
66
71
  the workspace cannot fill up with abandoned tabs.
67
72
 
68
- ## Why the results mean something
69
-
70
- **Each action mirrors real code.** Every action in a specification names the function whose guards it
71
- copies. When that function changes, the action must change with it, and the check must run again.
72
-
73
- **Each header says what the model leaves out.** Leaving behavior out narrows what a claim covers. A claim
74
- holds for the behaviors the model includes, and for a left-out feature only as far as that feature's code
75
- passes through the same guards. For example, every path that could resume a ticket checks the same
76
- `ended.by === 'human'` guard that I1 relies on. The headers list what is missing so that a reader can
77
- check this for themselves.
78
-
79
- **The bounds are small on purpose.** Two threads, two ticket slots and four messages; two issues, two
80
- comments and one crash; two workers and three launches. Small bounds let TLC visit every state in
81
- minutes, and ordering bugs almost always show up at small sizes. They are still bounds, not a proof for
82
- every size, and each header states its own.
83
-
84
- **Liveness rests on stated fairness.** The "eventually" claims (LT_Delivered, C6, W5) assume that an
85
- action which stays enabled eventually happens. Each specification writes its fairness assumptions down,
86
- next to the claims, so they can be argued with. Crashes and launches are bounded, so "eventually" always
87
- has an end.
88
-
89
- **Lessons from TLC, so you can skip them.** A property with primed variables (`[][...]_vars`) goes under
90
- `PROPERTY`, never `INVARIANT`. `UNCHANGED` takes a tuple `<<...>>`, not a set. An action that assigns
91
- `x'` must not also list `x` in its `UNCHANGED`: the contradiction silently disables the action, and only
92
- `-coverage 1` will show it. A specification whose runs can finish needs an explicit stuttering step
93
- (`Terminate == UNCHANGED vars`), or TLC reports every finished run as a deadlock; stuck work is then
94
- caught by the liveness claims. And naming a state `open` broke the parse, which is why the intake model
95
- says `opened`.
96
-
97
- ## What the proofs do not say
98
-
99
- A model proves things about the model. These cover the control flow described above, not the judge's
100
- decisions, GitHub's behavior (status codes, rate limits, pagination), SQLite, herdr, or the network.
101
- Newer behavior is tested but not modeled: reviews, the stop on a closed conversation, the outbox's check
102
- before a retry, and how each way a worker's run ends reaches the thread.
103
-
104
- What the specifications do guarantee: no implementation that follows these transitions can reach a
105
- forbidden state, and the liveness claims hold under the stated fairness. When the code a specification
106
- mirrors changes, upstream's ticket transitions included, the specification must change with it.
73
+ ## Interpreting the results
74
+
75
+ Each action names the function whose guards it mirrors. A change to that function may change which
76
+ states the implementation can reach. Update the model and rerun the check when a transition changes.
77
+ For example, I1 depends on every path that might resume a ticket checking `ended.by === 'human'`.
78
+
79
+ The ticket model has two threads, two ticket slots and four messages. The intake model has two
80
+ issues, two comments and one crash. The worker model has two workers and three launches. These
81
+ bounds let TLC explore every ordering in minutes. Each model header records its exclusions.
82
+
83
+ The eventual-delivery and cleanup properties (LT_Delivered, C6 and W5) also depend on fairness.
84
+ An action that stays enabled must eventually run. The bounds on crashes and launches give the
85
+ system a chance to finish delivery or cleanup.
86
+
87
+ ## Maintaining the models
88
+
89
+ - Put temporal formulas such as `[][A]_vars` under `PROPERTY`. Use `INVARIANT` for state predicates.
90
+ - Pass a tuple such as `<<x, y>>` to `UNCHANGED`. If an action assigns `x'`, omit `x` from that
91
+ tuple unless the assignment really must preserve its value. Use `-coverage 1` to spot disabled actions.
92
+ - Give a model with a valid terminal state an explicit stuttering step (`Terminate == UNCHANGED vars`).
93
+ This lets TLC distinguish normal termination from a deadlock. Liveness checks still detect stuck work.
94
+ - The intake model uses the state name `opened` because `open` caused a parse error.
95
+
96
+ ## Scope
97
+
98
+ The models describe control flow. The judge’s decisions, GitHub API behavior, SQLite, herdr and network
99
+ failures have their own behavior outside these bounds. Integration tests cover review routing,
100
+ closing a conversation, checking for a previous post before retrying, and notifying the thread
101
+ when a worker’s run ends. Consult the test suite for those cases and the model headers for the
102
+ assumptions behind each property.