owenloop 0.2.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +67 -0
- package/README.md +82 -7
- package/bin/owenloop.mjs +2 -2
- package/dist/add.d.ts +412 -0
- package/dist/add.js +999 -0
- package/dist/cli.d.ts +43 -0
- package/dist/cli.js +1865 -19
- package/dist/defs.d.ts +33 -0
- package/dist/defs.js +112 -19
- package/dist/engine.d.ts +338 -50
- package/dist/engine.js +854 -170
- package/dist/factory.d.ts +19 -0
- package/dist/factory.js +41 -5
- package/dist/hub.d.ts +514 -0
- package/dist/hub.js +783 -0
- package/dist/index.d.ts +4 -2
- package/dist/index.js +2 -1
- package/dist/model.d.ts +36 -0
- package/dist/store.d.ts +14 -2
- package/dist/store.js +269 -21
- package/dist/types.d.ts +92 -0
- package/dist/untar.d.ts +47 -0
- package/dist/untar.js +164 -0
- package/dist/util.d.ts +76 -0
- package/dist/util.js +96 -0
- package/docs/authoring.md +48 -1
- package/docs/cli.md +509 -3
- package/docs/design.md +72 -9
- package/docs/embedding.md +87 -7
- package/package.json +1 -1
package/docs/cli.md
CHANGED
|
@@ -6,16 +6,40 @@ in-process equivalent — see [`docs/embedding.md`](embedding.md).
|
|
|
6
6
|
|
|
7
7
|
Global flags: `--db <path>` (env `OWENLOOP_DB`, default `.owenloop/state.db`) and
|
|
8
8
|
`--defs <dir>` (env `OWENLOOP_DEFS`, default `./workflows`). Nothing is
|
|
9
|
-
remembered between invocations — pass both on every command.
|
|
9
|
+
remembered between invocations — pass both on every command. Opening the
|
|
10
|
+
**default** db path refuses a symlinked `.owenloop` directory — and a symlinked
|
|
11
|
+
`state.db` file (or any of its SQLite `-wal`/`-shm`/`-journal` sidecars) inside
|
|
12
|
+
a real `.owenloop` — rather than following it (filesystem-isolation guard
|
|
13
|
+
against a hostile checkout redirecting state writes); an explicit
|
|
14
|
+
`--db`/`OWENLOOP_DB` is operator intent and is created/opened as-is.
|
|
15
|
+
|
|
16
|
+
An unrecognized `--option` for a command is rejected — nonzero exit, the
|
|
17
|
+
offending flag named, and the nearest valid option suggested when close
|
|
18
|
+
enough — before any filesystem, keychain, or network effect (e.g. `push
|
|
19
|
+
--dryrn` no longer does a real push). `--db`/`--defs` remain accepted on
|
|
20
|
+
every command, as above. `--help` on any command (or bare `-h`/`help`)
|
|
21
|
+
prints this usage and exits 0 without doing any work.
|
|
22
|
+
|
|
23
|
+
Boolean flags (`--force`, `--dry-run`, `--all`, `--open`, `--terminal`,
|
|
24
|
+
`--recursive`, `--with-token`, `--shallow`, `--assume-provided`, and the bare
|
|
25
|
+
`--now` on `reap`) never take a following value — the next token is always a
|
|
26
|
+
positional or the next `--flag`, never consumed as this flag's argument. Use
|
|
27
|
+
`--flag=value` (e.g. `--now=<ms>` on `tick`) for flags that do take a value.
|
|
10
28
|
|
|
11
29
|
## Commands
|
|
12
30
|
|
|
13
31
|
| command | what it does |
|
|
14
32
|
|---|---|
|
|
15
33
|
| `defs` | list available workflow definitions |
|
|
34
|
+
| `add <owner>/<repo>[@ref]` | fetch, validate, and install a repo's workflow defs from GitHub (public repos only) — see below |
|
|
35
|
+
| `add --recover` | finish or undo a crash-interrupted install, offline — no network call — see below |
|
|
36
|
+
| `login [--hub <url>] [--with-token] [--as <slot>]` | authenticate the CLI against a hub — loopback OAuth, or `--with-token` from stdin — see [Hub](#hub-login--connect--push--logout) |
|
|
37
|
+
| `connect [--hub <url>] [--as <slot>]` | bind this project to a hub (writes `.owenloop/hub.json`) and verify the credential |
|
|
38
|
+
| `push [<defName>...] [--force] [--dry-run] [--as <slot>]` | publish local workflow defs to the bound hub (idempotent against the hub's own def hashes) |
|
|
39
|
+
| `logout [--hub <url>] [--as <slot>]` | delete the stored credential for a hub |
|
|
16
40
|
| `create <def> [--title t] [--provide name=json …] [--param k=v …]` | start an instance; prints `{workflow}` |
|
|
17
41
|
| `provide <wf> <name> [--value json]` | supply a seeded input after the fact |
|
|
18
|
-
| `tick <wf> [--now <
|
|
42
|
+
| `tick <wf> [--now=<ms>] [--shallow] [--label <l>]…` | claim and emit eligible **orders** (the jobs to run); deep by default — also descends into live `calls:` children (`--shallow` = this instance only); repeatable `--label` claims unlabeled steps plus matching-label steps — see below |
|
|
19
43
|
| `reap <wf> [--now]` | run the reaper; `--now` forces every claim stale (TTL 0) — see below |
|
|
20
44
|
| `runs <wf> [--open]` | list this instance's runs, joining claim state for open ones |
|
|
21
45
|
| `status <wf>` | derived view: `done`, `debts`, `eligible`, `blocked`, `inFlight` |
|
|
@@ -33,6 +57,442 @@ remembered between invocations — pass both on every command.
|
|
|
33
57
|
| `delete <wf>` | delete an instance and all its rows |
|
|
34
58
|
| `adopt <wf>` | re-pin an instance to the current definition and settle any new debts |
|
|
35
59
|
|
|
60
|
+
## `add` — installing shared workflow defs from GitHub
|
|
61
|
+
|
|
62
|
+
`owenloop add <owner>/<repo>[@ref]` fetches a public GitHub repo's
|
|
63
|
+
`workflows/**` folder (via GitHub's REST API and Node's built-in `fetch` — no
|
|
64
|
+
new dependency), validates every def with the same lint/validate/`check`
|
|
65
|
+
machinery `owenloop lint`/`owenloop check` use — plus a strict cross-def
|
|
66
|
+
backstop (include expansion, `calls:` target/inputs/output-count checks,
|
|
67
|
+
cycle detection) on the staged tree, so an error `loadDefsRaw` would
|
|
68
|
+
otherwise swallow still refuses the install — and only then installs them
|
|
69
|
+
under `<defsDir>/<owner>-<repo>-<hash>/`, where `<hash>` is the first 8 hex
|
|
70
|
+
characters of `sha256(owner/repo)`. The hash keeps distinct sources that used
|
|
71
|
+
to collide on the same `<owner>-<repo>` folder (e.g. `a-b/c` and `a/b-c`)
|
|
72
|
+
from clobbering each other. `owner` and `repo` are restricted to the
|
|
73
|
+
GitHub-legal charset (letters, digits, `.`, `_`, `-`) so the folder is always
|
|
74
|
+
a single safe path segment. A def that fails parse, lint, validation, or has
|
|
75
|
+
a definite `check` defect refuses the **whole** add — nothing is written,
|
|
76
|
+
and every reason is printed.
|
|
77
|
+
|
|
78
|
+
`ref` defaults to `HEAD` (the repo's default branch) and is pinned to the
|
|
79
|
+
resolved commit sha before anything is fetched or installed. Provenance is
|
|
80
|
+
recorded in `.owenloop/installed.json`:
|
|
81
|
+
|
|
82
|
+
```jsonc
|
|
83
|
+
{
|
|
84
|
+
"version": 1,
|
|
85
|
+
"installed": {
|
|
86
|
+
"<owner>/<repo>": {
|
|
87
|
+
"source": "<owner>/<repo>",
|
|
88
|
+
"ref": "HEAD",
|
|
89
|
+
"sha": "<40-char-commit-sha>",
|
|
90
|
+
"installedAt": 1699999999999,
|
|
91
|
+
"path": "<owner>-<repo>-<hash>",
|
|
92
|
+
"files": ["foo.yaml", "sub/bar.yaml"]
|
|
93
|
+
}
|
|
94
|
+
}
|
|
95
|
+
}
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Re-running `add` for the same repo is idempotent: the fetch is staged under
|
|
99
|
+
`<defsDir>/.owenloop-staging/`, validated, and swapped into place with an
|
|
100
|
+
atomic rename, replacing the previous install and lockfile entry so a file
|
|
101
|
+
removed upstream disappears locally too. The directory swap and the lockfile
|
|
102
|
+
write are one recoverable operation: the install is *committed* only when
|
|
103
|
+
`.owenloop/installed.json` is atomically replaced, and the displaced previous
|
|
104
|
+
directory (and any old-name directory) is kept until that write succeeds. Any
|
|
105
|
+
failure before that commit point — a validation error, a lock timeout, an
|
|
106
|
+
interrupted rename, a failure parking the old-name directory during migration,
|
|
107
|
+
or a lockfile-write failure *after* the directory swap — rolls the directory
|
|
108
|
+
state back, restoring the previous install and any old-name directory and
|
|
109
|
+
leaving the lockfile unchanged, with no staging debris. The one deliberate
|
|
110
|
+
exception is a rollback double fault — the follow-on step fails *and* restoring
|
|
111
|
+
the directory state fails too. For a lockfile-write double fault, the
|
|
112
|
+
displaced previous content is intentionally preserved under
|
|
113
|
+
`<defsDir>/.owenloop-staging/` and the error names that path; recover it
|
|
114
|
+
before re-running `add`. For a park double fault during old-name migration,
|
|
115
|
+
there's no staging backup to preserve — the old-name directory was never
|
|
116
|
+
moved, so it stays intact at its original path, and the error instead names
|
|
117
|
+
the newly installed content stranded at the destination path; re-running
|
|
118
|
+
`add` recovers automatically, discarding the stranded content and leaving the
|
|
119
|
+
previous install in place.
|
|
120
|
+
|
|
121
|
+
Those rollbacks cover *in-process* failures — a thrown error `add` catches. A
|
|
122
|
+
hard kill (a process crash, SIGKILL, or other termination) partway through the
|
|
123
|
+
commit skips them entirely, so `add` also keeps a one-record crash-recovery
|
|
124
|
+
journal at `.owenloop/add.journal`: it is written just before the first
|
|
125
|
+
destructive step (phase `applying`), advanced to `finalizing` the instant the
|
|
126
|
+
lockfile write — the commit point — succeeds, and removed once the install
|
|
127
|
+
finishes. The next `add` reads it under the same lock, *before* clearing
|
|
128
|
+
staging, and brings the tree back to a consistent (defs ⇔ ledger) state: at or
|
|
129
|
+
past the commit point it rolls **forward** (discards the retained backup and
|
|
130
|
+
finishes the install); before it, it rolls **back** (restores the previous
|
|
131
|
+
install, or discards an orphaned fresh-install directory *only* when the ledger
|
|
132
|
+
corroborates an interrupted old-name migration — a journal naming an existing
|
|
133
|
+
directory with no corroborating ledger, staging, or backup evidence is refused
|
|
134
|
+
fail-closed and never deletes it, and the error names the manual remedy).
|
|
135
|
+
Recovery is idempotent and re-derives
|
|
136
|
+
every path it touches from the current defs directory, so a crash *during*
|
|
137
|
+
recovery just replays. The journal is treated as hostile input exactly like the
|
|
138
|
+
lockfile: it is validated fail-closed (every path field a safe single segment),
|
|
139
|
+
its recorded defs directory must match this run's, and a symlink where a
|
|
140
|
+
directory is expected is refused — any bad shape, mismatch, or contradictory
|
|
141
|
+
on-disk state refuses with no filesystem mutation and leaves the journal in
|
|
142
|
+
place as evidence. A rollback double fault likewise leaves the journal behind,
|
|
143
|
+
so the next `add` retries the restore automatically before touching staging.
|
|
144
|
+
|
|
145
|
+
`add --recover` runs that same recovery on demand, standalone, with **no
|
|
146
|
+
network call** — for a machine that crashed mid-install and is still offline
|
|
147
|
+
when you need the tree usable again, rather than waiting on the normal `add`
|
|
148
|
+
path (which also recovers inline, but only after its SHA and tarball fetches).
|
|
149
|
+
It takes no `<owner>/<repo>` argument — `add --recover acme/widgets` refuses
|
|
150
|
+
rather than guessing whether you meant "recover then install" (that's just a
|
|
151
|
+
plain `owenloop add acme/widgets`, which recovers inline anyway). It acquires
|
|
152
|
+
the same `.owenloop/add.lock`, calls `recoverInterruptedInstall`, and prints
|
|
153
|
+
one of three outcomes: `{"ok":true,"recovered":false,...}` when there was no
|
|
154
|
+
journal to act on, or `{"ok":true,"recovered":true,"outcome":"rolled-forward"|"rolled-back",...}`
|
|
155
|
+
when it finished or undid the interrupted install. A refusal (bad, mismatched,
|
|
156
|
+
or contradictory journal) throws the same as the inline path: exit 1, nothing
|
|
157
|
+
mutated, the journal left in place as evidence.
|
|
158
|
+
|
|
159
|
+
The recovery guarantee covers *process* death — a crash, SIGKILL, or
|
|
160
|
+
termination — not sudden power loss. Journal and lockfile writes are atomic
|
|
161
|
+
tmp-file-plus-rename *without* `fsync`/`fdatasync` or a directory sync, so an
|
|
162
|
+
atomic rename prevents partially-visible JSON but does not force the data or the
|
|
163
|
+
directory entry to durable storage: a power failure can lose the journal or a
|
|
164
|
+
just-written ledger entry entirely. Real fsync-based durability across power
|
|
165
|
+
loss is a tracked follow-up, deliberately out of scope here.
|
|
166
|
+
|
|
167
|
+
Concurrent `add` runs in the same project serialize on a `.owenloop/add.lock`
|
|
168
|
+
file; one that can't acquire the lock within 10s fails cleanly instead of
|
|
169
|
+
interleaving with another install. `add` also refuses to replace a
|
|
170
|
+
destination folder the lockfile doesn't record this source as owning (e.g. a
|
|
171
|
+
hand-placed folder that happens to collide) — remove it manually or fix the
|
|
172
|
+
lockfile to proceed. A repo previously installed under the old
|
|
173
|
+
`<owner>-<repo>` naming is migrated to the new hashed folder automatically,
|
|
174
|
+
and the old one is removed only once the new lockfile entry is durably written.
|
|
175
|
+
|
|
176
|
+
The lock's stale-reclaim is liveness-aware, not purely age-based: a lock
|
|
177
|
+
whose recorded pid is alive on this host is never reclaimed no matter how
|
|
178
|
+
long it's held, and one whose pid is dead is reclaimed immediately. Age (the
|
|
179
|
+
10-minute window) governs reclaim only as a fail-closed fallback for a lock
|
|
180
|
+
this process can't attribute to a live owner — unparseable, or missing a
|
|
181
|
+
pid. A lock recorded from a **different host** is never age-reclaimed
|
|
182
|
+
either, since a pid liveness check proves nothing about a foreign PID space;
|
|
183
|
+
it's held until its own machine clears it. Each acquisition writes a
|
|
184
|
+
per-lock ownership token, and release only deletes the file if that token
|
|
185
|
+
still matches — so a holder that loses a race can never delete a lock a
|
|
186
|
+
fresh owner has since re-acquired. A lock file that can be `stat`'d but not
|
|
187
|
+
read (e.g. root-owned, or mid-write) no longer spins the acquire loop
|
|
188
|
+
sleeplessly; it falls through to the normal poll sleep and still respects
|
|
189
|
+
the `waitMs` timeout.
|
|
190
|
+
|
|
191
|
+
`add` never trusts `.owenloop/installed.json` for filesystem paths: the lockfile
|
|
192
|
+
is validated fail-closed on read. A file that parses but is structurally invalid
|
|
193
|
+
— an unsupported `version`, a malformed or key-mismatched entry, a non-hex
|
|
194
|
+
`sha`, an escaping `files` entry, or a `path` that is not a single safe folder
|
|
195
|
+
segment (any `..`, absolute, or separator-bearing `path` is refused, never
|
|
196
|
+
normalized) — is a hard error naming the offending entry and field, never
|
|
197
|
+
silently reset to empty (which would erase ownership records). This closes a
|
|
198
|
+
directory-migration path where a crafted committed lockfile could make `add`
|
|
199
|
+
move and then delete a directory outside the defs dir.
|
|
200
|
+
|
|
201
|
+
`add` also refuses a symlinked project `.owenloop` directory and a symlinked
|
|
202
|
+
**default** defs dir before any state write — the same filesystem-isolation
|
|
203
|
+
guard used elsewhere, closing the one spot `add` hadn't yet applied it to. A
|
|
204
|
+
hostile checkout shipping `.owenloop -> /elsewhere` or a symlinked `./workflows`
|
|
205
|
+
could otherwise redirect `add.lock`, `installed.json`, and the staged/committed
|
|
206
|
+
defs outside the project. `.owenloop` is guarded unconditionally, since its
|
|
207
|
+
lock and lockfile paths are always `cwd`-derived in `add` with no override; the
|
|
208
|
+
defs dir is guarded only on the default `cwd/workflows` fallback — an explicit
|
|
209
|
+
`--defs`/`OWENLOOP_DEFS` is operator intent and is installed through as-is,
|
|
210
|
+
matching the `--db`/`OWENLOOP_DB` rule above.
|
|
211
|
+
|
|
212
|
+
**Installed-def discovery.** Defs installed by `add` are discovered by default:
|
|
213
|
+
a plain `owenloop defs`/`create`/`tick` against the DEFAULT defs dir
|
|
214
|
+
(`cwd/workflows`) sees them by name, no `--defs` flag needed. `loadDefs` itself
|
|
215
|
+
stays a pure dir-scanner (top-level `*.yaml` plus immediate-subdir
|
|
216
|
+
`workflow.yaml`); the CLI folds installed subfolders in on top, ledger-driven and
|
|
217
|
+
bounded — it only loads folders named by the fail-closed-validated
|
|
218
|
+
`.owenloop/installed.json` entries, never a raw recurse of the tree.
|
|
219
|
+
|
|
220
|
+
- **Only under the default defs dir.** An explicit `--defs`/`OWENLOOP_DEFS` is
|
|
221
|
+
operator intent to target a literal dir and keeps the pure-scan behavior with
|
|
222
|
+
no fold-in — the rule is "was an override given", so even
|
|
223
|
+
`OWENLOOP_DEFS=$PWD/workflows` counts as an override and stays literal.
|
|
224
|
+
Pointing `--defs` straight at an install folder
|
|
225
|
+
(`--defs workflows/<owner>-<repo>-<hash>`) still works exactly as before.
|
|
226
|
+
- **Precedence.** Project-local (top-level) defs win over installed defs; among
|
|
227
|
+
installed entries the ledger sources are iterated in sorted order and the
|
|
228
|
+
first-loaded def with a given name wins. Every shadowed def is reported as a
|
|
229
|
+
`warning:` on stderr (stdout JSON stays clean), never a silent clobber.
|
|
230
|
+
- **Fail-open.** The fold-in never breaks base loading. A corrupt or
|
|
231
|
+
structurally-invalid `installed.json`, a missing install folder, or an install
|
|
232
|
+
folder that fails to load each emits a `warning:` on stderr and is skipped;
|
|
233
|
+
your project defs still load and the command still exits 0. The add-time
|
|
234
|
+
fail-closed lockfile validation is unchanged — discovery consumes it and simply
|
|
235
|
+
refuses to act on a bad ledger rather than crashing.
|
|
236
|
+
|
|
237
|
+
Public repos only — no auth/token support yet; a private repo (or a bad
|
|
238
|
+
ref) surfaces as a 404 from the sha-resolve step.
|
|
239
|
+
|
|
240
|
+
**Trust model — what `add` does and does not protect.** Installing a package
|
|
241
|
+
executes nothing at install time: `add` only fetches, validates, and writes
|
|
242
|
+
YAML under the defs dir. But an installed def's steps *run* later, with
|
|
243
|
+
whatever privileges the host process and its dispatcher grant their workers.
|
|
244
|
+
owenloop itself never executes a step body — `worker:`/`command:` are opaque
|
|
245
|
+
labels it carries through untouched and never shells out (see [What owenloop is
|
|
246
|
+
not](../README.md#what-owenloop-is-not) and
|
|
247
|
+
[`docs/authoring.md`](authoring.md#worker--declaring-the-executor)) — so the
|
|
248
|
+
real risk surface is the conductor or worker you point at these defs: the
|
|
249
|
+
prompts and `command:` strings that ship in a package are handed to agents that
|
|
250
|
+
typically run with your full local privileges. **Install only sources you
|
|
251
|
+
trust.**
|
|
252
|
+
|
|
253
|
+
**Pin a commit SHA for anything you re-add.** `owenloop add <owner>/<repo>@<ref>`
|
|
254
|
+
takes a branch, tag, or commit SHA as `<ref>` — all three resolve through
|
|
255
|
+
GitHub's `GET /repos/<owner>/<repo>/commits/<ref>`. A branch or tag can move
|
|
256
|
+
under you between re-adds; only a SHA guarantees the same bytes every time.
|
|
257
|
+
`add` already resolves whatever `ref` you give (default `HEAD`) to a concrete
|
|
258
|
+
commit sha before fetching, and records that sha in `.owenloop/installed.json`
|
|
259
|
+
(above) — so a single install is already pinned by that record; the SHA
|
|
260
|
+
recommendation is about following a *moving* ref across later re-adds/updates.
|
|
261
|
+
|
|
262
|
+
The protections described under **Untrusted-archive safety** below guard the
|
|
263
|
+
install *step* — staged all-or-nothing validation with an atomic swap, path
|
|
264
|
+
containment (including symlink-aware `bodyFile` checks), and archive resource
|
|
265
|
+
bounds — not the def *content* that later executes. `add` validates a def's
|
|
266
|
+
structure and contains its paths; it does not sandbox or sanitize what a step
|
|
267
|
+
body will do once a worker runs it. That trust decision stays yours.
|
|
268
|
+
|
|
269
|
+
**Untrusted-archive safety.** `add` treats the fetched repo as untrusted and
|
|
270
|
+
refuses the whole install (nothing written) on any of these:
|
|
271
|
+
|
|
272
|
+
- **Path containment.** An archive entry whose path would escape the install
|
|
273
|
+
dir (absolute, or a `..` component) is rejected, and every offender is named.
|
|
274
|
+
A def's `bodyFile` is likewise resolved symlink-aware and must be a regular
|
|
275
|
+
file inside the def's own directory — an absolute path, a `..` component, or
|
|
276
|
+
a symlink pointing outside is refused (both while staging and when a
|
|
277
|
+
previously-installed package is later loaded).
|
|
278
|
+
- **Resource bounds on extraction.** The download is capped at 256 MiB
|
|
279
|
+
compressed and 1 GiB expanded (a gzip bomb aborts at inflate time rather than
|
|
280
|
+
exhausting memory), 50k files, 100 MiB per file, and 1024-char entry paths.
|
|
281
|
+
The 256 MiB compressed cap is enforced *during* the download by a bounded
|
|
282
|
+
streaming reader — a response advertising an oversize `Content-Length` is
|
|
283
|
+
refused before any body is read, and a body that streams past the cap is
|
|
284
|
+
cancelled the moment it crosses it, so an oversized archive is never fully
|
|
285
|
+
buffered into memory (the extraction limits above still re-check the size once
|
|
286
|
+
the bytes are in hand).
|
|
287
|
+
- **Request timeouts.** The sha-resolve fetch times out after 30s and the
|
|
288
|
+
tarball download after 5 min, each surfacing as a friendly error.
|
|
289
|
+
|
|
290
|
+
## Hub (`login` / `connect` / `push` / `logout`)
|
|
291
|
+
|
|
292
|
+
These four commands publish local workflow defs to a hosted **hub** (default
|
|
293
|
+
`https://api.owenloop.com`; override per-command with `--hub <url>` or the
|
|
294
|
+
`OWENLOOP_HUB` env var). They are the only network-bound commands besides
|
|
295
|
+
`add`, and they talk only to endpoints the hub exposes today — no new
|
|
296
|
+
service-side surface. The hub URL is normalized to its origin
|
|
297
|
+
(`scheme://host[:port]`); path/query are dropped. `https` is required for
|
|
298
|
+
every hub origin except the loopback hosts (`127.0.0.1`, `::1`, `localhost`),
|
|
299
|
+
which may use `http` for local development — a remote `http` URL is rejected at
|
|
300
|
+
normalization time so a plaintext origin can never be persisted as a credential
|
|
301
|
+
key or project binding. A legacy `hub.json` carrying a remote-http origin is
|
|
302
|
+
likewise refused at push time, with a hint to re-run `owenloop connect`. The CLI
|
|
303
|
+
also never follows an HTTP redirect from a hub — a 3xx response to any hub/auth
|
|
304
|
+
request is treated as an error — so a compromised or misconfigured hub cannot
|
|
305
|
+
bounce credentials or workflow YAML to another origin (same-origin validation
|
|
306
|
+
covers only the initial URL; a redirect would otherwise re-send the request body
|
|
307
|
+
cross-origin).
|
|
308
|
+
|
|
309
|
+
**Request timeouts.** Every hub call — OAuth discovery, client registration,
|
|
310
|
+
code exchange, token refresh, `whoami`, the workflow list, and each push — is
|
|
311
|
+
bounded by a 30s deadline; a stalled hub surfaces as a friendly `hub did not
|
|
312
|
+
respond within 30s` error instead of hanging. `OWENLOOP_HUB_TIMEOUT_MS`
|
|
313
|
+
overrides the budget (a test knob).
|
|
314
|
+
|
|
315
|
+
**Response-size cap.** Every hub/auth response body is read through the same
|
|
316
|
+
bounded streaming reader, capped at 8 MiB — hub responses are small JSON
|
|
317
|
+
round-trips, so a body advertising or streaming past that cap is refused (the
|
|
318
|
+
stream cancelled) rather than buffered, closing the same memory-exhaustion gap
|
|
319
|
+
on the hub path that the `add` download cap closes on the archive path.
|
|
320
|
+
`OWENLOOP_HUB_MAX_RESPONSE_BYTES` overrides the cap (a test knob).
|
|
321
|
+
|
|
322
|
+
### `login` — authenticate the CLI against a hub
|
|
323
|
+
|
|
324
|
+
Two ways to get a credential, both of which **verify before storing** (a token
|
|
325
|
+
that can't call the hub is never written to disk):
|
|
326
|
+
|
|
327
|
+
- **Loopback OAuth (default).** `owenloop login` binds a single-use catcher on
|
|
328
|
+
`127.0.0.1:<random-port>`, dynamically registers a public client
|
|
329
|
+
(`token_endpoint_auth_method: none`), opens your browser to the hub's
|
|
330
|
+
authorize endpoint with an auth-code + PKCE (S256) challenge, and exchanges
|
|
331
|
+
the returned code for an access/refresh token. State is checked on the
|
|
332
|
+
callback (CSRF guard) and the flow times out after 5 minutes. The exact
|
|
333
|
+
loopback `redirect_uri` is sent in the registration because the hub matches
|
|
334
|
+
redirect URIs by exact string (no RFC 8252 variable-port allowance).
|
|
335
|
+
- **Paste a token.** `… | owenloop login --with-token` reads a single token
|
|
336
|
+
from stdin (never argv, so it stays out of your shell history and the process
|
|
337
|
+
table). An `olp_`-prefixed **agent** token or an `mcpat_`-prefixed **access**
|
|
338
|
+
token is accepted; anything else is rejected before any network call.
|
|
339
|
+
|
|
340
|
+
**Credential slots (`--as`).** A hub origin holds more than one credential, each
|
|
341
|
+
in a named **slot**, so a human sign-in and any number of agent tokens coexist
|
|
342
|
+
on the same machine without overwriting each other:
|
|
343
|
+
|
|
344
|
+
| `--as` value | slot | who it is |
|
|
345
|
+
|---|---|---|
|
|
346
|
+
| *(omitted)* | depends on the credential — see below | |
|
|
347
|
+
| `human` | `human` | you, via loopback OAuth or a pasted `mcpat_` token |
|
|
348
|
+
| `agent` | `agent:default` | an agent token with no account name |
|
|
349
|
+
| `agent:<account>` | `agent:<account>` | a named agent, e.g. `agent:ci` |
|
|
350
|
+
|
|
351
|
+
An account name is 1–64 characters matching `[A-Za-z0-9][A-Za-z0-9._-]*`;
|
|
352
|
+
anything else is a usage error. With `--as` omitted, a credential lands in the
|
|
353
|
+
slot it belongs to: loopback OAuth and pasted `mcpat_` tokens go to `human`,
|
|
354
|
+
`olp_` agent tokens go to `agent:default`. The two contradictions are refused as
|
|
355
|
+
usage errors **before any network call**, so nothing unverified is stored:
|
|
356
|
+
`--as human` with an `olp_` token, and `--as agent[:…]` with an OAuth or pasted
|
|
357
|
+
human credential. `login`'s JSON reports the `slot` it wrote.
|
|
358
|
+
|
|
359
|
+
`connect`, `push`, and `logout` take the same `--as` and act on exactly that
|
|
360
|
+
slot — there is no fallback to another slot, so `push --as agent:ci` with an
|
|
361
|
+
empty `agent:ci` fails rather than quietly pushing as you. `logout` without
|
|
362
|
+
`--as` removes only `human`.
|
|
363
|
+
|
|
364
|
+
**Where the credential lands.** On macOS it goes into the login **Keychain**
|
|
365
|
+
(`security`, service `owenloop:<hub origin>`, one item per slot, with the slot
|
|
366
|
+
name as the account) with the secret fed over stdin, never on the command line.
|
|
367
|
+
Elsewhere — or with `OWENLOOP_NO_KEYCHAIN=1` — it falls back to a `0600` file at
|
|
368
|
+
`$XDG_CONFIG_HOME/owenloop/credentials.json` (or `~/.config/owenloop/…`) inside
|
|
369
|
+
a `0700` directory, keyed `hubs[origin][slot]`. Either way the token is never
|
|
370
|
+
written into the repo or a `.env`. `login`'s JSON reports `storage: "keychain" |
|
|
371
|
+
"file"` and `kind`, and prints **no token value** to stdout/stderr. A credential
|
|
372
|
+
stored by an earlier release used a different keying and is **not** read; there
|
|
373
|
+
is deliberately no migration, so re-run `owenloop login`.
|
|
374
|
+
|
|
375
|
+
The backend is chosen once from your platform and env and then used for every
|
|
376
|
+
read and write — a keychain-backed CLI never silently drops to the file store.
|
|
377
|
+
If the keychain write fails (locked or unavailable), `login` errors out instead
|
|
378
|
+
of writing the secret elsewhere: unlock the keychain, or set
|
|
379
|
+
`OWENLOOP_NO_KEYCHAIN=1` to select the file store up front. Programmatic hosts
|
|
380
|
+
can read (only read) a stored credential through the same backend logic via the
|
|
381
|
+
package's exported `readStoredCredential` — see
|
|
382
|
+
[Embedding](embedding.md#whats-exported).
|
|
383
|
+
|
|
384
|
+
**Supplying the credential from your own tooling.** If your secrets live in a
|
|
385
|
+
secret manager, or you run on a host with no keychain, set
|
|
386
|
+
`OWENLOOP_CREDENTIAL_COMMAND` to a shell command line that prints a credential.
|
|
387
|
+
It takes precedence over both stores, so the full order is **external command →
|
|
388
|
+
keychain → file**, still chosen once. Nothing is auto-detected: the variable is
|
|
389
|
+
the only way to turn this on, and an unset or blank value leaves everything
|
|
390
|
+
exactly as described above.
|
|
391
|
+
|
|
392
|
+
The contract:
|
|
393
|
+
|
|
394
|
+
- The command runs as `/bin/sh -c "<your command>"`, so a pipeline or arguments
|
|
395
|
+
work (`my-helper --hub prod`).
|
|
396
|
+
- Context arrives in the **environment**, not on the command line:
|
|
397
|
+
`OWENLOOP_CREDENTIAL_ORIGIN` (the normalized hub origin) and
|
|
398
|
+
`OWENLOOP_CREDENTIAL_SLOT` (`human` or `agent:<account>`). Your command should
|
|
399
|
+
return the credential for exactly that pair.
|
|
400
|
+
- `OWENLOOP_CREDENTIAL_COMMAND` is **removed** from the command's own
|
|
401
|
+
environment, so a helper that shells back into `owenloop` cannot recurse.
|
|
402
|
+
- It must print a credential as a JSON object on **stdout** — the same shape
|
|
403
|
+
stored in `credentials.json`, e.g. `{"kind":"agent","accessToken":"olp_…"}` or
|
|
404
|
+
a full `{"kind":"oauth", …}` object. A bare token is not accepted. stdout is
|
|
405
|
+
captured and never logged; the command's **stderr passes straight through** to
|
|
406
|
+
your terminal, so put diagnostics there — never the secret.
|
|
407
|
+
- It must finish within 10s, overridable with
|
|
408
|
+
`OWENLOOP_CREDENTIAL_COMMAND_TIMEOUT_MS`.
|
|
409
|
+
|
|
410
|
+
A configured command is **authoritative**: a nonzero exit, a timeout, empty
|
|
411
|
+
output, or output that is not a well-formed credential is a hard error naming
|
|
412
|
+
the hub and the slot — never a quiet fall back to a keychain or file entry,
|
|
413
|
+
which would risk handing back a stale key. For the same reason `login` refuses
|
|
414
|
+
to run while the variable is set (unset it if you want to use the local store
|
|
415
|
+
again), and a refreshed OAuth token is not written to the local store — your
|
|
416
|
+
command owns the credential's lifecycle. `logout` still clears local entries.
|
|
417
|
+
|
|
418
|
+
Both branches verify the credential against `GET /api/whoami` before storing
|
|
419
|
+
it — a `401` there means the credential is never written to disk. On success
|
|
420
|
+
`login`'s JSON reports the org and identity it authenticated as (`org`,
|
|
421
|
+
`orgId`, `identity`, and `email` when the hub returns one), read straight from
|
|
422
|
+
`whoami`.
|
|
423
|
+
|
|
424
|
+
### `connect` — bind a project to a hub
|
|
425
|
+
|
|
426
|
+
`owenloop connect` writes `.owenloop/hub.json` recording which hub this project
|
|
427
|
+
publishes to, after re-verifying the stored credential against `GET
|
|
428
|
+
/api/whoami`. Run `login` first. The JSON reports the same org/identity fields
|
|
429
|
+
as `login`; re-connecting to the **same** origin reports no `switchedFrom`,
|
|
430
|
+
switching to a **different** hub reports `switchedFrom: <old origin>` and
|
|
431
|
+
rebinds the project to the new one.
|
|
432
|
+
|
|
433
|
+
A symlinked project `.owenloop` directory is refused with a clear error rather
|
|
434
|
+
than followed: a hostile checkout cannot ship `.owenloop -> /elsewhere` to
|
|
435
|
+
redirect the `hub.json` write outside the project (filesystem-isolation
|
|
436
|
+
guarantee). The same refusal covers the default `state.db` FILE (and its SQLite
|
|
437
|
+
`-wal`/`-shm`/`-journal` sidecars) inside a real `.owenloop` — a symlinked db
|
|
438
|
+
file would otherwise redirect the store's writes, since SQLite follows file
|
|
439
|
+
symlinks.
|
|
440
|
+
|
|
441
|
+
### `push` — publish local defs to the bound hub
|
|
442
|
+
|
|
443
|
+
`owenloop push [<defName>...]` publishes the project's workflow defs (all of
|
|
444
|
+
them, or just the named ones) to the hub the project is `connect`ed to. It
|
|
445
|
+
reuses the **exact** all-or-nothing validation gate `add` uses — lint, validate,
|
|
446
|
+
and a bounded `check` — across every selected def before a single byte is sent;
|
|
447
|
+
any definite defect aborts the whole push. stdout is machine-parseable JSON;
|
|
448
|
+
the human-readable diff (`+ new`, `~ changed`, `= unchanged`, `! failed`) goes
|
|
449
|
+
to stderr.
|
|
450
|
+
|
|
451
|
+
**Idempotency is server-side truth, not a client ledger.** `push` fetches the
|
|
452
|
+
hub's own view of every def (`GET /api/workflows`, which reports each def's
|
|
453
|
+
`hash`) and diffs local content against it directly — there is no
|
|
454
|
+
`.owenloop/hub.json` push ledger to go stale, drift, or need migrating. A def
|
|
455
|
+
whose content hash matches the hub's is `unchanged` and is never sent at all.
|
|
456
|
+
`--dry-run` reports the plan (`new`/`changed`/`unchanged`, and `wouldPush`)
|
|
457
|
+
without sending anything. A real push that does go out can still come back
|
|
458
|
+
`noop`: the hub's `create_workflow` is itself idempotent by content hash, so
|
|
459
|
+
if server truth and local truth briefly disagree (e.g. `--force` re-sending
|
|
460
|
+
content that's actually already there), the hub reports `{unchanged: true}`
|
|
461
|
+
and no new version is minted — `push`'s JSON distinguishes `pushed`
|
|
462
|
+
(version-forwarded) from `noop` (server said unchanged) from `unchanged`
|
|
463
|
+
(skipped locally, never sent). `--force` re-sends every selected def
|
|
464
|
+
regardless of the local diff. A `<defName>` that doesn't resolve is an error;
|
|
465
|
+
an `{ok:false}` (or a malformed `2xx` whose identity fields don't match the
|
|
466
|
+
pushed def) from the hub mid-batch records that def under `failed`, keeps the
|
|
467
|
+
defs that did land, and exits 1. A `429` (rate limited) instead halts the whole
|
|
468
|
+
batch: the current def is recorded as `failed`, the not-yet-attempted remainder
|
|
469
|
+
is reported under a `skipped` output key, and any `Retry-After` the hub sent is
|
|
470
|
+
surfaced in the error.
|
|
471
|
+
|
|
472
|
+
The def hash is computed by re-parsing the raw YAML with no checkout-specific
|
|
473
|
+
`baseDir` — the same canonicalization the hub applies — so it's portable
|
|
474
|
+
across checkouts and machines by construction: a fresh clone at a different
|
|
475
|
+
path diffs identically to the original checkout, with no one-time migration
|
|
476
|
+
or forced re-push. It's stable only within a pinned engine version; a
|
|
477
|
+
version bump that changes how defs canonicalize will read as `changed` on the
|
|
478
|
+
next push, not as an error, since `create_workflow` is idempotent either way.
|
|
479
|
+
|
|
480
|
+
On a `401`, an OAuth credential is refreshed once and the request retried; an
|
|
481
|
+
agent (`olp_`) token has no refresh path, so a `401` is a hard "re-mint it"
|
|
482
|
+
error.
|
|
483
|
+
|
|
484
|
+
**Include and bodyFile limitations.** A def whose file uses `include:` is
|
|
485
|
+
refused (`uses include:, not hub-pushable`): the hub's `create_workflow`
|
|
486
|
+
parses the raw YAML without include expansion, and a re-serialized expanded
|
|
487
|
+
def isn't round-trippable. A def using `bodyFile:` is refused the same way
|
|
488
|
+
(`uses bodyFile:, not hub-pushable`) — there's no checkout `baseDir` to
|
|
489
|
+
resolve the external file against once the YAML leaves this machine. Inline
|
|
490
|
+
both before pushing.
|
|
491
|
+
|
|
492
|
+
Exact-match redirect URIs (no RFC 8252 variable-port allowance) and no
|
|
493
|
+
device-code grant remain recorded follow-ups on the service, not gaps in the
|
|
494
|
+
CLI.
|
|
495
|
+
|
|
36
496
|
## Hand-driven walkthrough
|
|
37
497
|
|
|
38
498
|
The [`examples/workflows`](../examples/workflows) folder has a workflow per
|
|
@@ -98,7 +558,14 @@ how fresh its claim is, for reclaiming a worker you know is dead without
|
|
|
98
558
|
waiting out the TTL. Reaping re-arms the task immediately, so **the run that
|
|
99
559
|
held the cleared lease can no longer commit** — its next `green`/`close` fails
|
|
100
560
|
with `run <id> no longer holds its lease (reaped or superseded)`, the same
|
|
101
|
-
error a normal TTL-expired reap produces.
|
|
561
|
+
error a normal TTL-expired reap produces. Each entry in `details` carries a
|
|
562
|
+
`reason` explaining why that lease was cleared: `heartbeat-lost` (no beat within
|
|
563
|
+
the reap TTL — the job went silent), `max-lease-exceeded` (a configured
|
|
564
|
+
`maxLeaseMs`/`maxLease:` cap expired a still-beating lease — only ever seen when
|
|
565
|
+
a cap is set), `run-missing` / `run-closed` (the owning run is gone or already
|
|
566
|
+
closed), or `forced` (a `--now` stand-down cleared a lease that was still fresh
|
|
567
|
+
under the real TTL — reported instead of a misleading liveness reason).
|
|
568
|
+
`owenloop runs <wf> [--open]` and
|
|
102
569
|
`status <wf>`'s `inFlight` array are the read side: `runs` lists every run
|
|
103
570
|
this instance has ever had (with `--open` filtering to still-open ones, each
|
|
104
571
|
joined with its owning task's `claimedAt`/`heartbeatAt`/`attempts`), while
|
|
@@ -138,6 +605,7 @@ a worker needs nothing else to do the work:
|
|
|
138
605
|
```jsonc
|
|
139
606
|
{
|
|
140
607
|
"run": "r_…", // job id — pass it back to green/emit/seal/close
|
|
608
|
+
"workflow": "wf_…", // the instance this order belongs to — see deep tick below
|
|
141
609
|
"step": "builder", // which step this job is for
|
|
142
610
|
"key": "", // map jobs carry the element key + index
|
|
143
611
|
"inputs": ["plan"],
|
|
@@ -157,6 +625,44 @@ A worker reads `prompt` + `consumes` + `owes`, does the work, reports with
|
|
|
157
625
|
counts in `owes[]` let a workflow escalate on its own — e.g. switch to a
|
|
158
626
|
stronger model after two rejections — before the engine stalls the step.
|
|
159
627
|
|
|
628
|
+
**Deep tick and `order.workflow`.** `tick <wf>` is **deep by default**: it ticks
|
|
629
|
+
`<wf>` and then descends into every live `calls:` child, folding their orders
|
|
630
|
+
into the one result. So an order in the list may belong to a child instance,
|
|
631
|
+
not `<wf>` — always dispatch and commit (`green`/`emit`/`seal`/`close`) against
|
|
632
|
+
`order.workflow`, not the id you passed to `tick`. `--shallow` ticks only the
|
|
633
|
+
one instance (every order then carries `<wf>` itself); use it for a deliberate
|
|
634
|
+
single-instance drive. `reaped` sums across the tree and `dueAt` (when present)
|
|
635
|
+
is the earliest wake across all levels. A folded deferral in the deep result
|
|
636
|
+
carries its own `workflow` (absent = the root you ticked, present = a
|
|
637
|
+
descendant).
|
|
638
|
+
|
|
639
|
+
**Label routing (`--label`).** `tick <wf> --label <l>` (repeatable) filters
|
|
640
|
+
which steps this caller claims, but only steps that carry their own `labels:`
|
|
641
|
+
are ever excluded: a step is deferred with reason `label-mismatch` and left for
|
|
642
|
+
another caller **only** when its `labels:` are non-empty and share no value with
|
|
643
|
+
the filter you pass. A step with no `labels:` is claimed by every caller,
|
|
644
|
+
filtered or not, and a tick that passes no `--label` claims every eligible step
|
|
645
|
+
regardless of labels. This is **routing, not
|
|
646
|
+
authorization**: any caller that can reach the database can tick without a
|
|
647
|
+
filter and claim anything, so labels split work across cooperating
|
|
648
|
+
orchestrators, they never enforce a boundary. See
|
|
649
|
+
[`labels:`](authoring.md#labels--routing-a-step-to-a-particular-tick-caller) in
|
|
650
|
+
the authoring guide for the step-side declaration and the starvation hazard to
|
|
651
|
+
watch for.
|
|
652
|
+
|
|
653
|
+
**Child stalls on `status`.** `status <wf>`'s `calls:`-debt entries carry a
|
|
654
|
+
`child: { workflow, def, done, stalled, debts }` summary once a child has been
|
|
655
|
+
spawned. `child.stalled: true` means the child (or a grandchild below it) has a
|
|
656
|
+
worker stuck at `maxAttempts` with no green outcome — the parent debt is blocked
|
|
657
|
+
on stuck child work. This lets a conductor spot a wedged child from the parent
|
|
658
|
+
`status` alone, without separately walking into the child's own `status`.
|
|
659
|
+
|
|
660
|
+
**`wait --until` is single-instance.** `wait <wf> --until eligible|done` polls
|
|
661
|
+
`status <wf>`, which is that one instance's derived view — it does **not** see a
|
|
662
|
+
child's `eligible` orders or wait on child completion. To block on a tree, wait
|
|
663
|
+
on the instance that actually owes the work (often a child), or poll deep
|
|
664
|
+
`tick`/`status` yourself.
|
|
665
|
+
|
|
160
666
|
## Instance pinning — editing a workflow definition mid-flight
|
|
161
667
|
|
|
162
668
|
`create` snapshots the fully-expanded definition (post `include:`/`calls:`) onto the
|