@hasna/loops 0.4.27 → 0.4.29
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +205 -1
- package/README.md +232 -49
- package/dist/api/index.d.ts +20 -19
- package/dist/api/index.js +7770 -1342
- package/dist/cli/index.js +2976 -966
- package/dist/cli/safe-error-context.d.ts +1 -0
- package/dist/daemon/daemon.d.ts +1 -0
- package/dist/daemon/index.js +1899 -499
- package/dist/generated/storage-kit/index.d.ts +1 -1
- package/dist/generated/storage-kit/mode.d.ts +6 -12
- package/dist/index.d.ts +3 -3
- package/dist/index.js +5319 -1096
- package/dist/lib/advancement.d.ts +52 -0
- package/dist/lib/agent-adapter.d.ts +20 -2
- package/dist/lib/auth/route-policy.d.ts +14 -0
- package/dist/lib/auth/tenant-auth.d.ts +38 -0
- package/dist/lib/cloud/mode.d.ts +2 -8
- package/dist/lib/cloud/storage.d.ts +1 -2
- package/dist/lib/cloud/transport.d.ts +4 -18
- package/dist/lib/errors.d.ts +43 -1
- package/dist/lib/executor.d.ts +9 -0
- package/dist/lib/format.d.ts +2 -2
- package/dist/lib/goal/runner.d.ts +36 -3
- package/dist/lib/health.d.ts +1 -1
- package/dist/lib/labels.d.ts +4 -0
- package/dist/lib/loop-status.d.ts +4 -0
- package/dist/lib/migration.d.ts +70 -17
- package/dist/lib/mode.d.ts +2 -5
- package/dist/lib/mode.js +28 -37
- package/dist/lib/route/fields.d.ts +8 -0
- package/dist/lib/route/index.d.ts +2 -2
- package/dist/lib/route/throttle.d.ts +7 -0
- package/dist/lib/route/types.d.ts +3 -0
- package/dist/lib/run-completion.d.ts +18 -0
- package/dist/lib/scheduler.d.ts +5 -11
- package/dist/lib/storage/contract.d.ts +15 -1
- package/dist/lib/storage/index.d.ts +1 -1
- package/dist/lib/storage/index.js +4083 -486
- package/dist/lib/storage/pg-executor.d.ts +10 -5
- package/dist/lib/storage/postgres-loop-storage.d.ts +52 -26
- package/dist/lib/storage/postgres-schema.d.ts +8 -0
- package/dist/lib/storage/postgres-schema.js +1326 -1
- package/dist/lib/storage/postgres.d.ts +3 -1
- package/dist/lib/storage/postgres.js +1356 -27
- package/dist/lib/storage/provider-credentials.d.ts +84 -0
- package/dist/lib/storage/shared-database-transfer.d.ts +136 -0
- package/dist/lib/storage/sqlite.d.ts +15 -2
- package/dist/lib/storage/sqlite.js +1379 -217
- package/dist/lib/storage/tenant-backfill-s3.d.ts +53 -0
- package/dist/lib/storage/tenant-backfill.d.ts +40 -0
- package/dist/lib/store/index.d.ts +14 -8
- package/dist/lib/store.d.ts +129 -7
- package/dist/lib/store.js +1352 -218
- package/dist/lib/templates.d.ts +11 -0
- package/dist/lib/workflow-events.d.ts +6 -0
- package/dist/lib/workflow-provenance.d.ts +8 -0
- package/dist/lib/workflow-runner.d.ts +52 -4
- package/dist/mcp/index.js +2074 -657
- package/dist/runner/index.d.ts +20 -2
- package/dist/runner/index.js +2146 -183
- package/dist/sdk/http.d.ts +364 -4
- package/dist/sdk/http.js +379 -1
- package/dist/sdk/index.d.ts +7 -2
- package/dist/sdk/index.js +2341 -742
- package/dist/serve/index.d.ts +14 -0
- package/dist/serve/index.js +13269 -1830
- package/dist/types.d.ts +75 -3
- package/docs/AUTOMATION_RUNTIME_DESIGN.md +32 -32
- package/docs/CUTOVER-RUNBOOK.md +158 -31
- package/docs/DEPLOYMENT_MODES.md +84 -56
- package/docs/RUNTIME_BOUNDARY.md +26 -26
- package/docs/SHARED-DATABASE-TRANSFER.md +95 -0
- package/docs/SHARED_KIT_EXTRACTION_INVENTORY.md +631 -0
- package/docs/TRANSCRIPT_LOOP_PATTERNS.md +3 -3
- package/docs/UNIFIED_PRODUCT_CONTRACT.md +365 -0
- package/docs/USAGE.md +150 -56
- package/docs/workflows/transcript-feedback-to-loops.json +2 -2
- package/package.json +8 -4
- package/dist/lib/storage/pg-runner-claim.d.ts +0 -40
package/docs/DEPLOYMENT_MODES.md
CHANGED
|
@@ -1,11 +1,11 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Loops Deployment Modes
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Loops supports one active source of truth at a time. The public package
|
|
4
4
|
defines the mode vocabulary, local cache behavior, API shape, and runner
|
|
5
5
|
contract. Hosted multi-tenant operation is implemented outside this public
|
|
6
6
|
package.
|
|
7
7
|
|
|
8
|
-
Deployment modes describe **where the
|
|
8
|
+
Deployment modes describe **where the Loops runtime stores and executes**
|
|
9
9
|
loops and workflows (`local` SQLite, `self_hosted` control plane, or `cloud`
|
|
10
10
|
contract). They are runtime placement concerns, not the automation product
|
|
11
11
|
surface — specs, queues, approvals, and audit for product automations remain in
|
|
@@ -17,7 +17,7 @@ surface — specs, queues, approvals, and audit for product automations remain i
|
|
|
17
17
|
| Mode | Source of truth | Local storage role | Executor |
|
|
18
18
|
| --- | --- | --- | --- |
|
|
19
19
|
| `local` | SQLite in `LOOPS_DATA_DIR` | Authoritative | `loops-daemon` |
|
|
20
|
-
| `self_hosted` | Hasna-owned AWS/RDS control plane served by `loops-serve
|
|
20
|
+
| `self_hosted` | Hasna-owned AWS/RDS control plane served by `loops-serve` using the embeddable `loops-api` contract | Cache and offline spool | `loops-runner` foundation |
|
|
21
21
|
| `cloud` | A configured hosted control plane contract | Cache and offline spool | `loops-runner` foundation |
|
|
22
22
|
|
|
23
23
|
`local` remains the default. It must keep working without network access,
|
|
@@ -29,29 +29,28 @@ the Postgres storage adapter, migrations, HTTP SDK, and runner contract for this
|
|
|
29
29
|
mode.
|
|
30
30
|
|
|
31
31
|
`cloud` is the hosted control-plane contract. The public package exposes the
|
|
32
|
-
client and runner contract,
|
|
33
|
-
infrastructure stay outside this package. The public package must not depend on
|
|
32
|
+
client and runner contract, tenant authentication, and tenant isolation; account
|
|
33
|
+
provisioning and hosted infrastructure stay outside this package. The public package must not depend on
|
|
34
34
|
private hosted packages or resource names. This release exposes status
|
|
35
35
|
surfaces only.
|
|
36
36
|
|
|
37
37
|
## Mode Resolution
|
|
38
38
|
|
|
39
|
-
`
|
|
40
|
-
`cloud`.
|
|
39
|
+
`HASNA_LOOPS_STORAGE_MODE` may be set to `local`, `self_hosted`, or
|
|
40
|
+
`cloud`. Other spellings and legacy mode names are rejected.
|
|
41
41
|
|
|
42
|
-
When no explicit mode is set,
|
|
42
|
+
When no explicit mode is set, Loops resolves the mode from configuration:
|
|
43
43
|
|
|
44
|
-
1. `
|
|
45
|
-
2. `LOOPS_API_URL`, `HASNA_LOOPS_API_URL`, `LOOPS_DATABASE_URL`, or
|
|
44
|
+
1. `HASNA_LOOPS_API_URL` or
|
|
46
45
|
`HASNA_LOOPS_DATABASE_URL` selects `self_hosted`.
|
|
47
|
-
|
|
46
|
+
2. Otherwise Loops uses `local`.
|
|
48
47
|
|
|
49
|
-
|
|
50
|
-
`cloud`
|
|
48
|
+
Both non-local modes use the canonical `HASNA_LOOPS_API_URL`; an explicit
|
|
49
|
+
`HASNA_LOOPS_STORAGE_MODE=cloud` distinguishes hosted cloud from self-hosted.
|
|
51
50
|
|
|
52
51
|
Tokens are represented only as presence signals in status output. Self-hosted
|
|
53
|
-
status uses `
|
|
54
|
-
`
|
|
52
|
+
status uses `HASNA_LOOPS_API_KEY`. Cloud status uses
|
|
53
|
+
`HASNA_LOOPS_API_KEY`. URL credentials, query
|
|
55
54
|
strings, and fragments are not returned in status output.
|
|
56
55
|
|
|
57
56
|
## Commands
|
|
@@ -63,12 +62,12 @@ loops self-hosted status
|
|
|
63
62
|
loops self-hosted migrate --dry-run
|
|
64
63
|
loops self-hosted push --dry-run
|
|
65
64
|
loops self-hosted pull --dry-run
|
|
66
|
-
loops self-hosted runner-register --runner-id <id> --machine-id <machine>
|
|
67
|
-
loops self-hosted runner-register --runner-id <id> --machine-id <machine> --apply
|
|
68
65
|
loops cloud status
|
|
69
66
|
loops-api status
|
|
70
67
|
loops-serve version
|
|
71
|
-
|
|
68
|
+
HASNA_LOOPS_MIGRATOR_DATABASE_URL=... loops-serve migrate --dry-run
|
|
69
|
+
loops-serve db-credentials reconcile
|
|
70
|
+
HASNA_LOOPS_DATABASE_URL=... HASNA_LOOPS_AUTH_DATABASE_URL=... loops-serve serve
|
|
72
71
|
loops-runner status
|
|
73
72
|
loops export --file ./loops-export.json --dry-run
|
|
74
73
|
loops export --file ./loops-export.json
|
|
@@ -92,7 +91,7 @@ JSON uses these field names:
|
|
|
92
91
|
- `localStore.role`: `authoritative` in local mode, `cache_and_spool` in
|
|
93
92
|
non-local modes.
|
|
94
93
|
- `controlPlane.configured`: true only when the current mode has enough
|
|
95
|
-
configuration to be usable. Cloud requires both
|
|
94
|
+
configuration to be usable. Cloud requires both the canonical API URL and a token
|
|
96
95
|
presence signal.
|
|
97
96
|
- `controlPlane.apiUrl`: a display-safe URL without credentials, query string,
|
|
98
97
|
or fragment.
|
|
@@ -101,10 +100,11 @@ JSON uses these field names:
|
|
|
101
100
|
non-local modes it is a cache, offline spool, and audit copy.
|
|
102
101
|
- `schedulerState.remoteStore`: names the non-local scheduler contract:
|
|
103
102
|
`api_control_plane_contract`, `postgres_contract`,
|
|
104
|
-
`hosted_control_plane_contract`, `unconfigured`, or `none`.
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
storage adapter for normal control-plane CRUD
|
|
103
|
+
`hosted_control_plane_contract`, `unconfigured`, or `none`. The standalone
|
|
104
|
+
CLI never mutates Postgres directly; self-hosted apply goes through the
|
|
105
|
+
configured control-plane API import contract. `loops-serve` itself wires the
|
|
106
|
+
Postgres storage adapter for normal control-plane CRUD, id-preserving import,
|
|
107
|
+
and runner protocol routes.
|
|
108
108
|
- `schedulerState.remoteStore.objectArtifacts`: `object_store_contract` means
|
|
109
109
|
remote artifact/object storage is a control-plane contract. The public package
|
|
110
110
|
does not create or mutate S3 buckets, AWS resources, or hosted credentials.
|
|
@@ -118,22 +118,48 @@ JSON uses these field names:
|
|
|
118
118
|
package. It reads and writes Postgres directly, serves open foundation probes
|
|
119
119
|
(`GET /health`, `/ready`, `/version`, `/openapi.json`), gates `/v1` loop/run and
|
|
120
120
|
runner-protocol routes with API-key auth on non-local binds, and applies the
|
|
121
|
-
Postgres migrations
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
|
|
127
|
-
|
|
128
|
-
|
|
121
|
+
Postgres migrations, including the tenant-bound `api_keys` table, with the
|
|
122
|
+
prepare/backfill/enforce `loops-serve migrate` sequence.
|
|
123
|
+
|
|
124
|
+
The service requires separate database logins: `HASNA_LOOPS_DATABASE_URL` is
|
|
125
|
+
the tenant-scoped runtime role, while `HASNA_LOOPS_AUTH_DATABASE_URL` is the
|
|
126
|
+
authenticator-only role that can verify keys and append authentication audits.
|
|
127
|
+
`HASNA_LOOPS_MIGRATOR_DATABASE_URL` is an offline schema-administrator login;
|
|
128
|
+
tenant enforcement normalizes cluster role attributes and therefore requires
|
|
129
|
+
the PostgreSQL privilege to alter role security attributes.
|
|
130
|
+
|
|
131
|
+
The in-cluster credential reconciler is `loops-serve db-credentials reconcile`.
|
|
132
|
+
It is source-only infrastructure glue for self-hosted deployments that use an
|
|
133
|
+
RDS-managed master secret and separate app DSN secrets. The command must run
|
|
134
|
+
with the ECS task role (`AWS_CONTAINER_CREDENTIALS_RELATIVE_URI`) and AWS
|
|
135
|
+
Secrets Manager access to exactly four distinct same-region secret ARNs. It
|
|
136
|
+
rejects static AWS credentials, profiles, web-identity inputs, full credential
|
|
137
|
+
URIs, malformed secret ARNs, duplicate app secrets, endpoint mismatches, and
|
|
138
|
+
master-secret JSON that does not exactly match the expected RDS instance id,
|
|
139
|
+
endpoint, port, database, and master username.
|
|
140
|
+
|
|
141
|
+
The reconciler never prints or stores raw passwords outside Secrets Manager and
|
|
142
|
+
Postgres. For every app secret it writes an `AWSPENDING` DSN, changes that
|
|
143
|
+
login password inside a database transaction, tests a fresh verify-full
|
|
144
|
+
connection, then promotes the pending version to `AWSCURRENT`. If the current
|
|
145
|
+
secret/password pair is already valid, it leaves the secret untouched. Before
|
|
146
|
+
migration `0010_tenant_enforce`, runtime and authenticator logins are kept
|
|
147
|
+
detached from service roles; after `0010`, each is attached only to its exact
|
|
148
|
+
matching NOLOGIN role.
|
|
149
|
+
|
|
150
|
+
`loops-api` is the embeddable API contract in the same public package. It is not
|
|
151
|
+
a separate service because self-hosted users and the hosted service must share
|
|
152
|
+
the same public contract. `loops-serve` is the only shipped Postgres-backed
|
|
153
|
+
self-hosted host.
|
|
129
154
|
|
|
130
155
|
`loops-runner` is the process that connects a machine to a non-local control
|
|
131
156
|
plane. The current public package supports a bounded one-shot protocol:
|
|
132
|
-
|
|
133
|
-
`loops-runner run-once` execution for
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
157
|
+
claim polling, claim-token fenced lease heartbeat/finalization, and
|
|
158
|
+
`loops-runner run-once` execution for command, agent, and workflow targets.
|
|
159
|
+
Workflow execution uses runner-scoped workflow and goal APIs behind the claimed
|
|
160
|
+
run lease. Durable machine registration, full fleet daemon mode, and
|
|
161
|
+
always-on fleet observability still need follow-up releases before
|
|
162
|
+
`loops-runner` is advertised as a complete always-on worker.
|
|
137
163
|
|
|
138
164
|
## Migration And Sync
|
|
139
165
|
|
|
@@ -159,6 +185,12 @@ refuses to write a no-loss bundle unless the operator explicitly uses
|
|
|
159
185
|
requires `--apply`; existing rows with the same id are updated only with
|
|
160
186
|
`--replace`. The CLI creates a local SQLite backup before a safe apply.
|
|
161
187
|
|
|
188
|
+
`loops self-hosted push` applies an additional self-hosted safety rule. Imported
|
|
189
|
+
workflow definitions are archived, and imported loops are paused with
|
|
190
|
+
`nextRunAt`/`retryScheduledFor` cleared. That safety normalization can re-archive
|
|
191
|
+
or re-pause existing same-id rows even when `--replace` is not supplied; explicit
|
|
192
|
+
preserve flags are reserved for deliberate activation.
|
|
193
|
+
|
|
162
194
|
No-loss validation blocks unsupported or live state instead of silently
|
|
163
195
|
dropping it. The current migration bundle does not preserve workflow invocation
|
|
164
196
|
rows, workflow work items, workflow run/step/event history, or goal run history.
|
|
@@ -167,37 +199,34 @@ later release adds full table-preserving migration. Active daemon leases,
|
|
|
167
199
|
running loop runs, running workflow runs/steps, and leased work items also
|
|
168
200
|
block migration; finish or stop that work first.
|
|
169
201
|
|
|
170
|
-
Self-hosted sync commands
|
|
202
|
+
Self-hosted sync commands use the control-plane API:
|
|
171
203
|
|
|
172
204
|
```bash
|
|
173
205
|
loops self-hosted migrate --dry-run
|
|
174
206
|
loops self-hosted push --dry-run
|
|
207
|
+
loops self-hosted push --apply
|
|
175
208
|
loops self-hosted pull --dry-run
|
|
176
209
|
```
|
|
177
210
|
|
|
178
|
-
They inspect local state,
|
|
179
|
-
rows that would
|
|
180
|
-
|
|
181
|
-
|
|
182
|
-
|
|
183
|
-
|
|
184
|
-
|
|
211
|
+
They inspect local state, inspect `HASNA_LOOPS_API_URL` when configured, and
|
|
212
|
+
report the rows that would move. `loops self-hosted push --apply` sends the
|
|
213
|
+
id-preserving workflow and loop import bundle to `/v1/import`; imported
|
|
214
|
+
workflows are archived and imported loops are paused with run pointers cleared.
|
|
215
|
+
Local SQLite remains authoritative until an operator applies the import and
|
|
216
|
+
records the rollout evidence; in non-local modes it may remain a cache, offline
|
|
217
|
+
spool, and audit copy.
|
|
185
218
|
|
|
186
|
-
`
|
|
219
|
+
`HASNA_LOOPS_DATABASE_URL` selects the self-hosted
|
|
187
220
|
Postgres scheduler-state contract and is required by `loops-serve`. It does not
|
|
188
221
|
make the standalone `loops` CLI mutate a remote database by itself. Remote
|
|
189
222
|
execution still flows through a configured control-plane API and runner
|
|
190
|
-
protocol. `loops-runner` needs `
|
|
223
|
+
protocol. `loops-runner` needs `HASNA_LOOPS_API_URL` to claim
|
|
191
224
|
work; a database URL alone is migration/readiness configuration.
|
|
192
225
|
|
|
193
|
-
`loops self-hosted runner-register` is also preview-only unless `--apply` is
|
|
194
|
-
present. The dry run prints the runner id, machine id, labels, and
|
|
195
|
-
capabilities that would be posted, without exposing tokens.
|
|
196
|
-
|
|
197
226
|
Cross-machine rollout evidence should record: machine id/hostname, package
|
|
198
227
|
version, git/build version, command run, dry-run or apply mode, redacted API
|
|
199
228
|
URL, source and target store ids, backup path, bundle hash, schema version,
|
|
200
|
-
row counts, conflicts, blocked rows, runner
|
|
229
|
+
row counts, conflicts, blocked rows, runner machine record id, daemon/runner
|
|
201
230
|
status, and timestamp.
|
|
202
231
|
|
|
203
232
|
## Machine Placement
|
|
@@ -215,7 +244,7 @@ distinguishable from intentional fan-out.
|
|
|
215
244
|
This is separate from existing local OpenMachines dispatch. In `local` mode,
|
|
216
245
|
`loops-daemon` can still dispatch a loop target to a configured remote machine
|
|
217
246
|
through the existing OpenMachines transport. In `self_hosted` or `cloud`,
|
|
218
|
-
machine execution is runner-pull:
|
|
247
|
+
machine execution is runner-pull: `loops-runner` claims work from
|
|
219
248
|
the control plane. Operators should not treat local remote dispatch as cloud
|
|
220
249
|
mode.
|
|
221
250
|
|
|
@@ -225,9 +254,8 @@ Non-local execution needs these follow-up releases before it is complete:
|
|
|
225
254
|
|
|
226
255
|
- Long-running runner daemon mode with backoff, fleet observability, and
|
|
227
256
|
durable machine registration records.
|
|
228
|
-
-
|
|
229
|
-
|
|
230
|
-
definitions, run history, workflow history, work items, goals, and audit rows.
|
|
257
|
+
- Id-preserving self-hosted import coverage for run history, workflow history,
|
|
258
|
+
work items, goals, and audit rows.
|
|
231
259
|
- Hosted product integration outside the public package.
|
|
232
260
|
|
|
233
261
|
## Public Package Boundary
|
package/docs/RUNTIME_BOUNDARY.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
#
|
|
1
|
+
# Loops Runtime Boundary
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
Loops is the **runtime, scheduler, and workflow engine** for automations.
|
|
4
4
|
It executes work that external systems have already materialized or explicitly
|
|
5
5
|
handed off. It is **not** the automation domain model: specs, triggers, queue
|
|
6
6
|
ownership, approvals, DLQ/replay, idempotency, and audit evidence live in
|
|
@@ -14,15 +14,15 @@ deployment-mode vocabulary (local vs self-hosted vs cloud), see
|
|
|
14
14
|
|
|
15
15
|
## Ownership Split
|
|
16
16
|
|
|
17
|
-
| Concern | Owner |
|
|
17
|
+
| Concern | Owner | Loops role |
|
|
18
18
|
| --- | --- | --- |
|
|
19
|
-
| Automation specs and trigger materialization | `@hasna/automations` | None — do not store specs in
|
|
19
|
+
| Automation specs and trigger materialization | `@hasna/automations` | None — do not store specs in Loops |
|
|
20
20
|
| Product automation action queues, leases, DLQ, replay, idempotency, approvals, audit | `@hasna/automations` | Execute claimed actions; complete/fail by action id + runner id |
|
|
21
21
|
| Action compilation and action-target rendering | `@hasna/actions` | Accept planned upsert handoff; do not write SQLite rows directly |
|
|
22
|
-
| Scheduler, daemon, loop/workflow storage |
|
|
23
|
-
| Workflow invocation, admission, execution, run manifests |
|
|
24
|
-
| Provider routing, worktrees, run artifacts |
|
|
25
|
-
| Todos-task route drains (`auto:route`, `route_enabled`, `automation.allowed`) |
|
|
22
|
+
| Scheduler, daemon, loop/workflow storage | Loops | Authoritative in `local` mode |
|
|
23
|
+
| Workflow invocation, admission, execution, run manifests | Loops | After explicit handoff or todos-task route admission |
|
|
24
|
+
| Provider routing, worktrees, run artifacts | Loops | Bounded by route and template policy |
|
|
25
|
+
| Todos-task route drains (`auto:route`, `route_enabled`, `automation.allowed`) | Loops-native | Opt-in task/event routing — **not** OpenAutomations queue replacement |
|
|
26
26
|
|
|
27
27
|
The SDK exposes the canonical boundary strings via `openAutomationsRuntimeBinding()`:
|
|
28
28
|
|
|
@@ -41,37 +41,37 @@ console.log(binding.nonGoals);
|
|
|
41
41
|
|
|
42
42
|
- OpenAutomations owns automation specs, run materialization, queue state, DLQ,
|
|
43
43
|
replay, idempotency, and approvals.
|
|
44
|
-
-
|
|
44
|
+
- Loops may execute claimed actions through explicit command or SDK handoff
|
|
45
45
|
only.
|
|
46
|
-
-
|
|
46
|
+
- Loops may consume exported event envelopes only through explicit
|
|
47
47
|
`loops routes create` commands.
|
|
48
48
|
- Workers must complete or fail actions by action id and runner id so
|
|
49
49
|
OpenAutomations can enforce queue leases.
|
|
50
50
|
|
|
51
51
|
**Non-goals (summary):**
|
|
52
52
|
|
|
53
|
-
-
|
|
54
|
-
-
|
|
55
|
-
-
|
|
53
|
+
- Loops must not become the OpenAutomations product surface.
|
|
54
|
+
- Loops must not store automation specs or replace the OpenAutomations queue.
|
|
55
|
+
- Loops must not infer automation trigger semantics from event transport
|
|
56
56
|
alone.
|
|
57
57
|
|
|
58
58
|
## External Compiler Handoff Paths
|
|
59
59
|
|
|
60
60
|
External compilers (`@hasna/automations`, `@hasna/actions`, event routers)
|
|
61
|
-
materialize automation intent in their own stores.
|
|
61
|
+
materialize automation intent in their own stores. Loops receives **rendered
|
|
62
62
|
work** through one of the paths below. Do not bypass these boundaries by writing
|
|
63
|
-
|
|
63
|
+
Loops SQLite rows from another package.
|
|
64
64
|
|
|
65
65
|
### 1. OpenAutomations claim-queue (implemented)
|
|
66
66
|
|
|
67
|
-
OpenAutomations owns the action queue.
|
|
67
|
+
OpenAutomations owns the action queue. A Loops worker claims work, runs the
|
|
68
68
|
rendered target, and reports completion back to OpenAutomations.
|
|
69
69
|
|
|
70
70
|
```bash
|
|
71
71
|
# Worker claims from the OpenAutomations queue
|
|
72
72
|
automations queue claim --runner open-loops:<worker-id>
|
|
73
73
|
|
|
74
|
-
# After
|
|
74
|
+
# After Loops executes the handed-off command/workflow:
|
|
75
75
|
automations queue complete <action-id> --runner open-loops:<worker-id>
|
|
76
76
|
# or on failure:
|
|
77
77
|
automations queue fail <action-id> --runner open-loops:<worker-id> \
|
|
@@ -131,7 +131,7 @@ loops workflows upsert-one-shot ./workflow.json \
|
|
|
131
131
|
--json
|
|
132
132
|
```
|
|
133
133
|
|
|
134
|
-
|
|
134
|
+
Loops returns durable refs (`workflowId`, `loopId`, `runId`, `manifestPath`)
|
|
135
135
|
so `@hasna/actions` can inspect, cancel, or replay without querying SQLite.
|
|
136
136
|
Full semantics (idempotency on `idempotencyKey` + `specHash`, redaction,
|
|
137
137
|
strict-mode policy) are specified in the design doc — not duplicated here.
|
|
@@ -139,7 +139,7 @@ strict-mode policy) are specified in the design doc — not duplicated here.
|
|
|
139
139
|
### 3. Event envelope handoff (implemented, explicit only)
|
|
140
140
|
|
|
141
141
|
For workflows that should run from a normalized event envelope — **without**
|
|
142
|
-
OpenAutomations materializing automation specs inside
|
|
142
|
+
OpenAutomations materializing automation specs inside Loops — pipe an
|
|
143
143
|
exported envelope into the generic route:
|
|
144
144
|
|
|
145
145
|
```bash
|
|
@@ -147,9 +147,9 @@ automations --json webhooks event <route> --body-json '<json>' \
|
|
|
147
147
|
| loops --json routes create generic
|
|
148
148
|
```
|
|
149
149
|
|
|
150
|
-
This is **not** automation materialization in
|
|
150
|
+
This is **not** automation materialization in Loops. OpenAutomations still
|
|
151
151
|
owns deterministic automation specs, webhook normalization, queue state,
|
|
152
|
-
approvals, DLQ, and replay.
|
|
152
|
+
approvals, DLQ, and replay. Loops owns agent workflow invocation after the
|
|
153
153
|
operator routes the envelope to `loops routes create generic`.
|
|
154
154
|
|
|
155
155
|
```ts
|
|
@@ -158,9 +158,9 @@ console.log(eventHandoff.handlerCommand); // "loops routes create generic"
|
|
|
158
158
|
console.log(eventHandoff.boundary);
|
|
159
159
|
```
|
|
160
160
|
|
|
161
|
-
##
|
|
161
|
+
## Loops-Native Path: Todos-Task Routes
|
|
162
162
|
|
|
163
|
-
Todos-task routing is
|
|
163
|
+
Todos-task routing is a **Loops-native** admission path. It is separate from
|
|
164
164
|
the OpenAutomations product queue:
|
|
165
165
|
|
|
166
166
|
```bash
|
|
@@ -170,7 +170,7 @@ cat task-created-event.json | loops routes create todos-task \
|
|
|
170
170
|
```
|
|
171
171
|
|
|
172
172
|
Tasks must opt in (`auto:route`, `route_enabled=true`, or
|
|
173
|
-
`automation.allowed=true`).
|
|
173
|
+
`automation.allowed=true`). Loops admits deduped one-shot workflow loops,
|
|
174
174
|
drains them on a schedule, and writes run manifests under
|
|
175
175
|
`.hasna/loops/runs/<project-slug>/<subject-key>/<run-id>/`.
|
|
176
176
|
|
|
@@ -186,10 +186,10 @@ provider rules, and drain examples.
|
|
|
186
186
|
|
|
187
187
|
Do **not**:
|
|
188
188
|
|
|
189
|
-
- Store automation specs in
|
|
189
|
+
- Store automation specs in Loops loop/workflow rows.
|
|
190
190
|
- Infer automation triggers from event transport alone.
|
|
191
191
|
- Replace the OpenAutomations queue with loop/workflow rows.
|
|
192
|
-
- Write
|
|
192
|
+
- Write Loops SQLite directly from `@hasna/actions` or `@hasna/automations`.
|
|
193
193
|
- Dispatch or paste task prompts into tmux panes from route drains (use headless
|
|
194
194
|
workflow templates instead).
|
|
195
195
|
|
|
@@ -0,0 +1,95 @@
|
|
|
1
|
+
# Shared Apps To Dedicated Loops Database Transfer
|
|
2
|
+
|
|
3
|
+
Status: source-side tooling and protected workflow source only. Do not run this
|
|
4
|
+
against live AWS or any live database without an approved maintenance window,
|
|
5
|
+
fresh recovery evidence, and the protected GitHub environment approval.
|
|
6
|
+
|
|
7
|
+
This transfer is intentionally selective. The source is the shared RDS database
|
|
8
|
+
`apps`; the target is the dedicated Loops database `loops`. Never snapshot
|
|
9
|
+
restore the shared cluster or shared database into the target.
|
|
10
|
+
|
|
11
|
+
## Fixed Command
|
|
12
|
+
|
|
13
|
+
The only production command for this lane is the no-option service command:
|
|
14
|
+
|
|
15
|
+
```bash
|
|
16
|
+
bun dist/serve/index.js shared-to-dedicated-transfer
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
Operators do not pass DSNs, table names, shell snippets, SQL files, or archive
|
|
20
|
+
paths. The command reads only these ECS task secret environment variables:
|
|
21
|
+
|
|
22
|
+
```text
|
|
23
|
+
HASNA_LOOPS_TRANSFER_SOURCE_DATABASE_URL
|
|
24
|
+
HASNA_LOOPS_TRANSFER_TARGET_DATABASE_URL
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
The source DSN must point at database `apps`. The target DSN must point at
|
|
28
|
+
database `loops`. The command converts both DSNs into a private `pg_service.conf`
|
|
29
|
+
inside a mode `0700` ephemeral directory so PostgreSQL client argv contains only
|
|
30
|
+
`service=openloops_transfer_source` and `service=openloops_transfer_target`.
|
|
31
|
+
The archive directory is removed in `finally` after success or failure.
|
|
32
|
+
|
|
33
|
+
## Sequence
|
|
34
|
+
|
|
35
|
+
1. Verify PostgreSQL 16 client binaries: `pg_dump`, `pg_restore`, and `psql`.
|
|
36
|
+
2. Verify source migration ledger checksums for Loops migrations
|
|
37
|
+
`0001_core_runtime` through `0007_work_item_gate_deaths`.
|
|
38
|
+
3. Verify source quiescence: no active loop runs, workflow runs, workflow step
|
|
39
|
+
runs, runner leases, or leased/running work items.
|
|
40
|
+
4. Initialize the dedicated target schema through migration
|
|
41
|
+
`0007_work_item_gate_deaths` using the normal checksum ledger.
|
|
42
|
+
5. Verify the target ledger is exactly migrations `0001` through `0007` and no
|
|
43
|
+
later row is present.
|
|
44
|
+
6. Use `pg_dump --format=custom --data-only --no-owner --no-privileges` for the
|
|
45
|
+
exact Loops table allowlist:
|
|
46
|
+
|
|
47
|
+
```text
|
|
48
|
+
loops
|
|
49
|
+
workflow_specs
|
|
50
|
+
runner_machines
|
|
51
|
+
goals
|
|
52
|
+
loop_runs
|
|
53
|
+
workflow_invocations
|
|
54
|
+
workflow_runs
|
|
55
|
+
workflow_work_items
|
|
56
|
+
workflow_step_runs
|
|
57
|
+
workflow_events
|
|
58
|
+
goal_plan_nodes
|
|
59
|
+
goal_runs
|
|
60
|
+
runner_leases
|
|
61
|
+
audit_events
|
|
62
|
+
run_receipts
|
|
63
|
+
daemon_lease
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
7. Use `pg_restore --data-only --single-transaction --exit-on-error` into the
|
|
67
|
+
dedicated target.
|
|
68
|
+
8. Apply migration `0008_tenant_prepare` on the target. This creates the
|
|
69
|
+
tenant-prep tables and `api_keys` table needed for the next copy.
|
|
70
|
+
9. Use filtered `COPY` for only:
|
|
71
|
+
|
|
72
|
+
```sql
|
|
73
|
+
SELECT kid, app, agent, scopes, token_hash, issued_at, expires_at, revoked_at,
|
|
74
|
+
revoked_reason, last_used_at, created_by, created_at
|
|
75
|
+
FROM public.api_keys
|
|
76
|
+
WHERE app = 'loops'
|
|
77
|
+
ORDER BY kid;
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
10. Verify source and target row counts, canonical SHA-256 row evidence,
|
|
81
|
+
target non-loop API key count, FK orphan checks, and unexpected database
|
|
82
|
+
objects across schemas, tables, functions, sequences, and views.
|
|
83
|
+
11. Stop after producing evidence. The next approved sequence is:
|
|
84
|
+
`loops-serve tenant-backfill-s3`, then
|
|
85
|
+
`loops-serve migrate --enforce-tenancy`.
|
|
86
|
+
|
|
87
|
+
## Protected Workflow
|
|
88
|
+
|
|
89
|
+
`.github/workflows/shared-database-transfer.yml` is a manual protected source
|
|
90
|
+
surface. It accepts only a confirmation string, assumes the environment's OIDC
|
|
91
|
+
role, and runs one ECS Fargate task with the fixed command above. Database URLs
|
|
92
|
+
must come from the ECS task definition secrets, not from GitHub inputs,
|
|
93
|
+
repository variables, workflow logs, or operator shell overrides.
|
|
94
|
+
|
|
95
|
+
The workflow has no RDS restore or snapshot API path.
|