@collective-intelligence/cli 0.0.0-stage → 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,345 @@
1
- # Temporary Holding Version
1
+ # Collective Intelligence CLI
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ The public `ci` binary manages Extension Package development and immutable
4
+ release operations. It talks to the Backend only through generated OpenAPI
5
+ operations; it does not import private Backend runtime code.
6
+
7
+ ## Sign In and Enroll a Local App
8
+
9
+ Interactive developers sign in once; pipelines keep using a token. Every
10
+ command accepts either credential.
11
+
12
+ ```sh
13
+ # Device login: copy this command from the Dashboard (Organization settings →
14
+ # CLI), approve the code in the Dashboard, then the CLI stores a 30-day CLI
15
+ # Session in ~/.config/ci/profiles.json (mode 0600).
16
+ ci login https://api.devpunks.example.com --organization <organization-id>
17
+ ci whoami
18
+ ci orgs list
19
+ ci orgs use <organization-id>
20
+
21
+ # Create or replace one Organization-scoped Public Application by FQN and
22
+ # write its connection values.
23
+ ci apps enroll \
24
+ --fqn acme.portal.local \
25
+ --redirect-uri http://localhost:3000/login/callback \
26
+ --scopes context:read \
27
+ --env-file .env.local
28
+ ci logout
29
+ ```
30
+
31
+ - `ci login <API URL>` resolves the API origin, issuer and JWKS URL from the
32
+ API host's `/.well-known/ci-configuration`. A Dashboard URL is rejected as
33
+ "not a CI API URL"; the Dashboard shows the exact command under Organization
34
+ settings → CLI. `--organization <id>` selects the Current Organization;
35
+ without it, a re-login keeps the Profile's previous Organization for the same
36
+ API origin. `--no-browser` only prints the URL.
37
+ - Resolution order for the backend URL, credential and Organization: flag
38
+ (`--backend-url`, `--organization`), then environment (`CI_BACKEND_URL`,
39
+ `CI_MANAGEMENT_TOKEN`), then the Profile (`--profile` or `CI_PROFILE` selects
40
+ one). A Profile never supplies a session for a different origin.
41
+ - `CI_MANAGEMENT_TOKEN` can be an Organization API key with the
42
+ `public_applications_manage` scope. The key fixes its Organization.
43
+ - `ci apps enroll` requires `--fqn`, `--redirect-uri` (repeatable) and
44
+ `--scopes`. Defaults: `--type public`, name = FQN, allowed origins = redirect
45
+ URI origins, audience = first redirect URI origin. Loopback redirect URIs
46
+ (`http://localhost`, `http://127.0.0.1`, `http://[::1]`) need an explicit port.
47
+ - Output keys: `CI_ISSUER`, `CI_AUDIENCE`, `CI_APPLICATION_ID`, `CI_CLIENT_ID`,
48
+ `CI_ORGANIZATION_ID`, `CI_JWKS_URL`, `CI_SCOPES`, plus `CI_CLIENT_SECRET` for
49
+ `--type confidential`. `--env-file` updates only these keys; without it the
50
+ CLI prints dotenv; `--json` prints one object; `--prefix` replaces `CI_`.
51
+ - A re-run keeps the Application Id and Client Id. It keeps an existing
52
+ `CI_CLIENT_SECRET` only when the same file's `CI_CLIENT_ID` and `CI_ISSUER`
53
+ match the enrolled application; otherwise it fails and asks for
54
+ `--rotate-secret`. It never rotates without `--rotate-secret`. It refuses
55
+ package-owned applications and disabled applications (pass `--enable`).
56
+ - `ci doctor` checks discovery, the JWKS signing key, Organization access and
57
+ registered redirect URIs; `ci apps enroll` runs the same checks before it
58
+ writes output. Each discovery and JWKS request fails after 30 seconds
59
+ without a complete response.
60
+ - Each request of `ci login`, `logout`, `whoami`, `orgs`, `doctor` and
61
+ `apps` also fails after 30 seconds without a complete response; a login
62
+ token poll gets its own 30 seconds. Ctrl-C cancels the request in flight.
63
+ - `ci apps list|get|rotate-secret|disable` manage existing applications.
64
+
65
+ ## Deploy an Exact Release
66
+
67
+ `ci deploy` builds and validates one declaration, resolves its exact immutable
68
+ release, publishes/promotes when needed, and installs or updates it for the
69
+ Organization. Use separate publisher and Organization management credentials:
70
+
71
+ ```sh
72
+ export CI_PUBLISHER_TOKEN="publisher-credential"
73
+ export CI_MANAGEMENT_TOKEN="organization-management-credential"
74
+
75
+ ci deploy \
76
+ --backend-url https://ci.example.com \
77
+ --organization <organization-id> \
78
+ --declaration ./src/package.ts \
79
+ --gateway-url com.acme.gateways.one=https://one.example.com \
80
+ --gateway-url com.acme.gateways.two=https://two.example.com \
81
+ --non-interactive --json
82
+ ```
83
+
84
+ Repeat `--gateway-url <GatewayFQN>=<HTTPS-origin>` for each independently
85
+ addressed declared Gateway. Bindings are deployment inputs, outside the release
86
+ contents. Duplicate, unknown and missing bindings reject clearly. A rerun can
87
+ omit bindings already saved for the installation; supplying a changed origin
88
+ reconciles that binding while reusing the others. Unchanged completed work is
89
+ skipped. A new version updates the existing installation. Backend strict
90
+ publication/version policy rejects same-version changed bytes and silent
91
+ downgrades; targeted package and rollback commands remain available separately.
92
+ Use `--accept-breaking-changes` only to explicitly acknowledge a breaking change
93
+ under the Organization's authority.
94
+
95
+ JSON mode emits newline-delimited records with `kind: progress`, `result`, or
96
+ `error`. An accepted operation is progress, not success. The command follows
97
+ resumable durable operation streams and exits successfully only after every
98
+ required operation succeeds. Failed, blocked or awaiting-continuation outcomes
99
+ exit nonzero with the operation ID and Backend-authored guidance. The default
100
+ overall deadline is 300,000 ms; `--timeout-ms <positive-integer>` changes it.
101
+ Timeout errors preserve the operation ID and last observed event sequence;
102
+ rerun the same intent or use `ci package operation reconnect` to follow it.
103
+ Deterministic request keys include exact release, Organization, endpoint and
104
+ breaking-change acknowledgement and connector setup intent. Broad publisher/management credentials
105
+ are sent only to CI, never to a Gateway endpoint.
106
+
107
+ ## Connector Setup
108
+
109
+ Pass `--setup-file <path>` to `ci deploy`, `ci package install`, or
110
+ `ci package update`. The JSON file stays separate from release bytes:
111
+
112
+ ```json
113
+ {
114
+ "connections": {
115
+ "crm": { "connectionId": "2a4cb90a-0e00-4384-8a78-d5279fbb3a77" },
116
+ "warehouse": {
117
+ "newConnection": { "name": "Warehouse production", "fields": {} }
118
+ }
119
+ },
120
+ "adoptions": {
121
+ "com.acme.sales.instances.customers": {
122
+ "instanceId": "42e62b11-1297-4ad3-ae87-19e6cc5b458d"
123
+ }
124
+ }
125
+ }
126
+ ```
127
+
128
+ Replace the UUIDs with existing authorized Organization resources and supply
129
+ required native credential fields. Connection setup keys match the declaration's
130
+ `setupKey`; adoption keys are concrete declaration FQNs. The backend validates
131
+ Organization/template/ownership compatibility and keeps credentials in native
132
+ connection custody. Invalid input errors omit setup values.
133
+
134
+ ```sh
135
+ ci deploy --backend-url https://ci.example.com --organization <organization-id> \
136
+ --declaration ./src/package.ts --setup-file ./setup.json --non-interactive --json
137
+
138
+ ci package install --backend-url https://ci.example.com --organization <organization-id> \
139
+ --package com.acme.sales --release <exact-release-id> --idempotency-key <request-key> \
140
+ --setup-file ./setup.json
141
+
142
+ ci package update --backend-url https://ci.example.com --organization <organization-id> \
143
+ --installation <installation-id> --release <exact-release-id> \
144
+ --idempotency-key <request-key> --setup-file ./setup.json
145
+ ```
146
+
147
+ An explicit setup file submits installation/update even when the exact release
148
+ already serves. Deploy request keys include canonical setup contents: identical
149
+ retries reuse their intent; changed setup gets a different key. For package
150
+ install/update, reuse the explicit key only for identical intent; the backend
151
+ rejects conflicting replay. Setup flags apply only to install/update, not sync or
152
+ publication. A successful configuration operation does not mean records have
153
+ been imported; native schedules/manual actions run afterward.
154
+
155
+ ## Immutable Release Builds
156
+
157
+ Build one package declaration into a deterministic, runtime-only release:
158
+
159
+ ```sh
160
+ ci package build --declaration ./src/package.ts --out ./dist/release
161
+ ```
162
+
163
+ The output directory contains `release.tar`, the authoritative
164
+ `manifest.json`, and private maps under
165
+ `private-source-maps/<runtime-asset-digest>.map`. The archive contains only
166
+ the Backend, Dashboard, and Studio runtime assets declared and integrity
167
+ addressed by the manifest. Private source maps remain outside the runtime
168
+ archive and do not have a public path or URL.
169
+
170
+ Frontend builds keep host-provided modules external and convert CommonJS
171
+ `require` calls for those modules into ESM imports. A CommonJS dependency can
172
+ therefore use the host's React runtime without requiring a Node.js loader in the
173
+ browser. The ESM external-require plugin owns the host-module allowlist; do not
174
+ duplicate those modules in the top-level bundler `external` option, which would
175
+ bypass that conversion.
176
+ Dashboard extensions also share `@xyflow/react` with the host so their handles
177
+ use the same canvas provider as the Dashboard.
178
+ Dashboard and Studio extensions share `@tanstack/react-query` with the host so
179
+ their query hooks read the host's `QueryClient` instead of a bundled copy
180
+ without a provider.
181
+
182
+ Dashboard and Studio builds keep
183
+ `@collective-intelligence/extensions-sdk-frontend-react/workflow` as a
184
+ Host-provided import. The Host supplies its workflow rendering runtime;
185
+ the release archive does not bundle a fallback copy of that runtime.
186
+
187
+ Validate the complete materialized result before publication:
188
+
189
+ ```sh
190
+ ci package validate \
191
+ --archive ./dist/release/release.tar \
192
+ --manifest ./dist/release/manifest.json \
193
+ --package-manifest ./dist/release/package-manifest.json
194
+ ```
195
+
196
+ Validation checks the canonical manifest, archive and asset digests and
197
+ sizes, private-map integrity, and every relative runtime import without
198
+ evaluating Extension code. Use `--private-maps <directory>` only when the
199
+ private maps were moved from their default directory next to the manifest.
200
+
201
+ ## Development Sessions
202
+
203
+ Set an authenticated management token and name the Backend and every Host
204
+ origin that may load local assets:
205
+
206
+ ```sh
207
+ export CI_MANAGEMENT_TOKEN="replace-with-an-ai-manage-token"
208
+
209
+ ci package dev \
210
+ --backend-url https://ci-api.localhost:1355 \
211
+ --declaration ./src/package.ts \
212
+ --host-origin https://ci-dashboard.localhost:1355 \
213
+ --host-origin https://ci-studio.localhost:1355 \
214
+ --host-origin http://localhost:5274 \
215
+ --host-origin tauri://localhost
216
+ ```
217
+
218
+ `CI_BACKEND_URL` may replace `--backend-url` for `package dev` only.
219
+ `CI_MANAGEMENT_TOKEN` is always required and must authorize `ai:manage` on the
220
+ selected Backend. Origins are exact, explicit attachments: an omitted origin
221
+ cannot resolve or fetch the session's assets.
222
+
223
+ The foreground command creates one ephemeral Development Session per declared
224
+ package, publishes a Local Manifest, and serves Vite 8.2.1 modules and sanitized
225
+ source maps from an opaque loopback HTTP address. Each server issues an
226
+ unguessable capability in its module and source-map URLs so Backend and Bun
227
+ consumers without an `Origin` can fetch only the exact published URLs. Browser
228
+ requests with an `Origin` must still match an attached origin; a detached
229
+ origin is rejected even when it presents the capability URL. Responses are
230
+ `no-store`. The capability is a path prefix, so relative executable-chunk and
231
+ advertised source-map URLs retain authorization. Vite may emit a source-mapless
232
+ facade; the CLI omits that nonexistent map from the Local Manifest while its
233
+ imported executable chunk advertises a sanitized, fetchable map. Contributions
234
+ are kept distinct by Host and FQN. Source-only changes rebuild fresh content at
235
+ the already published URLs without incrementing the Local Manifest revision;
236
+ contribution identity or compatibility changes publish a new revision. The
237
+ local package root is watched, including imported sources and declaration
238
+ re-exports.
239
+
240
+ The command heartbeats every session every 5 seconds. The Backend gives a
241
+ package to one live command at a time: a second `package dev` for the same
242
+ package reports that another command owns it and retries until the owner stops
243
+ or misses heartbeats for about 15 seconds. When the Backend forgets a session,
244
+ for example after a restart, the command re-creates it and republishes the
245
+ current Local Manifest without developer action. Other heartbeat failures are
246
+ reported on stderr and retried on the next interval.
247
+
248
+ The session lives only for the command lifetime. On `SIGINT` or `SIGTERM`, the
249
+ CLI uses the same authenticated generated client to terminate every session
250
+ before closing its asset server. A failed initial publication also terminates
251
+ any session that was already created. Removing a package declaration terminates
252
+ its session immediately; shutdown terminates only the packages still declared.
253
+
254
+ The repository's built-in `@platform/extensions#dev` task uses
255
+ `CI_BACKEND_URL`, `CI_DASHBOARD_URL`, `CI_STUDIO_URL`, and `CI_STUDIO_PORT`. It
256
+ attaches the configured Dashboard and Studio URLs, the direct Studio WebView
257
+ origin at `http://localhost:${CI_STUDIO_PORT}`, and `tauri://localhost`, waits
258
+ for Backend `/health`, and requires the same `CI_MANAGEMENT_TOKEN`; missing
259
+ authentication fails visibly.
260
+
261
+ ## Release commands
262
+
263
+ Frontend release entries require the named `loadFrontendCatalog` export. The
264
+ built module also preserves the authored entry's named exports and its default
265
+ export when present, so catalog renderer references using `import.meta.url`
266
+ remain executable after bundling for Dashboard and Studio. Frontend source is
267
+ inspected statically during the build; the CLI does not execute catalog loaders
268
+ or renderers to discover exports.
269
+
270
+ For exact-FQN claims and credential-first publication setup, see the
271
+ [Extension package CLI guide](../../../docs/reference/package-release-cli.md).
272
+
273
+ Publication and administration also require an explicit `--backend-url` and a
274
+ `CI_MANAGEMENT_TOKEN` carrying the permission for that operation:
275
+
276
+ ```sh
277
+ ci package publish --backend-url https://ci.example.com \
278
+ --archive ./dist/release/release.tar \
279
+ --manifest ./dist/release/manifest.json \
280
+ --package-manifest ./dist/release/package-manifest.json \
281
+ --idempotency-key build-482-publish
282
+
283
+ # Only needed if private maps were moved from their default sibling directory:
284
+ ci package publish --backend-url https://ci.example.com \
285
+ --archive ./dist/release/release.tar \
286
+ --manifest ./dist/release/manifest.json \
287
+ --package-manifest ./dist/release/package-manifest.json \
288
+ --private-maps ./secure/private-source-maps \
289
+ --idempotency-key build-482-publish
290
+
291
+ ci package promote --backend-url https://ci.example.com \
292
+ --release release_123 --idempotency-key build-482-promote
293
+
294
+ ci package releases --backend-url https://ci.example.com \
295
+ --organization org_123 --package acme.analytics
296
+
297
+ ci package install --backend-url https://ci.example.com \
298
+ --organization org_123 --package acme.analytics \
299
+ --release release_123 --idempotency-key org-123-install-release-123
300
+
301
+ ci package update --backend-url https://ci.example.com \
302
+ --organization org_123 --installation installation_123 \
303
+ --release release_456 --idempotency-key org-123-update-release-456
304
+ ```
305
+
306
+ `ci deploy` orchestrates an exact package release. The `package` command group
307
+ retains targeted build, publication, installation, and recovery commands. The
308
+ former schema-only and Workflow-only deploy commands were removed; schemas,
309
+ applications, and Workflows are delivered through package releases.
310
+
311
+ Targeted `ci package install` and `ci package update` use generated current package
312
+ contracts. Connection-map files are not accepted. Use `ci deploy` with declared
313
+ Gateway URL bindings for a package's first Gateway enrollment; subsequent exact
314
+ reruns reuse the saved URLs when omitted.
315
+
316
+ Management tokens are supplied through the environment and are never embedded
317
+ in asset URLs, Local Manifests, or command output. The asset URL capability is
318
+ ephemeral, scoped to one local server process, and is not a Backend credential.
319
+
320
+ ## Dataset package instances
321
+
322
+ Managed and External Dataset declarations use the same `ci deploy`, `ci package
323
+ install` and `ci package update --setup-file` commands described above. Author the
324
+ required lifecycle and existing provider FQN with the [SDK Dataset examples](../extensions-sdk/README.md#concrete-managed-and-external-datasets).
325
+ Connection setup remains private input; the release references its `setupKey`.
326
+ Gateway-backed sources require existing manifest approval before admission.
327
+
328
+ For newly package-created Datasets, omitted `dataPolicy` grants no query access,
329
+ including to the deploying actor. Installation configures empty Managed backing
330
+ or a live External binding without importing rows or projection backfill.
331
+ Authorized Managed file/batch ingestion stays manual. Package-declared External
332
+ health/definition refresh schedules and subsequent explicit definition review use
333
+ native Dataset behavior; refresh does not approve or widen a policy implicitly.
334
+
335
+ ## Client secrets for package-created applications
336
+
337
+ Install creates a declared confidential application without a client secret.
338
+ An administrator holding `public_applications:manage` (CLI Session, Dashboard
339
+ session token or API key) runs `ci apps rotate-secret <application-id>`, calls
340
+ `POST /admin/organization/public-applications/<application-id>/credential/rotate`
341
+ with the `x-organization-id` header, or chooses **Rotate secret** on the
342
+ application in the Dashboard External Apps page. The response shows
343
+ `clientSecret` once; store it server-side immediately, because CI keeps only its
344
+ hash. Each rotation invalidates the previous secret. See
345
+ [Provision a secret for a package-created application](../../../docs/reference/package-release-cli.md#provision-a-secret-for-a-package-created-application).