primitive-admin 1.1.0-alpha.4 → 1.1.0-alpha.40

package/README.md

@@ -71,11 +71,15 @@ primitive apps --help
 The CLI uses browser-based OAuth for authentication. When you run `primitive login`, it opens your browser to authenticate via Google OAuth and stores credentials locally.
 
 ```bash
-# Login to the default server
+# Login to the environment resolved from .primitive/config.json
+# (project mode), or the default/PRIMITIVE_SERVER_URL server (legacy mode)
 primitive login
 
-# Login to a custom server
-primitive login --server https://api.example.com
+# Login to a specific named environment
+primitive -e prod login
+
+# Login to a custom server without a project config (legacy mode)
+PRIMITIVE_SERVER_URL=https://api.example.com primitive login
 
 # View current session
 primitive whoami
@@ -150,12 +154,13 @@ Manage users within an app.
 
 ```bash
 primitive users list [app-id]                           # List users
+primitive users create <email> [--role admin|member]    # Create/add user by email
 primitive users invite [app-id] <email> [--role admin]  # Invite user
 primitive users remove [app-id] <user-id>               # Remove user
 primitive users set-role [app-id] <user-id> <role>      # Change role
 primitive users transfer-owner [app-id] <new-owner-id>  # Transfer ownership
-primitive users invitations list [app-id]               # List pending invitations
-primitive users invitations delete [app-id] <inv-id>    # Delete invitation
+primitive users mint-jwt <user-id> [--role <role>]      # Mint test JWT (dev/test only)
+primitive users invitations [app-id]                    # List pending invitations
 ```
 
 ### Waitlist
@@ -191,12 +196,38 @@ primitive integrations logs <id>                  # View invocation logs
 - `--timeout <ms>` - Request timeout in milliseconds
 
 **Secrets management:**
+
+Integration-scoped secrets are **deprecated** and can no longer be created. Use
+app-level secrets instead (`primitive secrets set <KEY> --value <value>`) and reference
+them in integration config via `{{secrets.KEY}}`. See the **Secrets** section
+below. The remaining integration-secret commands only list/archive legacy
+secrets:
+
 ```bash
 primitive integrations secrets list <integration-id>
-primitive integrations secrets add <id> --data '{"apiKey":"..."}'
 primitive integrations secrets archive <id> <secret-id>
 ```
 
+### Secrets
+
+Manage encrypted app secrets (API keys, tokens, credentials). Values are encrypted at rest and never displayed after creation.
+
+```bash
+primitive secrets list [--app <app-id>]                            # List secrets (values never shown)
+primitive secrets set <KEY> --value <value> [--summary <text>]     # Create or update a secret
+primitive secrets delete <KEY>                                      # Delete a secret
+```
+
+**Examples:**
+```bash
+primitive secrets set OPENAI_API_KEY --value "sk-..." --summary "Production key"
+primitive secrets set STRIPE_SECRET --value "sk_live_..."
+primitive secrets list --json
+primitive secrets delete STRIPE_SECRET
+```
+
+Keys must be uppercase letters, digits, and underscores (e.g., `OPENAI_API_KEY`). Max 100 secrets per app, 2 KB per value. The `set` command is an upsert — it creates or updates automatically. Use `{{secrets.KEY}}` in workflows and `secrets.KEY` in CEL rules.
+
 ### Prompts
 
 Manage LLM prompt configurations.
@@ -248,6 +279,39 @@ primitive workflows runs list <workflow-id>
 primitive workflows runs status <workflow-id> <run-id>
 ```
 
+**Workflow TOML push-time validation (issue #685):** Every command that
+pushes a workflow TOML (`workflows create --from-file`,
+`workflows draft update --from-file`, `workflows configs create --from-file`,
+`workflows configs update --from-file`, and `primitive sync push`) runs the
+file through `cli/src/lib/workflow-toml-validator.ts` before sending it to
+the server. For a step whose `kind` the CLI knows about, the validator
+rejects any top-level field outside the allowlist — most commonly catching
+the footgun where `[steps.<id>.request]` is written under `[[steps]]` (TOML
+places the sub-table under `steps[N][<id>]`, not the intended
+`steps[N].request`, so the runtime silently runs the step with an empty
+`request` block). The correct form for the most-recent step is
+`[steps.request]`. For a step whose `kind` the CLI does NOT know (an older
+CLI run against a newer server that registered a new step kind), the
+per-field check is skipped — so a valid new-kind field is never falsely
+rejected — while the kind-agnostic misnested-header footgun detector still
+fires (issue #998).
+
+**The allowlist is generated, not hand-maintained (issue #998):** The field
+union and the known-kind set live in the committed artifact
+`cli/src/lib/generated-allowlist.ts`, generated from the server step runners
+(`src/workflows/steps/*.ts`) and the universal `BaseStepDefinition` shape by
+`cli/scripts/gen-allowlist.mjs`. The generator runs automatically at CLI
+`prebuild`; regenerate manually with `node cli/scripts/gen-allowlist.mjs`
+(or `pnpm -C cli gen:allowlist`). The union is derived from what each runner
+*reads* off the step config — not from what the `*StepDefinition` type
+declares (types over-declare and would weaken the footgun detector). A
+freshness guard (`node cli/scripts/gen-allowlist.mjs --check`, also asserted
+by `cli/tests/unit/workflow-toml-validator-drift-guard.test.ts`) fails if the
+committed artifact drifts from a fresh scan, so the historical hand-edit
+drift (#685/#802/#971) cannot recur. When you add a new step kind or a runner
+starts reading a new field, just regenerate and commit the artifact — there is
+no hand-maintained list to update.
+
 ### Tokens
 
 Manage long-lived API access tokens for headless/server authentication.
@@ -263,16 +327,17 @@ primitive tokens revoke <token-id> [app-id]                           # Revoke t
 **Create options:**
 - `--name <name>` - Token name (required)
 - `--user <user-id>` - App user ID to associate the token with (required)
-- `--ttl <duration>` - Token lifetime (e.g., 7d, 30d, 4w, 3m, 1y). Omit for never-expiring
+- `--ttl <duration>` - Token lifetime (e.g., 30m, 2h, 7d, 30d, 4w, 3mo, 1y). Omit for never-expiring. Units: `m`/`min`=minutes (≤1440), `h`=hours (≤24), `d`=days, `w`=weeks, `mo`=months, `y`=years. **Note:** as of #420 `m` means minutes (was months); use `mo` for months.
 
 ### Databases
 
-Manage online databases, permissions, and group permissions.
+Manage online databases and permissions. `list` returns databases the user has direct access to; group-shared databases are listed via `primitive groups databases`.
 
 ```bash
-primitive databases list [app-id]                     # List databases
+primitive databases list [app-id]                     # List databases (direct access)
 primitive databases create <title> [app-id]           # Create database
 primitive databases get <database-id> [app-id]        # Get details
+primitive databases update <database-id> [options]    # Update title or type
 primitive databases delete <database-id> [app-id]     # Delete database
 ```
 
@@ -283,14 +348,68 @@ primitive databases permissions grant <database-id> --user-id <uid> --permission
 primitive databases permissions revoke <database-id> <user-id> [app-id]
 ```
 
-**Group permissions:**
+Permission values: `owner` (set at creation), `manager`
+
+**Metadata:**
+```bash
+primitive databases metadata update <database-id> --data '{"key":"value"}'  # Merge-update metadata
+```
+
+**Operations:**
+```bash
+primitive databases operations list <database-id> [app-id]                     # List registered operations
+primitive databases operations execute <database-id> <op-name> --params '{}'   # Execute operation
+primitive databases operations execute <database-id> <op-name> --token <jwt>   # Execute as specific user
+```
+
+The `--token` flag lets you execute an operation as a specific user using a test JWT from `users mint-jwt`. Useful for testing access rules.
+
+**Records (schema introspection):**
+```bash
+primitive databases records models <database-id> [app-id]            # List model names
+primitive databases records describe <database-id> <model> [app-id]  # Show inferred schema
+```
+
+**Indexes:**
+```bash
+primitive databases indexes list <database-id> [--model <name>]       # List indexes
+primitive databases indexes create <database-id> <model> <field> [options]  # Create index
+primitive databases indexes drop <database-id> <model> <field>        # Drop index
+```
+
+**Export / Import:**
+```bash
+primitive databases export [app-id] <database-id> --output <dir>      # Export records, indexes, constraints
+primitive databases import [app-id] <path> --overwrite --dry-run      # Import from export directory
+```
+
+Export creates a directory with `metadata.json`, `records.jsonl`, `indexes.json`, and `constraints.json`. Import restores records and indexes into a new or existing database. Database type config (operations, triggers, access rules) is managed separately via `primitive sync` — run `sync push` on the target app before importing.
+
+### Documents
+
+Manage document ownership, group permissions, and export/import.
+
+```bash
+primitive documents transfer-owner [app-id] <document-id> <new-owner-id>  # Transfer ownership
+```
+
+**Group permissions on documents:**
+```bash
+primitive documents group-permissions list <document-id>
+primitive documents group-permissions grant <document-id> --group-type <type> --group-id <id> --permission <perm>
+primitive documents group-permissions revoke <document-id> <group-type> <group-id>
+```
+
+Permission values: `read-write`, `reader`
+
+**Export / Import:**
 ```bash
-primitive databases group-permissions list <database-id> [app-id]
-primitive databases group-permissions grant <database-id> --group-type <type> --group-id <id> --permission <perm> [app-id]
-primitive databases group-permissions revoke <database-id> <group-type> <group-id> [app-id]
+primitive documents export [app-id] <document-id> --output <dir>         # Export Yjs state, blobs, permissions, aliases
+primitive documents export-all [app-id] --user-id <id> --owned-only      # Export all docs for a user
+primitive documents import [app-id] <path> --overwrite --aliases overwrite|skip --dry-run  # Import from export
 ```
 
-Permission values: `owner`, `read-write`, `reader`
+Export creates a directory per document with `metadata.json`, `document.yjs` (Yjs state), `permissions.json` (for reference), and `blobs/` (attachments). Permissions are exported for reference but not restored during import — the importing admin is the new owner and manages sharing in the target app. Document IDs are preserved across import. User-scoped aliases can be restored with `--aliases overwrite` (update existing) or `--aliases skip` (keep existing, default).
 
 ### Groups
 
@@ -317,20 +436,46 @@ primitive groups members set-role <group-type> <group-id> <user-id> <role> [app-
 primitive groups memberships <user-id> [app-id]          # List user's group memberships
 ```
 
+**Group resource access:**
+```bash
+primitive groups documents <group-type> <group-id>       # List documents a group can access
+```
+
+Group permissions on documents are managed via `primitive documents group-permissions` (see [Documents](#documents) above).
+
 ### Analytics
 
 View usage analytics for an app.
 
 ```bash
-primitive analytics overview [app-id]              # Activity overview
-primitive analytics top-users [app-id]             # Most active users
-primitive analytics user [app-id] <user-ulid>      # User activity details
-primitive analytics integrations [app-id]          # Integration metrics
+# Overview & active users
+primitive analytics overview [app-id]                  # DAU / WAU / MAU + growth
+primitive analytics daily-active [app-id]              # Daily active users time series
+primitive analytics rolling-active [app-id]            # Rolling active users (28 points)
+primitive analytics cohort-retention [app-id]          # Weekly cohort retention matrix
+
+# Users
+primitive analytics top-users [app-id]                 # Most active users
+primitive analytics user-search [app-id] --query <q>   # Search by email or ULID
+primitive analytics user-detail <user-ulid> [app-id]   # User activity breakdown
+primitive analytics user-snapshot <user-ulid> [app-id] # Latest context snapshot
+
+# Events
+primitive analytics events [app-id]                    # Paginated event feed
+primitive analytics events-grouped [app-id]            # Events grouped by dimension
+
+# Features
+primitive analytics integrations [app-id]              # Integration usage metrics
+primitive analytics workflows [app-id]                 # Top workflows by runs
+primitive analytics prompts [app-id]                   # Top prompts by executions
 ```
 
-**Options:**
-- `--window-days <n>` - Time window (default: 30)
-- `--limit <n>` - Result limit for top-users
+**Common options:**
+- `--window-days <n>` - Time window in days (default varies per command)
+- `--limit <n>` - Result limit (top-users, workflows, prompts)
+- `--group-by <dim>` - Dimension for events-grouped (action, feature, route, country, deviceType, plan, day)
+- `--page <n>` - Page number for events feed (0-based)
+- `--json` - Output raw JSON
 
 ### Admins (Super-Admin Only)
 
@@ -412,6 +557,9 @@ baseUrl = "https://myapp.com"
 googleOAuthEnabled = true
 passkeyEnabled = true
 magicLinkEnabled = false
+otpEnabled = true                      # email one-time-passcode sign-in
+appleSignInEnabled = true              # Sign in with Apple
+appleAudiences = ["com.example.MyApp"] # allowed Apple audiences
 
 [cors]
 mode = "custom"
@@ -419,6 +567,19 @@ allowedOrigins = ["https://myapp.com"]
 allowCredentials = true
 ```
 
+Notes on `[auth]`:
+
+- Only keys you actually list are pushed. Omitting a key leaves the server's
+  current value untouched — it is **not** reset to `false`. To turn a method
+  off, set it explicitly (e.g. `magicLinkEnabled = false`).
+- An unrecognized `[auth]` key (for example a typo like `googleOauthEnabled`)
+  produces a warning on `push` and is ignored, rather than being silently
+  dropped. The push still applies the recognized keys.
+- `appleAudiences = []` and an omitted `appleAudiences` both mean "no
+  audiences" (the server stores them the same way), so `pull` omits the key
+  when there are no audiences. A `[] ` round-trip reading back as an omitted
+  key is expected, not data loss.
+
 ### Example: integrations/weather-api.toml
 
 ```toml
@@ -475,6 +636,129 @@ primitive apps list
 primitive apps list --json | jq '.[0].appId'
 ```
 
+### Stdout vs. stderr
+
+The CLI follows the same convention as `git`, `kubectl`, `aws`, and `gh`:
+
+- **stdout** carries the *data* the command produced — JSON documents under
+  `--json`, tabular listings (`apps list`), raw values (`primitive token`),
+  and the primary `label: value` fields a `get` / `show` / `describe` command
+  emits (e.g. `primitive integrations get <id>`).
+- **stderr** carries *diagnostics* — status text (`✓ Created…`,
+  `i Waiting for completion...`), warnings (`!` markers), progress lines, and
+  the `key: value` summaries shown *after* a side-effecting command runs
+  (e.g. the `Webhook ID: …` confirmation printed by `webhooks create`).
+
+This means `primitive <cmd> --json | jq` always works, regardless of any
+status / warning / progress text the command may print along the way.
+It also means `primitive integrations get <id> > out.txt` writes the
+integration's fields to `out.txt` while the status diagnostics still appear
+on the terminal via stderr.
+
+Conversely, redirecting stderr (`primitive <cmd> 2>/dev/null`) silences
+diagnostics — including success checkmarks like `✓ Pushed 3 changes` — so
+prefer `2>&1 >file.txt` if you want the full transcript.
+
+For contributors writing new commands, the output helpers in
+`cli/src/lib/output.ts` make the data-vs-diagnostic split explicit: use
+`json()` and `result(label, value)` for data the caller asked for, and
+`success()` / `info()` / `warn()` / `keyValue()` / `divider()` / `heading()`
+for diagnostics about what the command did.
+
+## Programmatic surfaces
+
+### `primitive help --json` — command manifest
+
+`primitive help --json` prints a machine-readable manifest of the full command
+tree (commands, subcommands, positional args, options, aliases, global options,
+and the CLI version) to **stdout** as a single JSON document, then exits `0`. It
+is hidden/undocumented in human `--help` output, auth-free, and runs entirely
+offline (no project config, no credentials, no network) — so it works in any
+directory, including one with no `.primitive/`.
+
+```bash
+primitive help --json | jq '.commands[] | select(.name=="sync") | .commands[].name'
+```
+
+Shape (`schemaVersion: 1`):
+
+```jsonc
+{
+  "schemaVersion": 1,
+  "name": "primitive",
+  "version": "1.0.49",
+  "globalOptions": [ { "flags": "-e, --env <name>", "description": "…", "required": true, "defaultValue": null } ],
+  "commands": [
+    {
+      "name": "sync",
+      "description": "…",
+      "aliases": [],
+      "args": [],
+      "options": [],
+      "commands": [
+        {
+          "name": "push",
+          "description": "…",
+          "aliases": [],
+          "args": [ { "name": "app-id", "required": false, "variadic": false, "description": "…" } ],
+          "options": [ { "flags": "--dir <path>", "description": "…", "required": true, "defaultValue": null } ],
+          "commands": []
+        }
+      ]
+    }
+  ]
+}
+```
+
+`schemaVersion` is the contract knob: consumers assert the shape they understand
+and fail loud if a future CLI bumps it (a breaking change for downstream tooling
+such as docs CI that drift-gates documented invocations against the published
+CLI).
+
+### `primitive-admin/validators` — TOML validators
+
+The TOML validators that `sync push` runs as its load-bearing gate are exported
+from the `primitive-admin/validators` subpath (ESM-only, with types) so external
+tooling can validate exactly the way the published CLI does — instead of
+re-implementing the flow and silently drifting.
+
+```ts
+import {
+  validateWorkflowTomlSource, // source-level wrapper (parse → expand → validate)
+  validateWorkflowToml,        // raw: pure fn over already-parsed TOML
+  formatWorkflowTomlErrors,
+  validateOperations,          // raw: pure fn over already-parsed operations[]
+  formatIssue,
+} from "primitive-admin/validators";
+```
+
+Two contracts:
+
+- **`validateWorkflowTomlSource(raw, filePath)`** — pass the **raw TOML source
+  string** and the workflow's path. It runs the *identical* parse →
+  `expandWorkflowTomlData()` (`include` fragment expansion) → `validateWorkflowToml`
+  → `formatWorkflowTomlErrors` path that `sync push` runs, so it cannot diverge
+  from `sync push` on workflow `include` fragments. Returns
+  `{ errors, formatted }` (`formatted` is `null` when valid). Fragment files
+  resolve relative to `<filePath>/../../workflow-fragments/`, matching the CLI.
+
+- **The raw validators** (`validateWorkflowToml`, `validateOperations`,
+  `formatWorkflowTomlErrors`, `formatIssue`) are **pure functions over
+  already-parsed TOML** — they do *not* read files or parse TOML. The consumer
+  parses (e.g. with `@iarna/toml`) and, for workflows, performs any `include`
+  expansion itself. There is **no coupling on a specific TOML parser version**:
+  pass whatever parsed object / `operations[]` you already have.
+
+  ```ts
+  import * as TOML from "@iarna/toml";
+  const data = TOML.parse(raw);
+  const errs = validateWorkflowToml(data);
+  if (errs.length) throw new Error(formatWorkflowTomlErrors(path, errs));
+  ```
+
+Internal helpers (`collectParamRefs`, `declaredParamNames`,
+`locateOperationLine`) are intentionally **not** exported.
+
 ## Exit Codes
 
 - `0` - Success
@@ -483,12 +767,20 @@ primitive apps list --json | jq '.[0].appId'
 
 ## Environment Variables
 
-- `PRIMITIVE_SERVER_URL` - Server URL for login (defaults to `https://primitiveapi.com`)
+- `PRIMITIVE_SERVER_URL` - Server URL for login in legacy mode (defaults to `https://primitiveapi.com`)
+
+**Login server resolution:**
+1. Project mode (`.primitive/config.json` present): the active environment's
+   `apiUrl`. Select the environment with `-e <name>`, `PRIMITIVE_ENV`, or
+   `defaultEnvironment`. An unresolvable environment fails loudly — `login`
+   never silently falls back to production.
+2. Legacy mode (no project config): `PRIMITIVE_SERVER_URL`, else the default
+   `https://primitiveapi.com`.
 
-**Server URL priority:**
-1. `--server` flag (highest priority)
-2. `PRIMITIVE_SERVER_URL` environment variable
-3. Default: `https://primitiveapi.com`
+> `login` no longer accepts `-s/--server` (removed in a breaking change). Use
+> `-e <env>` in a project, or `PRIMITIVE_SERVER_URL` for local dev / scripts.
+> `init` and `bootstrap` keep their own `-s/--server` (they run before a
+> project config exists).
 
 ## Development
 
@@ -545,11 +837,13 @@ cli/tests/
     config.test.ts    # Credentials and config management
     output.test.ts    # Output formatting functions
   integration/
-    api-client.test.ts  # API client HTTP tests
-    commands.test.ts    # CLI command tests
-    tokens.test.ts      # Token lifecycle tests
-    databases.test.ts   # Database CRUD, permissions, group permissions tests
-    groups.test.ts      # Group CRUD, members, memberships tests
+    api-client.test.ts      # API client HTTP tests
+    commands.test.ts        # CLI command tests
+    tokens.test.ts          # Token lifecycle tests
+    databases.test.ts       # Database CRUD, permissions, group permissions tests
+    documents.test.ts       # Document group permissions, group resource listing tests
+    groups.test.ts          # Group CRUD, members, memberships tests
+    classroom-e2e.test.ts   # End-to-end classroom app workflow (types, rules, operations, access)
 ```
 
 ## Troubleshooting