@magoz/provision 0.1.0 → 0.2.0

package/README.md

@@ -220,7 +220,8 @@ Revoke immediately if the host may be compromised.
 provision factory [--checkout <repo>] --domain <production-host>
                   [--sender-name <name>] [--email-from <address>]
                   [--factory <name>] [--only vercel,neon,r2,upstash,resend,reports,secrets,domain]
-                  [--on-existing ask|reuse|overwrite|abort] [--neon-parent-branch <id>] [--dry-run]
+                  [--on-existing ask|reuse|overwrite|abort] [--neon-parent-branch <id>]
+                  [--secret NAME[=base64-<bytes>|hex-<bytes>|vapid]]... [--dry-run]
 ```
 
 Creates what the repository declares (`provision.factory.steps`, narrowed by `--only`) for the
@@ -235,15 +236,26 @@ checkout's GitHub repository `<owner>/<repo>`. Resources are named `<repo>` (Ver
 | `upstash` | `<repo>-dev`/`-prod` teams                                                                         | `QSTASH_*`, pasted by you (see [Manual follow-ups](#manual-follow-ups)) |
 | `resend`  | sending-only keys (any verified domain)                                                            | `RESEND_API_KEY`                                                        |
 | `reports` | report tokens from the factory's receiver                                                          | `REPORT_RECEIVER_URL`, `REPORT_RECEIVER_TOKEN`                          |
-| `secrets` | generated `BETTER_AUTH_SECRET`, `INTEGRATION_CREDENTIAL_ENCRYPTION_KEY`, `CRON_SECRET`, VAPID pair | those, plus `AUTH_EMAIL_FROM`, `AI_PROVIDER_USAGE_ALERT_THRESHOLDS`     |
+| `secrets` | the secrets named with `--secret` (generated, never shown)                                         | those, plus `AUTH_EMAIL_FROM` when a sender is known                    |
 | `domain`  | attaches `--domain` to the project                                                                 | —                                                                       |
 
 - **Environments.** Pre-production values go to Development, Preview, and `test`; production values
   go to Production and are Sensitive. Each side gets its own keys and secrets.
 - **`--domain`** is the production host. The `r2` step allows browser uploads to the production
   bucket only from `https://<domain>`; the development bucket accepts any origin.
-- **`--email-from`** sets `AUTH_EMAIL_FROM` (`"<sender-name> <address>"`). The address's domain must
-  be verified in the factory's Resend account.
+- **`--email-from`** sets `AUTH_EMAIL_FROM` (`"<sender-name> <address>"`), defaulting to
+  `noreply@<resend.sending_domain>`. The address's domain must be verified in the factory's Resend
+  account. Without either, `AUTH_EMAIL_FROM` is not written.
+- **`--secret`** (repeatable) names a secret the repository's code reads; nothing is generated
+  that isn't asked for. `NAME` alone is 32 random bytes in base64; `NAME=hex-32` is hex;
+  `NAME=vapid` is a web-push pair written as `NAME_PUBLIC_KEY` (plain) and `NAME_PRIVATE_KEY`.
+  Pre-production and production get separate values. A secret already in Vercel, even in only
+  part of an environment group, is replaced only when `--on-existing` says so.
+- **Which flags.** The repository's code decides: the variables it reads (`.env.example`, its
+  config reads) minus what `vercel env ls` already lists. Provider keys come from their steps
+  (`--only`), generated secrets from `--secret`; third-party keys and plain settings are added
+  with `vercel env add`. The [agent skill](#agents) does this and prepares the command.
+- **R2 uploads** allow the headers a signed upload binds: `content-type` and `if-none-match`.
 - **Order.** `vercel` runs first; the other steps then run concurrently. Each step prints one block
   when it finishes, and a failing step lets the others finish before the run fails.
 
@@ -252,11 +264,13 @@ checkout's GitHub repository `<owner>/<repo>`. Resources are named `<repo>` (Ver
 ```bash
 # 1. Plan: reads every provider, changes nothing
 provision factory --checkout ~/src/acme-app --domain app.acme.com \
-  --sender-name Acme --email-from noreply@acme.com --dry-run
+  --sender-name Acme --email-from noreply@acme.com \
+  --secret BETTER_AUTH_SECRET --secret CRON_SECRET=hex-32 --dry-run
 
 # 2. Apply
 provision factory --checkout ~/src/acme-app --domain app.acme.com \
-  --sender-name Acme --email-from noreply@acme.com
+  --sender-name Acme --email-from noreply@acme.com \
+  --secret BETTER_AUTH_SECRET --secret CRON_SECRET=hex-32
 
 # 3. Rerun: should only report "reusing"
 provision factory --checkout ~/src/acme-app --domain app.acme.com \
@@ -456,7 +470,7 @@ passphrase read from the terminal: an agent that hides its variables still canno
 ```json
 {
   "contractVersion": 1,
-  "provisionVersion": "0.1.0",
+  "provisionVersion": "0.2.0",
   "commands": ["contract", "config", "factory", "env", "db"]
 }
 ```

package/dist/factory/commands.js

@@ -14,7 +14,7 @@ import { neonStep } from './steps/neon.js';
 import { r2Step } from './steps/r2.js';
 import { reportsStep } from './steps/reports.js';
 import { resendStep } from './steps/resend.js';
-import { secretsStep } from './steps/secrets.js';
+import { parseSecretSpecs, secretsStep } from './steps/secrets.js';
 import { upstashStep } from './steps/upstash.js';
 import { vercelStep } from './steps/vercel.js';
 const steps = {
@@ -123,6 +123,7 @@ export const runFactory = (options) => Effect.gen(function* () {
     if (emailFrom !== undefined && !/^[^\s@<>]+@[a-z0-9.-]+\.[a-z]{2,}$/.test(emailFrom)) {
         return yield* factoryError('--email-from must be an address such as noreply@acme.com');
     }
+    const secrets = yield* parseSecretSpecs(options.secret);
     const factory = yield* selectFactory(yield* readRegistry, options.factory, owner);
     if (factory.credentials.githubOwner !== owner) {
         return yield* factoryError(`factory ${factory.name} serves GitHub owner ${factory.credentials.githubOwner}, not ${owner}`);
@@ -138,6 +139,7 @@ export const runFactory = (options) => Effect.gen(function* () {
         senderName: Option.getOrElse(options.senderName, () => name),
         emailFrom,
         neonParentBranch: Option.getOrUndefined(options.neonParentBranch),
+        secrets,
         dryRun: options.dryRun,
         onExisting: options.onExisting,
         names: { project: name, development: `${name}-dev`, production: `${name}-prod` }
@@ -167,6 +169,7 @@ export const factoryCommand = Command.make('factory', {
     only: Flag.String('only').pipe(Flag.withDescription(`comma-separated subset of: ${factorySteps.join(', ')}`), Flag.optional),
     onExisting: Flag.Literals('on-existing', onExistingPolicies).pipe(Flag.withDescription('what to do when a resource exists (data is only ever reused)'), Flag.withDefault('ask')),
     neonParentBranch: Flag.String('neon-parent-branch').pipe(Flag.withDescription('existing-branch mode: sandbox parent branch id'), Flag.optional),
+    secret: Flag.String('secret').pipe(Flag.withDescription('repeatable: a secret the code reads, NAME[=base64-<bytes>|hex-<bytes>|vapid] (default base64-32), generated per environment group'), Flag.atLeast(0)),
     dryRun: Flag.Boolean('dry-run').pipe(Flag.withDescription('inspect and print planned changes without making them'), Flag.withDefault(false))
 }, options => humanOnly(runFactory(options).pipe(Effect.catch(error => Console.error(`provision factory: ${error.message}`).pipe(Effect.andThen(Effect.sync(() => {
     process.exitCode = 2;

package/dist/factory/context.js

@@ -103,6 +103,12 @@ const containedIn = (record, group) => recordTargets(record).every(target => gro
     (record.customEnvironmentIds ?? []).every(id => group.customEnvironmentIds.includes(id));
 /** Whether every key has a record covering the whole group. */
 export const groupHasKeys = (records, group, keys) => keys.every(key => records.some(record => record.key === key && covers(record, group)));
+/** Whether the group has every key everywhere (`complete`), some of it (`partial`), or none. */
+export const groupKeyState = (records, group, keys) => groupHasKeys(records, group, keys)
+    ? 'complete'
+    : keys.some(key => records.some(record => record.key === key && overlaps(record, group)))
+        ? 'partial'
+        : 'none';
 /**
  * Replaces the group's records for each key: overlapping records are deleted, then one record per
  * key is created. A record that spans environments outside the group is never touched.

package/dist/factory/providers/cloudflare.js

@@ -13,11 +13,14 @@ export const r2Endpoint = (accountId) => `https://${accountId}.r2.cloudflarestor
 // ---------------------------------------------------------------------------
 export const bucketExists = (admin, name) => requestJson('R2 bucket lookup', `${account(admin)}/r2/buckets/${encodeURIComponent(name)}`, envelope(Schema.Struct({ name: Schema.String })), { headers: headers(admin.apiToken), allowNotFound: true }).pipe(Effect.map(Option.isSome));
 export const createBucket = (admin, name) => requestJsonRequired('R2 bucket creation', `${account(admin)}/r2/buckets`, envelope(Schema.Struct({ name: Schema.String })), { method: 'POST', headers: headers(admin.apiToken), body: { name } }).pipe(Effect.asVoid);
-/** Browser uploads use presigned PUTs; CORS allows exactly that. */
+/**
+ * Browser uploads use presigned PUTs; CORS allows exactly that, with the headers a signed upload
+ * binds (`content-type`, and `if-none-match` for create-only uploads).
+ */
 const corsRules = (origin) => [
     {
         id: 'browser-uploads',
-        allowed: { methods: ['PUT'], origins: [origin], headers: ['content-type'] },
+        allowed: { methods: ['PUT'], origins: [origin], headers: ['content-type', 'if-none-match'] },
         exposeHeaders: ['etag'],
         maxAgeSeconds: 3600
     }

package/dist/factory/steps/secrets.js

@@ -1,10 +1,12 @@
 import { generateKeyPairSync, randomBytes } from 'node:crypto';
 import { Console, Effect, Option, Redacted } from 'effect';
-import { decide, envGroups, factoryError, groupHasKeys, requireProject, writeGroup } from '../context.js';
+import { decide, envGroups, factoryError, groupKeyState, requireProject, writeGroup } from '../context.js';
 import { requireVerifiedDomain } from '../providers/resend.js';
 import { listEnvRecords } from '../vercel-api.js';
-export const usageAlertThresholds = '50,80,95';
-const base64 = (bytes) => Redacted.make(randomBytes(bytes).toString('base64'));
+const generators = {
+    base64: (bytes) => Redacted.make(randomBytes(bytes).toString('base64')),
+    hex: (bytes) => Redacted.make(randomBytes(bytes).toString('hex'))
+};
 /** A web-push VAPID pair: uncompressed P-256 public point and raw private scalar, base64url. */
 export const generateVapidKeys = () => {
     const { privateKey, publicKey } = generateKeyPairSync('ec', { namedCurve: 'prime256v1' });
@@ -17,64 +19,79 @@ export const generateVapidKeys = () => {
         privateKey: Redacted.make(privateJwk.d ?? '')
     };
 };
-/** Generated per group: a separate value for pre-production and production. */
-const generatedSets = [
-    {
-        description: 'BETTER_AUTH_SECRET',
-        keys: ['BETTER_AUTH_SECRET'],
-        kind: 'secret',
-        entries: type => [{ key: 'BETTER_AUTH_SECRET', value: base64(32), type }]
-    },
-    {
-        description: 'INTEGRATION_CREDENTIAL_ENCRYPTION_KEY',
-        keys: ['INTEGRATION_CREDENTIAL_ENCRYPTION_KEY'],
-        kind: 'secret',
-        entries: type => [{ key: 'INTEGRATION_CREDENTIAL_ENCRYPTION_KEY', value: base64(32), type }]
-    },
-    {
-        description: 'CRON_SECRET',
-        keys: ['CRON_SECRET'],
-        kind: 'key',
-        entries: type => [
-            { key: 'CRON_SECRET', value: Redacted.make(randomBytes(32).toString('hex')), type }
-        ]
-    },
-    {
-        description: 'VAPID key pair',
-        keys: ['VAPID_PUBLIC_KEY', 'VAPID_PRIVATE_KEY'],
-        kind: 'secret',
-        entries: type => {
-            const pair = generateVapidKeys();
-            return [
-                { key: 'VAPID_PUBLIC_KEY', value: Redacted.make(pair.publicKey), type: 'plain' },
-                { key: 'VAPID_PRIVATE_KEY', value: pair.privateKey, type }
-            ];
+const secretName = /^[A-Z][A-Z0-9_]*$/;
+export const parseSecretSpecs = (values) => Effect.gen(function* () {
+    const specs = [];
+    for (const value of values) {
+        const [name = '', format = 'base64-32', ...rest] = value.split('=');
+        const sized = /^(base64|hex)-(\d+)$/.exec(format);
+        const bytes = Number(sized?.[2]);
+        if (!secretName.test(name) || rest.length > 0) {
+            return yield* factoryError(`--secret ${value}: expected NAME[=FORMAT] with an upper-case NAME, e.g. BETTER_AUTH_SECRET=base64-32`);
+        }
+        if (format === 'vapid')
+            specs.push({ name, format: { kind: 'vapid' } });
+        else if ((sized?.[1] === 'base64' || sized?.[1] === 'hex') && bytes >= 16 && bytes <= 128) {
+            specs.push({ name, format: { kind: sized[1], bytes } });
         }
+        else {
+            return yield* factoryError(`--secret ${value}: FORMAT is base64-<bytes>, hex-<bytes> (16 to 128 bytes), or vapid`);
+        }
+    }
+    const keys = specs.flatMap(spec => specKeys(spec));
+    const duplicates = keys.filter((key, index) => keys.indexOf(key) !== index);
+    if (duplicates.length > 0) {
+        return yield* factoryError(`--secret names a variable twice: ${[...new Set(duplicates)].join(', ')}`);
     }
-];
-const ensureSet = (context, handle, records, group, set, entries) => Effect.gen(function* () {
-    if (groupHasKeys(records, group, set.keys)) {
-        const decision = yield* decide(context, set.kind, `${set.description} (${group.label})`);
+    return specs;
+});
+const specKeys = (spec) => spec.format.kind === 'vapid'
+    ? [`${spec.name}_PUBLIC_KEY`, `${spec.name}_PRIVATE_KEY`]
+    : [spec.name];
+const generate = (spec, type) => {
+    if (spec.format.kind === 'vapid') {
+        const pair = generateVapidKeys();
+        return [
+            { key: `${spec.name}_PUBLIC_KEY`, value: Redacted.make(pair.publicKey), type: 'plain' },
+            { key: `${spec.name}_PRIVATE_KEY`, value: pair.privateKey, type }
+        ];
+    }
+    return [{ key: spec.name, value: generators[spec.format.kind](spec.format.bytes), type }];
+};
+/**
+ * Writes `entries` unless the group already has any of `keys`: whole or partial, an existing value
+ * is only replaced when `--on-existing` says so (never silently).
+ */
+const ensure = (context, handle, records, group, description, keys, entries) => Effect.gen(function* () {
+    const state = groupKeyState(records, group, keys);
+    if (state !== 'none') {
+        const partly = state === 'partial' ? ', set only in part of it' : '';
+        const decision = yield* decide(context, 'secret', `${description} (${group.label}${partly})`);
         if (decision === 'reuse')
             return;
     }
     yield* writeGroup(context, handle, group, entries());
 });
 /**
- * App secrets the CLI generates (never shown or stored elsewhere), plus plain settings shared by
- * all environments: `AUTH_EMAIL_FROM` and `AI_PROVIDER_USAGE_ALERT_THRESHOLDS`.
+ * The secrets the repository's code reads, named with `--secret` (generated here, never shown or
+ * stored elsewhere): a separate value for pre-production and for production (Sensitive). Plus
+ * `AUTH_EMAIL_FROM` for every environment when a sender is known (`--email-from`, else
+ * `noreply@<resend.sending_domain>`), on a domain verified in the factory's Resend account.
  */
 export const secretsStep = (context) => Effect.gen(function* () {
     const resend = context.credentials.resend;
-    if (resend === undefined) {
-        return yield* factoryError('the secrets step needs the resend section for AUTH_EMAIL_FROM');
-    }
     const fromAddress = context.emailFrom ??
-        (resend.sendingDomain === undefined ? undefined : `noreply@${resend.sendingDomain}`);
-    if (fromAddress === undefined) {
-        return yield* factoryError('the secrets step needs a sender: pass --email-from or set resend.sending_domain in the factory item');
+        (resend?.sendingDomain === undefined ? undefined : `noreply@${resend.sendingDomain}`);
+    if (context.secrets.length === 0 && fromAddress === undefined) {
+        yield* Console.log('  nothing to generate: pass --secret NAME=FORMAT or --email-from');
+        return;
+    }
+    if (fromAddress !== undefined) {
+        if (resend === undefined) {
+            return yield* factoryError('AUTH_EMAIL_FROM needs the resend section to verify its domain');
+        }
+        yield* requireVerifiedDomain(resend.apiKey, fromAddress.slice(fromAddress.indexOf('@') + 1));
     }
-    yield* requireVerifiedDomain(resend.apiKey, fromAddress.slice(fromAddress.indexOf('@') + 1));
     const handle = yield* requireProject(context);
     if (Option.isNone(handle)) {
         yield* Console.log('  would run after the vercel step creates the project');
@@ -82,23 +99,23 @@ export const secretsStep = (context) => Effect.gen(function* () {
     }
     const groups = envGroups(handle.value);
     const records = yield* listEnvRecords(context.credentials.vercel.teamId, handle.value.project.id);
-    for (const set of generatedSets) {
-        yield* ensureSet(context, handle.value, records, groups.preproduction, set, () => set.entries('encrypted'));
-        yield* ensureSet(context, handle.value, records, groups.production, set, () => set.entries('sensitive'));
+    for (const spec of context.secrets) {
+        const keys = specKeys(spec);
+        for (const [group, type] of [
+            [groups.preproduction, 'encrypted'],
+            [groups.production, 'sensitive']
+        ]) {
+            yield* ensure(context, handle.value, records, group, keys.join(', '), keys, () => generate(spec, type));
+        }
     }
-    const plain = [
-        {
+    if (fromAddress !== undefined) {
+        const entry = {
             key: 'AUTH_EMAIL_FROM',
             value: Redacted.make(`${context.senderName} <${fromAddress}>`),
             type: 'plain'
-        },
-        {
-            key: 'AI_PROVIDER_USAGE_ALERT_THRESHOLDS',
-            value: Redacted.make(usageAlertThresholds),
-            type: 'plain'
-        }
-    ];
-    for (const entry of plain) {
-        yield* ensureSet(context, handle.value, records, groups.all, { description: entry.key, keys: [entry.key], kind: 'key', entries: () => [entry] }, () => [entry]);
+        };
+        yield* ensure(context, handle.value, records, groups.all, entry.key, [entry.key], () => [
+            entry
+        ]);
     }
 });

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@magoz/provision",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Provision projects (factory), local checkouts (env), and disposable databases (db).",
   "license": "MIT",
   "repository": {
@@ -29,6 +29,7 @@
   },
   "dependencies": {
     "@effect/platform-node": "4.0.2",
+    "@effect/platform-node-shared": "4.0.2",
     "effect": "4.0.2"
   },
   "devDependencies": {

package/skills/provision/SKILL.md

@@ -119,24 +119,63 @@ Release both leases before deleting a worktree. Branches expire on their own aft
 Prerequisites: the GitHub repository exists under the factory's GitHub owner, there is a local
 checkout with that `origin`, and the factory is registered (`provision config list`).
 
-Prepare these for the human, in order:
+### First, work out what the repository needs
+
+Don't assume a template: read the code, then compare with Vercel.
+
+1. **Variables the code reads.** Search for config reads (`Config.String('…')`,
+   `Config.Redacted('…')`, `process.env.…`), `.env.example`, and any production or setup doc in
+   the repo. Note how the docs say each secret is generated (for example `openssl rand -base64 32`).
+2. **What Vercel already has** (names only, from the linked checkout):
+   `vercel env ls development`, `vercel env ls preview`, `vercel env ls production`,
+   `vercel env ls test`. Existing values are reused; never regenerate what exists unless the
+   human asks to rotate.
+3. **Sort each missing variable:**
+
+   | The code reads…                                                        | Comes from                                                   |
+   | ---------------------------------------------------------------------- | ------------------------------------------------------------ |
+   | `DATABASE_URL*` (Neon)                                                 | `neon` step                                                  |
+   | `R2_*` (S3-compatible storage)                                         | `r2` step (needs `--domain` for the upload origin)           |
+   | `QSTASH_*`                                                             | `upstash` step (the human pastes the console's `.env` block) |
+   | `RESEND_API_KEY`                                                       | `resend` step                                                |
+   | `AUTH_EMAIL_FROM`                                                      | `--email-from` (domain verified in the factory's Resend)     |
+   | `REPORT_RECEIVER_*`                                                    | `reports` step (factories with a receiver)                   |
+   | a random secret the app generates itself (auth, cron, encryption keys) | `--secret NAME[=FORMAT]`                                     |
+   | a third-party key (AI Gateway, OAuth apps, data APIs)                  | the human, with `vercel env add`                             |
+   | plain config (URLs, limits, flags)                                     | you, with `vercel env add … --value … --yes`                 |
+
+   `--secret` formats: `NAME` (32 random bytes, base64), `NAME=base64-<bytes>`,
+   `NAME=hex-<bytes>`, `NAME=vapid` (web-push pair: `NAME_PUBLIC_KEY` + `NAME_PRIVATE_KEY`).
+   Match the format the code or docs expect (a key decoded as 32 bytes of base64 needs
+   `base64-32`).
+
+4. **Pick the steps** with `--only`: `vercel` always, plus the steps whose variables the code reads,
+   plus `domain` when the production host is known. Leave out steps the app doesn't use.
+5. **Check conflicts** before handing over: a step stops if one Vercel record of its variable spans
+   pre-production and production (`… has a record that also targets environments outside …`).
+   Tell the human which record to delete or split first.
+
+Then show the human what you found (variables, where each comes from, what already exists) and
+the commands, in order:
 
 ```bash
 # 1. Plan: reads providers, changes nothing
-/usr/local/bin/provision factory --checkout <checkout> --domain <production-host> --dry-run
-
-# 2. Apply
 /usr/local/bin/provision factory --checkout <checkout> --domain <production-host> \
-  --sender-name <App name> --email-from noreply@<verified-domain>
+  --only vercel,neon,resend,secrets,domain \
+  --sender-name <App name> --email-from login@<verified-domain> \
+  --secret BETTER_AUTH_SECRET --secret CRON_SECRET=hex-32 --dry-run
 
-# 3. Confirm idempotence: should print only "reusing" lines
-/usr/local/bin/provision factory --checkout <checkout> --domain <production-host> \
-  --sender-name <App name> --email-from noreply@<verified-domain> --on-existing reuse
+# 2. Apply: the same command without --dry-run
+
+# 3. Confirm idempotence: the same command with --on-existing reuse; only "reusing" lines
 
 # 4. Local checkout (you can run this one)
 provision env --repo <checkout> --database
 ```
 
+Keep `--only` and the `--secret` flags identical across the three runs. Afterwards, the
+third-party keys and plain settings from step 3 still need `vercel env add`.
+
 Reading the output: each step prints one block. `would …` is a dry-run plan, `exists:` /
 `reusing …` means nothing changed, `skipped:` means the factory lacks that provider, and
 `manual: …` is a follow-up the human must do by hand. Relay `manual:` lines to the human.