caracms 0.6.0 → 0.8.0

package/README.md

@@ -27,8 +27,10 @@ one diff to review. Then look at what `init` would do:
 npx caracms init
 ```
 
-It prints every file it would add and every edit it would make, as a diff, and changes
-nothing. When it looks right, apply it and install what it added:
+It asks where the site will run, [CaraCMS Cloud](#caracms-cloud) or self-hosted, then
+prints every file it would add and every edit it would make, as a diff, and changes
+nothing. (It asks only in a terminal: an agent or a script passes `--cloud` or
+`--self-hosted`.) When it looks right, apply it and install what it added:
 
 ```sh
 npx caracms init --write
@@ -45,9 +47,12 @@ job of the `caracms-adopt` skill, one page at a time.
 
 Run from the site's root:
 
-- `caracms init [--write] [--railway]`: add caracms to an Astro site, or bring a set-up
-  site's skills, schemas and CLAUDE.md block up to date. `--railway` also writes a
-  Railway setup; without it, `init` prints the variables any host needs (Hosting, below).
+- `caracms init [--write] [--railway] [--cloud | --self-hosted]`: add caracms to an Astro
+  site, or bring a set-up site's skills, schemas and CLAUDE.md block up to date.
+  `--cloud` or `--self-hosted` says where the site runs, written into `caracms.config.ts`
+  as `services` ([Where the site runs](#where-the-site-runs-services)); `--cloud` on a
+  self-hosted site also prints the steps of the move. `--railway` also writes a Railway
+  setup; without it, `init` prints the variables any host needs (Hosting, below).
 - `caracms validate [--fix] [--json]`: check the content against `caracms.config.ts`.
   Exits 1 when there are errors. `--fix` runs `caracms schemas` first, then lays each
   content file out as the editor saves it (two-space JSON; only whitespace changes).
@@ -59,8 +64,8 @@ Run from the site's root:
   keeping every key ([Moving a site's images in](#moving-a-sites-images-in)).
 - `caracms start`: serve the built site (Hosting, below).
 
-The skills (`caracms-content`, `caracms-blocks` and `caracms-adopt`, in `skills/`) say how
-the rest works, matching the installed version.
+The skills (`caracms-content`, `caracms-blocks`, `caracms-adopt`, `caracms-tina` and
+`caracms-hosting`, in `skills/`) say how the rest works, matching the installed version.
 
 ## Collections and addresses
 
@@ -129,7 +134,7 @@ These go on the site's server. `caracms init` prints the same list.
 
 | Name | Needed | What it's for |
 |---|---|---|
-| `CARACMS_CLOUD_KEY` | instead of the GitHub and image variables | The site's key from CaraCMS Cloud (`cck_…`). It replaces the token or App, `GITHUB_REPO` and the image variables: see [CaraCMS Cloud](#caracms-cloud). |
+| `CARACMS_CLOUD_KEY` | instead of the GitHub, image, form and email variables | The site's key from CaraCMS Cloud (`cck_…`). It replaces the token or App, `GITHUB_REPO`, the image variables, and the form and email ones: see [CaraCMS Cloud](#caracms-cloud). |
 | `GITHUB_REPO` | required, but not with the cloud key | The site's repository, as owner/name. |
 | `CARACMS_GITHUB_TOKEN` | required, or the App, or the cloud key | A fine-grained token (`github_pat_…`) on that one repo: Contents read and write, Metadata read (Deployments read is optional). Never `GITHUB_TOKEN`. |
 | `GITHUB_APP_ID` | instead of the token | The GitHub App's ID, for anyone setting up many sites. Set the token or the App, not both. |
@@ -139,8 +144,8 @@ These go on the site's server. `caracms init` prints the same list.
 | `CARACMS_SECRET` | required | Signs the editors' sessions. A long random string. |
 | `CARACMS_CLIENT_IP_HEADER` | required off Railway, Fly and Render | Where the visitor's IP comes from: `x-real-ip` for nginx with `proxy_set_header X-Real-IP $remote_addr`, `x-forwarded-for` for Caddy, or `socket` if nothing sits in front of this server. `caracms start` won't start without it. |
 | `CARACMS_ALLOW` | for email sign-in | Who may sign in, comma-separated: addresses (`aoife@fernhill.ie`) or whole domains (`@fernhill.ie`). The client adds their own people in Site Settings. |
-| `RESEND_API_KEY` | for email sign-in | Sends the sign-in links (and form notifications) through Resend. |
-| `FORMS_FROM` | for email sign-in | The address those emails come from, on a domain verified in Resend. |
+| `RESEND_API_KEY` | for email sign-in, but not with the cloud key | Sends the sign-in links (and form notifications) through Resend. |
+| `FORMS_FROM` | for email sign-in, but not with the cloud key | The address those emails come from, on a domain verified in Resend. |
 | `CARACMS_BACKUP_USER` | for the backup sign-in | The backup sign-in, for when email sign-in links can't arrive. Both or neither. |
 | `CARACMS_BACKUP_PASSWORD` | for the backup sign-in | The backup sign-in's password. |
 | `CARACMS_RUNNING_COMMIT` | optional | The commit this server runs, where the host doesn't say (Railway and Render do). |
@@ -155,9 +160,9 @@ A site that uses images, video or forms needs these as well. The image variables
 | Feature | Variables |
 |---|---|
 | Images (the media bucket and imgproxy) | `CARACMS_MEDIA_URL` (imgproxy's address), `IMGPROXY_KEY`, `IMGPROXY_SALT`, `S3_BUCKET`, `S3_ENDPOINT`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`; on CaraCMS Cloud, none of them: `CARACMS_CLOUD_KEY` alone |
-| Video | `MUX_TOKEN_ID`, `MUX_TOKEN_SECRET` |
-| Form submissions (a second bucket, at the media bucket's endpoint and region) | `SUBMISSIONS_S3_BUCKET`, `SUBMISSIONS_S3_ACCESS_KEY_ID`, `SUBMISSIONS_S3_SECRET_ACCESS_KEY` |
-| Form emails | `RESEND_API_KEY`, `FORMS_FROM` (as above) |
+| Video | `MUX_TOKEN_ID`, `MUX_TOKEN_SECRET`; on CaraCMS Cloud, none of them: `CARACMS_CLOUD_KEY` alone |
+| Form submissions (a second bucket, at the media bucket's endpoint and region) | `SUBMISSIONS_S3_BUCKET`, `SUBMISSIONS_S3_ACCESS_KEY_ID`, `SUBMISSIONS_S3_SECRET_ACCESS_KEY`; on CaraCMS Cloud, none of them: `CARACMS_CLOUD_KEY` alone |
+| Form emails | `RESEND_API_KEY`, `FORMS_FROM` (as above); on CaraCMS Cloud, none of them |
 | Spam checks on forms | `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY` |
 | A password on the whole site before launch | `CARACMS_PRELAUNCH` |
 | Try again through Railway's API | `CARACMS_RAILWAY_TOKEN` (Railway sets the rest) |
@@ -204,7 +209,8 @@ It prints where the IP comes from when it starts.
 - Commit: `RAILWAY_GIT_COMMIT_SHA`, which Railway sets.
 - `caracms init --railway` writes `.railway/railway.ts`, which describes the whole
   project (the site, Redis, the buckets, imgproxy). Railway's own variables cover the
-  repo, so `GITHUB_REPO` isn't needed.
+  repo, so `GITHUB_REPO` isn't needed. With `--cloud` it writes only the site and Redis:
+  the cloud brings the rest.
 - Try again: `CARACMS_RAILWAY_TOKEN`, a project token. Railway has no deploy hook.
 
 ### Render
@@ -317,6 +323,64 @@ only, with the same permissions as the token.
 
 ### CaraCMS Cloud
 
+#### Where the site runs: `services`
+
+`caracms.config.ts` says where the site's services (GitHub access, images, forms and
+email, video) come from, and nothing else decides it:
+
+```ts
+export default defineSite({
+	services: 'cloud', // or 'self-hosted'
+	collections: { … },
+	…
+});
+```
+
+- **`'cloud'`**: everything comes through `CARACMS_CLOUD_KEY`. The self-hosted variables
+  (the GitHub token or App and `GITHUB_REPO`, the image, video, and form and email ones
+  below) are **ignored**, never read, and each is named once in the build log and once
+  when the server starts:
+
+  > S3_BUCKET is set, but this site runs on CaraCMS Cloud (caracms.config.ts): it's ignored. Remove it once the old bucket and imgproxy are retired.
+
+  So a site can move with its old variables still set, and remove them afterwards. A
+  production build or server with no key fails with "This site runs on CaraCMS Cloud
+  (caracms.config.ts), but CARACMS_CLOUD_KEY isn't set. Make a key for it in
+  app.caracms.com." (the build fails; a running server's `/admin` answers 503 with it, and
+  the public site keeps serving). It never falls back to the self-hosted variables.
+  `astro dev` without a key is local mode, as always.
+- **`'self-hosted'`**: the site's own variables, as [the variables](#the-variables) list
+  them. A `CARACMS_CLOUD_KEY` beside them is ignored the same way.
+- **Not set** (a site from before caracms 0.8.0): the key decides, and setting it beside
+  the site's own variables is refused, as each section below says. The next `caracms init`
+  adds the field.
+
+`caracms init --cloud` or `--self-hosted` writes the field (as the first line inside
+`defineSite({`). With neither, `init` asks in a terminal and, run by an agent or a script,
+leaves a set-up site's mode unset (and says so) or stops on a new one. Once the field is
+there, `init` reads it and never asks again.
+
+**Moving a self-hosted site onto the cloud.** `caracms init --cloud` on a self-hosted site
+writes the field and prints the steps of the move, in order, each with the check that it
+worked: make the site, its key and the App's install in app.caracms.com; push the field
+(the host's build fails until the key is set: the old deployment keeps serving); copy the
+images in with [`caracms media push`](#moving-a-sites-images-in); set the key on the host;
+upload the video clips again; remove every variable the cloud replaces (it names them all,
+marking the ones it found in `.env` and `.railway/railway.ts`; it can't see the host's own
+page, so check that too); retire what the cloud replaced; and, last, cut
+`.railway/railway.ts` down to the site and Redis. `init` never edits an existing
+`.railway/railway.ts`: `railway config apply` would remove the buckets with it. With a key
+in the environment or `.env`, `init --cloud` also checks it, showing the site, its team,
+the linked repo and whether the App is installed on it, or the cloud's reason for refusing
+it (it never prints the key). The `caracms-hosting` skill walks through all of this.
+
+**Leaving.** `caracms init --self-hosted` on a cloud site writes the field back and prints
+the way back: set up the site's own services ([Hosting](#hosting)), copy the site's folder
+from the cloud's media bucket to a bucket of your own (the keys in content stay the same),
+set the variables, then remove the key.
+
+#### GitHub access on CaraCMS Cloud
+
 A site run by CaraCMS can take its GitHub access from CaraCMS Cloud instead of a token or
 an App of its own. The cloud gives the site's server a short-lived token for the site's
 repo, and only that repo, about once an hour.
@@ -329,11 +393,14 @@ repo, and only that repo, about once an hour.
    site's server.
 
 The key replaces `CARACMS_GITHUB_TOKEN` (or `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY`)
-and `GITHUB_REPO`: the repo comes from the cloud when the server starts, so setting
-`GITHUB_REPO` beside the key is refused, and so is setting a token or an App as well. It
-also replaces the image variables ([Images](#images-on-caracms-cloud), below).
+and `GITHUB_REPO`: the repo comes from the cloud when the server starts. With `services: 'cloud'`
+they're ignored; on a site whose config doesn't say, setting `GITHUB_REPO` beside the key is
+refused, and so is setting a token or an App as well. It
+also replaces the image variables ([Images](#images-on-caracms-cloud), below), the
+video ones ([Video](#video-on-caracms-cloud)), and the form and email ones
+([Forms and email](#forms-and-email-on-caracms-cloud)).
 Everything else in [the variables](#the-variables) is still the site's own: `SITE_URL`,
-`REDIS_URL`, `CARACMS_SECRET`, the sign-in variables, and the video and form variables.
+`REDIS_URL`, `CARACMS_SECRET`, `CARACMS_ALLOW`, the backup sign-in, and Turnstile.
 There's no webhook: a check once a minute keeps the editor's Publish status up to date.
 
 If the cloud says no when the server starts (the App isn't installed on the repo, the
@@ -348,7 +415,8 @@ start.
 With the key, the site's images live in its own folder of CaraCMS Cloud's media bucket and
 are served, resized and signed, from `img.caracms.cloud`. Set none of `S3_ENDPOINT`,
 `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`, `IMGPROXY_KEY`,
-`IMGPROXY_SALT` or `CARACMS_MEDIA_URL`: setting any of them beside the key is refused with
+`IMGPROXY_SALT` or `CARACMS_MEDIA_URL`. With `services: 'cloud'` they're ignored; on a site
+whose config doesn't say, setting any of them beside the key is refused with
 
 > Set CARACMS_CLOUD_KEY or the site's own media variables, not both: remove S3_BUCKET.
 
@@ -368,12 +436,46 @@ builds. A running site keeps serving whatever the cloud does.
 `caracms media import <files…>` works with the key too (from the environment or the
 site's `.env`), with the same rules and output, and its uploads count like any other.
 
+#### Forms and email on CaraCMS Cloud
+
+With the key, the site needs no submissions bucket and no Resend account. Set none of
+`SUBMISSIONS_S3_BUCKET`, `SUBMISSIONS_S3_ACCESS_KEY_ID`, `SUBMISSIONS_S3_SECRET_ACCESS_KEY`,
+`RESEND_API_KEY` or `FORMS_FROM`. With `services: 'cloud'` they're ignored; on a site whose
+config doesn't say, setting any of them beside the key is refused, at build and when the
+server starts, with
+
+> Set CARACMS_CLOUD_KEY or the site's own forms and email variables, not both: remove RESEND_API_KEY and FORMS_FROM.
+
+The site's server still makes every check on a submission (the fields, the honeypot, the
+rate limit, Turnstile) and writes its email, as before. The cloud stores the submission
+and its attachments, and sends the email once it's stored. The editor's inbox reads them
+from the cloud, unchanged.
+
+- **Form emails** come from `forms@notify.caracms.com`, named "<brand name> – <form label>"
+  (for example "Urban Return – Contact"), with Reply-To the visitor. They go to the form's
+  recipients in Site Settings, at most the first ten (the server log says when some were
+  left off).
+- **Each site sends at most 50 form emails a day** (UTC) unless CaraCMS raises it. Past
+  that, a submission is still stored, and the inbox shows it as "Not emailed: this site
+  reached its daily email limit in CaraCMS Cloud (50). Ask CaraCMS to raise it." It isn't
+  sent later.
+- **Sign-in links** come from `no-reply@accounts.caracms.com`, in the same words as before.
+
+**On a laptop**, or anywhere that isn't a production server (`NODE_ENV=production`),
+nothing goes to the cloud, even with the key: submissions are kept in
+`.caracms/submissions/` and both emails are written to the server log, so a test enquiry
+never reaches a client's inbox.
+
+A site on an older caracms keeps its own bucket and Resend until it updates and removes
+them.
+
 #### Moving a site's images in
 
 `caracms media push` copies everything under `uploads/` in the site's own bucket into its
 CaraCMS Cloud folder, under the same keys, so the content doesn't change. It's the one
-command that takes both credentials, so give the site's own `S3_*` on its command line
-rather than in `.env` (the running site refuses both):
+command that takes both credentials. The old bucket's `S3_*` can stay in `.env` once the
+config says `services: 'cloud'` (the running site ignores them); on a site whose config
+doesn't say, give them on its command line instead (it refuses both):
 
 ```bash
 S3_ENDPOINT=https://t3.storageapi.dev S3_REGION=auto S3_BUCKET=… S3_ACCESS_KEY_ID=… S3_SECRET_ACCESS_KEY=… npx caracms media push
@@ -385,6 +487,30 @@ uploader, and never changes the source bucket. It prints `{ copied, skipped, fai
 A site bigger than its storage limit stops at the limit with the cloud's sentence: ask
 CaraCMS to raise it, then run it again.
 
+#### Video on CaraCMS Cloud
+
+With the key, the site's clips are uploaded to CaraCMS Cloud's Mux and listed from the
+site's own clips there; the editor works exactly as it does with a Mux account of your
+own. Set neither `MUX_TOKEN_ID` nor `MUX_TOKEN_SECRET`. With `services: 'cloud'` they're
+ignored; on a site whose config doesn't say, setting either beside the key is refused with
+
+> Set CARACMS_CLOUD_KEY or the site's own Mux variables, not both: remove MUX_TOKEN_ID.
+
+Clips play on the site straight from Mux (`stream.mux.com`, `image.mux.com`), as before:
+the public pages never call the cloud. The minutes a site stores count towards its video
+limit (60 minutes unless CaraCMS has set another), and once it's reached, a new upload is
+refused with the cloud's sentence: "This site has reached its video limit in CaraCMS
+Cloud (60 minutes). Ask CaraCMS to raise it." A clip counts once Mux has finished with
+it, so one upload can take a site past its limit; the next is refused.
+
+**Moving a site's video in** means uploading its clips again in the editor and placing
+them where the old ones were. There's no copy command. Until a clip is replaced, the old
+one keeps playing on the site from the site's old Mux account: the page only needs its
+playback ID. In the editor it shows its poster and "A clip" (it isn't in the cloud's
+library), and saving other changes on the page leaves it as it is. Remove `MUX_TOKEN_ID`
+and `MUX_TOKEN_SECRET` when you set the key; keep the old Mux account until every clip
+has been replaced.
+
 #### Working locally
 
 **Anyone working on a real site, wherever they are, uses that site's CaraCMS Cloud**: your

package/bin/caracms.mjs

@@ -31,7 +31,9 @@ import { createGate, describeClientIpSource, markProcessGated, parsePrelaunch, p
 
 const USAGE = `Usage: caracms <command>, from the site's root
 
-  init [--write] [--railway]  Add caracms to an existing Astro site. Shows the change; --write applies it.
+  init [--write] [--railway] [--cloud | --self-hosted]
+                              Add caracms to an existing Astro site, and say where it runs (CaraCMS
+                              Cloud or self-hosted). Shows the change; --write applies it.
   schemas                     Write schemas/ from caracms.config.ts, and add $schema to content files.
   validate [--fix] [--json]   Check content against caracms.config.ts. --fix runs \`schemas\` first,
                               then lays content files out as the editor saves them;
@@ -180,10 +182,16 @@ async function main(argv) {
 			if (rest.length) return usage(`start takes no arguments.`);
 			return start();
 		case 'init': {
-			const f = flags('init', rest, ['--write', '--railway']);
+			const f = flags('init', rest, ['--write', '--railway', '--cloud', '--self-hosted']);
 			if (f.error) return usage(f.error);
 			const { run } = await import('./init.mjs');
-			process.exitCode = await run({ root: process.cwd(), write: f.set.has('--write'), railway: f.set.has('--railway') });
+			process.exitCode = await run({
+				root: process.cwd(),
+				write: f.set.has('--write'),
+				railway: f.set.has('--railway'),
+				cloud: f.set.has('--cloud'),
+				selfHosted: f.set.has('--self-hosted'),
+			});
 			return;
 		}
 		case 'schemas':