caracms 0.2.0 → 0.3.0

package/README.md

@@ -44,15 +44,259 @@ job of the `caracms-adopt` skill, one page at a time.
 
 Run from the site's root:
 
-- `caracms init [--write]`: add caracms to an Astro site, or bring a set-up site's skills,
-  schemas and CLAUDE.md block up to date.
+- `caracms init [--write] [--railway]`: add caracms to an Astro site, or bring a set-up
+  site's skills, schemas and CLAUDE.md block up to date. `--railway` also writes a
+  Railway setup; without it, `init` prints the variables any host needs (Hosting, below).
 - `caracms validate [--fix] [--json]`: check the content against `caracms.config.ts`.
   Exits 1 when there are errors. `--fix` runs `caracms schemas` first, then lays each
   content file out as the editor saves it (two-space JSON; only whitespace changes).
 - `caracms schemas`: write `schemas/` from the config, and give each content file its
   `$schema`.
 - `caracms media import <files…>`: put local images into the site's media bucket.
-- `caracms start`: serve the built site.
+- `caracms start`: serve the built site (Hosting, below).
 
 The skills (`caracms-content`, `caracms-blocks` and `caracms-adopt`, in `skills/`) say how
 the rest works, matching the installed version.
+
+## Hosting
+
+caracms runs on any host that keeps one Node server running behind HTTPS: Railway,
+Render, Fly, or a VPS. What every host needs:
+
+- **One instance.** The site runs as one server process (two for a moment during a
+  deploy). The drafts and several caches live in its memory. On Fly that means one
+  machine; on Render, no autoscaling.
+- **HTTPS.** Editors' sessions only travel over https, so signing in needs it.
+  `http://` works for `localhost` only, and then in a Chromium browser: Safari won't
+  keep the session cookie on `http://localhost`.
+- **A deploy whenever `main` changes.** A Publish commits to `main`; caracms never
+  deploys anything itself. Railway, Render and Coolify deploy on a push. On Fly or a
+  plain VPS, add a GitHub Action (or Coolify, or similar) that deploys on a push to
+  `main`.
+- **`SITE_URL`, the same value at build and at run time.** Set it where the site is
+  built and where it runs. A www address at one and the bare domain at the other is
+  the likeliest setup mistake.
+- **Started with `caracms start`** (the site's `npm start`). It decides each visitor's
+  IP for the limits on signing in, forms and downloads. A site started any other way
+  answers those routes with a 503:
+
+  > This site wasn't started with `caracms start`, so it can't tell its visitors apart
+  > for the limits on signing in, forms and downloads. A developer can start it with
+  > `caracms start` (the site's start script).
+
+### The variables
+
+These go on the site's server. `caracms init` prints the same list.
+
+| Name | Needed | What it's for |
+|---|---|---|
+| `GITHUB_REPO` | required | The site's repository, as owner/name. |
+| `CARACMS_GITHUB_TOKEN` | required, or the App | A fine-grained token (`github_pat_…`) on that one repo: Contents read and write, Metadata read (Deployments read is optional). Never `GITHUB_TOKEN`. |
+| `GITHUB_APP_ID` | instead of the token | The GitHub App's ID, for anyone setting up many sites. Set the token or the App, not both. |
+| `GITHUB_APP_PRIVATE_KEY` | with `GITHUB_APP_ID` | The App's private key. |
+| `SITE_URL` | required | The site's public address, https://. The same value at build and at run time. |
+| `REDIS_URL` | required | The editor's drafts and sessions: `redis://` or `rediss://` (Upstash works, in the site's region). |
+| `CARACMS_SECRET` | required | Signs the editors' sessions. A long random string. |
+| `CARACMS_CLIENT_IP_HEADER` | required off Railway, Fly and Render | Where the visitor's IP comes from: `x-real-ip` for nginx with `proxy_set_header X-Real-IP $remote_addr`, `x-forwarded-for` for Caddy, or `socket` if nothing sits in front of this server. `caracms start` won't start without it. |
+| `CARACMS_ALLOW` | for email sign-in | Who may sign in, comma-separated: addresses (`aoife@fernhill.ie`) or whole domains (`@fernhill.ie`). The client adds their own people in Site Settings. |
+| `RESEND_API_KEY` | for email sign-in | Sends the sign-in links (and form notifications) through Resend. |
+| `FORMS_FROM` | for email sign-in | The address those emails come from, on a domain verified in Resend. |
+| `CARACMS_BACKUP_USER` | for the backup sign-in | The backup sign-in, for when email sign-in links can't arrive. Both or neither. |
+| `CARACMS_BACKUP_PASSWORD` | for the backup sign-in | The backup sign-in's password. |
+| `CARACMS_RUNNING_COMMIT` | optional | The commit this server runs, where the host doesn't say (Railway and Render do). |
+| `CARACMS_DEPLOY_HOOK_URL` | optional | Try again on a publish POSTs here to redeploy the site (the host's deploy hook). |
+| `GITHUB_WEBHOOK_SECRET` | optional | For a webhook added in the repo's settings. Without it, a check once a minute does the same job. |
+
+Set email sign-in, the backup sign-in or both, or nobody can sign in.
+
+A site that uses images, video or forms needs these as well. The image variables and
+`TURNSTILE_SITE_KEY` are read at build too, so set them in both places.
+
+| Feature | Variables |
+|---|---|
+| Images (the media bucket and imgproxy) | `CARACMS_MEDIA_URL` (imgproxy's address), `IMGPROXY_KEY`, `IMGPROXY_SALT`, `S3_BUCKET`, `S3_ENDPOINT`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY` |
+| Video | `MUX_TOKEN_ID`, `MUX_TOKEN_SECRET` |
+| Form submissions (a second bucket, at the media bucket's endpoint and region) | `SUBMISSIONS_S3_BUCKET`, `SUBMISSIONS_S3_ACCESS_KEY_ID`, `SUBMISSIONS_S3_SECRET_ACCESS_KEY` |
+| Form emails | `RESEND_API_KEY`, `FORMS_FROM` (as above) |
+| Spam checks on forms | `TURNSTILE_SITE_KEY`, `TURNSTILE_SECRET_KEY` |
+| A password on the whole site before launch | `CARACMS_PRELAUNCH` |
+| Try again through Railway's API | `CARACMS_RAILWAY_TOKEN` (Railway sets the rest) |
+
+When something required is missing or wrong, the public site still serves and `/admin`
+answers 503 with a sentence saying what to fix. For example:
+
+> Both a GitHub App (GITHUB_APP_ID) and a token (CARACMS_GITHUB_TOKEN) are set: set one
+> of them, not both.
+
+> The site's public address isn't known: set SITE_URL to it, like https://fernhill.ie,
+> both when the site is built and when it runs. Without it, sign-in links can't be sent
+> and the editor's saves are refused.
+
+> Both CARACMS_DEPLOY_HOOK_URL and CARACMS_RAILWAY_TOKEN are set: Try again redeploys
+> through one of them, so set only one.
+
+If the site is opened at an address other than `SITE_URL` (say `https://fernhill.ie`
+when `SITE_URL` is `https://www.fernhill.ie`), signing in and saving are refused with:
+
+> This site's address is set to https://www.fernhill.ie (SITE_URL), but it was opened at
+> https://fernhill.ie, so signing in and saving are refused here. Open it at
+> https://www.fernhill.ie, or set SITE_URL to https://fernhill.ie where the site is built
+> and where it runs.
+
+### The visitor's IP
+
+`caracms start` takes each visitor's IP from one source, set by
+`CARACMS_CLIENT_IP_HEADER`, one of five values: a header the nearest proxy sets and
+overwrites (`x-real-ip`, `fly-client-ip` or `cf-connecting-ip`), `x-forwarded-for` (its
+last entry, the one the nearest proxy added), or `socket` for a server visitors reach
+directly. Any other value is refused. On Railway, Fly and Render it has a
+default, and a set value always wins. Anywhere else it won't start without it:
+
+> Set CARACMS_CLIENT_IP_HEADER to the header your proxy sets: x-real-ip for nginx with
+> `proxy_set_header X-Real-IP $remote_addr`, x-forwarded-for for Caddy, or socket if
+> nothing sits in front of this server.
+
+It prints where the IP comes from when it starts.
+
+### Railway
+
+- IP: `x-real-ip`, by default.
+- Commit: `RAILWAY_GIT_COMMIT_SHA`, which Railway sets.
+- `caracms init --railway` writes `.railway/railway.ts`, which describes the whole
+  project (the site, Redis, the buckets, imgproxy). Railway's own variables cover the
+  repo, so `GITHUB_REPO` isn't needed.
+- Try again: `CARACMS_RAILWAY_TOKEN`, a project token. Railway has no deploy hook.
+
+### Render
+
+- IP: `x-forwarded-for`'s last entry, by default. Render documents no header it
+  overwrites, and this default hasn't been checked against a running Render service
+  yet. Check it before relying on the sign-in limits. `*.onrender.com` sits behind
+  Cloudflare: if the last entry there is a Cloudflare address, every visitor to the
+  `onrender.com` address shares one bucket (caracms never reads `CF-Connecting-IP` on a
+  host's own address). On the site's own domain it's read as usual.
+- Commit: `RENDER_GIT_COMMIT`, which Render sets.
+- One instance: don't turn on autoscaling.
+- A web service with build command `npm ci && npm run build` and start command
+  `npm start`. Render's environment variables are there at build and at run time.
+- Try again: set `CARACMS_DEPLOY_HOOK_URL` to the service's deploy hook. Render's hook
+  takes a bare POST.
+
+### Fly
+
+- IP: `fly-client-ip`, by default. Fly's proxy sets it and replaces a forged one.
+- Commit: Fly sets none. Pass it as a build argument, and declare it in the Dockerfile
+  before the build step (`ARG CARACMS_RUNNING_COMMIT`), so the build bakes it in:
+
+  ```sh
+  fly deploy --build-arg CARACMS_RUNNING_COMMIT=$(git rev-parse HEAD)
+  ```
+
+  In a GitHub Action, use `${{ github.sha }}`.
+- `SITE_URL` at build: the variables in `fly.toml`'s `[env]` and Fly's secrets are
+  for run time only. Pass `SITE_URL` (and the image variables, if the site has images)
+  as build arguments too, declared with `ARG`.
+- One machine: `fly launch --ha=false` (or `fly scale count 1` later). Fly starts two
+  by default.
+- Deploys: Fly doesn't deploy on a push. Add a GitHub Action that runs `fly deploy` on
+  a push to `main`.
+
+### A VPS, behind Caddy or nginx
+
+- IP: Caddy adds the visitor's address to `X-Forwarded-For`, so set
+  `CARACMS_CLIENT_IP_HEADER=x-forwarded-for`. With nginx, set
+  `CARACMS_CLIENT_IP_HEADER=x-real-ip` and pass the headers on:
+
+  ```nginx
+  location / {
+    proxy_pass http://127.0.0.1:8080;
+    proxy_set_header Host $host;
+    proxy_set_header X-Real-IP $remote_addr;
+    proxy_set_header X-Forwarded-Proto $scheme;
+  }
+  ```
+
+  Caddy passes the host and the scheme on by itself: `reverse_proxy 127.0.0.1:8080`.
+- Commit: a build in a git checkout bakes in `git rev-parse HEAD`. Otherwise set
+  `CARACMS_RUNNING_COMMIT`. Coolify sets `SOURCE_COMMIT`, which caracms reads.
+- Keep the server running with systemd or similar. `caracms start` listens on `PORT`
+  (default 8080).
+- Deploys: Coolify, or a GitHub Action that pulls, builds and restarts on a push to
+  `main`.
+- If the running commit isn't one in the repo (a server built from a commit that was
+  never pushed), no publish can ever show as live on it: each one reads as "taking
+  longer than usual".
+
+### Redis
+
+Any Redis works by URL: Railway's, Render Key Value, a local one, or Upstash. For
+Upstash, use its `rediss://` URL as `REDIS_URL`, with the database in the same region
+as the site.
+
+### GitHub: a token or an App
+
+caracms writes to the site's repo with one of two credentials. Set one, not both.
+
+**A fine-grained token** is the easy one:
+
+1. On GitHub: Settings → Developer settings → Personal access tokens → Fine-grained
+   tokens → Generate new token.
+2. Resource owner: whoever owns the site's repo. Repository access: only that one repo.
+3. Permissions: Contents, read and write. Metadata, read (GitHub adds it). Deployments,
+   read, if the host writes GitHub Deployments (Railway does): it lets the editor show
+   a failed deploy.
+4. Pick an expiry (a year at most; an organisation may set less) and set the token as
+   `CARACMS_GITHUB_TOKEN`.
+
+caracms never reads `GITHUB_TOKEN` (the `gh` CLI and GitHub Actions set it), and refuses
+a classic token:
+
+> CARACMS_GITHUB_TOKEN is a classic token, which reaches every repo its owner can: use a
+> fine-grained token (it starts with github_pat_) with access to this site's repo only.
+
+What a token costs:
+
+- Its rate limits belong to the person, shared by every site on their tokens: 5,000
+  requests an hour, and 80 commits a minute and 500 an hour. A few sites edited hard at
+  the same moment can reach the hourly cap. Past a handful of sites on one person's
+  tokens, use the App.
+- Commits show as the token's owner, not the App. Commit messages still carry the
+  editor's name.
+- Publish's commits on `main`, and merges from `main`, are unsigned (GitHub signs only
+  the drafts' commits). A "require signed commits" rule on `main` needs the App.
+- No webhook comes with it. Add one by hand in the repo's settings (`POST
+  /_caracms/github`, with `GITHUB_WEBHOOK_SECRET`), or leave it: a check once a minute
+  does the same job.
+- It expires, and it's tied to a person's account. The editor warns 14 days before it
+  expires, and says so if GitHub refuses it or a permission is missing.
+
+**A GitHub App** (`GITHUB_APP_ID`, `GITHUB_APP_PRIVATE_KEY`) has its own rate limit per
+site, signs every commit, and isn't tied to a person. Use it for anyone running more than
+a handful of sites, or a repo that requires signed commits. Install it on the site's repo
+only, with the same permissions as the token.
+
+### Behind Cloudflare
+
+caracms sees through Cloudflare's proxy (the orange cloud) by itself: when the address
+from the host's header is one of Cloudflare's and the request came in on the site's own
+domain, it uses `CF-Connecting-IP` instead. On a host's own address (`*.up.railway.app`,
+`*.onrender.com`, `*.fly.dev`), localhost or a bare IP it never does. Watch for:
+
+- **`CF-Connecting-IP` can be forged around Cloudflare.** Render and Fly give the site its
+  own address too (`*.onrender.com`, `*.fly.dev`), and anyone calling it directly can set
+  the header. caracms only trusts it on requests that came from Cloudflare's addresses,
+  so leave `CARACMS_CLIENT_IP_HEADER` alone: never set it to `cf-connecting-ip` unless
+  the server accepts Cloudflare's traffic alone.
+- **Cloudflare Workers.** A Worker on any Cloudflare account can call a server that isn't
+  itself on Cloudflare, from Cloudflare's addresses, and set `CF-Connecting-IP` to
+  anything. So put the site's own domain on Cloudflare's proxy: a Worker fetching a
+  proxied domain goes through your zone, where Cloudflare sets the header itself, and
+  one fetching the host's own address is ignored (above). A domain that isn't proxied
+  (the grey cloud) can be reached this way. On a VPS, also accept only your own zone's
+  traffic: Authenticated Origin Pulls with your zone's own certificate, or a Cloudflare
+  Tunnel.
+- **No "Cache Everything" rule over `/admin` or `/_caracms/*`.**
+- **Bot Fight Mode** (the free plan's) can't exempt a path, so it may challenge GitHub's
+  webhook. The once-a-minute check covers it.
+- **Cloudflare gives up on an answer after 100 seconds.** Publish answers within 60
+  seconds and carries on if it needs longer; the editor follows it.

package/bin/caracms.mjs

@@ -16,24 +16,26 @@
 //
 // It runs from the site's root (the directory with `dist/` and `content/`) and reads,
 // once at start:
-//   PORT (default 8080), HOST (default 0.0.0.0), SITE_URL, CARACMS_PRELAUNCH
-//   (`user:password`), and content/site-settings.json → brand.name.
+//   PORT (default 8080), HOST (default 0.0.0.0), the site's address (Astro's `site` from
+//   dist/server/caracms-build.json, else SITE_URL, else RAILWAY_PUBLIC_DOMAIN),
+//   CARACMS_PRELAUNCH (`user:password`), CARACMS_CLIENT_IP_HEADER (else the detected
+//   host's default; with neither, it exits with the fix), and content/site-settings.json → brand.name.
 
 import { existsSync, readFileSync } from 'node:fs';
 import { createServer } from 'node:http';
 import { join } from 'node:path';
 import { pathToFileURL } from 'node:url';
-import { createGate, parsePrelaunch, parseSiteUrl } from './gate.mjs';
+import { createGate, describeClientIpSource, markProcessGated, parsePrelaunch, parseSiteUrl, publicAddress, readBuildInfo, readClientIpSource } from './gate.mjs';
 
 const USAGE = `Usage: caracms <command>, from the site's root
 
-  init [--write]            Add caracms to an existing Astro site. Shows the change; --write applies it.
-  schemas                   Write schemas/ from caracms.config.ts, and add $schema to content files.
-  validate [--fix] [--json] Check content against caracms.config.ts. --fix runs \`schemas\` first,
-                            then lays content files out as the editor saves them;
-                            --json prints { errors, warnings }.
-  media import <files…>     Put local images into the site's media bucket; prints JSON.
-  start                     Serve the built site (dist/server/entry.mjs).
+  init [--write] [--railway]  Add caracms to an existing Astro site. Shows the change; --write applies it.
+  schemas                     Write schemas/ from caracms.config.ts, and add $schema to content files.
+  validate [--fix] [--json]   Check content against caracms.config.ts. --fix runs \`schemas\` first,
+                              then lays content files out as the editor saves them;
+                              --json prints { errors, warnings }.
+  media import <files…>       Put local images into the site's media bucket; prints JSON.
+  start                       Serve the built site (dist/server/entry.mjs).
 `;
 
 /** The command running, for `fatal`'s sentence: set by `main`. */
@@ -64,18 +66,28 @@ async function start() {
 	if (!Number.isInteger(port) || port < 0 || port > 65535) fatal(`PORT must be a port number, not "${portText}".`);
 	const host = process.env.HOST || '0.0.0.0';
 
+	// Decision 8: the one setting that stops the public site, on purpose. Read before the site loads.
+	const ipSource = readClientIpSource(process.env);
+	if (ipSource.problem) fatal(ipSource.problem);
+
 	// A built site is a production server: the submissions store and the email sender read NODE_ENV (M6 Decision 7),
 	// and this command is the one thing that starts the built site. An explicit NODE_ENV is left as it is.
 	process.env.NODE_ENV ??= 'production';
 	// The adapter starts its own server on import unless told not to.
 	process.env.ASTRO_NODE_AUTOSTART = 'disabled';
+	// Decision 8: the built site runs in this process (the import below), and the only
+	// server it gets is `createServer(gate)`, so every request it sees has had
+	// x-caracms-client-ip replaced by the gate. The mark tells `clientIp()` so; a server
+	// started any other way lacks it and answers 503, a forged header or not.
+	markProcessGated();
 	const built = await import(pathToFileURL(entry).href);
 	if (typeof built.handler !== 'function') fatal(`${entry} exports no handler. Is the site built with @astrojs/node?`);
 	if (built.options && built.options.mode !== 'standalone') fatal(`the site's Node adapter must be in standalone mode, not "${built.options.mode}".`);
 
-	const siteUrl = process.env.SITE_URL;
+	const publicSite = publicAddress(process.env, readBuildInfo(root));
+	const siteUrl = publicSite?.value;
 	const prelaunch = process.env.CARACMS_PRELAUNCH;
-	const gate = createGate({ handler: built.handler, siteUrl, prelaunch, brandName: readBrandName(root) });
+	const gate = createGate({ handler: built.handler, clientIpSource: ipSource.source, siteUrl, prelaunch, brandName: readBrandName(root) });
 
 	const server = createServer(gate);
 	server.on('error', (err) => fatal(err.message));
@@ -83,9 +95,13 @@ async function start() {
 		const address = server.address();
 		const shown = typeof address === 'object' && address ? `${address.address.includes(':') ? `[${address.address}]` : address.address}:${address.port}` : `${host}:${port}`;
 		const site = parseSiteUrl(siteUrl);
-		const indexing = site?.indexable ? `indexable on ${site.host} only` : 'never indexable (SITE_URL is unset, or not a custom domain)';
+		const indexing = site?.indexable
+			? `indexable on ${site.host} only (${publicSite.from})`
+			: `never indexable (${!publicSite ? 'no address: SITE_URL is unset' : !site ? `${publicSite.from} isn't an http(s) address` : `the address, from ${publicSite.from}, is localhost, a host's own address or a bare IP`})`;
 		const password = parsePrelaunch(prelaunch) ? 'on (CARACMS_PRELAUNCH)' : 'off';
-		console.log(`caracms start: listening on http://${shown}\n  indexing: ${indexing}\n  pre-launch password: ${password}`);
+		console.log(
+			`caracms start: listening on http://${shown}\n  client IP: ${describeClientIpSource(ipSource)}\n  indexing: ${indexing}\n  pre-launch password: ${password}`,
+		);
 	});
 
 	const stop = () => {
@@ -137,10 +153,10 @@ async function main(argv) {
 			if (rest.length) return usage(`start takes no arguments.`);
 			return start();
 		case 'init': {
-			const f = flags('init', rest, ['--write']);
+			const f = flags('init', rest, ['--write', '--railway']);
 			if (f.error) return usage(f.error);
 			const { run } = await import('./init.mjs');
-			process.exitCode = await run({ root: process.cwd(), write: f.set.has('--write') });
+			process.exitCode = await run({ root: process.cwd(), write: f.set.has('--write'), railway: f.set.has('--railway') });
 			return;
 		}
 		case 'schemas':

package/bin/cloudflare-ips.mjs

@@ -0,0 +1,38 @@
+// Cloudflare's published IP ranges, from https://www.cloudflare.com/ips-v4/ and
+// https://www.cloudflare.com/ips-v6/ (listed at cloudflare.com/ips), fetched 2026-10-24.
+//
+// The gate (gate.mjs) uses them to tell a request that came through Cloudflare's proxy:
+// when the address its IP source gives is in one of these, it takes CF-Connecting-IP
+// instead (runs-anywhere spec, Decision 8), but only on the site's own domain: a Worker
+// on anyone's account calls from these ranges and can set CF-Connecting-IP to a host's
+// own address, so the gate's Host rule (isPlatformHost) is what keeps that out, not this
+// list. Cloudflare changes them rarely; a stale list means one shared bucket for a new
+// range. To refresh: copy both lists here and change the date above.
+
+export const CLOUDFLARE_IPV4 = [
+	'173.245.48.0/20',
+	'103.21.244.0/22',
+	'103.22.200.0/22',
+	'103.31.4.0/22',
+	'141.101.64.0/18',
+	'108.162.192.0/18',
+	'190.93.240.0/20',
+	'188.114.96.0/20',
+	'197.234.240.0/22',
+	'198.41.128.0/17',
+	'162.158.0.0/15',
+	'104.16.0.0/13',
+	'104.24.0.0/14',
+	'172.64.0.0/13',
+	'131.0.72.0/22',
+];
+
+export const CLOUDFLARE_IPV6 = [
+	'2400:cb00::/32',
+	'2606:4700::/32',
+	'2803:f800::/32',
+	'2405:b500::/32',
+	'2405:8100::/32',
+	'2a06:98c0::/29',
+	'2c0f:f248::/32',
+];