caracms 0.5.0 → 0.6.0

package/README.md

@@ -53,7 +53,10 @@ Run from the site's root:
   content file out as the editor saves it (two-space JSON; only whitespace changes).
 - `caracms schemas`: write `schemas/` from the config, and give each content file its
   `$schema`.
-- `caracms media import <files…>`: put local images into the site's media bucket.
+- `caracms media import <files…>`: put local images into the site's media bucket (or, with
+  `CARACMS_CLOUD_KEY`, its folder in CaraCMS Cloud); prints `[{ file, key, w, h, existed }]`.
+- `caracms media push`: copy a site's originals from its own bucket into CaraCMS Cloud,
+  keeping every key ([Moving a site's images in](#moving-a-sites-images-in)).
 - `caracms start`: serve the built site (Hosting, below).
 
 The skills (`caracms-content`, `caracms-blocks` and `caracms-adopt`, in `skills/`) say how
@@ -126,7 +129,7 @@ These go on the site's server. `caracms init` prints the same list.
 
 | Name | Needed | What it's for |
 |---|---|---|
-| `CARACMS_CLOUD_KEY` | instead of the GitHub variables | The site's key from CaraCMS Cloud (`cck_…`). It replaces the token or App and `GITHUB_REPO`: see [CaraCMS Cloud](#caracms-cloud). |
+| `CARACMS_CLOUD_KEY` | instead of the GitHub and image variables | The site's key from CaraCMS Cloud (`cck_…`). It replaces the token or App, `GITHUB_REPO` and the image variables: see [CaraCMS Cloud](#caracms-cloud). |
 | `GITHUB_REPO` | required, but not with the cloud key | The site's repository, as owner/name. |
 | `CARACMS_GITHUB_TOKEN` | required, or the App, or the cloud key | A fine-grained token (`github_pat_…`) on that one repo: Contents read and write, Metadata read (Deployments read is optional). Never `GITHUB_TOKEN`. |
 | `GITHUB_APP_ID` | instead of the token | The GitHub App's ID, for anyone setting up many sites. Set the token or the App, not both. |
@@ -151,7 +154,7 @@ A site that uses images, video or forms needs these as well. The image variables
 
 | Feature | Variables |
 |---|---|
-| Images (the media bucket and imgproxy) | `CARACMS_MEDIA_URL` (imgproxy's address), `IMGPROXY_KEY`, `IMGPROXY_SALT`, `S3_BUCKET`, `S3_ENDPOINT`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY` |
+| Images (the media bucket and imgproxy) | `CARACMS_MEDIA_URL` (imgproxy's address), `IMGPROXY_KEY`, `IMGPROXY_SALT`, `S3_BUCKET`, `S3_ENDPOINT`, `S3_REGION`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`; on CaraCMS Cloud, none of them: `CARACMS_CLOUD_KEY` alone |
 | Video | `MUX_TOKEN_ID`, `MUX_TOKEN_SECRET` |
 | Form submissions (a second bucket, at the media bucket's endpoint and region) | `SUBMISSIONS_S3_BUCKET`, `SUBMISSIONS_S3_ACCESS_KEY_ID`, `SUBMISSIONS_S3_SECRET_ACCESS_KEY` |
 | Form emails | `RESEND_API_KEY`, `FORMS_FROM` (as above) |
@@ -327,11 +330,11 @@ repo, and only that repo, about once an hour.
 
 The key replaces `CARACMS_GITHUB_TOKEN` (or `GITHUB_APP_ID` and `GITHUB_APP_PRIVATE_KEY`)
 and `GITHUB_REPO`: the repo comes from the cloud when the server starts, so setting
-`GITHUB_REPO` beside the key is refused, and so is setting a token or an App as well.
+`GITHUB_REPO` beside the key is refused, and so is setting a token or an App as well. It
+also replaces the image variables ([Images](#images-on-caracms-cloud), below).
 Everything else in [the variables](#the-variables) is still the site's own: `SITE_URL`,
-`REDIS_URL`, `CARACMS_SECRET`, the sign-in variables, and the image, video and form
-variables. There's no webhook: a check once a minute keeps the editor's Publish status up
-to date.
+`REDIS_URL`, `CARACMS_SECRET`, the sign-in variables, and the video and form variables.
+There's no webhook: a check once a minute keeps the editor's Publish status up to date.
 
 If the cloud says no when the server starts (the App isn't installed on the repo, the
 site has no repo yet, the key was revoked), `/admin` answers 503 with the cloud's own
@@ -340,10 +343,59 @@ says no later, while the editor is open, the editor shows the same sentence unti
 cloud gives a token again. A repo changed in CaraCMS Cloud reaches the site at its next
 start.
 
-**On a laptop**, a cloud key turns `astro dev` into live mode on the site's real repo,
-as the App's variables do. Use a key from CaraCMS Cloud's dev deployment there (set its
-address as `CARACMS_CLOUD_URL`), never a production key: the dev deployment only gives
-out tokens for throwaway repos. Leave `CARACMS_CLOUD_URL` unset everywhere else.
+#### Images on CaraCMS Cloud
+
+With the key, the site's images live in its own folder of CaraCMS Cloud's media bucket and
+are served, resized and signed, from `img.caracms.cloud`. Set none of `S3_ENDPOINT`,
+`S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY_ID`, `S3_SECRET_ACCESS_KEY`, `IMGPROXY_KEY`,
+`IMGPROXY_SALT` or `CARACMS_MEDIA_URL`: setting any of them beside the key is refused with
+
+> Set CARACMS_CLOUD_KEY or the site's own media variables, not both: remove S3_BUCKET.
+
+The image host, the bucket, the site's folder and its signing key come from the cloud:
+once when the site is built, and once when its server starts (on its first request). The
+signing key is never in a variable, a log or anything sent to a browser. Uploads go to the
+cloud through short-lived links the cloud makes for the site's server, and count towards
+the site's storage limit; a full site's uploads are refused with the cloud's sentence. The
+keys in content are the same `uploads/<hash>/<name>.<ext>` as anywhere else, so leaving
+the cloud is copying the folder to a bucket of your own.
+
+**A build needs the cloud.** If it says no (a revoked key, a team switched off), the build
+fails with its sentence; if it can't be reached, with "CaraCMS Cloud couldn't be reached,
+so the site can't be built. Try again in a minute." A site with no repo linked yet still
+builds. A running site keeps serving whatever the cloud does.
+
+`caracms media import <files…>` works with the key too (from the environment or the
+site's `.env`), with the same rules and output, and its uploads count like any other.
+
+#### Moving a site's images in
+
+`caracms media push` copies everything under `uploads/` in the site's own bucket into its
+CaraCMS Cloud folder, under the same keys, so the content doesn't change. It's the one
+command that takes both credentials, so give the site's own `S3_*` on its command line
+rather than in `.env` (the running site refuses both):
+
+```bash
+S3_ENDPOINT=https://t3.storageapi.dev S3_REGION=auto S3_BUCKET=… S3_ACCESS_KEY_ID=… S3_SECRET_ACCESS_KEY=… npx caracms media push
+```
+
+It checks each file's bytes against the hash in its key, skips anything already in the
+cloud (so it can be stopped and run again), carries over each file's size, name and
+uploader, and never changes the source bucket. It prints `{ copied, skipped, failed }`.
+A site bigger than its storage limit stops at the limit with the cloud's sentence: ask
+CaraCMS to raise it, then run it again.
+
+#### Working locally
+
+**Anyone working on a real site, wherever they are, uses that site's CaraCMS Cloud**: your
+laptop, a colleague's, a client's developer, a staging deploy. Each gets **its own key**
+for the site, labelled ("Rich's laptop", "staging") in app.caracms.com, so revoking one
+never touches the live site. Leave `CARACMS_CLOUD_URL` unset: images render from
+`img.caracms.cloud` and the library lists the site's real folder. A cloud key turns
+`astro dev` into live mode on the site's real repo, as the App's variables do. Uploads
+from a laptop land in the real folder and count towards its storage, which is harmless:
+nothing is overwritten or deleted, and nothing is live until it's published.
+`CARACMS_CLOUD_URL` is only for working on CaraCMS Cloud itself, against its dev deployment.
 
 ### Behind Cloudflare
 

package/bin/caracms.mjs

@@ -1,5 +1,5 @@
 #!/usr/bin/env node
-// The `caracms` command (M8 → Contracts): `init`, `schemas`, `validate`, `media import`
+// The `caracms` command (M8 → Contracts): `init`, `schemas`, `validate`, `media import`, `media push`
 // and `start`, each run from the site's root. Exit codes: 0 fine, 1 problems found or
 // refused, 2 bad usage.
 //
@@ -36,7 +36,10 @@ const USAGE = `Usage: caracms <command>, from the site's root
   validate [--fix] [--json]   Check content against caracms.config.ts. --fix runs \`schemas\` first,
                               then lays content files out as the editor saves them;
                               --json prints { errors, warnings }.
-  media import <files…>       Put local images into the site's media bucket; prints JSON.
+  media import <files…>       Put local images into the site's media bucket (or its CaraCMS Cloud
+                              folder, with CARACMS_CLOUD_KEY); prints JSON.
+  media push                  Copy the site's own bucket's uploads/ into its CaraCMS Cloud folder,
+                              keeping every key (S3_* and CARACMS_CLOUD_KEY); prints JSON.
   start                       Serve the built site (dist/server/entry.mjs).
 `;
 

package/bin/media.mjs

@@ -20,22 +20,43 @@
 // - Credentials: `S3_ENDPOINT`, `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY_ID` and
 //   `S3_SECRET_ACCESS_KEY`, from the environment, else the site's `.env` in the cwd.
 // - S3's SigV4 by hand, path-style requests (Railway's endpoint takes both styles).
+// - **On CaraCMS Cloud** (caracms-cloud M3 Decision 6): with `CARACMS_CLOUD_KEY` (and
+//   `CARACMS_CLOUD_URL` for another deployment; both read as the `S3_*` are) it checks
+//   and uploads through the cloud's media API instead (the list, object and upload-URL
+//   routes), with the same rules, output and exit codes. The key with any of the site's
+//   own media variables is refused with the running site's sentence: one credential.
 //
 // Output: one JSON array on stdout, `[{ "file", "key", "w", "h", "existed" }]`, `file`
 // as given. Messages go to stderr. Exit codes: 0 fine, 1 failed, 2 bad usage.
+//
+// `caracms media push` (caracms-cloud M3 Decision 6a) moves a site's originals into the
+// cloud, keeping every key: it reads the site's own bucket with `S3_*` and writes each
+// `uploads/` object into the site's cloud folder through upload URLs with
+// `CARACMS_CLOUD_KEY` (the one command that takes both). Each object's bytes are checked
+// against the hash in its key, and a key not of the upload shape is skipped as failed;
+// one already in the cloud is skipped, so it can be stopped and run again. Its metadata
+// (`w`, `h`, `name`, `by`) is carried over. Nothing in the source bucket is changed.
+// Output: `{ "copied", "skipped", "failed": [{ "key", "reason" }] }` (counts, and the
+// failures), progress on stderr; exit 0 when nothing failed, 1 otherwise (the cloud's
+// `upload_limit` stops the run there, with its sentence), 2 for bad usage.
 
 import { createHash, createHmac } from 'node:crypto';
 import { readFileSync } from 'node:fs';
 import { readFile } from 'node:fs/promises';
 import { basename, join, resolve } from 'node:path';
 
-const USAGE = 'Usage: caracms media import <files…>';
+const USAGE = 'Usage: caracms media import <files…>\n       caracms media push';
+
+/** The production API's address (src/live/cloud.ts `DEFAULT_CLOUD_URL`; a test keeps the two equal). */
+export const DEFAULT_CLOUD_URL = 'https://glad-bird-412.ap-southeast-2.convex.site';
 
 /** What the upload route takes (src/editor/types.ts: MEDIA_MAX_BYTES, MEDIA_MAX_PIXELS). */
 export const MAX_BYTES = 40_000_000;
 export const MAX_PIXELS = 50_000_000;
 
 export const ENV_NAMES = ['S3_ENDPOINT', 'S3_REGION', 'S3_BUCKET', 'S3_ACCESS_KEY_ID', 'S3_SECRET_ACCESS_KEY'];
+/** The site's own media variables, which the cloud key replaces (src/live/mode.ts `OWN_MEDIA_VARIABLES`). */
+export const OWN_MEDIA_NAMES = [...ENV_NAMES, 'IMGPROXY_KEY', 'IMGPROXY_SALT', 'CARACMS_MEDIA_URL'];
 
 // ── The key rule (upload.ts keyName, slug.ts slugify) ───────────────────────────
 
@@ -458,28 +479,51 @@ export function parseDotenv(text) {
 	return out;
 }
 
-/** The five `S3_*` values: the environment's, else `.env`'s in `cwd`. Names what's missing. */
-export function readS3Env(env, cwd) {
+/** A variable's value: the environment's, else `.env`'s in `cwd` (read once, when first needed). */
+function envReader(env, cwd) {
 	let file = null;
-	const fromFile = () => {
-		if (file) return file;
-		try {
-			file = parseDotenv(readFileSync(join(cwd, '.env'), 'utf8'));
-		} catch {
-			file = {};
+	return (name) => {
+		if (env[name]) return env[name];
+		if (!file) {
+			try {
+				file = parseDotenv(readFileSync(join(cwd, '.env'), 'utf8'));
+			} catch {
+				file = {};
+			}
 		}
-		return file;
+		return file[name] || '';
 	};
+}
+
+/** The five `S3_*` values: the environment's, else `.env`'s in `cwd`. Names what's missing. */
+export function readS3Env(env, cwd) {
+	const get = envReader(env, cwd);
 	const values = {};
 	const missing = [];
 	for (const name of ENV_NAMES) {
-		const v = env[name] || fromFile()[name];
+		const v = get(name);
 		if (v) values[name] = v;
 		else missing.push(name);
 	}
 	return { values, missing };
 }
 
+const listed = (names) => (names.length <= 1 ? names.join('') : `${names.slice(0, -1).join(', ')} and ${names[names.length - 1]}`);
+
+/**
+ * The cloud key and address, as the `S3_*` are read, or null without a key; `own` names the
+ * site's own media variables also set (the running site's "not both", src/media.ts).
+ */
+export function readCloudEnv(env, cwd) {
+	const get = envReader(env, cwd);
+	const key = get('CARACMS_CLOUD_KEY').trim();
+	if (!key) return null;
+	const url = (get('CARACMS_CLOUD_URL').trim() || DEFAULT_CLOUD_URL).replace(/\/+$/, '');
+	return { key, url, own: OWN_MEDIA_NAMES.filter((name) => get(name)) };
+}
+
+export const bothMessage = (names) => `Set CARACMS_CLOUD_KEY or the site's own media variables, not both: remove ${listed(names)}.`;
+
 // ── S3 (SigV4 by hand, path style) ─────────────────────────────────────────────
 
 const hmac = (key, data) => createHmac('sha256', key).update(data).digest();
@@ -515,10 +559,11 @@ export function signS3(creds, { method, key = '', query = {}, body, headers = {}
 }
 
 /** One signed request. Only GET (a listing), HEAD and PUT: nothing here deletes. */
-async function s3(creds, method, key, { query, body, headers } = {}) {
+async function s3(creds, method, key, { query, body, headers, bytes = false } = {}) {
 	if (method !== 'GET' && method !== 'HEAD' && method !== 'PUT') throw new Error(`No ${method} here.`);
 	const signed = signS3(creds, { method, key, query, body, headers });
-	const res = await fetch(signed.url, { method, headers: signed.headers, body: body ? new Uint8Array(body) : undefined, signal: AbortSignal.timeout(60_000) });
+	const res = await fetch(signed.url, { method, headers: signed.headers, body: body ? new Uint8Array(body) : undefined, signal: AbortSignal.timeout(bytes ? 120_000 : 60_000) });
+	if (bytes) return { res, text: '', bytes: res.ok ? Buffer.from(await res.arrayBuffer()) : (await res.body?.cancel().catch(() => {}), null) };
 	const text = method === 'HEAD' ? '' : await res.text();
 	return { res, text };
 }
@@ -533,6 +578,22 @@ async function first(creds, prefix) {
 	return m ? unxml(m[1]) : null;
 }
 
+/** Every key under a prefix, following S3's continuation tokens (`push`'s source). */
+async function listAll(creds, prefix) {
+	const keys = [];
+	let token = null;
+	for (let n = 0; n < 1000; n++) {
+		const query = { 'list-type': '2', prefix, ...(token ? { 'continuation-token': token } : {}) };
+		const { res, text } = await s3(creds, 'GET', '', { query });
+		if (!res.ok) throw new Error(`listing ${prefix} failed: ${res.status} ${text.slice(0, 300)}`);
+		for (const m of text.matchAll(/<Contents>[\s\S]*?<Key>([\s\S]*?)<\/Key>/g)) keys.push(unxml(m[1]));
+		const next = /<IsTruncated>true<\/IsTruncated>/.test(text) ? /<NextContinuationToken>([\s\S]*?)<\/NextContinuationToken>/.exec(text)?.[1] : null;
+		if (!next || next === token) break;
+		token = unxml(next);
+	}
+	return keys;
+}
+
 /** An existing object's stored `w` and `h`, when it has them. */
 async function storedSize(creds, key) {
 	const { res } = await s3(creds, 'HEAD', key);
@@ -543,6 +604,74 @@ async function storedSize(creds, key) {
 	return w && h ? { w, h } : null;
 }
 
+// ── CaraCMS Cloud's media API (the M3 spec's Contracts) ─────────────────────────
+
+/** The cloud said no, with its sentence (`message`, shown as sent), or couldn't be reached (`error: 'unreachable'`). */
+export class CloudRefused extends Error {
+	constructor(error, message) {
+		super(message);
+		this.error = error;
+	}
+}
+
+/** One call: the site key as the bearer, a JSON body for a POST, the answer's JSON or a CloudRefused. */
+async function cloudApi(cloud, method, path, body) {
+	let res;
+	let answer;
+	try {
+		res = await fetch(`${cloud.url}${path}`, {
+			method,
+			headers: { authorization: `Bearer ${cloud.key}`, accept: 'application/json', 'user-agent': 'caracms', ...(body ? { 'content-type': 'application/json' } : {}) },
+			body: body ? JSON.stringify(body) : undefined,
+			signal: AbortSignal.timeout(60_000),
+		});
+		answer = await res.json();
+	} catch {
+		throw new CloudRefused('unreachable', "CaraCMS Cloud couldn't be reached. Try again in a minute.");
+	}
+	if (!answer || typeof answer !== 'object') throw new CloudRefused('unreachable', "CaraCMS Cloud couldn't be reached. Try again in a minute.");
+	if (!res.ok) {
+		if (typeof answer.error !== 'string' || typeof answer.message !== 'string') throw new CloudRefused('unreachable', `CaraCMS Cloud answered ${res.status}.`);
+		throw new CloudRefused(answer.error, answer.message);
+	}
+	return answer;
+}
+
+/** The first key under a prefix in the site's cloud folder. */
+async function cloudFirst(cloud, prefix) {
+	const answer = await cloudApi(cloud, 'GET', `/v1/media/list?${new URLSearchParams({ prefix })}`);
+	return Array.isArray(answer.objects) && typeof answer.objects[0]?.key === 'string' ? answer.objects[0].key : null;
+}
+
+/** An object's details in the cloud, or null for `not_found`. */
+async function cloudObject(cloud, key) {
+	try {
+		return await cloudApi(cloud, 'GET', `/v1/media/object?${new URLSearchParams({ key })}`);
+	} catch (error) {
+		if (error instanceof CloudRefused && error.error === 'not_found') return null;
+		throw error;
+	}
+}
+
+/**
+ * Write a new object: an upload URL for exactly these bytes, then the PUT with the headers it
+ * names (Content-Length is fetch's own). Returns the storage's status: 200s are stored, 412
+ * means the key is already there.
+ */
+async function cloudPut(cloud, { key, body, contentType, meta }) {
+	const answer = await cloudApi(cloud, 'POST', '/v1/media/upload-url', { key, size: body.byteLength, contentType, sha256: sha256hex(body), meta });
+	if (typeof answer.url !== 'string' || !answer.headers || typeof answer.headers !== 'object') throw new CloudRefused('unreachable', 'CaraCMS Cloud gave no upload URL.');
+	const headers = Object.fromEntries(Object.entries(answer.headers).filter(([name, v]) => typeof v === 'string' && !/^(content-length|host)$/i.test(name)));
+	let res;
+	try {
+		res = await fetch(answer.url, { method: 'PUT', headers, body: new Uint8Array(body), signal: AbortSignal.timeout(120_000) });
+	} catch {
+		throw new CloudRefused('unreachable', "CaraCMS Cloud's storage couldn't be reached. Try again in a minute.");
+	}
+	await res.body?.cancel().catch(() => {});
+	return res.status;
+}
+
 // ── The command ────────────────────────────────────────────────────────────────
 
 /**
@@ -561,6 +690,15 @@ export async function run(argv, io = {}) {
 		out.write(`${USAGE}\n`);
 		return 0;
 	}
+	if (sub === 'push') {
+		const extra = rest.filter((a) => a !== '--');
+		if (extra.length > 0) {
+			say(extra[0].startsWith('-') ? `Unknown option: ${extra[0]}` : 'media push takes no arguments: it copies everything under uploads/.');
+			say(USAGE);
+			return 2;
+		}
+		return push({ env, cwd, out, say });
+	}
 	if (sub !== 'import') {
 		say(sub ? `Unknown media command: ${sub}` : 'Which media command?');
 		say(USAGE);
@@ -579,11 +717,21 @@ export async function run(argv, io = {}) {
 		return 2;
 	}
 
-	const { values: creds, missing } = readS3Env(env, cwd);
-	if (missing.length > 0) {
-		say(`Missing ${missing.join(', ')}: set ${missing.length === 1 ? 'it' : 'them'} in the environment or in this site's .env.`);
+	// The cloud key, when it's set, is the one credential (caracms-cloud M3 Decisions 6 and 19).
+	const cloud = readCloudEnv(env, cwd);
+	if (cloud && cloud.own.length > 0) {
+		say(bothMessage(cloud.own));
 		return 1;
 	}
+	let creds = null;
+	if (!cloud) {
+		const { values, missing } = readS3Env(env, cwd);
+		if (missing.length > 0) {
+			say(`Missing ${missing.join(', ')}: set ${missing.length === 1 ? 'it' : 'them'} in the environment or in this site's .env (or set CARACMS_CLOUD_KEY for a site on CaraCMS Cloud).`);
+			return 1;
+		}
+		creds = values;
+	}
 
 	// Every file read and checked before anything is written.
 	const items = [];
@@ -630,33 +778,163 @@ export async function run(argv, io = {}) {
 	try {
 		for (const item of items) {
 			const { file, bytes, key, hash, sniffed } = item;
-			const existing = await first(creds, `uploads/${hash}/`);
+			const existing = cloud ? await cloudFirst(cloud, `uploads/${hash}/`) : await first(creds, `uploads/${hash}/`);
 			if (existing) {
-				const size = (await storedSize(creds, existing)) ?? { w: sniffed.w, h: sniffed.h };
+				const size = (cloud ? sizeOf(await cloudObject(cloud, existing)) : await storedSize(creds, existing)) ?? { w: sniffed.w, h: sniffed.h };
 				results.push({ file, key: existing, w: size.w, h: size.h, existed: true });
 				say(`${file}: already there, ${existing}`);
 				continue;
 			}
 			const name = cleanName(basename(file)) || key.slice(key.lastIndexOf('/') + 1);
-			const { res, text } = await s3(creds, 'PUT', key, {
-				body: bytes,
-				headers: {
-					'content-type': sniffed.mime,
-					'x-amz-meta-w': String(sniffed.w),
-					'x-amz-meta-h': String(sniffed.h),
-					'x-amz-meta-name': encodeURIComponent(name),
-					'x-amz-meta-by': 'import',
-				},
-			});
-			if (!res.ok) throw new Error(`writing ${key} failed: ${res.status} ${text.slice(0, 300)}`);
+			if (cloud) {
+				const status = await cloudPut(cloud, { key, body: bytes, contentType: sniffed.mime, meta: { w: sniffed.w, h: sniffed.h, name, by: 'import' } });
+				// 412: the key (and so these bytes) is already there; it went in between the check and the write.
+				if (status === 412) {
+					results.push({ file, key, w: sniffed.w, h: sniffed.h, existed: true });
+					say(`${file}: already there, ${key}`);
+					continue;
+				}
+				if (status < 200 || status >= 300) throw new Error(`writing ${key} failed: the storage answered ${status}`);
+			} else {
+				const { res, text } = await s3(creds, 'PUT', key, {
+					body: bytes,
+					headers: {
+						'content-type': sniffed.mime,
+						'x-amz-meta-w': String(sniffed.w),
+						'x-amz-meta-h': String(sniffed.h),
+						'x-amz-meta-name': encodeURIComponent(name),
+						'x-amz-meta-by': 'import',
+					},
+				});
+				if (!res.ok) throw new Error(`writing ${key} failed: ${res.status} ${text.slice(0, 300)}`);
+			}
 			results.push({ file, key, w: sniffed.w, h: sniffed.h, existed: false });
 			say(`${file}: ${key} (${sniffed.w}×${sniffed.h})`);
 		}
 	} catch (error) {
-		say(`The bucket refused or didn't answer: ${error?.message ?? error}`);
+		// The cloud's own sentence as sent (a full site, a revoked key); anything else as before.
+		say(error instanceof CloudRefused ? error.message : `The bucket refused or didn't answer: ${error?.message ?? error}`);
 		if (results.length > 0) say(`These went in before it stopped: ${JSON.stringify(results)}`);
 		return 1;
 	}
 	out.write(`${JSON.stringify(results, null, 2)}\n`);
 	return 0;
 }
+
+/** A cloud object's `w` and `h`, when it has them. */
+const sizeOf = (object) => (object && Number.isInteger(object.w) && object.w > 0 && Number.isInteger(object.h) && object.h > 0 ? { w: object.w, h: object.h } : null);
+
+// ── media push (caracms-cloud M3 Decision 6a) ──────────────────────────────────
+
+/** The upload route's key shape (the M3 spec's Decision 15): the hash, the slugified name, an extension the library takes. */
+const UPLOAD_KEY = /^uploads\/([0-9a-f]{12})\/([\p{Ll}\p{Lo}\p{N}]+(?:-[\p{Ll}\p{Lo}\p{N}]+)*)\.(jpg|png|webp|avif|heic|heif|gif|svg|pdf|docx|doc|xlsx|xls|pptx|ppt|csv|zip)$/u;
+/** A type for an object whose source has none (src/editor/server/media/sniff.ts and src/files.ts name the same). */
+const TYPES = {
+	jpg: 'image/jpeg', png: 'image/png', webp: 'image/webp', avif: 'image/avif', heic: 'image/heic', heif: 'image/heif', gif: 'image/gif', svg: 'image/svg+xml',
+	pdf: 'application/pdf', docx: 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', doc: 'application/msword',
+	xlsx: 'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet', xls: 'application/vnd.ms-excel',
+	pptx: 'application/vnd.openxmlformats-officedocument.presentationml.presentation', ppt: 'application/vnd.ms-powerpoint', csv: 'text/csv', zip: 'application/zip',
+};
+/** Refusals about one key: report it and carry on. Any other (the cap, a revoked key, the storage away) stops the run. */
+const PER_KEY = new Set(['bad_key', 'bad_request', 'too_large', 'exists']);
+
+const decoded = (v) => {
+	if (!v) return null;
+	try {
+		return decodeURIComponent(v);
+	} catch {
+		return v;
+	}
+};
+const metaSide = (v) => (v != null && /^\d+$/.test(v) && Number(v) > 0 ? Number(v) : undefined);
+
+async function push({ env, cwd, out, say }) {
+	const cloud = readCloudEnv(env, cwd);
+	const { values: creds, missing } = readS3Env(env, cwd);
+	if (!cloud || missing.length > 0) {
+		const need = [...(cloud ? [] : ['CARACMS_CLOUD_KEY']), ...missing];
+		say(`Missing ${listed(need)}: media push reads the site's own bucket (S3_*) and writes to its CaraCMS Cloud folder (CARACMS_CLOUD_KEY). Set ${need.length === 1 ? 'it' : 'them'} in the environment or in this site's .env.`);
+		return 1;
+	}
+
+	let copied = 0;
+	let skipped = 0;
+	const failed = [];
+	const done = (code) => {
+		out.write(`${JSON.stringify({ copied, skipped, failed }, null, 2)}\n`);
+		return code;
+	};
+
+	let keys;
+	try {
+		keys = (await listAll(creds, 'uploads/')).sort();
+	} catch (error) {
+		say(`The site's bucket refused or didn't answer: ${error?.message ?? error}`);
+		return 1;
+	}
+	say(`${keys.length} object${keys.length === 1 ? '' : 's'} under uploads/ in ${creds.S3_BUCKET}.`);
+
+	for (const key of keys) {
+		const shape = UPLOAD_KEY.exec(key);
+		if (!shape) {
+			failed.push({ key, reason: "It isn't of the upload key's shape (uploads/<12 hex>/<name>.<ext>, an extension the library takes)." });
+			say(`failed ${key}: not of the upload key's shape`);
+			continue;
+		}
+		try {
+			if (await cloudObject(cloud, key)) {
+				skipped++;
+				say(`skipped ${key} (already in the cloud)`);
+				continue;
+			}
+			const { res, bytes } = await s3(creds, 'GET', key, { bytes: true });
+			if (!bytes) {
+				failed.push({ key, reason: `The site's bucket answered ${res.status} for it.` });
+				say(`failed ${key}: the site's bucket answered ${res.status}`);
+				continue;
+			}
+			if (bytes.byteLength > MAX_BYTES) {
+				failed.push({ key, reason: `It's ${(bytes.byteLength / 1e6).toFixed(1)} MB; the library takes up to ${MAX_BYTES / 1e6} MB.` });
+				say(`failed ${key}: over ${MAX_BYTES / 1e6} MB`);
+				continue;
+			}
+			if (sha256hex(bytes).slice(0, 12) !== shape[1]) {
+				failed.push({ key, reason: "Its bytes don't match the hash in its key, so it wasn't sent." });
+				say(`failed ${key}: its bytes don't match the hash in its key`);
+				continue;
+			}
+			const h = res.headers;
+			const meta = {
+				...(metaSide(h.get('x-amz-meta-w')) ? { w: metaSide(h.get('x-amz-meta-w')) } : {}),
+				...(metaSide(h.get('x-amz-meta-h')) ? { h: metaSide(h.get('x-amz-meta-h')) } : {}),
+				name: decoded(h.get('x-amz-meta-name')) || key.slice(key.lastIndexOf('/') + 1),
+				by: decoded(h.get('x-amz-meta-by')),
+			};
+			const contentType = h.get('content-type') && h.get('content-type') !== 'application/octet-stream' ? h.get('content-type') : TYPES[shape[3]];
+			const status = await cloudPut(cloud, { key, body: bytes, contentType, meta });
+			if (status === 412) {
+				skipped++;
+				say(`skipped ${key} (already in the cloud)`);
+			} else if (status >= 200 && status < 300) {
+				copied++;
+				say(`copied ${key}`);
+			} else {
+				failed.push({ key, reason: `The cloud's storage answered ${status}.` });
+				say(`failed ${key}: the cloud's storage answered ${status}`);
+			}
+		} catch (error) {
+			if (error instanceof CloudRefused && PER_KEY.has(error.error)) {
+				failed.push({ key, reason: error.message });
+				say(`failed ${key}: ${error.message}`);
+				continue;
+			}
+			// The cap (upload_limit), a key that stopped working, the cloud or its storage away: stop here.
+			const reason = error instanceof CloudRefused ? error.message : `The site's bucket didn't answer: ${error?.message ?? error}`;
+			failed.push({ key, reason });
+			say(`Stopped at ${key}, after copying ${copied}: ${reason}`);
+			return done(1);
+		}
+	}
+	say(`Done: ${copied} copied, ${skipped} already there, ${failed.length} failed.`);
+	return done(failed.length === 0 ? 0 : 1);
+}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "caracms",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "description": "A visual CMS for Astro sites, with content in git.",
   "license": "MIT",
   "type": "module",

package/skills/caracms-adopt/SKILL.md

@@ -331,7 +331,9 @@ editor's uploads are) and prints one JSON array on stdout:
 ```
 
 - It reads `S3_ENDPOINT`, `S3_REGION`, `S3_BUCKET`, `S3_ACCESS_KEY_ID` and
-  `S3_SECRET_ACCESS_KEY` from the environment, else the site's `.env`.
+  `S3_SECRET_ACCESS_KEY` from the environment, else the site's `.env`. On CaraCMS Cloud it
+  reads `CARACMS_CLOUD_KEY` instead (the same places) and uploads to the site's cloud
+  folder, with the same output; with both set it refuses.
 - The same image again writes nothing and prints its key with `"existed": true`. It never
   deletes anything.
 - JPEG, PNG, WebP, AVIF, HEIC/HEIF, GIF and SVG, up to 40 MB and 50 megapixels. If any file