codex-snapshots 0.1.0 → 0.1.1

package/deploy/aliyun/README.md

@@ -0,0 +1,311 @@
+# Deploy the Share API on Alibaba Cloud ECS
+
+This deployment runs `codex-snapshot-share` on `127.0.0.1:8787`, exposes it through Nginx + HTTPS, and lets the GitHub Pages site list and render public sessions.
+
+Replace these examples before running commands:
+
+- `snapshots.example.com`: your public API domain
+- `your-client-id` / `your-client-secret`: your GitHub OAuth App credentials
+- `your-github-login`: your GitHub login; this account can delete any shared session
+- `https://ffffhx.github.io/codex-snapshots/`: the public static site
+
+## 1. Prepare Alibaba Cloud
+
+1. Point a DNS `A` record for `snapshots.example.com` to the ECS public IP.
+2. In the ECS security group, allow inbound TCP `80` and `443`.
+3. Do not expose port `8787` publicly; the service binds to `127.0.0.1`.
+
+## 2. Install host dependencies
+
+On the ECS host, install Node.js 18 or newer, Nginx, Certbot, Git, OpenSSL, and rsync.
+
+Ubuntu/Debian example:
+
+```bash
+sudo apt-get update
+sudo apt-get install -y nginx certbot python3-certbot-nginx git openssl rsync
+node --version
+```
+
+Alibaba Cloud Linux/RHEL-style systems usually use `dnf` or `yum` packages instead of `apt`.
+
+You can also let the deploy helper install the common dependencies on Ubuntu/Debian, Alibaba Cloud Linux, or RHEL-style hosts:
+
+```bash
+deploy/aliyun/deploy-to-ecs.sh \
+  --ssh root@1.2.3.4 \
+  --domain snapshots.example.com \
+  --install-deps \
+  --configure-local
+```
+
+The helper installs Node.js 20 when the host does not already have Node.js 18+.
+
+## 3. Deploy from your local machine
+
+Optionally create a local deployment config so you do not have to type the long command each time:
+
+```bash
+cp deploy/aliyun/deploy.env.example deploy/aliyun/deploy.env
+$EDITOR deploy/aliyun/deploy.env
+deploy/aliyun/deploy-to-ecs.sh --config deploy/aliyun/deploy.env --dry-run
+node deploy/aliyun/doctor.mjs --config deploy/aliyun/deploy.env
+deploy/aliyun/deploy-to-ecs.sh --config deploy/aliyun/deploy.env
+```
+
+`deploy/aliyun/deploy.env` is ignored by git because it contains host details, OAuth secrets, and optional legacy token values.
+The dry run validates that you replaced template values like `root@1.2.3.4`, `snapshots.example.com`, `change-me`, and `you@example.com` before anything connects to ECS.
+The doctor command adds local readiness checks for DNS, local `ssh`/`rsync`/`node`, GitHub CLI when Pages auto-config is enabled, auth config, and the deploy dry run. Pass `--offline` if DNS has not propagated yet.
+The deploy and install scripts also exclude local secret files such as `.env`, `deploy/aliyun/deploy.env`, private-key files, and local `backups/` from the ECS rsync payload.
+
+Run the preflight check before deploying:
+
+```bash
+node deploy/aliyun/preflight.mjs \
+  --domain snapshots.example.com \
+  --ssh root@1.2.3.4
+```
+
+This checks DNS, SSH access, Node.js 18+, Nginx, Certbot, rsync, OpenSSL, systemd, and root/passwordless-sudo privileges on the ECS host.
+When you deploy with `--issue-cert`, the deploy helper also checks that Certbot has the Nginx plugin.
+
+After DNS points at the ECS public IP and SSH works, deploy the current checkout from your local machine:
+
+```bash
+SNAPSHOT_GITHUB_CLIENT_ID=your-client-id \
+SNAPSHOT_GITHUB_CLIENT_SECRET=your-client-secret \
+SNAPSHOT_SESSION_SECRET="$(openssl rand -base64 48)" \
+SNAPSHOT_GITHUB_OWNER_LOGIN=your-github-login \
+deploy/aliyun/deploy-to-ecs.sh \
+  --ssh root@1.2.3.4 \
+  --domain snapshots.example.com \
+  --configure-local
+```
+
+This runs the preflight checks, rsyncs the repository to the ECS host, installs the systemd service, and writes the Nginx bootstrap config. If you have already checked the host and want to skip preflight, pass `--no-preflight`.
+GitHub OAuth makes publish/delete use GitHub login instead of a shared token. The site owner in `SNAPSHOT_GITHUB_OWNER_LOGIN` can delete any shared session; other GitHub users can delete only their own sessions.
+Keep the same `SNAPSHOT_SESSION_SECRET` across redeploys so existing login cookies remain valid. If you omit it while OAuth vars are present, the deploy helper generates one for that deploy.
+
+After port `80` works, issue HTTPS and switch to the HTTPS proxy template:
+
+```bash
+deploy/aliyun/deploy-to-ecs.sh \
+  --ssh root@1.2.3.4 \
+  --domain snapshots.example.com \
+  --issue-cert \
+  --email you@example.com
+```
+
+The deploy script verifies the ECS API at the end. In GitHub OAuth mode it skips command-line token publishing because writes require a browser GitHub login; configure Pages in step 6, then run the public loop verification in step 7.
+
+You can also let the deploy command configure GitHub Pages and the local publisher in the same run:
+
+```bash
+deploy/aliyun/deploy-to-ecs.sh \
+  --ssh root@1.2.3.4 \
+  --domain snapshots.example.com \
+  --install-deps \
+  --issue-cert \
+  --email you@example.com \
+  --configure-pages \
+  --wait-pages \
+  --configure-local \
+  --reinstall-daemon
+```
+
+`--configure-pages` requires GitHub CLI auth on your local machine. With `--wait-pages`, the deploy script waits for the Pages workflow and then runs the full public verification.
+
+## 4. Install the service manually
+
+Clone this repository on the ECS host, then run the installer from the repository root:
+
+```bash
+git clone https://github.com/ffffhx/codex-snapshots.git
+cd codex-snapshots
+
+sudo DOMAIN=snapshots.example.com \
+  SNAPSHOT_GITHUB_CLIENT_ID=your-client-id \
+  SNAPSHOT_GITHUB_CLIENT_SECRET=your-client-secret \
+  SNAPSHOT_SESSION_SECRET="$(openssl rand -base64 48)" \
+  SNAPSHOT_GITHUB_OWNER_LOGIN=your-github-login \
+  SNAPSHOT_SHARE_SITE_URL=https://ffffhx.github.io/codex-snapshots/ \
+  SNAPSHOT_SHARE_PUBLIC_API_URL=https://snapshots.example.com \
+  deploy/aliyun/install-share-api.sh
+```
+
+The first run installs an HTTP bootstrap Nginx config if a TLS certificate is not present yet.
+
+## 5. Enable HTTPS manually
+
+Issue a certificate after DNS and port `80` are working:
+
+```bash
+sudo certbot --nginx -d snapshots.example.com
+```
+
+Then re-run the installer so it switches Nginx to the committed HTTPS proxy template:
+
+```bash
+sudo DOMAIN=snapshots.example.com \
+  SNAPSHOT_GITHUB_CLIENT_ID=your-client-id \
+  SNAPSHOT_GITHUB_CLIENT_SECRET=your-client-secret \
+  SNAPSHOT_SESSION_SECRET="$(openssl rand -base64 48)" \
+  SNAPSHOT_GITHUB_OWNER_LOGIN=your-github-login \
+  SNAPSHOT_SHARE_SITE_URL=https://ffffhx.github.io/codex-snapshots/ \
+  SNAPSHOT_SHARE_PUBLIC_API_URL=https://snapshots.example.com \
+  deploy/aliyun/install-share-api.sh
+```
+
+Verify:
+
+```bash
+curl https://snapshots.example.com/api/snapshots/health
+curl https://snapshots.example.com/api/snapshots
+```
+
+## 6. Connect GitHub Pages
+
+Set this repository variable in GitHub:
+
+```text
+CODEX_SNAPSHOTS_PUBLIC_API_URL=https://snapshots.example.com
+```
+
+You can set it and trigger the Pages workflow with GitHub CLI:
+
+```bash
+deploy/aliyun/configure-github-pages-api.sh \
+  --api-url https://snapshots.example.com \
+  --repo ffffhx/codex-snapshots
+```
+
+The helper refuses placeholder or local-only API URLs so the public site does not get configured to fetch from `127.0.0.1` or `snapshots.example.com`.
+The Pages workflow runs the same validation before writing `site/assets/config.js`; empty is allowed, but invalid, example, localhost, and private-network API URLs fail the deploy step.
+
+The Pages workflow writes `site/assets/config.js`, so the public homepage fetches:
+
+```text
+https://snapshots.example.com/api/snapshots
+```
+
+If this variable is missing, the public site shows that the share API is not configured instead of attempting to fetch from each visitor's `127.0.0.1`.
+
+## 7. Verify the public loop
+
+After the Pages workflow finishes, run the read-only checks:
+
+```bash
+node deploy/aliyun/verify-public-share.mjs \
+  --api-url https://snapshots.example.com \
+  --site-url https://ffffhx.github.io/codex-snapshots/
+```
+
+This proves:
+
+- the ECS API is reachable over HTTPS
+- public reads work
+- CORS allows the public GitHub Pages site
+- the static site config points at the ECS API
+
+GitHub OAuth publishing is verified from the browser: open the local viewer, click “发布分享”, log in with GitHub when prompted, and confirm the shared session appears on the public site. For legacy token mode only, add `--publish --token <same-token>` to create one small verification snapshot from the command line.
+
+To verify only the ECS API before Pages has deployed, add `--skip-site-config`.
+After configuring the local publisher, add `--check-local-config` to prove that `~/.codex-snapshots-agent.json` also points at the same Aliyun API and public site.
+
+## 8. Publish from your local viewer
+
+Configure your local viewer with the public API and site:
+
+```bash
+SNAPSHOT_SHARE_API_URL=https://snapshots.example.com \
+SNAPSHOT_SHARE_SITE_URL=https://ffffhx.github.io/codex-snapshots/ \
+deploy/aliyun/configure-local-publisher.sh
+```
+
+The helper rejects placeholder and local-only API URLs before it writes `~/.codex-snapshots-agent.json`.
+That file stores the public API/site URLs, so the local viewer knows which remote share API to use. With GitHub OAuth, the browser session cookie handles publish/delete auth after login.
+
+Verify the local publisher config:
+
+```bash
+node deploy/aliyun/verify-public-share.mjs \
+  --api-url https://snapshots.example.com \
+  --site-url https://ffffhx.github.io/codex-snapshots/ \
+  --skip-site-config \
+  --check-local-config
+```
+
+Then start the local viewer:
+
+```bash
+pnpm snapshot serve --port 4321
+```
+
+For the macOS LaunchAgent, let the helper reinstall the daemon with the public API and site URLs:
+
+```bash
+SNAPSHOT_SHARE_API_URL=https://snapshots.example.com \
+SNAPSHOT_SHARE_SITE_URL=https://ffffhx.github.io/codex-snapshots/ \
+deploy/aliyun/configure-local-publisher.sh --reinstall-daemon
+```
+
+The local publisher config can also be edited manually in `~/.codex-snapshots-agent.json`:
+
+```json
+{
+  "snapshotShareApiUrl": "https://snapshots.example.com",
+  "snapshotShareSiteUrl": "https://ffffhx.github.io/codex-snapshots"
+}
+```
+
+## Operations
+
+Useful ECS commands:
+
+```bash
+sudo systemctl status codex-snapshot-share
+sudo journalctl -u codex-snapshot-share -f
+sudo nginx -t
+sudo systemctl reload nginx
+sudo cp /var/lib/codex-snapshots/shares.json ~/shares.json.backup
+```
+
+You can run the same health checks from your local machine:
+
+```bash
+deploy/aliyun/check-ecs-status.sh \
+  --ssh root@1.2.3.4 \
+  --domain snapshots.example.com
+```
+
+This checks systemd status, local health, install-file permissions, that port `8787` is only listening locally, Nginx syntax/config, and the public `/api/snapshots/health` endpoint.
+
+Back up the share data from ECS to your local checkout:
+
+```bash
+deploy/aliyun/backup-share-data.sh \
+  --ssh root@1.2.3.4
+```
+
+Restore a local backup to ECS:
+
+```bash
+deploy/aliyun/restore-share-data.sh \
+  --ssh root@1.2.3.4 \
+  --file backups/codex-snapshots/shares-20260101T000000Z.json
+```
+
+The restore script validates the JSON locally and saves the previous remote file next to the live data before replacing it.
+
+The share data is stored in:
+
+```text
+/var/lib/codex-snapshots/shares.json
+```
+
+Local publisher checks:
+
+```bash
+curl https://snapshots.example.com/api/snapshots/health
+curl https://snapshots.example.com/api/snapshots
+```

package/deploy/aliyun/backup-share-data.sh

@@ -0,0 +1,109 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SSH_TARGET=""
+SSH_OPTS=()
+REMOTE_FILE="/var/lib/codex-snapshots/shares.json"
+OUTPUT_DIR="backups/codex-snapshots"
+
+usage() {
+  cat <<'EOF'
+Usage:
+  deploy/aliyun/backup-share-data.sh --ssh root@1.2.3.4
+
+Options:
+  --ssh TARGET          SSH target, for example root@1.2.3.4.
+  --identity-file FILE  SSH private key for the ECS host.
+  --port PORT           SSH port.
+  --remote-file FILE    Remote share data file. Defaults to /var/lib/codex-snapshots/shares.json.
+  --output-dir DIR      Local backup directory. Defaults to backups/codex-snapshots.
+  -h, --help            Show help.
+EOF
+}
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --ssh)
+      SSH_TARGET="${2:-}"
+      shift 2
+      ;;
+    --identity-file)
+      SSH_OPTS+=("-i" "${2:-}")
+      shift 2
+      ;;
+    --port)
+      SSH_OPTS+=("-p" "${2:-}")
+      shift 2
+      ;;
+    --remote-file)
+      REMOTE_FILE="${2:-}"
+      shift 2
+      ;;
+    --output-dir)
+      OUTPUT_DIR="${2:-}"
+      shift 2
+      ;;
+    -h|--help)
+      usage
+      exit 0
+      ;;
+    *)
+      echo "Unknown option: $1" >&2
+      usage >&2
+      exit 1
+      ;;
+  esac
+done
+
+if [[ -z "${SSH_TARGET}" ]]; then
+  echo "Missing --ssh target." >&2
+  usage >&2
+  exit 1
+fi
+
+shell_quote() {
+  printf "'"
+  printf "%s" "$1" | sed "s/'/'\"'\"'/g"
+  printf "'"
+}
+
+mkdir -p "${OUTPUT_DIR}"
+timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
+output_file="${OUTPUT_DIR}/shares-${timestamp}.json"
+tmp_file="${output_file}.tmp"
+remote_file_q="$(shell_quote "${REMOTE_FILE}")"
+
+cleanup() {
+  rm -f "${tmp_file}"
+}
+trap cleanup EXIT
+
+remote_cmd=$(
+  cat <<EOF
+set -e
+if [ "\$(id -u)" -eq 0 ]; then
+  cat ${remote_file_q}
+else
+  sudo -n cat ${remote_file_q}
+fi
+EOF
+)
+
+SSH_CMD=(ssh)
+if ((${#SSH_OPTS[@]})); then
+  SSH_CMD+=("${SSH_OPTS[@]}")
+fi
+
+"${SSH_CMD[@]}" "${SSH_TARGET}" "${remote_cmd}" > "${tmp_file}"
+
+node -e '
+const { readFileSync } = require("node:fs");
+const file = process.argv[1];
+JSON.parse(readFileSync(file, "utf8"));
+' "${tmp_file}"
+
+mv "${tmp_file}" "${output_file}"
+chmod 0600 "${output_file}"
+trap - EXIT
+
+echo "Backed up share data to ${output_file}"

package/deploy/aliyun/check-ecs-status.sh

@@ -0,0 +1,149 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SSH_TARGET=""
+SSH_OPTS=()
+DOMAIN=""
+API_URL="${SNAPSHOT_SHARE_PUBLIC_API_URL:-${SNAPSHOT_SHARE_API_URL:-}}"
+
+usage() {
+  cat <<'EOF'
+Usage:
+  deploy/aliyun/check-ecs-status.sh --ssh root@1.2.3.4 --domain snapshots.example.com
+
+Options:
+  --ssh TARGET          SSH target, for example root@1.2.3.4.
+  --domain DOMAIN      Public API domain. Used to check https://DOMAIN/api/snapshots/health.
+  --api-url URL        Public API URL. Overrides --domain for public health check.
+  --identity-file FILE SSH private key for the ECS host.
+  --port PORT          SSH port.
+  -h, --help           Show help.
+EOF
+}
+
+while [[ $# -gt 0 ]]; do
+  case "$1" in
+    --ssh)
+      SSH_TARGET="${2:-}"
+      shift 2
+      ;;
+    --domain)
+      DOMAIN="${2:-}"
+      shift 2
+      ;;
+    --api-url)
+      API_URL="${2:-}"
+      shift 2
+      ;;
+    --identity-file)
+      SSH_OPTS+=("-i" "${2:-}")
+      shift 2
+      ;;
+    --port)
+      SSH_OPTS+=("-p" "${2:-}")
+      shift 2
+      ;;
+    -h|--help)
+      usage
+      exit 0
+      ;;
+    *)
+      echo "Unknown option: $1" >&2
+      usage >&2
+      exit 1
+      ;;
+  esac
+done
+
+if [[ -z "${SSH_TARGET}" ]]; then
+  echo "Missing --ssh target." >&2
+  usage >&2
+  exit 1
+fi
+
+if [[ -z "${API_URL}" && -n "${DOMAIN}" ]]; then
+  API_URL="https://${DOMAIN}"
+fi
+
+remote_cmd='
+set -e
+PATH="$PATH:/usr/local/sbin:/usr/sbin:/sbin"
+
+run_root() {
+  if [ "$(id -u)" -eq 0 ]; then
+    "$@"
+  else
+    sudo -n "$@"
+  fi
+}
+
+echo "== systemd =="
+systemctl is-enabled codex-snapshot-share.service || true
+systemctl is-active codex-snapshot-share.service || true
+systemctl --no-pager --lines=12 status codex-snapshot-share.service || true
+
+echo
+echo "== local health =="
+curl --fail --show-error --silent --max-time 8 http://127.0.0.1:8787/api/snapshots/health
+echo
+
+echo
+echo "== install files =="
+run_root test -f /etc/codex-snapshots/share-api.env
+run_root test -f /etc/systemd/system/codex-snapshot-share.service
+run_root test -d /var/lib/codex-snapshots
+run_root test -f /etc/nginx/conf.d/codex-snapshots.conf
+if command -v stat >/dev/null 2>&1; then
+  run_root stat -c "env %a %U:%G %n" /etc/codex-snapshots/share-api.env
+  run_root stat -c "state %a %U:%G %n" /var/lib/codex-snapshots
+fi
+
+echo
+echo "== local bind =="
+if command -v ss >/dev/null 2>&1; then
+  ss -ltn | grep ":8787" || {
+    echo "share API is not listening on port 8787" >&2
+    exit 21
+  }
+  if ss -ltn | grep ":8787" | grep -Eq "0\\.0\\.0\\.0:8787|\\[::\\]:8787|\\*:8787"; then
+    echo "share API port 8787 is exposed beyond localhost" >&2
+    exit 22
+  fi
+else
+  echo "ss command not found; skipped bind-scope check"
+fi
+
+echo
+echo "== nginx =="
+run_root nginx -t
+run_root grep -q "proxy_pass http://127.0.0.1:8787" /etc/nginx/conf.d/codex-snapshots.conf
+'
+
+SSH_CMD=(ssh)
+if ((${#SSH_OPTS[@]})); then
+  SSH_CMD+=("${SSH_OPTS[@]}")
+fi
+
+"${SSH_CMD[@]}" "${SSH_TARGET}" "${remote_cmd}"
+
+if [[ -n "${API_URL}" ]]; then
+  echo
+  echo "== public health =="
+  HEALTH_PAYLOAD="$(curl --fail --show-error --silent --max-time 12 "${API_URL%/}/api/snapshots/health")"
+  printf "%s\n" "${HEALTH_PAYLOAD}"
+  if printf "%s" "${HEALTH_PAYLOAD}" | grep -q '"storage"'; then
+    echo "public health response exposes the storage path" >&2
+    exit 31
+  fi
+  echo
+
+  echo "== public CORS =="
+  CORS_HEADERS="$(curl --fail --show-error --silent --max-time 12 -D - -o /dev/null \
+    -H "Origin: https://ffffhx.github.io" \
+    "${API_URL%/}/api/snapshots?limit=1")"
+  printf "%s\n" "${CORS_HEADERS}" | grep -qi '^access-control-allow-origin: \*' || {
+    echo "public list response is missing access-control-allow-origin: *" >&2
+    exit 32
+  }
+  echo "CORS read check passed."
+fi

package/deploy/aliyun/codex-snapshot-share.env.example

@@ -0,0 +1,29 @@
+# Copy to /etc/codex-snapshots/share-api.env on the ECS host.
+# Generate a strong value, for example:
+#   openssl rand -base64 32
+SNAPSHOT_SHARE_TOKEN=change-me
+
+# Public GitHub Pages site that should render shared sessions.
+SNAPSHOT_SHARE_SITE_URL=https://ffffhx.github.io/codex-snapshots/
+
+# Public HTTPS API origin served by Nginx on the ECS host.
+SNAPSHOT_SHARE_PUBLIC_API_URL=https://snapshots.example.com
+
+# The public site path for the static share viewer.
+SNAPSHOT_SHARE_VIEWER_PATH=/share/
+
+# Persistent storage on the ECS host. Back this file up.
+SNAPSHOT_SHARE_DATA_FILE=/var/lib/codex-snapshots/shares.json
+
+# Keep writes/deletes authenticated. Public reads remain available.
+SNAPSHOT_SHARE_ALLOW_ANONYMOUS=false
+
+# Optional GitHub OAuth publish/delete auth.
+# GitHub OAuth App callback URL:
+#   https://snapshots.example.com/api/auth/github/callback
+# SNAPSHOT_GITHUB_CLIENT_ID=change-me
+# SNAPSHOT_GITHUB_CLIENT_SECRET=change-me
+# SNAPSHOT_SESSION_SECRET=change-me-generate-with-openssl-rand-base64-48
+# SNAPSHOT_GITHUB_OWNER_LOGIN=your-github-login
+# SNAPSHOT_GITHUB_OWNER_ID=
+# SNAPSHOT_AUTH_ALLOWED_ORIGINS=https://ffffhx.github.io

package/deploy/aliyun/codex-snapshot-share.service

@@ -0,0 +1,26 @@
+[Unit]
+Description=Codex Snapshots Share API
+Documentation=https://github.com/ffffhx/codex-snapshots
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+User=codexsnap
+Group=codexsnap
+WorkingDirectory=/opt/codex-snapshots
+Environment=NODE_ENV=production
+Environment=HOST=127.0.0.1
+Environment=PORT=8787
+EnvironmentFile=/etc/codex-snapshots/share-api.env
+ExecStart=/usr/bin/env node /opt/codex-snapshots/server/share-api.mjs
+Restart=always
+RestartSec=3
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectHome=true
+ProtectSystem=strict
+ReadWritePaths=/var/lib/codex-snapshots
+
+[Install]
+WantedBy=multi-user.target