@lamalibre/create-lamaste 4.0.0

package/vendor/docs/01-concepts/certificates.md

@@ -0,0 +1,452 @@
+# Certificates
+
+> Lamaste manages three types of TLS certificates: Let's Encrypt certificates for public domains, self-signed certificates for IP access, and mTLS client certificates for admin authentication.
+
+## In Plain English
+
+When you visit a website over HTTPS, your browser checks the site's certificate to verify two things: the connection is encrypted, and the site is who it claims to be. Without a certificate, your browser shows a warning and the connection is insecure.
+
+Lamaste needs certificates for every domain it serves — `panel.example.com`, `auth.example.com`, `myapp.example.com`, and so on. Buying certificates would be expensive and tedious, so Lamaste uses Let's Encrypt, a free certificate authority that issues certificates automatically.
+
+Think of Let's Encrypt like a free notary service. Lamaste asks "I need a certificate for myapp.example.com," and Let's Encrypt says "prove you control that domain." Lamaste proves it by serving a specific file on that domain (an HTTP challenge). Let's Encrypt verifies the file is there and issues the certificate. The whole process takes a few seconds and happens without any human intervention.
+
+Let's Encrypt certificates expire every 90 days. Lamaste sets up automatic renewal so you never need to think about it. A system timer checks twice daily whether any certificates need renewing and handles it silently.
+
+For the IP-based admin panel (`https://<ip>:9292`), there is no domain to verify, so Lamaste uses a self-signed certificate instead. Your browser shows a security warning for self-signed certificates, but the connection is still encrypted. The real authentication for the admin panel comes from the [mTLS client certificate](mtls.md), not from the server certificate.
+
+## For Users
+
+### Certificate types in Lamaste
+
+| Type              | Where used                                                         | Issued by         | Validity | Auto-renew          |
+| ----------------- | ------------------------------------------------------------------ | ----------------- | -------- | ------------------- |
+| Let's Encrypt     | Domain-based vhosts (`*.example.com`)                              | Let's Encrypt CA  | 90 days  | Yes (systemd timer) |
+| Self-signed TLS   | IP-based panel (`https://IP:9292`)                                 | Lamaste installer | 10 years | No (long validity)  |
+| mTLS CA           | nginx client cert verification                                     | Lamaste installer | 10 years | No (long validity)  |
+| mTLS client       | Browser authentication                                             | Lamaste CA        | 2 years  | Manual rotation     |
+| Agent cert        | Agent-to-panel mTLS (`CN=agent:<label>`)                           | Lamaste CA        | 2 years  | Manual rotation     |
+| Plugin-agent cert | Delegated plugin agent mTLS (`CN=plugin-agent:<delegator>:<name>`) | Lamaste CA        | 2 years  | Manual rotation     |
+
+### When certificates are issued
+
+- **Self-signed + mTLS certs** — created during `npx @lamalibre/create-lamaste@4.0.0` installation
+- **Core Let's Encrypt certs** — issued during onboarding for `panel`, `auth`, and `tunnel` subdomains
+- **App Let's Encrypt certs** — issued when you create a new tunnel or a managed static site
+- **Custom-domain site certs** — issued when a custom-domain site's DNS is verified: one certificate lineage, named after the site's domain, covering the domain and every redirect alias (re-issued when aliases change)
+
+### Viewing certificates
+
+The Certificates page in the management panel shows all certificates with:
+
+- Domain name
+- Issuer (Let's Encrypt or self-signed)
+- Expiry date
+- Days remaining
+- Status (valid, expiring soon, expired)
+
+### Certificate renewal
+
+Let's Encrypt certificates are automatically renewed by a systemd timer (`certbot.timer`). The timer runs twice daily and renews any certificate expiring within 30 days.
+
+You can also manually trigger renewal from the Certificates page:
+
+1. Find the certificate in the list
+2. Click "Renew"
+3. Lamaste runs `certbot renew --force-renewal` for that certificate — without the random delay of up to about 8 minutes certbot otherwise adds before a non-interactive renewal (that delay is meant for `certbot.timer`, not for an operator waiting on the page)
+4. nginx reloads to pick up the new certificate
+
+### mTLS client certificate rotation
+
+Your admin client certificate expires after 2 years. The Certificates page shows when it expires. To rotate it:
+
+1. On the server, run `sudo lamaste-server reset-admin` (the panel runs unprivileged and does not issue admin certificates)
+2. Lamaste generates a new key pair and certificate, signed by the same CA, and revokes the old one
+3. A new `.p12` file is generated with a new random password, which the command prints
+4. Download the new `.p12` file (Certificates page, or `scp`) and import it into your browser
+5. Remove the old certificate from your browser/keychain
+
+**Important:** after rotation, the old certificate is immediately invalid.
+
+### What happens when certificates expire
+
+| Certificate     | If it expires                          | Impact                                                          | Recovery                                          |
+| --------------- | -------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------- |
+| Let's Encrypt   | Browsers show security warning         | Visitors see "not secure" for that domain                       | Renew via panel or `certbot renew`                |
+| Self-signed TLS | Browser warning changes                | Minimal — browsers already show a warning for self-signed certs | Re-run installer                                  |
+| mTLS CA         | Client certs can no longer be verified | Admin panel becomes inaccessible                                | Re-provision PKI (requires SSH)                   |
+| mTLS client     | nginx rejects the certificate          | Admin panel inaccessible from that browser                      | Rotate from panel (if another device still works) |
+
+The most critical scenario is the mTLS CA expiring, but it has a 10-year validity, making this a distant concern.
+
+## For Developers
+
+### Let's Encrypt implementation
+
+Lamaste uses certbot with the nginx plugin for certificate issuance. The implementation lives in `@lamalibre/lamaste@4.0.0/server` (`packages/core/lib/src/server/certbot.ts`), wrapped by `packages/server/daemon/src/lib/certbot.js`.
+
+The panel runs as the unprivileged `lamaste` user and never runs `certbot` or `openssl` on Let's Encrypt files directly. It calls two root-owned wrapper scripts through sudo:
+
+| Wrapper                             | Commands                                                                                       |
+| ----------------------------------- | ---------------------------------------------------------------------------------------------- |
+| `/usr/local/sbin/lamaste-certbot`   | `issue <email> <cert-name> <domain>...`, `renew <cert-name> [force]`, `renew-all`, `list`      |
+| `/usr/local/sbin/lamaste-cert-info` | `<lineage> enddate` (expiry), `<lineage> checkend` (valid 24 h more?), `<lineage> san` (names) |
+
+Each wrapper validates every argument — lowercase DNS hostnames only (no wildcards, no path characters, at most 100 per certificate), a syntactically valid email — and executes certbot or openssl with a fixed argument vector. They replace sudoers rules such as `certbot renew --cert-name * --non-interactive` and `openssl x509 -enddate -noout -in /etc/letsencrypt/live/*`: a sudoers `*` also matches spaces, so those rules accepted extra flags like `--deploy-hook <command>` or `-engine <library>` — root code execution for the `lamaste` user. The wrappers ship in `packages/provisioners/server/scripts/` and are installed (`root:root`, 0755) by the installer and by every redeploy.
+
+The mTLS PKI needs no wrapper. `/etc/lamalibre/lamaste/pki` belongs to the `lamaste` user — the panel is the CA — so agent certificates and CSRs (enrollment, hardware-bound rotation) are signed by the panel itself with `openssl x509 -req` (`packages/server/daemon/src/lib/pki-sign.js`). `signAgentCsr` refuses any CN that is not an `agent:<label>` or `plugin-agent:<label>` label, and the callers check the CN is exactly the label they expect; the panel never issues `CN=admin`. Admin certificates come only from `sudo lamaste-server reset-admin` on the server. (Earlier versions signed through a root wrapper, `lamaste-sign-csr`, and renamed PKI files through `lamaste-pki-rename`; both are removed on upgrade.)
+
+#### Issuing a certificate
+
+```javascript
+export async function issueCert(fqdn, email, exec) {
+  try {
+    // lamaste-certbot runs: certbot certonly --nginx --cert-name <fqdn> -d <fqdn>
+    //                       --email <email> --agree-tos --non-interactive
+    await exec('sudo', ['/usr/local/sbin/lamaste-certbot', 'issue', email, fqdn, fqdn]);
+  } catch (err) {
+    const stderr = err.stderr || err.message;
+
+    if (stderr.includes('too many certificates') || stderr.includes('rate limit')) {
+      throw new Error(`Let's Encrypt rate limit reached for ${fqdn}...`);
+    }
+
+    if (stderr.includes('DNS problem') || stderr.includes('NXDOMAIN')) {
+      throw new Error(`DNS is not pointing ${fqdn} to this server...`);
+    }
+
+    throw new Error(`Failed to issue certificate for ${fqdn}: ${stderr}`);
+  }
+
+  return {
+    issued: true,
+    domain: fqdn,
+    certPath: `/etc/letsencrypt/live/${fqdn}/fullchain.pem`,
+    keyPath: `/etc/letsencrypt/live/${fqdn}/privkey.pem`,
+  };
+}
+```
+
+The error handling distinguishes between rate limits, DNS problems, and nginx configuration issues, providing specific guidance for each failure mode.
+
+#### HTTP-01 challenge
+
+The `--nginx` plugin uses the HTTP-01 challenge method:
+
+```
+1. Certbot creates a challenge response for /.well-known/acme-challenge/<token>
+2. Certbot temporarily adds a port-80 server block for the name to nginx
+   (cloned from the lamalibre-lamaste-http-redirect default server)
+3. Let's Encrypt's servers request http://fqdn/.well-known/acme-challenge/<token>
+4. If the response matches, the domain is verified
+5. Let's Encrypt issues the certificate
+6. Certbot restores the nginx configuration
+```
+
+This requires that the FQDN resolves to the VPS IP and that port 80 is reachable from the internet — for the first issuance and for every renewal. The installer's port-80 catch-all, which redirects every other plain-HTTP request to HTTPS, does not interfere: certbot answers the challenge path ahead of the redirect and restores the file afterwards. See [DNS and Domains](dns-and-domains.md) for DNS configuration details.
+
+#### Core certificate issuance
+
+During onboarding, certificates are issued sequentially for the three core subdomains:
+
+```javascript
+export async function issueCoreCerts(domain, email) {
+  const subdomains = ['panel', 'auth', 'tunnel'];
+  const results = [];
+
+  for (const sub of subdomains) {
+    const fqdn = `${sub}.${domain}`;
+    const result = await issueCert(fqdn, email);
+    results.push(result);
+  }
+
+  return results;
+}
+```
+
+Sequential issuance avoids triggering Let's Encrypt rate limits from parallel requests.
+
+#### Tunnel certificate issuance
+
+When creating a tunnel, the system checks for existing coverage before issuing a new certificate:
+
+```javascript
+export async function issueTunnelCert(fqdn, email) {
+  // 1. Check for wildcard cert
+  const baseDomain = fqdn.split('.').slice(1).join('.');
+  if (await hasWildcardCert(baseDomain)) {
+    return { skipped: true, reason: 'wildcard', certPath: `.../${baseDomain}/` };
+  }
+
+  // 2. Check for existing valid individual cert
+  const existing = await isCertValid(fqdn);
+  if (existing.valid) {
+    return { skipped: true, reason: 'exists', certPath: `.../${fqdn}/` };
+  }
+
+  // 3. Issue new certificate
+  await issueCert(fqdn, email);
+  return { skipped: false, certPath: `.../${fqdn}/` };
+}
+```
+
+This three-step check prevents unnecessary certificate issuance and respects rate limits.
+
+#### Custom-domain site certificate issuance
+
+A custom-domain static site may have redirect aliases (e.g. `www.myblog.net`). `issueSiteCert(fqdn, aliases, email)` in `@lamalibre/lamaste@4.0.0/server` keeps them on **one** lineage named after the primary domain:
+
+```bash
+sudo /usr/local/sbin/lamaste-certbot issue admin@example.com myblog.net myblog.net www.myblog.net
+# runs: certbot certonly --nginx --cert-name myblog.net -d myblog.net -d www.myblog.net \
+#         --email admin@example.com --agree-tos --non-interactive
+```
+
+`--cert-name` pins the lineage directory to `/etc/letsencrypt/live/myblog.net/`, so re-issuing for a changed alias set never drifts to `myblog.net-0001`. Before issuing, it reads the current certificate's names (`lamaste-cert-info myblog.net san`) and skips issuance when the certificate already covers every name — or, with `{ match: 'exact' }`, when it holds exactly those names.
+
+When the aliases of a live site change, the certificate is first issued for the **union** of the old and new names (valid whichever vhost nginx serves during the switch), the vhost is rewritten and the site saved, and then the certificate is re-issued for **exactly** the new names (`match: 'exact'`), so a renewal never has to validate a name the site no longer serves. If that last step fails, `PATCH /api/sites/:id` still succeeds and returns a `warning` — see the [Sites API](../04-api-reference/sites.md).
+
+#### Input validation
+
+The `issueTunnelCert` function validates both the FQDN and email before proceeding:
+
+```javascript
+if (!/^[a-z0-9]([a-z0-9.-]*[a-z0-9])?$/.test(fqdn)) {
+  throw new Error(`Invalid FQDN: ${fqdn}`);
+}
+
+if (!/^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(email)) {
+  throw new Error(`Invalid email: ${email}`);
+}
+```
+
+These checks prevent command injection through the certbot CLI arguments. The `lamaste-certbot` wrapper repeats them as root, so a compromised panel process cannot pass anything else either.
+
+### Certificate validity checks
+
+The `isCertValid` function uses OpenSSL to check if a certificate exists and is valid for at least 24 more hours:
+
+```javascript
+export async function isCertValid(fqdn, exec) {
+  const certPath = `/etc/letsencrypt/live/${fqdn}/fullchain.pem`;
+  if (!HOSTNAME_RE.test(fqdn)) {
+    return { valid: false, certPath: null, expiryDate: null };
+  }
+
+  try {
+    // lamaste-cert-info runs: openssl x509 -checkend 86400 -noout -in <lineage>/fullchain.pem
+    await exec('sudo', ['/usr/local/sbin/lamaste-cert-info', fqdn, 'checkend']);
+  } catch (err) {
+    // Exit 1: the certificate exists but expires within a day.
+    // Anything else (3: no certificate, 2: refused) means there is none to use.
+    if (err.exitCode === 1) {
+      return { valid: false, certPath, expiryDate: null };
+    }
+    return { valid: false, certPath: null, expiryDate: null };
+  }
+
+  const expiry = await readLetsEncryptExpiry(fqdn, exec); // lamaste-cert-info <fqdn> enddate
+  return { valid: true, certPath, expiryDate: expiry?.expiresAt ?? null };
+}
+```
+
+`openssl x509 -checkend 86400` exits with code 0 if the certificate is valid for at least 86400 seconds (24 hours) and code 1 otherwise; the wrapper passes that through and uses exit code 3 for a lineage with no certificate. This provides a simple binary check without parsing dates.
+
+### Listing certificates
+
+The `listCerts` function parses the output of `certbot certificates` (`lamaste-certbot list`) to build a structured list:
+
+```javascript
+export async function listCerts(exec) {
+  const { stdout } = await exec('sudo', ['/usr/local/sbin/lamaste-certbot', 'list']);
+
+  if (stdout.includes('No certificates found')) return [];
+
+  const blocks = stdout.split('Certificate Name:').slice(1);
+  return blocks.map((block) => {
+    // Parse: name, domains, expiry, cert path, key path, validity
+    // ...
+    return { name, domains, expiryDate, daysRemaining, certPath, keyPath, isValid };
+  });
+}
+```
+
+This list is combined with mTLS certificate information from `packages/lamaste-serverd/src/lib/mtls.js` to present a unified certificate view in the panel.
+
+### Auto-renewal setup
+
+During onboarding provisioning, the certbot systemd timer is enabled:
+
+```javascript
+export async function setupAutoRenew() {
+  await execa('sudo', ['systemctl', 'enable', 'certbot.timer']);
+  await execa('sudo', ['systemctl', 'start', 'certbot.timer']);
+
+  const { stdout } = await execa('systemctl', ['is-active', 'certbot.timer']);
+  if (stdout.trim() !== 'active') {
+    throw new Error('Certbot timer is not active after enabling.');
+  }
+  return { enabled: true };
+}
+```
+
+The certbot timer runs twice daily (standard certbot behavior). It only renews certificates that expire within 30 days, so most runs are no-ops.
+
+After renewal, certbot automatically reloads nginx via a deploy hook, ensuring the new certificate is served immediately.
+
+### Self-signed certificate for IP access
+
+The installer generates a self-signed certificate during nginx setup:
+
+```bash
+openssl req -x509 -nodes \
+  -days 3650 \
+  -newkey rsa:2048 \
+  -keyout /etc/lamalibre/lamaste/pki/self-signed-key.pem \
+  -out /etc/lamalibre/lamaste/pki/self-signed.pem \
+  -subj "/CN=203.0.113.42/O=Lamaste" \
+  -addext "subjectAltName=IP:203.0.113.42"
+```
+
+The `-addext subjectAltName=IP:...` flag embeds the IP address in the Subject Alternative Name field. Without this, modern browsers reject the certificate even for IP-based access.
+
+The self-signed certificate uses a 2048-bit key (smaller than the 4096-bit mTLS keys) because it serves a less critical role — it is only for IP-based access where the browser already shows a warning.
+
+### Certificate file layout
+
+```
+/etc/letsencrypt/live/
+├── panel.example.com/
+│   ├── fullchain.pem        Let's Encrypt cert chain
+│   ├── privkey.pem          Private key
+│   ├── cert.pem             Server certificate only
+│   └── chain.pem            Intermediate CA chain
+├── auth.example.com/
+│   └── (same structure)
+├── tunnel.example.com/
+│   └── (same structure)
+└── myapp.example.com/
+    └── (same structure)
+
+/etc/lamalibre/lamaste/pki/
+├── self-signed.pem          Self-signed TLS cert (IP access)
+├── self-signed-key.pem      Self-signed TLS key
+├── ca.crt                   mTLS CA certificate
+├── ca.key                   mTLS CA private key
+├── client.crt               mTLS client certificate
+├── client.key               mTLS client private key
+├── client.p12               PKCS12 bundle for browser import
+├── .p12-password            Password for the .p12 file
+├── revoked.json             Revoked certificate serials
+├── enrollment-tokens.json   Pending hardware-bound enrollment tokens
+└── agents/
+    ├── registry.json        Agent certificate registry
+    └── <label>/
+        └── client.p12       Agent PKCS12 bundle
+```
+
+### mTLS certificate expiry monitoring
+
+The panel server reads mTLS certificate expiry dates using OpenSSL:
+
+```javascript
+export async function readCertExpiry(certPath) {
+  // No sudo: the PKI directory belongs to the panel's user.
+  const { stdout } = await execa('openssl', ['x509', '-in', certPath, '-enddate', '-noout']);
+  const match = stdout.match(/notAfter=(.+)/);
+  const expiryDate = new Date(match[1]);
+  const daysUntilExpiry = Math.floor((expiryDate.getTime() - Date.now()) / (1000 * 60 * 60 * 24));
+  return { expiresAt: expiryDate.toISOString(), daysUntilExpiry };
+}
+```
+
+Certificates with 30 or fewer days remaining are flagged as `expiringSoon` in the API response.
+
+### Source files
+
+| File                                                      | Purpose                                                 |
+| --------------------------------------------------------- | ------------------------------------------------------- |
+| `packages/lamaste-serverd/src/lib/certbot.js`             | Let's Encrypt issuance, renewal, listing                |
+| `packages/lamaste-serverd/src/lib/mtls.js`                | mTLS cert expiry, rotation, download                    |
+| `packages/lamaste-serverd/src/routes/management/certs.js` | Certificate management API                              |
+| `packages/create-lamaste/src/tasks/nginx.js`              | Self-signed cert generation                             |
+| `packages/create-lamaste/src/tasks/mtls.js`               | mTLS CA and client cert generation                      |
+| `packages/lamaste-serverd/src/lib/csr-signing.js`         | CSR signing for hardware-bound and delegated enrollment |
+| `packages/lamaste-serverd/src/lib/enrollment.js`          | Enrollment token management (regular and delegated)     |
+| `packages/lamaste-serverd/src/routes/enrollment.js`       | Public enrollment route                                 |
+
+## Quick Reference
+
+### Certificate types
+
+| Type            | Location                                     | Issued by        | Validity | Key size     |
+| --------------- | -------------------------------------------- | ---------------- | -------- | ------------ |
+| Let's Encrypt   | `/etc/letsencrypt/live/<fqdn>/`              | Let's Encrypt CA | 90 days  | 2048-bit RSA |
+| Self-signed TLS | `/etc/lamalibre/lamaste/pki/self-signed.pem` | Self             | 10 years | 2048-bit RSA |
+| mTLS CA         | `/etc/lamalibre/lamaste/pki/ca.crt`          | Self             | 10 years | 4096-bit RSA |
+| mTLS client     | `/etc/lamalibre/lamaste/pki/client.crt`      | Lamaste CA       | 2 years  | 4096-bit RSA |
+
+### certbot commands
+
+```bash
+# List all certificates
+sudo certbot certificates
+
+# Renew a specific certificate
+sudo certbot renew --cert-name panel.example.com
+
+# Renew all certificates due for renewal
+sudo certbot renew
+
+# Issue a new certificate
+sudo certbot certonly --nginx -d myapp.example.com --email admin@example.com --agree-tos --non-interactive
+
+# Check auto-renewal timer
+systemctl status certbot.timer
+```
+
+### OpenSSL commands
+
+```bash
+# View certificate details
+openssl x509 -in /etc/letsencrypt/live/panel.example.com/fullchain.pem -text -noout
+
+# Check certificate expiry date
+openssl x509 -in /etc/letsencrypt/live/panel.example.com/fullchain.pem -enddate -noout
+
+# Check if cert is valid for 30+ days
+openssl x509 -checkend 2592000 -noout -in /path/to/cert.pem
+
+# Verify certificate chain
+openssl verify -CAfile /etc/letsencrypt/live/panel.example.com/chain.pem \
+  /etc/letsencrypt/live/panel.example.com/cert.pem
+```
+
+### API endpoints
+
+| Method | Path                                         | Description                                  |
+| ------ | -------------------------------------------- | -------------------------------------------- |
+| GET    | `/api/certs`                                 | List all certificates (Let's Encrypt + mTLS) |
+| POST   | `/api/certs/:domain/renew`                   | Force renewal of a specific certificate      |
+| POST   | `/api/certs/mtls/rotate`                     | Rotate the mTLS client certificate           |
+| GET    | `/api/certs/mtls/download`                   | Download the current `.p12` bundle           |
+| POST   | `/api/certs/agent/enroll`                    | Generate enrollment token (admin)            |
+| POST   | `/api/certs/agent/enroll-delegated`          | Delegated enrollment token (agent)           |
+| POST   | `/api/enroll`                                | Enroll agent with token (public)             |
+| POST   | `/api/certs/admin/upgrade-to-hardware-bound` | Upgrade admin auth (admin)                   |
+| GET    | `/api/certs/admin/auth-mode`                 | Get admin auth mode (admin)                  |
+
+### Let's Encrypt rate limits
+
+| Limit                              | Value               | Notes                                   |
+| ---------------------------------- | ------------------- | --------------------------------------- |
+| Certificates per registered domain | 50/week             | Covers all subdomains of `example.com`  |
+| Duplicate certificates             | 5/week              | Same set of domains                     |
+| Failed validations                 | 5/hour              | Per account per hostname                |
+| Certificate renewals               | Exempt from 50/week | Renewals do not count against the limit |
+
+### Related documentation
+
+- [mTLS](mtls.md) — client certificate details and PKCS12 bundles
+- [DNS and Domains](dns-and-domains.md) — DNS must be correct for Let's Encrypt
+- [nginx Reverse Proxy](nginx-reverse-proxy.md) — nginx uses these certificates
+- [Security Model](security-model.md) — TLS as part of defense-in-depth