palmier 0.6.0 → 0.6.2

package/palmier-server/PRODUCTION.md

@@ -0,0 +1,355 @@
+# Production Deployment Guide
+
+Single-instance deployment on Ubuntu 24 with TLS support. NATS and PostgreSQL run in Docker; the Palmier web server runs directly on the host via systemd. Caddy handles TLS termination with automatic Let's Encrypt certificates. Deployments are automated via GitHub Actions — pushing to `main` triggers a CI build, then deploys to the VPS over SSH.
+
+## Architecture
+
+```
+Internet
+   │
+   ├── palmier.me        → Cloudflare → Caddy → redirect to www
+   ├── www.palmier.me    → Cloudflare Pages (landing site, static)
+   ├── app.palmier.me    → Cloudflare → Caddy → localhost:3000 (Palmier server, HTTPS)
+   └── nats.palmier.me   → Caddy (direct) → localhost:9222 (NATS WebSocket)
+                          → port 4222 direct (NATS TCP for hosts)
+
+Caddy (ports 80/443, auto TLS)
+
+Docker
+   ├── NATS        (ports 4222 TCP public, 9222 WS localhost)
+   └── PostgreSQL  (port 5432 localhost)
+
+systemd
+   └── Palmier Server (port 3000 localhost)
+```
+
+## Prerequisites
+
+Three DNS records pointing to your server's public IP:
+
+- `palmier.me` — root domain (Cloudflare proxied, redirects to `www`)
+- `www.palmier.me` — Landing site (Cloudflare Pages, static)
+- `app.palmier.me` — PWA + API over HTTPS (Cloudflare proxied)
+- `nats.palmier.me` — NATS WebSocket + TCP (DNS only, no Cloudflare proxy)
+
+### Install Node.js 24+
+
+```bash
+curl -fsSL https://deb.nodesource.com/setup_24.x | sudo bash -
+sudo apt install -y nodejs
+```
+
+### Install pnpm
+
+```bash
+sudo npm install -g pnpm
+```
+
+### Install Docker
+
+```bash
+sudo apt install -y docker.io docker-compose-v2
+sudo usermod -aG docker $USER
+# Log out and back in for group to take effect
+```
+
+### Install Caddy
+
+```bash
+sudo apt install -y caddy
+```
+
+## 1. TLS with Caddy
+
+Caddy auto-provisions Let's Encrypt certificates with zero configuration.
+
+Create `/etc/caddy/Caddyfile`:
+
+```
+palmier.me {
+    redir https://www.palmier.me{uri} permanent
+}
+
+app.palmier.me {
+    reverse_proxy localhost:3000
+}
+
+nats.palmier.me {
+    @websocket {
+        header Connection *Upgrade*
+        header Upgrade websocket
+    }
+    reverse_proxy @websocket localhost:9222
+}
+```
+
+This gives you:
+
+- `https://app.palmier.me` — PWA + API (auto TLS, Cloudflare CDN for static assets)
+- `wss://nats.palmier.me` — NATS WebSocket (auto TLS, direct to VPS)
+- `https://www.palmier.me` — Landing site (Cloudflare Pages, no VPS involvement)
+
+LAN mode is served directly by the palmier host binary (reverse-proxying PWA assets from `app.palmier.me`), so no separate domain is needed.
+
+Caddy auto-provisions Let's Encrypt certs for both domains. Set Cloudflare SSL/TLS mode to **Full (strict)** so Cloudflare verifies the origin cert when connecting to your VPS.
+
+No need to configure TLS in NATS itself.
+
+```bash
+sudo systemctl enable --now caddy
+```
+
+## 2. Docker Compose for NATS + PostgreSQL
+
+Create a directory for production Docker config:
+
+```bash
+mkdir -p ~/palmier-prod
+```
+
+### docker-compose.yml
+
+Create `~/palmier-prod/docker-compose.yml`:
+
+```yaml
+services:
+  nats:
+    image: nats:2-alpine
+    restart: unless-stopped
+    ports:
+      - "4222:4222"
+      - "127.0.0.1:9222:9222"
+    volumes:
+      - ./nats.conf:/etc/nats/nats.conf:ro
+      - nats-data:/data
+    command: ["-c", "/etc/nats/nats.conf"]
+
+  postgres:
+    image: postgres:17-alpine
+    restart: unless-stopped
+    ports:
+      - "127.0.0.1:5432:5432"
+    environment:
+      POSTGRES_DB: palmier
+      POSTGRES_USER: palmier
+      POSTGRES_PASSWORD: <generate-a-strong-password>
+    volumes:
+      - pgdata:/var/lib/postgresql/data
+
+volumes:
+  nats-data:
+  pgdata:
+```
+
+### nats.conf
+
+Create `~/palmier-prod/nats.conf`:
+
+```
+listen: 0.0.0.0:4222
+
+websocket {
+  listen: "0.0.0.0:9222"
+  no_tls: true
+}
+
+authorization {
+  token: "<generate-a-new-token>"
+}
+```
+
+Generate secrets:
+
+```bash
+# NATS token
+openssl rand -hex 32
+
+# Postgres password
+openssl rand -hex 16
+```
+
+Start the containers:
+
+```bash
+cd ~/palmier-prod
+docker compose up -d
+```
+
+## 3. NATS TCP for Remote Hosts
+
+Hosts connect to NATS over TCP (port 4222). Since Caddy only handles HTTP/WebSocket, port 4222 must be open on the firewall for hosts to connect. The NATS token provides authentication.
+
+For encrypted host connections, you can add a TLS-terminating TCP proxy (e.g., nginx stream block) in front of port 4222. For a single-user setup, token auth alone is sufficient.
+
+## 4. Firewall
+
+```bash
+sudo ufw allow 22/tcp      # SSH (must allow before enabling!)
+sudo ufw allow 80/tcp      # Caddy HTTP -> HTTPS redirect
+sudo ufw allow 443/tcp     # Caddy HTTPS
+sudo ufw allow 4222/tcp    # NATS TCP for hosts
+sudo ufw enable
+```
+
+## 5. Clone the Repository
+
+Set up VPS git access first (see [VPS Git Access](#vps-git-access)), then clone:
+
+```bash
+mkdir -p ~/source
+cd ~/source
+git clone git@github.com:caihongxu/palmier-server.git
+cd palmier-server
+```
+
+The first push to `main` will build and start the server automatically via GitHub Actions.
+
+## 6. Configure Environment
+
+Create `server/.env`:
+
+```bash
+PORT=3000
+DATABASE_URL=postgresql://palmier:<your-pg-password>@localhost:5432/palmier
+
+NATS_URL=nats://localhost:4222
+NATS_HOST_URL=nats://nats.palmier.me:4222
+NATS_WS_URL=wss://nats.palmier.me
+NATS_TOKEN=<your-nats-token-from-nats.conf>
+
+VAPID_PUBLIC_KEY=<generated>
+VAPID_PRIVATE_KEY=<generated>
+VAPID_MAILTO=mailto:you@palmier.me
+```
+
+Key differences from development:
+
+- `NATS_WS_URL` uses `wss://` through Caddy (not direct `ws://`)
+- `NATS_HOST_URL` uses your public domain so remote hosts can reach NATS
+
+Generate VAPID keys:
+
+```bash
+npx web-push generate-vapid-keys
+```
+
+## 7. Palmier systemd Service
+
+Create `/etc/systemd/system/palmier-server.service`:
+
+```ini
+[Unit]
+Description=Palmier Server
+After=network.target docker.service
+
+[Service]
+Type=simple
+User=hongxu
+WorkingDirectory=/home/hongxu/source/palmier-server/server
+ExecStart=/usr/bin/node dist/index.js
+Restart=on-failure
+RestartSec=5
+EnvironmentFile=/home/hongxu/source/palmier-server/server/.env
+
+[Install]
+WantedBy=multi-user.target
+```
+
+Enable and start:
+
+```bash
+sudo systemctl daemon-reload
+sudo systemctl enable --now palmier-server
+```
+
+## 8. Verify
+
+At this point, all services should be running (Docker from step 2, Caddy from step 1, Palmier from step 7). Verify:
+
+```bash
+sudo systemctl status caddy
+sudo systemctl status palmier-server
+cd ~/palmier-prod && docker compose ps
+```
+
+- `https://app.palmier.me` — should load the PWA (server mode)
+- `https://app.palmier.me/health` — should return OK
+- Logs: `journalctl -u palmier-server -f`
+- Docker logs: `cd ~/palmier-prod && docker compose logs -f`
+
+## CI/CD with GitHub Actions
+
+Deployments are automated via two GitHub Actions workflows in `.github/workflows/`:
+
+- **`ci.yml`** — runs on every push (except `main`) and pull requests. Builds server + PWA to catch errors early.
+- **`deploy.yml`** — runs on push to `main`. Builds in CI first, then SSHs into the VPS to pull, build, and restart the service.
+
+### GitHub Secrets
+
+Add these in the repo settings (Settings → Secrets and variables → Actions):
+
+| Secret | Description |
+|---|---|
+| `VPS_HOST` | Droplet IP address |
+| `VPS_USER` | SSH username (e.g., `hongxu`) |
+| `SSH_PRIVATE_KEY` | Ed25519 private key for SSH access |
+
+### VPS SSH Setup
+
+Generate a deploy key pair and add the public key to the VPS:
+
+```bash
+ssh-keygen -t ed25519 -C "github-actions-deploy" -f ~/.ssh/palmier-deploy -N ""
+```
+
+Append the public key to the VPS `~/.ssh/authorized_keys`. Add the private key as the `SSH_PRIVATE_KEY` GitHub secret.
+
+### VPS Git Access
+
+The VPS needs to pull from the private repo. Add a read-only deploy key:
+
+1. Generate a key on the VPS: `ssh-keygen -t ed25519 -C "palmier-server-deploy" -f ~/.ssh/deploy-key -N ""`
+2. Add the public key to the repo (Settings → Deploy keys, read-only)
+3. Configure SSH on the VPS to use the key for GitHub:
+   ```
+   # ~/.ssh/config
+   Host github.com
+     IdentityFile ~/.ssh/deploy-key
+   ```
+
+### Passwordless sudo for deploy
+
+The deploy workflow needs to restart the systemd service. Allow the deploy user to do this without a password:
+
+```bash
+sudo visudo -f /etc/sudoers.d/palmier-deploy
+```
+
+Add:
+
+```
+hongxu ALL=(ALL) NOPASSWD: /usr/bin/systemctl restart palmier-server
+```
+
+### Deploying
+
+Push to `main` and the deploy workflow handles everything automatically. To deploy manually:
+
+```bash
+ssh hongxu@<vps-ip>
+cd ~/source/palmier-server
+git pull
+pnpm install --frozen-lockfile
+cd pwa && pnpm build && cd ..
+cd server && pnpm build && cd ..
+sudo systemctl restart palmier-server
+```
+
+## Component Summary
+
+| Component | Runtime | Ports |
+|---|---|---|
+| PostgreSQL | Docker | 5432 (localhost only) |
+| NATS | Docker | 4222 (public, for hosts), 9222 (localhost, proxied by Caddy) |
+| Palmier Server | systemd + Node.js | 3000 (localhost only) |
+| Caddy | systemd | 80, 443 (public) |

package/palmier-server/README.md

@@ -0,0 +1,187 @@
+# Palmier Server
+
+![Deploy](https://github.com/caihongxu/palmier-server/actions/workflows/deploy.yml/badge.svg)
+
+Palmier is a platform for remotely scheduling, managing, and executing autonomous AI tasks on host machines via a progressive web app. Uses NATS for real-time communication (pub/sub and request-reply) and push notifications for task progress.
+
+## Architecture
+
+```
++-----------+       +----------------+       +------------------+
+|           | HTTP  |                | NATS  |                  |
+|    PWA    |------>|   Web Server   |<----->|   NATS Server    |
+|  (React)  |       |   (Express)    |       |                  |
+|           |       |                |       |                  |
++-----------+       +----------------+       +------------------+
+      |                    |                         ^
+      |  NATS WebSocket    |  PostgreSQL             |  NATS
+      +------------------------------------------->  |
+                           |                         |
+                      +----v----+            +-------+-------+
+                      |         |            |               |
+                      |   DB    |            |  Host(s)      |
+                      |  (PG)   |            |  (separate    |
+                      |         |            |   repo)       |
+                      +---------+            +---------------+
+```
+
+- **PWA** -- React 19 + Vite progressive web app. Connects to NATS over WebSocket for real-time task updates and to the web server for host registration and push notifications. No user accounts — paired hosts are stored in localStorage.
+- **Web Server** -- Express + TypeScript API server. Handles host registration, push notifications (subscribes to `host-event.>` pub/sub for confirmation and completion events), and push notification relay (for host CLI requests via NATS). In production, also serves the built PWA static files.
+- **NATS Server** -- Message broker. Provides pub/sub messaging and request-reply for real-time communication between all components.
+- **Host** -- Runs on remote Linux/Windows machines to execute tasks via pluggable agent tools (e.g., Claude Code, Codex, Gemini). Each agent implements an `AgentTool` interface that handles command construction. Communicates with the platform over NATS and exposes a local HTTP server with agent-facing endpoints (`/notify`, `/request-input`) for task execution flows. See the [palmier](https://github.com/caihongxu/palmier) repo.
+
+## Prerequisites
+
+- Node.js 24+
+- pnpm (`npm install -g pnpm`)
+- PostgreSQL 17
+- NATS server with WebSocket enabled
+
+## Getting Started
+
+1. **Clone the repository**
+
+   ```bash
+   git clone https://github.com/caihongxu/palmier-server.git
+   cd palmier-server
+   ```
+
+2. **Install dependencies** (uses pnpm workspaces)
+
+   ```bash
+   pnpm install
+   ```
+
+3. **Configure environment variables**
+
+   ```bash
+   cp server/.env.example server/.env
+   ```
+
+   Edit `server/.env` and fill in the values (see table below).
+
+4. **Generate VAPID keys** for web push notifications
+
+   ```bash
+   npx web-push generate-vapid-keys
+   ```
+
+   Copy the public and private keys into `server/.env`.
+
+5. **Create the PostgreSQL database**
+
+   ```bash
+   createdb palmier
+   ```
+
+   Update `DATABASE_URL` in `server/.env` with your connection string. Tables are created automatically on server startup.
+
+6. **Start the NATS server** with the included config
+
+   ```bash
+   nats-server -c nats.conf
+   ```
+
+   The config enables WebSocket on port 9222 and token-based auth.
+
+## Development
+
+Development uses two servers: Vite for the PWA (with hot module replacement) and Express for the API. Vite proxies `/api/*` requests to Express so everything appears same-origin.
+
+```bash
+# Terminal 1 -- API server (port 3000)
+cd server
+pnpm dev
+
+# Terminal 2 -- PWA dev server (port 5173, proxies /api to :3000)
+cd pwa
+pnpm dev
+
+# To proxy to a remote API server instead of localhost:
+# Linux / macOS:
+API_URL=https://app.palmier.me pnpm dev
+# Windows (PowerShell):
+$env:API_URL="https://app.palmier.me"; pnpm dev
+```
+
+Open `http://localhost:5173` in your browser.
+
+### Production
+
+In production, Express serves the built PWA static files directly — one server, one port.
+
+```bash
+# Build the PWA
+cd pwa && pnpm build
+
+# Start the server (serves API + PWA on port 3000)
+cd server && pnpm start
+```
+
+Open `http://localhost:3000`.
+
+### Host Setup
+
+The host runs on a separate Linux machine. See the [palmier README](https://github.com/caihongxu/palmier) for full details.
+
+1. On the host machine, in your project directory, run:
+   ```bash
+   palmier init
+   ```
+   The interactive wizard detects agents, configures access modes (HTTP port, LAN access), shows a summary for confirmation, registers with the server, saves config to `~/.config/palmier/host.json`, installs a background daemon, and generates a pairing code.
+
+2. Enter the pairing code in the PWA to connect your device to the host.
+
+## Environment Variables
+
+| Variable | Description | Example |
+|---|---|---|
+| `PORT` | HTTP server port | `3000` |
+| `DATABASE_URL` | PostgreSQL connection string | `postgresql://user:password@localhost:5432/palmier` |
+| `NATS_URL` | NATS server URL (TCP, for server's own connection) | `nats://localhost:4222` |
+| `NATS_HOST_URL` | NATS URL sent to hosts during registration (use LAN IP) | `nats://192.168.1.100:4222` |
+| `NATS_WS_URL` | NATS WebSocket URL sent to PWA clients | `wss://nats.palmier.me` (prod) or `ws://192.168.1.100:9222` (LAN) |
+| `NATS_TOKEN` | NATS authentication token | *(from nats.conf)* |
+| `VAPID_PUBLIC_KEY` | VAPID public key for web push | *(generated via web-push)* |
+| `VAPID_PRIVATE_KEY` | VAPID private key for web push | *(generated via web-push)* |
+| `VAPID_MAILTO` | Contact email for VAPID | `mailto:admin@example.com` |
+
+> **LAN setup note:** `NATS_URL` uses `localhost` because the server connects to NATS locally. `NATS_HOST_URL`, `NATS_WS_URL` must use the LAN IP so remote hosts and browsers can reach them. Firewall must allow inbound on ports 3000 (HTTP), 4222 (NATS TCP), 5173 (Vite dev), and 9222 (NATS WebSocket).
+
+## API Endpoints
+
+All endpoints are prefixed with `/api`. No user authentication is required.
+
+| Method | Path | Description |
+|---|---|---|
+| `POST` | `/api/hosts/register` | Register a new host (returns hostId + NATS config) |
+| `GET` | `/api/config` | Get NATS WebSocket credentials for the PWA |
+| `POST` | `/api/push/subscribe` | Register a push notification subscription |
+| `DELETE` | `/api/push/subscribe` | Remove a push notification subscription |
+| `GET` | `/api/push/vapid-key` | Get the VAPID public key |
+| `POST` | `/api/push/respond` | Respond to a pending task confirmation via push notification |
+| `GET` | `/health` | Health check |
+
+
+## Key Implementation Notes
+
+- **No user accounts** — the PWA stores paired hosts in localStorage. Each device pairs with a host via a pairing code and receives a client token.
+- **Client tokens** are generated and validated on the host, not the server. They are included in every NATS RPC payload and as Bearer tokens for HTTP requests.
+- **NATS RPC** — the RPC method is derived from the NATS subject (e.g., `...rpc.task.list` → `task.list`), not the message body. The body contains request parameters plus `clientToken`. All NATS requests go through a centralized `request()` helper in `HostConnectionContext` that handles encoding/decoding and logging.
+- **Pairing** — `palmier pair` (or auto-pair after `palmier init`) generates a 6-char pairing code. The PWA enters the code, which routes to the host via NATS (`pair.<CODE>`) or HTTP (`POST /pair`). The host validates the code and returns a client token.
+- **Task IDs** are generated by the host as UUIDs.
+- **Triggers can be enabled/disabled** — the `triggers_enabled` frontmatter field (default `true`) controls whether systemd timers are installed. When disabled, timers are removed but the task can still be run manually. The "Enable Triggers" checkbox only appears in the form when triggers exist.
+- **Host responses** return flat task objects (frontmatter fields at the top level, not nested) for `task.list`, `task.create`, and `task.update`.
+- **NATS "503"** means "no responders" — the dashboard silently handles this when no host is connected, showing an empty task list instead of an error.
+- **Helmet CSP** is disabled (`contentSecurityPolicy: false`) to allow NATS WebSocket connections and inline Vite scripts during dev.
+- **Static file serving** is conditional — Express only serves `pwa/dist/` if the directory exists, so it doesn't interfere during dev when using Vite.
+- **No CORS** needed — Vite proxy handles same-origin in dev, Express static serving handles it in production.
+- **Push notifications** — the PWA registers a service worker (`injectManifest` strategy via vite-plugin-pwa) and subscribes the browser for Web Push. The Web Server subscribes to `host-event.>` and sends push notifications for confirmation requests, task completions, and task failures.
+- **Markdown rendering** — Generated task plans and task results are rendered as rich formatted text using `react-markdown` with `remark-gfm` (GitHub Flavored Markdown), supporting tables, strikethrough, task lists, and autolinks.
+- **Task confirmation** — the Dashboard discovers pending confirmations from the `task.list` RPC response (tasks with a pending request in the serve daemon's in-memory registry, reported via `task.status`). When found, it shows a full-screen confirmation modal. Push notification action buttons trigger `POST /api/push/respond`, which forwards to the `task.user_input` NATS RPC.
+- **Task event tracking** — task lifecycle events are persisted to `status.json` on the host (for crash detection) and broadcast via `host-event.<host_id>.<task_id>` pub/sub and HTTP SSE. The PWA loads initial status from `task.list` and subscribes to events for real-time updates.
+- **NATS config** (`nats.conf`) enables WebSocket on port 9222 (for browser clients) and token-based auth.
+
+## Related Repositories
+
+- [palmier](https://github.com/caihongxu/palmier) -- The host binary, published as `palmier` on npm. Install with `npm install -g palmier`. Uses npm (not pnpm).

package/palmier-server/nats.conf

@@ -0,0 +1,15 @@
+# NATS Server Configuration for Palmier
+
+# TCP listener (for server + agent)
+listen: 0.0.0.0:4222
+
+# WebSocket listener (for PWA browser clients)
+websocket {
+  listen: "0.0.0.0:9222"
+  no_tls: true
+}
+
+# Token-based authentication
+authorization {
+  token: "592b229f9df46e9ecaedbfcf9e2e5bcc6432367c114b20a0c7854b6f4542184a"
+}

package/palmier-server/package.json

@@ -0,0 +1,8 @@
+{
+  "name": "palmier-server",
+  "private": true,
+  "packageManager": "pnpm@10.32.1",
+  "pnpm": {
+    "onlyBuiltDependencies": ["bcrypt", "esbuild"]
+  }
+}